segmentation-offloads.rst 7.7 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184
  1. .. SPDX-License-Identifier: GPL-2.0
  2. =====================
  3. Segmentation Offloads
  4. =====================
  5. Introduction
  6. ============
  7. This document describes a set of techniques in the Linux networking stack
  8. to take advantage of segmentation offload capabilities of various NICs.
  9. The following technologies are described:
  10. * TCP Segmentation Offload - TSO
  11. * UDP Fragmentation Offload - UFO
  12. * IPIP, SIT, GRE, and UDP Tunnel Offloads
  13. * Generic Segmentation Offload - GSO
  14. * Generic Receive Offload - GRO
  15. * Partial Generic Segmentation Offload - GSO_PARTIAL
  16. * SCTP acceleration with GSO - GSO_BY_FRAGS
  17. TCP Segmentation Offload
  18. ========================
  19. TCP segmentation allows a device to segment a single frame into multiple
  20. frames with a data payload size specified in skb_shinfo()->gso_size.
  21. When TCP segmentation requested the bit for either SKB_GSO_TCPV4 or
  22. SKB_GSO_TCPV6 should be set in skb_shinfo()->gso_type and
  23. skb_shinfo()->gso_size should be set to a non-zero value.
  24. TCP segmentation is dependent on support for the use of partial checksum
  25. offload. For this reason TSO is normally disabled if the Tx checksum
  26. offload for a given device is disabled.
  27. In order to support TCP segmentation offload it is necessary to populate
  28. the network and transport header offsets of the skbuff so that the device
  29. drivers will be able determine the offsets of the IP or IPv6 header and the
  30. TCP header. In addition as CHECKSUM_PARTIAL is required csum_start should
  31. also point to the TCP header of the packet.
  32. For IPv4 segmentation we support one of two types in terms of the IP ID.
  33. The default behavior is to increment the IP ID with every segment. If the
  34. GSO type SKB_GSO_TCP_FIXEDID is specified then we will not increment the IP
  35. ID and all segments will use the same IP ID. If a device has
  36. NETIF_F_TSO_MANGLEID set then the IP ID can be ignored when performing TSO
  37. and we will either increment the IP ID for all frames, or leave it at a
  38. static value based on driver preference.
  39. UDP Fragmentation Offload
  40. =========================
  41. UDP fragmentation offload allows a device to fragment an oversized UDP
  42. datagram into multiple IPv4 fragments. Many of the requirements for UDP
  43. fragmentation offload are the same as TSO. However the IPv4 ID for
  44. fragments should not increment as a single IPv4 datagram is fragmented.
  45. UFO is deprecated: modern kernels will no longer generate UFO skbs, but can
  46. still receive them from tuntap and similar devices. Offload of UDP-based
  47. tunnel protocols is still supported.
  48. IPIP, SIT, GRE, UDP Tunnel, and Remote Checksum Offloads
  49. ========================================================
  50. In addition to the offloads described above it is possible for a frame to
  51. contain additional headers such as an outer tunnel. In order to account
  52. for such instances an additional set of segmentation offload types were
  53. introduced including SKB_GSO_IPXIP4, SKB_GSO_IPXIP6, SKB_GSO_GRE, and
  54. SKB_GSO_UDP_TUNNEL. These extra segmentation types are used to identify
  55. cases where there are more than just 1 set of headers. For example in the
  56. case of IPIP and SIT we should have the network and transport headers moved
  57. from the standard list of headers to "inner" header offsets.
  58. Currently only two levels of headers are supported. The convention is to
  59. refer to the tunnel headers as the outer headers, while the encapsulated
  60. data is normally referred to as the inner headers. Below is the list of
  61. calls to access the given headers:
  62. IPIP/SIT Tunnel::
  63. Outer Inner
  64. MAC skb_mac_header
  65. Network skb_network_header skb_inner_network_header
  66. Transport skb_transport_header
  67. UDP/GRE Tunnel::
  68. Outer Inner
  69. MAC skb_mac_header skb_inner_mac_header
  70. Network skb_network_header skb_inner_network_header
  71. Transport skb_transport_header skb_inner_transport_header
  72. In addition to the above tunnel types there are also SKB_GSO_GRE_CSUM and
  73. SKB_GSO_UDP_TUNNEL_CSUM. These two additional tunnel types reflect the
  74. fact that the outer header also requests to have a non-zero checksum
  75. included in the outer header.
  76. Finally there is SKB_GSO_TUNNEL_REMCSUM which indicates that a given tunnel
  77. header has requested a remote checksum offload. In this case the inner
  78. headers will be left with a partial checksum and only the outer header
  79. checksum will be computed.
  80. Generic Segmentation Offload
  81. ============================
  82. Generic segmentation offload is a pure software offload that is meant to
  83. deal with cases where device drivers cannot perform the offloads described
  84. above. What occurs in GSO is that a given skbuff will have its data broken
  85. out over multiple skbuffs that have been resized to match the MSS provided
  86. via skb_shinfo()->gso_size.
  87. Before enabling any hardware segmentation offload a corresponding software
  88. offload is required in GSO. Otherwise it becomes possible for a frame to
  89. be re-routed between devices and end up being unable to be transmitted.
  90. Generic Receive Offload
  91. =======================
  92. Generic receive offload is the complement to GSO. Ideally any frame
  93. assembled by GRO should be segmented to create an identical sequence of
  94. frames using GSO, and any sequence of frames segmented by GSO should be
  95. able to be reassembled back to the original by GRO. The only exception to
  96. this is IPv4 ID in the case that the DF bit is set for a given IP header.
  97. If the value of the IPv4 ID is not sequentially incrementing it will be
  98. altered so that it is when a frame assembled via GRO is segmented via GSO.
  99. Partial Generic Segmentation Offload
  100. ====================================
  101. Partial generic segmentation offload is a hybrid between TSO and GSO. What
  102. it effectively does is take advantage of certain traits of TCP and tunnels
  103. so that instead of having to rewrite the packet headers for each segment
  104. only the inner-most transport header and possibly the outer-most network
  105. header need to be updated. This allows devices that do not support tunnel
  106. offloads or tunnel offloads with checksum to still make use of segmentation.
  107. With the partial offload what occurs is that all headers excluding the
  108. inner transport header are updated such that they will contain the correct
  109. values for if the header was simply duplicated. The one exception to this
  110. is the outer IPv4 ID field. It is up to the device drivers to guarantee
  111. that the IPv4 ID field is incremented in the case that a given header does
  112. not have the DF bit set.
  113. SCTP acceleration with GSO
  114. ===========================
  115. SCTP - despite the lack of hardware support - can still take advantage of
  116. GSO to pass one large packet through the network stack, rather than
  117. multiple small packets.
  118. This requires a different approach to other offloads, as SCTP packets
  119. cannot be just segmented to (P)MTU. Rather, the chunks must be contained in
  120. IP segments, padding respected. So unlike regular GSO, SCTP can't just
  121. generate a big skb, set gso_size to the fragmentation point and deliver it
  122. to IP layer.
  123. Instead, the SCTP protocol layer builds an skb with the segments correctly
  124. padded and stored as chained skbs, and skb_segment() splits based on those.
  125. To signal this, gso_size is set to the special value GSO_BY_FRAGS.
  126. Therefore, any code in the core networking stack must be aware of the
  127. possibility that gso_size will be GSO_BY_FRAGS and handle that case
  128. appropriately.
  129. There are some helpers to make this easier:
  130. - skb_is_gso(skb) && skb_is_gso_sctp(skb) is the best way to see if
  131. an skb is an SCTP GSO skb.
  132. - For size checks, the skb_gso_validate_*_len family of helpers correctly
  133. considers GSO_BY_FRAGS.
  134. - For manipulating packets, skb_increase_gso_size and skb_decrease_gso_size
  135. will check for GSO_BY_FRAGS and WARN if asked to manipulate these skbs.
  136. This also affects drivers with the NETIF_F_FRAGLIST & NETIF_F_GSO_SCTP bits
  137. set. Note also that NETIF_F_GSO_SCTP is included in NETIF_F_GSO_SOFTWARE.