tls-offload.rst 22 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530
  1. .. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
  2. ==================
  3. Kernel TLS offload
  4. ==================
  5. Kernel TLS operation
  6. ====================
  7. Linux kernel provides TLS connection offload infrastructure. Once a TCP
  8. connection is in ``ESTABLISHED`` state user space can enable the TLS Upper
  9. Layer Protocol (ULP) and install the cryptographic connection state.
  10. For details regarding the user-facing interface refer to the TLS
  11. documentation in :ref:`Documentation/networking/tls.rst <kernel_tls>`.
  12. ``ktls`` can operate in three modes:
  13. * Software crypto mode (``TLS_SW``) - CPU handles the cryptography.
  14. In most basic cases only crypto operations synchronous with the CPU
  15. can be used, but depending on calling context CPU may utilize
  16. asynchronous crypto accelerators. The use of accelerators introduces extra
  17. latency on socket reads (decryption only starts when a read syscall
  18. is made) and additional I/O load on the system.
  19. * Packet-based NIC offload mode (``TLS_HW``) - the NIC handles crypto
  20. on a packet by packet basis, provided the packets arrive in order.
  21. This mode integrates best with the kernel stack and is described in detail
  22. in the remaining part of this document
  23. (``ethtool`` flags ``tls-hw-tx-offload`` and ``tls-hw-rx-offload``).
  24. * Full TCP NIC offload mode (``TLS_HW_RECORD``) - mode of operation where
  25. NIC driver and firmware replace the kernel networking stack
  26. with its own TCP handling, it is not usable in production environments
  27. making use of the Linux networking stack for example any firewalling
  28. abilities or QoS and packet scheduling (``ethtool`` flag ``tls-hw-record``).
  29. The operation mode is selected automatically based on device configuration,
  30. offload opt-in or opt-out on per-connection basis is not currently supported.
  31. TX
  32. --
  33. At a high level user write requests are turned into a scatter list, the TLS ULP
  34. intercepts them, inserts record framing, performs encryption (in ``TLS_SW``
  35. mode) and then hands the modified scatter list to the TCP layer. From this
  36. point on the TCP stack proceeds as normal.
  37. In ``TLS_HW`` mode the encryption is not performed in the TLS ULP.
  38. Instead packets reach a device driver, the driver will mark the packets
  39. for crypto offload based on the socket the packet is attached to,
  40. and send them to the device for encryption and transmission.
  41. RX
  42. --
  43. On the receive side if the device handled decryption and authentication
  44. successfully, the driver will set the decrypted bit in the associated
  45. :c:type:`struct sk_buff <sk_buff>`. The packets reach the TCP stack and
  46. are handled normally. ``ktls`` is informed when data is queued to the socket
  47. and the ``strparser`` mechanism is used to delineate the records. Upon read
  48. request, records are retrieved from the socket and passed to decryption routine.
  49. If device decrypted all the segments of the record the decryption is skipped,
  50. otherwise software path handles decryption.
  51. .. kernel-figure:: tls-offload-layers.svg
  52. :alt: TLS offload layers
  53. :align: center
  54. :figwidth: 28em
  55. Layers of Kernel TLS stack
  56. Device configuration
  57. ====================
  58. During driver initialization device sets the ``NETIF_F_HW_TLS_RX`` and
  59. ``NETIF_F_HW_TLS_TX`` features and installs its
  60. :c:type:`struct tlsdev_ops <tlsdev_ops>`
  61. pointer in the :c:member:`tlsdev_ops` member of the
  62. :c:type:`struct net_device <net_device>`.
  63. When TLS cryptographic connection state is installed on a ``ktls`` socket
  64. (note that it is done twice, once for RX and once for TX direction,
  65. and the two are completely independent), the kernel checks if the underlying
  66. network device is offload-capable and attempts the offload. In case offload
  67. fails the connection is handled entirely in software using the same mechanism
  68. as if the offload was never tried.
  69. Offload request is performed via the :c:member:`tls_dev_add` callback of
  70. :c:type:`struct tlsdev_ops <tlsdev_ops>`:
  71. .. code-block:: c
  72. int (*tls_dev_add)(struct net_device *netdev, struct sock *sk,
  73. enum tls_offload_ctx_dir direction,
  74. struct tls_crypto_info *crypto_info,
  75. u32 start_offload_tcp_sn);
  76. ``direction`` indicates whether the cryptographic information is for
  77. the received or transmitted packets. Driver uses the ``sk`` parameter
  78. to retrieve the connection 5-tuple and socket family (IPv4 vs IPv6).
  79. Cryptographic information in ``crypto_info`` includes the key, iv, salt
  80. as well as TLS record sequence number. ``start_offload_tcp_sn`` indicates
  81. which TCP sequence number corresponds to the beginning of the record with
  82. sequence number from ``crypto_info``. The driver can add its state
  83. at the end of kernel structures (see :c:member:`driver_state` members
  84. in ``include/net/tls.h``) to avoid additional allocations and pointer
  85. dereferences.
  86. TX
  87. --
  88. After TX state is installed, the stack guarantees that the first segment
  89. of the stream will start exactly at the ``start_offload_tcp_sn`` sequence
  90. number, simplifying TCP sequence number matching.
  91. TX offload being fully initialized does not imply that all segments passing
  92. through the driver and which belong to the offloaded socket will be after
  93. the expected sequence number and will have kernel record information.
  94. In particular, already encrypted data may have been queued to the socket
  95. before installing the connection state in the kernel.
  96. RX
  97. --
  98. In RX direction local networking stack has little control over the segmentation,
  99. so the initial records' TCP sequence number may be anywhere inside the segment.
  100. Normal operation
  101. ================
  102. At the minimum the device maintains the following state for each connection, in
  103. each direction:
  104. * crypto secrets (key, iv, salt)
  105. * crypto processing state (partial blocks, partial authentication tag, etc.)
  106. * record metadata (sequence number, processing offset and length)
  107. * expected TCP sequence number
  108. There are no guarantees on record length or record segmentation. In particular
  109. segments may start at any point of a record and contain any number of records.
  110. Assuming segments are received in order, the device should be able to perform
  111. crypto operations and authentication regardless of segmentation. For this
  112. to be possible device has to keep small amount of segment-to-segment state.
  113. This includes at least:
  114. * partial headers (if a segment carried only a part of the TLS header)
  115. * partial data block
  116. * partial authentication tag (all data had been seen but part of the
  117. authentication tag has to be written or read from the subsequent segment)
  118. Record reassembly is not necessary for TLS offload. If the packets arrive
  119. in order the device should be able to handle them separately and make
  120. forward progress.
  121. TX
  122. --
  123. The kernel stack performs record framing reserving space for the authentication
  124. tag and populating all other TLS header and tailer fields.
  125. Both the device and the driver maintain expected TCP sequence numbers
  126. due to the possibility of retransmissions and the lack of software fallback
  127. once the packet reaches the device.
  128. For segments passed in order, the driver marks the packets with
  129. a connection identifier (note that a 5-tuple lookup is insufficient to identify
  130. packets requiring HW offload, see the :ref:`5tuple_problems` section)
  131. and hands them to the device. The device identifies the packet as requiring
  132. TLS handling and confirms the sequence number matches its expectation.
  133. The device performs encryption and authentication of the record data.
  134. It replaces the authentication tag and TCP checksum with correct values.
  135. RX
  136. --
  137. Before a packet is DMAed to the host (but after NIC's embedded switching
  138. and packet transformation functions) the device validates the Layer 4
  139. checksum and performs a 5-tuple lookup to find any TLS connection the packet
  140. may belong to (technically a 4-tuple
  141. lookup is sufficient - IP addresses and TCP port numbers, as the protocol
  142. is always TCP). If connection is matched device confirms if the TCP sequence
  143. number is the expected one and proceeds to TLS handling (record delineation,
  144. decryption, authentication for each record in the packet). The device leaves
  145. the record framing unmodified, the stack takes care of record decapsulation.
  146. Device indicates successful handling of TLS offload in the per-packet context
  147. (descriptor) passed to the host.
  148. Upon reception of a TLS offloaded packet, the driver sets
  149. the :c:member:`decrypted` mark in :c:type:`struct sk_buff <sk_buff>`
  150. corresponding to the segment. Networking stack makes sure decrypted
  151. and non-decrypted segments do not get coalesced (e.g. by GRO or socket layer)
  152. and takes care of partial decryption.
  153. Resync handling
  154. ===============
  155. In presence of packet drops or network packet reordering, the device may lose
  156. synchronization with the TLS stream, and require a resync with the kernel's
  157. TCP stack.
  158. Note that resync is only attempted for connections which were successfully
  159. added to the device table and are in TLS_HW mode. For example,
  160. if the table was full when cryptographic state was installed in the kernel,
  161. such connection will never get offloaded. Therefore the resync request
  162. does not carry any cryptographic connection state.
  163. TX
  164. --
  165. Segments transmitted from an offloaded socket can get out of sync
  166. in similar ways to the receive side-retransmissions - local drops
  167. are possible, though network reorders are not. There are currently
  168. two mechanisms for dealing with out of order segments.
  169. Crypto state rebuilding
  170. ~~~~~~~~~~~~~~~~~~~~~~~
  171. Whenever an out of order segment is transmitted the driver provides
  172. the device with enough information to perform cryptographic operations.
  173. This means most likely that the part of the record preceding the current
  174. segment has to be passed to the device as part of the packet context,
  175. together with its TCP sequence number and TLS record number. The device
  176. can then initialize its crypto state, process and discard the preceding
  177. data (to be able to insert the authentication tag) and move onto handling
  178. the actual packet.
  179. In this mode depending on the implementation the driver can either ask
  180. for a continuation with the crypto state and the new sequence number
  181. (next expected segment is the one after the out of order one), or continue
  182. with the previous stream state - assuming that the out of order segment
  183. was just a retransmission. The former is simpler, and does not require
  184. retransmission detection therefore it is the recommended method until
  185. such time it is proven inefficient.
  186. Next record sync
  187. ~~~~~~~~~~~~~~~~
  188. Whenever an out of order segment is detected the driver requests
  189. that the ``ktls`` software fallback code encrypt it. If the segment's
  190. sequence number is lower than expected the driver assumes retransmission
  191. and doesn't change device state. If the segment is in the future, it
  192. may imply a local drop, the driver asks the stack to sync the device
  193. to the next record state and falls back to software.
  194. Resync request is indicated with:
  195. .. code-block:: c
  196. void tls_offload_tx_resync_request(struct sock *sk, u32 got_seq, u32 exp_seq)
  197. Until resync is complete driver should not access its expected TCP
  198. sequence number (as it will be updated from a different context).
  199. Following helper should be used to test if resync is complete:
  200. .. code-block:: c
  201. bool tls_offload_tx_resync_pending(struct sock *sk)
  202. Next time ``ktls`` pushes a record it will first send its TCP sequence number
  203. and TLS record number to the driver. Stack will also make sure that
  204. the new record will start on a segment boundary (like it does when
  205. the connection is initially added).
  206. RX
  207. --
  208. A small amount of RX reorder events may not require a full resynchronization.
  209. In particular the device should not lose synchronization
  210. when record boundary can be recovered:
  211. .. kernel-figure:: tls-offload-reorder-good.svg
  212. :alt: reorder of non-header segment
  213. :align: center
  214. Reorder of non-header segment
  215. Green segments are successfully decrypted, blue ones are passed
  216. as received on wire, red stripes mark start of new records.
  217. In above case segment 1 is received and decrypted successfully.
  218. Segment 2 was dropped so 3 arrives out of order. The device knows
  219. the next record starts inside 3, based on record length in segment 1.
  220. Segment 3 is passed untouched, because due to lack of data from segment 2
  221. the remainder of the previous record inside segment 3 cannot be handled.
  222. The device can, however, collect the authentication algorithm's state
  223. and partial block from the new record in segment 3 and when 4 and 5
  224. arrive continue decryption. Finally when 2 arrives it's completely outside
  225. of expected window of the device so it's passed as is without special
  226. handling. ``ktls`` software fallback handles the decryption of record
  227. spanning segments 1, 2 and 3. The device did not get out of sync,
  228. even though two segments did not get decrypted.
  229. Kernel synchronization may be necessary if the lost segment contained
  230. a record header and arrived after the next record header has already passed:
  231. .. kernel-figure:: tls-offload-reorder-bad.svg
  232. :alt: reorder of header segment
  233. :align: center
  234. Reorder of segment with a TLS header
  235. In this example segment 2 gets dropped, and it contains a record header.
  236. Device can only detect that segment 4 also contains a TLS header
  237. if it knows the length of the previous record from segment 2. In this case
  238. the device will lose synchronization with the stream.
  239. Stream scan resynchronization
  240. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  241. When the device gets out of sync and the stream reaches TCP sequence
  242. numbers more than a max size record past the expected TCP sequence number,
  243. the device starts scanning for a known header pattern. For example
  244. for TLS 1.2 and TLS 1.3 subsequent bytes of value ``0x03 0x03`` occur
  245. in the SSL/TLS version field of the header. Once pattern is matched
  246. the device continues attempting parsing headers at expected locations
  247. (based on the length fields at guessed locations).
  248. Whenever the expected location does not contain a valid header the scan
  249. is restarted.
  250. When the header is matched the device sends a confirmation request
  251. to the kernel, asking if the guessed location is correct (if a TLS record
  252. really starts there), and which record sequence number the given header had.
  253. The kernel confirms the guessed location was correct and tells the device
  254. the record sequence number. Meanwhile, the device had been parsing
  255. and counting all records since the just-confirmed one, it adds the number
  256. of records it had seen to the record number provided by the kernel.
  257. At this point the device is in sync and can resume decryption at next
  258. segment boundary.
  259. In a pathological case the device may latch onto a sequence of matching
  260. headers and never hear back from the kernel (there is no negative
  261. confirmation from the kernel). The implementation may choose to periodically
  262. restart scan. Given how unlikely falsely-matching stream is, however,
  263. periodic restart is not deemed necessary.
  264. Special care has to be taken if the confirmation request is passed
  265. asynchronously to the packet stream and record may get processed
  266. by the kernel before the confirmation request.
  267. Stack-driven resynchronization
  268. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  269. The driver may also request the stack to perform resynchronization
  270. whenever it sees the records are no longer getting decrypted.
  271. If the connection is configured in this mode the stack automatically
  272. schedules resynchronization after it has received two completely encrypted
  273. records.
  274. The stack waits for the socket to drain and informs the device about
  275. the next expected record number and its TCP sequence number. If the
  276. records continue to be received fully encrypted stack retries the
  277. synchronization with an exponential back off (first after 2 encrypted
  278. records, then after 4 records, after 8, after 16... up until every
  279. 128 records).
  280. Error handling
  281. ==============
  282. TX
  283. --
  284. Packets may be redirected or rerouted by the stack to a different
  285. device than the selected TLS offload device. The stack will handle
  286. such condition using the :c:func:`sk_validate_xmit_skb` helper
  287. (TLS offload code installs :c:func:`tls_validate_xmit_skb` at this hook).
  288. Offload maintains information about all records until the data is
  289. fully acknowledged, so if skbs reach the wrong device they can be handled
  290. by software fallback.
  291. Any device TLS offload handling error on the transmission side must result
  292. in the packet being dropped. For example if a packet got out of order
  293. due to a bug in the stack or the device, reached the device and can't
  294. be encrypted such packet must be dropped.
  295. RX
  296. --
  297. If the device encounters any problems with TLS offload on the receive
  298. side it should pass the packet to the host's networking stack as it was
  299. received on the wire.
  300. For example authentication failure for any record in the segment should
  301. result in passing the unmodified packet to the software fallback. This means
  302. packets should not be modified "in place". Splitting segments to handle partial
  303. decryption is not advised. In other words either all records in the packet
  304. had been handled successfully and authenticated or the packet has to be passed
  305. to the host's stack as it was on the wire (recovering original packet in the
  306. driver if device provides precise error is sufficient).
  307. The Linux networking stack does not provide a way of reporting per-packet
  308. decryption and authentication errors, packets with errors must simply not
  309. have the :c:member:`decrypted` mark set.
  310. A packet should also not be handled by the TLS offload if it contains
  311. incorrect checksums.
  312. Performance metrics
  313. ===================
  314. TLS offload can be characterized by the following basic metrics:
  315. * max connection count
  316. * connection installation rate
  317. * connection installation latency
  318. * total cryptographic performance
  319. Note that each TCP connection requires a TLS session in both directions,
  320. the performance may be reported treating each direction separately.
  321. Max connection count
  322. --------------------
  323. The number of connections device can support can be exposed via
  324. ``devlink resource`` API.
  325. Total cryptographic performance
  326. -------------------------------
  327. Offload performance may depend on segment and record size.
  328. Overload of the cryptographic subsystem of the device should not have
  329. significant performance impact on non-offloaded streams.
  330. Statistics
  331. ==========
  332. Following minimum set of TLS-related statistics should be reported
  333. by the driver:
  334. * ``rx_tls_decrypted_packets`` - number of successfully decrypted RX packets
  335. which were part of a TLS stream.
  336. * ``rx_tls_decrypted_bytes`` - number of TLS payload bytes in RX packets
  337. which were successfully decrypted.
  338. * ``rx_tls_ctx`` - number of TLS RX HW offload contexts added to device for
  339. decryption.
  340. * ``rx_tls_del`` - number of TLS RX HW offload contexts deleted from device
  341. (connection has finished).
  342. * ``rx_tls_resync_req_pkt`` - number of received TLS packets with a resync
  343. request.
  344. * ``rx_tls_resync_req_start`` - number of times the TLS async resync request
  345. was started.
  346. * ``rx_tls_resync_req_end`` - number of times the TLS async resync request
  347. properly ended with providing the HW tracked tcp-seq.
  348. * ``rx_tls_resync_req_skip`` - number of times the TLS async resync request
  349. procedure was started by not properly ended.
  350. * ``rx_tls_resync_res_ok`` - number of times the TLS resync response call to
  351. the driver was successfully handled.
  352. * ``rx_tls_resync_res_skip`` - number of times the TLS resync response call to
  353. the driver was terminated unsuccessfully.
  354. * ``rx_tls_err`` - number of RX packets which were part of a TLS stream
  355. but were not decrypted due to unexpected error in the state machine.
  356. * ``tx_tls_encrypted_packets`` - number of TX packets passed to the device
  357. for encryption of their TLS payload.
  358. * ``tx_tls_encrypted_bytes`` - number of TLS payload bytes in TX packets
  359. passed to the device for encryption.
  360. * ``tx_tls_ctx`` - number of TLS TX HW offload contexts added to device for
  361. encryption.
  362. * ``tx_tls_ooo`` - number of TX packets which were part of a TLS stream
  363. but did not arrive in the expected order.
  364. * ``tx_tls_skip_no_sync_data`` - number of TX packets which were part of
  365. a TLS stream and arrived out-of-order, but skipped the HW offload routine
  366. and went to the regular transmit flow as they were retransmissions of the
  367. connection handshake.
  368. * ``tx_tls_drop_no_sync_data`` - number of TX packets which were part of
  369. a TLS stream dropped, because they arrived out of order and associated
  370. record could not be found.
  371. * ``tx_tls_drop_bypass_req`` - number of TX packets which were part of a TLS
  372. stream dropped, because they contain both data that has been encrypted by
  373. software and data that expects hardware crypto offload.
  374. Notable corner cases, exceptions and additional requirements
  375. ============================================================
  376. .. _5tuple_problems:
  377. 5-tuple matching limitations
  378. ----------------------------
  379. The device can only recognize received packets based on the 5-tuple
  380. of the socket. Current ``ktls`` implementation will not offload sockets
  381. routed through software interfaces such as those used for tunneling
  382. or virtual networking. However, many packet transformations performed
  383. by the networking stack (most notably any BPF logic) do not require
  384. any intermediate software device, therefore a 5-tuple match may
  385. consistently miss at the device level. In such cases the device
  386. should still be able to perform TX offload (encryption) and should
  387. fallback cleanly to software decryption (RX).
  388. Out of order
  389. ------------
  390. Introducing extra processing in NICs should not cause packets to be
  391. transmitted or received out of order, for example pure ACK packets
  392. should not be reordered with respect to data segments.
  393. Ingress reorder
  394. ---------------
  395. A device is permitted to perform packet reordering for consecutive
  396. TCP segments (i.e. placing packets in the correct order) but any form
  397. of additional buffering is disallowed.
  398. Coexistence with standard networking offload features
  399. -----------------------------------------------------
  400. Offloaded ``ktls`` sockets should support standard TCP stack features
  401. transparently. Enabling device TLS offload should not cause any difference
  402. in packets as seen on the wire.
  403. Transport layer transparency
  404. ----------------------------
  405. The device should not modify any packet headers for the purpose
  406. of the simplifying TLS offload.
  407. The device should not depend on any packet headers beyond what is strictly
  408. necessary for TLS offload.
  409. Segment drops
  410. -------------
  411. Dropping packets is acceptable only in the event of catastrophic
  412. system errors and should never be used as an error handling mechanism
  413. in cases arising from normal operation. In other words, reliance
  414. on TCP retransmissions to handle corner cases is not acceptable.
  415. TLS device features
  416. -------------------
  417. Drivers should ignore the changes to TLS the device feature flags.
  418. These flags will be acted upon accordingly by the core ``ktls`` code.
  419. TLS device feature flags only control adding of new TLS connection
  420. offloads, old connections will remain active after flags are cleared.