thunderbolt.rst 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284
  1. .. SPDX-License-Identifier: GPL-2.0
  2. ======================
  3. USB4 and Thunderbolt
  4. ======================
  5. USB4 is the public specification based on Thunderbolt 3 protocol with
  6. some differences at the register level among other things. Connection
  7. manager is an entity running on the host router (host controller)
  8. responsible for enumerating routers and establishing tunnels. A
  9. connection manager can be implemented either in firmware or software.
  10. Typically PCs come with a firmware connection manager for Thunderbolt 3
  11. and early USB4 capable systems. Apple systems on the other hand use
  12. software connection manager and the later USB4 compliant devices follow
  13. the suit.
  14. The Linux Thunderbolt driver supports both and can detect at runtime which
  15. connection manager implementation is to be used. To be on the safe side the
  16. software connection manager in Linux also advertises security level
  17. ``user`` which means PCIe tunneling is disabled by default. The
  18. documentation below applies to both implementations with the exception that
  19. the software connection manager only supports ``user`` security level and
  20. is expected to be accompanied with an IOMMU based DMA protection.
  21. Security levels and how to use them
  22. -----------------------------------
  23. The interface presented here is not meant for end users. Instead there
  24. should be a userspace tool that handles all the low-level details, keeps
  25. a database of the authorized devices and prompts users for new connections.
  26. More details about the sysfs interface for Thunderbolt devices can be
  27. found in ``Documentation/ABI/testing/sysfs-bus-thunderbolt``.
  28. Those users who just want to connect any device without any sort of
  29. manual work can add following line to
  30. ``/etc/udev/rules.d/99-local.rules``::
  31. ACTION=="add", SUBSYSTEM=="thunderbolt", ATTR{authorized}=="0", ATTR{authorized}="1"
  32. This will authorize all devices automatically when they appear. However,
  33. keep in mind that this bypasses the security levels and makes the system
  34. vulnerable to DMA attacks.
  35. Starting with Intel Falcon Ridge Thunderbolt controller there are 4
  36. security levels available. Intel Titan Ridge added one more security level
  37. (usbonly). The reason for these is the fact that the connected devices can
  38. be DMA masters and thus read contents of the host memory without CPU and OS
  39. knowing about it. There are ways to prevent this by setting up an IOMMU but
  40. it is not always available for various reasons.
  41. The security levels are as follows:
  42. none
  43. All devices are automatically connected by the firmware. No user
  44. approval is needed. In BIOS settings this is typically called
  45. *Legacy mode*.
  46. user
  47. User is asked whether the device is allowed to be connected.
  48. Based on the device identification information available through
  49. ``/sys/bus/thunderbolt/devices``, the user then can make the decision.
  50. In BIOS settings this is typically called *Unique ID*.
  51. secure
  52. User is asked whether the device is allowed to be connected. In
  53. addition to UUID the device (if it supports secure connect) is sent
  54. a challenge that should match the expected one based on a random key
  55. written to the ``key`` sysfs attribute. In BIOS settings this is
  56. typically called *One time saved key*.
  57. dponly
  58. The firmware automatically creates tunnels for Display Port and
  59. USB. No PCIe tunneling is done. In BIOS settings this is
  60. typically called *Display Port Only*.
  61. usbonly
  62. The firmware automatically creates tunnels for the USB controller and
  63. Display Port in a dock. All PCIe links downstream of the dock are
  64. removed.
  65. The current security level can be read from
  66. ``/sys/bus/thunderbolt/devices/domainX/security`` where ``domainX`` is
  67. the Thunderbolt domain the host controller manages. There is typically
  68. one domain per Thunderbolt host controller.
  69. If the security level reads as ``user`` or ``secure`` the connected
  70. device must be authorized by the user before PCIe tunnels are created
  71. (e.g the PCIe device appears).
  72. Each Thunderbolt device plugged in will appear in sysfs under
  73. ``/sys/bus/thunderbolt/devices``. The device directory carries
  74. information that can be used to identify the particular device,
  75. including its name and UUID.
  76. Authorizing devices when security level is ``user`` or ``secure``
  77. -----------------------------------------------------------------
  78. When a device is plugged in it will appear in sysfs as follows::
  79. /sys/bus/thunderbolt/devices/0-1/authorized - 0
  80. /sys/bus/thunderbolt/devices/0-1/device - 0x8004
  81. /sys/bus/thunderbolt/devices/0-1/device_name - Thunderbolt to FireWire Adapter
  82. /sys/bus/thunderbolt/devices/0-1/vendor - 0x1
  83. /sys/bus/thunderbolt/devices/0-1/vendor_name - Apple, Inc.
  84. /sys/bus/thunderbolt/devices/0-1/unique_id - e0376f00-0300-0100-ffff-ffffffffffff
  85. The ``authorized`` attribute reads 0 which means no PCIe tunnels are
  86. created yet. The user can authorize the device by simply entering::
  87. # echo 1 > /sys/bus/thunderbolt/devices/0-1/authorized
  88. This will create the PCIe tunnels and the device is now connected.
  89. If the device supports secure connect, and the domain security level is
  90. set to ``secure``, it has an additional attribute ``key`` which can hold
  91. a random 32-byte value used for authorization and challenging the device in
  92. future connects::
  93. /sys/bus/thunderbolt/devices/0-3/authorized - 0
  94. /sys/bus/thunderbolt/devices/0-3/device - 0x305
  95. /sys/bus/thunderbolt/devices/0-3/device_name - AKiTiO Thunder3 PCIe Box
  96. /sys/bus/thunderbolt/devices/0-3/key -
  97. /sys/bus/thunderbolt/devices/0-3/vendor - 0x41
  98. /sys/bus/thunderbolt/devices/0-3/vendor_name - inXtron
  99. /sys/bus/thunderbolt/devices/0-3/unique_id - dc010000-0000-8508-a22d-32ca6421cb16
  100. Notice the key is empty by default.
  101. If the user does not want to use secure connect they can just ``echo 1``
  102. to the ``authorized`` attribute and the PCIe tunnels will be created in
  103. the same way as in the ``user`` security level.
  104. If the user wants to use secure connect, the first time the device is
  105. plugged a key needs to be created and sent to the device::
  106. # key=$(openssl rand -hex 32)
  107. # echo $key > /sys/bus/thunderbolt/devices/0-3/key
  108. # echo 1 > /sys/bus/thunderbolt/devices/0-3/authorized
  109. Now the device is connected (PCIe tunnels are created) and in addition
  110. the key is stored on the device NVM.
  111. Next time the device is plugged in the user can verify (challenge) the
  112. device using the same key::
  113. # echo $key > /sys/bus/thunderbolt/devices/0-3/key
  114. # echo 2 > /sys/bus/thunderbolt/devices/0-3/authorized
  115. If the challenge the device returns back matches the one we expect based
  116. on the key, the device is connected and the PCIe tunnels are created.
  117. However, if the challenge fails no tunnels are created and error is
  118. returned to the user.
  119. If the user still wants to connect the device they can either approve
  120. the device without a key or write a new key and write 1 to the
  121. ``authorized`` file to get the new key stored on the device NVM.
  122. DMA protection utilizing IOMMU
  123. ------------------------------
  124. Recent systems from 2018 and forward with Thunderbolt ports may natively
  125. support IOMMU. This means that Thunderbolt security is handled by an IOMMU
  126. so connected devices cannot access memory regions outside of what is
  127. allocated for them by drivers. When Linux is running on such system it
  128. automatically enables IOMMU if not enabled by the user already. These
  129. systems can be identified by reading ``1`` from
  130. ``/sys/bus/thunderbolt/devices/domainX/iommu_dma_protection`` attribute.
  131. The driver does not do anything special in this case but because DMA
  132. protection is handled by the IOMMU, security levels (if set) are
  133. redundant. For this reason some systems ship with security level set to
  134. ``none``. Other systems have security level set to ``user`` in order to
  135. support downgrade to older OS, so users who want to automatically
  136. authorize devices when IOMMU DMA protection is enabled can use the
  137. following ``udev`` rule::
  138. ACTION=="add", SUBSYSTEM=="thunderbolt", ATTRS{iommu_dma_protection}=="1", ATTR{authorized}=="0", ATTR{authorized}="1"
  139. Upgrading NVM on Thunderbolt device, host or retimer
  140. ----------------------------------------------------
  141. Since most of the functionality is handled in firmware running on a
  142. host controller or a device, it is important that the firmware can be
  143. upgraded to the latest where possible bugs in it have been fixed.
  144. Typically OEMs provide this firmware from their support site.
  145. There is also a central site which has links where to download firmware
  146. for some machines:
  147. `Thunderbolt Updates <https://thunderbolttechnology.net/updates>`_
  148. Before you upgrade firmware on a device, host or retimer, please make
  149. sure it is a suitable upgrade. Failing to do that may render the device
  150. in a state where it cannot be used properly anymore without special
  151. tools!
  152. Host NVM upgrade on Apple Macs is not supported.
  153. Once the NVM image has been downloaded, you need to plug in a
  154. Thunderbolt device so that the host controller appears. It does not
  155. matter which device is connected (unless you are upgrading NVM on a
  156. device - then you need to connect that particular device).
  157. Note an OEM-specific method to power the controller up ("force power") may
  158. be available for your system in which case there is no need to plug in a
  159. Thunderbolt device.
  160. After that we can write the firmware to the non-active parts of the NVM
  161. of the host or device. As an example here is how Intel NUC6i7KYK (Skull
  162. Canyon) Thunderbolt controller NVM is upgraded::
  163. # dd if=KYK_TBT_FW_0018.bin of=/sys/bus/thunderbolt/devices/0-0/nvm_non_active0/nvmem
  164. Once the operation completes we can trigger NVM authentication and
  165. upgrade process as follows::
  166. # echo 1 > /sys/bus/thunderbolt/devices/0-0/nvm_authenticate
  167. If no errors are returned, the host controller shortly disappears. Once
  168. it comes back the driver notices it and initiates a full power cycle.
  169. After a while the host controller appears again and this time it should
  170. be fully functional.
  171. We can verify that the new NVM firmware is active by running the following
  172. commands::
  173. # cat /sys/bus/thunderbolt/devices/0-0/nvm_authenticate
  174. 0x0
  175. # cat /sys/bus/thunderbolt/devices/0-0/nvm_version
  176. 18.0
  177. If ``nvm_authenticate`` contains anything other than 0x0 it is the error
  178. code from the last authentication cycle, which means the authentication
  179. of the NVM image failed.
  180. Note names of the NVMem devices ``nvm_activeN`` and ``nvm_non_activeN``
  181. depend on the order they are registered in the NVMem subsystem. N in
  182. the name is the identifier added by the NVMem subsystem.
  183. Upgrading NVM when host controller is in safe mode
  184. --------------------------------------------------
  185. If the existing NVM is not properly authenticated (or is missing) the
  186. host controller goes into safe mode which means that the only available
  187. functionality is flashing a new NVM image. When in this mode, reading
  188. ``nvm_version`` fails with ``ENODATA`` and the device identification
  189. information is missing.
  190. To recover from this mode, one needs to flash a valid NVM image to the
  191. host controller in the same way it is done in the previous chapter.
  192. Networking over Thunderbolt cable
  193. ---------------------------------
  194. Thunderbolt technology allows software communication between two hosts
  195. connected by a Thunderbolt cable.
  196. It is possible to tunnel any kind of traffic over a Thunderbolt link but
  197. currently we only support Apple ThunderboltIP protocol.
  198. If the other host is running Windows or macOS, the only thing you need to
  199. do is to connect a Thunderbolt cable between the two hosts; the
  200. ``thunderbolt-net`` driver is loaded automatically. If the other host is
  201. also Linux you should load ``thunderbolt-net`` manually on one host (it
  202. does not matter which one)::
  203. # modprobe thunderbolt-net
  204. This triggers module load on the other host automatically. If the driver
  205. is built-in to the kernel image, there is no need to do anything.
  206. The driver will create one virtual ethernet interface per Thunderbolt
  207. port which are named like ``thunderbolt0`` and so on. From this point
  208. you can either use standard userspace tools like ``ifconfig`` to
  209. configure the interface or let your GUI handle it automatically.
  210. Forcing power
  211. -------------
  212. Many OEMs include a method that can be used to force the power of a
  213. Thunderbolt controller to an "On" state even if nothing is connected.
  214. If supported by your machine this will be exposed by the WMI bus with
  215. a sysfs attribute called "force_power".
  216. For example the intel-wmi-thunderbolt driver exposes this attribute in:
  217. /sys/bus/wmi/devices/86CCFD48-205E-4A77-9C48-2021CBEDE341/force_power
  218. To force the power to on, write 1 to this attribute file.
  219. To disable force power, write 0 to this attribute file.
  220. Note: it's currently not possible to query the force power state of a platform.