iommu.rst 7.6 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209
  1. .. SPDX-License-Identifier: GPL-2.0
  2. .. iommu:
  3. =====================================
  4. IOMMU Userspace API
  5. =====================================
  6. IOMMU UAPI is used for virtualization cases where communications are
  7. needed between physical and virtual IOMMU drivers. For baremetal
  8. usage, the IOMMU is a system device which does not need to communicate
  9. with userspace directly.
  10. The primary use cases are guest Shared Virtual Address (SVA) and
  11. guest IO virtual address (IOVA), wherein the vIOMMU implementation
  12. relies on the physical IOMMU and for this reason requires interactions
  13. with the host driver.
  14. .. contents:: :local:
  15. Functionalities
  16. ===============
  17. Communications of user and kernel involve both directions. The
  18. supported user-kernel APIs are as follows:
  19. 1. Bind/Unbind guest PASID (e.g. Intel VT-d)
  20. 2. Bind/Unbind guest PASID table (e.g. ARM SMMU)
  21. 3. Invalidate IOMMU caches upon guest requests
  22. 4. Report errors to the guest and serve page requests
  23. Requirements
  24. ============
  25. The IOMMU UAPIs are generic and extensible to meet the following
  26. requirements:
  27. 1. Emulated and para-virtualised vIOMMUs
  28. 2. Multiple vendors (Intel VT-d, ARM SMMU, etc.)
  29. 3. Extensions to the UAPI shall not break existing userspace
  30. Interfaces
  31. ==========
  32. Although the data structures defined in IOMMU UAPI are self-contained,
  33. there are no user API functions introduced. Instead, IOMMU UAPI is
  34. designed to work with existing user driver frameworks such as VFIO.
  35. Extension Rules & Precautions
  36. -----------------------------
  37. When IOMMU UAPI gets extended, the data structures can *only* be
  38. modified in two ways:
  39. 1. Adding new fields by re-purposing the padding[] field. No size change.
  40. 2. Adding new union members at the end. May increase the structure sizes.
  41. No new fields can be added *after* the variable sized union in that it
  42. will break backward compatibility when offset moves. A new flag must
  43. be introduced whenever a change affects the structure using either
  44. method. The IOMMU driver processes the data based on flags which
  45. ensures backward compatibility.
  46. Version field is only reserved for the unlikely event of UAPI upgrade
  47. at its entirety.
  48. It's *always* the caller's responsibility to indicate the size of the
  49. structure passed by setting argsz appropriately.
  50. Though at the same time, argsz is user provided data which is not
  51. trusted. The argsz field allows the user app to indicate how much data
  52. it is providing; it's still the kernel's responsibility to validate
  53. whether it's correct and sufficient for the requested operation.
  54. Compatibility Checking
  55. ----------------------
  56. When IOMMU UAPI extension results in some structure size increase,
  57. IOMMU UAPI code shall handle the following cases:
  58. 1. User and kernel has exact size match
  59. 2. An older user with older kernel header (smaller UAPI size) running on a
  60. newer kernel (larger UAPI size)
  61. 3. A newer user with newer kernel header (larger UAPI size) running
  62. on an older kernel.
  63. 4. A malicious/misbehaving user passing illegal/invalid size but within
  64. range. The data may contain garbage.
  65. Feature Checking
  66. ----------------
  67. While launching a guest with vIOMMU, it is strongly advised to check
  68. the compatibility upfront, as some subsequent errors happening during
  69. vIOMMU operation, such as cache invalidation failures cannot be nicely
  70. escalated to the guest due to IOMMU specifications. This can lead to
  71. catastrophic failures for the users.
  72. User applications such as QEMU are expected to import kernel UAPI
  73. headers. Backward compatibility is supported per feature flags.
  74. For example, an older QEMU (with older kernel header) can run on newer
  75. kernel. Newer QEMU (with new kernel header) may refuse to initialize
  76. on an older kernel if new feature flags are not supported by older
  77. kernel. Simply recompiling existing code with newer kernel header should
  78. not be an issue in that only existing flags are used.
  79. IOMMU vendor driver should report the below features to IOMMU UAPI
  80. consumers (e.g. via VFIO).
  81. 1. IOMMU_NESTING_FEAT_SYSWIDE_PASID
  82. 2. IOMMU_NESTING_FEAT_BIND_PGTBL
  83. 3. IOMMU_NESTING_FEAT_BIND_PASID_TABLE
  84. 4. IOMMU_NESTING_FEAT_CACHE_INVLD
  85. 5. IOMMU_NESTING_FEAT_PAGE_REQUEST
  86. Take VFIO as example, upon request from VFIO userspace (e.g. QEMU),
  87. VFIO kernel code shall query IOMMU vendor driver for the support of
  88. the above features. Query result can then be reported back to the
  89. userspace caller. Details can be found in
  90. Documentation/driver-api/vfio.rst.
  91. Data Passing Example with VFIO
  92. ------------------------------
  93. As the ubiquitous userspace driver framework, VFIO is already IOMMU
  94. aware and shares many key concepts such as device model, group, and
  95. protection domain. Other user driver frameworks can also be extended
  96. to support IOMMU UAPI but it is outside the scope of this document.
  97. In this tight-knit VFIO-IOMMU interface, the ultimate consumer of the
  98. IOMMU UAPI data is the host IOMMU driver. VFIO facilitates user-kernel
  99. transport, capability checking, security, and life cycle management of
  100. process address space ID (PASID).
  101. VFIO layer conveys the data structures down to the IOMMU driver. It
  102. follows the pattern below::
  103. struct {
  104. __u32 argsz;
  105. __u32 flags;
  106. __u8 data[];
  107. };
  108. Here data[] contains the IOMMU UAPI data structures. VFIO has the
  109. freedom to bundle the data as well as parse data size based on its own flags.
  110. In order to determine the size and feature set of the user data, argsz
  111. and flags (or the equivalent) are also embedded in the IOMMU UAPI data
  112. structures.
  113. A "__u32 argsz" field is *always* at the beginning of each structure.
  114. For example:
  115. ::
  116. struct iommu_cache_invalidate_info {
  117. __u32 argsz;
  118. #define IOMMU_CACHE_INVALIDATE_INFO_VERSION_1 1
  119. __u32 version;
  120. /* IOMMU paging structure cache */
  121. #define IOMMU_CACHE_INV_TYPE_IOTLB (1 << 0) /* IOMMU IOTLB */
  122. #define IOMMU_CACHE_INV_TYPE_DEV_IOTLB (1 << 1) /* Device IOTLB */
  123. #define IOMMU_CACHE_INV_TYPE_PASID (1 << 2) /* PASID cache */
  124. #define IOMMU_CACHE_INV_TYPE_NR (3)
  125. __u8 cache;
  126. __u8 granularity;
  127. __u8 padding[6];
  128. union {
  129. struct iommu_inv_pasid_info pasid_info;
  130. struct iommu_inv_addr_info addr_info;
  131. } granu;
  132. };
  133. VFIO is responsible for checking its own argsz and flags. It then
  134. invokes appropriate IOMMU UAPI functions. The user pointers are passed
  135. to the IOMMU layer for further processing. The responsibilities are
  136. divided as follows:
  137. - Generic IOMMU layer checks argsz range based on UAPI data in the
  138. current kernel version.
  139. - Generic IOMMU layer checks content of the UAPI data for non-zero
  140. reserved bits in flags, padding fields, and unsupported version.
  141. This is to ensure not breaking userspace in the future when these
  142. fields or flags are used.
  143. - Vendor IOMMU driver checks argsz based on vendor flags. UAPI data
  144. is consumed based on flags. Vendor driver has access to
  145. unadulterated argsz value in case of vendor specific future
  146. extensions. Currently, it does not perform the copy_from_user()
  147. itself. A __user pointer can be provided in some future scenarios
  148. where there's vendor data outside of the structure definition.
  149. IOMMU code treats UAPI data in two categories:
  150. - structure contains vendor data
  151. (Example: iommu_uapi_cache_invalidate())
  152. - structure contains only generic data
  153. (Example: iommu_uapi_sva_bind_gpasid())
  154. Sharing UAPI with in-kernel users
  155. ---------------------------------
  156. For UAPIs that are shared with in-kernel users, a wrapper function is
  157. provided to distinguish the callers. For example,
  158. Userspace caller ::
  159. int iommu_uapi_sva_unbind_gpasid(struct iommu_domain *domain,
  160. struct device *dev,
  161. void __user *udata)
  162. In-kernel caller ::
  163. int iommu_sva_unbind_gpasid(struct iommu_domain *domain,
  164. struct device *dev, ioasid_t ioasid);