uacce.rst 7.0 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176
  1. .. SPDX-License-Identifier: GPL-2.0
  2. Introduction of Uacce
  3. ---------------------
  4. Uacce (Unified/User-space-access-intended Accelerator Framework) targets to
  5. provide Shared Virtual Addressing (SVA) between accelerators and processes.
  6. So accelerator can access any data structure of the main cpu.
  7. This differs from the data sharing between cpu and io device, which share
  8. only data content rather than address.
  9. Because of the unified address, hardware and user space of process can
  10. share the same virtual address in the communication.
  11. Uacce takes the hardware accelerator as a heterogeneous processor, while
  12. IOMMU share the same CPU page tables and as a result the same translation
  13. from va to pa.
  14. ::
  15. __________________________ __________________________
  16. | | | |
  17. | User application (CPU) | | Hardware Accelerator |
  18. |__________________________| |__________________________|
  19. | |
  20. | va | va
  21. V V
  22. __________ __________
  23. | | | |
  24. | MMU | | IOMMU |
  25. |__________| |__________|
  26. | |
  27. | |
  28. V pa V pa
  29. _______________________________________
  30. | |
  31. | Memory |
  32. |_______________________________________|
  33. Architecture
  34. ------------
  35. Uacce is the kernel module, taking charge of iommu and address sharing.
  36. The user drivers and libraries are called WarpDrive.
  37. The uacce device, built around the IOMMU SVA API, can access multiple
  38. address spaces, including the one without PASID.
  39. A virtual concept, queue, is used for the communication. It provides a
  40. FIFO-like interface. And it maintains a unified address space between the
  41. application and all involved hardware.
  42. ::
  43. ___________________ ________________
  44. | | user API | |
  45. | WarpDrive library | ------------> | user driver |
  46. |___________________| |________________|
  47. | |
  48. | |
  49. | queue fd |
  50. | |
  51. | |
  52. v |
  53. ___________________ _________ |
  54. | | | | | mmap memory
  55. | Other framework | | uacce | | r/w interface
  56. | crypto/nic/others | |_________| |
  57. |___________________| |
  58. | | |
  59. | register | register |
  60. | | |
  61. | | |
  62. | _________________ __________ |
  63. | | | | | |
  64. ------------- | Device Driver | | IOMMU | |
  65. |_________________| |__________| |
  66. | |
  67. | V
  68. | ___________________
  69. | | |
  70. -------------------------- | Device(Hardware) |
  71. |___________________|
  72. How does it work
  73. ----------------
  74. Uacce uses mmap and IOMMU to play the trick.
  75. Uacce creates a chrdev for every device registered to it. New queue is
  76. created when user application open the chrdev. The file descriptor is used
  77. as the user handle of the queue.
  78. The accelerator device present itself as an Uacce object, which exports as
  79. a chrdev to the user space. The user application communicates with the
  80. hardware by ioctl (as control path) or share memory (as data path).
  81. The control path to the hardware is via file operation, while data path is
  82. via mmap space of the queue fd.
  83. The queue file address space:
  84. ::
  85. /**
  86. * enum uacce_qfrt: qfrt type
  87. * @UACCE_QFRT_MMIO: device mmio region
  88. * @UACCE_QFRT_DUS: device user share region
  89. */
  90. enum uacce_qfrt {
  91. UACCE_QFRT_MMIO = 0,
  92. UACCE_QFRT_DUS = 1,
  93. };
  94. All regions are optional and differ from device type to type.
  95. Each region can be mmapped only once, otherwise -EEXIST returns.
  96. The device mmio region is mapped to the hardware mmio space. It is generally
  97. used for doorbell or other notification to the hardware. It is not fast enough
  98. as data channel.
  99. The device user share region is used for share data buffer between user process
  100. and device.
  101. The Uacce register API
  102. ----------------------
  103. The register API is defined in uacce.h.
  104. ::
  105. struct uacce_interface {
  106. char name[UACCE_MAX_NAME_SIZE];
  107. unsigned int flags;
  108. const struct uacce_ops *ops;
  109. };
  110. According to the IOMMU capability, uacce_interface flags can be:
  111. ::
  112. /**
  113. * UACCE Device flags:
  114. * UACCE_DEV_SVA: Shared Virtual Addresses
  115. * Support PASID
  116. * Support device page faults (PCI PRI or SMMU Stall)
  117. */
  118. #define UACCE_DEV_SVA BIT(0)
  119. struct uacce_device *uacce_alloc(struct device *parent,
  120. struct uacce_interface *interface);
  121. int uacce_register(struct uacce_device *uacce);
  122. void uacce_remove(struct uacce_device *uacce);
  123. uacce_register results can be:
  124. a. If uacce module is not compiled, ERR_PTR(-ENODEV)
  125. b. Succeed with the desired flags
  126. c. Succeed with the negotiated flags, for example
  127. uacce_interface.flags = UACCE_DEV_SVA but uacce->flags = ~UACCE_DEV_SVA
  128. So user driver need check return value as well as the negotiated uacce->flags.
  129. The user driver
  130. ---------------
  131. The queue file mmap space will need a user driver to wrap the communication
  132. protocol. Uacce provides some attributes in sysfs for the user driver to
  133. match the right accelerator accordingly.
  134. More details in Documentation/ABI/testing/sysfs-driver-uacce.