blk-mq.rst 7.3 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153
  1. .. SPDX-License-Identifier: GPL-2.0
  2. ================================================
  3. Multi-Queue Block IO Queueing Mechanism (blk-mq)
  4. ================================================
  5. The Multi-Queue Block IO Queueing Mechanism is an API to enable fast storage
  6. devices to achieve a huge number of input/output operations per second (IOPS)
  7. through queueing and submitting IO requests to block devices simultaneously,
  8. benefiting from the parallelism offered by modern storage devices.
  9. Introduction
  10. ============
  11. Background
  12. ----------
  13. Magnetic hard disks have been the de facto standard from the beginning of the
  14. development of the kernel. The Block IO subsystem aimed to achieve the best
  15. performance possible for those devices with a high penalty when doing random
  16. access, and the bottleneck was the mechanical moving parts, a lot slower than
  17. any layer on the storage stack. One example of such optimization technique
  18. involves ordering read/write requests according to the current position of the
  19. hard disk head.
  20. However, with the development of Solid State Drives and Non-Volatile Memories
  21. without mechanical parts nor random access penalty and capable of performing
  22. high parallel access, the bottleneck of the stack had moved from the storage
  23. device to the operating system. In order to take advantage of the parallelism
  24. in those devices' design, the multi-queue mechanism was introduced.
  25. The former design had a single queue to store block IO requests with a single
  26. lock. That did not scale well in SMP systems due to dirty data in cache and the
  27. bottleneck of having a single lock for multiple processors. This setup also
  28. suffered with congestion when different processes (or the same process, moving
  29. to different CPUs) wanted to perform block IO. Instead of this, the blk-mq API
  30. spawns multiple queues with individual entry points local to the CPU, removing
  31. the need for a lock. A deeper explanation on how this works is covered in the
  32. following section (`Operation`_).
  33. Operation
  34. ---------
  35. When the userspace performs IO to a block device (reading or writing a file,
  36. for instance), blk-mq takes action: it will store and manage IO requests to
  37. the block device, acting as middleware between the userspace (and a file
  38. system, if present) and the block device driver.
  39. blk-mq has two group of queues: software staging queues and hardware dispatch
  40. queues. When the request arrives at the block layer, it will try the shortest
  41. path possible: send it directly to the hardware queue. However, there are two
  42. cases that it might not do that: if there's an IO scheduler attached at the
  43. layer or if we want to try to merge requests. In both cases, requests will be
  44. sent to the software queue.
  45. Then, after the requests are processed by software queues, they will be placed
  46. at the hardware queue, a second stage queue were the hardware has direct access
  47. to process those requests. However, if the hardware does not have enough
  48. resources to accept more requests, blk-mq will places requests on a temporary
  49. queue, to be sent in the future, when the hardware is able.
  50. Software staging queues
  51. ~~~~~~~~~~~~~~~~~~~~~~~
  52. The block IO subsystem adds requests in the software staging queues
  53. (represented by struct blk_mq_ctx) in case that they weren't sent
  54. directly to the driver. A request is one or more BIOs. They arrived at the
  55. block layer through the data structure struct bio. The block layer
  56. will then build a new structure from it, the struct request that will
  57. be used to communicate with the device driver. Each queue has its own lock and
  58. the number of queues is defined by a per-CPU or per-node basis.
  59. The staging queue can be used to merge requests for adjacent sectors. For
  60. instance, requests for sector 3-6, 6-7, 7-9 can become one request for 3-9.
  61. Even if random access to SSDs and NVMs have the same time of response compared
  62. to sequential access, grouped requests for sequential access decreases the
  63. number of individual requests. This technique of merging requests is called
  64. plugging.
  65. Along with that, the requests can be reordered to ensure fairness of system
  66. resources (e.g. to ensure that no application suffers from starvation) and/or to
  67. improve IO performance, by an IO scheduler.
  68. IO Schedulers
  69. ^^^^^^^^^^^^^
  70. There are several schedulers implemented by the block layer, each one following
  71. a heuristic to improve the IO performance. They are "pluggable" (as in plug
  72. and play), in the sense of they can be selected at run time using sysfs. You
  73. can read more about Linux's IO schedulers `here
  74. <https://www.kernel.org/doc/html/latest/block/index.html>`_. The scheduling
  75. happens only between requests in the same queue, so it is not possible to merge
  76. requests from different queues, otherwise there would be cache trashing and a
  77. need to have a lock for each queue. After the scheduling, the requests are
  78. eligible to be sent to the hardware. One of the possible schedulers to be
  79. selected is the NONE scheduler, the most straightforward one. It will just
  80. place requests on whatever software queue the process is running on, without
  81. any reordering. When the device starts processing requests in the hardware
  82. queue (a.k.a. run the hardware queue), the software queues mapped to that
  83. hardware queue will be drained in sequence according to their mapping.
  84. Hardware dispatch queues
  85. ~~~~~~~~~~~~~~~~~~~~~~~~
  86. The hardware queue (represented by struct blk_mq_hw_ctx) is a struct
  87. used by device drivers to map the device submission queues (or device DMA ring
  88. buffer), and are the last step of the block layer submission code before the
  89. low level device driver taking ownership of the request. To run this queue, the
  90. block layer removes requests from the associated software queues and tries to
  91. dispatch to the hardware.
  92. If it's not possible to send the requests directly to hardware, they will be
  93. added to a linked list (``hctx->dispatch``) of requests. Then,
  94. next time the block layer runs a queue, it will send the requests laying at the
  95. ``dispatch`` list first, to ensure a fairness dispatch with those
  96. requests that were ready to be sent first. The number of hardware queues
  97. depends on the number of hardware contexts supported by the hardware and its
  98. device driver, but it will not be more than the number of cores of the system.
  99. There is no reordering at this stage, and each software queue has a set of
  100. hardware queues to send requests for.
  101. .. note::
  102. Neither the block layer nor the device protocols guarantee
  103. the order of completion of requests. This must be handled by
  104. higher layers, like the filesystem.
  105. Tag-based completion
  106. ~~~~~~~~~~~~~~~~~~~~
  107. In order to indicate which request has been completed, every request is
  108. identified by an integer, ranging from 0 to the dispatch queue size. This tag
  109. is generated by the block layer and later reused by the device driver, removing
  110. the need to create a redundant identifier. When a request is completed in the
  111. drive, the tag is sent back to the block layer to notify it of the finalization.
  112. This removes the need to do a linear search to find out which IO has been
  113. completed.
  114. Further reading
  115. ---------------
  116. - `Linux Block IO: Introducing Multi-queue SSD Access on Multi-core Systems <http://kernel.dk/blk-mq.pdf>`_
  117. - `NOOP scheduler <https://en.wikipedia.org/wiki/Noop_scheduler>`_
  118. - `Null block device driver <https://www.kernel.org/doc/html/latest/block/null_blk.html>`_
  119. Source code documentation
  120. =========================
  121. .. kernel-doc:: include/linux/blk-mq.h
  122. .. kernel-doc:: block/blk-mq.c