kcsan.rst 14 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316
  1. The Kernel Concurrency Sanitizer (KCSAN)
  2. ========================================
  3. The Kernel Concurrency Sanitizer (KCSAN) is a dynamic race detector, which
  4. relies on compile-time instrumentation, and uses a watchpoint-based sampling
  5. approach to detect races. KCSAN's primary purpose is to detect `data races`_.
  6. Usage
  7. -----
  8. KCSAN is supported by both GCC and Clang. With GCC we require version 11 or
  9. later, and with Clang also require version 11 or later.
  10. To enable KCSAN configure the kernel with::
  11. CONFIG_KCSAN = y
  12. KCSAN provides several other configuration options to customize behaviour (see
  13. the respective help text in ``lib/Kconfig.kcsan`` for more info).
  14. Error reports
  15. ~~~~~~~~~~~~~
  16. A typical data race report looks like this::
  17. ==================================================================
  18. BUG: KCSAN: data-race in generic_permission / kernfs_refresh_inode
  19. write to 0xffff8fee4c40700c of 4 bytes by task 175 on cpu 4:
  20. kernfs_refresh_inode+0x70/0x170
  21. kernfs_iop_permission+0x4f/0x90
  22. inode_permission+0x190/0x200
  23. link_path_walk.part.0+0x503/0x8e0
  24. path_lookupat.isra.0+0x69/0x4d0
  25. filename_lookup+0x136/0x280
  26. user_path_at_empty+0x47/0x60
  27. vfs_statx+0x9b/0x130
  28. __do_sys_newlstat+0x50/0xb0
  29. __x64_sys_newlstat+0x37/0x50
  30. do_syscall_64+0x85/0x260
  31. entry_SYSCALL_64_after_hwframe+0x44/0xa9
  32. read to 0xffff8fee4c40700c of 4 bytes by task 166 on cpu 6:
  33. generic_permission+0x5b/0x2a0
  34. kernfs_iop_permission+0x66/0x90
  35. inode_permission+0x190/0x200
  36. link_path_walk.part.0+0x503/0x8e0
  37. path_lookupat.isra.0+0x69/0x4d0
  38. filename_lookup+0x136/0x280
  39. user_path_at_empty+0x47/0x60
  40. do_faccessat+0x11a/0x390
  41. __x64_sys_access+0x3c/0x50
  42. do_syscall_64+0x85/0x260
  43. entry_SYSCALL_64_after_hwframe+0x44/0xa9
  44. Reported by Kernel Concurrency Sanitizer on:
  45. CPU: 6 PID: 166 Comm: systemd-journal Not tainted 5.3.0-rc7+ #1
  46. Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
  47. ==================================================================
  48. The header of the report provides a short summary of the functions involved in
  49. the race. It is followed by the access types and stack traces of the 2 threads
  50. involved in the data race.
  51. The other less common type of data race report looks like this::
  52. ==================================================================
  53. BUG: KCSAN: data-race in e1000_clean_rx_irq+0x551/0xb10
  54. race at unknown origin, with read to 0xffff933db8a2ae6c of 1 bytes by interrupt on cpu 0:
  55. e1000_clean_rx_irq+0x551/0xb10
  56. e1000_clean+0x533/0xda0
  57. net_rx_action+0x329/0x900
  58. __do_softirq+0xdb/0x2db
  59. irq_exit+0x9b/0xa0
  60. do_IRQ+0x9c/0xf0
  61. ret_from_intr+0x0/0x18
  62. default_idle+0x3f/0x220
  63. arch_cpu_idle+0x21/0x30
  64. do_idle+0x1df/0x230
  65. cpu_startup_entry+0x14/0x20
  66. rest_init+0xc5/0xcb
  67. arch_call_rest_init+0x13/0x2b
  68. start_kernel+0x6db/0x700
  69. Reported by Kernel Concurrency Sanitizer on:
  70. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.3.0-rc7+ #2
  71. Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
  72. ==================================================================
  73. This report is generated where it was not possible to determine the other
  74. racing thread, but a race was inferred due to the data value of the watched
  75. memory location having changed. These can occur either due to missing
  76. instrumentation or e.g. DMA accesses. These reports will only be generated if
  77. ``CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN=y`` (selected by default).
  78. Selective analysis
  79. ~~~~~~~~~~~~~~~~~~
  80. It may be desirable to disable data race detection for specific accesses,
  81. functions, compilation units, or entire subsystems. For static blacklisting,
  82. the below options are available:
  83. * KCSAN understands the ``data_race(expr)`` annotation, which tells KCSAN that
  84. any data races due to accesses in ``expr`` should be ignored and resulting
  85. behaviour when encountering a data race is deemed safe.
  86. * Disabling data race detection for entire functions can be accomplished by
  87. using the function attribute ``__no_kcsan``::
  88. __no_kcsan
  89. void foo(void) {
  90. ...
  91. To dynamically limit for which functions to generate reports, see the
  92. `DebugFS interface`_ blacklist/whitelist feature.
  93. * To disable data race detection for a particular compilation unit, add to the
  94. ``Makefile``::
  95. KCSAN_SANITIZE_file.o := n
  96. * To disable data race detection for all compilation units listed in a
  97. ``Makefile``, add to the respective ``Makefile``::
  98. KCSAN_SANITIZE := n
  99. Furthermore, it is possible to tell KCSAN to show or hide entire classes of
  100. data races, depending on preferences. These can be changed via the following
  101. Kconfig options:
  102. * ``CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY``: If enabled and a conflicting write
  103. is observed via a watchpoint, but the data value of the memory location was
  104. observed to remain unchanged, do not report the data race.
  105. * ``CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC``: Assume that plain aligned writes
  106. up to word size are atomic by default. Assumes that such writes are not
  107. subject to unsafe compiler optimizations resulting in data races. The option
  108. causes KCSAN to not report data races due to conflicts where the only plain
  109. accesses are aligned writes up to word size.
  110. DebugFS interface
  111. ~~~~~~~~~~~~~~~~~
  112. The file ``/sys/kernel/debug/kcsan`` provides the following interface:
  113. * Reading ``/sys/kernel/debug/kcsan`` returns various runtime statistics.
  114. * Writing ``on`` or ``off`` to ``/sys/kernel/debug/kcsan`` allows turning KCSAN
  115. on or off, respectively.
  116. * Writing ``!some_func_name`` to ``/sys/kernel/debug/kcsan`` adds
  117. ``some_func_name`` to the report filter list, which (by default) blacklists
  118. reporting data races where either one of the top stackframes are a function
  119. in the list.
  120. * Writing either ``blacklist`` or ``whitelist`` to ``/sys/kernel/debug/kcsan``
  121. changes the report filtering behaviour. For example, the blacklist feature
  122. can be used to silence frequently occurring data races; the whitelist feature
  123. can help with reproduction and testing of fixes.
  124. Tuning performance
  125. ~~~~~~~~~~~~~~~~~~
  126. Core parameters that affect KCSAN's overall performance and bug detection
  127. ability are exposed as kernel command-line arguments whose defaults can also be
  128. changed via the corresponding Kconfig options.
  129. * ``kcsan.skip_watch`` (``CONFIG_KCSAN_SKIP_WATCH``): Number of per-CPU memory
  130. operations to skip, before another watchpoint is set up. Setting up
  131. watchpoints more frequently will result in the likelihood of races to be
  132. observed to increase. This parameter has the most significant impact on
  133. overall system performance and race detection ability.
  134. * ``kcsan.udelay_task`` (``CONFIG_KCSAN_UDELAY_TASK``): For tasks, the
  135. microsecond delay to stall execution after a watchpoint has been set up.
  136. Larger values result in the window in which we may observe a race to
  137. increase.
  138. * ``kcsan.udelay_interrupt`` (``CONFIG_KCSAN_UDELAY_INTERRUPT``): For
  139. interrupts, the microsecond delay to stall execution after a watchpoint has
  140. been set up. Interrupts have tighter latency requirements, and their delay
  141. should generally be smaller than the one chosen for tasks.
  142. They may be tweaked at runtime via ``/sys/module/kcsan/parameters/``.
  143. Data Races
  144. ----------
  145. In an execution, two memory accesses form a *data race* if they *conflict*,
  146. they happen concurrently in different threads, and at least one of them is a
  147. *plain access*; they *conflict* if both access the same memory location, and at
  148. least one is a write. For a more thorough discussion and definition, see `"Plain
  149. Accesses and Data Races" in the LKMM`_.
  150. .. _"Plain Accesses and Data Races" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/explanation.txt#n1922
  151. Relationship with the Linux-Kernel Memory Consistency Model (LKMM)
  152. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  153. The LKMM defines the propagation and ordering rules of various memory
  154. operations, which gives developers the ability to reason about concurrent code.
  155. Ultimately this allows to determine the possible executions of concurrent code,
  156. and if that code is free from data races.
  157. KCSAN is aware of *marked atomic operations* (``READ_ONCE``, ``WRITE_ONCE``,
  158. ``atomic_*``, etc.), but is oblivious of any ordering guarantees and simply
  159. assumes that memory barriers are placed correctly. In other words, KCSAN
  160. assumes that as long as a plain access is not observed to race with another
  161. conflicting access, memory operations are correctly ordered.
  162. This means that KCSAN will not report *potential* data races due to missing
  163. memory ordering. Developers should therefore carefully consider the required
  164. memory ordering requirements that remain unchecked. If, however, missing
  165. memory ordering (that is observable with a particular compiler and
  166. architecture) leads to an observable data race (e.g. entering a critical
  167. section erroneously), KCSAN would report the resulting data race.
  168. Race Detection Beyond Data Races
  169. --------------------------------
  170. For code with complex concurrency design, race-condition bugs may not always
  171. manifest as data races. Race conditions occur if concurrently executing
  172. operations result in unexpected system behaviour. On the other hand, data races
  173. are defined at the C-language level. The following macros can be used to check
  174. properties of concurrent code where bugs would not manifest as data races.
  175. .. kernel-doc:: include/linux/kcsan-checks.h
  176. :functions: ASSERT_EXCLUSIVE_WRITER ASSERT_EXCLUSIVE_WRITER_SCOPED
  177. ASSERT_EXCLUSIVE_ACCESS ASSERT_EXCLUSIVE_ACCESS_SCOPED
  178. ASSERT_EXCLUSIVE_BITS
  179. Implementation Details
  180. ----------------------
  181. KCSAN relies on observing that two accesses happen concurrently. Crucially, we
  182. want to (a) increase the chances of observing races (especially for races that
  183. manifest rarely), and (b) be able to actually observe them. We can accomplish
  184. (a) by injecting various delays, and (b) by using address watchpoints (or
  185. breakpoints).
  186. If we deliberately stall a memory access, while we have a watchpoint for its
  187. address set up, and then observe the watchpoint to fire, two accesses to the
  188. same address just raced. Using hardware watchpoints, this is the approach taken
  189. in `DataCollider
  190. <http://usenix.org/legacy/events/osdi10/tech/full_papers/Erickson.pdf>`_.
  191. Unlike DataCollider, KCSAN does not use hardware watchpoints, but instead
  192. relies on compiler instrumentation and "soft watchpoints".
  193. In KCSAN, watchpoints are implemented using an efficient encoding that stores
  194. access type, size, and address in a long; the benefits of using "soft
  195. watchpoints" are portability and greater flexibility. KCSAN then relies on the
  196. compiler instrumenting plain accesses. For each instrumented plain access:
  197. 1. Check if a matching watchpoint exists; if yes, and at least one access is a
  198. write, then we encountered a racing access.
  199. 2. Periodically, if no matching watchpoint exists, set up a watchpoint and
  200. stall for a small randomized delay.
  201. 3. Also check the data value before the delay, and re-check the data value
  202. after delay; if the values mismatch, we infer a race of unknown origin.
  203. To detect data races between plain and marked accesses, KCSAN also annotates
  204. marked accesses, but only to check if a watchpoint exists; i.e. KCSAN never
  205. sets up a watchpoint on marked accesses. By never setting up watchpoints for
  206. marked operations, if all accesses to a variable that is accessed concurrently
  207. are properly marked, KCSAN will never trigger a watchpoint and therefore never
  208. report the accesses.
  209. Key Properties
  210. ~~~~~~~~~~~~~~
  211. 1. **Memory Overhead:** The overall memory overhead is only a few MiB
  212. depending on configuration. The current implementation uses a small array of
  213. longs to encode watchpoint information, which is negligible.
  214. 2. **Performance Overhead:** KCSAN's runtime aims to be minimal, using an
  215. efficient watchpoint encoding that does not require acquiring any shared
  216. locks in the fast-path. For kernel boot on a system with 8 CPUs:
  217. - 5.0x slow-down with the default KCSAN config;
  218. - 2.8x slow-down from runtime fast-path overhead only (set very large
  219. ``KCSAN_SKIP_WATCH`` and unset ``KCSAN_SKIP_WATCH_RANDOMIZE``).
  220. 3. **Annotation Overheads:** Minimal annotations are required outside the KCSAN
  221. runtime. As a result, maintenance overheads are minimal as the kernel
  222. evolves.
  223. 4. **Detects Racy Writes from Devices:** Due to checking data values upon
  224. setting up watchpoints, racy writes from devices can also be detected.
  225. 5. **Memory Ordering:** KCSAN is *not* explicitly aware of the LKMM's ordering
  226. rules; this may result in missed data races (false negatives).
  227. 6. **Analysis Accuracy:** For observed executions, due to using a sampling
  228. strategy, the analysis is *unsound* (false negatives possible), but aims to
  229. be complete (no false positives).
  230. Alternatives Considered
  231. -----------------------
  232. An alternative data race detection approach for the kernel can be found in the
  233. `Kernel Thread Sanitizer (KTSAN) <https://github.com/google/ktsan/wiki>`_.
  234. KTSAN is a happens-before data race detector, which explicitly establishes the
  235. happens-before order between memory operations, which can then be used to
  236. determine data races as defined in `Data Races`_.
  237. To build a correct happens-before relation, KTSAN must be aware of all ordering
  238. rules of the LKMM and synchronization primitives. Unfortunately, any omission
  239. leads to large numbers of false positives, which is especially detrimental in
  240. the context of the kernel which includes numerous custom synchronization
  241. mechanisms. To track the happens-before relation, KTSAN's implementation
  242. requires metadata for each memory location (shadow memory), which for each page
  243. corresponds to 4 pages of shadow memory, and can translate into overhead of
  244. tens of GiB on a large system.