locktorture.rst 5.6 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169
  1. ==================================
  2. Kernel Lock Torture Test Operation
  3. ==================================
  4. CONFIG_LOCK_TORTURE_TEST
  5. ========================
  6. The CONFIG LOCK_TORTURE_TEST config option provides a kernel module
  7. that runs torture tests on core kernel locking primitives. The kernel
  8. module, 'locktorture', may be built after the fact on the running
  9. kernel to be tested, if desired. The tests periodically output status
  10. messages via printk(), which can be examined via the dmesg (perhaps
  11. grepping for "torture"). The test is started when the module is loaded,
  12. and stops when the module is unloaded. This program is based on how RCU
  13. is tortured, via rcutorture.
  14. This torture test consists of creating a number of kernel threads which
  15. acquire the lock and hold it for specific amount of time, thus simulating
  16. different critical region behaviors. The amount of contention on the lock
  17. can be simulated by either enlarging this critical region hold time and/or
  18. creating more kthreads.
  19. Module Parameters
  20. =================
  21. This module has the following parameters:
  22. Locktorture-specific
  23. --------------------
  24. nwriters_stress
  25. Number of kernel threads that will stress exclusive lock
  26. ownership (writers). The default value is twice the number
  27. of online CPUs.
  28. nreaders_stress
  29. Number of kernel threads that will stress shared lock
  30. ownership (readers). The default is the same amount of writer
  31. locks. If the user did not specify nwriters_stress, then
  32. both readers and writers be the amount of online CPUs.
  33. torture_type
  34. Type of lock to torture. By default, only spinlocks will
  35. be tortured. This module can torture the following locks,
  36. with string values as follows:
  37. - "lock_busted":
  38. Simulates a buggy lock implementation.
  39. - "spin_lock":
  40. spin_lock() and spin_unlock() pairs.
  41. - "spin_lock_irq":
  42. spin_lock_irq() and spin_unlock_irq() pairs.
  43. - "rw_lock":
  44. read/write lock() and unlock() rwlock pairs.
  45. - "rw_lock_irq":
  46. read/write lock_irq() and unlock_irq()
  47. rwlock pairs.
  48. - "mutex_lock":
  49. mutex_lock() and mutex_unlock() pairs.
  50. - "rtmutex_lock":
  51. rtmutex_lock() and rtmutex_unlock() pairs.
  52. Kernel must have CONFIG_RT_MUTEX=y.
  53. - "rwsem_lock":
  54. read/write down() and up() semaphore pairs.
  55. Torture-framework (RCU + locking)
  56. ---------------------------------
  57. shutdown_secs
  58. The number of seconds to run the test before terminating
  59. the test and powering off the system. The default is
  60. zero, which disables test termination and system shutdown.
  61. This capability is useful for automated testing.
  62. onoff_interval
  63. The number of seconds between each attempt to execute a
  64. randomly selected CPU-hotplug operation. Defaults
  65. to zero, which disables CPU hotplugging. In
  66. CONFIG_HOTPLUG_CPU=n kernels, locktorture will silently
  67. refuse to do any CPU-hotplug operations regardless of
  68. what value is specified for onoff_interval.
  69. onoff_holdoff
  70. The number of seconds to wait until starting CPU-hotplug
  71. operations. This would normally only be used when
  72. locktorture was built into the kernel and started
  73. automatically at boot time, in which case it is useful
  74. in order to avoid confusing boot-time code with CPUs
  75. coming and going. This parameter is only useful if
  76. CONFIG_HOTPLUG_CPU is enabled.
  77. stat_interval
  78. Number of seconds between statistics-related printk()s.
  79. By default, locktorture will report stats every 60 seconds.
  80. Setting the interval to zero causes the statistics to
  81. be printed -only- when the module is unloaded.
  82. stutter
  83. The length of time to run the test before pausing for this
  84. same period of time. Defaults to "stutter=5", so as
  85. to run and pause for (roughly) five-second intervals.
  86. Specifying "stutter=0" causes the test to run continuously
  87. without pausing.
  88. shuffle_interval
  89. The number of seconds to keep the test threads affinitied
  90. to a particular subset of the CPUs, defaults to 3 seconds.
  91. Used in conjunction with test_no_idle_hz.
  92. verbose
  93. Enable verbose debugging printing, via printk(). Enabled
  94. by default. This extra information is mostly related to
  95. high-level errors and reports from the main 'torture'
  96. framework.
  97. Statistics
  98. ==========
  99. Statistics are printed in the following format::
  100. spin_lock-torture: Writes: Total: 93746064 Max/Min: 0/0 Fail: 0
  101. (A) (B) (C) (D) (E)
  102. (A): Lock type that is being tortured -- torture_type parameter.
  103. (B): Number of writer lock acquisitions. If dealing with a read/write
  104. primitive a second "Reads" statistics line is printed.
  105. (C): Number of times the lock was acquired.
  106. (D): Min and max number of times threads failed to acquire the lock.
  107. (E): true/false values if there were errors acquiring the lock. This should
  108. -only- be positive if there is a bug in the locking primitive's
  109. implementation. Otherwise a lock should never fail (i.e., spin_lock()).
  110. Of course, the same applies for (C), above. A dummy example of this is
  111. the "lock_busted" type.
  112. Usage
  113. =====
  114. The following script may be used to torture locks::
  115. #!/bin/sh
  116. modprobe locktorture
  117. sleep 3600
  118. rmmod locktorture
  119. dmesg | grep torture:
  120. The output can be manually inspected for the error flag of "!!!".
  121. One could of course create a more elaborate script that automatically
  122. checked for such errors. The "rmmod" command forces a "SUCCESS",
  123. "FAILURE", or "RCU_HOTPLUG" indication to be printk()ed. The first
  124. two are self-explanatory, while the last indicates that while there
  125. were no locking failures, CPU-hotplug problems were detected.
  126. Also see: Documentation/RCU/torture.rst