kselftest.rst 11 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350
  1. ======================
  2. Linux Kernel Selftests
  3. ======================
  4. The kernel contains a set of "self tests" under the tools/testing/selftests/
  5. directory. These are intended to be small tests to exercise individual code
  6. paths in the kernel. Tests are intended to be run after building, installing
  7. and booting a kernel.
  8. You can find additional information on Kselftest framework, how to
  9. write new tests using the framework on Kselftest wiki:
  10. https://kselftest.wiki.kernel.org/
  11. On some systems, hot-plug tests could hang forever waiting for cpu and
  12. memory to be ready to be offlined. A special hot-plug target is created
  13. to run the full range of hot-plug tests. In default mode, hot-plug tests run
  14. in safe mode with a limited scope. In limited mode, cpu-hotplug test is
  15. run on a single cpu as opposed to all hotplug capable cpus, and memory
  16. hotplug test is run on 2% of hotplug capable memory instead of 10%.
  17. kselftest runs as a userspace process. Tests that can be written/run in
  18. userspace may wish to use the `Test Harness`_. Tests that need to be
  19. run in kernel space may wish to use a `Test Module`_.
  20. Running the selftests (hotplug tests are run in limited mode)
  21. =============================================================
  22. To build the tests::
  23. $ make -C tools/testing/selftests
  24. To run the tests::
  25. $ make -C tools/testing/selftests run_tests
  26. To build and run the tests with a single command, use::
  27. $ make kselftest
  28. Note that some tests will require root privileges.
  29. Kselftest supports saving output files in a separate directory and then
  30. running tests. To locate output files in a separate directory two syntaxes
  31. are supported. In both cases the working directory must be the root of the
  32. kernel src. This is applicable to "Running a subset of selftests" section
  33. below.
  34. To build, save output files in a separate directory with O= ::
  35. $ make O=/tmp/kselftest kselftest
  36. To build, save output files in a separate directory with KBUILD_OUTPUT ::
  37. $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest
  38. The O= assignment takes precedence over the KBUILD_OUTPUT environment
  39. variable.
  40. The above commands by default run the tests and print full pass/fail report.
  41. Kselftest supports "summary" option to make it easier to understand the test
  42. results. Please find the detailed individual test results for each test in
  43. /tmp/testname file(s) when summary option is specified. This is applicable
  44. to "Running a subset of selftests" section below.
  45. To run kselftest with summary option enabled ::
  46. $ make summary=1 kselftest
  47. Running a subset of selftests
  48. =============================
  49. You can use the "TARGETS" variable on the make command line to specify
  50. single test to run, or a list of tests to run.
  51. To run only tests targeted for a single subsystem::
  52. $ make -C tools/testing/selftests TARGETS=ptrace run_tests
  53. You can specify multiple tests to build and run::
  54. $ make TARGETS="size timers" kselftest
  55. To build, save output files in a separate directory with O= ::
  56. $ make O=/tmp/kselftest TARGETS="size timers" kselftest
  57. To build, save output files in a separate directory with KBUILD_OUTPUT ::
  58. $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest
  59. Additionally you can use the "SKIP_TARGETS" variable on the make command
  60. line to specify one or more targets to exclude from the TARGETS list.
  61. To run all tests but a single subsystem::
  62. $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests
  63. You can specify multiple tests to skip::
  64. $ make SKIP_TARGETS="size timers" kselftest
  65. You can also specify a restricted list of tests to run together with a
  66. dedicated skiplist::
  67. $ make TARGETS="bpf breakpoints size timers" SKIP_TARGETS=bpf kselftest
  68. See the top-level tools/testing/selftests/Makefile for the list of all
  69. possible targets.
  70. Running the full range hotplug selftests
  71. ========================================
  72. To build the hotplug tests::
  73. $ make -C tools/testing/selftests hotplug
  74. To run the hotplug tests::
  75. $ make -C tools/testing/selftests run_hotplug
  76. Note that some tests will require root privileges.
  77. Install selftests
  78. =================
  79. You can use the "install" target of "make" (which calls the `kselftest_install.sh`
  80. tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
  81. or in a user specified location via the `INSTALL_PATH` "make" variable.
  82. To install selftests in default location::
  83. $ make -C tools/testing/selftests install
  84. To install selftests in a user specified location::
  85. $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
  86. Running installed selftests
  87. ===========================
  88. Found in the install directory, as well as in the Kselftest tarball,
  89. is a script named `run_kselftest.sh` to run the tests.
  90. You can simply do the following to run the installed Kselftests. Please
  91. note some tests will require root privileges::
  92. $ cd kselftest_install
  93. $ ./run_kselftest.sh
  94. To see the list of available tests, the `-l` option can be used::
  95. $ ./run_kselftest.sh -l
  96. The `-c` option can be used to run all the tests from a test collection, or
  97. the `-t` option for specific single tests. Either can be used multiple times::
  98. $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep
  99. For other features see the script usage output, seen with the `-h` option.
  100. Packaging selftests
  101. ===================
  102. In some cases packaging is desired, such as when tests need to run on a
  103. different system. To package selftests, run::
  104. $ make -C tools/testing/selftests gen_tar
  105. This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
  106. default, `.gz` format is used. The tar compression format can be overridden by
  107. specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
  108. option is supported, such as::
  109. $ make -C tools/testing/selftests gen_tar FORMAT=.xz
  110. `make gen_tar` invokes `make install` so you can use it to package a subset of
  111. tests by using variables specified in `Running a subset of selftests`_
  112. section::
  113. $ make -C tools/testing/selftests gen_tar TARGETS="bpf" FORMAT=.xz
  114. .. _tar's auto-compress: https://www.gnu.org/software/tar/manual/html_node/gzip.html#auto_002dcompress
  115. Contributing new tests
  116. ======================
  117. In general, the rules for selftests are
  118. * Do as much as you can if you're not root;
  119. * Don't take too long;
  120. * Don't break the build on any architecture, and
  121. * Don't cause the top-level "make run_tests" to fail if your feature is
  122. unconfigured.
  123. Contributing new tests (details)
  124. ================================
  125. * Use TEST_GEN_XXX if such binaries or files are generated during
  126. compiling.
  127. TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by
  128. default.
  129. TEST_CUSTOM_PROGS should be used by tests that require custom build
  130. rules and prevent common build rule use.
  131. TEST_PROGS are for test shell scripts. Please ensure shell script has
  132. its exec bit set. Otherwise, lib.mk run_tests will generate a warning.
  133. TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests.
  134. TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the
  135. executable which is not tested by default.
  136. TEST_FILES, TEST_GEN_FILES mean it is the file which is used by
  137. test.
  138. * First use the headers inside the kernel source and/or git repo, and then the
  139. system headers. Headers for the kernel release as opposed to headers
  140. installed by the distro on the system should be the primary focus to be able
  141. to find regressions.
  142. * If a test needs specific kernel config options enabled, add a config file in
  143. the test directory to enable them.
  144. e.g: tools/testing/selftests/android/config
  145. Test Module
  146. ===========
  147. Kselftest tests the kernel from userspace. Sometimes things need
  148. testing from within the kernel, one method of doing this is to create a
  149. test module. We can tie the module into the kselftest framework by
  150. using a shell script test runner. ``kselftest/module.sh`` is designed
  151. to facilitate this process. There is also a header file provided to
  152. assist writing kernel modules that are for use with kselftest:
  153. - ``tools/testing/kselftest/kselftest_module.h``
  154. - ``tools/testing/kselftest/kselftest/module.sh``
  155. How to use
  156. ----------
  157. Here we show the typical steps to create a test module and tie it into
  158. kselftest. We use kselftests for lib/ as an example.
  159. 1. Create the test module
  160. 2. Create the test script that will run (load/unload) the module
  161. e.g. ``tools/testing/selftests/lib/printf.sh``
  162. 3. Add line to config file e.g. ``tools/testing/selftests/lib/config``
  163. 4. Add test script to makefile e.g. ``tools/testing/selftests/lib/Makefile``
  164. 5. Verify it works:
  165. .. code-block:: sh
  166. # Assumes you have booted a fresh build of this kernel tree
  167. cd /path/to/linux/tree
  168. make kselftest-merge
  169. make modules
  170. sudo make modules_install
  171. make TARGETS=lib kselftest
  172. Example Module
  173. --------------
  174. A bare bones test module might look like this:
  175. .. code-block:: c
  176. // SPDX-License-Identifier: GPL-2.0+
  177. #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
  178. #include "../tools/testing/selftests/kselftest/module.h"
  179. KSTM_MODULE_GLOBALS();
  180. /*
  181. * Kernel module for testing the foobinator
  182. */
  183. static int __init test_function()
  184. {
  185. ...
  186. }
  187. static void __init selftest(void)
  188. {
  189. KSTM_CHECK_ZERO(do_test_case("", 0));
  190. }
  191. KSTM_MODULE_LOADERS(test_foo);
  192. MODULE_AUTHOR("John Developer <jd@fooman.org>");
  193. MODULE_LICENSE("GPL");
  194. Example test script
  195. -------------------
  196. .. code-block:: sh
  197. #!/bin/bash
  198. # SPDX-License-Identifier: GPL-2.0+
  199. $(dirname $0)/../kselftest/module.sh "foo" test_foo
  200. Test Harness
  201. ============
  202. The kselftest_harness.h file contains useful helpers to build tests. The
  203. test harness is for userspace testing, for kernel space testing see `Test
  204. Module`_ above.
  205. The tests from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as
  206. example.
  207. Example
  208. -------
  209. .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
  210. :doc: example
  211. Helpers
  212. -------
  213. .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
  214. :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP
  215. FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN FIXTURE_VARIANT
  216. FIXTURE_VARIANT_ADD
  217. Operators
  218. ---------
  219. .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
  220. :doc: operators
  221. .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
  222. :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE
  223. ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE
  224. ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT
  225. EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE
  226. EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE