123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209 |
- .. SPDX-License-Identifier: GPL-2.0+
- Sandbox tests
- =============
- Test Design
- -----------
- Most uclasses and many functions of U-Boot have sandbox tests. This allows much
- of the code to be checked in an developer-friendly environment.
- Sandbox provides a way to write and run unit tests. The traditional approach to
- unit tests is to build lots of little executables, one for each test or
- category of tests. With sandbox, so far as possible, all the tests share a
- small number of executables (e.g. 'u-boot' for sandbox, 'u-boot-spl' and
- 'u-boot' for sandbox_spl) and can be run very quickly. The vast majority of
- tests can run on the 'sandbox' build,
- Available tests
- ---------------
- Some of the available tests are:
- - command_ut: Unit tests for command parsing and handling
- - compression: Unit tests for U-Boot's compression algorithms, useful for
- security checking. It supports gzip, bzip2, lzma and lzo.
- - image: Unit tests for images:
- - test/image/test-imagetools.sh - multi-file images
- - test/py/tests/test-fit.py - FIT images
- - tracing: test/trace/test-trace.sh tests the tracing system (see
- README.trace)
- - verified boot: test/py/tests/test_vboot.py
- If you change or enhance any U-Boot subsystem, you should write or expand a
- test and include it with your patch series submission. Test coverage in some
- older areas of U-Boot is still somewhat limited and we need to work to improve
- it.
- Note that many of these tests are implemented as commands which you can
- run natively on your board if desired (and enabled).
- To run all tests, use 'make check'.
- Running sandbox tests directly
- ------------------------------
- Typically tests are run using the pytest suite. Running pytests on sandbox is
- easy and always gets things right. For example some tests require files to be
- set up before they can work.
- But it is also possible to run some sandbox tests directly. For example, this
- runs the dm_test_gpio() test which you can find in test/dm/gpio.c::
- $ ./u-boot -T -c "ut dm gpio"
- U-Boot 2021.01
- Model: sandbox
- DRAM: 128 MiB
- WDT: Started with servicing (60s timeout)
- MMC: mmc2: 2 (SD), mmc1: 1 (SD), mmc0: 0 (SD)
- In: serial
- Out: vidconsole
- Err: vidconsole
- Model: sandbox
- SCSI:
- Net: eth0: eth@10002000, eth5: eth@10003000, eth3: sbe5, eth6: eth@10004000
- Test: dm_test_gpio: gpio.c
- Test: dm_test_gpio: gpio.c (flat tree)
- Failures: 0
- The -T option tells the U-Boot sandbox to run with the 'test' devicetree
- (test.dts) instead of -D which selects the normal sandbox.dts - this is
- necessary because many tests rely on nodes or properties in the test devicetree.
- If you try running tests without -T then you may see failures, like::
- $ ./u-boot -c "ut dm gpio"
- U-Boot 2021.01
- DRAM: 128 MiB
- WDT: Not found!
- MMC:
- In: serial
- Out: serial
- Err: serial
- SCSI:
- Net: No ethernet found.
- Please run with test device tree:
- ./u-boot -d arch/sandbox/dts/test.dtb
- Test: dm_test_gpio: gpio.c
- test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22)
- Test: dm_test_gpio: gpio.c (flat tree)
- test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22)
- Failures: 2
- The message above should provide a hint if you forget to use the -T flag. Even
- running with -D will produce different results.
- You can easily use gdb on these tests, without needing --gdbserver::
- $ gdb u-boot --args -T -c "ut dm gpio"
- ...
- (gdb) break dm_test_gpio
- Breakpoint 1 at 0x1415bd: file test/dm/gpio.c, line 37.
- (gdb) run -T -c "ut dm gpio"
- Starting program: u-boot -T -c "ut dm gpio"
- Test: dm_test_gpio: gpio.c
- Breakpoint 1, dm_test_gpio (uts=0x5555558029a0 <global_dm_test_state>)
- at files/test/dm/gpio.c:37
- 37 ut_assertok(gpio_lookup_name("b4", &dev, &offset, &gpio));
- (gdb)
- You can then single-step and look at variables as needed.
- Running sandbox_spl tests directly
- ----------------------------------
- SPL is the phase before U-Boot proper. It is present in the sandbox_spl build,
- so you can run SPL like this::
- ./spl/u-boot-spl
- SPL tests are special in that they run (only in the SPL phase, of course) if the
- -u flag is given::
- ./spl/u-boot-spl -u
- U-Boot SPL 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700)
- Running 5 driver model tests
- Test: dm_test_of_plat_base: of_platdata.c (flat tree)
- Test: dm_test_of_plat_dev: of_platdata.c (flat tree)
- Test: dm_test_of_plat_parent: of_platdata.c (flat tree)
- Test: dm_test_of_plat_phandle: of_platdata.c (flat tree)
- Test: dm_test_of_plat_props: of_platdata.c (flat tree)
- Failures: 0
- U-Boot 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700)
- DRAM: 128 MiB
- ...
- It is not possible to run SPL tests in U-Boot proper, firstly because they are
- not built into U-Boot proper and secondly because the environment is very
- different, e.g. some SPL tests rely on of-platdata which is only available in
- SPL.
- Note that after running, SPL continues to boot into U-Boot proper. You can add
- '-c exit' to make U-Boot quit without doing anything further. It is not
- currently possible to run SPL tests and then stop, since the pytests require
- that U-Boot produces the expected banner.
- You can use the -k flag to select which tests run::
- ./spl/u-boot-spl -u -k dm_test_of_plat_parent
- Of course you can use gdb with sandbox_spl, just as with sandbox.
- Running all tests directly
- --------------------------
- A fast way to run all sandbox tests is::
- ./u-boot -T -c "ut all"
- It typically runs single-thread in 6 seconds on 2021 hardware, with 2s of that
- to the delays in the time test.
- This should not be considered a substitute for 'make check', but can be helpful
- for git bisect, etc.
- What tests are built in?
- ------------------------
- Whatever sandbox build is used, which tests are present is determined by which
- source files are built. For sandbox_spl, the of_platdata tests are built
- because of the build rule in test/dm/Makefile::
- ifeq ($(CONFIG_SPL_BUILD),y)
- obj-$(CONFIG_SPL_OF_PLATDATA) += of_platdata.o
- else
- ...other tests for non-spl
- endif
- You can get a list of tests in a U-Boot ELF file by looking for the
- linker_list::
- $ nm /tmp/b/sandbox_spl/spl/u-boot-spl |grep 2_dm_test
- 000000000001f200 D _u_boot_list_2_dm_test_2_dm_test_of_plat_base
- 000000000001f220 D _u_boot_list_2_dm_test_2_dm_test_of_plat_dev
- 000000000001f240 D _u_boot_list_2_dm_test_2_dm_test_of_plat_parent
- 000000000001f260 D _u_boot_list_2_dm_test_2_dm_test_of_plat_phandle
- 000000000001f280 D _u_boot_list_2_dm_test_2_dm_test_of_plat_props
- Writing tests
- -------------
- See :doc:`tests_writing` for how to write new tests.
|