# Introduction Welcome to the NodeMCU self-test suite. Here you will find our growing effort to ensure that our software behaves as we think it should and that we do not regress against earlier versions. Our tests are written using [NTest](./NTest/NTest.md), a lightweight yet featureful framework for specifying unit tests. # Building and Running Test Software on NodeMCU Devices Naturally, to test NodeMCU on its intended hardware, you will need one or more NodeMCU-capable boards. At present, the test environment is specified using two ESP8266 Devices Under Test (DUTs), but we envision expanding this to mixed ESP8266/ESP32 environments as well. Test programs live beside this file. While many test programs run on the NodeMCU DUTs, but there is reason to want to orchestrate DUTs and the environment using the host. Files matching the glob `NTest_*.lua` are intended for on-DUT execution. ## Manual Test Invocation At the moment, the testing regime and host-based orchestration is still in development, and so things are a little more manual than perhaps desired. The `NTest`-based test programs all assume that they can `require "NTest"`, and so the easiest route to success is to * build an LFS image containing * [package.loader support for LFS](../lua_examples/lfs/_init.lua) * [NTest itself](./NTest/NTest.lua) * Any additional Lua support modules required (e.g., [mcp23017 support](../lua_modules/mcp23017/mcp23017.lua) ) * build a firmware with the appropriate C modules * program the board with your firmware and LFS images * ensure that `package.loader` is patched appropriately on startup * transfer the `NTest_foo` program you wish to run to the device SPIFFS (or have included it in the LFS). * at the interpreter prompt, say `dofile("NTest_foo.lua")` (or `node.LFS.get("NTest_foo")()`) to run the `foo` test program. ## Experimental Host Orchestration Enthusiastic testers are encouraged to try using our very new, very experimental host test runner, [tap-driver.expect](./tap-driver.expect). To use this program, in addition to the above, the LFS environment should contain [NTestTapOut](./tests/utils/NTestTapOut.lua), an output adapter for `NTest`, making it speak a slight variant of the [Test Anything Protocol](https://testanything.org/). This structured output is scanned for by the script on the host. You'll need `expect` and TCL and some TCL libraries available; on Debian, that amounts to apt install tcl tcllib tclx8.4 expect This program should be invoked from beside this file with something like TCLLIBPATH=./expectnmcu ./tap-driver.expect -serial /dev/ttyUSB3 -lfs ./lfs.img NTest_file.lua This will... * transfer and install the specified LFS module (and reboot the device to load LFS) * transfer the test program * run the test program with `NTest` shimmed to use the `NTestTapOut` output handler * summarize the results * return 0 if and only if all tests have passed This tool is quite flexible and takes a number of other options and flags controlling aspects of its behavior: * Additional files, Lua or otherwise, may be transferred by specifing them before the test to run (e.g., `./tap-driver.expect a.lua b.lua NTest_foo.lua`); dually, a `-noxfer` flag will suppress transferring even the last file. All transferred files are moved byte-for-byte to the DUT's SPIFFS with names, but not directory components, preserved. * The `-lfs LFS.img` option need not be specified and, if not given, any existing `LFS` image will remain on the device for use by the test. * A `-nontestshim` flag will skip attempting to shim the given test program with `NTestTapOut`; the test program is expected to provide its own TAP output. The `-tpfx` argument can be used to override the leading `TAP: ` sigil used by the `NTestTapOut` output handler. * A `-runfunc` option indicates that the last argument is not a file to transfer but rather a function to be run. It will be invoked at the REPL with a single argument, the shimmed `NTest` constructor, unless `-nontestshim` is given, in which case the argument will be `nil`. * A `-notests` option suppresses running tests (making the tool merely another option for loading files to the device). Transfers will be significantly faster if [pipeutils](../lua_examples/pipeutils.lua) is available to `require` on the DUT, but a fallback strategy exists if not. We suggest either including `pipeutils` in LFS images, in SPIFFS, or as the first file to be transferred. # NodeMCU Testing Environment Herein we define the environment our testing framework expects to see when it runs. It is composed of two ESP8266 devices, each capable of holding an entire NodeMCU firmware, LFS image, and SPIFFS file system, as well as additional peripheral hardware. It is designed to fit comfortably on a breadboard and so should be easily replicated and integrated into any firmware validation testing. The test harness runs from a dedicated host computer, which is expected to have reset- and programming-capable UART links to both ESP8266 devices, as found on almost all ESP8266 boards with USB to UART adapters, but the host does not necessarily need to use USB to connect, so long as TXD, RXD, DTR, and RTS are wired across. A particular implementation of this can be found at [Test Harness](HardwareTestHarness.md). ## Peripherals ### I2C Bus There is an I2C bus hanging off DUT 0. Attached hardware is used both as tests of modules directly and also to facilitate testing other modules (e.g., gpio). #### MCP23017: I/O Expander At address 0x20. An 16-bit tristate GPIO expander, this chip is used to test I2C, GPIO, and ADC functionality. This chip's interconnections are as follows: MPC23017 | Purpose ---------|-------------------------------------------------------------- /RESET |DUT0 reset. This resets the chip whenever the host computer resets DUT 0 over its serial link (using DTR/RTS). B 0 |4K7 resistor to DUT 0 ADC. B 1 |2K2 resistor to DUT 0 ADC. B 5 |DUT1 GPIO16/WAKE via 4K7 resitor B 6 |DUT0 GPIO13 via 4K7 resistor and DUT1 GPIO15 via 4K7 resistor B 7 |DUT0 GPIO15 via 4K7 resistor and DUT1 GPIO13 via 4K7 resistor Notes: - DUT 0's ADC pin is connected via a 2K2 reistor to this chip's port B, pin 1 and via a 4K7 resistor to port B, pin 0. This gives us the ability to produce approximately 0 (both pins low), 1.1 (pin 0 high, pin 1 low), 2.2 (pin 1 high, pin 0 low), and 3.3V (both pins high) on the ADC pin. - Port B pins 6 and 7 sit on the UART cross-wiring between DUT 0 and DUT 1. The 23017 will be tristated for inter-DUT UART tests, but these - Port B pins 2, 3, and 4, as well as all of port A, remain available for expansion. - The interrupt pins are not yet routed, but could be. We reserve DUT 0 GPIO 2 for this purpose with the understanding that the 23017's interrupt functionality will be disabled (INTA, INTB set to open-drain, GPINTEN set to 0) when not explicitly under test. ESP8266 Device 0 Connections ---------------------------- ESP | Usage ----------|---------------------------------------------------------- GPIO 0 |Used to enter programming mode; otherwise unused in test environment. GPIO 1 |Primary UART transmit; reserved for host communication GPIO 2 |[reserved for 1-Wire] [+ reserved for 23017 INT[AB] connections] GPIO 3 |Primary UART recieve; reserved for host communication GPIO 4 |I2C SDA GPIO 5 |I2C SCL GPIO 6 |[Reserved for on-chip flash] GPIO 7 |[Reserved for on-chip flash] GPIO 8 |[Reserved for on-chip flash] GPIO 9 |[Reserved for on-chip flash] GPIO 10 |[Reserved for on-chip flash] GPIO 11 |[Reserved for on-chip flash] GPIO 12 | GPIO 13 |Secondary UART RX; DUT 1 GPIO 15, I/O expander B 6 GPIO 14 | GPIO 15 |Secondary UART TX; DUT 1 GPIO 13, I/O expander B 7 GPIO 16 | ADC 0 |Resistor divider with I/O expander ESP8266 Device 1 Connections ---------------------------- ESP | Usage ----------|---------------------------------------------------------- GPIO 0 |Used to enter programming mode; otherwise unused in test environment. GPIO 1 |Primary UART transmit; reserved for host communication GPIO 2 |[Reserved for WS2812] GPIO 3 |Primary UART recieve; reserved for host communication GPIO 4 | GPIO 5 | GPIO 6 |[Reserved for on-chip flash] GPIO 7 |[Reserved for on-chip flash] GPIO 8 |[Reserved for on-chip flash] GPIO 9 |[Reserved for on-chip flash] GPIO 10 |[Reserved for on-chip flash] GPIO 11 |[Reserved for on-chip flash] GPIO 12 |HSPI MISO GPIO 13 |Secondary UART RX; DUT 0 GPIO 15, I/O exp B 7 via 4K7 Also used as HSPI MOSI for SPI tests GPIO 14 |HSPI CLK GPIO 15 |Secondary UART TX; DUT 0 GPIO 13, I/O exp B 6 via 4K7 Also used as HSPI /CS for SPI tests GPIO 16 |I/O expander B 5 via 4K7 resistor, for deep-sleep tests ADC 0 |