telemetry.md 2.9 KB

Running Telemetry Benchmarks on Fuchsia

[TOC]

General instruction on running and debugging benchmarks can be found in the tools/perf/README.md.

Fuchsia uses web_engine_shell to run integration tests. Be sure to build any components you wish to deploy on your device, along with web_engine_shell.

Supported benchmarks

The list of supported benchmarks can be found in the bot_platforms.py.

The list of stories that are skipped can be found in expectations.config, with the fuchsia tag.

Run on a local device


NOTE This has not been tested as these instructions were developed while remote work was the only available option.


After building chromium, be sure to have a paved device that has gone through OOBEattached to the host. Run the following from the chromium build directory if you want to run a specific benchmark:

$ ../../content/test/gpu/run_telemetry_benchmark_fuchsia.py \
--browser=web-engine-shell  --output-format=histograms  \
--experimental-tbmv3-metrics -d --host=localhost \
[--fuchsia-out-dir=/path/to/fuchsia/out/dir] \  # Deploy custom fuchsia.
[benchmark] [--story-filter=<story name>]

For instance, to run the simplest story, load:chrome:blank from the benchmark system_health.memory_desktop:

$ ../../content/test/gpu/run_telemetry_benchmark_fuchsia.py \
--browser=web-engine-shell  --output-format=histograms  \
--experimental-tbmv3-metrics -d --host=localhost system_health.memory_desktop \
--story-filter=load:chrome:blank

If no benchmark or filter is specified, all supported benchmarks will run.

Run on an emulator

If you wish to run the tests on an emulator, simply drop the -d flag. This will start an emulator and run through the supported tests, like so.

$ ../../content/test/gpu/run_telemetry_benchmark_fuchsia.py \
--browser=web-engine-shell  --output-format=histograms  \
--experimental-tbmv3-metrics system_health.memory_desktop \
--story-filter=load:chrome:blank

Note that this needs to be run from an x64 build directory.

Run on a remote device

As connecting to a device that is not connected to a workstation is more common, this flow is what is recommended and tested.

Be sure to first open a tunnel to your device and have a pm server running and serving to your device:

$ ../../content/test/gpu/run_telemetry_benchmark_fuchsia.py \
--browser=web-engine-shell  --output-format=histograms  \
--experimental-tbmv3-metrics -d --host=localhost \
--ssh-config=<sshconfig file>  [benchmark] [--story-filter=<story name>]

The SSH config path is typically $HOME/.fuchsia/sshconfig.

See the above section on how to use the benchmark and --story-filter.