Skip to content

8860 - Run benchmarks twice

drebs requested to merge drebs/soledad:8860 into master

This MR runs benchmarks twice: once for time results and a second time for resource results.

It does this by adding a command-line option for pytest called --watch-resources, and running tox twice in the benchmarking stage, once without and a second time with the command line option set.

As a side fix, it also:

  • moves the benchmark-all-commits env setup to its own script and executes it based on the presence of the BENCHMARK_ALL_COMMITS environment variable.
  • avoids reinstalling pysqlcipher if unneeded (good for local dev and run of benchmarks where the environment is already setup).

Merge request reports