diff --git a/scripts/benchmark/README.md b/scripts/benchmark/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0910219a9211843a4855c3af11f67cfa41a12914
--- /dev/null
+++ b/scripts/benchmark/README.md
@@ -0,0 +1,31 @@
+Benchmark all commits
+=====================
+
+The script in this directory is used to run the benchmarking tests for all
+commits in the repository.
+
+It is actually not possible to run all benchmarking tests for all commits
+because:
+
+  - at some points in the repo history, the tested code didn't exist.
+
+  - at some poitns in the repo history, the teste code might have been
+    refactored and the api would not match the one used in tests.
+
+So what this script does is the following:
+
+  - check out .gitlab-ci.yml from current master.
+  - use gitlab-runner to exec a build locally.
+  - store timestamp and exit status of each run.
+  - checkout previous commit and iterate.
+
+How to use the script
+---------------------
+
+Once you have gitlab-runner setup in the machine and the elasticsearch
+credentials, use the following commands to run the script:
+
+    git clone https://0xacab.org/leap/soledad
+    cd soledad/
+    export PYTEST_OPTS="--benchmark-autosave --benchmark-storage=\"elasticsearch+https://USER:PASSWORD@moose.leap.se:9200\""
+    scripts/benchmark/benchmark-all-commits.sh