Changes

Page history
Update benchmarks authored by drebs's avatar drebs
Show whitespace changes
Inline Side-by-side
benchmarks.md
View page @ 3bc5ff3f
......@@ -24,38 +24,40 @@ These are the configuration options that tests that are automatically calibrated
The following table was taken from a run of benchmarks by the CI on top of (effectivelly) the current master branch: https://0xacab.org/drebs/soledad/-/jobs/13422
| **module** | **name** | **mean time (ms)** | **rounds** | **function** |
| ---------------- | ---------------------------- | -----------------: | ---------: | ------------------------ |
| `test_resources` | `test_memory_intensive` | 3843.1 | 5 | `monitored_benchmark` |
| `test_resources` | `test_cpu_intensive` | 5915.6 | 5 | `monitored_benchmark` |
| `test_crypto` | `test_decrypt_doc_10k` | 0.798 | 492 | `txbenchmark` |
| `test_crypto` | `test_decrypt_doc_100k` | 2.719 | 229 | `txbenchmark` |
| `test_crypto` | `test_decrypt_doc_500k` | 9.758 | 55 | `txbenchmark` |
| `test_crypto` | `test_decrypt_doc_1M` | 19.468 | 31 | `txbenchmark` |
| `test_crypto` | `test_decrypt_doc_10M` | 192.162 | 5 | `txbenchmark` |
| `test_crypto` | `test_encrypt_doc_10k` | 0.803 | 501 | `txbenchmark` |
| `test_crypto` | `test_encrypt_doc_100k` | 2.040 | 416 | `txbenchmark` |
| `test_crypto` | `test_encrypt_doc_500k` | 7.852 | 127 | `txbenchmark` |
| `test_crypto` | `test_encrypt_doc_1M` | 14.751 | 63 | `txbenchmark` |
| `test_crypto` | `test_encrypt_doc_10M` | 241.375 | 5 | `txbenchmark` |
| `test_crypto` | `test_decrypt_raw_10k` | 0.171 | 1695 | `monitored_benchmark` |
| `test_crypto` | `test_decrypt_raw_100k` | 0.912 | 961 | `monitored_benchmark` |
| `test_crypto` | `test_decrypt_raw_500k` | 3.937 | 249 | `monitored_benchmark` |
| `test_crypto` | `test_decrypt_raw_1M` | 7.833 | 136 | `monitored_benchmark` |
| `test_crypto` | `test_decrypt_raw_10M` | 80.620 | 10 | `monitored_benchmark` |
| `test_crypto` | `test_encrypt_raw_10k` | 0.185 | 916 | `monitored_benchmark` |
| `test_crypto` | `test_encrypt_raw_100k` | 0.860 | 914 | `monitored_benchmark` |
| `test_crypto` | `test_encrypt_raw_500k` | 3.946 | 254 | `monitored_benchmark` |
| `test_crypto` | `test_encrypt_raw_1M` | 7.748 | 125 | `monitored_benchmark` |
| `test_crypto` | `test_encrypt_raw_10M` | 82.109 | 12 | `monitored_benchmark` |
| `test_sync` | `test_download_20_500k` | 4426.7 | 4 | `txbenchmark_with_setup` |
| `test_sync` | `test_download_100_100k` | 4629.2 | 4 | `txbenchmark_with_setup` |
| `test_sync` | `test_download_1000_10k` | 10.3160 | 4 | `txbenchmark_with_setup` |
| `test_sync` | `test_upload_20_500k` | 4167.3 | 4 | `txbenchmark_with_setup` |
| `test_sync` | `test_upload_100_100k` | 4519.6 | 4 | `txbenchmark_with_setup` |
| `test_sync` | `test_upload_1000_10k` | 6831.7 | 4 | `txbenchmark_with_setup` |
| `test_sync` | `test_nothing_to_sync` | 195.7985 | 4 | `txbenchmark_with_setup` |
| `test_misc` | `test_initialization` | 233.5269 | 5 | `monitored_benchmark` |
| `test_sqlcipher` | `test_async_create_100_100k` | 1867.1 | 5 | `txbenchmark` |
| `test_sqlcipher` | `test_async_create_20_500k` | 2343.2 | 5 | `txbenchmark` |
| `test_sqlcipher` | `test_async_create_1000_10k` | 4008.3 | 5 | `txbenchmark` |
\ No newline at end of file
The last two columns are used to estimate what would be a maximum sampling interval that makes sense for each function. The **total time** is estimated from the mean time times the number of rounds, and the *maximum sampling interval* is the maximum number that would provide good memory statistics for each test.
| **module** | **name** | **mean time (ms)** | **rounds** | **function** | **~total time (s)** | **maximum sampling interval (s)** |
| ---------------- | ---------------------------- | -----------------: | ---------: | ------------------------ | --------------------: | --------------------: |
| `test_resources` | `test_memory_intensive` | 3843.1 | 5 | `monitored_benchmark` | 2 | 0.1 |
| `test_resources` | `test_cpu_intensive` | 5915.6 | 5 | `monitored_benchmark` | 3 | 0.1 |
| `test_crypto` | `test_decrypt_doc_10k` | 0.798 | 492 | `txbenchmark` | 0.5 | 0.1 |
| `test_crypto` | `test_decrypt_doc_100k` | 2.719 | 229 | `txbenchmark` | 0.5 | 0.1 |
| `test_crypto` | `test_decrypt_doc_500k` | 9.758 | 55 | `txbenchmark` | 0.5 | 0.1 |
| `test_crypto` | `test_decrypt_doc_1M` | 19.468 | 31 | `txbenchmark` | 0.6 | 0.1 |
| `test_crypto` | `test_decrypt_doc_10M` | 192.162 | 5 | `txbenchmark` | 1.2 | 0.1 |
| `test_crypto` | `test_encrypt_doc_10k` | 0.803 | 501 | `txbenchmark` | 0.4 | 0.1 |
| `test_crypto` | `test_encrypt_doc_100k` | 2.040 | 416 | `txbenchmark` | 0.8 | 0.1 |
| `test_crypto` | `test_encrypt_doc_500k` | 7.852 | 127 | `txbenchmark` | 1 | 0.1 |
| `test_crypto` | `test_encrypt_doc_1M` | 14.751 | 63 | `txbenchmark` | 0.9 | 0.1 |
| `test_crypto` | `test_encrypt_doc_10M` | 241.375 | 5 | `txbenchmark` | 1.2 | 0.1 |
| `test_crypto` | `test_decrypt_raw_10k` | 0.171 | 1695 | `monitored_benchmark` | 0.3 | 0.1 |
| `test_crypto` | `test_decrypt_raw_100k` | 0.912 | 961 | `monitored_benchmark` | 1 | 0.1 |
| `test_crypto` | `test_decrypt_raw_500k` | 3.937 | 249 | `monitored_benchmark` | 1 | 0.1 |
| `test_crypto` | `test_decrypt_raw_1M` | 7.833 | 136 | `monitored_benchmark` | 1 | 0.1 |
| `test_crypto` | `test_decrypt_raw_10M` | 80.620 | 10 | `monitored_benchmark` | 0.8 | 0.1 |
| `test_crypto` | `test_encrypt_raw_10k` | 0.185 | 916 | `monitored_benchmark` | 0.2 | 0.1 |
| `test_crypto` | `test_encrypt_raw_100k` | 0.860 | 914 | `monitored_benchmark` | 0.9 | 0.1 |
| `test_crypto` | `test_encrypt_raw_500k` | 3.946 | 254 | `monitored_benchmark` | 1 | 0.1 |
| `test_crypto` | `test_encrypt_raw_1M` | 7.748 | 125 | `monitored_benchmark` | 1 | 0.1 |
| `test_crypto` | `test_encrypt_raw_10M` | 82.109 | 12 | `monitored_benchmark` | 0.9 | 0.1 |
| `test_sync` | `test_download_20_500k` | 4426.7 | 4 | `txbenchmark_with_setup` | 18 | 1 |
| `test_sync` | `test_download_100_100k` | 4629.2 | 4 | `txbenchmark_with_setup` | 19 | 1 |
| `test_sync` | `test_download_1000_10k` | 10316.0 | 4 | `txbenchmark_with_setup` | 40 | 1 |
| `test_sync` | `test_upload_20_500k` | 4167.3 | 4 | `txbenchmark_with_setup` | 17 | 1 |
| `test_sync` | `test_upload_100_100k` | 4519.6 | 4 | `txbenchmark_with_setup` | 18 | 1 |
| `test_sync` | `test_upload_1000_10k` | 6831.7 | 4 | `txbenchmark_with_setup` | 28 | 1 |
| `test_sync` | `test_nothing_to_sync` | 195.7985 | 4 | `txbenchmark_with_setup` | 0.8 | 0.1 |
| `test_misc` | `test_initialization` | 233.5269 | 5 | `monitored_benchmark` | 1.2 | 0.1 |
| `test_sqlcipher` | `test_async_create_100_100k` | 1867.1 | 5 | `txbenchmark` | 10 | 1 |
| `test_sqlcipher` | `test_async_create_20_500k` | 2343.2 | 5 | `txbenchmark` | 12 | 1 |
| `test_sqlcipher` | `test_async_create_1000_10k` | 4008.3 | 5 | `txbenchmark` | 20 | 1 |
\ No newline at end of file