Graph soledad benchmarks with Kibana
Current WIP Dashboard Documentation
Todo
-
Install elasticsearch, logstash and kibana on moose -
Enable elasticsearch auth so it can get reached from outside -
Manually puting real metrics into elasticsearch -
Fix commit_info.project
andcommit_info.branch
-
Include missing commit_info.id
-
Add commit_info.datetime
to pytest-benchmark output -
Fix couchdb access in docker benchmark tests -
Make sure we dont leak elasticsearch credentials from the CI -
Merge soledad pull req to benchmark and push to elasticsearch: soledad!74 (merged) (@drebs) -
Integrate pushing the metrics from CI builds
-
-
Puppetize -
/etc/elasticsearch/moose/elasticsearch.yml
-
/etc/kibana/kibana.yml
-
-
Set concurrent=1
inquelea:/etc/gitlab-runner/config.toml
-
Add hostname to json output of benchmarks -
Make sure commit_info.branch contains the correct branch, because gitlab-ci checks out a detatched HEAD by default -
Database locks in benchmark tests on quelea docker runner -
Finish one graph with all metrics for one test we want to show (min, max, mean, stdeev etc.) -
Use the above template for creating graphs for each (?) test -
Use upstream pytest-benchmark in tox.ini once all pending pull reqs are merged: -
Add commit_info.author_time: https://github.com/ionelmc/pytest-benchmark/pull/75 -
Fix update hooks specs: https://github.com/ionelmc/pytest-benchmark/pull/74
-
-
Two dashboards: one for historic benchmarks (run on weasel/quelea) and one for every new commit on beluga -
Decrease standard deviation by a) increasing the benchmark rounds for each test or/and b) re-running all tests multiple times again -
improve the stdev plotting: mean +/- 3stdev -
Test SSDs+HDs on quelea -
Improve kibana installation -
Use auth_key_sha256 for passwords in /etc/elasticsearch/moose/elasticsearch.yml
-
Secure /etc/kibana/kibana.yml
-
Puppetize Installation of elasticsearch-readonlyrest-plugin
-
-
Integrate graphs into gitlab
Gather questions for elastic call
-
Add summary to each graph that explains what's measured -
Add time unit to y-axis -
Group commits per week ? -
Saved search to refine an existing search