Old commits benchmark tests get killed randomly
I wanted to graph some older commit's benchmark metrics, but the tests get magically killed randomly:
--- git/soledad ‹master* ⁇› » git co d92f1d1840d28ee6c2058556b047db75ed737f77
Note: checking out 'd92f1d1840d28ee6c2058556b047db75ed737f77'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at d92f1d1... [bug] fix shared database initialization
--- git/soledad » cd testing
--- soledad/testing » tox -e benchmark
benchmark develop-inst-nodeps: /home/varac/dev/projects/leap/git/soledad/testing
benchmark installed: appdirs==1.4.3,asn1crypto==0.22.0,attrs==16.3.0,Automat==0.5.0,Beaker==1.8.1,cffi==1.10.0,configparser==3.5.0,constantly==15.1.0,CouchDB==1.1,coverage==4.3.4,cryptography==1.8.1,decorator==4.0.11,docutils==0.13.1,enum34==1.1.6,extras==1.0.0,fancycompleter==0.7,fixtures==3.0.0,funcsigs==1.0.2,functools32==3.2.3.post2,greenlet==0.4.12,idna==2.5,incremental==16.10.1,ipaddress==1.0.18,jsonschema==2.6.0,leap.common==0.5.4,-e git+gitolite@leap.se:/soledad.git@d92f1d1840d28ee6c2058556b047db75ed737f77#egg=leap.soledad.client&subdirectory=client,-e git+gitolite@leap.se:/soledad.git@d92f1d1840d28ee6c2058556b047db75ed737f77#egg=leap.soledad.common&subdirectory=common,-e git+gitolite@leap.se:/soledad.git@d92f1d1840d28ee6c2058556b047db75ed737f77#egg=leap.soledad.server&subdirectory=server,linecache2==1.0.0,mock==2.0.0,ordereddict==1.1,packaging==16.8,Paste==2.0.3,pathlib2==2.2.1,pbr==2.0.0,pdbpp==0.8.3,pkg-resources==0.0.0,py==1.4.33,pyasn1==0.2.3,pyasn1-modules==0.0.8,pycparser==2.17,Pygments==2.2.0,pyOpenSSL==16.2.0,pyparsing==2.2.0,pyrepl==0.8.4,pysqlcipher==2.6.9,pytest==3.0.7,pytest-benchmark==3.0.0,pytest-cov==2.4.0,pytest-twisted==1.5,python-cjson==1.2.1,python-dateutil==2.6.0,python-mimeparse==1.6.0,pyzmq==16.0.2,repoze.lru==0.6,requests==2.13.0,Routes==2.4.1,scandir==1.5,scrypt==0.8.0,service-identity==16.0.0,setuptools-trial==0.6.0,six==1.10.0,statistics==1.0.3.5,-e git+gitolite@leap.se:/soledad.git@d92f1d1840d28ee6c2058556b047db75ed737f77#egg=test_soledad&subdirectory=testing,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,Twisted==17.1.0,txZMQ==0.8.0,unittest2==1.1.0,wmctrl==0.3,zope.interface==4.3.3,zope.proxy==4.2.0
benchmark runtests: PYTHONHASHSEED='3764654556'
benchmark runtests: commands[0] | py.test --benchmark-only
=============================================================================================== test session starts ===============================================================================================
platform linux2 -- Python 2.7.12+, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
benchmark: 3.0.0 (defaults: timer=time.time disable_gc=False min_rounds=5 min_time=5.00us max_time=1.00s calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/varac/dev/projects/leap/git/soledad/testing, inifile: pytest.ini
plugins: benchmark-3.0.0, twisted-1.5, cov-2.4.0
collected 673 items
tests/benchmarks/test_crypto.py FFFFFFFFFF..........[1] 21175 killed tox -e benchmark
It's not super important although it would be good to have metrics from the past. Does anybody know why this happens ?