Benchmarks fail with TLS certificate error
Since November 14 2017 benchmarks are failing with a certificate error when instantiating the ElasticSearch python module inside pytest-benchmarks.
From: https://0xacab.org/leap/soledad/-/jobs/26786
00:01:44 benchmark-memory runtests: commands[1] | py.test --subdir=benchmarks --watch-memory --couch-url http://couchdb:5984 --benchmark-json=/tmp/tmp.Q05Ni6iQeP
00:01:46 INTERNALERROR> Traceback (most recent call last):
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/_pytest/main.py", line 107, in wrap_session
00:01:46 INTERNALERROR> config._do_configure()
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/_pytest/config.py", line 935, in _do_configure
00:01:46 INTERNALERROR> self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 750, in call_historic
00:01:46 INTERNALERROR> self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
00:01:46 INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
00:01:46 INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
00:01:46 INTERNALERROR> res = hook_impl.function(*args)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/pytest_benchmark/plugin.py", line 427, in pytest_configure
00:01:46 INTERNALERROR> bs = config._benchmarksession = BenchmarkSession(config)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/pytest_benchmark/session.py", line 40, in __init__
00:01:46 INTERNALERROR> netrc=config.getoption("benchmark_netrc")
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/pytest_benchmark/utils.py", line 454, in load_storage
00:01:46 INTERNALERROR> return ElasticsearchStorage(*args, **kwargs)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/pytest_benchmark/storage/elasticsearch.py", line 47, in __init__
00:01:46 INTERNALERROR> self._es = elasticsearch.Elasticsearch(self._es_hosts, serializer=BenchmarkJSONSerializer())
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 170, in __init__
00:01:46 INTERNALERROR> self.transport = transport_class(_normalize_hosts(hosts), **kwargs)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 108, in __init__
00:01:46 INTERNALERROR> self.set_connections(hosts)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 155, in set_connections
00:01:46 INTERNALERROR> connections = map(_create_connection, hosts)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 154, in _create_connection
00:01:46 INTERNALERROR> return self.connection_class(**kwargs)
00:01:46 INTERNALERROR> File "/builds/leap/soledad/.tox/benchmark-memory/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 91, in __init__
00:01:46 INTERNALERROR> raise ImproperlyConfigured("Root certificates are missing for certificate "
00:01:46 INTERNALERROR> ImproperlyConfigured: Root certificates are missing for certificate validation. Either pass them in using the ca_certs parameter or install certifi to use it automatically.
00:01:46 ERROR: InvocationError: '/builds/leap/soledad/.tox/benchmark-memory/bin/py.test --subdir=benchmarks --watch-memory --couch-url http://couchdb:5984 --benchmark-json=/tmp/tmp.Q05Ni6iQeP'
00:01:46 ___________________________________ summary ____________________________________
00:01:46 ERROR: benchmark-memory: commands failed
+ echo benchmark-memory
+ grep ^benchmark-
+ [ -z benchmark-memory ]
+ dirname scripts/benchmark/run-benchmarks-ci-job.sh
+ basedir=scripts/benchmark
+ scripts/benchmark/check-for-outliers.py --status-code 0 /tmp/tmp.Q05Ni6iQeP
Traceback (most recent call last):
File "scripts/benchmark/check-for-outliers.py", line 166, in <module>
tests = parse_file(args.file)
File "scripts/benchmark/check-for-outliers.py", line 42, in parse_file
data = json.loads(f.read())
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Edited by drebs