leap test ordering: check stunnel/couchdb first
We often run into the situation where (on jessie nodes) stunnel runs out of memory and doesn't restart by itself.
This will result in a broken provider.
leap test unstable
will show this:
~/leap/git/bitmask leap test unstable
= [tuatara] PASS: Network > Can connect to internet?
= [tuatara] PASS: Network > Is stunnel running?
= [tuatara] PASS: Network > Is shorewall running?
= [tuatara] PASS: Network > Are server certificates valid?
= [tuatara] PASS: OpenVPN > Are daemons running?
= [tuatara] PASS: OpenVPN > Can connect to openvpn?
= [tuatara] 6 tests: 6 passes, 0 skips, 0 warnings, 0 failures, 0 errors
= [donkey] PASS: Network > Can connect to internet?
= [donkey] PASS: Network > Is stunnel running?
= [donkey] PASS: Network > Is shorewall running?
= [donkey] PASS: Network > Are server certificates valid?
= [donkey] FAIL: Webapp > Can contact couchdb? [webapp.rb:14]:
= [donkey] Expected a response from GET http://webapp:REDACTED@localhost:4000, but got "Connection reset by peer" instead.
= [donkey] (via stunnel to aardwolf.unstable.bitmask.i:15984, aka 199.119.112.205)
= [donkey] Tests halted on failure (because of --no-continue).
= bailing out
This is very misleading because most users will start investigating the webapp on donkey, but the underlying problem is stunnel on aardwolf being down, as we can see when we run leap test unstable --continue
below.
So we need to change the order in which the checks are executed, from the ground-level nodes/services (stunnel, couchdb) first down to highlevel services (webapp) at last.
~/leap/git/bitmask leap test unstable --continue
= [tuatara] PASS: Network > Can connect to internet?
= [tuatara] PASS: Network > Is stunnel running?
= [tuatara] PASS: Network > Is shorewall running?
= [tuatara] PASS: Network > Are server certificates valid?
= [tuatara] PASS: OpenVPN > Are daemons running?
= [tuatara] PASS: OpenVPN > Can connect to openvpn?
= [tuatara] 6 tests: 6 passes, 0 skips, 0 warnings, 0 failures, 0 errors
= [donkey] PASS: Network > Can connect to internet?
= [donkey] PASS: Network > Is stunnel running?
= [donkey] PASS: Network > Is shorewall running?
= [donkey] PASS: Network > Are server certificates valid?
= [donkey] FAIL: Webapp > Can contact couchdb? [webapp.rb:14]:
= [donkey] Expected a response from GET http://webapp:REDACTED@localhost:4000, but got "Connection reset by peer" instead.
= [donkey] (via stunnel to aardwolf.unstable.bitmask.i:15984, aka 199.119.112.205)
= [donkey] PASS: Webapp > Are daemons running?
= [donkey] FAIL: Webapp > Can access webapp? [webapp.rb:31]:
= [donkey] Expected success code from GET https://unstable.bitmask.net/, but got 500 instead.
= [donkey] <!DOCTYPE html>
= [donkey] <html>
= [donkey] <head>
= [donkey] <meta content='width=device-width, initial-scale=1.0' name='viewport'>
= [donkey] <title>unstable.bitmask.net</title>
= [donkey] <meta content='Leap Web' name='description'>
= [donkey] <link rel="stylesheet" media="all" href="/assets/application-ebe937e880e2c82cc4965397cafbc76afca20a69a8df2bb5407df10a36a33230.css" />
= [donkey] <script src="/assets/application-2c69b8452ef0714289b4be76afc9179d3d2fbde31c719e2dcf5b425f62c3b50e.js"></script>
= [donkey] <meta name="csrf-param" content="authenticity_token" />
= [donkey] <meta name="csrf-token" content="mbgG2O9LQbxq9104rU+ZCeS+YpdBCnWDCA2wLFxBJzObCJgt8Xr4q4rtWwH0oRyJTMlgMXprH4AZMaXAMQCB9w==" />
= [donkey] </meta>
= [donkey] </meta>
= [donkey] </head>
= [donkey] <body>
= [donkey] <div id='wrap'>
= [donkey] <div id='masthead'>
= [donkey] <div class='title'>
= [donkey] <span class='sitename'>
= [donkey] <a href='/'>unstable.bitmask.net</a>
= [donkey] </span>
= [donkey] </div>
= [donkey] </div>
= [donkey] <div id='main'>
= [donkey] <div class='container-fluid'>
= [donkey] <div class='row'>
= [donkey] <div class='col-md-12'>
= [donkey] <div id='messages'>
= [donkey] </div>
= [donkey] <div class='hero-unit'>
= [donkey] <h1>Ouch!</h1>
= [donkey] <h2>We ran into a server error.</h2>
= [donkey] <p class='lead'>The problem has been logged and we will look into it.</p>
= [donkey] <a class='btn btn-primary btn-large' href='/'>
= [donkey] <span class="glyphicon glyphicon-home "></span>
= [donkey] Home
= [donkey] </a>
= [donkey] </div>
= [donkey] </div>
= [donkey] </div>
= [donkey] </div>
= [donkey] </div>
= [donkey] <div id='push'>
= [donkey] </div>
= [donkey] </div>
= [donkey] <div id='footer'>
= [donkey] <div class='locales row'>
= [donkey] <div class='wrap-text'>
= [donkey] <a class="locale" href="/500?locale=es">Español</a> <a class="locale active" href="/500?locale=en">English</a> <a class="locale" href="/500?locale=pt">Português</a> <a class="locale" href="/500?locale=de">Deutsch</a>
= [donkey] </div>
= [donkey] </div>
= [donkey] <div class='links row'>
= [donkey] <div class='wrap-text'>
= [donkey] <a href="/"><span class="glyphicon glyphicon-home "></span>Home</a>
= [donkey] <a href="/privacy-policy"><span class="glyphicon glyphicon-eye-close "></span>Privacy Policy</a>
= [donkey] <a href="/terms-of-service"><span class="glyphicon glyphicon-file "></span>Terms of Service</a>
= [donkey] </div>
= [donkey] </div>
= [donkey] </div>
= [donkey] </body>
= [donkey] </html>
= [donkey] rake aborted!
= [donkey] HTTPClient::KeepAliveDisconnected: HTTPClient::KeepAliveDisconnected: Connection reset by peer @ io_fillbuf - fd:7
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:813:in `rescue in block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:804:in `block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:801:in `parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:784:in `read_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:561:in `get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1299:in `do_get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1245:in `do_get_block'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1138:in `rescue in protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1132:in `protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:192:in `send_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:184:in `send_and_parse_response'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:170:in `execute'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:97:in `head'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/server.rb:50:in `database!'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:38:in `prepare_database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/support/inheritable_attributes.rb:48:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/document.rb:91:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:14:in `block (2 levels) in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:12:in `block in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:11:in `create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:47:in `save'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:16:in `block (2 levels) in <top (required)>'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `times'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `block in <top (required)>'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
= [donkey] Caused by:
= [donkey] Errno::ECONNRESET: Connection reset by peer @ io_fillbuf - fd:7
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:805:in `gets'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:805:in `block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:801:in `parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:784:in `read_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:561:in `get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1299:in `do_get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1245:in `do_get_block'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1138:in `rescue in protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1132:in `protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:192:in `send_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:184:in `send_and_parse_response'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:170:in `execute'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:97:in `head'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/server.rb:50:in `database!'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:38:in `prepare_database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/support/inheritable_attributes.rb:48:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/document.rb:91:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:14:in `block (2 levels) in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:12:in `block in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:11:in `create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:47:in `save'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:16:in `block (2 levels) in <top (required)>'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `times'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `block in <top (required)>'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
= [donkey] Caused by:
= [donkey] HTTPClient::KeepAliveDisconnected: HTTPClient::KeepAliveDisconnected: Connection reset by peer @ io_fillbuf - fd:7
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:813:in `rescue in block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:804:in `block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:801:in `parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:784:in `read_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:561:in `get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1299:in `do_get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1245:in `do_get_block'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1133:in `protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:192:in `send_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:184:in `send_and_parse_response'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:170:in `execute'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:97:in `head'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/server.rb:50:in `database!'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:38:in `prepare_database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/support/inheritable_attributes.rb:48:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/document.rb:91:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:14:in `block (2 levels) in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:12:in `block in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:11:in `create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:47:in `save'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:16:in `block (2 levels) in <top (required)>'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `times'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `block in <top (required)>'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
= [donkey] Caused by:
= [donkey] Errno::ECONNRESET: Connection reset by peer @ io_fillbuf - fd:7
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:805:in `gets'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:805:in `block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:801:in `parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:784:in `read_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:561:in `get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1299:in `do_get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1245:in `do_get_block'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1133:in `protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:192:in `send_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:184:in `send_and_parse_response'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:170:in `execute'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:97:in `head'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/server.rb:50:in `database!'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:38:in `prepare_database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/support/inheritable_attributes.rb:48:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/document.rb:91:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:14:in `block (2 levels) in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:12:in `block in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:11:in `create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:47:in `save'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:16:in `block (2 levels) in <top (required)>'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `times'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `block in <top (required)>'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
= [donkey] Tasks: TOP => generate_invites
= [donkey] (See full trace by running task with --trace)
= [donkey] FAIL: Webapp > Can create and authenticate and delete user via API? [webapp.rb:37]:
= [donkey] Expected success code from POST https://api.unstable.bitmask.net:4430/1/users.json, but got 500 instead.
= [donkey] {
= [donkey] "error": "server failed",
= [donkey] "message": "Server Error"
= [donkey] }
= [donkey] rake aborted!
= [donkey] HTTPClient::KeepAliveDisconnected: HTTPClient::KeepAliveDisconnected: Connection reset by peer @ io_fillbuf - fd:7
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:813:in `rescue in block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:804:in `block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:801:in `parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:784:in `read_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:561:in `get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1299:in `do_get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1245:in `do_get_block'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1138:in `rescue in protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1132:in `protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:192:in `send_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:184:in `send_and_parse_response'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:170:in `execute'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:97:in `head'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/server.rb:50:in `database!'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:38:in `prepare_database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/support/inheritable_attributes.rb:48:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/document.rb:91:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:14:in `block (2 levels) in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:12:in `block in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:11:in `create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:47:in `save'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:16:in `block (2 levels) in <top (required)>'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `times'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `block in <top (required)>'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
= [donkey] Caused by:
= [donkey] Errno::ECONNRESET: Connection reset by peer @ io_fillbuf - fd:7
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:805:in `gets'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:805:in `block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:801:in `parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:784:in `read_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:561:in `get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1299:in `do_get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1245:in `do_get_block'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1138:in `rescue in protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1132:in `protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:192:in `send_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:184:in `send_and_parse_response'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:170:in `execute'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:97:in `head'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/server.rb:50:in `database!'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:38:in `prepare_database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/support/inheritable_attributes.rb:48:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/document.rb:91:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:14:in `block (2 levels) in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:12:in `block in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:11:in `create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:47:in `save'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:16:in `block (2 levels) in <top (required)>'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `times'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `block in <top (required)>'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
= [donkey] Caused by:
= [donkey] HTTPClient::KeepAliveDisconnected: HTTPClient::KeepAliveDisconnected: Connection reset by peer @ io_fillbuf - fd:7
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:813:in `rescue in block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:804:in `block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:801:in `parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:784:in `read_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:561:in `get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1299:in `do_get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1245:in `do_get_block'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1133:in `protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:192:in `send_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:184:in `send_and_parse_response'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:170:in `execute'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:97:in `head'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/server.rb:50:in `database!'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:38:in `prepare_database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/support/inheritable_attributes.rb:48:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/document.rb:91:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:14:in `block (2 levels) in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:12:in `block in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:11:in `create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:47:in `save'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:16:in `block (2 levels) in <top (required)>'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `times'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `block in <top (required)>'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
= [donkey] Caused by:
= [donkey] Errno::ECONNRESET: Connection reset by peer @ io_fillbuf - fd:7
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:805:in `gets'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:805:in `block in parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:801:in `parse_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:784:in `read_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient/session.rb:561:in `get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1299:in `do_get_header'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1245:in `do_get_block'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1133:in `protect_keep_alive_disconnected'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:192:in `send_request'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:184:in `send_and_parse_response'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:170:in `execute'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/connection.rb:97:in `head'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/server.rb:50:in `database!'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:38:in `prepare_database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/support/inheritable_attributes.rb:48:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/connection.rb:26:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest-2.0.0/lib/couchrest/document.rb:91:in `database'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:14:in `block (2 levels) in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:12:in `block in create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:81:in `run_callbacks'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:11:in `create'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/couchrest_model-2.1.0.rc1/lib/couchrest/model/persistence.rb:47:in `save'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:16:in `block (2 levels) in <top (required)>'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `times'
= [donkey] /srv/leap/webapp/lib/tasks/invite_code.rake:13:in `block in <top (required)>'
= [donkey] /srv/leap/webapp/vendor/bundle/ruby/2.1.0/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
= [donkey] Tasks: TOP => generate_invites
= [donkey] (See full trace by running task with --trace)
= [donkey] FAIL: Webapp > Can sync Soledad? [webapp.rb:50]:
= [donkey] Expected success code from POST https://api.unstable.bitmask.net:4430/1/users.json, but got 500 instead.
= [donkey] {
= [donkey] "error": "server failed",
= [donkey] "message": "Server Error"
= [donkey] }
= [donkey] 9 tests: 5 passes, 0 skips, 0 warnings, 4 failures, 0 errors
= [aardwolf] PASS: Network > Can connect to internet?
= [aardwolf] FAIL: Network > Is stunnel running? [network.rb:52]:
= [aardwolf] There should be 1 stunnel processes running for `/etc/stunnel/couch_server.conf`.
= [aardwolf] Expected: 1
= [aardwolf] Actual: 0
= [aardwolf] PASS: Network > Is shorewall running?
= [aardwolf] PASS: Network > Are server certificates valid?
= [aardwolf] FAIL: CouchDB > Are daemons running? [couchdb.rb:12]:
= [aardwolf] No running process for bin/beam
= [aardwolf] FAIL: CouchDB > Is CouchDB running? [couchdb.rb:23]:
= [aardwolf] Expected a response from GET http://localhost:5984, but got "Connection refused - connect(2) for "localhost" port 5984" instead.
= [aardwolf] (via direct localhost connection)
= [aardwolf] FAIL: CouchDB > Do ACL users exist? [couchdb.rb:60]:
= [aardwolf] Expected a response from GET http://admin:REDACTED@localhost:5984/_users/_all_docs, but got "Connection refused - connect(2) for "localhost" port 5984" instead.
= [aardwolf] (via direct localhost connection)
= [aardwolf] FAIL: CouchDB > Do required databases exist? [couchdb.rb:78]:
= [aardwolf] Expected a response from GET http://admin:REDACTED@localhost:5984/customers, but got "Connection refused - connect(2) for "localhost" port 5984" instead.
= [aardwolf] (via direct localhost connection)
= [aardwolf] FAIL: CouchDB > Can records be created? [couchdb.rb:108]:
= [aardwolf] Expected a response from POST http://admin:REDACTED@localhost:5984/tokens_593, but got "Connection refused - connect(2) for "localhost" port 5984" instead.
= [aardwolf] (via direct localhost connection)
= [aardwolf] FAIL: CouchDB > Delete mail storage used in mx tests [bonafide_helper.rb:127]:
= [aardwolf] Expected success code from GET https://api.unstable.bitmask.net:4430/1/users/0.json?login=test_user_email, but got 500 instead.
= [aardwolf] {
= [aardwolf] "error": "server failed",
= [aardwolf] "message": "Server Error"
= [aardwolf] }
= [aardwolf] PASS: Soledad > Is Soledad running?
= [aardwolf] 12 tests: 4 passes, 0 skips, 0 warnings, 7 failures, 0 errors
= [demodex] PASS: Network > Can connect to internet?
= [demodex] PASS: Network > Is stunnel running?
= [demodex] PASS: Network > Is shorewall running?
= [demodex] PASS: Network > Are server certificates valid?
= [demodex] 4 tests: 4 passes, 0 skips, 0 warnings, 0 failures, 0 errors
= [cat] PASS: Network > Can connect to internet?
= [cat] PASS: Network > Is stunnel running?
= [cat] PASS: Network > Is shorewall running?
= [cat] PASS: Network > Are server certificates valid?
= [cat] FAIL: Mx > Can contact couchdb? [mx.rb:18]:
= [cat] Expected a response from GET http://leap_mx:REDACTED@localhost:4000/identities, but got "Connection reset by peer" instead.
= [cat] (via stunnel to aardwolf.unstable.bitmask.i:15984, aka 199.119.112.205)
= [cat] FAIL: Mx > Can query identities db? [mx.rb:160]:
= [cat] Expected a response from GET http://leap_mx:REDACTED@localhost:4000/identities, but got "Connection reset by peer" instead.
= [cat] (via stunnel to aardwolf.unstable.bitmask.i:15984, aka 199.119.112.205)
= [cat] PASS: Mx > Are MX daemons running?
= [cat] FAIL: Mx > Can deliver email? [bonafide_helper.rb:96]:
= [cat] Expected success code from GET https://api.unstable.bitmask.net:4430/1/identities/test_user_bad@unstable.bitmask.net.json, but got 500 instead.
= [cat] {
= [cat] "error": "server failed",
= [cat] "message": "Server Error"
= [cat] }
= [cat] FAIL: Mx > Can postfix query leapmx? [mx.rb:160]:
= [cat] Expected a response from GET http://leap_mx:REDACTED@localhost:4000/identities, but got "Connection reset by peer" instead.
= [cat] (via stunnel to aardwolf.unstable.bitmask.i:15984, aka 199.119.112.205)
= [cat] 9 tests: 5 passes, 0 skips, 0 warnings, 4 failures, 0 errors