Tapicero stops working randomly without any trace in the logs
on a singlenode mx setup, with 0.6.0 deployed between rc1 and rc2, tapicero dies randomly. fortunately, i could see this shortly before tapicero died (ignore the shorewall error, we needed to disable shorewall in order to get docker running):
root@pixelated:/home/leap/configuration# leap test --continue = [pixelated.try.pixelated-project.org] PASS: Network > Can connect to internet? = [pixelated.try.pixelated-project.org] PASS: Network > Is stunnel running? = [pixelated.try.pixelated-project.org] FAIL: Network > Is shorewall running? [network.rb:58]: = [pixelated.try.pixelated-project.org] Error running `/sbin/shorewall status`: = [pixelated.try.pixelated-project.org] Shorewall-4.5.5.3 Status at pixelated - Thu Dec 11 20:36:20 CET 2014 = [pixelated.try.pixelated-project.org] Shorewall is stopped = [pixelated.try.pixelated-project.org] State:Cleared (Thu Dec 11 09:04:55 CET 2014) = [pixelated.try.pixelated-project.org] PASS: CouchDB > Are daemons running? = [pixelated.try.pixelated-project.org] PASS: CouchDB > Is CouchDB running? = [pixelated.try.pixelated-project.org] PASS: CouchDB > Is cluster membership ok? = [pixelated.try.pixelated-project.org] PASS: CouchDB > Are configured nodes online? = [pixelated.try.pixelated-project.org] PASS: CouchDB > Do ACL users exist? = [pixelated.try.pixelated-project.org] PASS: CouchDB > Do required databases exist? = [pixelated.try.pixelated-project.org] PASS: CouchDB > Can records be created? = [pixelated.try.pixelated-project.org] PASS: Mx > Can contact couchdb? = [pixelated.try.pixelated-project.org] FAIL: Mx > Can contact couchdb via haproxy? [mx.rb:27]: = [pixelated.try.pixelated-project.org] Expected a 200 status code from GET http://leap_mx:REDACTED@localhost:4096, but got 503 instead. = [pixelated.try.pixelated-project.org] (via haproxy) = [pixelated.try.pixelated-project.org]503 Service Unavailable
= [pixelated.try.pixelated-project.org] No server is available to handle this request. = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] PASS: Mx > Are MX daemons running? = [pixelated.try.pixelated-project.org] PASS: Soledad > Is Soledad running? = [pixelated.try.pixelated-project.org] PASS: Webapp > Can contact couchdb? = [pixelated.try.pixelated-project.org] FAIL: Webapp > Can contact couchdb via haproxy? [webapp.rb:22]: = [pixelated.try.pixelated-project.org] Expected a 200 status code from GET http://localhost:4096, but got 503 instead. = [pixelated.try.pixelated-project.org] (via haproxy) = [pixelated.try.pixelated-project.org]503 Service Unavailable
= [pixelated.try.pixelated-project.org] No server is available to handle this request. = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] PASS: Webapp > Are daemons running? = [pixelated.try.pixelated-project.org] FAIL: Webapp > Can access webapp? [webapp.rb:40]: = [pixelated.try.pixelated-project.org] Expected a 200 status code from GET https://try.pixelated-project.org/, but got 500 instead. = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] try.pixelated-project.org = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org]
= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] try.pixelated-project.org = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]Ouch!
= [pixelated.try.pixelated-project.org]We ran into a server error.
= [pixelated.try.pixelated-project.org]The problem has been logged and we will look into it.
= [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] Home = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] = [pixelated.try.pixelated-project.org] FAIL: Webapp > Can create and authenticate and delete user via API? [webapp.rb:45]: = [pixelated.try.pixelated-project.org] Expected a 200 status code from POST https://api.try.pixelated-project.org:4430/1/users.json, but got 422 instead. = [pixelated.try.pixelated-project.org] {"errors":{"base":["503 Service Unavailable"]}} = [pixelated.try.pixelated-project.org] FAIL: Webapp > Can sync Soledad? [webapp.rb:53]: = [pixelated.try.pixelated-project.org] Expected a 200 status code from POST https://api.try.pixelated-project.org:4430/1/users.json, but got 422 instead. = [pixelated.try.pixelated-project.org] {"errors":{"base":["503 Service Unavailable"]}} = [pixelated.try.pixelated-project.org] 20 tests: 14 passes, 0 skips, 0 warnings, 6 failures, 0 errors= [pixelated.try.pixelated-project.org] Home = [pixelated.try.pixelated-project.org] Privacy Policy = [pixelated.try.pixelated-project.org] Terms of Service = [pixelated.try.pixelated-project.org]= [pixelated.try.pixelated-project.org]
(from redmine: created on 2014-12-11, closed on 2015-04-16, relates #6017 (closed), relates #6539 (closed), relates #6556 (closed), relates #6821 (closed), relates #6832 (closed))