leap_mx crashes bigcouch trying to store a certain, big mail
see also the last part of #5024 (closed)
on 13:35:11, leap_mx tries to deliver this single leftover mail:
root@starfish:~# ls -alh /var/mail/vmail/Maildir/new/ total 9.5M drwx------ 2 vmail vmail 28K Jan 30 13:14 . drwx------ 5 vmail vmail 4.0K Jul 10 2013 .. -rw------- 1 vmail vmail 9.5M Jan 22 15:49 1390405744.Vfe12I2675cM454881.starfish
the log shows this:
2014-01-30 13:25:11+0000 [-] Starting processing skipped mail... 2014-01-30 13:25:11+0000 [-] -------------------------------------------------- 2014-01-30 13:25:11+0000 [-] Processing new mail at '/var/mail/vmail/.bashrc' 2014-01-30 13:25:11+0000 [-] Don't know how to deliver mail '/var/mail/vmail/.bashrc', skipping... 2014-01-30 13:25:11+0000 [-] Processing new mail at '/var/mail/vmail/.profile' 2014-01-30 13:25:11+0000 [-] Don't know how to deliver mail '/var/mail/vmail/.profile', skipping... 2014-01-30 13:25:11+0000 [-] Processing new mail at '/var/mail/vmail/.bash_logout' 2014-01-30 13:25:11+0000 [-] Don't know how to deliver mail '/var/mail/vmail/.bash_logout', skipping... 2014-01-30 13:25:11+0000 [-] Processing new mail at '/var/mail/vmail/Maildir/new/1390405744.Vfe12I2675cM454881.starfish' 2014-01-30 13:25:12+0000 [-] Mail owner: 847a6ece58ce498e4cd86cb10a6e9f98 2014-01-30 13:25:12+0000 [-] Starting factory 2014-01-30 13:25:12+0000 [HTTP11ClientProtocol,client] Encrypting message to 847a6ece58ce498e4cd86cb10a6e9f98's pubkey 2014-01-30 13:25:14+0000 [HTTP11ClientProtocol,client] Exporting message for 847a6ece58ce498e4cd86cb10a6e9f98 2014-01-30 13:25:35+0000 [HTTP11ClientProtocol,client] Error processing skipped mail: '/var/mail/vmail/Maildir/new/1390405744.Vfe12I2675cM454881.starfish' 2014-01-30 13:25:35+0000 [HTTP11ClientProtocol,client] Unhandled Error Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 575, in _runCallbacks current.result = callback(current.result, *args, **kw) File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1126, in gotResult _inlineCallbacks(r, g, deferred) File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1068, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) --- --- File "/usr/lib/python2.7/dist-packages/leap/mx/mail_receiver.py", line 268, in _process_skipped yield self._step_process_mail_backend(fpath) File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1070, in _inlineCallbacks result = g.send(result) File "/usr/lib/python2.7/dist-packages/leap/mx/mail_receiver.py", line 317, in _step_process_mail_backend do_remove = yield self._export_message(uuid, doc) File "/usr/lib/python2.7/dist-packages/leap/mx/mail_receiver.py", line 180, in _export_message db.put_doc(doc) File "/usr/lib/python2.7/dist-packages/leap/soledad/common/couch.py", line 547, in put_doc self._put_doc(old_doc, doc) File "/usr/lib/python2.7/dist-packages/leap/soledad/common/couch.py", line 504, in _put_doc headers={'content-type': 'application/json'}) File "/usr/lib/pymodules/python2.7/couchdb/http.py", line 405, in put_json status, headers, data = self.put(*a, **k) File "/usr/lib/pymodules/python2.7/couchdb/http.py", line 384, in put return self._request('PUT', path, body=body, headers=headers, **params) File "/usr/lib/pymodules/python2.7/couchdb/http.py", line 419, in _request credentials=self.credentials) File "/usr/lib/pymodules/python2.7/couchdb/http.py", line 310, in request raise ServerError((status, error)) couchdb.http.ServerError: (504, "\nThe server didn't respond in time.\n\n") 2014-01-30 13:25:35+0000 [HTTP11ClientProtocol,client] ++++++++++++++++++++++++++++++++++++++++++++++++++ 2014-01-30 13:25:35+0000 [HTTP11ClientProtocol,client] Done processing skipped mail 2014-01-30 13:25:35+0000 [HTTP11ClientProtocol,client] Stopping factory 504 Gateway Time-out
meanwhile in the bigcouch log on elk:
[Thu, 30 Jan 2014 13:25:35 GMT] [info] [<0.26104.1>] [dbbcc5c4] undefined - - 'GET' / 200 [Thu, 30 Jan 2014 13:25:35 GMT] [info] [<0.26105.1>] [cf3680c3] undefined - - 'GET' / 200 [Thu, 30 Jan 2014 13:25:36 GMT] [info] [<0.26104.1>] [dbbcc5c4] 127.0.0.1 undefined GET / 200 ok 119 [Thu, 30 Jan 2014 13:25:36 GMT] [info] [<0.26105.1>] [cf3680c3] 127.0.0.1 undefined GET / 200 ok 124 [Thu, 30 Jan 2014 13:25:37 GMT] [info] [<0.93.0>] [--------] Apache CouchDB has started on http://127.0.0.1:5986/ [Thu, 30 Jan 2014 13:25:39 GMT] [info] [<0.185.0>] [abd2b4a9] undefined - - 'GET' / 200 [Thu, 30 Jan 2014 13:25:39 GMT] [info] [<0.186.0>] [59ea786f] undefined - - 'GET' / 200 [Thu, 30 Jan 2014 13:25:39 GMT] [error] [<0.185.0>] [abd2b4a9] Uncaught error in HTTP request: {exit,normal} [Thu, 30 Jan 2014 13:25:39 GMT] [error] [<0.186.0>] [59ea786f] Uncaught error in HTTP request: {exit,normal} [Thu, 30 Jan 2014 13:25:39 GMT] [info] [<0.185.0>] [abd2b4a9] Stacktrace: [{mochiweb_request,send,2, [{file,"src/mochiweb_request.erl"}, {line,146}]}, {mochiweb_request,respond,2, [{file,"src/mochiweb_request.erl"}, {line,330}]}, {couch_httpd,send_response,4, [{file,"src/couch_httpd.erl"}, {line,618}]}, {chttpd,handle_request,1, [{file,"src/chttpd.erl"}, {line,198}]}, {mochiweb_http,headers,5, [{file,"src/mochiweb_http.erl"}, {line,126}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"}, {line,227}]}] [Thu, 30 Jan 2014 13:25:39 GMT] [info] [<0.186.0>] [59ea786f] Stacktrace: [{mochiweb_request,send,2, [{file,"src/mochiweb_request.erl"}, {line,146}]}, {mochiweb_request,respond,2, [{file,"src/mochiweb_request.erl"}, {line,330}]}, {couch_httpd,send_response,4, [{file,"src/couch_httpd.erl"}, {line,618}]}, {chttpd,handle_request,1, [{file,"src/chttpd.erl"}, {line,198}]}, {mochiweb_http,headers,5, [{file,"src/mochiweb_http.erl"}, {line,126}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"}, {line,227}]}] [Thu, 30 Jan 2014 13:25:39 GMT] [info] [<0.185.0>] [abd2b4a9] undefined - - 'GET' / 500 [Thu, 30 Jan 2014 13:25:39 GMT] [info] [<0.186.0>] [59ea786f] undefined - - 'GET' / 500
(from redmine: created on 2014-01-30, closed on 2016-08-09)