prevent 503 response codes from haproxy / stunnel / couch

Round about once per day the webapp receives a 503 response (service not available) from the couchdb connection. This causes the webapp to crash and i will make sure it retrys and then fails without crashing.

But besides that there should not be a 503 response in the first place. I ruled out some possible causes:

• couch is up an running at the given time
• there's no deploy happening

So it looks like HAproxy - maybe in connection with stunnel is causing this. However i have not been able to find anything suspicious in syslog from stunnel. I have also not been able to track down any haproxy logs at all. The only thing i found are HAproxy stats which i will attach.

In the attached file you will see that 59 requests resulted in 5xx response codes on the Backend side of things. But there were non on the couch_4000 side.

The last time this happened on mudskipper was on the 25th of may (today) at system time 06:29.

(from redmine: created on 2016-05-25, blocks #8144 (closed))