bigcouch is reaching the per-process limit of max. open files
production couch nodes have many errors in bigcouch.log
root@ant:~# grep '\[error\]' /opt/bigcouch/var/log/bigcouch.log | wc -l 1358560 root@thrips:~# grep '\[error\]' /opt/bigcouch/var/log/bigcouch.log | wc -l 3790
in comparism to dev.bm (and there almost certainly all errors are body_too_large errors):
--- ~ » ssh clam "grep '\[error\]' /opt/bigcouch/var/log/bigcouch.log|wc -l" 31 --- ~ » ssh elk "grep '\[error\]' /opt/bigcouch/var/log/bigcouch.log|wc -l" 9 --- ~ » ssh urchin "grep '\[error\]' /opt/bigcouch/var/log/bigcouch.log|wc -l" 33
i attached a part of bigcouch.log
(from redmine: created on 2014-01-09, closed on 2014-11-20, relates #4931, relates #4922 (closed), relates #4930, relates #4926, relates #4965 (closed), relates #5432, relates #6187, relates #6444 (closed), blocks #4932 (closed))