failing encdecpool test on parallel environment after some runs

If i run tox -e parallel some times in a row, i eventually reach a failing test in encdecpool. It seems to vary from 2 to 5 runs to be able to see the problem.

I'm using:

• soledad development branch (e8747096)
• couchdb 1.6.0
• ssd disk

The following is the output of runnign only the encdecpool tests in parallel:

$ tox -e parallel tests/sync/test_encdecpool.py
GLOB sdist-make: /home/drebs/dev/leap/repos/soledad/testing/setup.py
parallel inst-nodeps: /home/drebs/dev/leap/repos/soledad/testing/.tox/dist/test_soledad-0.0.0.zip
parallel installed:
apipkg==1.4,Beaker==1.6.3,cffi==1.7.0,configparser==3.5.0,CouchDB==1.0.1,cryptography==1.4,
decorator==4.0.10,enum34==1.1.6,execnet==1.4.1,extras==1.0.0,fancycompleter==0.5,
fixtures==3.0.0,funcsigs==1.0.2,functools32==3.2.3.post2,greenlet==0.4.10,idna==2.1,
ipaddress==1.0.16,jsonschema==2.5.1,leap.common==0.5.2,-e git+ssh://code.leap.se/soledad.git
@e8747096045933754d3f8ac2608cce844f6b0fee#egg=leap.soledad.client&subdirectory=client,
-e git+ssh://code.leap.se/soledad.git@e8747096045933754d3f8ac2608cce844f6b0fee#egg=
leap.soledad.common&subdirectory=common,-e git+ssh://code.leap.se/soledad.git
@e8747096045933754d3f8ac2608cce844f6b0fee#egg=leap.soledad.server&subdirectory=server,
linecache2==1.0.0,mock==2.0.0,ordereddict==1.1,Paste==2.0.3,pathlib2==2.1.0,
pbr==1.10.0,pdbpp==0.8.3,py==1.4.31,pyasn1==0.1.9,pycparser==2.14,Pygments==2.1.3,
pyOpenSSL==16.0.0,pyrepl==0.8.4,pysqlcipher==2.6.5,pytest==2.9.2,pytest-twisted==1.5,
pytest-xdist==1.14,python-dateutil==2.5.3,python-mimeparse==1.5.2,pyzmq==15.3.0,
repoze.lru==0.6,requests==2.10.0,Routes==2.3.1,scrypt==0.7.1,setuptools-trial==0.6.0,
six==1.10.0,test-soledad==0.0.0,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,
Twisted==16.3.0,txZMQ==0.7.4,unittest2==1.1.0,wmctrl==0.3,zope.interface==4.2.0,
zope.proxy==4.2.0
parallel runtests: PYTHONHASHSEED='2498300587'
parallel runtests: commands[0] | py.test tests/sync/test_encdecpool.py -n 4
===================================== test session starts =====================================
platform linux2 -- Python 2.7.9, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/drebs/dev/leap/repos/soledad/testing, inifile: pytest.ini
plugins: twisted-1.5, xdist-1.14
gw0 [10] / gw1 [10] / gw2 [10] / gw3 [10]
scheduling tests via LoadScheduling
.F........
========================================== FAILURES ===========================================
___________________ TestSyncEncrypterPool.test_encrypt_doc_and_get_it_back ____________________
[gw2] linux2 -- Python 2.7.9 /home/drebs/dev/leap/repos/soledad/testing/.tox/parallel/bin/python2
result = None, g = 
deferred = 

    def _inlineCallbacks(result, g, deferred):
        """
        See L{inlineCallbacks}.
        """
        # This function is complicated by the need to prevent unbounded recursion
        # arising from repeatedly yielding immediately ready deferreds.  This while
        # loop and the waiting variable solve that by manually unfolding the
        # recursion.
    
        waiting = [True, # waiting for result?
                   None] # result
    
        while 1:
            try:
                # Send the last result back as the result of the yield expression.
                isFailure = isinstance(result, failure.Failure)
                if isFailure:
                    result = result.throwExceptionIntoGenerator(g)
                else:
>                   result = g.send(result)

.tox/parallel/local/lib/python2.7/site-packages/twisted/internet/defer.py:1128: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/sync/test_encdecpool.py:76: in test_encrypt_doc_and_get_it_back
    self.assertIsNotNone(encrypted)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = 
msg = 'unexpectedly None'

    def fail(self, msg=None):
        """
            Absolutely fail the test.  Do not pass go, do not collect $200.
    
            @param msg: the message that will be displayed as the reason for the
            failure
            """
>       raise self.failureException(msg)
E       FailTest: unexpectedly None

.tox/parallel/local/lib/python2.7/site-packages/twisted/trial/_synctest.py:368: FailTest
============================= 1 failed, 9 passed in 8.63 seconds ==============================
ERROR: InvocationError: '/home/drebs/dev/leap/repos/soledad/testing/.tox/parallel/bin/py.test tests/sync/test_encdecpool.py -n 4'
___________________________________________ summary ___________________________________________
ERROR:   parallel: commands failed

(from redmine: created on 2016-08-21, closed on 2016-09-03)

• Relations:
  • relates #8283 (closed)