Skip to content

Allow tests to be logged multiple times with pytest-xdist #206

Description

@davehunt

From pytest created by davehunt : pytest-dev/pytest#1193

Note that this is not related to pytest-dev/pytest#927

In pytest-rerunfailures the test protocol is run multiple times in the event of a failure. An enhancement will log each iteration of the test with an outcome of 'rerun' until the maximum number of reruns is reached, at which point the genuine outcome is preserved. This works fine unless running the tests without pytest-xdist, which causes issues when the first rerun of a test is attempted to be removed from the scheduler:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/pytest-2.8.3-py2.7.egg/_pytest/main.py", line 90, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/pytest-2.8.3-py2.7.egg/_pytest/main.py", line 121, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/pytest-2.8.3-py2.7.egg/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/pytest-2.8.3-py2.7.egg/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/pytest-2.8.3-py2.7.egg/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/pytest-2.8.3-py2.7.egg/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/xdist/dsession.py", line 521, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/xdist/dsession.py", line 539, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/xdist/dsession.py", line 643, in slave_testreport
INTERNALERROR>     self.sched.remove_item(node, rep.item_index, rep.duration)
INTERNALERROR>   File "/Users/dhunt/.virtualenvs/tmp-aae67d87c3b61cd1/lib/python2.7/site-packages/xdist/dsession.py", line 280, in remove_item
INTERNALERROR>     self.node2pending[node].remove(item_index)
INTERNALERROR> ValueError: list.remove(x): x not in list

For now I can avoid logging multiple reports if we're running tests in parallel, but that does mean that the only indication of multiple tests running will be the multiple logstart calls. Ideally there should be no difference between running tests in serial or parallel.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions