Terminate tee subprocesses before waiting#718
Closed
EspenHa wants to merge 1 commit intoIDSIA:masterfrom
Closed
Conversation
Send termination signals to tee_stdout and tee_stderr, before waiting. This is needed because on some systems there seems to be deadlock issues related to using multiprocessing combined with fd capture. IDSIA#705 (comment) shows an example of this. This change reliably fixes the problem on the system I am using, and hopefully for others as well. There should be no ill effects from terminating the subprocesses because this is done after the stdin pipe is closed. Fixes IDSIA#705
Collaborator
|
Hey, thanks for tackling this. It seems like one of the stdout capturing tests is failing now: def test_fd_tee_output(capsys):
expected_lines = {
"captured stdout",
"captured stderr",
"stdout from C",
"and this is from echo",
}
capture_mode, capture_stdout = get_stdcapturer("fd")
output = ""
with capsys.disabled():
print("before (stdout)")
print("before (stderr)")
with capture_stdout() as out:
print("captured stdout")
print("captured stderr", file=sys.stderr)
output += out.get()
libc.puts(b"stdout from C")
libc.fflush(None)
os.system("echo and this is from echo")
output += out.get()
output += out.get()
print("after (stdout)")
print("after (stderr)")
> assert set(output.strip().split("\n")) == expected_lines
E AssertionError: assert {'and this is...tdout from C'} == {'and this is ...tdout from C'}
E Extra items in the right set:
E 'captured stderr'
E Use -v to get the full diff
tests/test_stdout_capturing.py:57: AssertionError To me it looks like |
Author
|
The test passes for the systems I have access to (three different linux systems). |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Send termination signals to
tee_stdoutandtee_stderr, before waiting.This is needed because on some systems there seems to be deadlock issues related to using multiprocessing combined with fd capture. #705 (comment) shows an example of this.
This change reliably fixes the problem on the system I am using, and hopefully for others as well. There should be no ill effects from terminating the subprocesses because this is done after the
stdin pipe is closed.
Fixes #705