Bug #1528
openctfestimate,py deadlock on subprocess.wait()
0%
Description
According to python doc, this can happen when using stdout=PIPE and/or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data.
This starts happening on CentOS installation and new ctffind3 that I downloaded. Maybe the buffer is smaller here or the output is larger.
Updated by Anchi Cheng almost 13 years ago
- Status changed from Assigned to In Code Review
- Assignee changed from Anchi Cheng to Jim Pulokas
r16257 fixed this particular case. Not sure how many of other cases where we use wait() need to be changed. According to python doc communicate() can only be used when the returned data can be handled by memory.
Updated by Jim Pulokas over 12 years ago
The warning in the python docs only indicate that it will block if the child stdout or stderr are connected to PIPE, but in this case only the stdin is connected to a PIPE. The output is connected to an open file object (logf), so not sure how the warning applies in this case. I guess the doc is wrong because it obviously fixes it to use communicate.
Other cases where this could hit us again in appion:
Checking where we use proc.wait():
grep -r "proc.wait()" * |grep -v svnreturns 165 lines. Those are not necessarily all bad because many do not redirect stdout,stderr to pipes or files, they just let it out through the parent process stdout/err. Can also check how many Popen() were created with stdout/err/in redirection:
grep -r Popen * | grep -v svn | grep stdreturns 63 lines
I will try to fix as many as possible, but I think we don't have to worry about getting them all since no other problems have been reported.