Problems running IMAGIC commands through the subprocess module

Added by Michael Cianfrocco over 11 years ago

While trying to run 2D alignments through appion, we have seen that imagic-based 2D alignment (Iterative MSA-MRA) do not run because the script sends up waiting for the imagic command to finish (it will wait indefinitely). This occurs at the very beginning of the 2D alignment script, when it creates a test image from imagic:

... Getting stack data for stackid=2
Old stack info: 'untilted, tiltrun stack'
 ... Getting stack data for stackid=2
Old stack info: 'untilted, tiltrun stack'
 ... Stack 2 pixel size: 4.38
 ... querying one stack particle from stackid=2 on Thu Mar 21 14:13:24 2013
EMAN: proc2d /labdata/allab/leginondata/appion/13feb23w/stacks/stack2/start.hed /labdata/allab/leginondata/appion/13feb23w/align/topol6/13mar21o13.hed apix=4.38 last=199 hp=2000 lp=10
EMAN 1.9 ($Date: 2009/02/18 05:12:22 $).
Run 'proc2d help' for detailed help.
200 images0 complete
executing IMAGIC command: /programs/x/imagic/leschziner/stand/testim.e
waiting for IMAGIC...

I went on to find out that the appion script DOES perform the imagic command, but the subprocess module that it uses to monitor the progress never realizes that the job finished. i was even able to create a small test script that recapitulated the waiting result:

#!/usr/bin/env python

from appionlib.apImagic import imagicFilters

imagicFilters.mask2D(200,0.2,maskfile='mask_imagictest')

This test script will create an imagic mask file, but then will wait indefinitely. I went into the pymagic.py script and found that the variable used to following the command (subprocess.poll()) returns a value of 'None' instead of a value of '0' to signify that it completed.

Since we are using SBGrid here, I asked them for insight into this problem since we are running the imagic program over the SBGrid installation. They did not know how to overcome this problem. I even copied a local version of imagic to the same workstation, and I got the same problem.

Has anyone seen this before? We are running imagic version 120619e on cent os 5 (red hat) using appion branch version 2.2.

Also, I should say that the 2D alignments using XMIPP, EMAN, and SPIDER work without any problem on our installation of appion.

Thank you for your help,
mike

Replies (1)

RE: Problems running IMAGIC commands through the subprocess module - Added by Michael Cianfrocco over 11 years ago

I think I inadvertently resolved this issue. While attempting to debug the script, I put the following 'print' command (to see the commands within the executable file) within the following pymagic definition:

        def close(self):
                print self.imagicexe                  #debug
                self.wait()
                for file in ['fort.1', 'jnkASSIGN1']:
                        if os.path.exists(file):
                                try: os.remove(file)
                                except: pass
                self.logf.close()

In addition to commenting out most of the commands within the wait() definition:

        def wait(self):
                ### waits until IMAGIC quits

                ### set wait times
                waittime = 2.0
                time.sleep(waittime)
                #self.logf.flush()
                ### check number 1
                #if self.imagicproc.poll() is None:
                #       waiting = True
                #       time.sleep(waittime)
                #else:
                #       self.imagicproc.wait()
                #       return
                ### check number 2
                #if self.imagicproc.poll() is None:
                #       waiting = True
                #       sys.stderr.write("waiting for IMAGIC")
                #else:
                #self.imagicproc.wait()
                #return
                ### continuous check
                #while self.imagicproc.poll() is None:
                #       if waittime > 10:
                #               sys.stderr.write(".")
                #       time.sleep(waittime)
                #       waittime *= 1.1
                #       self.logf.flush()
                #if waiting is True:
                #tdiff = time.time()-self.starttime
                #if tdiff > 20:
                #tstr = self.timeString(tdiff)
                #sys.stderr.write("\nIMAGIC completed in "+tstr+"\n")
                #       else:
                #               sys.stderr.write("\n")
                #self.imagicproc.wait()

As far as I can tell, this works without any problems. I know that it's a serious hack, but, in searching the internet for help to my subprocess problem, there were not any well documented solutions.

(1-1/1)

Project

General

Profile

Leginon