Bug #1535
opendanger in losing recon volume result from failed reconupload
0%
Description
When uploadEMANRefine.py is run and failed in the middle of the run, reruning it would not work giving the following error because EMAN_result directory is removed and recreated at the beginning of the script while threed.1a.mrc in recon directory has been replaces as a link to the now non-existing volume in the new EMAN_result.
Dmitry, please at least give your thought on what you'd like it to do even if you can't work on it now.
Traceback (most recent call last):
File "/usr/bin/uploadEMANRefine.py", line 605, in <module>
upload3D.start()
File "/usr/bin/uploadEMANRefine.py", line 592, in start
self.createChimeraVolumeSnapshot(newvol, iteration)
File "/usr/lib/python2.6/site-packages/appionlib/reconUploader.py", line 589, in createChimeraVolumeSnapshot
sym='c1', mass=self.params['mass'])
File "/usr/lib/python2.6/site-packages/appionlib/apChimera.py", line 89, in filterAndChimera
apDisplay.printError("Volume file is not valid")
File "/usr/lib/python2.6/site-packages/appionlib/apDisplay.py", line 57, in printError
raise Exception, colorString("\n *** FATAL ERROR ***\n"+text+"\n\a","red")
Exception:
*** FATAL ERROR ***
Volume file is not valid
Updated by Dmitry Lyumkis almost 13 years ago
Anchi, I don't see anywhere in the code where the eman_results directory would be removed, it's only created if it doesn't exist. What I do remember, however, is that there is a unique timestamp that can be associated with every upload. This is set by the reconUploader.py, if not provided, or you can provide it yourself. If you already have a timestamp associated with an unfinished eman upload, can you try providing it in the upload (according to what has already been created, e.g. --timestamp=11jul18d50). I've done this on numerous occasions, where, for example, I would upload iterations 1-3, then kill the job, then upload iterations 4-6 (giving the correct timestamp), etc. If that works, I'll document this in.
On the other hand, perhaps in the webpages we can make it mandatory to have a timestamp there (defaulted as the date), so that this sort of confusion doesn't happen.
Updated by Anchi Cheng over 12 years ago
Dmitry,
Found another case of this in uploadFrealignRefine.py I fixed this one by moving the symbolic link after the iteration is completely uploaded. I suspect that the problem with eman upload is the same. python os.symlink must be rather aggressive that forces an existing file to symbolic link. I assume that your original intention is to avoid duplication. r16883
Updated by Anchi Cheng over 12 years ago
By the way, in Frealign case, the file that becomes a link is in the one in recon directory and linked to the volume in result_path. Indeed because each upload attempt wants to create new time-stamped volume, it does not detect that there is one from previous trial.
Updated by Anchi Cheng over 11 years ago
- Affected Version changed from Appion/Leginon 2.2.0 (trunk) to Appion/Leginon 2.2.0
r17660 just allows me to test easier in uploadEMANRefine.py It is not a fix of this prblem in that script.