Bug #5048
closedGenerating missing stack file error
0%
Description
Unable to generate missing stack file or create stack file when making a substack from 2D classes. Working as user mfalzone on SEMC cluster.
generateMissingStack.py --projectid=197 --expid=4260 --stackid=35
Connected to database: 'nyap_197'
... Getting stack data for stackid=35
Old stack info: 'sub ... 15881 particle substack with 4,7,9,11,13,15,16,19,21,43,40,59,58,36,57,81,99,50,29,25,24,66,47,48,69,93 classes included'
... Querying original particles from stackid=35
... original stackid: 33
connecting
Original stack: /gpfs/appion/mfalzone/17jun23b/stacks/centered31/ali.hed
... generating stack: '/gpfs/appion/mfalzone/17jun23b/stacks/alignsub18/ali.hed' with 15881 particles
... creating a new stack
/gpfs/appion/mfalzone/17jun23b/stacks/alignsub18/ali.hed
from the oldstack
/gpfs/appion/mfalzone/17jun23b/stacks/centered31/ali.hed
Traceback (most recent call last):
File "/opt/myamisnap/bin/generateMissingStack.py", line 35, in <module>
apVirtualStack.generateMissingStack(params['stackid'])
File "/opt/myamisnap/lib/appionlib/apVirtualStack.py", line 25, in generateMissingStack
apStack.makeNewStack(vstackdata['filename'], fname, plist)
File "/opt/myamisnap/lib/appionlib/apStack.py", line 50, in makeNewStack
partlist = emanLstFileToPartList(listfile)
File "/opt/myamisnap/lib/appionlib/apStack.py", line 70, in emanLstFileToPartList
f = open(listfile, "r")
TypeError: coercing to Unicode: need string or buffer, list found
Updated by Carl Negro over 8 years ago
I merged changes from trunk to myami-beta that should fix this; this will be pushed to our production at midnight tonight.
Updated by Ashleigh Raczkowski over 8 years ago
Still seeing the same error message today.
Updated by Ashleigh Raczkowski over 8 years ago
This seems to work for other sessions but not for 17jun23b. I a re-running maxlike in case something was wrong with the first two alignments.
Update: no change.
Updated by Anchi Cheng over 8 years ago
- Status changed from New to Assigned
- Assignee set to Gabriel Lander
- Priority changed from Normal to Urgent
- Target version set to Appion/Leginon 3.3
This is a real bug also in the trunk.
This particular missing stack is a substack of the existing stack (stackid=35) which selects 15881 particles from 51760. This size of substack need to use particlelist file to pass the selections.
plist sent from apVirtualStack.py, line 25 is a list
apStack.makeNewStack function did not make a list file with it.
Therefore, when apStack.emanLstFileToPartList(listfile) is called, it failed since listfile is still plist.
Updated by Neil Voss over 8 years ago
Okay, I am sure there are more of these, so I made a change to take either a list or a string for the makeNewStack function.
see commits commit:c519afed and commit:3cd178915
Updated by Neil Voss over 8 years ago
- Status changed from Assigned to In Test
- Assignee changed from Gabriel Lander to Carl Negro
put Carl in charge to make sure commit gets moved over.
Updated by Carl Negro over 8 years ago
Merged, this will update at NYSBC tonight. Thanks
Updated by Ashleigh Raczkowski over 8 years ago
Tried to generate the stack again today:
generateMissingStack.py --projectid=197 --expid=4260 --stackid=35
Connected to database: 'nyap_197'
... Getting stack data for stackid=35
Old stack info: 'sub ... 15881 particle substack with 4,7,9,11,13,15,16,19,21,43,40,59,58,36,57,81,99,50,29,25,24,66,47,48,69,93 classes included'
... Querying original particles from stackid=35
... original stackid: 33
connecting
Original stack: /gpfs/appion/mfalzone/17jun23b/stacks/centered31/ali.hed
... generating stack: '/gpfs/appion/mfalzone/17jun23b/stacks/alignsub18/ali.hed' with 15881 particles
... creating a new stack
/gpfs/appion/mfalzone/17jun23b/stacks/alignsub18/ali.hed
from the oldstack
/gpfs/appion/mfalzone/17jun23b/stacks/centered31/ali.hed
... creating sub-stack from large stack
... ProcessStack2: Free memory: 6.5 GB
... ProcessStack2: Box size: 56
... ProcessStack2: Memory used per part: 12.2 kB
... ProcessStack2: Max particles in memory: 557831
... ProcessStack2: Particles allowed in memory: 1000
... ProcessStack2: Number of particles in stack: 15881
... ProcessStack2: Particle loop num chunks: 16
... ProcessStack2: Particle loop step size: 992
... ProcessStack2: partnum 1 to 992 of 15881
... going to read 15881 particles from file
... readParticlesListFromFile 0 to 46288
Traceback (most recent call last):
File "/opt/myamisnap/bin/generateMissingStack.py", line 35, in <module>
apVirtualStack.generateMissingStack(params['stackid'])
File "/opt/myamisnap/lib/appionlib/apVirtualStack.py", line 25, in generateMissingStack
apStack.makeNewStack(vstackdata['filename'], fname, plist)
File "/opt/myamisnap/lib/appionlib/apStack.py", line 54, in makeNewStack
stackTools.createSubStack(oldstack, newstack, partlist, msg=True)
File "/opt/myamisnap/lib/appionlib/StackClass/stackTools.py", line 81, in createSubStack
subStack.start(instack, partlist)
File "/opt/myamisnap/lib/appionlib/StackClass/ProcessStack.py", line 143, in start
stackarray = self.stackClass.readParticlesFromFile(partlist)
File "/opt/myamisnap/lib/appionlib/StackClass/baseClass.py", line 143, in readParticlesFromFile
partdatalist = self._readParticleListFromFile(particleNumbers)
File "/opt/myamisnap/lib/appionlib/StackClass/imagicClass.py", line 52, in _readParticleListFromFile
a = apImagicFile.readSingleParticleFromStack(self.filename, partnum=partnum, msg=self.debug)
File "/opt/myamisnap/lib/appionlib/apImagicFile.py", line 492, in readSingleParticleFromStack
apDisplay.printError("particle numbering starts at 1")
File "/opt/myamisnap/lib/appionlib/apDisplay.py", line 65, in printError
raise Exception, colorString("\n *** FATAL ERROR ***\n"+text+"\n\a","red")
Exception:
*** FATAL ERROR ***
particle numbering starts at 1
Updated by Neil Voss over 8 years ago
Hi Ashleigh, I just pushed a fix to the trunk, so give it a try tomorrow.
Updated by Neil Voss over 8 years ago
Oh, I guess I need Carl to merge this as well.
Updated by Carl Negro over 8 years ago
I see one of your commits in trunk and one in myami-beta? Is the trunk up to date where it should be?
Updated by Neil Voss over 8 years ago
Hmm, I did a git rebase command to merge my last two commits and maybe something went wrong.
It is commit commit:a5af8b0c8b7adb2e8eb19ee036299c1d236be594
commit commit:4470bc481af80f087660e3a5491a2c917103336a should not exist, so I reverted it
Updated by Neil Voss over 8 years ago
I think the myami-beta commit is not really there, redmine just thinks it is there.