Bug #5048
closedGenerating missing stack file error
0%
Description
Unable to generate missing stack file or create stack file when making a substack from 2D classes. Working as user mfalzone on SEMC cluster.
generateMissingStack.py --projectid=197 --expid=4260 --stackid=35 Connected to database: 'nyap_197' ... Getting stack data for stackid=35 Old stack info: 'sub ... 15881 particle substack with 4,7,9,11,13,15,16,19,21,43,40,59,58,36,57,81,99,50,29,25,24,66,47,48,69,93 classes included' ... Querying original particles from stackid=35 ... original stackid: 33 connecting Original stack: /gpfs/appion/mfalzone/17jun23b/stacks/centered31/ali.hed ... generating stack: '/gpfs/appion/mfalzone/17jun23b/stacks/alignsub18/ali.hed' with 15881 particles ... creating a new stack /gpfs/appion/mfalzone/17jun23b/stacks/alignsub18/ali.hed from the oldstack /gpfs/appion/mfalzone/17jun23b/stacks/centered31/ali.hed Traceback (most recent call last): File "/opt/myamisnap/bin/generateMissingStack.py", line 35, in <module> apVirtualStack.generateMissingStack(params['stackid']) File "/opt/myamisnap/lib/appionlib/apVirtualStack.py", line 25, in generateMissingStack apStack.makeNewStack(vstackdata['filename'], fname, plist) File "/opt/myamisnap/lib/appionlib/apStack.py", line 50, in makeNewStack partlist = emanLstFileToPartList(listfile) File "/opt/myamisnap/lib/appionlib/apStack.py", line 70, in emanLstFileToPartList f = open(listfile, "r") TypeError: coercing to Unicode: need string or buffer, list found
Updated by Carl Negro over 7 years ago
I merged changes from trunk to myami-beta that should fix this; this will be pushed to our production at midnight tonight.
Updated by Ashleigh Raczkowski over 7 years ago
Still seeing the same error message today.
Updated by Ashleigh Raczkowski over 7 years ago
This seems to work for other sessions but not for 17jun23b. I a re-running maxlike in case something was wrong with the first two alignments.
Update: no change.
Updated by Anchi Cheng over 7 years ago
- Status changed from New to Assigned
- Assignee set to Gabriel Lander
- Priority changed from Normal to Urgent
- Target version set to Appion/Leginon 3.3
This is a real bug also in the trunk.
This particular missing stack is a substack of the existing stack (stackid=35) which selects 15881 particles from 51760. This size of substack need to use particlelist file to pass the selections.
plist sent from apVirtualStack.py, line 25 is a list
apStack.makeNewStack function did not make a list file with it.
Therefore, when apStack.emanLstFileToPartList(listfile) is called, it failed since listfile is still plist.
Updated by Neil Voss over 7 years ago
Okay, I am sure there are more of these, so I made a change to take either a list or a string for the makeNewStack function.
see commits commit:c519afed and commit:3cd178915
Updated by Neil Voss over 7 years ago
- Status changed from Assigned to In Test
- Assignee changed from Gabriel Lander to Carl Negro
put Carl in charge to make sure commit gets moved over.
Updated by Carl Negro over 7 years ago
Merged, this will update at NYSBC tonight. Thanks
Updated by Ashleigh Raczkowski over 7 years ago
Tried to generate the stack again today:
generateMissingStack.py --projectid=197 --expid=4260 --stackid=35 Connected to database: 'nyap_197' ... Getting stack data for stackid=35 Old stack info: 'sub ... 15881 particle substack with 4,7,9,11,13,15,16,19,21,43,40,59,58,36,57,81,99,50,29,25,24,66,47,48,69,93 classes included' ... Querying original particles from stackid=35 ... original stackid: 33 connecting Original stack: /gpfs/appion/mfalzone/17jun23b/stacks/centered31/ali.hed ... generating stack: '/gpfs/appion/mfalzone/17jun23b/stacks/alignsub18/ali.hed' with 15881 particles ... creating a new stack /gpfs/appion/mfalzone/17jun23b/stacks/alignsub18/ali.hed from the oldstack /gpfs/appion/mfalzone/17jun23b/stacks/centered31/ali.hed ... creating sub-stack from large stack ... ProcessStack2: Free memory: 6.5 GB ... ProcessStack2: Box size: 56 ... ProcessStack2: Memory used per part: 12.2 kB ... ProcessStack2: Max particles in memory: 557831 ... ProcessStack2: Particles allowed in memory: 1000 ... ProcessStack2: Number of particles in stack: 15881 ... ProcessStack2: Particle loop num chunks: 16 ... ProcessStack2: Particle loop step size: 992 ... ProcessStack2: partnum 1 to 992 of 15881 ... going to read 15881 particles from file ... readParticlesListFromFile 0 to 46288 Traceback (most recent call last): File "/opt/myamisnap/bin/generateMissingStack.py", line 35, in <module> apVirtualStack.generateMissingStack(params['stackid']) File "/opt/myamisnap/lib/appionlib/apVirtualStack.py", line 25, in generateMissingStack apStack.makeNewStack(vstackdata['filename'], fname, plist) File "/opt/myamisnap/lib/appionlib/apStack.py", line 54, in makeNewStack stackTools.createSubStack(oldstack, newstack, partlist, msg=True) File "/opt/myamisnap/lib/appionlib/StackClass/stackTools.py", line 81, in createSubStack subStack.start(instack, partlist) File "/opt/myamisnap/lib/appionlib/StackClass/ProcessStack.py", line 143, in start stackarray = self.stackClass.readParticlesFromFile(partlist) File "/opt/myamisnap/lib/appionlib/StackClass/baseClass.py", line 143, in readParticlesFromFile partdatalist = self._readParticleListFromFile(particleNumbers) File "/opt/myamisnap/lib/appionlib/StackClass/imagicClass.py", line 52, in _readParticleListFromFile a = apImagicFile.readSingleParticleFromStack(self.filename, partnum=partnum, msg=self.debug) File "/opt/myamisnap/lib/appionlib/apImagicFile.py", line 492, in readSingleParticleFromStack apDisplay.printError("particle numbering starts at 1") File "/opt/myamisnap/lib/appionlib/apDisplay.py", line 65, in printError raise Exception, colorString("\n *** FATAL ERROR ***\n"+text+"\n\a","red") Exception: *** FATAL ERROR *** particle numbering starts at 1
Updated by Neil Voss over 7 years ago
Hi Ashleigh, I just pushed a fix to the trunk, so give it a try tomorrow.
Updated by Neil Voss over 7 years ago
Oh, I guess I need Carl to merge this as well.
Updated by Carl Negro over 7 years ago
I see one of your commits in trunk and one in myami-beta? Is the trunk up to date where it should be?
Updated by Neil Voss over 7 years ago
Hmm, I did a git rebase command to merge my last two commits and maybe something went wrong.
It is commit commit:a5af8b0c8b7adb2e8eb19ee036299c1d236be594
commit commit:4470bc481af80f087660e3a5491a2c917103336a should not exist, so I reverted it
Updated by Neil Voss over 7 years ago
I think the myami-beta commit is not really there, redmine just thinks it is there.
Updated by Anchi Cheng almost 7 years ago
- Status changed from In Test to Closed