Bug #3752: Generating missing virtual substack from virtual substack gives header nz is None error - Appion - Electron Microscopy Group

Actions

Copy link

Bug #3752

closed

Generating missing virtual substack from virtual substack gives header nz is None error

Added by William Rice about 9 years ago. Updated about 9 years ago.

Status:

Closed

Priority:

Normal

Assignee:

William Rice

Category:

Target version:

Appion/Leginon 3.3

Start date:

11/10/2015

Due date:

% Done:

Estimated time:

Affected Version:

Appion/Leginon 3.2

Show in known bugs:

Yes

Workaround:

follow the error message to make the parent stack

Description

Made a substack by deleting several bad class averages (3rd round of this). Getting errors when wokring with this substack, same error with any alignment runs. Tried to make the "real stack" generated an identical error:

[rice@guppy ~]$ generateMissingStack.py --projectid=33 --expid=755 --stackid=39
Error:  image path is not writable: /emg/data00/leginon
Connected to database: 'nyap_33'
 ... Getting stack data for stackid=39
Old stack info: 'wjr further removal after serden som of xmipp max liklihood number 8  ... 13244 particle substack with 54,55,57,94,93,92,91,83,82,74,52,53,9,64,65,66,67 classes excluded'
 ... Querying original particles from stackid=39
connecting
 ... Getting stack data for stackid=39
Old stack info: 'wjr further removal after serden som of xmipp max liklihood number 8  ... 13244 particle substack with 54,55,57,94,93,92,91,83,82,74,52,53,9,64,65,66,67 classes excluded'
 ... Stack 39 pixel size: 1.47
 ... generating stack: '/gpfs/appion/rice/15sep24a/stacks/clustersub75/start.hed' with 13244 particles
Traceback (most recent call last):
  File "/opt/myamisnap/bin/generateMissingStack.py", line 54, in <module>
    a.run()
  File "/opt/myamisnap/lib/appionlib/proc2dLib.py", line 45, in run
    self.approc2d.checkConflicts()
  File "/opt/myamisnap/lib/appionlib/proc2dLib.py", line 112, in checkConflicts
    self.inputNumParticles = self.inheader['nz']
TypeError: 'NoneType' object is unsubscriptable

Related issues 2 (0 open — 2 closed)

Actions

Copy link

Updated by William Rice about 9 years ago

Moved data to /gpfs and Anchi updated database. Still not working. Running Maximum Liklihood manually after qsub -I:

[rice@guppy-19 ~]$ /opt/myamisnap/bin/appion maxlikeAlignment.py --description="maxlike 13244 stack 39" --stack=39 --lowpass=10 --highpass=2000 --num-part=13244 --num-ref=30 --bin=3 --angle-interval=5 --max-iter=15 --nproc=8 --fast --fast-mode=normal --mirror --savemem --commit --converge=normal --rundir=/gpfs/appion/rice/15sep24a/align/wjrmaxlike10 --runname=wjrmaxlike10 --projectid=33 --expid=755 --jobtype=partalign
Error:  image path is not writable: /emg/data00/leginon

 ... Time stamp: 15nov11j22
 ... Function name: maxlikeAlignment
 ... Appion directory: /opt/myamisnap/lib
 ... Using split database
Connected to database: 'nyap_33'
 ... Committing data to database
 ... Querying original particles from stackid=39
connecting
 ... Getting stack data for stackid=39
Old stack info: 'wjr further removal after serden som of xmipp max liklihood number 8  ... 13244 particle substack with 54,55,57,94,93,92,91,83,82,74,52,53,9,64,65,66,67 classes excluded'
 ... Run directory: /gpfs/appion/rice/15sep24a/align/wjrmaxlike10
!!! WARNING: directory '/gpfs/appion/rice/15sep24a/align/wjrmaxlike10' already exists.
 ... Writing function log to: maxlikeAlignment.log
 ... Uploading ScriptData....
 ... Found 1 processors on this machine
 ... Running Appion version 'myami-3.2'
self.params['maxlikejobid'] 20
 ... Getting stack data for stackid=39
Old stack info: 'wjr further removal after serden som of xmipp max liklihood number 8  ... 13244 particle substack with 54,55,57,94,93,92,91,83,82,74,52,53,9,64,65,66,67 classes excluded'
 ... Stack 39 pixel size: 1.47
 ... Getting stack data for stackid=39
Old stack info: 'wjr further removal after serden som of xmipp max liklihood number 8  ... 13244 particle substack with 54,55,57,94,93,92,91,83,82,74,52,53,9,64,65,66,67 classes excluded'
Estimated first iteration time: 2 hr 43 min
Traceback (most recent call last):
  File "/opt/myamisnap/bin/maxlikeAlignment.py", line 498, in <module>
    maxLike.start()
  File "/opt/myamisnap/bin/maxlikeAlignment.py", line 418, in start
    a.run()
  File "/opt/myamisnap/lib/appionlib/proc2dLib.py", line 45, in run
    self.approc2d.checkConflicts()
  File "/opt/myamisnap/lib/appionlib/proc2dLib.py", line 112, in checkConflicts
    self.inputNumParticles = self.inheader['nz']
TypeError: 'NoneType' object is unsubscriptable

Actions

Copy link

Updated by Anchi Cheng about 9 years ago

Project changed from 138 to Appion
Subject changed from Having trouble working with substack to Generating missing virtual substack from virtual substack gives header nz is None error
Status changed from New to Assigned
Assignee set to Gabriel Lander
Target version set to Appion/Leginon 3.3
Affected Version set to Appion/Leginon 3.2
Show in known bugs set to Yes
Workaround updated (diff)

This will need a recursive virtual stack regeneration mechanism to solve this problem.

In the mean time r19334 gives error message for what file is missing so the user can back track.

Actions

Copy link

Updated by Anchi Cheng about 9 years ago

Related to Bug #3688: cl2d and virtual stacks added

Actions

Copy link

Updated by Anchi Cheng about 9 years ago

Related to Feature #3357: Virtual stacks added

Actions

Copy link

Updated by Gabriel Lander about 9 years ago

I didn't look closely enough at Anchi's changes in r19115, it screwed up the recursive search. I fixed it now in r19338, this should be working again. Please try.

Actions

Copy link