Appion

could this have something to do with Neil's recent update to makestack2.py for custom sorting of CTF values, r18426?

It was an EMAN1 CTF issue, fixed now, r18434.

It doesn't work for me but maybe I am not pointing to the right install?
I am using cronus3.scripps.edu/betamyamiweb/
Thanks

Can I get the command line and the directory?

Actually, it appears to be using the old version of the code that is why it crashed.

I will have to look why you are getting repeated "no best value" messages.

Hi Neil,

In this directory there are 6 stack jobs all of which crashed:
/ami/data15/appion/14jul17b/stacks/

stack1 had already started before the change was made, so I don't know if that would be helpful to see exactly what changed and made it crash.

stack2 through 6 all have the same "no best value" error.

Let me know if you need any more information.

FYI,
Any changes made to the code will not be available in betamyamiweb until the next morning. If it is critical to have the fix today, Sargis will need to push the code manually.

Okay, all is working once you start running the latest version.

I was unable to reproduce the "no best value" errors, but I think that is because you are running makestack before all of the CTFFIND images have finished. It is pretty complex code, but I think is actually rejecting the images in this case.

I believe the error comes from the fact we are not using the latest version (awaiting for the update to be made next night) as in my case ctffind was done estimating defocus.

my lab was able to get this working by removing the "--sort-type= " option from the command line. The Ward lab is also using this workaround until the trunk updates.

Sargis, if it is not too much trouble for you, would you mind to please make the update?

Hmm, I am confused by your workaround Gabe. The code that fails is in EMAN1 applyctf and should not relate to sort type.

You're right, this is a separate issue - Is the garibaldi trunk updated automatically? I think that's the problem - the web interface was updated but the myami/trunk on garibaldi is still running an older version of myami.

Updated myami/trunk on garibaldi. JC gave me write access permission to myami trunk on garibaldi so I can update it when needed. I'll be out till next Friday. Please email hpc_ca@scripps.edu if it needs to be updated again before that.

Thanks Sargis for updating but the issue is still not fixed as i
am getting the same error message.

Can anyone who is avaible please try and resolve this! We have no pipeline if we can’t make stacks!
Can we at the very least revert back to the old makestack while we try and resolve this?
We really do need a trunk and a branch!

It sounds like you have several different myami versions all over the place.

Everything is running trunk. The trunk is updated on all web servers, workstations, and work nodes nightly EXCEPT garibaldi. Garibaldi is now updated to trunk, so everyone is using the same trunk version.
David, please send the makestack2.py command you're trying to run.

Hi Gabe,

Here is an example of a makestack job file that doesn't work that I just submitted, after I took out the "--sort-type= " flag:

#!/bin/csh
#PBS -l walltime=240:00:00
#PBS -l nodes=1:ppn=2
#PBS -m e
#PBS -j oe
#PBS -V

1. Target Host: guppy
2. This job file has been created for the guppy processing host.
3. Changes may be required to run this on another host.

updateAppionDB.py 327 R 416
webcaller.py '/opt/myamisnap/bin/appion makestack2.py --single=start.hed --selectionid=43 --invert --phaseflip --flip-type=emanimage --boxsize=448 --bin=2 --ctfrunid=12 --description="test again" --runname=stack11 --rundir=/ami/data15/appion/14jul17b/stacks/stack11 --commit --preset=en --projectid=416 --session=14jul17b --no-rejects --no-wait --continue --expid=13705 ' /ami/data15/appion/14jul17b/stacks/stack11/stack11.appionsub.log
updateAppionDB.py 327 D 416
exit

And the resulting error:
Starting image 1 ( skip:38, remain:557 ) id:3062172, file: 14jul17b_013sq_02hl_02en
... Pixel size: 1.36376
... Found 46 particles
... Boxing is done on /ami/data00/leginon/14jul17b/rawdata/14jul17b_00013sq_v02_00002hl_v02_00002en.mrc

Traceback (most recent call last):
File "/opt/myamisnap/bin/makestack2.py", line 1213, in <module>
makeStack.run()
File "/opt/myamisnap/lib/appionlib/appionLoop2.py", line 90, in run
results = self.loopProcessImage(imgdata)
File "/opt/myamisnap/lib/appionlib/appionLoop2.py", line 140, in loopProcessImage
return self.processImage(imgdata)
File "/opt/myamisnap/lib/appionlib/apParticleExtractor.py", line 504, in processImage
total_processed_particles = self.processParticles(imgdata,partdatas,shiftdata)
File "/opt/myamisnap/bin/makestack2.py", line 77, in processParticles
self.boxedpartdatas, self.imgstackfile, self.partmeantree = self.boxParticlesFromImage(imgdata,partdatas,shiftdata)
File "/opt/myamisnap/bin/makestack2.py", line 235, in boxParticlesFromImage
self._boxParticlesFromImage(imgdata,parttree, imgstackfile)
File "/opt/myamisnap/bin/makestack2.py", line 251, in _boxParticlesFromImage
imgpath = self.phaseFlipWholeImage(imgpath, imgdata)
File "/opt/myamisnap/bin/makestack2.py", line 555, in phaseFlipWholeImage
defocus, ampconst = self.getDefocusAmpConstForImage(imgdata, True)
AttributeError: 'Makestack2Loop' object has no attribute 'getDefocusAmpConstForImage'

the ctf has been estimated for the entire data set.

Thanks!!

Hi Melody and Gabe, You may be importing an older version of myami in the PYTHONPATH, the fix is in apParticleExtration.py in appionlib.

I have to pick up the kids in 20 minutes, so if you can try it quickly then I can help, but otherwise I will check my email again at 9pm Central (7pm Pacific).

Hi Melody, I think you'll have to wait until overnight for things to work.

"The trunk is updated on all web servers, workstations, and work nodes nightly"

Hi Neil - no I don;t think that is it as Sargis did a push update today. Bridget

Well, I just checked and it was not updated on guppy.

Oh OK, thanks Sorry. So we may all be chasing shadows here.

Hi,

I've submitted a similar job on garibaldi (also removing the "--sort-type= " flag). So far it is running!!

Thanks all!

Just tried on garibaldi using the option "only use ctffind values" and it works as well.

Thanks a lot.

David