SOLVED: ctffind4.py gets confused about its location
Added by Patrick Goetz about 7 years ago
I followed the link here: http://emg.nysbc.org/projects/appion/wiki/Install_Grigorieff_lab_software
to install ctffind4 and friends. The make install step puts all the executables into /usr/local/bin, but this seems not to work with Appion:
cnsit@kraken:~$ ctffind4.py --ampcontrast=0.07 --fieldsize=1024 --resmin=50 --resmax=4 --defstep=0.1 --numstep=25 --dast=0.05 --parallel --min_phase_shift=10 --max_phase_shift=170 --phase_search_step=10 --runname=ctffind4run1 --rundir=/data2/appion/17nov28z/ctf/ctffind4run1 --preset=upload --commit --projectid=1 --session=17nov28z --no-rejects --no-wait --continue --bestdb --expid=2 --jobtype=ctfestimate
/usr/lib/python2.7/dist-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
... Time stamp: 17dec13l50
... Function name: ctffind4
... Appion directory: /usr/local/lib/python2.7/dist-packages
... Processing hostname: kraken
... Using split database
Connected to database: 'ap1'
... Committing data to database
... Looking up session, 17nov28z
... Found project id=1 for session 17nov28z in 1.98 msec
... Run directory: /data2/appion/17nov28z/ctf/ctffind4run1
!!! WARNING: directory '/data2/appion/17nov28z/ctf/ctffind4run1' already exists.
... Writing function log to: ctffind4.log
... Uploading ScriptData....
... Found 32 processors on this machine
... Looking up session, 17nov28z
... Running Appion version 'myami-beta'
... Reading old done dictionary: ctffind4.donedict
... Found 1 done dictionary entries
... Querying database for preset 'upload' images from session '17nov28z' ...
... 1 images recevied in 6.44 msec
... Found 1 images in 6.56 msec
... Remove processed images
... finished skipping in 6.41 msec
... Process images old to new
Traceback (most recent call last):
File "/usr/local/bin/ctffind4.py", line 466, in <module>
imgLoop.run()
File "/usr/local/lib/python2.7/dist-packages/appionlib/appionLoop2.py", line 66, in run
self.preLoopFunctions()
File "/usr/local/bin/ctffind4.py", line 98, in preLoopFunctions
self.ctfprgmexe = self.getCtfProgPath()
File "/usr/local/bin/ctffind4.py", line 111, in getCtfProgPath
apDisplay.printError(exename+" was not found at: "+apParam.getAppionDirectory())
File "/usr/local/lib/python2.7/dist-packages/appionlib/apDisplay.py", line 65, in printError
raise Exception, colorString("\n *** FATAL ERROR ***\n"+text+"\n\a","red")
Exception:
*** FATAL ERROR ***
ctffind4 was not found at: /usr/local/lib/python2.7/dist-packages
If ctffind4 needs to go into the local python dist-packages, it's not clear what the folder name should be or how exactly it would be found without a .egg.info file.
Any instructions for how this needs to be set up to work properly with the Appion tools?
Replies (11)
RE: ctffind4.py gets confused about its location - Added by Sargis Dallakyan about 7 years ago
Is /usr/local/bin in your PATH? Reading the code at tffind4.py#L107 it uses `which ctffind4` to find the exceutable. If it fails, it tries to use getAppionDirectory. And if it fails too, it raises that error.
RE: ctffind4.py gets confused about its location - Added by Anchi Cheng about 7 years ago
See Processing_Server_Installation for that table of the executable alias we use. You can make a soft link of the particular download you get to this name inside one of the path Sargis mentioned.
RE: ctffind4.py gets confused about its location - Added by Patrick Goetz about 7 years ago
Sargis: /usr/local/bin is in my path:
cnsit@kraken:~$ echo $PATH /home/cnsit/bin:/home/cnsit/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/snap/bin:/usr/local/EMAN2/bin
cnsit@kraken:~$ which ctffind4.py /usr/local/bin/ctffind4.py
I'm confused by line 107 in ctffind.py, though. It's never going to find ctffind4 because that file doesn't exist without an extension:
cnsit@kraken:~$ cd /usr/local/bin cnsit@kraken:/usr/local/bin$ ls ctf* ctfestimate.py ctffind4-pow_avrot_plot.py ctffind4.py ctffind64.exe ctfRefine.py ctftilt64.exe cnsit@kraken:/usr/local/bin$
Maybe I need to link ctffind4 --> ctffind4.py?
RE: ctffind4.py gets confused about its location - Added by Patrick Goetz about 7 years ago
Anchi -
I think you mean this page: http://emg.nysbc.org/projects/appion/wiki/Processing_Server_Installation
(the one with s/appion/leginon/ is different. I've read all these pages a dozen times.
There is a typo in ctffind4 line in the table on that page. The left hand column needs to be ctffind4.py, not ctffind4 -- this is what confused me.
I still get a fatal error after executing
ln -s ctffind4.py ctffind4
but this time it might be another user error. Will check and close the ticket if it is.
RE: ctffind4.py gets confused about its location - Added by Sargis Dallakyan about 7 years ago
Patrick: `which ctffind4` needs to find ctffind4 not the .py file. I've updated http://emg.nysbc.org/projects/appion/wiki/Install_Grigorieff_lab_software page which was writtent for ctffind3.
RE: ctffind4.py gets confused about its location - Added by Patrick Goetz about 7 years ago
Sargis: yep, I figured that was the case, hence my response to Anchi. However, the Grigorieff Lab executable isn't called ctffind:
cnsit@kraken:/usr/local/bin$ ls -l ctf* -rwxr-xr-x 1 root root 18728 Nov 29 15:56 ctfestimate.py lrwxrwxrwx 1 root root 11 Dec 14 09:24 ctffind4 -> ctffind4.py -rwxr-xr-x 1 root root 463 Nov 29 15:56 ctffind4-pow_avrot_plot.py -rwxr-xr-x 1 root root 18775 Nov 29 15:56 ctffind4.py -rwxr-xr-x 1 root root 1478576 Jun 9 2014 ctffind64.exe -rwxr-xr-x 1 root root 33884 Nov 29 15:56 ctfRefine.py -rwxr-xr-x 1 root root 1516390 Jun 9 2014 ctftilt64.exe
(I created the cttffind4 link myself)
That's the source of confusion on this page: http://emg.nysbc.org/projects/appion/wiki/Processing_Server_Installation
The left hand entry in the table on that page for ctffind4 should be ctffind4.py. On the page you edited the note should read:
NOTE: Once installed, you must rename ctffind4.py to ctffind4 and ctftilt.exe to ctftilt64.exe or create symbolic links. Otherwise, you will receive an error that ctffind4 was not found during ctf estimation
RE: ctffind4.py gets confused about its location - Added by Anchi Cheng about 7 years ago
No, the left hand entry in the table is the name of the program package given by the distributor of the package.
ctffind4.py is the python script in Appion that wrap the program to run by producing the input parameters according to the database record of the images. What you link above will do more harm than good because when you do
which ctffind4.py
It should point to your appion script installation location.
appion executable column gives what Appion script will look for this name in "which" run from shell.
Therefore, you should, in /usr/local/bin
ln -s ctffind64.exe ctffind4 ln -s ctftilt64.exe ctftilt.exe
RE: ctffind4.py gets confused about its location - Added by Anchi Cheng about 7 years ago
Patrick,
Regarding the location of the mapping table, you were correct.
RE: ctffind4.py gets confused about its location - Added by Anchi Cheng about 7 years ago
I modified Package_executable_alias_name_in_Appion to have this explanation
RE: SOLVED: ctffind4.py gets confused about its location - Added by Patrick Goetz about 7 years ago
Anchi, No, the left hand entry in the table is the name of the program package given by the distributor of the package.
Exactly. The distributor of the package calls this program ctffind64.exe, not ctffind4, which is why that table is confusing. Also, for someone new to the system (like me), I didn't remember exactly which ctf* stuff came from Grigorieff Lab, and thought they supplied ctffind4.py as well. After I looked at the installation dates, I realized what was going on.
Anyway, creating the links above solved the problem. Thank you.
RE: SOLVED: ctffind4.py gets confused about its location - Added by Anchi Cheng about 7 years ago
Sorry, Patrick,
Package name is the name of the software the webpage where you download it call it. It the name your user will ask you to install. You will not find a user refering this software as ctffind64.exe
Your question is a reasonable one for a system admin who has not worked with em lab before, and it did make our documentation better.
Thank you.