SOLVED: Missing library for findem64.exe

Added by Patrick Goetz almost 8 years ago

I'm at a loss for how to proceed with findem64.exe. This program requires a shared library libg2c.so

cnsit@kraken:~$ findem64.exe
findem64.exe: error while loading shared libraries: libg2c.so.0: cannot open shared object file: No such file or directory

which hasn't shipped with Ubuntu for 8 years due to changes in gcc. I can't find it in the CentOS7/RHEL repos either. Nor can I find compat-gcc-34-g77, as described here:

http://emg.nysbc.org/projects/appion/wiki/Compile_FindEM

Normally, the solution would be to compile the program by hand on a sympathetic platform and link the library statically rather than as a .so/DLL, as indicated here:

http://emg.nysbc.org/projects/appion/wiki/Install_FindEM_from_source

however, I don't have a findem folder in myami-beta/appion at all.

Has this program been superseded by another tool? In any case it is still being called by templateCorrelator.py

evan@armadillo:/home/local/evan/17dec11_E3_E5$ templateCorrelator.py --thread_findem --nodes=2 --ppn=2 --mem=16 --walltime=3 --cput=3 --runname=tmplrun1 --rundir=/cryodata/appion/17dec14w/extract/tmplrun1 --projectid=1 --session=17dec14w --no-rejects --no-wait --no-commit --no-continue --mrclist=17dec14w_17dec11_E3_E5_0059 --pdiam=195 --lpval=15 --hpval=0 --medianval=2 --pixlimit=4 --binval=4 --planereg --minthresh=0.35 --maxthresh=5 --maxpeaks=1500 --peaktype=centerofmass --maxsize=1.0 --overlapmult=1.5 --template_list=1 --range_list=0,360,10

 ... Time stamp: 17dec18m06
 ... Function name: templateCorrelator
 ... Appion directory: /usr/local/lib/python2.7/dist-packages
 ... Processing hostname: armadillo
Usage: templateCorrelator.py --projectid=## --runname=<runname> --session=<session> --preset=<preset> --description='<text>' --commit [options]

templateCorrelator.py: error: no such option: --nodes
evan@armadillo:/home/local/evan/17dec11_E3_E5$ templateCorrelator.py --thread_findem --ppn=2 --mem=16 --walltime=3 --cput=3 --runname=tmplrun1 --rundir=/cryodata/appion/17dec14w/extract/tmplrun1 --projectid=1 --session=17dec14w --no-rejects --no-wait --no-commit --no-continue --mrclist=17dec14w_17dec11_E3_E5_0059 --pdiam=195 --lpval=15 --hpval=0 --medianval=2 --pixlimit=4 --binval=4 --planereg --minthresh=0.35 --maxthresh=5 --maxpeaks=1500 --peaktype=centerofmass --maxsize=1.0 --overlapmult=1.5 --template_list=1 --range_list=0,360,10

 ... Time stamp: 17dec18m07
 ... Function name: templateCorrelator
 ... Appion directory: /usr/local/lib/python2.7/dist-packages
 ... Processing hostname: armadillo
Usage: templateCorrelator.py --projectid=## --runname=<runname> --session=<session> --preset=<preset> --description='<text>' --commit [options]

templateCorrelator.py: error: no such option: --ppn
evan@armadillo:/home/local/evan/17dec11_E3_E5$ templateCorrelator.py --thread_findem --mem=16 --walltime=3 --cput=3 --runname=tmplrun1 --rundir=/cryodata/appion/17dec14w/extract/tmplrun1 --projectid=1 --session=17dec14w --no-rejects --no-wait --no-commit --no-continue --mrclist=17dec14w_17dec11_E3_E5_0059 --pdiam=195 --lpval=15 --hpval=0 --medianval=2 --pixlimit=4 --binval=4 --planereg --minthresh=0.35 --maxthresh=5 --maxpeaks=1500 --peaktype=centerofmass --maxsize=1.0 --overlapmult=1.5 --template_list=1 --range_list=0,360,10

 ... Time stamp: 17dec18m07
 ... Function name: templateCorrelator
 ... Appion directory: /usr/local/lib/python2.7/dist-packages
 ... Processing hostname: armadillo
Usage: templateCorrelator.py --projectid=## --runname=<runname> --session=<session> --preset=<preset> --description='<text>' --commit [options]

templateCorrelator.py: error: no such option: --mem
evan@armadillo:/home/local/evan/17dec11_E3_E5$ templateCorrelator.py --thread_findem --runname=tmplrun1 --rundir=/cryodata/appion/17dec14w/extract/tmplrun1 --projectid=1 --session=17dec14w --no-rejects --no-wait --no-commit --no-continue --mrclist=17dec14w_17dec11_E3_E5_0059 --pdiam=195 --lpval=15 --hpval=0 --medianval=2 --pixlimit=4 --binval=4 --planereg --minthresh=0.35 --maxthresh=5 --maxpeaks=1500 --peaktype=centerofmass --maxsize=1.0 --overlapmult=1.5 --template_list=1 --range_list=0,360,10

 ... Time stamp: 17dec18m08
 ... Function name: templateCorrelator
 ... Appion directory: /usr/local/lib/python2.7/dist-packages
 ... Processing hostname: armadillo
 ... Using split database
Connected to database: 'ap1'
!!! WARNING: Not committing data to database
 ... Found 16 processors on this machine
 ... Found project id=1 for session 17dec14w in 9.58 msec
 ... Run directory: /cryodata/appion/17dec14w/extract/tmplrun1
 ... Writing function log to: templateCorrelator.log
 ... Uploading ScriptData....
 ... Found 16 processors on this machine
 ... Running Appion version 'myami-beta'
 ... Creating new done dictionary: templateCorrelator.donedict
 ... Looking up session, 17dec14w
Querying database for 1 specific images ... 
 ... Found 1 images in 5.69 msec
 ... Remove processed images

 ... finished skipping in 9.37 msec
 ... Process images old to new
 ... getting templates
 ... getting template: /cryodata/appion/17dec14w/templates/template17dec18m02/PKS_Picking_Template-17dec18m02.mrc
 ... reading MRC: origTemplate1.mrc size:(96, 96) dtype:float32
 ... rescaling template 1: 3.6->2.05
 ... scaling template by a factor of 1.75609756098
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:59: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:70: RuntimeWarning: invalid value encountered in true_divide
  ret = ret.dtype.type(ret / rcount)
 ... changing box size from 169 to 160
 ... Median filter of size 2 pixels
 ... Binning image by a factor of 4
 ... Applying a 2D plane regression and subtraction
 ... plane_regress: x-slope: 0.399, y-slope: -0.845, xy-intercept: 0.520
 ... Applying a low pass filter of 15.0 A (apix 2.0A) of type gauss
 ... Applying pixel limit filter of ± 4.0 standard deviations
.. Pixel Limits -37.706 <> 37.706
 ... filtered image in 3.57 msec
 ... scaled & filtered 1 file(s)
Template list: ['filterTemplate1.mrc']
 ... Looking up session, 17dec14w

Beginning Main Loop
 ... Pixel size: 2.05
 ... Median filter of size 2 pixels
 ... Binning image by a factor of 4
 ... Applying a 2D plane regression and subtraction
 ... plane_regress: x-slope: -0.169, y-slope: 0.304, xy-intercept: 0.181
 ... Applying a low pass filter of 15.0 A (apix 2.0A) of type gauss
 ... Applying pixel limit filter of ± 4.0 standard deviations
.. Pixel Limits -1.620 <> 1.620
 ... .. replacing 523 of 1048576 pixels with pixel limit
 ... filtered image in 4.8 sec
 ... writing MRC: 17dec14w_17dec11_E3_E5_0059.dwn.mrc size:(1024, 1024) dtype:float64
 ... image file, 17dec14w_17dec11_E3_E5_0059.dwn.mrc
 ... template file, filterTemplate1.mrc
 ... threading findem64.exe (1 of 1)
 ... 
FindEM finished in 157.16 msec

Traceback (most recent call last):
  File "/usr/local/bin/templateCorrelator.py", line 8, in <module>
    imgLoop.run()
  File "/usr/local/lib/python2.7/dist-packages/appionlib/appionLoop2.py", line 89, in run
    results = self.loopProcessImage(imgdata)
  File "/usr/local/lib/python2.7/dist-packages/appionlib/particleLoop2.py", line 101, in loopProcessImage
    self.peaktree = filterLoop.FilterLoop.loopProcessImage(self, imgdata)
  File "/usr/local/lib/python2.7/dist-packages/appionlib/filterLoop.py", line 83, in loopProcessImage
    peaktree = self.processImage(imgdata, self.filtarray)
  File "/usr/local/lib/python2.7/dist-packages/appionlib/apTemplateCorrelator.py", line 196, in processImage
    cclist = self.runTemplateCorrelator(imgdata)
  File "/usr/local/lib/python2.7/dist-packages/appionlib/apTemplateCorrelator.py", line 148, in runTemplateCorrelator
    ccmaplist = apFindEM.runFindEM(imgdata, self.params, thread=self.params['threadfindem'])
  File "/usr/local/lib/python2.7/dist-packages/appionlib/apFindEM.py", line 158, in runFindEM
    apDisplay.printError("findem.exe did not run or crashed.\n")
  File "/usr/local/lib/python2.7/dist-packages/appionlib/apDisplay.py", line 65, in printError
    raise Exception, colorString("\n *** FATAL ERROR ***\n"+text+"\n\a","red")
Exception: 
 *** FATAL ERROR ***
findem.exe did not run or crashed.

Replies (5)

RE: Missing library for findem64.exe - Added by Sargis Dallakyan almost 8 years ago

Thanks for the message. I've update the wiki to include a link to FindEM source at http://emg.nysbc.org/projects/software/wiki/FindEM

I haven't compiled FindEM from source for years. I can make a Singularity container for FindEM if it's hard to build it from source.

RE: Missing library for findem64.exe - Added by Patrick Goetz almost 8 years ago

Singularity containers is one of the things I've been tasked with looking at after this project wraps up. I will try and compile the code, but if it comes this boxing the app, how would findem64.exe interact with the rest of the Appion suite if it's in a container?

RE: Missing library for findem64.exe - Added by Sargis Dallakyan almost 8 years ago

I have been experimenting with Singularity containers and I find it more suitable for HPC than Docker. Appion is calling findem64.exe through system calls (subprocess.Popen). As long as you have all your in/out files in your home folder, there won't be any difference if findem64.exe is an executable or a Singularity container. If in/out file are in a shared folder, then Singularity container needs to be modified to mount that folder and it will act the same as findem64.exe executable.

RE: Missing library for findem64.exe - Added by Patrick Goetz almost 8 years ago

I solved this problem by creating CentOS 6 virtual machine, installing the compat-gcc-34-g77 package, and then harvesting libg2c.so.0.0.0 from this package.
I also had to harvest libgfortran.so.1.0.0 from the EMAN2 distribution. Then

# mv libg2c.so.0.0.0 /usr/local/lib
# mv libgfortran.so.1.0.0 /usr/local/lib
# cd /usr/local/lib
# ln -s libg2c.so.0.0.0 libg2c.so.0
# ln -s libgfortran.so.1.0.0 libgfortran.so.1

That's it, although users might need to set LD_LIBRARY_PATH:

export LD_LIBRARY_PATH=/usr/local/lib

Because linux maintains ABI compatibility, and because this program is strictly computational, this works on Ubuntu 16.04.

RE: SOLVED: Missing library for findem64.exe - Added by Anchi Cheng almost 8 years ago

Thanks, Patrick

(1-5/5)

Project

General

Profile

Leginon