Actions
Bug #3438
closednoise subtraction in ctfdisplay normalization error
Added by Anchi Cheng about 9 years ago. Updated about 8 years ago.
Start date:
08/26/2015
Due date:
% Done:
0%
Estimated time:
Affected Version:
Appion/Leginon 3.2
Show in known bugs:
No
Workaround:
Description
Running on the same image attached #3436 (200 kV, Cs=2 mm, apix=1.47 A)
I tried these parameters as well as the faster --numstep=50. Got the same problem below.
ctfestimate.py --runname=ctffindrun6 --rundir=/emg/data00/appion/acheng/15aug25g/ctf/ctffindrun6 --limit=1 --commit --preset=en-a --projectid=5 --session=15aug25g --no-rejects --no-wait --continue --ampcarbon=0.15 --ampice=0.07 --fieldsize=512 --medium=carbon --bin=2 --resmin=30 --resmax=4 --defstep=1000 --numstep=50 --dast=500 --expid=605 --jobtype=ctfestimate
Final params: def1: 9.95e-07 | def2: 1.01e-06 | angle: 15.7 | defratio 1.01
... Defocus Astig Percent Diff 0.58 -- 9.948e-07, 1.007e-06
Computing power spectra in 1024x1024 blocks...................................
... Computing median of power spectra series
... Compute PSD with fieldsize 1024 and 35 images complete in 9.4 sec
... Preform a rotational average and remove spikes...
PART 1: SETUP PARAMETERS AND ELLIPTICAL AVERAGE
... Number of available peaks is 100
... Determine and subtract noise model
PART 2: BACKGROUND NOISE SUBTRACTION
... starting peak extension
Peak Extension Complete in 0.76 msec
Traceback (most recent call last):
File "/home/acheng/myami/appion/bin/ctfestimate.py", line 449, in <module>
imgLoop.run()
File "/home/acheng/myami/appion/appionlib/appionLoop2.py", line 93, in run
self.loopCommitToDatabase(imgdata)
File "/home/acheng/myami/appion/appionlib/appionLoop2.py", line 143, in loopCommitToDatabase
return self.commitToDatabase(imgdata)
File "/home/acheng/myami/appion/bin/ctfestimate.py", line 349, in commitToDatabase
ctfinsert.validateAndInsertCTFData(imgdata, self.ctfvalues, self.ctfrun, self.params['rundir'])
File "/home/acheng/myami/appion/appionlib/apCtf/ctfinsert.py", line 45, in validateAndInsertCTFData
ctfvalues = runCTFdisplayTools(imgdata, ctfvalues, opimagedir, fftpath, fftfreq)
File "/home/acheng/myami/appion/appionlib/apCtf/ctfinsert.py", line 83, in runCTFdisplayTools
ctfdisplaydict = ctfdisplay.makeCtfImages(imgdata, ctfvalues, fftpath, fftfreq)
File "/home/acheng/myami/appion/appionlib/apCtf/ctfdisplay.py", line 1267, in makeCtfImages
ctfdisplaydict = a.CTFpowerspec(imgdata, ctfdata, fftpath, fftfreq, twod=twod)
File "/home/acheng/myami/appion/appionlib/apCtf/ctfdisplay.py", line 1149, in CTFpowerspec
normpowerspec = self.normalizeCtf(powerspec, twod=twod)
File "/home/acheng/myami/appion/appionlib/apCtf/ctfdisplay.py", line 244, in normalizeCtf
scale /= scale.max()
ValueError: zero-size array to ufunc.reduce without identity
ctf estimation is used all the time. If the new change in the background fitting is causing this much trouble, should we revert and make it more as an option for now ?
Files
| beforeUpdate.png (413 KB) beforeUpdate.png | Gabriel Lander, 08/31/2015 06:13 PM | ||
| afterUpdate.png (573 KB) afterUpdate.png | Gabriel Lander, 08/31/2015 06:13 PM |
Updated by Neil Voss about 9 years ago
- Status changed from Assigned to In Test
- Assignee changed from Neil Voss to Anchi Cheng
Hi Anchi, I do not have time to work on this right now, but I think I fixed it in r19107.
Updated by Anchi Cheng about 9 years ago
- Assignee changed from Anchi Cheng to Gabriel Lander
The fix works at our end.
Gabe ?
Actions
#3
Updated by Gabriel Lander about 9 years ago
- File afterUpdate.png afterUpdate.png added
- File beforeUpdate.png beforeUpdate.png added
The 2D image no longer appear to be binned by two, and the confidences that show up are much worse than previously. I'm attaching two images showing the result of the CTF results run on the same images, before & after the update.
Updated by Anchi Cheng about 9 years ago
In order to let nysbc users keep going, I created branch 3.2 with ctfdisplay.py reverted to r19012. This way Neil can debug when he has time.
Updated by Neil Voss about 9 years ago
Made some tweeks to trunk ctf to help with close to focus images. Needs testing. I will try to visit in free time, but very limited right now.
Updated by Neil Voss about 9 years ago
- Related to Bug #3494: difficult image for new ctfdisplay.py to do noise subtraction added
Updated by Neil Voss about 9 years ago
- Related to Bug #3495: CTF matplotlib font path Error added
Updated by Gabriel Lander almost 9 years ago
Still not working (running r19192):
====================================
Starting image 1 ( skip:9, remain:131 ) id:3771558, file: 15sep24b_03sq_02hl_011en
... Pixel size: 2.05044
!!! WARNING: no best CTF value for image 15sep24b_03sq_02hl_011en
... Xmag=76081, dstep=1.56e-05, mpix=2.05e-10
!!! WARNING: Defocus minimum is less than zero
Defocus search range: 1000 A to 32000 A (0.10 to 3.20 um)
... running ctf estimation at Thu Sep 24 10:54:31 2015
/gpfs/home/glander/myami/appion/bin/ctffind64.exe
15sep24b_03sq_02hl_011en.mrc
15sep24b_03sq_02hl_011en-pow.mrc
2.2,120.0,0.15,76081.2313455,15.6,2
512,40.0,8.0,1000.0,32000.0,1000.0,500.0
... ctf estimation completed in 61.8 sec
... nominal=7.0e-07, bestdef=7.0e-07,
... def_1=7.5e-07, def_2=8.1e-07, astig_angle=-14.4, cross_corr=0.413,
... reading MRC: 15sep24b_03sq_02hl_011en-pow.mrc size:(512, 512) dtype:float32
... writing JPEG: 15sep24b_03sq_02hl_011en-pow.jpg
==== Committing data to database ====
... Committing ctf parameters for 15sep24b_03sq_02hl_011en to database
... Reading image...
[CTF param] def1: 7.55e-07 | def2: 8.09e-07 | angle: -14.4 | ampcontr 0.15 | defratio 1.072
Final params: def1: 7.55e-07 | def2: 8.09e-07 | angle: -14.4 | defratio 1.07
... Defocus Astig Percent Diff 3.46 -- 7.546e-07, 8.086e-07
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/n
umeric.py:1846: DeprecationWarning: using a non-integer number instead of an integer will result in
an error in the future
res = empty((N,)+dimensions, dtype=dtype)
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/n
umeric.py:1849: DeprecationWarning: using a non-integer number instead of an integer will result in
an error in the future
tmp.shape = (1,)*i + (dim,)+(1,)*(N-i-1)
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/n
umeric.py:1851: DeprecationWarning: using a non-integer number instead of an integer will result in
an error in the future
val = zeros(newdim, dtype)
Computing power spectra in 4096x4096 blocks./gpfs/home/glander/myami/appion/appionlib/apCtf/ctfpower
.py:101: DeprecationWarning: using a non-integer number instead of an integer will result in an erro
r in the future
cutout = image[x1:x2, y1:y2]
... Computing median of power spectra series
... Compute PSD with fieldsize 4096 and 1 images complete in 6.45 sec
!!! WARNING: Requested resolution (3.000) is not available (4.101)
... Preform a rotational average and remove spikes...
PART 1: SETUP PARAMETERS AND ELLIPTICAL AVERAGE
... Number of available peaks is 41
/gpfs/home/glander/myami/appion/appionlib/apCtf/ctftools.py:490: DeprecationWarning: using a non-int
eger number instead of an integer will result in an error in the future
xdataint = xdataint[:outercutsize]
... Determine and subtract noise model
PART 2: BACKGROUND NOISE SUBTRACTION
... starting peak extension
Peak Extension Complete in 0.73 msec
['/gpfs/home/glander/myami/appion/appionlib', '/gpfs/home/glander/myami/appion']
Found 0 ctf values
Traceback (most recent call last):
File "/gpfs/home/glander/myami/appion/bin/ctfestimate.py", line 451, in <module>
imgLoop.run()
File "/gpfs/home/glander/myami/appion/appionlib/appionLoop2.py", line 93, in run
self.loopCommitToDatabase(imgdata)
File "/gpfs/home/glander/myami/appion/appionlib/appionLoop2.py", line 143, in loopCommitToDatabase
return self.commitToDatabase(imgdata)
File "/gpfs/home/glander/myami/appion/bin/ctfestimate.py", line 351, in commitToDatabase
ctfinsert.validateAndInsertCTFData(imgdata, self.ctfvalues, self.ctfrun, self.params['rundir'])
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfinsert.py", line 45, in validateAndInsert
CTFData
ctfvalues = runCTFdisplayTools(imgdata, ctfvalues, opimagedir, fftpath, fftfreq)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfinsert.py", line 83, in runCTFdisplayTool
s
ctfdisplaydict = ctfdisplay.makeCtfImages(imgdata, ctfvalues, fftpath, fftfreq)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 1266, in makeCtfImages
ctfdisplaydict = a.CTFpowerspec(imgdata, ctfdata, fftpath, fftfreq, twod=twod)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 1148, in CTFpowerspec
normpowerspec = self.normalizeCtf(powerspec, twod=twod)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 243, in normalizeCtf
scale /= scale.max()
File "/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/nump
y/core/_methods.py", line 26, in _amax
return umr_maximum(a, axis, None, out, keepdims)
ValueError: zero-size array to reduction operation maximum which has no identity
Updated by Neil Voss almost 9 years ago
Hi Gabe, I added some debugging info to help diagnose the problem, because it ended up with no overlap region.
Updated by Gabriel Lander almost 9 years ago
Here's the output with the debug (also had to comment out an if statement for this to work - r19202
===============================================
Starting image 1 ( skip:34, remain:406 ) id:3771558, file: 15sep24b_03sq_02hl_011en
... Pixel size: 2.05044
Found 1 ctf values
0.167 -- ctffindrun1
*** 0.167
[CTF run] method: ctffind | runname ctffindrun1
[CTF param] def1: 7.53e-07 | def2: 8.07e-07 | angle: -16.4 | ampcontr 0.15 | defratio 1.071
[CTF stats] conf_30-10: 0.986 | conf_5peak: 0.973 | res_0.8: 6.0A | res_0.5 5.6A
... Xmag=76081, dstep=1.56e-05, mpix=2.05e-10
!!! WARNING: Defocus minimum is less than zero
Defocus search range: 1000 A to 32530 A (0.10 to 3.25 um)
... running ctf estimation at Fri Sep 25 13:44:40 2015
/gpfs/home/glander/myami/appion/bin/ctffind64.exe
15sep24b_03sq_02hl_011en.mrc
15sep24b_03sq_02hl_011en-pow.mrc
2.2,120.0,0.15,76081.2313455,15.6,2
512,40.0,8.0,1000.0,32530.3,1000.0,500.0
... ctf estimation completed in 60.55 sec
... nominal=7.0e-07, bestdef=7.5e-07,
... def_1=7.5e-07, def_2=8.1e-07, astig_angle=-14.4, cross_corr=0.413,
... reading MRC: 15sep24b_03sq_02hl_011en-pow.mrc size:(512, 512) dtype:float32
... writing JPEG: 15sep24b_03sq_02hl_011en-pow.jpg
==== Committing data to database ====
... Committing ctf parameters for 15sep24b_03sq_02hl_011en to database
15sep24b_03sq_02hl_011en
Pixelsize (A/pix) 2.05044
... Reading image...
[CTF param] def1: 7.55e-07 | def2: 8.09e-07 | angle: -14.4 | ampcontr 0.15 | defratio 1.072
Final params: def1: 7.55e-07 | def2: 8.09e-07 | angle: -14.4 | defratio 1.07
... Defocus Astig Percent Diff 3.46 -- 7.546e-07, 8.086e-07
volts -- 120000
amplitude_contrast -- 0.15
cross_correlation -- 0.41284
angle_astigmatism -- -14.37
cs -- 2.2
confidence_d -- 0.64253
confidence -- 0.41284
graph1 -- /gpfs/group/em/appion/glander/15sep24b/ctf/ctffindruntest/opimages/15sep24b_03sq_02hl_011en-pow.jpg
defocusinit -- 7.5303e-07
nominal -- 7e-07
defocus1 -- 7.54591e-07
defocus2 -- 8.08636e-07
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/numeric.py:1846: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
res = empty((N,)+dimensions, dtype=dtype)
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/numeric.py:1849: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
tmp.shape = (1,)*i + (dim,)+(1,)*(N-i-1)
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/numeric.py:1851: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
val = zeros(newdim, dtype)
Computing power spectra in 4096x4096 blocks.
... Computing median of power spectra series
... Compute PSD with fieldsize 4096 and 1 images complete in 6.26 sec
!!! WARNING: Requested resolution (3.000) is not available (4.101)
... Preform a rotational average and remove spikes...
orig pixel 2.050 freq 1.191e-04
trim pixel 2.050 freq 1.191e-04
PART 1: SETUP PARAMETERS AND ELLIPTICAL AVERAGE
... Number of available peaks is 41
Elliptical CTF limits 24.6 A -->> 4.2A
... Determine and subtract noise model
PART 2: BACKGROUND NOISE SUBTRACTION
extremaToIndex Debug
... requestVal = 508.09663437
... extremaIndex = 0
... extremaList[extremaIndex-1] = 4471.40382371
... extremaVal = extremaList[extremaIndex] = 508.09663437
... extremaList[extremaIndex+1] = 729.80361294
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.0605
... index = 85
... raddata[index-1] = 0.0604
... raddata[index] = 0.0606
... raddata[index+1] = 0.0608
extremaToIndex Debug
... requestVal = 508.09663437
... extremaIndex = 0
... extremaList[extremaIndex-1] = 4471.40382371
... extremaVal = extremaList[extremaIndex] = 508.09663437
... extremaList[extremaIndex+1] = 729.80361294
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.0605
... index = 85
... raddata[index-1] = 0.0604
... raddata[index] = 0.0606
... raddata[index+1] = 0.0608
extremaToIndex Debug
... requestVal = 780.95377723
... extremaIndex = 1
... extremaList[extremaIndex-1] = 508.09663437
... extremaVal = extremaList[extremaIndex] = 729.80361294
... extremaList[extremaIndex+1] = 900.31406112
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.0869
... index = 197
... raddata[index-1] = 0.0866
... raddata[index] = 0.0868
... raddata[index+1] = 0.0870
extremaToIndex Debug
... requestVal = 671.81092008
... extremaIndex = 1
... extremaList[extremaIndex-1] = 508.09663437
... extremaVal = extremaList[extremaIndex] = 729.80361294
... extremaList[extremaIndex+1] = 900.31406112
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.0869
... index = 197
... raddata[index-1] = 0.0866
... raddata[index] = 0.0868
... raddata[index+1] = 0.0870
extremaToIndex Debug
... requestVal = 944.66806294
... extremaIndex = 2
... extremaList[extremaIndex-1] = 729.80361294
... extremaVal = extremaList[extremaIndex] = 900.31406112
... extremaList[extremaIndex+1] = 1045.06266452
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1072
... index = 284
... raddata[index-1] = 0.1069
... raddata[index] = 0.1072
... raddata[index+1] = 0.1074
extremaToIndex Debug
... requestVal = 835.52520580
... extremaIndex = 2
... extremaList[extremaIndex-1] = 729.80361294
... extremaVal = extremaList[extremaIndex] = 900.31406112
... extremaList[extremaIndex+1] = 1045.06266452
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1072
... index = 284
... raddata[index-1] = 0.1069
... raddata[index] = 0.1072
... raddata[index+1] = 0.1074
extremaToIndex Debug
... requestVal = 1108.38234865
... extremaIndex = 3
... extremaList[extremaIndex-1] = 900.31406112
... extremaVal = extremaList[extremaIndex] = 1045.06266452
... extremaList[extremaIndex+1] = 1173.68298368
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1244
... index = 358
... raddata[index-1] = 0.1243
... raddata[index] = 0.1245
... raddata[index+1] = 0.1247
extremaToIndex Debug
... requestVal = 999.23949151
... extremaIndex = 3
... extremaList[extremaIndex-1] = 900.31406112
... extremaVal = extremaList[extremaIndex] = 1045.06266452
... extremaList[extremaIndex+1] = 1173.68298368
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1244
... index = 358
... raddata[index-1] = 0.1243
... raddata[index] = 0.1245
... raddata[index+1] = 0.1247
extremaToIndex Debug
... requestVal = 1272.09663437
... extremaIndex = 5
... extremaList[extremaIndex-1] = 1173.68298368
... extremaVal = extremaList[extremaIndex] = 1291.06529892
... extremaList[extremaIndex+1] = 1400.09913043
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1537
... index = 483
... raddata[index-1] = 0.1535
... raddata[index] = 0.1537
... raddata[index+1] = 0.1540
[85, 197, 284, 358]
[197, 284, 358, 483]
Noise mergeIndexes [85, 197, 197, 284, 284, 358, 358, 483]
... starting peak extension
Peak Extension Complete in 0.81 msec
... fitting noise section 1 of 4
... finished in 115.84 msec
... fitting noise section 2 of 4
... finished in 113.82 msec
... fitting noise section 3 of 4
... finished in 112.11 msec
... fitting noise section 4 of 4
... finished in 112.32 msec
section 0 mergedata (85,)
insert from 85 to 197 (112)
overlap from 197 to 197 (0)
Traceback (most recent call last):
File "/gpfs/home/glander/myami/appion/bin/ctfestimate.py", line 451, in <module>
imgLoop.run()
File "/gpfs/home/glander/myami/appion/appionlib/appionLoop2.py", line 93, in run
self.loopCommitToDatabase(imgdata)
File "/gpfs/home/glander/myami/appion/appionlib/appionLoop2.py", line 143, in loopCommitToDatabase
return self.commitToDatabase(imgdata)
File "/gpfs/home/glander/myami/appion/bin/ctfestimate.py", line 351, in commitToDatabase
ctfinsert.validateAndInsertCTFData(imgdata, self.ctfvalues, self.ctfrun, self.params['rundir'])
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfinsert.py", line 45, in validateAndInsertCTFData
ctfvalues = runCTFdisplayTools(imgdata, ctfvalues, opimagedir, fftpath, fftfreq)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfinsert.py", line 83, in runCTFdisplayTools
ctfdisplaydict = ctfdisplay.makeCtfImages(imgdata, ctfvalues, fftpath, fftfreq)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 1264, in makeCtfImages
ctfdisplaydict = a.CTFpowerspec(imgdata, ctfdata, fftpath, fftfreq, twod=twod)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 1146, in CTFpowerspec
normpowerspec = self.normalizeCtf(powerspec, twod=twod)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 245, in normalizeCtf
scale /= scale.max()
File "/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/_methods.py", line 26, in _amax
return umr_maximum(a, axis, None, out, keepdims)
ValueError: zero-size array to reduction operation maximum which has no identity
Updated by Neil Voss almost 9 years ago
Can you include the output again, I am trying to learn something.
Updated by Gabriel Lander almost 9 years ago
... Time stamp: 15sep25o17
... Function name: ctfestimate
... Appion directory: /gpfs/home/glander/myami/appion
!!! WARNING: There is more than one appion directory in your PYTHONPATH
['/gpfs/home/glander/myami/appion/appionlib', '/gpfs/home/glander/myami/appion']
... Using split database
Connected to database: 'ap472'
... Committing data to database
... Looking up session, 15sep24b
... Found project id=472 for session 15sep24b in 5.72 msec
... Run directory: /gpfs/group/em/appion/glander/15sep24b/ctf/ctffindruntest
!!! WARNING: directory '/gpfs/group/em/appion/glander/15sep24b/ctf/ctffindruntest' already exists.
... Writing function log to: ctfestimate.log
... Uploading ScriptData....
... Found 1 processors on this machine
... Running Appion version 'trunk'
... Reading old done dictionary: ctfestimate.donedict
... Found 35 done dictionary entries
... Querying database for preset 'en' images from session '15sep24b' ...
... Found 441 images in 27.66 sec
... Remove processed images
.skipping processed images
............................................................................
... finished skipping in 4.74 sec
!!! WARNING: skipped 34 of 441 images
... [[ 0 no reprocess | 0 rejected | 0 wrong tilt | 0 in donedict ]]
... Process images old to new
... Running program ctffind64.exe
Beginning Main Loop
Starting image 1 ( skip:34, remain:406 ) id:3771558, file: 15sep24b_03sq_02hl_011en
... Pixel size: 2.05044
Found 1 ctf values
0.167 -- ctffindrun1
*** 0.167
[CTF run] method: ctffind | runname ctffindrun1
[CTF param] def1: 7.53e-07 | def2: 8.07e-07 | angle: -16.4 | ampcontr 0.15 | defratio 1.071
[CTF stats] conf_30-10: 0.986 | conf_5peak: 0.973 | res_0.8: 6.0A | res_0.5 5.6A
... Xmag=76081, dstep=1.56e-05, mpix=2.05e-10
!!! WARNING: Defocus minimum is less than zero
Defocus search range: 1000 A to 32530 A (0.10 to 3.25 um)
... running ctf estimation at Fri Sep 25 14:17:38 2015
/gpfs/home/glander/myami/appion/bin/ctffind64.exe
15sep24b_03sq_02hl_011en.mrc
15sep24b_03sq_02hl_011en-pow.mrc
2.2,120.0,0.15,76081.2313455,15.6,2
512,40.0,8.0,1000.0,32530.3,1000.0,500.0
... ctf estimation completed in 60.39 sec
... nominal=7.0e-07, bestdef=7.5e-07,
... def_1=7.5e-07, def_2=8.1e-07, astig_angle=-14.4, cross_corr=0.413,
... reading MRC: 15sep24b_03sq_02hl_011en-pow.mrc size:(512, 512) dtype:float32
... writing JPEG: 15sep24b_03sq_02hl_011en-pow.jpg
==== Committing data to database ====
... Committing ctf parameters for 15sep24b_03sq_02hl_011en to database
15sep24b_03sq_02hl_011en
Pixelsize (A/pix) 2.05044
... Reading image...
[CTF param] def1: 7.55e-07 | def2: 8.09e-07 | angle: -14.4 | ampcontr 0.15 | defratio 1.072
Final params: def1: 7.55e-07 | def2: 8.09e-07 | angle: -14.4 | defratio 1.07
... Defocus Astig Percent Diff 3.46 -- 7.546e-07, 8.086e-07
volts -- 120000
amplitude_contrast -- 0.15
cross_correlation -- 0.41284
angle_astigmatism -- -14.37
cs -- 2.2
confidence_d -- 0.64253
confidence -- 0.41284
graph1 -- /gpfs/group/em/appion/glander/15sep24b/ctf/ctffindruntest/opimages/15sep24b_03sq_02hl_011en-pow.jpg
defocusinit -- 7.5303e-07
nominal -- 7e-07
defocus1 -- 7.54591e-07
defocus2 -- 8.08636e-07
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/numeric.py:1846: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
res = empty((N,)+dimensions, dtype=dtype)
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/numeric.py:1849: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
tmp.shape = (1,)*i + (dim,)+(1,)*(N-i-1)
/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/numeric.py:1851: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
val = zeros(newdim, dtype)
Computing power spectra in 4096x4096 blocks.
... Computing median of power spectra series
... Compute PSD with fieldsize 4096 and 1 images complete in 6.2 sec
!!! WARNING: Requested resolution (3.000) is not available (4.101)
... Preform a rotational average and remove spikes...
orig pixel 2.050 freq 1.191e-04
trim pixel 2.050 freq 1.191e-04
PART 1: SETUP PARAMETERS AND ELLIPTICAL AVERAGE
... Number of available peaks is 41
... setting the number of sections to default: 4
Elliptical CTF limits 24.6 A -->> 4.2A
... Determine and subtract noise model
PART 2: BACKGROUND NOISE SUBTRACTION
extremaToIndex Debug
... requestVal = 508.09663437
... extremaIndex = 0
... extremaList[extremaIndex-1] = 4471.40382371
... extremaVal = extremaList[extremaIndex] = 508.09663437
... extremaList[extremaIndex+1] = 729.80361294
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.0605
... index = 85
... raddata[index-1] = 0.0604
... raddata[index] = 0.0606
... raddata[index+1] = 0.0608
extremaToIndex Debug
... requestVal = 508.09663437
... extremaIndex = 0
... extremaList[extremaIndex-1] = 4471.40382371
... extremaVal = extremaList[extremaIndex] = 508.09663437
... extremaList[extremaIndex+1] = 729.80361294
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.0605
... index = 85
... raddata[index-1] = 0.0604
... raddata[index] = 0.0606
... raddata[index+1] = 0.0608
extremaToIndex Debug
... requestVal = 780.95377723
... extremaIndex = 1
... extremaList[extremaIndex-1] = 508.09663437
... extremaVal = extremaList[extremaIndex] = 729.80361294
... extremaList[extremaIndex+1] = 900.31406112
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.0869
... index = 197
... raddata[index-1] = 0.0866
... raddata[index] = 0.0868
... raddata[index+1] = 0.0870
extremaToIndex Debug
... requestVal = 671.81092008
... extremaIndex = 1
... extremaList[extremaIndex-1] = 508.09663437
... extremaVal = extremaList[extremaIndex] = 729.80361294
... extremaList[extremaIndex+1] = 900.31406112
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.0869
... index = 197
... raddata[index-1] = 0.0866
... raddata[index] = 0.0868
... raddata[index+1] = 0.0870
extremaToIndex Debug
... requestVal = 944.66806294
... extremaIndex = 2
... extremaList[extremaIndex-1] = 729.80361294
... extremaVal = extremaList[extremaIndex] = 900.31406112
... extremaList[extremaIndex+1] = 1045.06266452
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1072
... index = 284
... raddata[index-1] = 0.1069
... raddata[index] = 0.1072
... raddata[index+1] = 0.1074
extremaToIndex Debug
... requestVal = 835.52520580
... extremaIndex = 2
... extremaList[extremaIndex-1] = 729.80361294
... extremaVal = extremaList[extremaIndex] = 900.31406112
... extremaList[extremaIndex+1] = 1045.06266452
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1072
... index = 284
... raddata[index-1] = 0.1069
... raddata[index] = 0.1072
... raddata[index+1] = 0.1074
extremaToIndex Debug
... requestVal = 1108.38234865
... extremaIndex = 3
... extremaList[extremaIndex-1] = 900.31406112
... extremaVal = extremaList[extremaIndex] = 1045.06266452
... extremaList[extremaIndex+1] = 1173.68298368
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1244
... index = 358
... raddata[index-1] = 0.1243
... raddata[index] = 0.1245
... raddata[index+1] = 0.1247
extremaToIndex Debug
... requestVal = 999.23949151
... extremaIndex = 3
... extremaList[extremaIndex-1] = 900.31406112
... extremaVal = extremaList[extremaIndex] = 1045.06266452
... extremaList[extremaIndex+1] = 1173.68298368
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1244
... index = 358
... raddata[index-1] = 0.1243
... raddata[index] = 0.1245
... raddata[index+1] = 0.1247
extremaToIndex Debug
... requestVal = 1272.09663437
... extremaIndex = 5
... extremaList[extremaIndex-1] = 1173.68298368
... extremaVal = extremaList[extremaIndex] = 1291.06529892
... extremaList[extremaIndex+1] = 1400.09913043
... trimFreq = 0.00011907
... extremaVal*trimFreq = 0.1537
... index = 483
... raddata[index-1] = 0.1535
... raddata[index] = 0.1537
... raddata[index+1] = 0.1540
[85, 197, 284, 358]
[197, 284, 358, 483]
Noise mergeIndexes [85, 197, 197, 284, 284, 358, 358, 483]
... starting peak extension
Peak Extension Complete in 0.83 msec
... fitting noise section 1 of 4
... finished in 116.54 msec
... fitting noise section 2 of 4
... finished in 115.2 msec
... fitting noise section 3 of 4
... finished in 113.08 msec
... fitting noise section 4 of 4
... finished in 114.06 msec
section 0 mergedata (85,)
insert from 85 to 197 (112)
overlap from 197 to 197 (0)
Traceback (most recent call last):
File "/gpfs/home/glander/myami/appion/bin/ctfestimate.py", line 451, in <module>
imgLoop.run()
File "/gpfs/home/glander/myami/appion/appionlib/appionLoop2.py", line 93, in run
self.loopCommitToDatabase(imgdata)
File "/gpfs/home/glander/myami/appion/appionlib/appionLoop2.py", line 143, in loopCommitToDatabase
return self.commitToDatabase(imgdata)
File "/gpfs/home/glander/myami/appion/bin/ctfestimate.py", line 351, in commitToDatabase
ctfinsert.validateAndInsertCTFData(imgdata, self.ctfvalues, self.ctfrun, self.params['rundir'])
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfinsert.py", line 45, in validateAndInsertCTFData
ctfvalues = runCTFdisplayTools(imgdata, ctfvalues, opimagedir, fftpath, fftfreq)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfinsert.py", line 83, in runCTFdisplayTools
ctfdisplaydict = ctfdisplay.makeCtfImages(imgdata, ctfvalues, fftpath, fftfreq)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 1277, in makeCtfImages
ctfdisplaydict = a.CTFpowerspec(imgdata, ctfdata, fftpath, fftfreq, twod=twod)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 1159, in CTFpowerspec
normpowerspec = self.normalizeCtf(powerspec, twod=twod)
File "/gpfs/home/glander/myami/appion/appionlib/apCtf/ctfdisplay.py", line 258, in normalizeCtf
scale /= scale.max()
File "/opt/applications/python/lib/python2.6/site-packages/numpy-1.9.1-py2.6-linux-x86_64.egg/numpy/core/_methods.py", line 26, in _amax
return umr_maximum(a, axis, None, out, keepdims)
ValueError: zero-size array to reduction operation maximum which has no identity
Updated by Neil Voss almost 9 years ago
hmm did not print what I wanted, I updated to see if I can get it to print. So, post again if you want otherwise, I have a test case back at lab I can test on Monday.
Updated by Gabriel Lander almost 9 years ago
any updates? we have 38 users here using myami/3.0 for ctf estimation now.
Updated by Neil Voss over 8 years ago
- Related to Feature #3950: Create CTF test dataset within Docker added
Updated by Gabriel Lander about 8 years ago
- Status changed from In Test to Closed
Actions