Bug #2552
openprime number decomposition for CTF estimation
0%
Description
I've recently noticed that my CTF estimation runs off of the K2 data run very slow (on the order of 2-20 minutes per image!). This occurs because ACE2 takes extremely long to generate an FFT (although CTFFind3 seems to have less problems). The prime number decomposition for the images off of Tecnai I (boxsize 3712) is [2, 2, 2, 2, 2, 2, 2, 29]. The 29 seems to cause problems. I would like to put in some code to check for prime numbers, and then clip the image prior to CTF estimation. I think we can just use the function getPrevEvenPrime() in pyami to clip. The question is whether to make this universal, for all CTF estimation algorithms, or just ACE2- (and I guess phasor???)-specific. Thoughts?
Updated by Gabriel Lander about 11 years ago
I've noticed the same extremely sluggish processing of K2 data with ACE2 (~5 minutes per image). I think Anchi already has the code to pad out K2 images for display/binning purposes, it should be pretty straightforward to incorporate this into ACE2, as well as all the post-analysis (noise subtraction, peak extension, etc). We shouldn't do any padding/clipping for CTFfind since it seems to handle these irregular dimensions without a problem (~20 seconds per image), likely since it was developed to work with rectangular film data. Additionally I'm concerned that padding or clipping will affect the code we have in appion to deal with location-specific CTF correction for tilted images.
Updated by Anchi Cheng about 11 years ago
The good news is that you only need to add 1 pixel on each side to get a good number.
See #2505 for what I did.
Updated by Neil Voss about 11 years ago
It is amazing how the prime number come into this. I wrote some code for ACE2 to do subfield averaging to work better with rectangular images, but I have not had time to test it.
Updated by Dmitry Lyumkis about 11 years ago
- Status changed from New to In Code Review
- Assignee changed from Dmitry Lyumkis to Anchi Cheng
added some code to the pre-processing do the following for pyace2.py
1) if image contains bad prime number, find nearest good image size (dimension larger than original)
2) pad out image to that dimension, using image mean as a constant frame
this makes ACE2 FFT calculation drop from ~10 minutes to 20 seconds. tested on K2 data, and ACE2 seems to estimate well, on par with the other packages (I'm comparing with CTFFind and phasor). Let me know if anyone finds problems.
This is revision 17944 (sorry didn't associate with commit)
Dmitry
Updated by Anchi Cheng about 11 years ago
- Status changed from In Code Review to In Test
- Assignee changed from Anchi Cheng to Gabriel Lander
Gabe, you should have noticed by now if this helped.
Updated by Gabriel Lander about 11 years ago
- Assignee changed from Gabriel Lander to Saikat Chowdhury
Saikat, how long does ACE2 take to estimate the CTF now? You're doing a lot more data collection than I am these days, so you'd have a better handle on this than me.
The ACE2 step should take about 20 seconds per image.
Updated by Gabriel Lander about 11 years ago
On garibaldi using the myami/trunk the ACE2 step still takes ~4 minutes per image. Does this have to be recompiled on garibaldi?