Project

General

Profile

Actions

Bug #585

closed

ace2 "wienerfilter"

Added by Gabriel Lander almost 14 years ago. Updated about 12 years ago.

Status:
Won't Fix or Won't Do
Priority:
Normal
Assignee:
Amber Herold
Category:
-
Start date:
05/27/2010
Due date:
% Done:

0%

Estimated time:
Affected Version:
Appion/Leginon 2.0.1
Show in known bugs:
No
Workaround:

Description

To what extent does the ace2 ctf correction "wienerfilter" the images? How much are the amplitudes adjusted? I just found out that no one in the Nogales lab has been using the ACE2 CTF correction in makestack because they're afraid that ACE2 correction is going to screw with the amplitudes in addition to phaseflipping. They've all been using the "eman phaseflip by stack", because it only phaseflips. Perhaps we should change this label so it doesn't scare users away from using ACE2 correction (because it's obviously the best)?


Files

20100528091223059.png (114 KB) 20100528091223059.png Gabriel Lander, 05/29/2010 02:11 AM
Actions #1

Updated by Neil Voss almost 14 years ago

You'll have to talk to Craig. I remember Scott was also very skeptical and had a conversation with Craig about it.

Actions #2

Updated by Scott Stagg almost 14 years ago

I have not been able to get ace2 working well here, so I haven't been able to test ace2 correction to my satisfaction. I recall talking with Craig about it, but I don't remember what he said. Regardless, if it's not just phase flipping then it has to be doing something with the amplitudes. My personal opinion is that you shouldn't mess with the amplitudes until the very end of your refinement.

Actions #3

Updated by Gabriel Lander almost 14 years ago

From Craig....

Oh boy. let's see... If I recall correctly the 'wiener filter' option in ACE2 isn't trying to correct the overall amplitudes but just the variations in amplitudes caused by the CTF. Can't recall wether this is really the common use of "wiener filter" in EM since wiener filtering is just a process of trying to optimally boost signal to noise when trying to 'undo' some kind of convolution (since deconvolution is especially sensitive to noise). In my case I'm trying to 'undo' the effect of the CTF, but NOT the envelope function. So to avoid saying something technically incorrect, if I haven't already, I'll just describe what ACE2 does and why I think it's OK.
So the black line is the CTF in the power spectrum. Phase flipping just tries to invert the phase sign of every other hump in the graph (but the graph would still have the exact same appearance). ACE2, tries to correct the dips in the graph to undo the CTF, and has some 'noise' clamping, which can be adjusted, that leaves the values alone if they get too close to zero (since division by zero is a big no-no, and would boost noise a lot more). The desired result of this is the dashed line, which is an attempt to de-convolute the CTF, but NOT the background noise and envelope functions. Wiener filtering, when also applied with the noise and envelope functions thrown in, would try to recreate something more like the large dashed line, which is what I believe a lot of people refer to as amplitude correction (or also wiener filtering!).

As you can see, the ACE2 filtering will boost the signal AND noise in the CTF humps, and as you get closer to the minima you get disproportionately more noise, which a lot of people don't like.... BUT this is before averaging! All things being consistent, it is probably better to average first and deconvolve the higher SNR average, BUT CTF isn't consistent from image to image!! (it is reasonable to assume noise and envelope are though). So in essence, even if you are boosting the noise a little too much at the lower reaches of the throughs, at least you are getting the buried signal closer to its 'real' value before averaging, whereas if you average first it's almost as if you are treating the amplitude variations caused by the CTF as part of the 'noise' you are trying to average away (and those variations are large!!). Even more optimal, which I don't think is done right now (by anyone), would be for ACE2 to store the masked zero locations so that they can be excluded in later Fourier space averaging (thereby minimizing the effects of the clamped regions).

Once ACE2 has corrected the images, and they have been averaged, one would hopefully get a nice version of the ACE2 dashed line with the zero clips filled in nicely and the 'true' signal nicely averaged together. One can then take this (hopefully) high SNR average and optionally apply `another` wiener filter to de-convolute the background noise and envelope functions, which it is reasonable to assume, were consistent throughout all the images. Any envelope functions fitted to this average will also be a lot more reliable for using in correction (or, instead, what most people do: just boost the high end until they can see what they want to see without hopefully making the map look like a porcupine).

The only equally 'good' alternative I can think of is to group particles by defocus, then average each group, correct the CTF and background noise in these averages, average the averages, then correct the envelope in the final average. That sound like a huge pain in the ass though, and would only work as well as the particles can be grouped by consistent defocus. So, as long as one is reasonably sure the defocus estimate is of good accuracy, and doesn't become overly zealous in pushing the zero clamping in the CTF correction, I think the ACE2 route is probably the most bang for the buck. (And let's face it, a poor estimate of defocus will wreck any route, including simple phase flipping).

Actions #4

Updated by Gabriel Lander almost 14 years ago

So this isn't a traditional "weinerfilter" by a typical user's definition... Do we change the option name? Document this anywhere? ideas?
Bueller?

Actions #5

Updated by Anchi Cheng almost 14 years ago

I have not looked into this that much, but you can examine Craig's code under appion/ace2.ace2correct.m Wiener filter as far as I know is used to prevent an explosion of noise as the signal get close to zero. For example, what Craig wanted to do was to divide the measured image FFT by CTF. However, CTF goes through zero. division of zero gives infinity. By adding a term in the denominator, it keeps the noise near CTF node from dominating the corrected FFT. In Craig's program this what he calls snr term is set to 0.15. If you break up images into defocus group, you will improve the overall signal to noise ratio, and therefore reduce snr probably but the term is still needed.

In a way I think the objection to altering the amplitude of the image FFT by what implied by CTF is due to the possibility of error in defocus and astigmatism estimation, not the amplitude contrast term that changes gradually and smoothly by resolution. A bad job on the former is more likely to cause alignment and classification error than the latter. Therefore, the former can cause more problem in reconstruction refinement. Of course, my experience is limited, maybe I am totally off-track.

Actions #6

Updated by Anchi Cheng almost 14 years ago

Anchi Cheng wrote:

... If you break up images into defocus group, you will improve the overall signal to noise ratio, and therefore reduce snr probably but the term is still needed.

I am taking it back. averaging does not alter the distribution of noise. Therefore, even there, you can't reduce snr term that way.

Actions #7

Updated by Amber Herold almost 14 years ago

  • Target version set to Appion/Leginon 2.1.0
  • Affected Version set to Appion/Leginon 2.0.1
Actions #8

Updated by Amber Herold almost 14 years ago

  • Target version changed from Appion/Leginon 2.1.0 to Appion/Leginon 2.2.0
  • Show in known bugs set to No
Actions #9

Updated by Amber Herold about 12 years ago

  • Target version changed from Appion/Leginon 2.2.0 to Appion/Leginon Future Version
Actions #10

Updated by Anchi Cheng about 12 years ago

  • Status changed from New to Assigned
  • Assignee set to Anonymous

Craig,

Can you answer this old question?

Actions #11

Updated by Gabriel Lander about 12 years ago

  • Status changed from Assigned to Won't Fix or Won't Do
  • Assignee changed from Anonymous to Amber Herold

I think that because this option does affect the amplitudes, whether it's a true "Wienerfilter" by traditional EM terminology or not, I think we should leave it as is. There is also a "ACE2 phaseflip only" option that won't affect the amplitudes in case users want to use that. I think we can close this issue.

Actions

Also available in: Atom PDF