Appion

Hi Scott,

That is a good question that I have thought long about.

ANGLES:
BEFORE: 50% of ACE2 values before the change had bad astig angles in radians, ACE1 did not insert angles (no astig), and CTFFIND had good angles, but in degrees.
AFTER: Now, all angles are in degrees.

DEFOCUS:
The order of the defocus does not matter to any of the programs, I just do the conversion, so that we have an internal convention that matters. So before and after are equivalent.
BEFORE: defocus1 or defocus2 could be a larger number.
AFTER: defocus1 is always less than defocus2. Astig angle is rotated by 90 degrees to switch order.

Neil

OK. I think I understand, but I'm getting some inconsistent results. It sounds like you're saying that if I use the values directly from the database they will work directly with Frealign. In other words, if I use ctfdb.getCtfValueForImage() and convert the defocus values to angstroms, then df1, df2, and angast can go straight into the frealign param file. However, the apFrealign code is a little different:

                if ctfdata is None:
                    ctfdata, confidence=ctfdb.getBestCtfValueForImage(imagedata, msg=False)
                if ctfdata is not None:
                    # use defocus & astigmatism values
                    particleparams['df1']=abs(ctfdata['defocus1']*1e10)
                    particleparams['df2']=abs(ctfdata['defocus2']*1e10)
                    particleparams['angast']=-ctfdata['angle_astigmatism']

Oh, yeah, I think CTFFIND and FREALIGN use the negative of the database values. Despite the figure in the CTFFIND paper, Mindell 2003, showing the opposite. You'll note that we take the negative of the CTFFIND value after estimation, this has not changed since its initial implementation in myami 2.0 or whatever.

That is an interesting result on the resolution. I wish I could say more about it. You would figure that CTFFIND and FREALIGN would give the same results. This is something, I am doing for my paper, but it might be interesting to plot the CTFFIND defocus against the ACE1 defocus values.

Another piece of data...I swapped the sign on angast for the frealign reconstruction I was talking about above, and the resolution went to 5.4 instead of 5.8 that I was getting before. I guess that demonstrates that sign convention wasn't the problem. I feel like I'm using CTFFIND properly. I used the "fine tune previously estimated defocus" settings. I think all the high-resolution reconstructions used CTFFIND, so do y'all think the reason that it's not working great for me is the difference between CCD data and film?

I disagree the ACE1 fit (from its graph) goes awry at about 9 Angstroms. It is just hard to see, because the normalization is off.

I will try to test sometime, first class is tonight.

Also, the decay of signal around 8A (.12%) and then it comes back around 6.5A (.154) is usually evidence of astigmatism.

I agree that the normalization is off with the ACE1 graphs, but what I'm looking at is the position of the minima. In the final ACE1 graph, there are overlapping minima in the simulated and calculated CTF all the way out to the end of the graph. In the new CTF graph, the minima start to diverge at the third minimum.

I'm sure you're right about there being some astigmatism.

Okay, your second image is convincing. The other one I had a hard time believing your argument about something we could not see.

ACE1 code from getgamma.m and getctf.m

gmma = 2*pi*(0.25*Cs*(lambda.^3).*(s.^4) + 0.5*defocus.*lambda.*s.^2); 
ctf = (sqrt(1-A^2)*sin(gmma) + A*cos(gmma)).^2;  

<hr/>

so I quickly rewrote my python code to generate the CTF, so it uses the same equations above verbatim. Now the problem comes with the sign of the defocus...

What I found was that if I flip the sign of the defocus (for my original equation as well), it fits the data perfect. Now the question is, what is going wrong?

So, are you sure the images you send me are not over-focused? I reprocessed the CTF of my high-res virus data (~5.8 A) and it was happier before.

I suppose it's possible that the image is over focus. I will take a new carbon image where I'm positive it is underfocus, and I'll try again.

Thanks, I want to make sure this all correct.

Hi Scott, based on the last results you are using an old version, could you please test on the latest version.

I can tell empirically that the current algorithm is getting the defocus correct out to beyond 6 Å. I would say that you should change nothing in the ACE1 code, and instead have a conditional in your new code that says if you're using ACE1, then when you simulate the CTF then you take the negative of the defocus. I assume that ACE is getting the defocus correct because it is essentially doing a double negative (switching the sign of both Cs and defocus). If you insist on changing the ACE1 code, then I could test it on the super high-res carbon images that are attached to this issue. However, I must insist that the new ACE code produce exactly the same defocus that it is currently producing. I don't care what is going on in the guts as long as it gets the answer right, and right now ACE1 gives the best results for ice images of any CTF program that we use.

Note: the CTF is not symmetric

CTF(gamma) != CTF(-gamma)

Again, I am not sure this is it, I just found this discrepancy. I have tried all forms of the +Cs and + defcous on Anke's 30S data which goes out past 5 Ångstroms and has clearly visible particles. Having a +Cs fails the fit past 1/10A.

You are pointing out that the peaks don't match up and I think this maybe the cause. I do NOT want to change the code, because if you change it one place you could miss another; it is a steep slope. It should not effect the defocus estimate at lower resolutions, so there is no need to fix and long as it is working. But, in my opinion, it is impossible to see the ACE1 fit out very far.

I think we need to have a conference call about this. You keep saying that ACE1 is not estimating the CTF properly past 10 Å, but empirical evidence says that it performs the best. Does everyone think we can get a Scripps, Berkley, and FSU conference call? It would need to be hosted by Scripps, I think.

I can work on setting up a call. Finding a time that works for everyone may be the hard part. Scott and Neil, can you suggest times that work for you?

I would like to investigate this some more. Let me write it up in a more coherent format. Because I am not really convinced myself. I still trying figure what the heck is going on, but I can get matches out to 5A.

My feeling is that as long as you are close enough you can get high resolution. For instance, I got a 5.8A virus with the wrong astigmatism angles from ACE2. From ACE1, it may use the wrong Cs in the estimation, but the EMAN1 uses the correct formula in the phase flipping, so all is good. There is just too many discrepancies to mention. If you look at ACE1's astig formula (in the paper), it is ridiculously wrong.

Also, I will NOT be making any changes to the code, because I do not have time.

I need to investigate this, too. I've confirmed that defocus of ace1 in database is indeed positive all the time but would like to know what the raw output is like and test run some images I took for this. Only if I can get ace1 to run here !!!.

I ran Ace1 on Goby a couple weeks ago, and had Sargis install missing libs on Guppy (see #2088) but I may not have tested on Guppy since then...