Bug #2202
closedK2 DD corrected frame alignment fails with certain gain references
100%
Description
I am filing this bug due to the problems that I have had aligning my K2 frames in the dataset 12dec22a.
the main problem that I have experienced is that, when using different gain-normalization images for correcting the raw frame stack, the program dosefgpu_driftcorr gives drastically different results, often times failing, as judged by the shifts and summed image. While I was collecting data, I was testing out different methods of gain corrections, then creating a raw-frame stack, then testing Yifan's program outside of Appion, and looking at the Log files & summed images. In particular, I was interested to see that the alignment parameters are consistent (i.e. they decrease / increase, but they do not jump all over the place). The gain corrections that were tested during that data collection were (all differences for the bright only, and I think I am listing them correctly, although it might be worth looking into the database to double check):
2000 ms * 10 frames -- collected at the start of the session on 12/22, hardware gain corrections were done beforehand, but given what you have said, I wouldn't assume that they had any effect
250 ms * 10 frames -- collected morning of 12/23
5000 ms * 10 frames -- collected 12/23, ~1 PM, hardware gain corrections were redone, and I believe I unchecked / rechecked the box to reupload them (although I'm not sure that this was done correctly and that re-uploading actually occurred)
5000 ms * 10 frames -- collected 12/23, ~3:30 PM
All this is to say that, the main thing that I would like to determine is whether or not I can reuse a specific set of gain references in order to make my aligned frame stack, then test the frame alignment. For example, one thing that would be great is to reuse the gain references from 12dec05a (collected under otherwise identical conditions) in order to align 12dec22a.
Anchi, if you would like to test this for yourself to see what I am referring to, you can go into the directory /ami/data00/appion/12dec22a/ddstack/testanchi (should be read/writable by all), create a single corrected frame stack using a specific set of gain references, then align using dosefgpu_driftcorr.
Let me know if you need any more information, and I'll be happy to provide whatever I can.
Files
Updated by Anchi Cheng almost 12 years ago
- Status changed from New to Assigned
- Assignee changed from Anchi Cheng to Dmitry Lyumkis
- Affected Version changed from Appion/Leginon 2.1.0 to Appion/Leginon 3.0.0
Dmitry,
Could you give me a specific image that failed with its own references so that I can see if I can improve it?
Updated by Dmitry Lyumkis almost 12 years ago
How about this one:
12dec22a_gr2_00012gr_00013sq_v01_00033hl_00004ed.mrc
logfile: /ami/data00/appion/12dec22a/ddstack/ddstack2/12dec22a_gr2_00012gr_00013sq_v01_00033hl_00004ed_st_Log.txt
or this one:
12dec22a_gr2_00012gr_00013sq_v01_00009hl_00003ed.mrc
logfile: /ami/data00/appion/12dec22a/ddstack/ddstack2/12dec22a_gr2_00012gr_00013sq_v01_00009hl_00003ed_st_Log.txt
and here's one that appears highly drifty / astigmatic, and I can't seem to be able to correct it (the scope was coma-free aligned)
12dec22a_gr2_00012gr_00013sq_v01_00012hl_00002ed.mrc
logfile: /ami/data00/appion/12dec22a/ddstack/ddstack2/12dec22a_gr2_00012gr_00013sq_v01_00012hl_00002ed_st_Log.txt
...
I have been going through the directory "ddstack2" to try and pick out good examples. A number of the images do seem to align. The way I am judging this is by looking at the shifts, and in select cases at the FFTs. For about 1/2 of the images, The shifts follow a specific trend, which is expected for a 5-second exposure (e.g. from -4 to 4 pixels). For the other half, there is no trend and the shifts fluctuate randomly around +/-1 (you can test this, for example in bash, `for file in *ed_st_Log.txt ; do tail -22 $file ; done`). The FFTs for these images don't look terrible, but the signal is weak, weaker than in my previous data. In addition, many of them appear astigmatic, even though I know the scope was well aligned, which you can see from the coma-free tableuas in the data. Perhaps this is all due to a weak signal, since the thon rings are consistently weaker than in my other session, 12dec05a.
For example, compare any image from 12dec22a with this image from 12dec05a, which doesn't even have any carbon in it:
12dec05a_gr1_00012gr_00057sq_v01_00006hl_00003ed-a.mrc
or, more comparable images (and FFTs)
12dec05a_gr1_00012gr_00057sq_v01_00007hl_00003ed-a.mrc ...... vs. ...... 12dec22a_gr2_00011gr_00031sq_v02_00021hl_00003ed-a.mrc
Updated by Anchi Cheng almost 12 years ago
feature #2216 allows the specification with the image id of the required image.
I tested on 12dec22a_gr2_00012gr_00013sq_v01_00033hl_00004ed.mrc and the alignment result (on defcon1) is not jumping.
I still need to find out why it failed previously, but this will keep you moving.
Updated by Dmitry Lyumkis almost 12 years ago
Yes, this is much better than the previous logfile. Did you do anything different, apart from specifying the reference image? (I don't remember if I was trying to run on amibox04 or defcon1). What is the reference image (2026767)? Which table are they located in, and what reference images should I be trying?
Updated by Dmitry Lyumkis almost 12 years ago
- File 12dec22a_gr2_00012gr_00019sq_v03_00034hl_00003ed-a.jpg 12dec22a_gr2_00012gr_00019sq_v03_00034hl_00003ed-a.jpg added
- File 12dec22a_00055fc_tableau.jpg 12dec22a_00055fc_tableau.jpg added
It's interesting that it is still consistently failing in one direction. The thon rings in Y are consistently more pronounced than in X.
Attached is the coma-free alignment tableu and a typical "aligned" and summed image from the session.
Updated by Dmitry Lyumkis almost 12 years ago
- Status changed from Assigned to In Test
Updated by Dmitry Lyumkis almost 12 years ago
- Status changed from In Test to Closed