Project

General

Profile

Actions

Task #2071

closed

Bug #2068: Resolve CTF problem effort by NRAMM local team

Ctf testing matrix

Added by Amber Herold almost 12 years ago. Updated almost 12 years ago.

Status:
Closed
Priority:
Immediate
Assignee:
Amber Herold
Category:
-
Target version:
-
Start date:
10/01/2012
Due date:
% Done:

0%

Estimated time:

Description

  1. We need to identify all test data needed to test all features of CTF Estimation found in appion.
  2. We need to know what the output from these data should be.
  3. We need to identify all the tests that need to be run.

CTF Estimation Test Matrix

Ace2 Tests
Note: After running command, results are found in ...imagename.mrc.ctf.txt

Astigmatism Over/Under Focus Command Image Name Defocus Ang Astigmatism Amplitude Contrast Confidence
none over ace2.exe -i 12oct01e_00059en.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0 12oct01e_00059en.mrc 5.568862e-07 6.559742e-07 -70.235983 0.091264 0.815827
none under ace2.exe -i 12oct01e_00055en.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0 12oct01e_00055en.mrc 1.280421e-06 1.318144e-06 15.267624 0.192086 0.947226
direction 1 over ace2.exe -i 12oct01e_00041ea.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0 12oct01e_00041ea.mrc 1.337478e-06 1.482890e-06 72.485569 0.050000 0.863265
direction 1 under ace2.exe -i 12oct01e_00032ea.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0 12oct01e_00032ea.mrc 1.773057e-06 1.867818e-06 -66.550039 0.182214 0.957607
direction 2 over ace2.exe -i 12oct01e_00050eb.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0 12oct01e_00050eb.mrc 1.477478e-06 1.674565e-06 -64.236799 0.050000 0.876410
direction 2 under ace2.exe -i 12oct01e_00054eb.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0 12oct01e_00054eb.mrc 1.533405e-06 1.649353e-06 29.912167 0.194930 0.958123

Ace1 Tests
Note: After running command, results are found in

Astigmatism Over/Under Focus Command Image Name Defocus Ang Astigmatism Amplitude Contrast Confidence
none over
none under

CTFFind Tests
Note: After running command, results are found in rundir/logfiles/imagename-ctfprog.log if run in appion. Output files end in -pow.mrc and are attached to this issue.

Astigmatism Over/Under Focus Command Image Name Defocus Ang Astigmatism Cross Correlation
none over ctffind64.exe 12oct01e_059en.mrc 12oct01e_059en-pow.mrc 2.0,200.0,0.15,109841.827768,15.0,2 512,40.0,8.0,1000.0,30000.0,1000.0,100.0 12oct01e_00059en.mrc 6072.85 5544.12 -17.61 0.47946
none under ctffind64.exe 12oct01e_055en.mrc 12oct01e_055en-pow.mrc 2.0,200.0,0.15,109841.827768,15.0,2 512,40.0,8.0,1000.0,40000.0,1000.0,100.0 12oct01e_00055en.mrc 13154.68 13285.13 -18.97 0.40741
direction 1 over ctffind64.exe 12oct01e_037ea.mrc 12oct01e_037ea-pow.mrc 2.0,200.0,0.15,109841.827768,15.0,2 512,40.0,8.0,1000.0,35000.0,1000.0,100.0 12oct01e_00037ea.mrc 8318.40 8167.91 50.30 0.48868
direction 1 under ctffind64.exe 12oct01e_032ea.mrc 12oct01e_032ea-pow.mrc 2.0,200.0,0.15,109841.827768,15.0,2 512,40.0,8.0,1000.0,40000.0,1000.0,100.0 12oct01e_00032ea.mrc 18052.81 18536.83 65.26 0.36839
direction 2 over ctffind64.exe 12oct01e_048eb.mrc 12oct01e_048eb-pow.mrc 2.0,200.0,0.15,109841.827768,15.0,2 512,40.0,8.0,1000.0,30000.0,1000.0,500.0 12oct01e_00048eb.mrc 4500.14 6016.58 64.72 0.45517
direction 2 under ctffind64.exe 12oct01e_054eb.mrc 12oct01e_054eb-pow.mrc 2.0,200.0,0.15,109841.827768,15.0,2 512,40.0,8.0,1000.0,40000.0,1000.0,500.0 12oct01e_00054eb.mrc 15613.98 16342.15 -28.88 0.37626

Xmipp CTF Tests
Note: After running command, results are found in

Astigmatism Over/Under Focus Command Image Name Defocus Ang Astigmatism Amplitude Contrast Confidence
none over
none under
direction 1 over
direction 1 under
direction 2 over
direction 2 under

CTF Correction Test Matrix

Estimation Package Correction Method Astigmation Over/Under Focus Command Image Name
Ace2 ACE2 wiener none over 12oct01e_00059en.mrc
Ace2 ACE2 wiener none under 12oct01e_00055en.mrc
Ace2 ACE2 wiener direction 1 over 12oct01e_00041ea.mrc
Ace2 ACE2 wiener direction 1 under 12oct01e_00032ea.mrc
Ace2 ACE2 wiener direction 2 over 12oct01e_00050eb.mrc
Ace2 ACE2 wiener direction 2 under 12oct01e_00054eb.mrc
Ace2 ACE2 phaseflip none over
Ace2 ACE2 phaseflip none under
Ace2 ACE2 phaseflip direction 1 over
Ace2 ACE2 phaseflip direction 1 under
Ace2 ACE2 phaseflip direction 2 over
Ace2 ACE2 phaseflip direction 2 under
Ace2 SPIDER phaseflip none over
Ace2 SPIDER phaseflip none under
Ace2 SPIDER phaseflip direction 1 over
Ace2 SPIDER phaseflip direction 1 under
Ace2 SPIDER phaseflip direction 2 over
Ace2 SPIDER phaseflip direction 2 under
Ace2 EMAN phaseflip whole image none over
Ace2 EMAN phaseflip whole image none under
Ace2 EMAN phaseflip whole image direction 1 over
Ace2 EMAN phaseflip whole image direction 1 under
Ace2 EMAN phaseflip whole image direction 2 over
Ace2 EMAN phaseflip whole image direction 2 under
Ace2 EMAN phaseflip by boxed stack per image none over
Ace2 EMAN phaseflip by boxed stack per image none under
Ace2 EMAN phaseflip by boxed stack per image direction 1 over
Ace2 EMAN phaseflip by boxed stack per image direction 1 under
Ace2 EMAN phaseflip by boxed stack per image direction 2 over
Ace2 EMAN phaseflip by boxed stack per image direction 2 under
Ace2 EMAN phaseflip by tilt location none over
Ace2 EMAN phaseflip by tilt location none under
Ace2 EMAN phaseflip by tilt location direction 1 over
Ace2 EMAN phaseflip by tilt location direction 1 under
Ace2 EMAN phaseflip by tilt location direction 2 over
Ace2 EMAN phaseflip by tilt location direction 2 under
Ace1 ACE2 wiener none over
Ace1 ACE2 wiener none under
Ace1 ACE2 phaseflip none over
Ace1 ACE2 phaseflip none under
Ace1 SPIDER phaseflip none over
Ace1 SPIDER phaseflip none under
Ace1 EMAN phaseflip whole image none over
Ace1 EMAN phaseflip whole image none under
Ace1 EMAN phaseflip by boxed stack per image none over
Ace1 EMAN phaseflip by boxed stack per image none under
Ace1 EMAN phaseflip by tilt location none over
Ace1 EMAN phaseflip by tilt location none under
CTFFind ACE2 wiener none over 12oct01e_00059en.mrc
CTFFind ACE2 wiener none under 12oct01e_00055en.mrc
CTFFind ACE2 wiener direction 1 over 12oct01e_00037ea.mrc
CTFFind ACE2 wiener direction 1 under 12oct01e_00032ea.mrc
CTFFind ACE2 wiener direction 2 over 12oct01e_00048eb.mrc
CTFFind ACE2 wiener direction 2 under 12oct01e_00054eb.mrc
CTFFind ACE2 phaseflip none over
CTFFind ACE2 phaseflip none under
CTFFind ACE2 phaseflip direction 1 over
CTFFind ACE2 phaseflip direction 1 under
CTFFind ACE2 phaseflip direction 2 over
CTFFind ACE2 phaseflip direction 2 under
CTFFind SPIDER phaseflip none over
CTFFind SPIDER phaseflip none under
CTFFind SPIDER phaseflip direction 1 over
CTFFind SPIDER phaseflip direction 1 under
CTFFind SPIDER phaseflip direction 2 over
CTFFind SPIDER phaseflip direction 2 under
CTFFind EMAN phaseflip whole image none over
CTFFind EMAN phaseflip whole image none under
CTFFind EMAN phaseflip whole image direction 1 over
CTFFind EMAN phaseflip whole image direction 1 under
CTFFind EMAN phaseflip whole image direction 2 over
CTFFind EMAN phaseflip whole image direction 2 under
CTFFind EMAN phaseflip by boxed stack per image none over
CTFFind EMAN phaseflip by boxed stack per image none under
CTFFind EMAN phaseflip by boxed stack per image direction 1 over
CTFFind EMAN phaseflip by boxed stack per image direction 1 under
CTFFind EMAN phaseflip by boxed stack per image direction 2 over
CTFFind EMAN phaseflip by boxed stack per image direction 2 under
CTFFind EMAN phaseflip by tilt location none over
CTFFind EMAN phaseflip by tilt location none under
CTFFind EMAN phaseflip by tilt location direction 1 over
CTFFind EMAN phaseflip by tilt location direction 1 under
CTFFind EMAN phaseflip by tilt location direction 2 over
CTFFind EMAN phaseflip by tilt location direction 2 under
Xmipp CTF ACE2 wiener none over
Xmipp CTF ACE2 wiener none under
Xmipp CTF ACE2 wiener direction 1 over
Xmipp CTF ACE2 wiener direction 1 under
Xmipp CTF ACE2 wiener direction 2 over
Xmipp CTF ACE2 wiener direction 2 under
Xmipp CTF ACE2 phaseflip none over
Xmipp CTF ACE2 phaseflip none under
Xmipp CTF ACE2 phaseflip direction 1 over
Xmipp CTF ACE2 phaseflip direction 1 under
Xmipp CTF ACE2 phaseflip direction 2 over
Xmipp CTF ACE2 phaseflip direction 2 under
Xmipp CTF SPIDER phaseflip none over
Xmipp CTF SPIDER phaseflip none under
Xmipp CTF SPIDER phaseflip direction 1 over
Xmipp CTF SPIDER phaseflip direction 1 under
Xmipp CTF SPIDER phaseflip direction 2 over
Xmipp CTF SPIDER phaseflip direction 2 under
Xmipp CTF EMAN phaseflip whole image none over
Xmipp CTF EMAN phaseflip whole image none under
Xmipp CTF EMAN phaseflip whole image direction 1 over
Xmipp CTF EMAN phaseflip whole image direction 1 under
Xmipp CTF EMAN phaseflip whole image direction 2 over
Xmipp CTF EMAN phaseflip whole image direction 2 under
Xmipp CTF EMAN phaseflip by boxed stack per image none over
Xmipp CTF EMAN phaseflip by boxed stack per image none under
Xmipp CTF EMAN phaseflip by boxed stack per image direction 1 over
Xmipp CTF EMAN phaseflip by boxed stack per image direction 1 under
Xmipp CTF EMAN phaseflip by boxed stack per image direction 2 over
Xmipp CTF EMAN phaseflip by boxed stack per image direction 2 under
Xmipp CTF EMAN phaseflip by tilt location none over
Xmipp CTF EMAN phaseflip by tilt location none under
Xmipp CTF EMAN phaseflip by tilt location direction 1 over
Xmipp CTF EMAN phaseflip by tilt location direction 1 under
Xmipp CTF EMAN phaseflip by tilt location direction 2 over
Xmipp CTF EMAN phaseflip by tilt location direction 2 under

Files

12oct01e_00054eb.mrc (64 MB) 12oct01e_00054eb.mrc Amber Herold, 10/04/2012 12:28 PM
12oct01e_00059en.mrc (64 MB) 12oct01e_00059en.mrc Amber Herold, 10/04/2012 12:29 PM
12oct01e_00032ea.mrc (64 MB) 12oct01e_00032ea.mrc Amber Herold, 10/04/2012 12:29 PM
12oct01e_00037ea.mrc (64 MB) 12oct01e_00037ea.mrc Amber Herold, 10/04/2012 12:29 PM
12oct01e_00041ea.mrc (64 MB) 12oct01e_00041ea.mrc Amber Herold, 10/04/2012 12:29 PM
12oct01e_00048eb.mrc (64 MB) 12oct01e_00048eb.mrc Amber Herold, 10/04/2012 12:29 PM
12oct01e_00050eb.mrc (64 MB) 12oct01e_00050eb.mrc Amber Herold, 10/04/2012 12:29 PM
12oct01e_059en-pow.mrc (1 MB) 12oct01e_059en-pow.mrc Amber Herold, 10/05/2012 11:45 AM
12oct01e_032ea-pow.mrc (1 MB) 12oct01e_032ea-pow.mrc Amber Herold, 10/05/2012 11:45 AM
12oct01e_037ea-pow.mrc (1 MB) 12oct01e_037ea-pow.mrc Amber Herold, 10/05/2012 11:45 AM
12oct01e_048eb-pow.mrc (1 MB) 12oct01e_048eb-pow.mrc Amber Herold, 10/05/2012 11:45 AM
12oct01e_054eb-pow.mrc (1 MB) 12oct01e_054eb-pow.mrc Amber Herold, 10/05/2012 11:45 AM
12oct01e_055en-pow.mrc (1 MB) 12oct01e_055en-pow.mrc Amber Herold, 10/05/2012 11:45 AM
12oct01e_00055en.mrc (64 MB) 12oct01e_00055en.mrc Amber Herold, 10/08/2012 11:31 AM
Actions #1

Updated by Bridget Carragher almost 12 years ago

The matrix is a good idea in theory but seems we will never get all these tests done in practice. So how about just picking some obvious ones and making sure those work. We certainly never did these exhaustive tests in the past. I would also assume for now during this "emergency" that all we should test are underfocus as that would be 99% of the images we take and I would just post a warning to users that we are unsure of the behavior for overfocus images.

Actions #2

Updated by Anchi Cheng almost 12 years ago

This is for the future testing and the over/under only need to be done at the algorithm level and one image each.

I will aquire one image for each case for this today.

Actions #3

Updated by Neil Voss almost 12 years ago

Currently, the pipeline is only designed to handle underfocus. Overfocus should fail, which may be a good test nonetheless.

Actions #6

Updated by Amber Herold almost 12 years ago

  • File deleted (12oct01e_00055en.mrc)
Actions #7

Updated by Amber Herold almost 12 years ago

Actions #8

Updated by Amber Herold almost 12 years ago

r17167 adds testAce2.py to myami. It uses the standard python unittest class to run all the tests outlined in the Ace2 CTF Estimation test matrix and compare the results to what is expected.

### This class is used to test the Ace2 program. It will dowload the needed test images from the AMI website, 
### then run ace2.exe and compare the results to what is ecpected. It will remove results files from a prior run
### when running subsequent tests. It does not remove the files after a test is run so that the usr can review 
### the result files if needed. Any functions in this class that begin with "test_" is a seperate test that
### will be executed by unittest.

### To run this test:
### 1. Create a temporary test directory and change directories to it. ex. mkdir ace2test, cd ace2test
### 2. run this file by providing the path to it. ex. python ../workspace/myami/appion/test/testAce2.py
### 3. review the results printed to the terminal.

Actions #9

Updated by Amber Herold almost 12 years ago

  • Status changed from Assigned to Closed
Actions

Also available in: Atom PDF