Project

General

Profile

Actions

Bug #1873

closed

confict in icosahedral convention and refinement packages

Added by Anchi Cheng over 12 years ago. Updated about 9 years ago.

Status:
Closed
Priority:
Urgent
Assignee:
Category:
-
Target version:
Start date:
06/20/2012
Due date:
% Done:

0%

Estimated time:
Affected Version:
Appion/Leginon 2.2.0
Show in known bugs:
No
Workaround:

Description

EMAN does not take Crowther convension (I hope EMAN2 would)
Frealign does not take EMAN convension
Not sure about xmipp.

selectModelForm.php is letting user to combine any symmetry with the packages.

Will need a long-term solution is we want to continue a refinement from one package to another.

Any good idea from virologists?


Related issues 3 (1 open2 closed)

Related to Appion - Bug #2397: ParticleFile produced by EMAN for reconUploader does not match particle correctlyIn Code ReviewDmitry Lyumkis06/14/2013

Actions
Related to Appion - Bug #2396: uploadEMANRefine.py does not handle mirror correctlyClosedAmber Herold06/14/2013

Actions
Related to Appion - Bug #1872: different icosahedral symmetry convension makes euler angle passing to Frealign difficultWon't Fix or Won't Do Amber Herold06/20/2012

Actions
Actions #1

Updated by Anchi Cheng over 12 years ago

I see that prepFrealign.py which we retire at 2.2 converts all icos model into (2 3 5) in Frealign refinement but does not do anything special upon upload which would mean that the uploaded euler could not be used to project the initial model database entry into the view of the particle although would be fine with the refined density.

Current prepRefineFrealign.py retains (2 3 5) and (2 5 3) that frealign accepts, but it therefore can not accept (5 3 2) unless converted.

Was it meant to save in database eulers only one icos convention regardless of package ?

Actions #2

Updated by Anchi Cheng over 12 years ago

  • Status changed from New to Assigned
  • Assignee set to Anchi Cheng

Since no one has strong opinion, Dmitry and I decided to stick with one icos convention in the database (currently 532). The conversion is made when it is inserted to the database. Models will be kept in its original convention and refinement package will take all conventions it can handle. This means only eman-refined map need to converted during preparation if used as initial model for frealign.

r16899 and r16900 does this conversion when needed.

Still need check conversion during preparation.

Actions #3

Updated by Anchi Cheng over 12 years ago

  • Assignee changed from Anchi Cheng to Dmitry Lyumkis

I have to put this down for the next month and half. Here is the current.

r16985 have all my current version that seems to work by checking Xmipp project3d images against Frealign the projections shown in projection match except that apEulerCalc.convertXmippEulersToFrealign(Xphi, Xtheta, Xpsi) should return identical Eulers since xmipp spider single files and mrc stack are in the same orientation now.

Dmitry, please check apEulerCalc.convertXmippEulersToFrealign() assuming such orientation.

Still a mystery: even though the projection images matches, the reconstruction from Frealign using these Eulers still look bad. May need to investigate if Xmipp really is using these Euler internally for reconstruction.

Actions #4

Updated by Anchi Cheng over 11 years ago

  • Status changed from Assigned to In Code Review
  • Affected Version changed from Appion/Leginon 2.2.0 (trunk) to Appion/Leginon 2.2.0

Issue #2382 explains the bad reconstruction since the shifts were applied incorrectly.

r17832 makes the model file to be rotated to the following convention during prepRefine:
Xmipp: (2 3 5) 3DEM/Viper
Frealign: (2 3 5) 3DEM/Viper
EMAN: (5 3 2) EMAN

Earlier revisions already convert the resulting Eulers back to (5 3 2) during upload.

I think Neil has done some reconstruction of a virus recently. Maybe he can test.

Actions #5

Updated by Neil Voss over 11 years ago

Hi Anchi,

Yeah it was not rotating my model correctly, so I rotated it by hand. It was a mess, my frealign jobs would hang forever - probably my install of TORQUE and the agent.py - so I had to manually run all the job files. prepFrealign and uploadFrealign worked great other than the rotation issue. Eulers were converted fine from EMAN to FREALIGN.

I find that to convert between EMAN and FREALIGN some virus models needed a 90 rotation before proc3d icos5fTo2f and others did not.

Actions #6

Updated by Anchi Cheng over 11 years ago

r17634 enforces the (2 3 5) for Xmip and Frealign during refinement.

Here is what I know:
Frealign I symmetry is the same as Xmipp I1 (2 3 5)
Frealign I2 symmetry is the same as Xmipp I2 (2 5 3)

We have not had a native EMAN icos reconstruction for a while. I may have to pull some old ones out to be sure of its true orientation.

Actions #7

Updated by Anchi Cheng over 11 years ago

Looks like EMAN (5 3 2) is the same as Xmipp I4 (5 3 2) 5-fold on z, a 2-fold on y, one of the front most 5-fold on xz-plane pointing to positive x.

Confirmed by old eman reconstructions of rotavirus and two uploaded models (afhv and cnv). I did icos5fTo2f followed by rot=0,0,90 All gave the same result of (2 3 5) Viper.

Actions #8

Updated by Scott Stagg over 11 years ago

I will add what I know. I've never been able to get the reconstruction stuff working properly here at FSU (I've never had the time to troubleshoot). Anyway, I hacked into the Appion code, so that I don't have to upload reconstructions. My current workflow works with EMAN and Frealign and uses Appion libraries but not database. I use EMAN do get initial icosahedral Eulers (5 fold on Z). Then, use Appion code to convert those to the Frealign convention (2-fold on Z). Then I use the newly generated Frealign param file to make a reconstruction in Frealign. The resulting reconstruction will be in the correct orientation. Then I run a Frealign refinement with runFrealign.py that is in Appion but may only work at FSU.

Actions #9

Updated by Anchi Cheng over 11 years ago

  • Status changed from In Code Review to Assigned
  • Assignee changed from Dmitry Lyumkis to Anchi Cheng
  • Priority changed from Normal to Urgent

Found several bugs regarding the new uploadEMANRefine.py now that I can finally run EMAN refinement here. I will continue on this now that I have confirmed that the Euler transfer works right for GroEL from EMAN.

Actions #10

Updated by Anchi Cheng over 11 years ago

r17675 and r17659 debug r17632 prepRefineEman.py

Actions #11

Updated by Anchi Cheng over 11 years ago

r17678 consider default orientation at each conversion point so that the eulers are correct for the default at each point.
EMAN, database: 532
Frealign,Xmipp, particle.txt: 235

The refinement and upload are now correct for both starting orientation between EMAN and Frealign.

The remaining problems:
1. The rotated model has a 0.5 pixel shift EMAN failed the test dataset because of it.
2. A rotation of the reconstructed volume back to the initial model might be needed? Not sure if it causes too much confusion.

Actions #12

Updated by Anchi Cheng over 11 years ago

  • Target version set to Appion/Leginon 3.0.0

r17753 fixes the problem with rotation axis. EMAN defines its center of icosahedral object from even pixel cube at (x,y,z)=(0,-1,1) away from what frealign and xmipp defined. This causes rotation of an off-centered model map because even more off-center after the operation. EMAN is particularly sensitive to this offset while Frealign does not mind as much.

Actions #13

Updated by Amber Herold almost 11 years ago

Anchi,
Are you leaving this open for your point that "A rotation of the reconstructed volume back to the initial model might be needed?"?

Actions #14

Updated by Anchi Cheng almost 11 years ago

  • Status changed from Assigned to In Code Review
  • Assignee changed from Anchi Cheng to Amber Herold

Code review and testing by someone other than me is more important. The point about converting back to other convention can be a separate issue when others feel the same.

Actions #15

Updated by Amber Herold over 10 years ago

  • Assignee changed from Amber Herold to Anchi Cheng

I took a look at the code. Looks like you've implemented what you set out to implement.
Who should test?

Actions #16

Updated by Anchi Cheng about 9 years ago

  • Status changed from In Code Review to Closed

no one other than me would test. Considered done.

Actions

Also available in: Atom PDF