Appion

First thing I see is that the empty classes are replaced with the stack average, so that is confusing.

Second, I align all the classes to one another, so in the RELION version the object is pointing in various directions whereas in Appion they are point in the same direction.

Third, I cannot explain the class average differences, the Appion version seems like something I would expect. The contrast is too high, but more what I expect. I would recommend opening the appion stack in another viewer (e.g., EMAN) where you can scale the contrast, because all the details are washed out.

It seems for sure that these are not the same particles contrast not withstanding. This is pretty worrying that a bug this serious is in there. Please can anyone with input try and help sort this out.
Thanks,
Bridget

I was hoping if the contrast is corrected, we might see the relationship between the two. To help see what is going on.

I could also be that RELION added a feature (such as mirroring) since I got it working this summer.

Is it possible that we should not put it as an Particle Alignment step ? The forcing of aligning the reference step might may be part of the problem. Sorjs said that Relion does the normalization by making the region outside the mask to have standard deviation of 1. This normalization is important in his statistical model. The references out of Relions are masked and without any noise outside the mask.

Achi it is a good point to do a preprocess the stack using "relion_preprocess" command. But here the relion results I attached is pulled from the appion folder which is still from the appion wrapper command with the option "--dont_checknorm" option.

I processed this stack in relion separately and results look same. I am mainly bothered about the visualization in appion, somehow the preprocess done with the relion results to show in web viewer has some bug in it.

oops!! Anchi, I did not read this statement carefully - " The forcing of aligning the reference step might may be part of the problem." I meant the same.

As Anchi points out, I read the alignment results from RELION and create my own aligned stack and then average the classes and upload.

• Alignment parameter is wrong
• particle numbers are shifted or wrong, washing out classes

I tested for these extensively, but I cannot rule out error.

I do not think the pre-normalization is important, but it is possible that RELION does not use 100% of the intensity in the average, but a weighted average of the particle in the class based on the probabilities.

For aligning the references, I run a second instance of RELION, get a second set of alignment parameters and merge them with the first alignment parameters (using trig). Then I create my own aligned stack and then average the classes and upload

I think our problem comes from making "the average of the aligned particles in a class" as a reference class at the end of the upload. We should stop doing that. Maxlike method classes are certainly weighted sum of the particles. I don't think have a complete output of the relative weight,. There are a few single number propability values in _data.star that is not enough to reconstruct the weight.

The upload process should simply upload the shift/rotated references as using what comes out from what Neil referred as "second instance of RELION" that operates on the original "classes" as Relion calls it.

Hope I am not making it more confusing. I illustrate what I think would work with this excercise:
I uploaded the iter030_classes.mrcs from 16nov20d/align/rmaxlike2/iter030 into session 16nov04z and run relion maxlike 2D classification in Appion as it would be in what Neil called the "second instance". If you view the alignestack, then we have the desired effect that keeps what Relion is famous for - Well defined lobes with high resolution feature, and yet aligned relative to each other (what Appion wants to have).

http://emgweb.nysbc.org/betamyamiweb/processing/viewstack.php?file=/gpfs/appion/acheng/16nov04z/align/rmaxlike1/alignstack.hed&expId=2875&ps=6.0E-10

To display the number of particles "Belong" to a class (I think it is an over statement, it probably only means the largest propablity), then the upload script can attach the number from reading into _data.star file.

The discrepancies between the RELION probability classes and Appion forced align classes have never been so profound.

It is like quantum mechanics, particles can exist in multiple alignments and classes until to make a measurement and force them into a particular state, which is their most probable state. I preferred the forced align classes, because if I make a substack or do an RCT, I am using the forced parameters, not the probability distribution. But if you are doing a rough analysis of the data, you want the pretty pictures, so maybe we should provide both.

I am still concerned about the differences between the two sets, they are usually pretty close. If I had time, I would probably want to do some more testing.

Sadly, I have no time.

It may store the weights, but each particle has a weight (probability) for every shift, every rotation, and every class as it is needed in the integral, so it would be a BIG file.

Thanks Anchi - that sounds like what we want. Given that Neil does not have time to work on this can you perhaps recruit Carl? Perhaps with your help? Not sure who else has the time or skill set?
Best regards,
Bridget

Thanks.

A interesting result from what I have seen though is that for well separated classes, the unweighted average of the particles in class is more similar to thee weighted one. Your previous example was a better one to observe problems.

Hi, this time I am trying a different dataset which is more heterogeneous to test as anchi suggested.

But somehow I can't run the job ..it is complaining about the total memory should be <16GB per node. But each node in our cluster has 256GB(I think).

This is the error,
Error: Total MPI threads per node * MPI memory/thread is greater than max memory per node.
( 23 procs/node * 1 threads/proc * 4 Gb/thread = 92 Gb > 16 Gb )

This is an appion configuration issue, not an Appion bug. I will take care of it at NYSBC.

Hi Ashleigh,

I am confused by your bug report. Relion is notorious for having empty classes in 2d alignment and there is nothing I can do about it on the Appion side of things, which is why I always use the same program in Xmipp.

Perhaps they fixed the empty class problem in Relion 2. Can't you just upload the Relion 2 class averages? What error do you get?