Appion

FEI has apparently broken the MRC format by adding a way-too-large header with some unconventional metadata. Anchi forwards me an excerpt of an email thread explaining this; below is a quote from Henderson:

The FEI header uses only the CCP4 locations 1-4 and location 24. All the others (i.e. 5-23 and
25-256) contain zeroes. The first three locations contain NX, NY and NZ correctly as 4096, 4096 and 1. Location 4 contains the MODE, which is 6 (non-standard, but most people seem to agree this was added by the tomography community as an extension to define unsigned INTEGER*2 numbers). At the moment we allow only MODES 0 to 4, but I expect it would be easy enough add MODE 6.

FEI then enters NSYMBT as 131072, whereas most programs expect 0, 80 or a small number, which is used to define the symmetry information. This is where FEI use it to define how many bytes to skip before the data starts. The problem is that any program that expects symmetry information here (such as the CCP4 and MRC image processing programs) will now find junk and crash, as observed.

It would also be useful if the other fields in the header contained the intended numbers. It would be desirabe to include :-

MX, MY, MZ (these can be the same as NX, NY, NZ)

CELLA (these would be the edge lengths of the image in Angstroms, so if the pixel size was 1.75 Å and the image was 4096x4096, these would be 7168.0, 7168.0, and the third number could be 1.0, which is arbitrary since it is a 2D and not a 3D image)

CELLB (the angle between the axes, these would all be 90.0 degrees)

MAPC, MAPR, MAPS - should be 1, 2, 3, specifying X as the most rapid axis, then Y, with Z slowest, which does not matetr since there is only one image.

DMIN,DMAX,DMEAN - minimum, maximum and mean density in the image histogram

ISPG - 0 (it is already 0)

NSYMBT - should be 0 because there is no symmetry information, because there is no spacegroup.

EXTRA - you can use these 100 bytes for anything you like.

ORIGIN - leave blank

MAP - this should be a character string to describe the file as a MAP

MACHST - machine stamp, I'm not sure how many people use this (Martyn?)

RMS - useful to include this calculation of the rms deviation from the mean density

NLABEL - number of labels used, set to 10 if all 10x80 byte in the final label field contain information

LABEL - 800 bytes (10x80) of descriptive information. This could include lots of useful stuff about the microscope settings, but is not enough to allow the 131,072 bytes that are there currently.

Altogether, the standard header allows for 900 bytes of completely free user-defined information

I'm trying to scrub the MRCs of their problematic metadata so as to more easily dovetail with Appion. For example, manualmask.py preprocesses the micrographs to give all-black fields (no data). Appion displays these micrographs correctly, except for a black bar that Anchi says corresponds to the extraneous header information getting parsed as pixel intensities.

Interestingly, e2display.py can show the micrographs correctly, but v2 cannot.

I've attached a script that goes some way to fixing the micrographs, but that apparently still gives bad results with manusalmask.py. I'm assuming this 'feature' should be in testing/debugging stage.

After processing by this script, v2 displays the data (but with the contrast/intensity set incorrectly.)