leginon and 4K x 4K images

Added by Anonymous over 16 years ago

Hi,

We finally have our permanent server set up, and we are having some issues with saving 4K x 4K images. Our setup is to have the leginon database and fileserver running on an external linux box, and the leginon client running on a Windows PC next to the Tecnai PC. The linux drive is mounted by samba on both Windows computers. We can save all image sizes properly except the 4K x 4K size. When I do a gain normalization at 4K by 4K, both the bright field images and the dark field images get stored properly on the remote drive (float mode, size = 64 MB). I am able to collect and save raw images at 4K by 4K, but these raw images get saved as mode 6 (unsigned, 32 MB), even though they are supposed to be saved as float. The raw images are added properly to the database. Trying to collect corrected images at 4K by 4K fails, sometimes leaving the correction node hanging (little green arrow keeps updating). These images are not added to the database. 2K by 2K and smaller images have no problems, and everything is stored as float.

There had been no issue with 4Kx4K images when saving directly on a remotely mounted Windows drive. I know the documentation says there is an issue with samba and large images, but it seems that the bright and darkfield images are saved correctly, but the raw images are saved in the wrong mode. Could there be something in the code that is causing it to change the mode with large image sizes and remote drives?

Thanks,

Bill

Replies (2)

- Added by Jim Pulokas over 16 years ago

Our mrc.py module (in our pyami package) uses the numpy package to write data to disk. For some reason, numpy has a limit on how much data it can write to a samba share in one call to its write function. In recent versions of mrc.py, we now split up the write operation into smaller chunks that do not seem to cause a problem. Can you check that you are using a recent version of our pyami package. The version that is included in our 1.5b1 download (pyami revision 69) should have the fix. Or you can check out the latest from the svn repository like this:

svn co http://emg.nysbc.org/svn/pyami/trunk pyami-trunk
cd pyami-trunk
python setup.py install

If you think you are using the latest mrc.py, and still having problems, then this may be a new bug you have discovered. Let me know if that is the case, and I will give you some debugging tests to try.

Jim

- Added by Anonymous about 16 years ago

Thanks Jim. That worked, but only on the Manual application. I still get the same errors in MSI-Edge and MSI-Raster.

On a related note, I would like to be able to save 16-bit integer images, as opposed to 32-bit float. I know that there is an option to do this in several of the MSI nodes, but it doesn't work. The problem seems to be that I can't find a way to save a 16-bit gain reference in the corrections node. Is there an option there to do this?

Thanks,

Bill

(1-2/2)

Project

General

Profile

Leginon