Project

General

Profile

Issue viewing uploaded images

Added by Douglas Walker over 1 year ago

Hello,
I am having issues with images that I am trying to upload to Appion. I have tried uploading both .dm3 and .tif files and I get different errors when I try each one.

With an attempt to upload a dm3 I get the following error:

!!! WARNING: EMAN failed with subprocess error code 127
Traceback (most recent call last):
File "/usr/local/anaconda2/bin/appion/imageloader.py", line 6, in <module>
app.run()
File "/usr/local/anaconda2/lib/python2.7/site-packages/appionlib/apImageUpload.py", line 256, in run
info = self.readUploadInfo(self.batchinfo[imgnum])
File "/usr/local/anaconda2/lib/python2.7/site-packages/appionlib/apImageUpload.py", line 625, in readUploadInfo
apDisplay.printError("image conversion to mrc did not execute properly")
File "/usr/local/anaconda2/lib/python2.7/site-packages/appionlib/apDisplay.py", line 74, in printError
raise Exception, colorString("\n * FATAL ERROR *\n"+text+"\n\a","red")
Exception: * FATAL ERROR *
image conversion to mrc did not execute properly

And when I try to upload a .tif file, it seems like the code executes properly in terminal, but when I try to view the images in my project viewer, only a white box with a partial error message is displayed. When I check the logs, the entire error message reads as follows:

REQUEST: filename=/myamiImages/leginon/22aug30z/rawdata/22aug30z_A1_2to4_1to100_0050.mrc&info=1&cache=0&oformat=JSON&frame=0
Running Read(139877296586448,{'info': True, 'frame': 0, 'filename': '/myamiImages/leginon/22aug30z/rawdata/22aug30z_A1_2to4_1to100_0050.mrc'})
Running Format(139877296586064,{'oformat': 'JSON', 'rgb': False, 'overlay': '', 'overlaycolor': 0.1})
REQUEST: pipeline=standard&filename=/myamiImages/leginon/22aug30z/rawdata/22aug30z_A1_2to4_1to100_0050.mrc&shape=512x0&scaletype=stdev&scalemin=-5&scalemax=5&oformat=JPEG&overlaycolor=0.35
Running Read(139877296586192,{'info': False, 'filename': '/myamiImages/leginon/22aug30z/rawdata/22aug30z_A1_2to4_1to100_0050.mrc'})
REDUX ERROR 1661976961 reading entire stack not allowed: /myamiImages/leginon/22aug30z/rawdata/22aug30z_A1_2to4_1to100_0050.mrc
Traceback (most recent call last):
File "/usr/local/anaconda2/lib/python2.7/site-packages/redux/server.py", line 37, in run_process
result = pipeline.process(**kwargs)
File "/usr/local/anaconda2/lib/python2.7/site-packages/redux/pipeline.py", line 144, in process
result = pipe(result)
File "/usr/local/anaconda2/lib/python2.7/site-packages/redux/pipe.py", line 163, in call
return self.run(input, **self.kwargs)
File "/usr/local/anaconda2/lib/python2.7/site-packages/redux/pipes/read.py", line 61, in run
raise ValueError('reading entire stack not allowed: %s' % (filename,))
ValueError: reading entire stack not allowed: /myamiImages/leginon/22aug30z/rawdata/22aug30z_A1_2to4_1to100_0050.mrc

At my institution, software installs are all managed by the IT department. The guy who has been helping me with my install initially had issues getting EMAN to install correctly, but thinks that he got that resolved and now is unsure of what to do next.

Let me know if you need anything else from me to help solve this issue,
Thanks!


Replies (15)

RE: Issue viewing uploaded images - Added by Anchi Cheng over 1 year ago

Is the tif file an mrc stack ? The error suggests that the conversion made it into a 3D mrc.

RE: Issue viewing uploaded images - Added by Douglas Walker over 1 year ago

The tif Files should just be 2D images, but the upload did try to convert them to mrc files.
I tried uploading 50 different .tifs and when I look at the project which holds the files, there are 50 different uploaded files that all show that error, so I don't think it tried to combined them into a 3D image.

RE: Issue viewing uploaded images - Added by Anchi Cheng over 1 year ago

dm3 format upload was written quite a few years ago. There might be change in the format that the library used is not doing right. Can you give us the camera, Lattitude and GMS version where it is produced from ? I can use that to ask Gatan.

If you can make a small tif file that does not convert correctly and attach here or create an Issue on redmine, I can ask others to look into solutions.

Thanks.

Anchi

RE: Issue viewing uploaded images - Added by Douglas Walker over 1 year ago

I will follow up with information on the camera at a later point, but in the meantime, I will attach one of my tif files here. Does it effect things that I made these tifs by converting .jpgs?

Thanks,

RE: Issue viewing uploaded images - Added by Anchi Cheng over 1 year ago

I want to check the header with this, so converted by another program is not good. Please send me a file from the same program as the ones you try to upload. Thanks

RE: Issue viewing uploaded images - Added by Douglas Walker over 1 year ago

The file I sent is one of the ones that I tried to upload.
I converted them by changing the extension because Appion will not upload .jpgs

As far as the questions you asked previously:
As answered by my facility manager:
"Our camera is a Gatan UltraScan 1000. Our camera software is Gatan Digital Micrograph 2.12. It predates when Gata started calling it GMS (Gatan Microscopy Suite). I am not sure what Lattitude is? Do you mean exposure latitude? If so, I gain normalize our camera at 4000 counts per pixel per second."

Let me know if you need anything else,

RE: Issue viewing uploaded images - Added by Douglas Walker over 1 year ago

Hello,
Is there anything more on this front? Anything that I should do to try to get this working for me?

RE: Issue viewing uploaded images - Added by Anchi Cheng over 1 year ago

Sorry for the delay. I just opened the file you sent now and that reveals a big problem. images we want to upload is in general represents only pixel values from the camera. Therefore, they would be a grey scale image without any other color nor scale bar. That is not so of your tiff image. I would not recommend trying upload something like this.

RE: Issue viewing uploaded images - Added by Douglas Walker over 1 year ago

Ok, so you think that that would solve the whole issue if I were to use images that were only grey-scaled?
Is it the same problem with my .dm3 files? Or is there a different problem here?

RE: Issue viewing uploaded images - Added by Douglas Walker over 1 year ago

Is my problem with the dm3 files the same? Is the EMAN failing because my dm3 files contain color? If not, what can I do to try to fix this?

RE: Issue viewing uploaded images - Added by Gabriel Lander over 1 year ago

I believe EMAN1.9 is being used to convert anything that is not an mrc or tif formatted file. Can you try running the command:
proc2d <input file> out.mrc edgenorm flip mrc
with one of your dm3 files and let us know if it converts properly? You can check that it converted by using an MRC viewer (e.g. boxer from EMAN)

RE: Issue viewing uploaded images - Added by Douglas Walker over 1 year ago

It's telling me that there is no such command as proc2d

RE: Issue viewing uploaded images - Added by Gabriel Lander over 1 year ago

I see. That's the issue then - your options are to install EMAN 1.9 on the machine that is performing the upload, or find a program that can batch convert your files to mrc & upload those instead so that it bypasses the EMAN 1.9 requirement.

RE: Issue viewing uploaded images - Added by Douglas Walker over 1 year ago

When my IT guy tried doing the whole appion install, he sent me this:

"It looks like the Appion autoinstaller didn't install EMAN, and the instructions for installing EMAN seem to not work correctly on CentOS 7."

Do you know a better, reliable way to intall EMAN?

RE: Issue viewing uploaded images - Added by Gabriel Lander over 1 year ago

I've never had to install EMAN, this has always been taken care of by the IT folks. EMAN2 may be an option since it's udpated, then you could use that to manually convert your dm3 files to mrc for upload.

    (1-15/15)