Project

General

Profile

Forums » Appion and Leginon Web tools »

Redux problems

Added by Christopher Lilienthal over 10 years ago

Hello,

I have been trying to figure this issue out for a few days and I can't make any progress so I am hoping someone has an insight.

On the web server I've installed the necessary packages but when trying to view the test dataset images I see in the image placeholder: "Redux failed to return anything". The redux log file contains the following:

*****************************************
redux server started: Sat Jul  5 21:59:52 2014
subversion revision: None
host: 127.0.0.1,  port: 55123

REQUEST: filename=/var/www/html/myamiweb/test/img/08mar13e_00010gr_00004sq_v02_00005hl.mrc&info=1&cache=0&oformat=JSON&frame=0
Running Read(32846288,filename=/var/www/html/myamiweb/test/img/08mar13e_00010gr_00004sq_v02_00005hl.mrc,frame=0,info=True)
Running Format(32846224,oformat=JSON,overlay=,overlaycolor=0.1,rgb=False)
REQUEST: pipeline=standard&filename=/var/www/html/myamiweb/test/img/08mar13e_00010gr_00004sq_v02_00005hl.mrc&shape=512x512&scaletype=stdev&scalemin=-5&scalemax=5&oformat=JPEG&overlaycolor=0.35
IN MEMORY: Format(32846352,oformat=JPEG,overlay=,overlaycolor=0.35,rgb=False)

REQUEST: filename=/var/www/html/myamiweb/test/img/08mar13e_00010gr_00004sq_v02_00005hl.mrc&info=1&cache=0&oformat=JSON&frame=0
Running Read(32846160,filename=/var/www/html/myamiweb/test/img/08mar13e_00010gr_00004sq_v02_00005hl.mrc,frame=0,info=True)
Running Format(32846352,oformat=JSON,overlay=,overlaycolor=0.1,rgb=False)
REQUEST: pipeline=standard&filename=/var/www/html/myamiweb/test/img/08mar13e_00010gr_00004sq_v02_00005hl.mrc&shape=512x512&scaletype=stdev&scalemin=-5&scalemax=5&oformat=JPEG&overlaycolor=0.35
IN MEMORY: Format(32846416,oformat=JPEG,overlay=,overlaycolor=0.35,rgb=False)

With my data that was uploaded previously I see the following message:

Redux Error (long float) can't write str to text stream

and the redux.log gives:

REQUEST: filename=/lana/leginon/14may30x/rawdata/14may30x_tilt197.dm3.mrc&info=1&cache=0&oformat=JSON&frame=0
Running Read(32820944,filename=/lana/leginon/14may30x/rawdata/14may30x_tilt197.dm3.mrc,frame=0,info=True)
Running Format(32820624,oformat=JSON,overlay=,overlaycolor=0.1,rgb=False)
REQUEST: pipeline=standard&filename=/lana/leginon/14may30x/rawdata/14may30x_tilt197.dm3.mrc&shape=512x512&scaletype=stdev&scalemin=-5&scalemax=5&oformat=JPEG&overlaycolor=0.35
NOT IN MEMORY: Format(32821008,oformat=JPEG,overlay=,overlaycolor=0.35,rgb=False)

NOT IN MEMORY: Scale(32821072,scalemax=5.0,scalemin=-5.0,scaletype=stdev)

NOT IN MEMORY: Shape(32820624,shape=(512, 512))

NOT IN MEMORY: Read(32820944,filename=/lana/leginon/14may30x/rawdata/14may30x_tilt197.dm3.mrc,info=False)

Running Read(32820944,filename=/lana/leginon/14may30x/rawdata/14may30x_tilt197.dm3.mrc,info=False)
Running Shape(32820624,shape=(512, 512))
1404617128.01
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/redux/server.py", line 39, in run_process
    result = pipeline.process(**kwargs)
  File "/usr/lib/python2.6/site-packages/redux/pipeline.py", line 147, in process
    results.put(done, result)
  File "/usr/lib/python2.6/site-packages/redux/cache.py", line 40, in put
    return self._put(pipeline, result)
  File "/usr/lib/python2.6/site-packages/redux/cache.py", line 35, in _put
    self.file_put(pipeline, result)
  File "/usr/lib/python2.6/site-packages/redux/cache.py", line 82, in file_put
    final_pipe.put_result(f, result)
  File "/usr/lib/python2.6/site-packages/redux/pipe.py", line 90, in put_result
    cPickle.dump(result, f, cPickle.HIGHEST_PROTOCOL)
  File "/usr/lib64/python2.6/io.py", line 1515, in write
    s.__class__.__name__)
TypeError: can't write str to text stream
  • I have the reduxd server running as root and the redux config file is:
# cat /etc/myami/redux.cfg
## redux config file

[server]
host: localhost
port: 55123

# Enable/disable the cache by setting enable = yes or no.
# Configure disk cache path and size
# Configure memory cache size
# Sizes are in MB

[cache]
enable: yes
path: /var/cache/myami/redux/
disksize:  500
memsize: 500

# default uses current directory... better to specify full path
[log]
file: /var/log/redux/redux.log

Any help would be much appreciated.


Replies (4)

RE: Redux problems - Added by Anchi Cheng over 10 years ago

Most likely the uploaded mrc file from dm is not MRC original format, which caused the error you see. Uploading only make copy of the file, not making the header conversion.

Could you upload the problem dm3.mrc file in this thread ? We will see what we can do with that.

RE: Redux problems - Added by Christopher Lilienthal over 10 years ago

Attached is one of the files giving me the issue. I converted it using EMAN2. The other problems were from the test images but they did not generate any errors…

Thanks for taking a look!

tilt197.dm3.mrc (64 MB) tilt197.dm3.mrc problem file

RE: Redux problems - Added by Christopher Lilienthal over 10 years ago

In case this helps…
I am using release 18363 on RHEL Server 6.5
I tried the redux test on the Install Redux page and this is what I received while logged in as root:

redux --filename=/usr/local/src/myami/redux/gst/test.jpg --oformat=PNG > /usr/local/src/myami/test.png

*** Using custom copy of fftw3 wrapper
calc_fftw3: 2 CPUs found, setting threads=2
calc_fftw3: system wisdom imported
calc_fftw3: local wisdom imported
**power thread lock in effectNOT IN MEMORY: Format(33730960,oformat=PNG,overlay=,overlaycolor=0.1,rgb=False)
NOT IN MEMORY: Read(33730896,filename=/usr/local/src/myami/redux/gst/test.jpg,info=False)
Running Read(33730896,filename=/usr/local/src/myami/redux/gst/test.jpg,info=False)
Running Format(33730960,oformat=PNG,overlay=,overlaycolor=0.1,rgb=False)
Traceback (most recent call last):
  File "/usr/bin/redux", line 4, in <module>
    redux.client.run()
  File "/usr/lib/python2.6/site-packages/redux/client.py", line 141, in run
    result = client.process_kwargs(**kwargs)
  File "/usr/lib/python2.6/site-packages/redux/client.py", line 57, in process_kwargs
    return pl.process(**kwargs)
  File "/usr/lib/python2.6/site-packages/redux/pipeline.py", line 147, in process
    results.put(done, result)
  File "/usr/lib/python2.6/site-packages/redux/cache.py", line 40, in put
    return self._put(pipeline, result)
  File "/usr/lib/python2.6/site-packages/redux/cache.py", line 35, in _put
    self.file_put(pipeline, result)
  File "/usr/lib/python2.6/site-packages/redux/cache.py", line 82, in file_put
    final_pipe.put_result(f, result)
  File "/usr/lib/python2.6/site-packages/redux/pipes/format.py", line 86, in put_result
    f.write(result)
  File "/usr/lib64/python2.6/io.py", line 1515, in write
    s.__class__.__name__)

The test.png file is completely empty.

Thanks

RE: Redux problems - Added by Sargis Dallakyan over 10 years ago

I've tried tilt197.dm3.mrc and it worked fine for me in our test server: 

[root@fly ~]# redux --filename=/home/sargis/Downloads/tilt197.dm3.mrc --oformat=PNG > test.png
*** Using custom copy of fftw3 wrapper
calc_fftw3: 4 CPUs found, setting threads=4
**power thread lock in effectWarning:  You have not configured Images path in leginon.cfg!  Using current directory.
NOT IN MEMORY: Format(31886992,oformat=PNG,overlay=,overlaycolor=0.1,rgb=False)
NOT IN MEMORY: Read(31886928,filename=/home/sargis/Downloads/tilt197.dm3.mrc,info=False)
Running Read(31886928,filename=/home/sargis/Downloads/tilt197.dm3.mrc,info=False)
Running Format(31886992,oformat=PNG,overlay=,overlaycolor=0.1,rgb=False)
[root@fly ~]# more /etc/issue
CentOS release 6.4 (Final)

RHEL and CentOS are supposed to be compatible. I'll try to install redux on CentOS 6.5 to see if I can reproduce the error. Also, I can't find test.jpg in https://emg.nysbc.org/projects/appion/repository/show/trunk/redux/gst.

Redux works fine when I run it on example.png from there.

    (1-4/4)