Project

General

Profile

Actions
Copy link

Bug #4824

closed

reduxd cache does not work with python-fs version 2.0 (CentOS 7)

Added by Neil Voss over 7 years ago. Updated over 7 years ago.

Status:
Won't Fix or Won't Do
Priority:
Normal
Assignee:
Neil Voss
Category:
-
Target version:
-
Start date:
03/10/2017
Due date:
% Done:

0%

Estimated time:
Affected Version:
Appion/Leginon 3.3
Show in known bugs:
No
Workaround:

Description

Work arounds:
  • disable cache in /etc/myami/reduxd.cfg
  • downgrade to older python-fs, version 0.5.4

Test setup that works

sudo pip uninstall -y fs
sudo pip install fs==0.5.4
sudo /etc/init.d/reduxd restart

Install latest fs version (2.0 as of March 10, 2017)

sudo pip uninstall -y fs
sudo pip install fs
sudo /etc/init.d/reduxd restart


import redux.server
  File "/usr/lib64/python2.7/site-packages/redux/server.py", line 16, in <module>
import redux.pipeline
  File "/usr/lib64/python2.7/site-packages/redux/pipeline.py", line 20, in <module>
results = redux.cache.Cache(disk_cache_path, disk_cache_size, size_max=mem_cache_size)
  File "/usr/lib64/python2.7/site-packages/redux/cache.py", line 19, in __init__
self.diskcache = cachefs.CacheFS(disk_cache_path, disk_cache_size)
  File "/usr/lib64/python2.7/site-packages/redux/cachefs.py", line 108, in __init__
files = list(self.walkfiles())
AttributeError

that issue is easily fixed in cachefs.py

-               files = list(self.walkfiles())
+               try:
+                       files = list(self.walkfiles())
+               except AttributeError:
+                       files = list(self.walk.files())

but then it gets much worse:

Running Read(47509968,{'info': False, 'filename': '/emg/data/leginon/06jul12a/rawdata/06jul12a_35gr_63sq_12hl_04em.mrc'})
Running Shape(47510352,{'shape': (1024, 1024)})
REDUX ERROR 1489182773 makedir() got an unexpected keyword argument 'recursive'
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/redux/server.py", line 37, in run_process
    result = pipeline.process(**kwargs)
  File "/usr/lib64/python2.7/site-packages/redux/pipeline.py", line 147, in process
    results.put(done, result)
  File "/usr/lib64/python2.7/site-packages/redux/cache.py", line 39, in put
    return self._put(pipeline, result)
  File "/usr/lib64/python2.7/site-packages/redux/cache.py", line 34, in _put
    self.file_put(pipeline, result)
  File "/usr/lib64/python2.7/site-packages/redux/cache.py", line 79, in file_put
    self.diskcache.makedir(path, recursive=True, allow_recreate=True)
TypeError: makedir() got an unexpected keyword argument 'recursive'

and many other errors follow, it is going to take a big rewrite.

Actions
Copy link
 #1

Updated by Neil Voss over 7 years ago

  • Affected Version changed from Appion/Leginon 3.2 to Appion/Leginon 3.3

Hi Scott,

I am trying to get a full install of appion working in CentOS 7.

Is this the reason that you could not get redux to work with CentOS 7, because besides this it is working fine for me. Using php v5.4.

Actions
Copy link
 #2

Updated by Scott Stagg over 7 years ago

I don't remember. There were lots of things broken for me. One of the things I had to do was disable one of the FFT libraries. It tries to choose from two or something like that. I was never able to get caching working, like ever. I never bothered to figure out why. Our images load fast enough without caching so I don't worry about it. I'm going to include Donny here in case he has anything to add.

Actions
Copy link

Also available in: Atom PDF