Project

General

Profile

Actions
Copy link

Bug #4824

closed

reduxd cache does not work with python-fs version 2.0 (CentOS 7)

Added by Neil Voss over 7 years ago. Updated over 7 years ago.

Status:
Won't Fix or Won't Do
Priority:
Normal
Assignee:
Neil Voss
Category:
-
Target version:
-
Start date:
03/10/2017
Due date:
% Done:

0%

Estimated time:
Affected Version:
Appion/Leginon 3.3
Show in known bugs:
No
Workaround:

Description

Work arounds:
  • disable cache in /etc/myami/reduxd.cfg
  • downgrade to older python-fs, version 0.5.4

Test setup that works

sudo pip uninstall -y fs
sudo pip install fs==0.5.4
sudo /etc/init.d/reduxd restart

Install latest fs version (2.0 as of March 10, 2017)

sudo pip uninstall -y fs
sudo pip install fs
sudo /etc/init.d/reduxd restart


import redux.server
  File "/usr/lib64/python2.7/site-packages/redux/server.py", line 16, in <module>
import redux.pipeline
  File "/usr/lib64/python2.7/site-packages/redux/pipeline.py", line 20, in <module>
results = redux.cache.Cache(disk_cache_path, disk_cache_size, size_max=mem_cache_size)
  File "/usr/lib64/python2.7/site-packages/redux/cache.py", line 19, in __init__
self.diskcache = cachefs.CacheFS(disk_cache_path, disk_cache_size)
  File "/usr/lib64/python2.7/site-packages/redux/cachefs.py", line 108, in __init__
files = list(self.walkfiles())
AttributeError

that issue is easily fixed in cachefs.py

-               files = list(self.walkfiles())
+               try:
+                       files = list(self.walkfiles())
+               except AttributeError:
+                       files = list(self.walk.files())

but then it gets much worse:

Running Read(47509968,{'info': False, 'filename': '/emg/data/leginon/06jul12a/rawdata/06jul12a_35gr_63sq_12hl_04em.mrc'})
Running Shape(47510352,{'shape': (1024, 1024)})
REDUX ERROR 1489182773 makedir() got an unexpected keyword argument 'recursive'
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/redux/server.py", line 37, in run_process
    result = pipeline.process(**kwargs)
  File "/usr/lib64/python2.7/site-packages/redux/pipeline.py", line 147, in process
    results.put(done, result)
  File "/usr/lib64/python2.7/site-packages/redux/cache.py", line 39, in put
    return self._put(pipeline, result)
  File "/usr/lib64/python2.7/site-packages/redux/cache.py", line 34, in _put
    self.file_put(pipeline, result)
  File "/usr/lib64/python2.7/site-packages/redux/cache.py", line 79, in file_put
    self.diskcache.makedir(path, recursive=True, allow_recreate=True)
TypeError: makedir() got an unexpected keyword argument 'recursive'

and many other errors follow, it is going to take a big rewrite.

Actions
Copy link

Also available in: Atom PDF