Bug #3529
openmemmap error
0%
Description
I got the following error when I tried to make a substack:
... averaging stack for summary web page
Traceback (most recent call last):
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/alignSubStack.py", line 293, in <module>
subStack.start()
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/alignSubStack.py", line 279, in start
apStack.averageStack(stack=oldstack,outfile=outavg,partlist=includeParticle)
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apStack.py", line 328, in averageStack
particles = imagic.read(stack)
File "/panfs/storage.local/imb/stagg/software/myami_dev/pyami/imagic.py", line 73, in read
a = readImagicData(pair['img'], header_dict, frame)
File "/panfs/storage.local/imb/stagg/software/myami_dev/pyami/imagic.py", line 43, in readImagicData
a = numpy.memmap(filename, dtype=dtype, mode='r', offset=start, shape=shape, order='C')
File "/usr/lib64/python2.7/site-packages/numpy/core/memmap.py", line 257, in new
mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start)
mmap.error: [Errno 12] Cannot allocate memory
Updated by Neil Voss about 9 years ago
I think I am using memmap incorrectly.
http://stackoverflow.com/questions/1661986/why-doesnt-pythons-mmap-work-with-large-files
We need to scrap memmap, I do not think 1 particle per iteration is ideal, but we can check system memory and do a chunk of particles at a time.
I already have the tools do implement the functionality, but I do not have the time. Probably a good idea to create a class for handling stack files.
Neil
P.S. Scott you should add yourself as a watcher on these bugs as well.
Updated by Scott Stagg about 9 years ago
Well what should we do? Appion trunk and branch3.2 is broken almost entirely with the current setup.
Updated by Neil Voss about 9 years ago
So what is using the new approc2d right now? Makestack should be okay. It is only crashing on the average stack step correct?
Updated by Neil Voss about 9 years ago
I changed averageStack back to EMAN1 in r19126
Updated by Gabriel Lander almost 9 years ago
since we're back to using eman1 I added code to write an LST file so this will work with virtual substacks