Appion

I thought I'd switch to redmine from email for my similar issue. I'm on trunk with numpil.py, and I'm still getting this error after svn updating.

manualpicker.py --runname=manrun1 --rundir=/lustre/cryo/lustre/appiondata/16jun20a/extract/manrun1 --commit -m 16jun20a_00006sq_00004hl_00011ex.mrc,16jun20a_00006sq_00004hl_00010ex.mrc,16jun20a_00006sq_00004hl_00009ex.mrc --projectid=1 --session=16jun20a --no-rejects --no-wait --continue --median=2 --lowpass=15 --highpass=0 --planereg --bin=4 --diam=200 --pixlimit=4.0 --shape=plus --expid=11321 --jobtype=manualpicker
  File "/panfs/storage.local/imb/stagg/software/myamiss/pyami/numpil.py", line 143, in fromstring
    return getattr(Image, getPilFromStringFuncName())(data,decoder_name, *args)
  File "/panfs/storage.local/imb/stagg/software/myamiss/pyami/numpil.py", line 143, in fromstring
    return getattr(Image, getPilFromStringFuncName())(data,decoder_name, *args)
  File "/panfs/storage.local/imb/stagg/software/myamiss/pyami/numpil.py", line 143, in fromstring
    return getattr(Image, getPilFromStringFuncName())(data,decoder_name, *args)
...
 File "/panfs/storage.local/imb/stagg/software/myamiss/pyami/numpil.py", line 131, in getPilFromStringFuncName
    im = Image.new('1', (1,1))
  File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 1801, in new
    return Image()._new(core.fill(mode, size, color))
  File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 462, in _new
    new = Image()
RuntimeError: maximum recursion depth exceeded

I don't know what to say. A work around is to remove the replaced definition in pyami/numpil.py and put a flag specific to sites:

For those have to use fromstring remove this at the end of numpil.py so it retain its original definition.

Image2.frombytes = fromstring

See above.

Alex, Image.new('1', (1,1)) is called because it need to verify which attribute to use. If you can find out from which module it is doing Image.fromstring from, it would help me finding out how the confusion got started. In the mean time, I think the code removal I mentioned above will work for you.

What versions of Python, scipy, and PIL are you using? I recall the mention of pillow is that something that I need to install?

pillow is a fork of PIL that has replaces PIL for those use newer versions of python. You will either install the original PIL or pillow. The two should not coexist because in the python module import, they are called the same name.

I thought replacing the attributes by checking works, and it did work for us for manualpicker and manualmasker etc. but there are places that there is another import Image from PIL like in apProTomo2Aligner.py. Maybe it got confused again. It seems that in Alex's case scipy.misc.imsave uses PIL but could not distinguish the local numpil.py extended module from the original. I guess it is the problem of me saying

Image2 = Image

Neil, if you know how to distinguish PIL from pillow with a less harmful call like a version text, we should try that.

Hi Anchi, do not forget we need to override tostring and tobytes as well.

Here is the hack I did on my machine:

vi site-packages/PIL/Image.py

    def fromstring(self, *args, **kw):
        raise Exception("fromstring() has been removed. " +
                        "Please call frombytes() instead.")

    def fromstring(self, *args, **kw):
        return self.frombytes(*args, **kw)

You have to do it in two locations...

Unfortunately, this is not a great solution for general users. Is there a way to overwrite the function in Appion? Just so annoying, but CentOS 6 still uses the classic PIL (last real update was Dec 2006), we could just force users to upgrade to pillow, it is pretty straightforward.

I just upgraded to Pillow 3.2, and I'm getting a new error.

Traceback (most recent call last):
  File "/panfs/storage.local/imb/stagg/software/myamiss/appion/bin/manualpicker.py", line 733, in <module>
    imgLoop.run()
  File "/panfs/storage.local/imb/stagg/software/myamiss/appion/appionlib/appionLoop2.py", line 88, in run
    results = self.loopProcessImage(imgdata)
  File "/panfs/storage.local/imb/stagg/software/myamiss/appion/appionlib/particleLoop2.py", line 101, in loopProcessImage
    self.peaktree = filterLoop.FilterLoop.loopProcessImage(self, imgdata)
  File "/panfs/storage.local/imb/stagg/software/myamiss/appion/appionlib/filterLoop.py", line 83, in loopProcessImage
    peaktree = self.processImage(imgdata, self.filtarray)
  File "/panfs/storage.local/imb/stagg/software/myamiss/appion/bin/manualpicker.py", line 484, in processImage
    peaktree = self.runManualPicker(imgdata)
  File "/panfs/storage.local/imb/stagg/software/myamiss/appion/bin/manualpicker.py", line 651, in runManualPicker
    self.app.panel.openImageFile(imgpath)
  File "/panfs/storage.local/imb/stagg/software/myamiss/appion/bin/manualpicker.py", line 63, in openImageFile
    self.setImage(image.astype(numpy.float32))
  File "/panfs/storage.local/imb/stagg/software/myamiss/leginon/gui/wx/ImagePanel.py", line 316, in setImage
    self.setNumericImage(imagedata)
  File "/panfs/storage.local/imb/stagg/software/myamiss/leginon/gui/wx/ImagePanel.py", line 404, in setNumericImage
    self.setBitmap()
  File "/panfs/storage.local/imb/stagg/software/myamiss/leginon/gui/wx/ImagePanel.py", line 191, in setBitmap
    wximage = self.numpyToWxImage(self.imagedata)
  File "/panfs/storage.local/imb/stagg/software/myamiss/leginon/gui/wx/ImagePanel.py", line 226, in numpyToWxImage
    imagedata = Image.fromstring("L", (w, h), normarray.tostring())
  File "/panfs/storage.local/imb/stagg/software/myami_python/lib/python2.7/site-packages/PIL/Image.py", line 2077, in fromstring
    "Please call frombytes() instead.")
Exception: fromstring() has been removed. Please call frombytes() instead.

Alex threatened to assign this to me. HELP! I'm drowning in complaints right now. What can I do quickly to get this working?

We need to get this fixed, but I cannot do any more work on it until I defend next week. Consider me a lame duck assignee until then!

As per this link https://github.com/TileStache/TileStache/issues/237

I uninstalled Pillow 3.2 with pip, and then reinstalled with

sudo pip install Pillow==2.9.0

...and manual masker works on my machine now. Not sure how this will affect other programs. If we need other programs to use Pillow 3.2 I can write up a virtual environment for manual masker that'll use 2.9.0.

Hi Scott, You can do my edit above or use Carl's system to install older version.

OK, I did that, but now the recursion depth error is back.

OK. I fixed it (I think) by putting in a check to see if the Pillow version is greater than 3. Please test it at NYSBC. If it barfs maybe you can tune my conditional at the end of numpil.py to something that works for y'all.

Scott's fix works on my machine. It was already working on my machine though, so we'll have to keep an eye on it.

Appion-Protomo works again. Thanks Scott!

I just realized my conditional wasn't quite right for checking Pillow version. I made a small but significant change. Can y'all test again when you get a chance?

Hi all,
sorry I lost the thread on this - where does this bug stand? We're still getting a "fromstring" error in manualpicker.py:

Beginning Main Loop
... Pixel size: 0.637
... reading filtered image from mrc file
Traceback (most recent call last):
File "/gpfs/home/glander/myami/appion/bin/manualpicker.py", line 733, in <module>
imgLoop.run()
File "/gpfs/home/glander/myami/appion/appionlib/appionLoop2.py", line 88, in run
results = self.loopProcessImage(imgdata)
File "/gpfs/home/glander/myami/appion/appionlib/particleLoop2.py", line 101, in loopProcessImage
self.peaktree = filterLoop.FilterLoop.loopProcessImage(self, imgdata)
File "/gpfs/home/glander/myami/appion/appionlib/filterLoop.py", line 83, in loopProcessImage
peaktree = self.processImage(imgdata, self.filtarray)
File "/gpfs/home/glander/myami/appion/bin/manualpicker.py", line 484, in processImage
peaktree = self.runManualPicker(imgdata)
File "/gpfs/home/glander/myami/appion/bin/manualpicker.py", line 651, in runManualPicker
self.app.panel.openImageFile(imgpath)
File "/gpfs/home/glander/myami/appion/bin/manualpicker.py", line 63, in openImageFile
self.setImage(image.astype(numpy.float32))
File "/gpfs/home/glander/myami/leginon/gui/wx/ImagePanel.py", line 322, in setImage
self.setNumericImage(imagedata)
File "/gpfs/home/glander/myami/leginon/gui/wx/ImagePanel.py", line 410, in setNumericImage
self.setBitmap()
File "/gpfs/home/glander/myami/leginon/gui/wx/ImagePanel.py", line 191, in setBitmap
wximage = self.numpyToWxImage(self.imagedata)
File "/gpfs/home/glander/myami/leginon/gui/wx/ImagePanel.py", line 226, in numpyToWxImage
imagedata = Image.fromstring("L", (w, h), normarray.tostring())
File "/opt/applications/python/2.7.11/gnu/lib/python2.7/site-packages/PIL/Image.py", line 2077, in fromstring
"Please call frombytes() instead.")

Got a report from an outside user who has Pillow 3.1.2 and had to downgrade to 2.9.0 on his appion 3.2. I thought I mention it here.