Bug #1401
closeddogpicker fails with scipy 0.9.0
0%
Description
This was from Ben Eisenbraun at Harvard:
ApDogPicker.py --image=groel-stain.mrc --outfile=picks.txt --invert --thresh=0.7 --diam=10
... Time stamp: 11aug18m18
... Function name: ApDogPicker
... Appion directory: /nfs/programs/x86_64-linux/dogpicker/0.2
!!! WARNING: Not committing data to database
... Writing function log to: ApDogPicker.log
particle pixel sizes= [ 3.93075689 6.36009825]
diameter list= [ 7.86199999 12.72000027]
Traceback (most recent call last):
File "/programs/x/dogpicker/0.2/ApDogPicker.py", line 164, in <module>
dogpicker.start()
File "/programs/x/dogpicker/0.2/ApDogPicker.py", line 125, in start
msg=True, bin=1)
File "/nfs/programs/x86_64-linux/dogpicker/0.2/appionlib/apPeaks.py", line 95,
in findPeaksInMap
varyThreshold(imgmap, thresh, maxsize)
File "/nfs/programs/x86_64-linux/dogpicker/0.2/appionlib/apPeaks.py", line 331
, in varyThreshold
blobtree, percentcov = findBlobs(ccmap, thresh, maxsize=maxsize)
File "/nfs/programs/x86_64-linux/dogpicker/0.2/appionlib/apPeaks.py", line 397
, in findBlobs
maxsize, minsize, maxmoment, elim, summary)
File "/nfs/programs/x86_64-linux/dogpicker/0.2/pyami/imagefun.py", line 338, i
+n find_blobs
File "/nfs/programs/x86_64-linux/dogpicker/0.2/pyami/imagefun.py", line 267, i
+n scipyblobs
stds = scipy.ndimage.standard_deviation(im,labels,range(1,n+1))
File "/programs/x/python/2.7/lib/python2.7/site-packages/scipy/ndimage/measure
+ments.py", line 649, in standard_deviation
return numpy.sqrt(variance(input, labels, index))
File "/programs/x/python/2.7/lib/python2.7/site-packages/scipy/ndimage/measure
+ments.py", line 596, in variance
count, sum, sum_c_sq = _stats(input, labels, index, centered=True)
File "/programs/x/python/2.7/lib/python2.7/site-packages/scipy/ndimage/measure
+ments.py", line 434, in _stats
sums_c = _sum_centered(labels.ravel())
File "/programs/x/python/2.7/lib/python2.7/site-packages/scipy/ndimage/measure
+ments.py", line 408, in _sum_centered
centered_input = input - means[labels]
ValueError: operands could not be broadcast together with shapes (512,512)
(262144)
$ python --version
Python 2.7
$ python -c "import scipy; print scipy.version.version"
0.9.0rc1Files
Updated by Jim Pulokas over 13 years ago
Neil tested on Fedora 14 (scipy 0.9.0) and gets the same error.
Jim tested on Ubuntu 11.04 (scipy 0.8.0) and does not get the error.
Makes is seem like a new scipy thing.
Is it only a problem if using the individually packaged dogpicker? Maybe the trunk works?
Updated by Neil Voss over 13 years ago
I tried it in the trunk too, and got the same error in Fedora 15 (not 14). It is important to note that this bug affects Leginon too.
Updated by Scott Stagg over 13 years ago
I have a related (I think) update. I am getting an error with tempateCorrelator that I believe I have tracked down to scipy. When I run templateCorrelator, I get the following error:
Traceback (most recent call last):
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/templateCorrelator.py", line 190, in <module>
imgLoop.run()
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/appionLoop2.py", line 77, in run
self.loopCommitToDatabase(imgdata)
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/particleLoop2.py", line 134, in loopCommitToDatabase
runname=self.params['runname'], msg=(not self.params['background']))
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apParticle.py", line 265, in insertParticlePeaks
particlesq.insert()
File "/panfs/storage.local/imb/stagg/software/myami_dev/sinedon/data.py", line 390, in insert
db.insert(self, **kwargs)
File "/panfs/storage.local/imb/stagg/software/myami_dev/sinedon/dbdatakeeper.py", line 243, in insert
self._insert(newdata, force=force)
File "/panfs/storage.local/imb/stagg/software/myami_dev/sinedon/dbdatakeeper.py", line 281, in _insert
raise InsertError(e.args[-1])
sinedon.dbdatakeeper.InsertError: Unknown column 'nan' in 'field list'
It seems that the program is trying to insert 'nan' into an incompatible field in the db. I printed out the query in the apParticle library before it inserts, and this is what I get:
correlation 0.165345197916 <type 'numpy.float64'> peakmoment 5.83389095603 <type 'numpy.float64'> peakstddev 0.00204974048294 <type 'numpy.float64'> peakarea 5 <type 'numpy.int64'> label templ181 <type 'str'> correlation 0.165707156062 <type 'numpy.float64'> peakmoment 1.0 <type 'float'> peakstddev nan <type 'numpy.float64'> peakarea 1 <type 'numpy.int64'> label templ181 <type 'str'>
Notice stddev is nan in the second query. This value comes from blobfinder in imagefun. I can only think that it is a scipy version problem. I am using scipy 0.8 on this test. I didn't get this error a few months ago when I was using 0.6. Interestingly, dogpicker works for me which seems weird because I would think that it uses the same blob finding algorithm. On, the other hand, I haven't done extensive testing.
Is there a plan for what to do here? Should I try to revert to an earlier scipy?
Updated by Anchi Cheng over 12 years ago
- Status changed from New to Won't Fix or Won't Do
Updated by Jim Pulokas over 12 years ago
I created a mini repository that you can download and install locally on your machine to make installing scipy 0.10.1 very clean without dependency issues. Download the attached tar.bz file and unpack it with
tar jxf ubuntu-scipy-0.10.1.tar.bzPut it somewhere where you will not loose it, like /usr/local, or leave it in your home directory if you trust that. Then go into /etc/apt/sources.list.d/ and create a new file, called scipy-0.10.1.list or whatever with a line that points to your new repository. For example:
deb file:///somewhere/whatever/ubuntu-scipy-0.10.1 ./Then update apt and install scipy:
sudo apt-get update sudo apt-get install python-scipy