Bug #2912
closed
Added by Scott Stagg over 10 years ago.
Updated about 9 years ago.
Affected Version:
Appion/Leginon 3.1.0
Description
We are having an issue with RCT. I am currently on the trunk distribution. When we go to test the region finder (click the acquisition icon in the RCT node), we are getting an error where Leginon says 'libCV.FindRegions failed'. I traced this down to the rctacquisition.py module in the testAcquire function. I worked around the error by commenting out the try/except statement and just executing FindRegions on it own line. See below:
# find regions
minsize = self.settings['minsize']
maxsize = self.settings['maxsize']
timeout = 300
regions, image = libCVwrapper.FindRegions(im, minsize, maxsize)
print "This is modified!!!!!!!!!!!!!!!"
self.logger.info('running libCV.FindRegions, timeout = %d' % (timeout,))
#try:
# regions,image = pyami.timedproc.call('leginon.libCVwrapper', 'FindRegions', args=(im,minsize,maxsize), timeout=timeout)
#except:
# self.logger.error('libCV.FindRegions failed')
# regions = []
# image = None
Somehow the timer or the try is messing things up. Do y'all have this problem at Scripps?
Files
Just found the same problem in the the rctacquisition.py trackStage function:
result = libCVwrapper.MatchImages(arrayold, arraynew, minsize, maxsize)
#try:
# result = pyami.timedproc.call('leginon.libCVwrapper', 'MatchImages', args=(arrayold, arraynew, minsize, maxsize), timeout=timeout)
# self.logger.info("result matrix= "+str(numpy.asarray(result*100, dtype=numpy.int8).ravel()))
#except:
# self.logger.error('libCV MatchImages failed')
# return None,None
#difftilt = degrees(abs(tilts[int(i)])-abs(tilts[int(i-1)]))
#result = self.apTiltShiftMethod(arrayold, arraynew, difftilt)
I haven't seen this error before. timeout = 300 seems to be ad hoc number. I've looked at pyami.timedproc.call and it seems that setting timeout = None, or removing timeout=timeout from pyami.timedproc.call would have similar effect.
- Assignee set to Scott Stagg
Scott.
How long did the libCV function run ? Is it possible that it exceeded the timeout ? I tested timedcall.call and it does not seem to have problem.
It may also be possible to be a pythonpath problem since we don't have this problem here. Is your pyami folder parallel to leginon and there is a pythonpath pointing to the parent directory ?
Hi guys,
I am experiencing the same error in Leginon 3.0. I first noticed it when trying to setup the RCT node. Clicking on the test acquire camera icon reproduced the "FindRegions failed" error.
If I apply the workaround suggested by Scott, it works and points are detected.
I also had to edit the MatchImages function to allow RCT to progress through the standard target submission process. It was previously giving a "MatchImages failed" error.
However, this step is very slow. Each MatchImage calculation (after each consecutive tilt step) can take varying amounts of times (up to minutes) and often hogs the cpus of the processing computer.
I hope this info helps with the debug.
Regards,
Jason
Jason,
Your information is very valuable. matchImages algorithm needs to check all permutations between features it finds in the two images. Just that part of operation is N^2 problem. We have two possibilities for the slowness:
1. libCV runs much slower on your computer than ours.
2. The images and parameters used give too many features to match efficiently.
To test the first case, I attach three files here. Put them in the same folder.
With the same environment that you run leginon, run the python file
python ./testMatchImages.py
My output on is
Image min and max are 0 589896
Image 1: keypoints: 765; descriptors: 1275
Image min and max are 0 597938
Image 2: keypoints: 651; descriptors: 1005
Found 460 matches between images
Found matrix that fit 68 out of the 460 matches.
Freeing all mem
1.00 -0.01 0.00
-0.01 0.94 0.00
-53.05 3.07 1.00
process time = 2.5 seconds
The computer I ran this with has
- cpu: Intel(R) Xeon(R) CPU 5160 @ 3.00GHz
- mem: 4 GB
- Centos 6.3
Thanks for the test script. As I suspected, there seems to be problem with my libCV instance. When I meant slow, I didn't mean seconds, I meant minutes.
Perhaps this is why it's timing out in the error described above.
My results are as follows:
Image min and max are 0 589896
Image 1: keypoints: 765; descriptors: 1171
Image min and max are 0 597938
Image 2: keypoints: 652; descriptors: 909
Found 433 matches between images
Found matrix that fit 79 out of the 433 matches.
Freeing all mem
1.00 -0.01 0.00
-0.00 0.94 0.00
-53.89 3.54 1.00
process time = 1284.1 seconds
On a "Intel(R) Pentium(R) 4 CPU 3.00GHz" with 3 gigs of memory.
Is there a way to get a more verbose output from MatchImage to see where it is going wrong?
O.K. This is most strange.
With Craig's help, I recompiled libcv with some debug prints and then found that it got stuck just like you two do. We have been going back and forth with the change applied in r17816. Issue #1291 was resolved by using AffineTransformImage function in place of SeparableAffineTransform. Then Issue #1500 was resolved by changing it back. Now we have this problem. I replace the current SeparableAffineTransform call with AffineTransformImage, and then it runs and fast.
Craig said it means that it hits a bug that wasn't encountered before. I wonder if it has to do with compiling with different compilers. He promised to look into it, but I don't know how long it will be.
For now, revert the change in r17816. It might be just what you need.
I changed SeparableAffineTransform(source,patch,TR,IT) back to AffineTransformImage(source,patch,TR,IT) in csift.c before recompiling and it's much faster:
Image min and max are 0 589896
Image 1: keypoints: 765; descriptors: 1273
Image min and max are 0 597938
Image 2: keypoints: 652; descriptors: 1005
Found 457 matches between images
Found matrix that fit 65 out of the 457 matches.
Freeing all mem
1.00 -0.00 0.00
-0.01 0.94 0.00
-53.36 1.19 1.00
process time = 5.4 seconds
I will test it in practise when I have machine time next week.
Thanks!
- Status changed from New to Closed
committed the revert to AffineTransformImage in trunk (r18735) and 3.1 (r18736) since this same solution fixes the problem at OIST as well.
Found another case in which this error happens at U Washington.
The source of the problem is related to permission of the file. They put all leginon files in a path that is included in PYTHONPATH environment variable. However, the files have permission
-rw-r--r-- 1 root root 23571 Apr 8 12:30 rctacquisition.py
pyami.timedproc.call uses subprocess which requires the file it calls to have executable permission. Changing all files *.py in that leginon and pyami directory owned by root to mode 755 solved the problem.
Equivalent test script for openCV version is attached.
Test result on a moderate poweful computer bought in 2012
33920 keypoints in image1, 33394 keypoints in image2
33920 preliminary matches
y-shift: -58 pixels with 3057.30103574 votes
526 matches
[[ 1.00169064e+00 -1.31436628e-02 0.00000000e+00]
[ -2.79313522e-03 9.31863128e-01 0.00000000e+00]
[ -5.77252661e+01 9.57291887e+00 1.00000000e+00]]
process time = 25.5 seconds
- Related to Task #3546: make autoinstalled rctacquisition.py executable added
- Related to Feature #3020: rct feature matching using opencv added
Also available in: Atom
PDF