Bug #2912
closedRCT error
0%
Description
We are having an issue with RCT. I am currently on the trunk distribution. When we go to test the region finder (click the acquisition icon in the RCT node), we are getting an error where Leginon says 'libCV.FindRegions failed'. I traced this down to the rctacquisition.py module in the testAcquire function. I worked around the error by commenting out the try/except statement and just executing FindRegions on it own line. See below:
# find regions minsize = self.settings['minsize'] maxsize = self.settings['maxsize'] timeout = 300 regions, image = libCVwrapper.FindRegions(im, minsize, maxsize) print "This is modified!!!!!!!!!!!!!!!" self.logger.info('running libCV.FindRegions, timeout = %d' % (timeout,)) #try: # regions,image = pyami.timedproc.call('leginon.libCVwrapper', 'FindRegions', args=(im,minsize,maxsize), timeout=timeout) #except: # self.logger.error('libCV.FindRegions failed') # regions = [] # image = None
Somehow the timer or the try is messing things up. Do y'all have this problem at Scripps?
Files
Updated by Scott Stagg over 10 years ago
Just found the same problem in the the rctacquisition.py trackStage function:
result = libCVwrapper.MatchImages(arrayold, arraynew, minsize, maxsize) #try: # result = pyami.timedproc.call('leginon.libCVwrapper', 'MatchImages', args=(arrayold, arraynew, minsize, maxsize), timeout=timeout) # self.logger.info("result matrix= "+str(numpy.asarray(result*100, dtype=numpy.int8).ravel())) #except: # self.logger.error('libCV MatchImages failed') # return None,None #difftilt = degrees(abs(tilts[int(i)])-abs(tilts[int(i-1)])) #result = self.apTiltShiftMethod(arrayold, arraynew, difftilt)
Updated by Sargis Dallakyan over 10 years ago
I haven't seen this error before. timeout = 300 seems to be ad hoc number. I've looked at pyami.timedproc.call and it seems that setting timeout = None, or removing timeout=timeout from pyami.timedproc.call would have similar effect.
Updated by Anchi Cheng about 10 years ago
- Assignee set to Scott Stagg
Scott.
How long did the libCV function run ? Is it possible that it exceeded the timeout ? I tested timedcall.call and it does not seem to have problem.
It may also be possible to be a pythonpath problem since we don't have this problem here. Is your pyami folder parallel to leginon and there is a pythonpath pointing to the parent directory ?
Updated by Jason van Rooyen about 10 years ago
Hi guys,
I am experiencing the same error in Leginon 3.0. I first noticed it when trying to setup the RCT node. Clicking on the test acquire camera icon reproduced the "FindRegions failed" error.
If I apply the workaround suggested by Scott, it works and points are detected.
I also had to edit the MatchImages function to allow RCT to progress through the standard target submission process. It was previously giving a "MatchImages failed" error.
However, this step is very slow. Each MatchImage calculation (after each consecutive tilt step) can take varying amounts of times (up to minutes) and often hogs the cpus of the processing computer.
I hope this info helps with the debug.
Regards,
Jason
Updated by Anchi Cheng about 10 years ago
- File arraynew.mrc arraynew.mrc added
- File arrayold.mrc arrayold.mrc added
- File testMatchImages.py testMatchImages.py added
- Assignee changed from Scott Stagg to Jason van Rooyen
Jason,
Your information is very valuable. matchImages algorithm needs to check all permutations between features it finds in the two images. Just that part of operation is N^2 problem. We have two possibilities for the slowness:
1. libCV runs much slower on your computer than ours.
2. The images and parameters used give too many features to match efficiently.
With the same environment that you run leginon, run the python file
python ./testMatchImages.py
My output on is
Image min and max are 0 589896 Image 1: keypoints: 765; descriptors: 1275 Image min and max are 0 597938 Image 2: keypoints: 651; descriptors: 1005 Found 460 matches between images Found matrix that fit 68 out of the 460 matches. Freeing all mem 1.00 -0.01 0.00 -0.01 0.94 0.00 -53.05 3.07 1.00 process time = 2.5 seconds
The computer I ran this with has
- cpu: Intel(R) Xeon(R) CPU 5160 @ 3.00GHz
- mem: 4 GB
- Centos 6.3
Updated by Jason van Rooyen about 10 years ago
Thanks for the test script. As I suspected, there seems to be problem with my libCV instance. When I meant slow, I didn't mean seconds, I meant minutes.
Perhaps this is why it's timing out in the error described above.
My results are as follows:
Image min and max are 0 589896
Image 1: keypoints: 765; descriptors: 1171
Image min and max are 0 597938
Image 2: keypoints: 652; descriptors: 909
Found 433 matches between images
Found matrix that fit 79 out of the 433 matches.
Freeing all mem
1.00 -0.01 0.00
-0.00 0.94 0.00
-53.89 3.54 1.00
process time = 1284.1 seconds
On a "Intel(R) Pentium(R) 4 CPU 3.00GHz" with 3 gigs of memory.
Is there a way to get a more verbose output from MatchImage to see where it is going wrong?
Updated by Anchi Cheng about 10 years ago
O.K. This is most strange.
With Craig's help, I recompiled libcv with some debug prints and then found that it got stuck just like you two do. We have been going back and forth with the change applied in r17816. Issue #1291 was resolved by using AffineTransformImage function in place of SeparableAffineTransform. Then Issue #1500 was resolved by changing it back. Now we have this problem. I replace the current SeparableAffineTransform call with AffineTransformImage, and then it runs and fast.
Craig said it means that it hits a bug that wasn't encountered before. I wonder if it has to do with compiling with different compilers. He promised to look into it, but I don't know how long it will be.
For now, revert the change in r17816. It might be just what you need.
Updated by Jason van Rooyen about 10 years ago
I changed SeparableAffineTransform(source,patch,TR,IT) back to AffineTransformImage(source,patch,TR,IT) in csift.c before recompiling and it's much faster:
Image min and max are 0 589896
Image 1: keypoints: 765; descriptors: 1273
Image min and max are 0 597938
Image 2: keypoints: 652; descriptors: 1005
Found 457 matches between images
Found matrix that fit 65 out of the 457 matches.
Freeing all mem
1.00 -0.00 0.00
-0.01 0.94 0.00
-53.36 1.19 1.00
process time = 5.4 seconds
I will test it in practise when I have machine time next week.
Thanks!
Updated by Anchi Cheng almost 10 years ago
- Status changed from New to Closed
committed the revert to AffineTransformImage in trunk (r18735) and 3.1 (r18736) since this same solution fixes the problem at OIST as well.
Updated by Anchi Cheng over 9 years ago
Found another case in which this error happens at U Washington.
The source of the problem is related to permission of the file. They put all leginon files in a path that is included in PYTHONPATH environment variable. However, the files have permission
-rw-r--r-- 1 root root 23571 Apr 8 12:30 rctacquisition.py
pyami.timedproc.call uses subprocess which requires the file it calls to have executable permission. Changing all files *.py in that leginon and pyami directory owned by root to mode 755 solved the problem.
Updated by Anchi Cheng about 9 years ago
- File testOpenCVMatchImages.py testOpenCVMatchImages.py added
Equivalent test script for openCV version is attached.
Test result on a moderate poweful computer bought in 2012
33920 keypoints in image1, 33394 keypoints in image2 33920 preliminary matches y-shift: -58 pixels with 3057.30103574 votes 526 matches [[ 1.00169064e+00 -1.31436628e-02 0.00000000e+00] [ -2.79313522e-03 9.31863128e-01 0.00000000e+00] [ -5.77252661e+01 9.57291887e+00 1.00000000e+00]] process time = 25.5 seconds
Updated by Anchi Cheng about 9 years ago
- Related to Task #3546: make autoinstalled rctacquisition.py executable added
Updated by Neil Voss over 7 years ago
- Related to Feature #3020: rct feature matching using opencv added