Rawtransfer.py not transfering all files.
Added by Harry Kao over 8 years ago
Hi,
Lately, perhaps after upgrading to myami 3.2, rawtransfer.y has not been transferring all the raw stack images to the corresponding project folder. There are about 50 stacks left untransferred with the time-stamp file name, per day.
Is there a manual way to retransfer these orphan stacks? And how can we repair the transfer function?
Thank you.
Harry
Replies (3)
RE: Rawtransfer.py not transfering all files. - Added by Anchi Cheng over 8 years ago
First, we need to figure out if they do need to be transferred. We have them, too, but more like 10 per day. I suspect some non-frame saving function did not change frame saving parameter properly. rawtransfer.py does takes in all files in K2PC frames directory and then decide if each one needs to be transferred from database query. Therefore, it should not miss any you actually want as it will recheck again in the next round.
Attached is a simple python script to check if the frames belongs to any recorded images. Run it with your Leginon environment so that it has access to its library and database without extension name in the case of K2 mrc stack.
For example,
python ./check_framename.py 20160426_110253
If they are not in the database, I will have to investigate when they were saved. The code change in 3.2 was supposed to reduce unnecessary frame saving in Issue #3614.
| check_framename.py (902 Bytes) check_framename.py |
RE: Rawtransfer.py not transfering all files. - Added by Harry Kao over 8 years ago
Thank you for the script.
I ran the check_framename.py script on some of the untransferred stacks, and the return message is always "2016XXXXX.mrc is an orphan that has no Leginon record"
On the most recently captured stack, successful query returns:
frame saving option was True in that acquisition
Image 16apr25a_XXXX.mrc was acquired with the CameraEMData
RE: Rawtransfer.py not transfering all files. - Added by Anchi Cheng over 8 years ago
Good to hear.
Until I find the exact place where these are produced, please just remove them daily. Will update this thread when I figure it out.