Rawtransfer.py on K2 computer issue
Added by Harry Kao over 8 years ago
Hi,
On the K2 computer, the frame stacks are first saved to drive X:\raw_stacks\, with just time stamps as the filenames.
We would like to transfer the stacks to z:\frames. The z drive is a mount point from a Linux file server, via the Samba protocol.
Here is the command I used:
python rawtransfer.py --method=mv --source_path=X:\raw_statcks --camera_host=k2-computer --destination_head=Z:\frames > rawtransfer.log
(in Leginon.cfg, the Image path is z:\Image)
1. The frame stacks are moved to the 'frames' directory under the python site-packages directory where rawtransfer.py was called. The new but wrong path created is ....\frames\file_server\Image\project_date\.....
Again, we would like to have the stacks moved to z:\frames\project_date\... on the K2. On the linux file server, the path should be /fileserver/frames/project_date/.....2. Will Appion have trouble locating the stacks, due the the data path name having the PC syntax (z:\frames\project_date\...) . We use Appion for real time frame alignment on another Linux workstation.
Thank you.
Replies (3)
RE: Rawtransfer.py on K2 computer issue - Added by Anchi Cheng over 8 years ago
Your plan to run rawtransfer.py on K2 PC rather than linux server will need you to do some extra work. I have not made general implementation of the specific case you have. There are groups doing it this way but I don't have their custom code.
Let me answer the second part first: Assuming that you still launch the main part of Leginon on a linux box, leginon.cfg image path on there would be in linux syntex and saved to the database as such. Appion will not have trouble locating the frame stacks once it is transferred. The leginon.cfg on K2PC would have no effect.
As for matching the right directory to transfer the frames to as in your first question. It is a bit tricky. At some point of rawtransfer.py, it query the database to get the frame path and frame file name to be moved to. Since the frame path defined at the start of the Leginon session is in linux format as I said above, it will need to translate that. rawtransfer.py as is does not do so. The mapping will need to be done when frames_path before it is used in
# only process destination frames_path starting with chosen head
if sys.platform != 'win32' and not frames_path.startswith(dest_head):
For example,
frames_path = 'Z:'+frames_path.replace('/','\\')
In addition, the reference copying (new in 3.2 version) will have the same problem.
I will make some general code but I found some deeper problem in the general implementation while investigating this today so it will take some time to make it right.
RE: Rawtransfer.py on K2 computer issue - Added by Harry Kao over 8 years ago
Hi Anchi,
I modified the 'frames_path' the way you suggested, and the stacks are now saved to the correct path.
I am not clear about the 'reference copying'. Is this related to the reference/gain correction images?
Thank you.
Harry
RE: Rawtransfer.py on K2 computer issue - Added by Anchi Cheng over 8 years ago
Glad to hear that it works out.
Reference Copying is a feature that makes a copy of the bright/dark/norm images as well as defect lists into a folder with the frames along with the needed modification of orientation so that people doing their frame processing outside appion can have an easier start. It is also cleaner for archiving.
The feature was turned off in rawtransfer.py if running from Windows because I was concerned of access to the reference folders. In the way you set up, this looks possible, so you may want to activate it.