DDD raw frame file transfer » History » Revision 2
Revision 1 (Anchi Cheng, 06/08/2012 09:45 AM) → Revision 2/20 (Jim Pulokas, 06/08/2012 03:00 PM)
h1. DDD raw frame file transfer
# Introduction:
Due to the limited disk space on the camera host computer, we wish to transfer the raw frame data to a location with more storage space.
The following assumes the camera host is a Windows system and the destination is a linux system. The "rawtransfer.py" script will run continuously in the background on the linux system, independent from Leginon. It will periodically check the Leginon database to see if any new raw frames have been acquired by Leginon, then pull them over the network.
# Initial setup of your systems:
Camera host (Windows): the folder where your raw frames are stored must be set to a shared folder, without a password in our example.
Destination host (Linux): The samba-client package must be installed in order to mount the shared Windows folder
# Mount the decamera computer D:\\RawFrames\DE12 on a computer that has direct access to the network drive you store Leginon data and Leginon.
For example:
<pre>
mount //decamera/DE12 /your_base_directory/dde
</pre>
# Start rawtransfer script in the background
It is currently set up to check every 30 seconds if any new files have been stored on the Direct Electron PC shared folder. If it finds new files, it will check the database to see if they are connected to a Leginon image. If it is from Leginon, then it will rsync the folder to the rawdata folder for that session. The name of the folder will be something like: 11nov28_00001sq_00001_hl_00001_en.frames (based on the same name as the Leginon mrc image). Then it will remove the original folder. It looks like it can copy one frame per second, but this could vary depending on network traffic.
The script is in leginon directory. Use a command like this:
<pre>RAW_FRAMES_PATH=/your_base_directory/dde nohup $PYTHONSITEPKG/leginon/rawtransfer.py > rawtransfer.log &</pre>