Leginon

Leginon philosophy is that we should provide the file transfer system that allow users to see the eer files named similarly to the sum image in their unix file system, not on Falcon server which has relatively limited space.

rawtransfer.py will do the filename change you want when it moves the movies parallel to your leginon/session/rawdata as frames/session/rawdata.

see https://emg.nysbc.org/projects/leginon/wiki/DDD_raw_frame_file_transfer

You can modify the leginon naming system, and rawtransfer.py should follow it.

Regarding where the eer files are saved. Advanced TEM Scripting restricts where the files it saves can go to. You can only have use directories under Z:\TEMScripting\EF-Falcon.

Anchi Cheng wrote:

Leginon philosophy is that we should provide the file transfer system that allow users to see the eer files named similarly to the sum image in their unix file system, not on Falcon server which has relatively limited space.

rawtransfer.py will do the filename change you want when it moves the movies parallel to your leginon/session/rawdata as frames/session/rawdata.

see https://emg.nysbc.org/projects/leginon/wiki/DDD_raw_frame_file_transfer

You can modify the leginon naming system, and rawtransfer.py should follow it.

Instead of mounting the camera offload server to our storage server, we copied a backup to use in rawtransfer.py in a similar fashion without having mount a network shared drive. We receive the following error:

$ rawtransfer.py --method=mv --source_path=/beebylab/data/myamiImages-raw --camera_host=glacios-9954637 --destination_head=/beebylab/data/myamiImages-frames --cleanup_delay_minutes=99999

Source path:  /beebylab/data/myamiImages-raw
Limit processing to destination frame path started with /beebylab/data/myamiImages-frames

Iterating...
**checking /beebylab/data/myamiImages-raw/20220829_20135332.eer
20220829_20135332.eer not from a saved image
...
...
...
**checking /beebylab/data/myamiImages-raw/20220829_20083310.eer
20220829_20083310.eer not from a saved image
Sleeping...

Is there an issue with the way we have tried to set this up?

Sorry for late reply.

Please check on your myamiweb image viewer image report (click on "i" when the image you want is displayed in the viewer). You should be able to expand the metadata tree and find camera->frames_name.
rawtransfer.py matches the name there with the file it finds on your camera storage server as mounted.

Since you have falcon4 running eer, you should also have in the data tree cramer->eer_frames = 1 (True)

You can adjust how self.frames_name is determined based on your customed settings. The key is to make pyscope.feicam.py getPreviousRawFramesName retrieves the same frames_name that corresponds to the basename set by camera_settings.SubPathPattern in the same python module. This frames_name should not include the extension eer.

Looking at rawtransfer.py code, the error message

20220829_20083310.eer not from a saved image

Should not include .eer part as the code sould have stripped it off at that point.

Hope this helps.

I could not find "camera->eer_frames = 1 (True)", nor any reference to .eer files in our rawtransfer.py

I have found that the leginon server is running an older version:

"Your current Appion version is 3.5 and revision is 22432" 

... Hopefully an update to the leginon server may be able to rectify this? (https://emg.nysbc.org/projects/leginon/repository/revisions/af461b5cdf74731549a20e7b1f36765db5b48ae5/diff/leginon/rawtransfer.py)

You should upgrade the whole system not just one file for this particular feature. Tracking of eer usage is system wide including new mysql table schema addition and how gain correction and frames are handled for frame alignment.

If you stay at 3.5, you will have to modify later also how frame alignment recognize it as eer and assign gain reference for motioncor2 as well, all manually.