image version in Leginon 1.3
Added by Anchi Cheng almost 18 years ago
This is a question sent by William Nicholson while this bulleton is down:
I meant to post this question to the bulletin board but the emg.nysbc.org and nramm.scripps.edu websites appear to be inaccessible right now. I noticed that in a recent Leginon run I got multiple versions of the square and hole images (with v0<number> after either sq or hl in the image name). Do you know why this would happen? The recent Leginon run was quite slow at collecting actual data and one of the suspected reasons is due to time spent collecting multiple versions of holes. (Another possibility is waiting in the Drift Manager node for drift to settle down.) The Leginon run was with one of the MSI application variants (actually a sort of hybrid of MSI-T and MSI-edge that uses templates for detecting holes in the Hole Targeting node and edge detection in the Exposure Targeting node) with queuing in both the Hole targeting and Exposure Targeting node. By the way, I can't find the section in the manual that describes the "double queuing" approach - a short section in the general MSI chapter I think refers to the MSI-Tomography chapter which does not appear to have any information about queuing. Also, you may be interested to know that I'm using "double queuing" as it seems to be the only way in Leeds to get round the deadlock problem with non-queued fully automated runs,
William
Replies (1)
- Added by Anchi Cheng almost 18 years ago
Hi, William,
Thanks for pointing out the lack of information on double queuing. I took a short cut there by refering to the tomography application but indeed the description there is not as complete. I will make the change.
Before Leginon v1.3, only one version of each image is saved on disks. Images that are produced by drift management for target adjustment only exist in memory. In v1.3, we added a different method for the scope to get to its target by using a multiple move-and-check procedure in Navigator, explained in the chapter of MSI in the section Special Operation Preference setup/ Minimize the use of image shift for the final exposure. In order to be compatible with that, the images from the drift management have to be saved. Unfortunately, this feature does not work with queuing so you did not get any benefit out of it but only problems,it seems.
For use with energy filter, your preference settings would be such that "adjust target for shift" are "no" for Grid, Z focus, and Square nodes and "yes" for other nodes that acquire images. Therefore, you would get extra image saved after when the queued targets from hl image is about to be processed and if drift is detected in "Focus" node drift check (Both are what we call a drift event). These would be marked as v1 while the originals v0. Saving these images certainly takes time but it should be minimal since this does not include normalization, despike and other image process that has to be done no matter what. On our network, the publish of an image takes 2 sec for 2kx2k (binned by 2) images and 1 sec for 1kx1k (binned by 4) and 512x512 (binned by 8 ) images (we are out of resolution on our timing table here). Since in our case the "hl" and "sq" preset at which most new version images are acquired uses a configuration of 512x512 or 1kx1k, the total delay per target after a drift event, the delay caused by this new addition is 1 sec per "hl" image that has targets on. and 1 sec per "sq" image that
has targets on.
If your delay is much longer, your image saving is slow. To check how long just the image publishing part takes, you can do the following:
1. Start Leginon, and your application.
2. Send the "hl" preset to the microscope as an initiallization.
3. Go to the node "Hole" which should have your normal settings.
4. Click on the "Simulate Target" tool to acquire an image with this
node. Note the first time stamp and the lase to get how long it takes.
This corresponds to the time takes for all processes in this node, including saving the image.
5. Click on the settings tool of the "Hole" node to deactivate the option "Save image to database".
6. Click on the "Simulate Target" tool again to find out how long it takes for all processes in this node except saving the image.
7. Don't foget to reactivate the "Save image to database" option, or you will lose data in your next run.
The difference in time spent between 6 and 4 is the time required to save an image.
You may also want to count how many v1 you get in your run and see if it adds up right. Note that only the v1 right before .mrc counts as an extra image acquire.
Anchi