Leginon

I looked at this again recently and it appears that the Focus node is not the main bottleneck although focussing may be taking more time than in the Stagg et al paper. Apparently it takes us about a minute to get one of the two images in a defocus pair in the Exposure node but from the Stagg et al paper it works out at about a minute for both images in a defocus pair (552 defocus pairs in 9.4 hours or 564 minutes). I don't know where the time could be going as assuming it is due to the network then a minute for one of the defocus pair images would suggest we're getting about 10 Mbps on the network. However, tests with netperf and also ftp'ing large files do suggest we get 100 Mbps,

William

"WVNicholson" wrote: Data acquisition with Leginon seems to be a bit slow - two to three times slower than times recently quoted in a recent paper on Leginon (Stagg et al, I think) - on a good day when there are not too many problems with drift. I'm not sure where the time could be going. It does appear to be somewhat more time than is spent acquiring the actual exposure images (although they are quite large). From logs it appears that a lot of the time is spent in the Focus node. The things I can think of are time spent sending the images over the network for the Focus node and correlation calculations in the Focus node (to get the image shift in order to subsequently compute the defocus and possibly stigmator correction). Does the Focus node fetch many images at the fa preset in order to determine the defocus? Is this number of images acquired for the Focus node at the fa preset dependent on whether the option is selected to carry out stigmator correction? We are currently working on a 100 Mbps network so maybe going to a gigabit network would help. On the other hand, if all the extra time is spent on correlation calculations then maybe we need to run Leginon on a faster system (i.e. new laptop or even a desktop with a reasonable performance),
William

I investigated further by running the Manual application in Leginon with the normal set up (client on microscope PC and main Leginon on a laptop) and Leginon on the microscope PC only and also timing image acquisition in DM. We appear to be losing some time on image display, image correction (bad pixels, despiking, etc for our bad camera) and also a few seconds on using a 100 Mbps rather than a gigabit network. The image acquisition itself seems to take about 40-45 seconds for a 4k x 4k image. I timed it with my watch at about 45 seconds in DM (which necessarily includes the display time) and the logged messages in Leginon running on the microscope PC only indicate it's about 42 seconds not including the Leginon display time. So this is still worse by about 10-15 seconds than the 30.5 seconds for a single image suggested by the quoted times in the Stagg et al paper. I'm not sure where the time could be going - if the microscope PC is slower or if it's something to do with the Ultrascan 4000SP camera - either a hardware problem, or the camera's slower due to the Gatan bin (shared with the other Gatan camera and other hardware) or the firewire interface (slower than the custom optical cable used by the multiport Ultrascan 4000 camera)?

William

"WVNicholson" wrote: I looked at this again recently and it appears that the Focus node is not the main bottleneck although focussing may be taking more time than in the Stagg et al paper. Apparently it takes us about a minute to get one of the two images in a defocus pair in the Exposure node but from the Stagg et al paper it works out at about a minute for both images in a defocus pair (552 defocus pairs in 9.4 hours or 564 minutes). I don't know where the time could be going as assuming it is due to the network then a minute for one of the defocus pair images would suggest we're getting about 10 Mbps on the network. However, tests with netperf and also ftp'ing large files do suggest we get 100 Mbps,
William

"WVNicholson" wrote: Data acquisition with Leginon seems to be a bit slow - two to three times slower than times recently quoted in a recent paper on Leginon (Stagg et al, I think) - on a good day when there are not too many problems with drift. I'm not sure where the time could be going. It does appear to be somewhat more time than is spent acquiring the actual exposure images (although they are quite large). From logs it appears that a lot of the time is spent in the Focus node. The things I can think of are time spent sending the images over the network for the Focus node and correlation calculations in the Focus node (to get the image shift in order to subsequently compute the defocus and possibly stigmator correction). Does the Focus node fetch many images at the fa preset in order to determine the defocus? Is this number of images acquired for the Focus node at the fa preset dependent on whether the option is selected to carry out stigmator correction? We are currently working on a 100 Mbps network so maybe going to a gigabit network would help. On the other hand, if all the extra time is spent on correlation calculations then maybe we need to run Leginon on a faster system (i.e. new laptop or even a desktop with a reasonable performance),
William