At what point is information written to the myami database?
Added by Patrick Goetz almost 6 years ago
We had a rather unfortunate incident over the weekend with an image collection session where at midnight images stopped being collected. Usually this is a result of a 7zip process affiliated with Digital Micrograph that locks the MRC file and freezes, preventing rawtransfer.py from transferring the images out of the NTFS folder and into the frames folder because the file is locked, but in this case there weren't any images in the D:\frames folder on the camera PC. Oddly enough, though, these images were in catalogued in the myami database (but with no actual associated image; i.e. empty preview and of course no associated images in the frames folder.) The lab manager logged out of the camera PC and logged back in again, allowing the image collection process to resume, but then a similar crash occurred on Sunday at midnight, and this time there were no images anywhere, including records in the myami database.
This raises the question, at precisely what point is an image documented in the myami database? I thought this was part of rawtransfer.py, but the evidence from Friday night suggests that this is being handled by the leginon component on the camera PC?
Replies (5)
RE: At what point is information written to the myami database? - Added by Anchi Cheng almost 6 years ago
The image is logged into the database when "getImage" call returns from the camera to the main Leginon program on the linux box. Even if frames are not saved for some camera issue, the sum image may still be returned and gives no error.
What you should check is whether webviewer shows the un-aligned image as it should. These are saved in your linux data server leginon directory. If it does show images, what really failed is the ability of saving frames. Are your K2 disk full or close to full ? If you use the solid-state drive on K2, please understand that the disk is used by K2 software during its frame processing before giving it to us. If K2 software has trouble, it will stop supplying the frames.
What rawtransfer.py does is not to log the image in the myami database, but to associate the frame movie with the image record already in the database at that point.
RE: At what point is information written to the myami database? - Added by Anchi Cheng almost 6 years ago
If K2 fails to save the frame movies, there might be error message on K2 PC GMS logger.
RE: At what point is information written to the myami database? - Added by Patrick Goetz almost 6 years ago
So the solid state drive on the K2 had no images on it when this problem occurred, which is why I suspected a glitch with Digital Micrograph (or some other process running on the camera PC). Can you elucidate on what you mean by GMS logger? I'm not sure what this is.
RE: At what point is information written to the myami database? - Added by Aguang Dai almost 6 years ago
Patrick Goetz wrote:
So the solid state drive on the K2 had no images on it when this problem occurred, which is why I suspected a glitch with Digital Micrograph (or some other process running on the camera PC). Can you elucidate on what you mean by GMS logger? I'm not sure what this is.
GMS and DM are the same software, Gatan re-named the software from DM to GMS now.
The GMS logger Anchi mentioned, I think it's the logging window at the bottom area of GMS software.
RE: At what point is information written to the myami database? - Added by Patrick Goetz almost 6 years ago
I was afraid of that. As far as we can tell, at least DM flushes the log whenever it's restarted, so we don't have a record of what happened when the event occurred, as DM was restarted in an effort to get image collection going again. (Aguang is our EM lab manager and knows all this; just reporting for the benefit of anyone else looking at this thread.)