Problems with Appion/Leginon
Added by Rick Baker over 5 years ago
Hi,
I recently had some issues while collecting data using Leginon and processing on-the-fly with Appion.
I was collecting data over the weekend, running MotionCor2 and CTFFIND4 through the Appion wrapper as I was collecting data. No problem.
For some reason, the frames stopped being transferred from the K2 computer to our Linux server (naturally, around midnight on Saturday night). Data collection was running fine, and unaligned summed micrographs (en.mrc) were appearing in the Appion webserver, but the frames were still sitting on the K2 computer. Sometime Sunday morning the transfer re-started on its own, after about 12 hours of being idle.
However, now when I try to run MotionCor2 through the Appion wrapper, it won't find the frames. If I try to re-start the original MotionCor2 session or start a new session, I always get this error:
Warning: /data/microscopes/frames/19jan05a/rawdata/19jan05a_gr_sq_hl_en.frames does not exist
The frame stacks are there, but there seems to be an issue with the extension (i.e. looking for en.frames but all files names en.frames.mrc)
This also only seems to be a problem with this dataset, as MotionCor2 is running fine on our current data collection. I assume this is related to the paused transfer protocol, but I don't know how to start trying to de-bug or solve this issue. I've reached out to our sys-admin and I'll update if he finds anything that could have caused this issue.
We are running myami/3.3
Rick
Replies (3)
RE: Problems with Appion/Leginon - Added by Rick Baker over 5 years ago
From our sys admin:
From the logs on Mozart, it looks like the rawtransfer service was copying frames up until about 11:11pm Saturday night. After that, the service no longer saw any new data to copy on the K2 camera mount. It continued like this until roughly 10:06am on Sunday morning, when it started seeing frames to copy again, which it then did.
However, there was an error that started occurring right at 11:11pm, and repeated during the ~11 hours when no data was copied:
Jan 13 08:17:25 mozart.al.ucsd.edu rawtransfer.py20921: rm: cannot remove ‘/cameras/k2-talos/20190112_23594712.mrc’: Device or resource busy
Jan 13 08:13:45 mozart.al.ucsd.edu rawtransfer.py20921: Destination path: /data/microscopes/frames/19jan05a/rawdata/19jan05a_grid_00028gr_00187sq_00091hl_rm: cannot remove ‘/cameras/k2-talos/20190112_23594712.mrc’: Device or resource busy
This all points to there being some sort of issue with the K2 machine's X: drive file share, and how the rawtransfer service on Mozart was accessing it.
RE: Problems with Appion/Leginon - Added by Anchi Cheng over 5 years ago
See this post RE: Is Leginon on Windows using 7-zip? I think it is the same issue.