Project

General

Profile

Setup Ptolemy CLI for exposure targeting » History » Version 3

William Rice, 11/01/2022 03:59 PM

1 1 Anchi Cheng
h1. Setup Ptolemy CLI for exposure targeting
2
3
This shell script needs to take three arguments
4
5
1. output json rootname.  Leginon will assign this according to the session name and node alias calling this.  For example, "22mar15a_Exposure_Targeting"
6
2. full input mrc file path. Leginon will assign this according to its session path.  For example "/data/leginon/22mar15a/rawdata/22mar15a_00001hl.mrc"
7
3. output directory name. Leginon will assign this to the directory where Leginon is launched, assuming write assess of the user.  For example, "/home/your_name"
8
9
Here is an example script used at SEMC
10
11
<pre>
12
#!/bin/sh -f
13
# Local runs
14
source /usr/local/anaconda3/etc/profile.d/conda.sh
15
conda activate /opt/condaenvs/ptolemy_env
16
echo $1
17
echo $2
18
echo $3
19
python /home/your_name/packages/ptolemy/medmag_cli.py -f json -o $3/$1.json $2
20
conda deactivate
21
</pre>
22
23 2 Anchi Cheng
h3. Testing example
24
25
Assuming you have /data/leginon/22mar15a/rawdata/22mar15a_00001hl.mrc, and call the above script hl_finding.sh you can test with
26
27
<pre>
28
hl_finding.sh my_test /data/leginon/22mar15a/rawdata/22mar15a_00001hl.mrc .
29
</pre>
30
You should get my_test.json written in your current directory.
31
32 1 Anchi Cheng
During leginon operation, the output json will be read by Leginon into memory and be replaced each time it is run.
33 3 William Rice
34
h1. Speeding up hole finding by running a holewatcher program
35
36
At NYU, we are running Ptolemy on a decently powered gpu workstation (2X Nvidia 1080, 128 GB RAM, 32 cores of Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz). We found that hole finding was taking 12-14s. However, time could be cut in half by keeping ptolemy running all the time rather than running a new instance every time it was needed. We modified the holefinder to holewatcher.py, which runs all of the time and waits for a file to appear. LEginon runs the corresponding holemaker.sh program below. The only variable to change is outdir, which needs to match the target directory used by holewatcher.py. Time was reduced to 6-8s by doing this, and also by turning on CUDA in ptolemy.
37
38
<pre>
39
#!/bin/sh -f
40
# changed for remote computer running a holewatcher program
41
# wjr 10-13-22
42
43
# modify outdir as needed to match that used by holewatcher.py
44
outdir=/data/cryoem/cryoemdata/arctica_holes
45
46
# sample output line 
47
# 22oct13a_Exposure_Targeting /data/cryoem/cryoemdata/leginon/22oct13a/rawdata/22oct13a_p3b9g2a_00002sq_v01_00002hl.mrc /data/cryoem/cryoemdata/leginon/22oct13a/rawdata
48
# the output file Leginon looks for is /data/cryoem/cryoemdata/leginon/22oct13a/rawdata/22oct13a_Exposure_Targeting.json
49
50
echo $1 $2 $3 > $outdir/$1.txt
51
52
# Leginon will look for the output immediately after this script ends, so this script needs a wait function
53
mrc_path=$3
54
file=$1
55
slash='/'
56
extension='.json'
57
outfile=$mrc_path$slash$file$extension
58
59
until [ -f $outfile ]
60
do
61
     sleep 0.1
62
     echo waiting for $outfile
63
done
64
echo Got it
65
</pre>