Setup Ptolemy CLI for exposure targeting » History » Version 3
William Rice, 11/01/2022 03:59 PM
1 | 1 | Anchi Cheng | h1. Setup Ptolemy CLI for exposure targeting |
---|---|---|---|
2 | |||
3 | This shell script needs to take three arguments |
||
4 | |||
5 | 1. output json rootname. Leginon will assign this according to the session name and node alias calling this. For example, "22mar15a_Exposure_Targeting" |
||
6 | 2. full input mrc file path. Leginon will assign this according to its session path. For example "/data/leginon/22mar15a/rawdata/22mar15a_00001hl.mrc" |
||
7 | 3. output directory name. Leginon will assign this to the directory where Leginon is launched, assuming write assess of the user. For example, "/home/your_name" |
||
8 | |||
9 | Here is an example script used at SEMC |
||
10 | |||
11 | <pre> |
||
12 | #!/bin/sh -f |
||
13 | # Local runs |
||
14 | source /usr/local/anaconda3/etc/profile.d/conda.sh |
||
15 | conda activate /opt/condaenvs/ptolemy_env |
||
16 | echo $1 |
||
17 | echo $2 |
||
18 | echo $3 |
||
19 | python /home/your_name/packages/ptolemy/medmag_cli.py -f json -o $3/$1.json $2 |
||
20 | conda deactivate |
||
21 | </pre> |
||
22 | |||
23 | 2 | Anchi Cheng | h3. Testing example |
24 | |||
25 | Assuming you have /data/leginon/22mar15a/rawdata/22mar15a_00001hl.mrc, and call the above script hl_finding.sh you can test with |
||
26 | |||
27 | <pre> |
||
28 | hl_finding.sh my_test /data/leginon/22mar15a/rawdata/22mar15a_00001hl.mrc . |
||
29 | </pre> |
||
30 | You should get my_test.json written in your current directory. |
||
31 | |||
32 | 1 | Anchi Cheng | During leginon operation, the output json will be read by Leginon into memory and be replaced each time it is run. |
33 | 3 | William Rice | |
34 | h1. Speeding up hole finding by running a holewatcher program |
||
35 | |||
36 | At NYU, we are running Ptolemy on a decently powered gpu workstation (2X Nvidia 1080, 128 GB RAM, 32 cores of Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz). We found that hole finding was taking 12-14s. However, time could be cut in half by keeping ptolemy running all the time rather than running a new instance every time it was needed. We modified the holefinder to holewatcher.py, which runs all of the time and waits for a file to appear. LEginon runs the corresponding holemaker.sh program below. The only variable to change is outdir, which needs to match the target directory used by holewatcher.py. Time was reduced to 6-8s by doing this, and also by turning on CUDA in ptolemy. |
||
37 | |||
38 | <pre> |
||
39 | #!/bin/sh -f |
||
40 | # changed for remote computer running a holewatcher program |
||
41 | # wjr 10-13-22 |
||
42 | |||
43 | # modify outdir as needed to match that used by holewatcher.py |
||
44 | outdir=/data/cryoem/cryoemdata/arctica_holes |
||
45 | |||
46 | # sample output line |
||
47 | # 22oct13a_Exposure_Targeting /data/cryoem/cryoemdata/leginon/22oct13a/rawdata/22oct13a_p3b9g2a_00002sq_v01_00002hl.mrc /data/cryoem/cryoemdata/leginon/22oct13a/rawdata |
||
48 | # the output file Leginon looks for is /data/cryoem/cryoemdata/leginon/22oct13a/rawdata/22oct13a_Exposure_Targeting.json |
||
49 | |||
50 | echo $1 $2 $3 > $outdir/$1.txt |
||
51 | |||
52 | # Leginon will look for the output immediately after this script ends, so this script needs a wait function |
||
53 | mrc_path=$3 |
||
54 | file=$1 |
||
55 | slash='/' |
||
56 | extension='.json' |
||
57 | outfile=$mrc_path$slash$file$extension |
||
58 | |||
59 | until [ -f $outfile ] |
||
60 | do |
||
61 | sleep 0.1 |
||
62 | echo waiting for $outfile |
||
63 | done |
||
64 | echo Got it |
||
65 | </pre> |