Appion

So is this sometimes a two step process, when defergpu is set?
Would I need to set up the interface similar to a preprefine job such that after the first step runs, then the user must start the second? We did add gpu info to the config file already, so we should be able to look it up to know if there is a gpu or not. Is there a way to query the gpu to see if it is busy?

Perhaps we should have a face to face before I start coding this...

Anchi, For catchupDDAlign, should the run name and output directory be the same as the original make DD stack, or something else?

At script level it does not need runname and rundir because it gets it from ddstackid. If you must put it in, then, yes, it is the same.

I tried a couple of times and the web interface didn't work to submit the job but when pasting the command in a terminal it does work.

Here is the error message I got:

Appion Directory /ami/data15/appion/12nov30d/ddstack/ddstack25
Job File Name ddstack25.appionsub.job
Hostname guppy.scripps.edu
Username dveesler
Local server ip 137.131.204.71
Project Id
Command /opt/myamisnap/bin/appion runJob.py /opt/myamisnap/bin/appion makeDDRawFrameStack.py --description="test" --projectid=373 --preset=es --session=12nov30d --runname=ddstack25 --rundir=/ami/data15/appion/12nov30d/ddstack/ddstack25 --no-rejects --no-wait --commit --limit=1 --continue --align --defergpu --expid=10730 --jobtype=makeddrawframestack --ppn=1 --nodes=1 --walltime=240 --jobid=334
ERROR in job submission. Check the cluster. makeddrawframestack ['/opt/myamisnap/bin/appion', 'makeDDRawFrameStack.py', '--description=test', '--projectid=373', '--preset=es', '--session=12nov30d', '--runname=ddstack25', '--rundir=/ami/data15/appion/12nov30d/ddstack/ddstack25', '--no-rejects', '--no-wait', '--commit', '--limit=1', '--continue', '--align', '--defergpu', '--expid=10730', '--jobtype=makeddrawframestack', '--ppn=1', '--nodes=1', '--walltime=240', '--jobid=334']

I submitted the following command into a terminal logged on guppy and it worked smoothly:

[dveesler@guppy ~]$ /opt/myamisnap/bin/appion makeDDRawFrameStack.py --description="test" --projectid=373 --preset=es --session=12nov30d --runname=ddstack25 --rundir=/ami/data15/appion/12nov30d/ddstack/ddstack25 --no-rejects --no-wait --commit --limit=1 --continue --align --defergpu --expid=10730 --jobtype=makeddrawframestack

David, there was a bug with all web submitted jobs last week that should be sorted out now. Could you please try launching this again to confirm functionality?

The web submission problem is persisting for the DD frames alignment. I just tried it again (cf. below).
However, I think we should have a binning option in the web interface because we have to add bin by 2 in the command line submission to decime super-resolution images from ~8Kx8K to ~4Kx4K in order to get Yifan's soft to work.

Appion Directory /ami/data15/appion/12nov30d/ddstack/ddstack26
Job File Name ddstack26.appionsub.job
Hostname guppy.scripps.edu
Username dveesler
Local server ip 137.131.204.44
Project Id
Command /opt/myamisnap/bin/appion runJob.py /opt/myamisnap/bin/appion makeDDRawFrameStack.py --description="test web interface" --projectid=373 --preset=es --session=12nov30d --runname=ddstack26 --rundir=/ami/data15/appion/12nov30d/ddstack/ddstack26 --no-rejects --no-wait --commit --limit=1 --continue --align --defergpu --expid=10730 --jobtype=makeddrawframestack --ppn=1 --nodes=1 --walltime=240 --jobid=339
ERROR in job submission. Check the cluster. makeddrawframestack ['/opt/myamisnap/bin/appion', 'makeDDRawFrameStack.py', '--description=test web interface', '--projectid=373', '--preset=es', '--session=12nov30d', '--runname=ddstack26', '--rundir=/ami/data15/appion/12nov30d/ddstack/ddstack26', '--no-rejects', '--no-wait', '--commit', '--limit=1', '--continue', '--align', '--defergpu', '--expid=10730', '--jobtype=makeddrawframestack', '--ppn=1', '--nodes=1', '--walltime=240', '--jobid=339']

I added the binning option to the web gui for makeDDrawFrameStack in r17498.

I changed the default binning to 1 in r17536

r18022 adds reference to web gui

Hey Anchi,
I took at look at the code in r18022.
I processing.inc, I would prefer not to create the new function

function gpuDriftCorrRef() { return showReference("gpudriftcorr"); }

Those function are only there for backward compatibility, and eventually I would like to remove them all and instead use

function showReference( $method )

So to add a reference you would use

showReference( "gpudriftcorr" )

Thoughts?

Sounds fine, except that now you need to input publication.inc everywhere. It is up to you. Fine with me either way.

showReference() is in processing.inc, so you should not need to add publication.inc. Did I miss something?