Actions
Notes from Recon meeting » History » Revision 1
Revision 1/8
| Next »
Amber Herold, 06/22/2011 03:39 PM
Notes from Recon meeting¶
Moving forward, refinements will all be split into 2 steps, prep and run.
Prepare refine¶
When the user selects to prep a refinement, a web form is provided to select the:- refinement method - eman, xmipp, frealign, etc...
- stack
- model
- run parameters - runname, rundir, description
- stack prep params - lp, hp, last particle, binning
The web then calls prepRefine.py located on the local cluster to prepare the refinement.
Run Refine¶
When the user selects to run a prepared refinement, a web form is provided to select the:- prepped refine
- cluster parameters - ppn, nodes, walltime, cputime, memory, mempernode
- refine params, both general and method specific
- verify the cluster params by checking default_cluster.php
- if needed, copy the stack and model to a location that can be accessed by the selected cluster
- verify the user is logged into the cluster
- pass the list of commands to runJob.py (extended from the Agent class), located on the remote cluster
- format the command tokens in a dictionary of key-value pairs
- set the job type which was passed in the command
- create an instance of the job class based on the job type
- create an instance of the processing host class
- launch the job based via the processing host
- update the job status in the appion database (do we have db access from the remote cluster?)
Object Model¶
Processing Host¶
Each processing host (eg. Garibaldi, Guppy, Trestles) will define a class extended from a base ProcessingHost class.
The extended classes know what headers need to be placed at the top of job files and they know how to execute a command based on the specific clusters requirements.
The base ProcessingHost class could be defined as follows:
abstract class ProcessingHost: def generateHeader(jobObject) - abstract, extended classes should define this, returns a string def executeCommand(command) - abstract, extending classes define this def createJobFile(header, commandList) - defined in base class, commandList is a 2D array, each row is a line in the job file. def launchJob(jobObject) - defined in base class, jobObject is an instance of the job class specific to the jobtype we are running header = generateHeader(jobObject) jobFile = createJobFile(header, jobObject.getCommandList()) executeCommand(jobFile)
Updated by Amber Herold over 13 years ago · 1 revisions