Adding refine runjob » History » Revision 11
Revision 10 (Amber Herold, 07/21/2014 12:44 PM) → Revision 11/17 (Amber Herold, 07/21/2014 12:45 PM)
h1. Adding refine job h3. How a cluster job file is built and launched * To run a cluster job, the user runs a command on the head node of the cluster that looks like this: <pre>runJob.py --jobtype=frealignrecon --outerMaskRadius=65 --innerMaskRadius=0 --symmetry="D7 (z)" --endIter=10 --percentDiscard=15 --wgh=0.07 --hp=300 --lp=10 --ffilt --psi --theta --phi --x --y --modelnames=initmodel0001.mrc --stackname=start.mrc --apix=1.63 --boxsize=120 --totalpart=300 --cs=2 --kv=120 --description="test frealign get preset" --runname=frealign_recon69 --rundir=/ami/data17/appion/zz07jul25b/recon/frealign_recon69 --nodes=2 --ppn=4 --rpn=8 --walltime=240 --cput=2400 --localhost=guppy.scripps.edu --remoterundir=/ami/data15/appion/zz07jul25b/recon/frealign_recon69 --projectid=303 --expid=8556</pre> * *runJob.py* (source:trunk/appion/bin/runJob.py) is (source:trunk/appion/bin/runJob.py)is a python script that passes all the command parameters to a python class called *apAgent* (source:trunk/appion/appionlib/apAgent.py). The *apAgent* object instantiates 2 more classes: # _Job object_: ** this can be a *genericJob* (source:trunk/appion/appionlib/apGenericJob.py) which does not require a specialized job file, or a job class based on *apRefineJob*, or *apRemoteJob* ** See source:trunk/appion/appionlib/apSparxISAC.py for an example based on remoteJob ** The job object is passed the cluster processing parameters and the name of the job ** Most importantly, this object is responsible for knowing what the guts of the job file should be. These guts are maintained in a list called *command_list* # _Processing Host object_: ** The base processing host will work for most resource managers, variations can create a new class based on apProcessingHost such as source:trunk/appion/appionlib/torqueHost.py ** The processing host class knows how to format the job file so that the resource manager running on the current cluster can read it ** It also keeps track of the correct commands to use to launch a job and check on the job status So, when a job is run with a command starting with runJob.py: # an apAgent object will be created # the apAgent object will create a processingHost object whose type (torque, moab, etc) depends on the specific cluster that is being run on # the apAgent object will create a job object based on the --jobtype parameter that was passed to runJob.py # the apAgent object will then call the launchJob() function defined in the processingHost class # inside the launchJob() function, the details of the job cluster parameters (number of processors, memory, etc) and the guts of the job file are requested from the Job Object # the information from the job object is used by the processingHost object to build the job file # the processingHost object submits the newly created job file to the cluster h3. Add job type to Agent. After you have added a new refinement job class it needs to be added to the job running agent by editing the file apAgent.py in appionlib. # Add the name of the module you created to the import statements at the top of the file. # In the method _createJobInst_ add the new refinment job type to the condition statements. <pre> Ex. elif "newJobType" == jobType: jobInstance = newModuleName.NewRefinementClass(command) </pre> !Agent_class_diag.png!