Appion

h1. Adding refine job

h3. How a cluster job file is built and launched

* To run a cluster job, the user runs a command on the head node of the cluster that looks like this:

<pre>runJob.py --jobtype=frealignrecon --outerMaskRadius=65 --innerMaskRadius=0 --symmetry="D7 (z)" --endIter=10 --percentDiscard=15 --wgh=0.07 --hp=300 --lp=10 --ffilt --psi --theta --phi --x --y --modelnames=initmodel0001.mrc --stackname=start.mrc --apix=1.63 --boxsize=120 --totalpart=300 --cs=2 --kv=120 --description="test frealign get preset" --runname=frealign_recon69 --rundir=/ami/data17/appion/zz07jul25b/recon/frealign_recon69 --nodes=2 --ppn=4 --rpn=8 --walltime=240 --cput=2400 --localhost=guppy.scripps.edu --remoterundir=/ami/data15/appion/zz07jul25b/recon/frealign_recon69 --projectid=303 --expid=8556</pre>

* *runJob.py* (source:trunk/appion/bin/runJob.py) is a python script that passes all the command parameters to a python class called *apAgent* (source:trunk/appion/appionlib/apAgent.py).

The *apAgent* object instantiates 2 more classes:

# _Job object_: 

** this can be a *genericJob* (source:trunk/appion/appionlib/apGenericJob.py) which does not require a specialized job file, or a job class based on *apRefineJob*, or *apRemoteJob*

** See source:trunk/appion/appionlib/apSparxISAC.py for an example based on remoteJob

** The job object is passed the cluster processing parameters and the name of the job

** Most importantly, this object is responsible for knowing what the guts of the job file should be. These guts are maintained in a list called *command_list* 

# _Processing Host object_: 

** The base processing host will work for most resource managers, variations can create a new class based on processingHost.py (source:trunk/appion/appionlib/processingHost.py) such as source:trunk/appion/appionlib/torqueHost.py

** The processing host class knows how to format the job file so that the resource manager running on the current cluster can read it

** It also keeps track of the correct commands to use to launch a job and check on the job status

* So, when a job is run with a command starting with runJob.py: 

## an apAgent object will be created

## the apAgent object will create a processingHost object whose type (torque, moab, etc) depends on the specific cluster that is being run on

## the apAgent object will create a job object based on the --jobtype parameter that was passed to runJob.py

## the apAgent object will then call the launchJob() function defined in the processingHost class

## inside the launchJob() function, the details of the job cluster parameters (number of processors, memory, etc) and the guts of the job file are requested from the Job Object

## the information from the job object is used by the processingHost object to build the job file 

## the processingHost object submits the newly created job file to the cluster 

h3. To add a new job type

# Write a new class in myami/appion/appionLib that is based on apRefineJob or apRemoteJob

 

# Add the new job type to the Agent class

 

After you have added a new refinement job class it needs to be added to the job running agent by editing the file apAgent.py in appionlib.  

 

# Add the name of the module you created to the import statements at the top of the file.

# In the method _createJobInst_ add the new refinment job type to the condition statements.

<pre>

  Ex.

  elif "newJobType" == jobType:

            jobInstance = newModuleName.NewRefinementClass(command)

</pre>

!Agent_class_diag.png!