Appion

h1. Remote Processing Setup

The web interface for Appion allows one to directly login to a computer and process Appion jobs. But this requires a job submission system to be installed.

The following applies to the both a web-server computer (setup earlier) and job submission system. The job submission system usually consists of a head node (main computer) and individual processing nodes (slave computers) for running jobs. As of these system can exist on an individual computer.

# [[Setup job submission server]]

# [[Configure web server to submit jobs]]

# [[Potential Problems]]

h2. Remote Appion Processing through ssh

A more advanced way to run appion script is done through an ssh session. This is equivalent of having you ssh into a computer and starting the appion processes.

There are two kinds of appion processes:

* The first is a single-node process that can be run on a stand-alone workstation or the head-node of a computer cluster without PBS.

* The second is a multiple-node process that requires PBS to run on the cluster.  

When you use "Just Show Command" option, it is always a single-node process, but if you run through ssh it could be either, depending on the demand of the process.  For example, imageuploader.py is typically always run as single-node process while maxlikeAlignment.py is either run on single-node with "Just Show Command" or as a PBS job submission when you run through ssh.

h2. Install the php-ssh2 extension

bq. For newer systems the extension is available through the repository, e.g., on Fedora 12 type "@sudo yum install php-pecl-ssh2@"

This enables an ssh session initialized from appionweb

# Download and install the ssh2 extension for php as instructed at its download site:

* http://www.php.net/manual/en/ssh2.installation.php

    Install openssl and compile libssh first before compiling php-ssh2 extension. Once you get the final ssh2.so, add it in your php.ini

The extension module is added to php in the same way as does the php-mrc module we distribute for the viewing mrc images through php.  To check whether it worked and for alternative way to make php recognize the module used in newer php, see http://emg.nysbc.org/documentation/leginon/bk02ch04s07.php under the section *Check php information* and *Alternative approach if mrc module does not show up in info.php output*

h2. configuration at web server side 

*IMPORTANT*: What we refer here as _your_cluster.php_ should not be taken literally.  For example, if you access your cluster through network with a name "bestclusterever", you should name your cluster configuration php file *bestclusterever.php*, not _your_cluster.php_.

# Go to your_dbem_1_5_1/processing directory

# Copy *default_cluster.php* to _your_cluster.php_

# [[ClusterPhpSettings|Edit your_cluster.php]] to correspond to your cluster configuration.

# Edit *config_processing.php* to add your cluster and, if desired, a stand-alone workstation as processing_host and register the _your_cluster.php_ you just created.

<pre>

$PROCESSING_HOSTS[]="your_stand-alone_processing_workstation";

$PROCESSING_HOSTS[]="your_cluster";

$CLUSTER_CONFIGS= array (

  'your_cluster'

);

</pre>

h2. Setting up PBS on processing server side 

Follow these instructions to [[Install PBS|setup PBS]].

h2. Testing 

h3. Testing php-ssh2 installation

Check your info.php as you did with mrctool installation.  Corrected installed extension should show up in the output of info.php.  Reference http://emg.nysbc.org/documentation/leginon/bk02ch04s07.php under the section *Check php information* and *Alternative approach if mrc module does not show up in info.php output*.

h3. Testing ssh log in 

Use the top right form on the processing page to log in as if doing an ssh session. The page will acknowledge that you have been logged in if the setup is correct.  You will be able to edit description of a run and to hide failed runs when logged in. The option for submitting the job appears at the bottom of the processing form whenever available.

h3. Testing simple appion processing job submission 

Upload template is a good example to try:

# Create a small mrc image as your template to be uploaded.

# Select Upload template in Import tools in the appionweb processing menu of an existing session.

# Type in the needed information.

# When you finish filling the form, instead of clicking on "Just show command", choose your processing host and run the job by clicking "Upload Template".

# For simple process such as this, the webpage will take a little while to refresh until the job is completed.

# If you get the message "Template is uploaded", the process is successful and if you refresh the page you will find the template in the available list. 

h3. Testing PBS-required appion processing job submission 

Particle selection such as DogPicker is a good example to try:

# Click on DoG Picking under Particle Selection menu

# Enter the required parameters

# When you finish filling a form for an appion processing, choose your processing host and run the job.  Pages for monitoring the job become available after the job is queued and subsequently begins running. If the status appears as "Running or Queued" at first, the setup is likely correct.

# After a while, the process will be completed and the status becomes "Done" when you click to have the Status updated on the page.

h3. Check _your_cluster.php_ setup 

For reconstructions involving iterations of different parameters such as EMAN reconstruction by refinement, the _your_cluster.php_ is used to generate the script.  Examine the script created on the web form and modify _your_cluster.php_  You can copy the script to your cluster and test run/modify it until it is correct.

______

[[Create a Test Project|< Create a Test Project]]

______