Setup Remote Processing¶

Remote Appion Processing through ssh¶

A more advanced way to run appion script is done through an ssh session. This is equivalent of having you ssh into a computer and starting the appion processes.

There are two kinds of appion processes. The first is a single-node process that can be run on a stand-alone workstation or the head-node of a computer cluster without PBS. The second is a multiple-node process that requires PBS to run on the cluster. When you use "Just Show Command" option, it is always a single-node process, but if you run through ssh it could be either, depending on the demand of the process. For example,
imageuploader.py is always run as single-node process while maxlikeAlignment.py is either run on single-node with "Just Show Command" or as a PBS job submission when you run through ssh.

Install and setup ssh2 extension for php at your web-server¶

This enables an ssh session initialized from appionweb

1. Download and install the ssh2 extension for php as instructed at its download site:
• http://www.php.net/manual/en/ssh2.installation.php
  Install openssl and compile libssh first before compiling php-ssh2 extension. Once you get the final ssh2.so, add it in your php.ini

The extension module is added to php in the same way as does the php-mrc module we distribute for the viewing mrc images through php. To check whether it worked and for alternative way to make php recognize the module used in newer php, see http://emg.nysbc.org/documentation/leginon/bk02ch04s07.php under the section Check php information and Alternative approach if mrc module does not show up in info.php output

configuration at web server side¶

IMPORTANT: What we refer here as your_cluster.php should not be taken literally. For example, if you access your cluster through network with a name "bestclusterever", you should name your cluster configuration php file bestclusterever.php, not your_cluster.php.

1. Go to your_dbem_1_5_1/processing directory
2. Copy default_cluster.php to your_cluster.php
3. Edit your_cluster.php to correspond to your cluster configuration.
4. Edit config_processing.php to add your cluster and, if desired, a stand-alone workstation as processing_host and register the your_cluster.php you just created.

$PROCESSING_HOSTS[]="your_stand-alone_processing_workstation";
$PROCESSING_HOSTS[]="your_cluster";
$CLUSTER_CONFIGS= array (
  'your_cluster'
);

Setting up PBS on processing server side¶

Follow these instructions to set up PBS on your processing server.

Testing¶

Testing php-ssh2 installation¶

Check your info.php as you did with mrctool installation. Corrected installed extension should show up in the output of info.php. Reference http://emg.nysbc.org/documentation/leginon/bk02ch04s07.php under the section Check php information and Alternative approach if mrc module does not show up in info.php output.

Testing ssh log in¶

Use the top right form on the processing page to log in as if doing an ssh session. The page will acknowledge that you have been logged in if the setup is correct. You will be able to edit description of a run and to hide failed runs when logged in. The option for submitting the job appears at the bottom of the processing form whenever available.

Testing simple appion processing job submission¶

Upload template is a good example to try:

1. Create a small mrc image as your template to be uploaded.
2. Select Upload template in Import tools in the appionweb processing menu of an existing session.
3. Type in the needed information.
4. When you finish filling the form, instead of clicking on "Just show command", choose your processing host and run the job by clicking "Upload Template".
5. For simple process such as this, the webpage will take a little while to refresh until the job is completed.
6. If you get the message "Template is uploaded", the process is successful and if you refresh the page you will find the template in the available list.

Testing PBS-required appion processing job submission¶

Particle selection such as DogPicker is a good example to try:

1. Click on DoG Picking under Particle Selection menu
2. Enter the required parameters
3. When you finish filling a form for an appion processing, choose your processing host and run the job. Pages for monitoring the job become available after the job is queued and subsequently begins running. If the status appears as "Running or Queued" at first, the setup is likely correct.
4. After a while, the process will be completed and the status becomes "Done" when you click to have the Status updated on the page.

Check your_cluster.php setup¶

For reconstructions involving iterations of different parameters such as EMAN reconstruction by refinement, the your_cluster.php is used to generate the script. Examine the script created on the web form and modify your_cluster.php You can copy the script to your cluster and test run/modify it until it is correct.