Setup Remote Processing » History » Revision 2
Revision 1 (Amber Herold, 04/06/2010 05:05 PM) → Revision 2/24 (Amber Herold, 04/06/2010 05:07 PM)
h1. Setup Remote Processing
h2. Remote Appion Processing through ssh
A more advanced way to run appion script is done through an ssh session. This is equivalent of having you ssh into a computer and starting the appion processes.
There are two kinds of appion processes. The first is a single-node process that can be run on a stand-alone workstation or the head-node of a computer cluster without PBS. The second is a multiple-node process that requires PBS to run on the cluster. When you use "Just Show Command" option, it is always a single-node process, but if you run through ssh it could be either, depending on the demand of the process. For example,
imageuploader.py is always run as single-node process while maxlikeAlignment.py is either run on single-node with "Just Show Command" or as a PBS job submission when you run through ssh.
h3. Install and setup ssh2 extension for php at your web-server
This enables an ssh session initialized from appionweb
# Download and install the ssh2 extension for php as instructed at its download site:
* http://www.php.net/manual/en/ssh2.installation.php
Install openssl and compile libssh first before compiling php-ssh2 extension. Once you get the final ssh2.so, add it in your php.ini
The extension module is added to php in the same way as does the php-mrc module we distribute for the viewing mrc images through php. To check whether it worked and for alternative way to make php recognize the module used in newer php, see http://emg.nysbc.org/documentation/leginon/bk02ch04s07.php under the section *Check php information* and *Alternative approach if mrc module does not show up in info.php output*
h3. configuration at web server side
*IMPORTANT*: What we refer here as _your_cluster.php_ should not be taken literally. For example, if you access your cluster through network with a name "bestclusterever", you should name your cluster configuration php file *bestclusterever.php*, not _your_cluster.php_.
# Go to your_dbem_1_5_1/processing directory
# Copy *default_cluster.php* to _your_cluster.php_
# [[ClusterPhpSettings|Edit your_cluster.php]] to correspond to your cluster configuration.
# Edit *config_processing.php* to add your cluster and, if desired, a stand-alone workstation as processing_host and register the _your_cluster.php_ you just created.
<pre>
$PROCESSING_HOSTS[]="your_stand-alone_processing_workstation";
$PROCESSING_HOSTS[]="your_cluster";
$CLUSTER_CONFIGS= array (
'your_cluster'
);
</pre>
h3. Setting up PBS on processing server side
Follow these instructions [[Start_with_existing_CentOS_installation#9-Install-a-PBS-job-submission-system|instructions to [[Install PBS|setup PBS]]. set up PBS on your processing server]].
h3. Testing
h4. Testing php-ssh2 installation
Check your info.php as you did with mrctool installation. Corrected installed extension should show up in the output of info.php. Reference http://emg.nysbc.org/documentation/leginon/bk02ch04s07.php under the section *Check php information* and *Alternative approach if mrc module does not show up in info.php output*.
h4. Testing ssh log in
Use the top right form on the processing page to log in as if doing an ssh session. The page will acknowledge that you have been logged in if the setup is correct. You will be able to edit description of a run and to hide failed runs when logged in. The option for submitting the job appears at the bottom of the processing form whenever available.
h4. Testing simple appion processing job submission
Upload template is a good example to try:
# Create a small mrc image as your template to be uploaded.
# Select Upload template in Import tools in the appionweb processing menu of an existing session.
# Type in the needed information.
# When you finish filling the form, instead of clicking on "Just show command", choose your processing host and run the job by clicking "Upload Template".
# For simple process such as this, the webpage will take a little while to refresh until the job is completed.
# If you get the message "Template is uploaded", the process is successful and if you refresh the page you will find the template in the available list.
h4. Testing PBS-required appion processing job submission
Particle selection such as DogPicker is a good example to try:
# Click on DoG Picking under Particle Selection menu
# Enter the required parameters
# When you finish filling a form for an appion processing, choose your processing host and run the job. Pages for monitoring the job become available after the job is queued and subsequently begins running. If the status appears as "Running or Queued" at first, the setup is likely correct.
# After a while, the process will be completed and the status becomes "Done" when you click to have the Status updated on the page.
h4. Check _your_cluster.php_ setup
For reconstructions involving iterations of different parameters such as EMAN reconstruction by refinement, the _your_cluster.php_ is used to generate the script. Examine the script created on the web form and modify _your_cluster.php_ You can copy the script to your cluster and test run/modify it until it is correct.