Project

General

Profile

Actions

Feature #2869

closed

Add an option to cluster parameters form to select which queue to submit a job to

Added by Amber Herold almost 10 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Sargis Dallakyan
Category:
-
Target version:
Start date:
07/28/2014
Due date:
% Done:

0%

Estimated time:

Related issues 2 (2 open0 closed)

Related to Appion - Feature #2839: Implement Iterative Stable Alignment and Clustering (ISAC) on a 2-D image stackAssignedDmitry Lyumkis07/15/2014

Actions
Related to Appion - Bug #2939: Appion Developers Conference Call NotesNew09/24/2014

Actions
Actions #1

Updated by Amber Herold almost 10 years ago

  • Target version changed from Appion/Leginon 3.1.0 to Appion/Leginon 3.2

Prior to this feature, the queue has been set by adding extra lines to the job file via the .appion.cfg file located on the processing host. When a job is launched from the web and goes through runJob.py, the .appion.cfg file is loaded and used.

Now we want to be able to select which queue the user would like to run on directly from the web GUI to give the user more flexibility and avoid having to edit job files once created.

To do this, we can now define all the available queues in the config.php file in the myamiweb directory. Each entry in the PROCESSING_HOSTS array can include a queues array which simply lists the names of the available queues. The Cluster class now includes a queues member variable which is used to populate the available queues in clusterParameters form. The Cluster class is also capable of asking the selected host for its list of available queues when the user is logged in, which may be more accurate or up to date than the config file, but may also be more confusing for users if there are queues that should not be used. We can remove that feature if needed. For generic jobs like particle picking, etc., the getSubmitForm() function in processing.inc has been updated to add a select box for queues if there is more than one queue available for the selected host.

For all job types, generic, remote, and refine, a new option --queue is added to the command sent to runJob.py. When runJob.py builds the job file, if --queue=name is defined, the queue will be listed in the file.
An important question for Anchi:
Should your apParellelTasks.py jobs also set the queue??

Still to do:
  • update the installation documentation
  • update the setup wizard to prompt for processing host queues

These changes will affect every job run in Appion, so if there are bugs, they should be discovered quickly. Keep this in mind if you see anything strange going on with job submission.

When I check in these changes, the new web front end will not be compatible with the old back end or old config.php files or old .appion.cfg files. The web code, processing code and config files must be updated at the same time. Therefore, I will wait to check in these changes for a time when Sargis is available to ensure all installations are updated correctly.

Sargis, will Monday August 4 be a good day for us to do the update??

Actions #2

Updated by Amber Herold almost 10 years ago

  • Related to Feature #2839: Implement Iterative Stable Alignment and Clustering (ISAC) on a 2-D image stack added
Actions #3

Updated by Sargis Dallakyan almost 10 years ago

Amber Herold wrote:

Sargis, will Monday August 4 be a good day for us to do the update?

Monday August 4 sounds good. This will be a good day for us to do web code, processing code and config files updates.

Actions #4

Updated by Amber Herold almost 10 years ago

Garibaldi has been been updated with these changes, so you should be able to select a queue when running there. I have found at least 4 features that are not behaving properly:
  1. Xmipp Maximum Likelihood Alignment
  2. Xmipp2 clustering 2D Alignment
  3. Simple
  4. Optimod

When you select a new processing host, the job is submitted right away instead of loading the available queues. To get around this, use the release version of these on myamiweb for today.

The bug is because we call this.form.submit() to reload the page with the queue info, but is somehow triggering the submit button, even though it is named "process" instead of "submit". It would be nice to replace this.form.submit with a bit of javascript to add the queue selector.

Actions #5

Updated by Amber Herold almost 10 years ago

  • Status changed from Assigned to In Code Review
  • Assignee changed from Amber Herold to Sargis Dallakyan

Turns out the issue with the submit is due to processing files that would submit the form for processing on

if ($_POST)
instead of
if ($_POST['process'])
. I found a number of other files with the same issue. There may be others that I missed so keep an eye out for that.

Everything should be functioning when the installations are updated tomorrow morning.

Ready for code review.

Actions #6

Updated by Neil Voss about 9 years ago

  • Related to Bug #2939: Appion Developers Conference Call Notes added
Actions #7

Updated by Anchi Cheng over 8 years ago

  • Status changed from In Code Review to Closed

Changed the rest of php scripts I could find in tomography part for completeness. These are outdated with protomo II integration in 3.2 and unrelated to the feature here.

Actions

Also available in: Atom PDF