Project

General

Profile

Actions

Feature #366

closed

Need to be able to define max number of procs per node for each processing host - Advanced version

Added by Neil Voss over 14 years ago. Updated over 13 years ago.

Status:
Closed
Priority:
High
Assignee:
Christopher Irving
Category:
-
Start date:
04/20/2010
Due date:
% Done:

100%

Estimated time:

Description

Need to be able to define max number of procs per node for each processing host. Goby is 4, Guppy is 8, and currently we have a hack in processing.inc that look like this:

source:trunk/myamiweb/processing/inc/processing.inc#L902

we should pull this from the config file.


Related issues 1 (0 open1 closed)

Related to Appion - Feature #422: Need to be able to dynamically define max number of procs per node for each processing hostDuplicate04/20/2010

Actions
Actions #1

Updated by Eric Hou over 14 years ago

We should have this number in the config file. So when Administrator enter server config file (config_processing.php), each server should have the max processing number go with it.

Thanks.

Eric

Actions #2

Updated by Jim Pulokas over 14 years ago

You may want to determine this from the cluster itself. You can query various info using commands such as:
qmgr -c "list queue @guppy"
qmgr -c "list node @guppy"
(or pbsnodes command is similar)

This will tell you the number of nodes, number of processors per node, which processors are currently in use, how much memory, wall time, etc. it available. It can give a lot of information that the user may want to know before submitting a job. For instance, if only three out of four nodes are free, then you may want to submit a job for three nodes rather than wait for all four to become free. Also, if a node needs to be taken down for maintainance, the web page would automatically be updated with the max nodes available, preventing a job request from being queued indefinitely because too many nodes were requested.

You may still want extra control on top of that, for instance, restricting users to less than the maximum number of nodes/procs available. Probably best to configure those types of restrictions on the server (qmgr) and then let the user query what resources are available to them. This would also be ideal in the case where you may want to give certain users more/less restriction than others. You can set up multiple queues where each one has different resource allocations.

See:
http://www.clusterresources.com/products/torque/docs/4.1queueconfig.shtml

Actions #3

Updated by Amber Herold over 14 years ago

Eric will add this field to the config file for now (getNumberofProcer). We will return to this issue after the 2.0 release. Please do not Close this issue OR create a new one for the Dynamic option.

Actions #4

Updated by Eric Hou over 14 years ago

For now, we change the Global variable $PROCESSING_HOST to an array holding the name of the host pair with the number of processing nodes. (example provided in the config.php.template).

To test this: edit your config file. Add more than 2 processing host.
Select any tools in image processing pipeline.
You should see a list of hosts show up.
and select processing node more than the number setup in the config file.
You should get an error.

Thanks.

Eric

Actions #5

Updated by Eric Hou over 14 years ago

  • Status changed from Assigned to In Code Review
  • Assignee changed from Eric Hou to Amber Herold
  • % Done changed from 0 to 90

r13671

Actions #6

Updated by Amber Herold over 14 years ago

  • Status changed from In Code Review to In Test
Actions #7

Updated by Neil Voss over 14 years ago

  • % Done changed from 90 to 100
Actions #8

Updated by Amber Herold over 14 years ago

  • Status changed from In Test to Closed
Actions #9

Updated by Amber Herold over 14 years ago

  • Subject changed from Need to be able to define max number of procs per node for each processing host to Need to be able to define max number of procs per node for each processing host - Advanced version
  • Status changed from Closed to New
  • Assignee deleted (Amber Herold)
  • Target version changed from Appion/Leginon 2.0.0 to Appion/Leginon 2.1.0

This has been implemented with the config file, need to revisit Jims comments for v2.1

Actions #10

Updated by Amber Herold over 14 years ago

  • Tracker changed from Bug to Feature
Actions #11

Updated by Neil Voss over 14 years ago

My favorite command is:

pbsnodes -a | egrep "resources_available.(mem|ncpus)" | sort | uniq -c

Actions #12

Updated by Amber Herold over 14 years ago

  • Target version changed from Appion/Leginon 2.1.0 to Appion/Leginon 2.2.0
Actions #13

Updated by Eric Hou about 14 years ago

  • Assignee set to Eric Hou
Actions #14

Updated by Amber Herold about 14 years ago

  • Deliverable set to Public cluster (sdsc)
Actions #15

Updated by Eric Hou over 13 years ago

  • Assignee changed from Eric Hou to Amber Herold
Actions #16

Updated by Amber Herold over 13 years ago

  • Assignee changed from Amber Herold to Christopher Irving
  • Target version changed from Appion/Leginon 2.2.0 to Appion/Leginon Future Version
Actions #17

Updated by Christopher Irving over 13 years ago

  • Status changed from New to Closed

This was fix by Amber in #532

Actions

Also available in: Atom PDF