Project

General

Profile

Job submission vs direct Appion Script running from terminal » History » Version 1

Anchi Cheng, 04/04/2012 10:29 PM

1 1 Anchi Cheng
h1. Job submission vs direct Appion Script running from terminal
2
3
We have two schools of developers. 
4
# Some has only single node computer and wants to run appionScript without going through resource manager by constructing Appion script and options directly on the terminal.  This means that appionScript need to be able to manage the resource without a resource manager.  The users would use *Show Commands* in myamiweb
5
# Some wants to move all appion job running through job submission so that resource managers are used in a large cluster of nodes.  The users would use *Submit job* in myamiweb.
6
7
8
This wiki is to organize my (Anchi) findings on what modules does what and how options are passed and set and what are logged into database and when in the two different way of running Appion currently.
9
10
h2 Show Command path
11
12
h3 during instance construct with "__init__"()
13
14
# createDefaultStats
15
# set self.functionname
16
# set self.appiondir
17
# setParams
18
## setupGlobalParseOptions (I have not found any script not doing this step even though it can be turned off)
19
## setupParserOptions (subclasses overwrite appionScript base class that adds stackid).
20
## convertOptions to self.params
21
# set processing database using projectid (don't see how we can have a situation without projectid any more).
22
# check conflicts
23
## checkConflicts from subclasses of appionScript
24
## checkGlobalConflicts
25
# set up run directory
26
# start pool of threads (set to 2 now and will time out if not used in 10 seconds)
27
# writeFunctionLog
28
# uploadScriptData
29
## ProgramName
30
## UserName
31
## Host
32
## ProgramRun (need cluster job id)
33
### query for ApAppionJobData by
34
#### path = self.params['rundir']
35
#### jobtype = self.functionname
36
### insert if not exist
37
## ParamName and ParamValues
38
39
h3 start()
40
41
h3 close()