Project

General

Profile

Job submission vs direct Appion Script running from terminal » History » Version 5

Anchi Cheng, 04/04/2012 11:20 PM

1 1 Anchi Cheng
h1. Job submission vs direct Appion Script running from terminal
2
3
We have two schools of developers. 
4
# Some has only single node computer and wants to run appionScript without going through resource manager by constructing Appion script and options directly on the terminal.  This means that appionScript need to be able to manage the resource without a resource manager.  The users would use *Show Commands* in myamiweb
5
# Some wants to move all appion job running through job submission so that resource managers are used in a large cluster of nodes.  The users would use *Submit job* in myamiweb.
6
7
8
This wiki is to organize my (Anchi) findings on what modules does what and how options are passed and set and what are logged into database and when in the two different way of running Appion currently.
9
10 2 Anchi Cheng
h2. Show Command path (*Bold* are not performed in BasicScript)
11 1 Anchi Cheng
12 4 Anchi Cheng
h3. "__init__()"
13 1 Anchi Cheng
14
# createDefaultStats
15 5 Anchi Cheng
# set self.functionname as basename of <pre>sys.argv[0]</pre>
16 2 Anchi Cheng
# *set self.appiondir*
17 1 Anchi Cheng
# setParams
18 2 Anchi Cheng
## *setupGlobalParseOptions* (I have not found any appion script not doing this step even though it can be turned off)
19 1 Anchi Cheng
## setupParserOptions (subclasses overwrite appionScript base class that adds stackid).
20
## convertOptions to self.params
21
# set processing database using projectid (don't see how we can have a situation without projectid any more).
22
# check conflicts
23
## checkConflicts from subclasses of appionScript
24 2 Anchi Cheng
## *checkGlobalConflicts*
25
# *set up run directory*
26
# *start pool of threads* (set to 2 now and will time out if not used in 10 seconds)
27 1 Anchi Cheng
# writeFunctionLog
28 2 Anchi Cheng
# *uploadScriptData*
29 1 Anchi Cheng
## ProgramName
30
## UserName
31
## Host
32
## ProgramRun (need cluster job id)
33
### query for ApAppionJobData by
34
#### path = self.params['rundir']
35
#### jobtype = self.functionname
36
### insert if not exist
37 2 Anchi Cheng
### *set ApAppionJobData to Run*
38 1 Anchi Cheng
## ParamName and ParamValues
39
40 2 Anchi Cheng
h3. start()
41 1 Anchi Cheng
42 2 Anchi Cheng
h3. close()
43
# closeFunctionLog
44
# *set ApAppionJobData to Done*