Job submission vs direct Appion Script running from terminal » History » Version 4
Anchi Cheng, 04/04/2012 11:18 PM
1 | 1 | Anchi Cheng | h1. Job submission vs direct Appion Script running from terminal |
---|---|---|---|
2 | |||
3 | We have two schools of developers. |
||
4 | # Some has only single node computer and wants to run appionScript without going through resource manager by constructing Appion script and options directly on the terminal. This means that appionScript need to be able to manage the resource without a resource manager. The users would use *Show Commands* in myamiweb |
||
5 | # Some wants to move all appion job running through job submission so that resource managers are used in a large cluster of nodes. The users would use *Submit job* in myamiweb. |
||
6 | |||
7 | |||
8 | This wiki is to organize my (Anchi) findings on what modules does what and how options are passed and set and what are logged into database and when in the two different way of running Appion currently. |
||
9 | |||
10 | 2 | Anchi Cheng | h2. Show Command path (*Bold* are not performed in BasicScript) |
11 | 1 | Anchi Cheng | |
12 | 4 | Anchi Cheng | h3. "__init__()" |
13 | 1 | Anchi Cheng | |
14 | # createDefaultStats |
||
15 | 4 | Anchi Cheng | # set self.functionname as basename of "sys.argv[0]" |
16 | 2 | Anchi Cheng | # *set self.appiondir* |
17 | 1 | Anchi Cheng | # setParams |
18 | 2 | Anchi Cheng | ## *setupGlobalParseOptions* (I have not found any appion script not doing this step even though it can be turned off) |
19 | 1 | Anchi Cheng | ## setupParserOptions (subclasses overwrite appionScript base class that adds stackid). |
20 | ## convertOptions to self.params |
||
21 | # set processing database using projectid (don't see how we can have a situation without projectid any more). |
||
22 | # check conflicts |
||
23 | ## checkConflicts from subclasses of appionScript |
||
24 | 2 | Anchi Cheng | ## *checkGlobalConflicts* |
25 | # *set up run directory* |
||
26 | # *start pool of threads* (set to 2 now and will time out if not used in 10 seconds) |
||
27 | 1 | Anchi Cheng | # writeFunctionLog |
28 | 2 | Anchi Cheng | # *uploadScriptData* |
29 | 1 | Anchi Cheng | ## ProgramName |
30 | ## UserName |
||
31 | ## Host |
||
32 | ## ProgramRun (need cluster job id) |
||
33 | ### query for ApAppionJobData by |
||
34 | #### path = self.params['rundir'] |
||
35 | #### jobtype = self.functionname |
||
36 | ### insert if not exist |
||
37 | 2 | Anchi Cheng | ### *set ApAppionJobData to Run* |
38 | 1 | Anchi Cheng | ## ParamName and ParamValues |
39 | |||
40 | 2 | Anchi Cheng | h3. start() |
41 | 1 | Anchi Cheng | |
42 | 2 | Anchi Cheng | h3. close() |
43 | # closeFunctionLog |
||
44 | # *set ApAppionJobData to Done* |