Appion

I've looked to see where the default memory of 2gb is coming. This is set in myami queue on garibaldi:

garibaldi00 align/cl2d7> qstat -f -Q myami
Queue: myami
    queue_type = Execution
    total_jobs = 2
    state_count = Transit:0 Queued:0 Held:0 Waiting:0 Running:2 Exiting:0 
    resources_max.cput = 200000:00:00
    resources_max.mem = 9588gb
    resources_max.ncpus = 1760
    resources_max.nodect = 173
    resources_max.walltime = 900:00:00
    resources_default.cput = 01:00:00
    resources_default.mem = 2gb <=====
    resources_default.ncpus = 1
    resources_default.nodect = 1
    resources_default.nodes = 1
    resources_default.walltime = 00:00:12
    mtime = 1385248723
    resources_assigned.mem = 45097156608b
    resources_assigned.ncpus = 2
    resources_assigned.nodect = 3
    max_user_run = 200
    keep_completed = 60
    enabled = True

I found 'memorymax' => '47' line in myamiweb/config.php for garibaldi host. I'll read through the code to see how to use this to set the default for requested memory.

Added Processing Host Parameters options to Xmipp 3 Clustering 2D Alignment as shown in the image below.

Users can now specify the amount of memory needed (in gb) and it defaults to 47gb for garibaldi.

I can add in the mem parameter today.

David,
Go ahead and give this a try. Should be working on longboard today. Everywhere else tomorrow.

Hi,

So i'm pretty sure cl2d2 will not work at all anymore.

On longboard/beta I get this error after the job is submitted:

!!! WARNING: could not create stack average, average.mrc
... Inserting CL2D Run into DB

lines= ['\tlibmpi.so.1 => /usr/lib64/libmpi.so.1 (0x0000003c67000000)\n', '\tlibmpi_cxx.so.1 => /usr/lib64/libmpi_cxx.so.1 (0x00007f4a85cb7000)\n']
/ami/data00/appion/14jul31e/align/cl2d15/alignedStack.hed
Traceback (most recent call last):
File "/opt/myamisnap/bin/runXmippCL2D.py", line 624, in <module>
cl2d.start()
File "/opt/myamisnap/bin/runXmippCL2D.py", line 605, in start
self.insertAlignStackRunIntoDatabase("alignedStack.hed")
File "/opt/myamisnap/bin/runXmippCL2D.py", line 386, in insertAlignStackRunIntoDatabase
apDisplay.printError("could not find average mrc file: "+avgmrcfile)
File "/opt/myamisnap/lib/appionlib/apDisplay.py", line 65, in printError
raise Exception, colorString("\n * FATAL ERROR *\n"+text+"\n\a","red")
Exception: * FATAL ERROR *
could not find average mrc file: /ami/data00/appion/14jul31e/align/cl2d15/average.mrc

And on cronus3/beta I get this error and it won't even submit
ERROR in job submission. Check the cluster setup. Ensure the ,.appio.cfg configuration file is correct (http://emg.nysbc.org/projects/appion/wiki/Configure_appioncfg) Job type: partalign partalign ['runXmippCL2D.py', '--stack=116', '--lowpass=15', '--highpass=2000', '--num-part=1999', '--num-ref=20', '--bin=2', '--max-iter=15', '--nproc=32', '--fast', '--classical_multiref', '--correlation', '--commit', '--nodes=4', '--ppn=8', '--mem=180', '--walltime=240', '--cput=24000', '--queue=myami', '--description=data00', '--runname=cl2d2', '--rundir=/ami/data00/appion/14jul31e/align/cl2d2', '--projectid=414', '--expid=13758', '--jobtype=partalign', '--jobid=548']

ERROR in job submission. Check the cluster setup. Ensure the ,.appio.cfg configuration file is correct (http://emg.nysbc.org/projects/appion/wiki/Configure_appioncfg) Job type: partalign partalign ['/opt/myamisnap/bin/appion', 'runXmippCL2D.py', '--stack=116', '--lowpass=15', '--highpass=2000', '--num-part=1999', '--num-ref=20', '--bin=2', '--max-iter=15', '--nproc=14', '--fast', '--classical_multiref', '--correlation', '--commit', '--nodes=4', '--ppn=4', '--walltime=2', '--cput=200', '--description=test', '--runname=cl2d6', '--rundir=/ami/data15/appion/14jul31e/align/cl2d6', '--projectid=414', '--expid=13758', '--jobtype=partalign', '--jobid=551']