Project

General

Profile

Appion Home » History » Version 21

Amber Herold, 03/31/2010 04:35 PM

1 1 Eric Hou
2 11 Amber Herold
h1. Complete Installation
3 6 Amber Herold
4
5 1 Eric Hou
6 16 Amber Herold
7 11 Amber Herold
h2. Four Parts of Leginon System
8 1 Eric Hou
9 11 Amber Herold
10 16 Amber Herold
!http://emg.nysbc.org/software/leginon/images/images/Leg_Proj_parts.png!
11 1 Eric Hou
12 11 Amber Herold
Leginon as a system that we distribute can be divided into three parts:
13 1 Eric Hou
14 6 Amber Herold
15 16 Amber Herold
16 11 Amber Herold
h2. Processing Server (Python-side)
17 1 Eric Hou
18
19 11 Amber Herold
Python (and some c) scripts that handle instrument control, data acquisition, and
20
processing.
21 1 Eric Hou
22
23
24
25 16 Amber Herold
26 11 Amber Herold
h2. Database Server (MySQL-side)
27 1 Eric Hou
28 11 Amber Herold
29
a MySQL server that handles the database
30
31
32 1 Eric Hou
33 11 Amber Herold
34 16 Amber Herold
35 11 Amber Herold
h2. Web Server (PHP-side)
36
37
38
This includes php and Java scripts at a webserver that we will create to retrieve image
39
and metadata from the database and file-storage system.
40
41
42 1 Eric Hou
43 11 Amber Herold
44 16 Amber Herold
45 11 Amber Herold
h2. File Server
46
47
48
This is up to you to set up to store lots of data coming out of Leginon system.
49
50
51
52
53 1 Eric Hou
54 11 Amber Herold
55 16 Amber Herold
56 11 Amber Herold
h2. What is in this chapter
57
58
59
60
Leginon runs under both the Linux and Microsoft Windows Operating systems. However, the
61
current php-mrctool can not be installed on Windows which means mrc images can not be viewed
62
on the web. See the section on Possible Computer Set-up Configurations for details.
63
64
65
The many components for the Leginon system are divided into packages and need to be
66
installed separately. It has been integrated with several third party software packages that
67
are necessary for its operation. The installation documentation will describe how to set-up
68
Leginon and give hints as to how to install and set-up the necessary third-party software.
69
This documentation is not intended to support the additional, but necessary third-party
70
software.
71
72
73
There has been interest in installing Leginon under Mac OS X. While this is possible in
74
theory, we have not been successful in installing a fully functional system. The two main
75
problems are (1) compilation of php-mrctool from the pre-installed php and (2) wxPython on Mac
76
is unable to hide bitmap objects which makes Leginon graphical user interface difficult to
77
use.
78
79
80 15 Amber Herold
See <link linkend="InstT_install">Installation Troubleshooting</link> and "Leginon Bulletin Board":http://emg.nysbc.org/bb/viewforum.php?f=2 searching
81 11 Amber Herold
for "install" if you run into problems.
82
83
84 1 Eric Hou
85 11 Amber Herold
86 16 Amber Herold
87 11 Amber Herold
h2.  Possible Computer Set-up Configurations
88
89
90
91
Computer(s) used for the whole Leginon system need to support five functions:
92
*  TEM/Camera control and data output (by python-side packages of Leginon
93
system)
94
95
96
*The computer attached to the microscope/camera is used for this function.
97
Therefore, it is on a Windows computer.
98
99
100
*  General Leginon operation such as target selection and image processing (processing
101
server of Leginon system)
102
103
104
*  Image data storage.
105
106
107
*  Other meta-data storage through MySQL database (by database server of Leginon
108
system)
109
110
111
*  Web server (using PHP-side of Leginon system).
112
113
114
*Important* The MRC module that is used to load mrc formatted files to the web
115
pages is not compiled on Windows. Therefore, this function must be on a linux box unless
116
you don't need to see the images***
117
118
119
120
121
122
The five functions can be distributed to five different computers on a network or all on
123
one single computer. However, since the latter minimal setup requires the use of the computer
124
attached to the microscope for all functions, it is not advisable.
125
126
127
Processing server of Leginion is a multi-platform software, meaning it can run on both
128
Microsoft Windows and Linux. It is also inherently designed to utilize distributed systems on
129
a network. MySQL used in the database server is also supported by multiple platforms. This is
130
not true for the Web server because the custom MRC module used in the web viewer only compiles
131
on linux machines. Therefore, one of the computer not part of the microscope must be a linux.
132
The following examples show several arrangements that take advantage of distributed system.
133
Other combinations are possible, but will basically be variations on one of the following
134
themes.
135 1 Eric Hou
136 11 Amber Herold
137 16 Amber Herold
138 11 Amber Herold
h2.  A: 1 Windows computer (attached to the Microscope) &amp; 1 Linux computer (for all
139
other functions, separate from the Microscope) (Good for single user and small-scale
140
acquisition)
141
142
143
144
145
146
*  Install the latest processing server side of Leginon release (and supporting
147
packages) on the Windows computer controlling the microscope and on the Linux computer
148
which is to run Leginon image acquisition and processing.
149
150
151
*  Install the additional modules needed for CCD camera and film interaction on the
152
Windows computer controlling the microscope.
153
154
155
*  Install MySQL, PHP, the Apache Web Server, and dbem tools only on the Linux
156
computer that will perform these functions, (i.e. NOT the computer controlling the
157
microscope).
158
159
160
161
162
163
164 1 Eric Hou
165 11 Amber Herold
166 16 Amber Herold
167 11 Amber Herold
h2.  B: 1 Windows computer (attached to the Microscope) &amp; 4 Linux computers (one for
168
each function) (Good for multiple microsopes, users and large-scale acquisition)
169
170
171
172
This is the set-up at NRAMM. It has the processing server side of Leginon installed on
173
the Windows computer attached to the microscope and on the Linux computer that is used for
174
running Leginon. A second Linux machine is dedicated to the database while the web-based
175
viewer is hosted by another server.
176
177
178
179
180
*  Install the python-side Leginon (and supporting packages) on the Windows computer
181
controlling the microscope and on the Linux computer that is to run Leginon. You can
182
make the latter installation accessable to any numbers of linux computers on your
183
network.
184
185
186
*  Install MySQL, PHP, the Apache Web Server, and dbem tools on the second Linux
187
computer that will serve as the database/web server/PHP host.
188
189
190
*  The computer hosting the data storage should be accessable by all linux
191
boxes.
192
193
194
195
196
197
198 1 Eric Hou
199 11 Amber Herold
200 16 Amber Herold
201 11 Amber Herold
h2.  C: 1 Windows computer (attached to the Microscope), 1 Windows computer running python-
202
and MySQL-side functions and 1 Linux computer as web server (Has been applied
203
successfully)
204
205
206
207
The data storage may be on one of these computers that is accessable by other
208
computers.
209
210
211
212
213 1 Eric Hou
214 11 Amber Herold
215 16 Amber Herold
216 11 Amber Herold
h2.  Ports that Leginon system uses:
217
218
219
220
This information is useful if you want to open specific ports between computers with a
221
firewall in between.
222
223
224
225
226
* MySQL normally uses port 3306 (but configurable in /etc/my.cnf)
227
228
229
* Leginon Client running on TEM host: port 55555
230
231
232
* Leginon main program on the processing server: dynamically assigned ports between
233
49152 and 65535
234
235
236
There is no strict port assignment since we could potentially have more than one
237
Leginon process running on the same linux host talking to different TEM hosts. It is
238
probably good enough if you only worry about opening up the first few of those ports in
239
your firewall (maybe 49152 through 49160, or something like that).
240
241
242
See more discussion at Leginon bulletin board thread on ""network problem, Leginon not seeing
243 15 Amber Herold
tecnai host":http://emg.nysbc.org/bb/viewtopic.php?t=7 "
244 11 Amber Herold
245
246
247
248
249
250 1 Eric Hou
251 11 Amber Herold
252 16 Amber Herold
253 15 Amber Herold
h2. Where to Register and download Leginon system
254 11 Amber Herold
255
256
257
"http://www.leginon.org/":http://www.leginon.org is the home. Please
258
register if you want to recieve support from Leginon team.
259
260 1 Eric Hou
261
*  python-side Leginon packages
262
263 11 Amber Herold
264 16 Amber Herold
direct link: "http://emg.nysbc.org/software/leginon/":http://emg.nysbc.org/software/leginon/ *  mrctools: including the tools for reading mrc images on the web
265 11 Amber Herold
266
267 16 Amber Herold
direct link: "http://emg.nysbc.org/software/mrctools/":http://emg.nysbc.org/software/mrctools/ *  dbemtools: including the tools for administration and webviewing
268 1 Eric Hou
269 11 Amber Herold
270 16 Amber Herold
direct link: "http://emg.nysbc.org/software/dbemtools/":http://emg.nysbc.org/software/dbemtools/ *  project tools (optional): For administration and webviewing of projects
271 11 Amber Herold
272
273 16 Amber Herold
direct link: "http://emg.nysbc.org/software/dbemtools/":http://emg.nysbc.org/software/dbemtools/ h2.  Processing server-side Leginon Installation
274 11 Amber Herold
275
276
277 1 Eric Hou
For the lack of better name, processing server-side Leginon refers to packages that
278 11 Amber Herold
performs the general functions of image acquisition and processing. The instructions here are
279 1 Eric Hou
for Linux. Refer to Windows Installation for Windows specific instructions.
280 11 Amber Herold
281
282 16 Amber Herold
283 11 Amber Herold
h2. Download svn checkout script from Leginon website
284
285
286
287
"http://www.leginon.org/":http://www.leginon.org/ Download
288 15 Amber Herold
svnget-1.6.sh to a convenient location such as your home directory.
289 11 Amber Herold
290
291
292
293 16 Amber Herold
294 11 Amber Herold
h2. Check out SVN Source Files from the depository
295
296
297
298
run svnget-1.6.sh. It does the
299 17 Amber Herold
following:<pre>mkdir Leginon-1.6-ALL
300 11 Amber Herold
cd Leginon-1.6-ALL
301
svn co http://emg.nysbc.org/svn/leginon/branches/1.6 Leginon
302 1 Eric Hou
svn co http://emg.nysbc.org/svn/pyami/branches/leg1.6 pyami
303 11 Amber Herold
svn co http://emg.nysbc.org/svn/pyScope/branches/leg1.6 pyScope
304
svn co http://emg.nysbc.org/svn/sinedon/branches/leg1.6 sinedon
305
svn co http://emg.nysbc.org/svn/numextension/branches/leg1.6 numextension
306
svn co http://emg.nysbc.org/svn/libcv/branches/leg1.6 libcv
307
svn co http://emg.nysbc.org/svn/ImageViewer/branches/leg1.6 ImageViewer</pre>
308
309
310
If the process fails, you probably do not have svn client installed. Skip to "Install
311
the supporting packages first if missing:", then try this again.
312
313
314
315
316 16 Amber Herold
317 11 Amber Herold
h2.  Perform system check:
318
319
320
321
In addition to the downloads from our svn depository, there are several other
322
requirements that you will get either from your OS installation source, or from its
323
respective website. The system check in the Leginon package checks your system to see if you
324
already have these requirements
325
326 1 Eric Hou
327 11 Amber Herold
328
<pre>cd your_download_area/Leginon-1.6-ALL/Leginon
329
python syscheck.py</pre>
330
331
332
333
If python is not installed, this, of course will not run. If you see any lines like "***
334
Failed...", then you have something missing. Otherwise, everything should result in
335
"OK".
336
337
338
339
340 16 Amber Herold
341 11 Amber Herold
h2.  Install the supporting packages first if missing:
342
343
344
345
Follow the instruction for your specific Linux distribution.
346 1 Eric Hou
347 11 Amber Herold
348
For example, SUSE users can use YaST to install them; RedHat and CentOS users can use
349
yum.
350
351
352
*SciPy may not build properly on some versions of SuSE due to an incompatible LAPACK
353
package that comes with SuSE. You can get scipy as well as a compatible LAPACK etc. from
354
http://repos.opensuse.org/science (need to specify your SuSE version and machine
355
etc.)
356
357
358 1 Eric Hou
359 11 Amber Herold
360 16 Amber Herold
361 11 Amber Herold
h2.  Required supporting packages:
362
363 1 Eric Hou
364 11 Amber Herold
365 14 Amber Herold
366 11 Amber Herold
|_.Name:|_.Download site:|_.yum package name|_.SuSE rpm name|
367 15 Amber Herold
|Python 2.4 or newer|"http://www.python.org":http://www.python.org |python|python-devel|
368
|wxPython 2.5.2.8 or newer|"http://www.wxpython.org":http://www.wxpython.org |wxPython|python-wxGTK|
369
|MySQL Python client 1.2 or newer|"http://sourceforge.net/projects/mysql-python":http://sourceforge.net/projects/mysql-python |MySQL-python|python-mysql|
370
|Python Imaging Library (PIL) 1.1.4 or newer|"http://www.pythonware.com/products/pil/":http://www.pythonware.com/products/pil/ |python-imaging|python-imaging|
371
|Python XML module 0.8.3 or newer|"http://pyxml.sourceforge.net":http://pyxml.sourceforge.net |PyXML|python-xml|
372 1 Eric Hou
|NumPy 1.0.1 or newer|"http://www.scipy.org":http://www.scipy.org |numpy|numpy|
373 15 Amber Herold
|SciPy 0.5.1 (tested, others may work)*|"http://www.scipy.org":http://www.scipy.org , "http://repos.opensuse.org/science":http://repos.opensuse.org/science *|scipy|python-scipy|
374 11 Amber Herold
375
376
377
378
*SciPy may not build properly on some versions of SuSE due to an incompatible LAPACK
379
package that comes with SuSE. You can get scipy as well as a compatible LAPACK etc. from
380
http://repos.opensuse.org/science (need to specify your SuSE version and machine
381 1 Eric Hou
etc.)
382 11 Amber Herold
383
384
385
386 16 Amber Herold
387 11 Amber Herold
h2.  Optional supporting packages:
388
389
390
391
392
393
*  For using Matlab to explore holefinding algorithm (Matlab HoleFinder)
394
395
396 16 Amber Herold
pymat: "http://sourceforge.net/projects/pymat":http://sourceforge.net/projects/pymat h2. Install the packages you downloaded from svn depository
397 11 Amber Herold
398
399
400
Here are the packages you need to install with python installer inside the
401
Leginon-1.6-ALL folder.
402 1 Eric Hou
403 11 Amber Herold
404 14 Amber Herold
405 11 Amber Herold
|_.Name:|_.Purpose:|
406
| Leginon|modular TEM image acquisition|
407
| pyami|general functions|
408
| pyScope|microscope control and monitoring|
409
| sinedon|Leginon/database interaction|
410
| numExtension|c extension for numerical processing|
411
| libCV|small c library of algorithm from computer vision field|
412
| ImageViewer|image viewing for tomography|
413
414
415
416
417
418
419
*  run syscheck.py again to make sure you have everything.
420
421 1 Eric Hou
422 19 Amber Herold
*  Install the package in _each folder_ with commands
423 11 Amber Herold
like
424
these<pre>cd /your_download_area/Leginon-1.6-ALL/Leginon
425 1 Eric Hou
python setup.py install
426 11 Amber Herold
427
cd ../pyami
428
python setup.py install
429
cd ../pyScope
430
python setup.py install
431
cd ../sinedon
432
python setup.py install
433
cd ../numextension
434
python setup.py install
435
cd ../libcv
436
python setup.py install
437
cd ../ImageViewer
438
python setup.py install</pre>
439
440
441
442
443
444
445
446
447 16 Amber Herold
448 11 Amber Herold
h2.  python-site-package-path: where the installed python
449
packages went:
450
451
452
Python installer put the packages you installed to its site-packages directory. This
453
enable all users on the same computer to access them. The easiest way to find where your
454
installed package is called by python is to load a module from the package using interactive
455
python command lines like this:
456
457
458
459
460
*  Start python command line from shell
461
462
463
464
<pre>> python</pre>
465
466
467
468
*  Import a module from the package. Let's try sinedon here. All packages installed
469
through the above setup.py script should go to the same place.
470
471
472
473
<pre>python> import sinedon</pre>
474
475
476
477 1 Eric Hou
*  If the module is loaded successfully, call the module attribute __path__ (two
478 11 Amber Herold
underscrolls before "path" and two underscrolls after) will return the location of the
479
module it is loaded from
480
481
482
483
<pre>python> sinedon.__path__
484
485
RETURNS, For example,
486 21 Amber Herold
python> <module 'sinedon' from '/usr/lib/python2.5/site-packages/sinedon/__init__.pyc'> </pre>
487 11 Amber Herold
488
489
In this case, /usr/lib/python2.5/site-packages/ is your python-site-package-path.
490
If you go to that directory, you will find all the packages you just installed.
491
492
493
494
495
496
497
498
499 16 Amber Herold
500 11 Amber Herold
h2.  Configure leginon.cfg:
501
502
503
Leginon can be configured at three levels:
504
505
506
507
508
*  A skeleton (default) configuration file is available:
509
510 1 Eric Hou
511 11 Amber Herold
512 19 Amber Herold
<pre> _<link linkend="package_path">[python-site-package-path]</link>_/Leginon/config/default.cfg</pre>
513 11 Amber Herold
514
515
516
*  Configurations for all users
517 1 Eric Hou
518 11 Amber Herold
519
520 19 Amber Herold
<pre> _<link linkend="package_path">[python-site-package-path]</link>_/Leginon/config/leginon.cfg</pre>
521 11 Amber Herold
522
523
524 1 Eric Hou
Example of the configuration that you should set: (Here we assume your leginon
525 11 Amber Herold
database is named "dbemdata", and Leginon log into the database as a user named
526
"usr_object" with no password
527
set).<pre>[Images]
528 19 Amber Herold
path: _your_storage_disk_path_/leginon
529 11 Amber Herold
</pre>
530
531 1 Eric Hou
532 11 Amber Herold
*  Configurations for individual users that overrides the all-user configuration if
533
exists
534
535
536
537
<pre> ~/leginon.cfg</pre>
538
539 1 Eric Hou
540 11 Amber Herold
541
This configuration should include all the minimal configuration above and
542
additional configuration for individual user for
543
convenience<pre>[User]
544 19 Amber Herold
Fullname: _your_name_  #The name you entered in the leginon administration web page. Note that it is case sensitive</pre>
545 11 Amber Herold
546
547
548
549
550
551
552
553 16 Amber Herold
554 11 Amber Herold
h2.  Names used in the example installation
555
556
557
From this point on, you will need to enter database user names for different
558
configuration files and settings. These are NOT Leginon user names but what is required by
559
MySQL database interaction. Consistent names are essential. In the example shown here, the
560
leginon database is called "dbemdata", the project database is called "projectdata". The
561
MySQL user name are the same, "usr_object", and in this case without a password.
562
563
564
565
566 16 Amber Herold
567 11 Amber Herold
h2.  Configure sinedon.cfg:
568
569
570
Sinedon is designed to be able to interact with multiple databases. It can be
571
configured at global or user level.
572
573
574
575
576
*  An example configuration file is available at:
577
578
579
580
<pre>/your_download_area/Leginon-1.6-ALL/sinedon/examples/sinedon.cfg</pre>
581
582 1 Eric Hou
583 11 Amber Herold
584
*  Configurations for all users should be placed at the following path
585
586
587
588 19 Amber Herold
<pre>  _<link linkend="package_path">[python-site-package-path]</link>_/sinedon/sinedon.cfg</pre>
589 11 Amber Herold
590
591
592 1 Eric Hou
*  Modify host,db,user,passwd to what is used. Note that the user here is MySQL user
593 11 Amber Herold
Leginon uses to communicate with the database for all Leginon users. For Leginon that
594
uses Project database, set the following:
595
596
597
598
<pre>[global]
599 19 Amber Herold
host: _your_database_host_
600 11 Amber Herold
user: usr_object
601
passwd:
602
603
[projectdata]
604
db: projectdata
605
606
[leginondata]
607
db: dbemdata
608
</pre>
609
610 1 Eric Hou
611 11 Amber Herold
612
*  Add database configuration if you intend to use grid-inserting robot. The Robot2
613
module uses the database to communicate to the robot. Applications that carries the
614
name "Robot" requires this to be set. In general, using the same database as the
615
general leginon database is fine.
616
617
618
619
<pre>[robot2]
620
db: dbemdata</pre>
621
622
623
624
625
626
627
628
629 1 Eric Hou
630 11 Amber Herold
631
632 16 Amber Herold
633 11 Amber Herold
h2. Database Server-side Installation
634
635
636
637
The following is for the computer that hosts the databases. This involves installing MySQL
638
server and creation/configuration of the leginon and project databases.
639
640 1 Eric Hou
641 11 Amber Herold
642 14 Amber Herold
643 11 Amber Herold
644 15 Amber Herold
|_.Name:|_.Download site:|
645 1 Eric Hou
|MySQL-Server 5.0 or higher|"http://www.mysql.com":http://www.mysql.com |
646 15 Amber Herold
|MySQL-Client 5.0 or higher|"http://www.mysql.com":http://www.mysql.com |
647 11 Amber Herold
648
649
650
651 16 Amber Herold
652 11 Amber Herold
h2.  MySQL
653
654
655
656
657
658
*  Install MySQL-Server
659
660
661
662
663
*  Use your package installer (yum, zypper, YaST) if available.
664
665 1 Eric Hou
666 11 Amber Herold
OR
667
668
669 16 Amber Herold
*  Download the latest MySQL-server RPM for Linux from "www.mysql.com":http://www.mysql.com *  Install the MySQL-server rpm:
670 11 Amber Herold
671
672
<pre> rpm -Uvh MySQL-server-5.0.xx-y.i386.rpm</pre>
673
(substitute correct version numbers)
674
675
676
677
678
679
680
*  Install MySQL-Client
681
682
683
684
685
*  Use your package installer (yum, zypper, YaST) if available.
686
687
688
OR
689
690
691 16 Amber Herold
*  Download the latest MySQL-client RPM for Linux from "www.mysql.com":http://www.mysql.com *  Install the MySQL-client rpm:
692 11 Amber Herold
693
694
<pre> rpm -Uvh MySQL-client-5.0.xx-y.i386.rpm</pre>
695
(substitute correct version numbers)
696
697
698
699
700
701
702
*  MySQL configuration file is usually located in /usr/share/mysql. There are
703
several examples there:
704
705
706
707
<pre>&gt; ls /usr/share/mysql/my*
708
/usr/share/mysql/my-huge.cnf
709
/usr/share/mysql/my-innodb-heavy-4G.cnf
710
/usr/share/mysql/my-large.cnf
711
/usr/share/mysql/my-medium.cnf
712
/usr/share/mysql/my-small.cnf
713
&gt; </pre>
714
715
716
717
*  Configure my.cnf in /etc using my-huge.cnf as the template
718
719
720
721
722
* <pre>> cp /usr/share/mysql/my-huge.cnf /etc/my.cnf</pre>
723
724
725
726
*  Edit /etc/my.cnf to add or change query cache variables like these:
727
728
<pre>query_cache_type= 1
729
query_cache_size = 100M
730
query_cache_limit= 100M</pre>
731
732
733
734
735
736
737
738
*  start MySQL Server
739
740
<pre> &gt;/etc/init.d/mysqld start</pre>
741
742
on some installation,
743
744
<pre> &gt;/etc/init.d/mysql start</pre>
745
746
For future reference: start | stop | restart MySQL Server with similar
747
commands:
748
749
<pre> &gt;/etc/init.d/mysqld start
750
&gt;/etc/init.d/mysqld stop
751
&gt;/etc/init.d/mysqld restart</pre>
752
753
If you want to start MySQL automatically at boot on SuSE
754
755
<pre> SuSE &gt;chkconfig mysql on
756
</pre>
757
758
*  For future reference, the database location will be:
759
760
761
762
<pre>&gt; cd /var/lib/mysql
763
Directory: /var/lib/mysql
764
&gt; ls
765
yourdbserver.pid
766
ib_logfile0
767
mysql
768
mysql.sock
769
test
770
&gt; </pre>
771
772
773
774
*  Create leginon database, here we call <link linkend="db_example_names"
775
>dbemdata</link>
776
777
<pre> &gt;mysqladmin create dbemdata</pre>
778
779
780
781
*  Create project database, here we call <link linkend="db_example_names"
782
>projectdata</link> (optional)
783
784
<pre> &gt;mysqladmin create projectdata</pre>
785
786
787
788
*  Connect to mysql db
789
790
<pre>&gt;mysql mysql
791
792
mysql&gt; select user, password, host from user;
793
+------+----------+-----------+
794
| user | password | host      |
795
+------+----------+-----------+
796
| root |          | localhost |
797
| root |          | host1     |
798
|      |          | host1     |
799
|      |          | localhost |
800
+------+----------+-----------+
801
4 rows in set (0.00 sec)</pre>
802
803
804
805
*  Create and grant privileges to a user called <link linkend="db_example_names"
806
>usr_object</link> for the databases on both the localhost and other hosts involved.
807
For example, use wild card '%' for all hosts. You may also set specific privilege to
808
the user. See MySQL Reference Manual for details
809
810
<pre>mysql&gt; create user usr_object@'localhost';
811
mysql&gt; grant all privileges on dbemdata.* to usr_object@'localhost';
812
mysql&gt; grant all privileges on projectdata.* to usr_object@'localhost';</pre>
813
814
Similarly,
815 1 Eric Hou
816 11 Amber Herold
<pre>mysql&gt; create usr_object@'%';
817
mysql&gt; grant all privileges on dbemdata.* to usr_object@'%';
818
mysql&gt; grant all privileges on projectdata.* to usr_object@'%';
819
</pre>
820
821
*  Change Root passworld
822
823 19 Amber Herold
<pre>mysql&gt; update user set password=password('_a passwd_') where user="root";
824 1 Eric Hou
Query OK, 2 rows affected (0.01 sec)
825 11 Amber Herold
Rows matched: 2  Changed: 2  Warnings: 0
826 1 Eric Hou
827 11 Amber Herold
mysql&gt;^D or exit;</pre>
828
From now on, you will need to specify the password to connect to the database as
829
root user like this:
830
831
<pre>>mysql -u root -p mysql</pre>
832
833 19 Amber Herold
_***If your password change does not take effect in your next
834 11 Amber Herold
database connection, you will need to flush the mysql pivileges cache as mysql root
835
and connecting with you old password (or no password in the above
836 19 Amber Herold
case):_
837 11 Amber Herold
838
<pre>>mysql -u root mysql</pre>
839
<pre>mysql&gt; flush privileges;
840
mysql&gt;^D or exit;</pre>
841
842
*  Check MySQL variables
843
844
<pre>&gt;mysql -u usr_object dbemdata
845
846
mysql&gt; SHOW VARIABLES LIKE 'query%';
847
+------------------------------+-----------+
848
| Variable_name                | Value     |
849
+------------------------------+-----------+
850
| ft_query_expansion_limit     | 20        |
851
| have_query_cache             | YES       |
852
| long_query_time              | 10        |
853
| query_alloc_block_size       | 8192      |
854 21 Amber Herold
| query_cache_limit            | 104857600 | <<---This should correspond to your change
855 11 Amber Herold
| query_cache_min_res_unit     | 4096      |
856 21 Amber Herold
| query_cache_size             | 104857600 | <<---This should correspond to your change
857
| query_cache_type             | ON        | <<---This should correspond to your change
858 11 Amber Herold
| query_cache_wlock_invalidate | OFF       |
859
| query_prealloc_size          | 8192      |
860
+------------------------------+-----------+
861
10 rows in set (0.00 sec)
862
863
mysql&gt; exit;</pre>
864
865
866
867
*  Make sure MySQL is running
868
869
<pre>prompt:~&gt; mysqlshow
870
+--------------+
871
| Databases    |
872
+--------------+
873
| mysql        |
874 1 Eric Hou
| dbemdata     |
875 11 Amber Herold
| projectdata  |
876
+--------------+</pre>
877
878
879
880
*  Or check with the following php script (if already installed)
881
882 21 Amber Herold
<pre><?
883 1 Eric Hou
mysql_connect('your_host.your_institute.edu', 'usr_object', '','dbemdata');
884 11 Amber Herold
echo mysql_stat();
885
?&gt; </pre>
886
887
888
Output
889
890
<pre> Uptime: 1452562 Threads: 1 Questions: 618 Slow queries: 0 Opens: 117 Flush tables: 1 Open tables: 106 Queries per second avg: 0.000</pre>
891
892
893
894
895
896
897
898
899
900 16 Amber Herold
901 11 Amber Herold
h2.  Web server set up and Installation
902
903
904
905
The following applies to the computer that will host the web-accessable image viewers ( a
906
part of dbem tools) and project manager (a part of project tools).
907 1 Eric Hou
908 11 Amber Herold
909 16 Amber Herold
910 11 Amber Herold
h2. Differences between Linux flavors
911
912
913
914
Different Linux flavors often put web server and mysql-related files in different
915
locations. This can be confusing. From experience, we found the equivalent on CentOS vs
916
SuSE. Here we list them for reference. If your system use a different naming and you are
917
willing to share your experience, please send us the list. We will add it here:
918
919 1 Eric Hou
920 11 Amber Herold
921 14 Amber Herold
|_.File or Command Head|_.CentOS|_.SuSE|
922 11 Amber Herold
| php.ini|/etc/| /etc/php5/apache2/|
923
| httpd.conf|/etc/httpd/conf/|/etc/php5/apache2/|
924
| default document_root|/var/www/html/|/srv/www/htdocs/|
925
| apache start/stop/restart command head|/etc/init.d/httpd|/etc/init.d/apache2|
926
| mysql start/stop/restart command head|/etc/init.d/mysqld|/etc/init.d/mysql|
927
928
929 1 Eric Hou
930
931 11 Amber Herold
932
933 16 Amber Herold
934 11 Amber Herold
h2. Prerequisites for dbemtools, projecttools, and mrctools extension
935
936
937
938
The dbemtools are mostly php scripts that run at the web server. The followings are
939
required before installation of dbemtools and the mrc extension that handles mrc files to be
940
displayed. Some of these packages may be found on the SuSE Linux DVD or included in common
941 1 Eric Hou
package repository. MySQL and the Apache Web Server can be downloaded from their respective
942
websites.
943 11 Amber Herold
944 1 Eric Hou
945 11 Amber Herold
946 14 Amber Herold
|_.Name:|_.Download site:|_.yum package name|_.SuSE rpm name|
947 11 Amber Herold
| Apache|"www.apache.org":http://www.apache.org | httpd| apache2|
948 15 Amber Herold
| php|"www.php.net":http://www.php.net | php| php|
949
| php-devel*|"rpmfind.net/linux/RPM/Development_Languages_PHP.html":http://rpmfind.net/linux/RPM/Development_Languages_PHP.html | php-devel| php-devel|
950
| php-gd (including GD library, its development libraries and header *)|"www.php.ned/gd":http://www.libgd.org (Use gd2)| php-gd, gd-devel| php-gd,gd-devel|
951
| fftw3 library (including development libraries and header *)|"www.fftw.org":http://www.fftw.org (Use fftw3.x)| fftw3-devel| fftw3-devel|
952
953 11 Amber Herold
954
955
956
* mrctools are compiled and added to php extension with php-devel package. Mrctools use
957
GD and FFTW3 that need to be compiled from their development libraries while the extension
958
is compiled. If GD and FFTW3 sources were downloaded and compiled directly on your computer,
959
these development files are included. If (as in most cases) GD and FFTW3 are installed from
960
rpm, they are not included. An error message will appear when you attempt to compile
961
mrctools. In this case, you will need separate download and installation of GD-devel and
962
FFTW3-devel. Search "http://rpmfind.net/linux/rpm2html/":http://rpmfind.ned/linux/rpm2html/ for GD-devel and FFTW3-devel for the rpm
963 15 Amber Herold
distribution needed for your system.
964 1 Eric Hou
965 11 Amber Herold
966
967
968 16 Amber Herold
969 11 Amber Herold
h2. Installation tools
970 1 Eric Hou
971 11 Amber Herold
972
973
Use the installation tools available for your linux distribution.
974
975
976
977
978
*  For example, Use yum utility to install.
979
980
981
*  For example, Use the SuSE Linux YaST2 utility or zypper (openSuSE 10.2 and above)
982
to install.
983
984
985
* "A list of required CENTOS
986 1 Eric Hou
rpms and instruction":http://emg.nysbc.org/bb/viewtopic.php?t=238 can be found at dbemtools bulletin board at
987 11 Amber Herold
leginon.org.
988 15 Amber Herold
989 11 Amber Herold
990
991
992
993
994
995
996 16 Amber Herold
997 11 Amber Herold
h2.  PHP, PHP-devel, gd, and fftw3
998
999
1000
Install the packages using your installation tools if available. For example, to
1001
install gd as php extension you may use
1002
1003
1004
1005 17 Amber Herold
<pre>CentOS> yum install php-gd
1006 11 Amber Herold
SuSE10.2 and above> zypper install php-gd</pre>
1007
1008
1009
1010
1011
1012 16 Amber Herold
1013 11 Amber Herold
h2.  Configure php.ini
1014
1015
1016
1017
Edit the following two sections in php.ini (found as /etc/php.ini on CentOS and
1018 1 Eric Hou
/etc/php5/apache2/php.ini on SuSE) so that they look like the following:
1019 11 Amber Herold
1020
1021
1022
<pre>register_argc_argv = On</pre>
1023
1024
1025
1026
1027
<pre>short_open_tag = On</pre>
1028
1029
1030
1031
1032
<pre>max_execution_time = 300     ; Maximum execution time of each script, in seconds
1033
max_input_time = 300     ; Maximum amount of time each script may spend parsing request data
1034
memory_limit = 256M      ; Maximum amount of memory a script may consume (8MB)</pre>
1035
1036
1037
1038
You may want to increase max_input_time and memory_limit if the server is heavily used.
1039
At NRAMM, max_input_time=600 and memory_limit=4000M.
1040
1041
1042
1043
1044 16 Amber Herold
1045 11 Amber Herold
h2.  Apache Web Server
1046
1047
1048
1049
1050
1051
*  Install the Apache Web Server with the YaST or yum utility.
1052
1053
1054
*  Find "httpd.conf".
1055
1056
1057
This is /etc/httpd/conf/httpd.conf on CentOS and /etc/php5/apache2/httpd.conf on
1058
SuSE
1059
1060
1061
*  Edit the "httpd.conf" configuration file to look like the following:
1062
1063
1064
1065 21 Amber Herold
<pre><IfModule mod_dir.c&gt;
1066 11 Amber Herold
DirectoryIndex index.htm index.shtm index.html index.shtml index.php
1067 21 Amber Herold
</IfModule&gt;</pre>
1068 11 Amber Herold
1069
1070 1 Eric Hou
(Note: It may be possible to edit httpd.conf in YaST2 as well.)
1071 11 Amber Herold
1072
1073
*  Restart the web server.
1074
1075
1076
1077
<pre>apachectl restart
1078
or
1079
/etc/init.d/httpd restart     (ON CentOS)
1080
or
1081
/etc/init.d/apache2 restart   (ON SuSE)</pre>
1082
1083
1084
1085
If you want to start the web server automatically at boot on SuSE
1086
1087
<pre> SuSE &gt;chkconfig apache2 on
1088
</pre>
1089
1090
1091
1092 1 Eric Hou
1093 11 Amber Herold
1094
1095
1096 16 Amber Herold
1097 11 Amber Herold
h2. Check php information
1098 1 Eric Hou
1099 11 Amber Herold
1100
Create the following info.php in your web server document root directory (/var/www/html
1101
on CentOS. /srv/www/htdocs on SuSE. You can find its location in httpd.conf mentioned above
1102
under the line starting
1103 21 Amber Herold
DocumentRoot).<pre><?php
1104 11 Amber Herold
phpinfo();
1105
?&gt;</pre>
1106
1107 1 Eric Hou
1108 11 Amber Herold
Visit this page at http://yourhost/info.php
1109
1110
1111
You will see comprehensive tables of php and apache information, including the location
1112
of the addition .ini files, extension, include path, and what extension is enabled.
1113
1114
1115
Here is an example screen shot of the part of the info.php page that tells you where
1116
php.ini and other configuration files are.
1117
1118
1119
1120 16 Amber Herold
!http://emg.nysbc.org/software/leginon/images/images/phpini.png!
1121 11 Amber Herold
1122
1123 1 Eric Hou
1124 11 Amber Herold
1125
1126 16 Amber Herold
1127 11 Amber Herold
h2.  mrctools Installation
1128
1129
1130
1131
mrctools are installed as php extension and are required for displaying mrc files live
1132
on the web browser.
1133
1134 1 Eric Hou
1135 16 Amber Herold
1136 11 Amber Herold
h2.  Install php-devel packages on the web server if missing:
1137
1138
1139
You can check whether php-devel is installed by
1140
typing<pre> &gt;phpize</pre>
1141
1142
1143
Follow the instruction for your specific Linux distribution.
1144
1145
1146
For example, SUSE users can use YaST or zypper to install them
1147
1148
1149
1150
1151 16 Amber Herold
1152 11 Amber Herold
h2.  php-GD/FFTW3-devel
1153
1154
1155
1156
Follow instructions from the download site. These may be included already. If not,
1157
mrctools installation will fail.
1158
1159
1160
1161 1 Eric Hou
1162 16 Amber Herold
1163 11 Amber Herold
h2.  mrctools Installation
1164
1165
1166
1167
mrctools are installed from php devel directory. This is usually /usr/include/php/ext
1168
where you will find other php extension source such as gd. After the installation, mrc.so
1169
should be placed in php's extension directory (Look for "extension_dir" in
1170 1 Eric Hou
http://your_host/<link linkend="install_infophp">info.php</link>
1171 11 Amber Herold
1172
1173
1174
1175 15 Amber Herold
*  Download dbem tools from "http://emg.nysbc.org/software/mrctools":http://emg.nysbc.org/software/mrctools .
1176 11 Amber Herold
1177
1178 15 Amber Herold
*  Follow the instruction at "http://emg.nysbc.org/software/mrctools/mrc_so.php":http://emg.nysbc.org/software/mrctools/mrc_so.php for installation and
1179 1 Eric Hou
testing. The next session about checking php information can also help debugging the
1180 11 Amber Herold
installation
1181
1182
1183
1184
1185
1186
1187
1188
1189 16 Amber Herold
1190 11 Amber Herold
h2.  Check php information
1191
1192
1193
1194
Visit or refresh http://yourhost/info.php <link linkend="install_infophp">which you
1195
created earlier</link>. It should have a section looking like this (The version should
1196
correspond to what you've just installed):
1197
1198
1199
1200 16 Amber Herold
!http://emg.nysbc.org/software/leginon/images/images/phpmrc.png!
1201 11 Amber Herold
1202
1203
1204
If mrc is not listed, the extension did not get added at the right order.
1205
1206
1207
1208
1209 16 Amber Herold
1210 11 Amber Herold
h2.  Alternative approach if mrc module does not show up in info.php output
1211
1212
1213
1214
* find in the <link linkend="install_infophp">info.php web page</link> the location
1215
of "additional .ini files parsed" in the first table (such as
1216
/etc/php.d/conf.d/*).
1217
1218
1219
*  Go to the directory and make a copy of any ini file to use as a template for
1220
mrc.ini
1221
1222
1223
1224
<pre>&gt;cd [additional_ini_directory]
1225
&gt;cp gd.ini mrc.ini</pre>
1226
1227
1228
1229
*  Edit mrc.ini to the following
1230
1231 1 Eric Hou
1232 11 Amber Herold
1233
<pre>; comment out next line to disable mrc extension in php
1234
extension=mrc.so</pre>
1235
1236
1237
1238
*  Comment out mrc extension from php.ini (found in /etc/php.ini/ on a typical PHP
1239
installation)
1240
1241
1242
1243
<pre>;extension=mrc.so</pre>
1244
1245
1246
1247
*  restart your webserver
1248
1249
1250
1251
<pre>&gt; /etc/init.d/httpd restart</pre>
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261 16 Amber Herold
1262 11 Amber Herold
h2.  Web Viewing Tools (dbemtools) Linux
1263
Installation
1264
1265
1266
1267
php and java scripts for viewing images and Leginon information through the web
1268
server.
1269
1270
1271
1272
1273 15 Amber Herold
*  Download dbem tools from "http://emg.nysbc.org/software/dbemtools":http://emg.nysbc.org/software/dbemtools .
1274 11 Amber Herold
1275
1276
*  Unpack the package in your [webdirectory]
1277
1278
1279
1280
<pre>&gt;cd [webdirectory]			#/var/www/html in this example
1281
[webdirectory&gt;tar zxvf dbem_1_5_1.tgz</pre>
1282
1283
1284
1285
*  Copy config.php.template to config.php and edit the latter by adding these MySQL
1286
parameters:
1287
1288
"config.php" should be located in /var/www/html/dbem_1_5_1/ on CentOS and
1289
/srv/www/htdocs/dbem_1_5_1/ on SuSE.
1290
1291
1292
1293 21 Amber Herold
<pre><?
1294 11 Amber Herold
// ---define dbem web tools base ---//
1295
define('BASE_URL',"/dbem_1_5_1/";
1296
// --- Leginon Viewer Configuration --- //
1297
1298 1 Eric Hou
// --- Set your leginon MySQL database server parameters
1299 11 Amber Herold
1300
$DB_HOST    = "[your database host]";
1301
$DB_USER    = "<link linkend="db_example_names">usr_object</link>";
1302
$DB_PASS    = "";
1303
$DB        = "<link linkend="db_example_names">dbemdata</link>";
1304
1305
// --- XML test dataset
1306
$XML_DATA = "test/viewerdata.xml";
1307
1308
// --- Project database config
1309
1310 1 Eric Hou
$PROJECT_URL = "/project_1_2";
1311 11 Amber Herold
$PROJECT_DB_HOST = "[your database host]";
1312
$PROJECT_DB_USER = "<link linkend="db_example_names">usr_object</link>";
1313
$PROJECT_DB_PASS = "";
1314
$PROJECT_DB = "<link linkend="db_example_names">projectdata</link>";
1315
</pre>
1316
1317
1318
1319
*  Test the set-up by visiting http://yourhost/dbem_1_5_1
1320 1 Eric Hou
1321 11 Amber Herold
1322
1323
1324
1325
1326
1327
1328 16 Amber Herold
1329 11 Amber Herold
h2.  Project management tools Linux Installation
1330
1331
1332
1333
Leginon sessions can be catagorized into different projects. The projects are created
1334
and viewed trhough a set of web tools. To access them, install the tools on the web server.
1335
The same project management tools are also used to create and track appion processing
1336
databases. If you want to use our processing pipeline, Appion, in the future, you must
1337
install this.
1338
1339
1340 16 Amber Herold
1341 11 Amber Herold
h2. Prerequisites for project tools
1342
1343
1344
1345
installed dbemtools and its prerequisites
1346
1347
1348
1349
1350 16 Amber Herold
1351 11 Amber Herold
h2. project tools
1352
1353
1354
Current release: project 1.2.
1355
1356
1357
1358
1359 15 Amber Herold
*  Download project tools from "http://emg.nysbc.org/software/dbemtools":http://emg.nysbc.org/software/dbemtools .
1360 11 Amber Herold
1361
1362
*  Unpack the package in your [webdirectory]
1363
1364 1 Eric Hou
1365 11 Amber Herold
1366
<pre>&gt;cd [webdirectory]			#/var/www/html in this example
1367
[webdirectory]&gt;tar zxvf project_1_2.tgz</pre>
1368
1369
1370
1371 1 Eric Hou
*  Go to the uncompressed directory
1372 17 Amber Herold
project_1_2.<pre>> cd project_1_2</pre>
1373 11 Amber Herold
1374
1375 19 Amber Herold
* Copy config.php.template to _config.php_
1376 11 Amber Herold
1377
1378
1379
<pre>> cp config.php.template to config.php</pre>
1380
1381
1382
1383 19 Amber Herold
* Edit _config.php_: You need to insert these MySQL
1384 11 Amber Herold
parameters in project tool's config.php
1385
1386
1387
1388 1 Eric Hou
<pre>// --- Project Configuration --- //
1389 11 Amber Herold
1390
// --- Leginon database config
1391
$DBEM_PATH="../dbem_1_5_1/";
1392
1393
$DB_HOST  = "[your database host]";
1394
$DB_USER  = "usr_object";
1395 1 Eric Hou
$DB_PASS  = "";
1396 11 Amber Herold
$DB   = "<link linkend="db_example_names">dbemdata</link>";
1397
1398
// --- Project database config
1399
$PROJECT_DB_HOST = "[your database host]";
1400 1 Eric Hou
$PROJECT_DB_USER = "<link linkend="db_example_names">usr_object</link>";
1401 11 Amber Herold
$PROJECT_DB_PASS = "";
1402
$PROJECT_DB = "<link linkend="db_example_names">projectdata</link>";
1403
</pre>
1404
1405
1406
1407
*  Test the set-up by visiting http://yourhost/project_1_2
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418 16 Amber Herold
1419 11 Amber Herold
h2.  Other Tools
1420
1421
1422
This section includes tools that may help with Leginon development.
1423
1424
1425 16 Amber Herold
1426 11 Amber Herold
h2.  phpMyAdmin
1427
1428
1429
1430
1431
1432 16 Amber Herold
*  Download phpMyAdmin from Yast2 or "www.phpmyadmin.net":http://www.phpmyadmin.net *  Download phpMyAdmin into the directory where the web server has been
1433 11 Amber Herold
installed.
1434
1435
1436
1437
1438
1439
1440
*  Check phpMyAdmin installation:
1441
1442
1443
1444
<pre>&gt;rpm -qa |grep -i phpmyadmin </pre>
1445
1446
1447
1448
*  Install from tarball:
1449
1450
1451
1452
<pre>> cd [webdirectory]
1453
[webdirectory]&gt;tar zxvf phpMyAdmin-2.x.tar.gz</pre>
1454
1455
1456
1457
or Install with yum if availabe:
1458
1459
1460
1461
<pre>> yum install phpMyAdmin</pre>
1462
1463
1464
1465
*  Create phpMyAdmin configuration file called config.inc.php from a
1466
sample.
1467
1468
1469
1470
<pre>[webdirectory]&gt;cd phpMyAdmin
1471
(if you install with yum, the program directory is /usr/share/phpMyAdmin)
1472
1473
> cp config.sample.inc.php config.inc.php
1474
</pre>
1475
1476
1477 1 Eric Hou
1478 20 Amber Herold
*  Make sure these are set correctly in _config.inc.php_. Note that these are for MySQL which we set earlier.
1479 11 Amber Herold
You can set specific user, but it is safer just leave it blank.
1480
1481
1482
1483
<pre>
1484
$cfg['Servers'][$i]['host']            = '[your host]';
1485
$cfg['Servers'][$i]['port']            = '';
1486
$cfg['Servers'][$i]['socket']          = '';
1487
$cfg['Servers'][$i]['connect_type']    = 'tcp';
1488
$cfg['Servers'][$i]['extension']       = 'mysql';
1489
$cfg['Servers'][$i]['compress']        = FALSE;
1490
$cfg['Servers'][$i]['controluser']     = '';
1491
$cfg['Servers'][$i]['auth_type']     = 'http';
1492
$cfg['Servers'][$i]['user']          = '';
1493
$cfg['Servers'][$i]['password']      = '';
1494
</pre>
1495
1496
1497
1498
*  For additional security, you may restrict mysql root access
1499
1500
1501
1502
<pre>
1503 1 Eric Hou
$cfg['Servers'][$i]['AllowRoot']     = FALSE;
1504 11 Amber Herold
</pre>
1505
1506
1507
1508 1 Eric Hou
*  If you want to access phpMyAdmin from another computer, you can add it to its
1509 11 Amber Herold
web access configuration file found as /etc/httpd/conf.d/phpMyAdmin.conf in a
1510
typical installation
1511
1512
1513 1 Eric Hou
1514 11 Amber Herold
<pre>
1515 21 Amber Herold
<Directory /usr/share/phpMyAdmin/>
1516 11 Amber Herold
order deny,allow
1517
deny from all
1518
allow from 127.0.0.1
1519
allow from YOUR_IP_ADDRESS
1520 21 Amber Herold
</Directory>
1521 11 Amber Herold
</pre>
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535 16 Amber Herold
1536 11 Amber Herold
h2.  Windows installation
1537
1538
1539
1540 16 Amber Herold
1541 11 Amber Herold
h2.  Processing-side Leginon Windows Installation
1542
1543
1544
1545 16 Amber Herold
1546 11 Amber Herold
h2.  Install Python and Support Packages (Note that python 2.5 must be used):
1547
1548
1549
1550
This list does not include pyton XML module because it is included in the python
1551
package for window.
1552
1553 1 Eric Hou
1554 11 Amber Herold
1555 14 Amber Herold
|_.Name:|_.Download site:|
1556 11 Amber Herold
|Python 2.5*|"http://www.python.org":http://www.python.org |
1557 15 Amber Herold
|Python for Windows extension (pywin32) |"http://sourceforge.net/projects/pywin32/":http://sourceforge.net/projects/pywin32/ |
1558
|wxPython 2.5.2.8 or newer|"http://www.wxpython.org":http://www.wxpython.org |
1559
|MySQL Python client 1.2 or newer|"http://sourceforge.net/projects/mysql-python":http://sourceforge.net/projects/mysql-python |
1560
|Python Imaging Library (PIL) 1.1.4 or newer|"http://www.pythonware.com/products/pil/":http://www.pythonware.com/products/pil/ |
1561
|NumPy 1.0b5 (tested, others may work)|"http://www.scipy.org":http://www.scipy.org |
1562
|SciPy 0.5.1 or newer|"http://www.scipy.org":http://www.scipy.org |
1563
|Tortoise SVN client|"http://tortoisesvn.tigris.org":http://tortoisesvn.tigris.org |
1564
1565 11 Amber Herold
1566
1567 1 Eric Hou
1568 11 Amber Herold
*Python 2.5 is the only python version that we have compiled numExtension. libCV and
1569
comarray in. Therefore no other python version works for now.
1570
1571
1572
Execute the installer file and follow the directions.
1573
1574
1575
1576 1 Eric Hou
1577 16 Amber Herold
1578 11 Amber Herold
h2. Packages required from NRAMM
1579
1580
1581
1582
Here are the packages you need to install with python installer
1583
1584 1 Eric Hou
1585 11 Amber Herold
1586 14 Amber Herold
|_.Name:|_.Purpose:|
1587 11 Amber Herold
| Leginon|modular TEM image acquisition|
1588
| pyami|general functions|
1589
| sinedon|Leginon/database interaction|
1590
| pyScope|microscope control and monitoring|
1591
| ImageViewer|image viewing for tomography|
1592
1593
1594
1595
1596
Because numextension and libCV requires extra compilers, we have created window
1597 1 Eric Hou
installer for them for python 2.5 and made them available through "http://www.leginon.org/":http://www.leginon.org/
1598
1599 14 Amber Herold
|_.Downloadfile Name|_.Installed Python Package File|_.Purpose:|
1600 13 Amber Herold
|NumExtension-1.2.0.win32-py2.5.exe|numextension.pyd|c extension for numerical processing|
1601 11 Amber Herold
|libCV-0.2.win32-py2.5.exe|libCV.pyd|small c library of algorithm from computer vision field|
1602
1603
1604 1 Eric Hou
1605 11 Amber Herold
1606
1607
1608 16 Amber Herold
1609 11 Amber Herold
h2. Check out SVN Source Files from the depository
1610
1611
1612
1613
Use your mouse to do the following
1614
* Create Leginon-1.6-ALL directory somewhere at your convenience
1615
1616
1617
* Change directory into Leginon-1.6-ALL
1618
1619 1 Eric Hou
1620 11 Amber Herold
* Right-click the mouse botton in this directory window and select Tortoise svn
1621 17 Amber Herold
Checkout in the menu:!http://emg.nysbc.org/software/leginon/images/images/svnmenu.png!
1622 1 Eric Hou
1623 11 Amber Herold
1624
* Set up svn checkout window like this for <link linkend="windows_packages">EACH
1625 17 Amber Herold
of the leginon packages</link>.!http://emg.nysbc.org/software/leginon/images/images/svnco.png!
1626 11 Amber Herold
1627
1628 1 Eric Hou
1629 11 Amber Herold
1630
1631
1632
1633 16 Amber Herold
1634 11 Amber Herold
h2. Install the packages you downloaded from NRAMM svn depository
1635
1636 1 Eric Hou
1637 11 Amber Herold
1638
1639
1640
* Start a command line Window from Start Menu
1641
1642
1643
* Install the package in each folder with commands such as
1644
<pre>cd Your_Download_Place\Leginon-1.6-ALL\leginon
1645
c:\\python25\python.exe setup.py install</pre>
1646
1647
1648
1649
1650
1651
1652
1653
1654 16 Amber Herold
1655 11 Amber Herold
h2. Download the two Window Installer Files from Leginon website
1656
1657
1658
1659
"http://www.leginon.org/":http://www.leginon.org/ h2. Install individual packages
1660 15 Amber Herold
1661 11 Amber Herold
1662
1663
Excute the installer files and follow the instruction.
1664
1665
1666
1667
1668 16 Amber Herold
1669 11 Amber Herold
h2.  Mapping Drives:
1670
1671
1672
1673
If you plan to run Leginon directly on the Windows machine, such as in <link
1674
linkend="config_C">configuration C</link>, and your data files are served through a
1675
Samba server on a Linux machine, you will need to map the network drive. For example, if
1676
your Samba server has a hostname your_smbserver, and you have set up a share called
1677
[your_share_point] which points to /your_data_path/ and leginon data will be saved under a
1678
folder in /your_data_path/leginon/
1679 1 Eric Hou
1680 11 Amber Herold
1681
1682
1683
*  Start, My Computer
1684 1 Eric Hou
1685 11 Amber Herold
1686
*  Tools menu, Map network drive
1687 1 Eric Hou
1688 11 Amber Herold
1689
*  Use an unmapped drive such as Z:
1690
1691 1 Eric Hou
Enter shared path in Windows format
1692 17 Amber Herold
as<pre>\\your_smbserver\your_share_point</pre>
1693 11 Amber Herold
1694
1695
*  Add the drive and the Linux path to leginon.cfg on the Windows machine
1696 17 Amber Herold
as<pre>[Drive Mapping]
1697 11 Amber Herold
Z:/your_data_path</pre>
1698
1699
1700
*  Add image path to leginon.cfg on the Windows machine in Linux format
1701 17 Amber Herold
as<pre>[Images]
1702 11 Amber Herold
path:/your_data_path/leginon</pre>
1703
1704
1705
1706
1707
1708
1709
1710
1711 16 Amber Herold
1712 11 Amber Herold
h2.  Configure leginon.cfg:
1713
1714
1715
Follow instruction in "<link linkend="leginon_cfg">Configure leginon.cfg</link>" in
1716
the section for Linux installation but note the location of the configuration files
1717
follows. In addition, if the storage disk is mapped onto the Windows PC as drive Z, this
1718
mapping should be included in leginon.cfg. See above.
1719
1720
1721
1722
1723
*  Configurations for all users
1724
1725
1726
1727 21 Amber Herold
<pre> <Python directory&gt;\Lib\site-packages\Leginon\config\leginon.cfg</pre>
1728 11 Amber Herold
1729
1730
Example:
1731
<pre> C:\Python25\Lib\site-packages\Leginon\config\leginon.cfg</pre>
1732
1733
1734
*  Configurations for individual users
1735
1736
1737
1738 21 Amber Herold
<pre> <Home directory&gt;\leginon.cfg</pre>
1739 11 Amber Herold
1740
1741
Example:
1742
<pre> C:\Documents and Settings\Leginon User\leginon.cfg </pre>
1743
1744
1745
*  A skeleton (default) configuration file is available:
1746
1747
1748
1749
<pre> C:\Python25\Lib\site-packages\Leginon\config\default.cfg</pre>
1750
1751
1752
1753
1754
1755
1756
1757
1758 1 Eric Hou
1759 16 Amber Herold
1760 11 Amber Herold
h2.  Configure sinedon.cfg:
1761
1762
1763
Sinedon is designed to be able to interact with multiple databases.
1764
1765
Follow instruction in "<link linkend="sinedon_cfg">Configure sinedon.cfg</link>" in
1766
the section for Linux installation but note the location of the configuration files
1767
follows.
1768
1769
1770
1771
1772
* For all users, put sinedon.cfg with the installed package
1773
as<pre>C:\Python25\Lib\site-packages\sinedon\sinedon.cfg</pre>
1774
1775
1776
*  Your home directory on
1777
Windows:<pre>C:\Documents and Settings\your_name&gt;</pre>
1778
1779
1780
* the skeleton sinedon configuration file
1781
is<pre>C:\Python25\Lib\site-packages\sinedon\examples\sinedon.cfg</pre>
1782
1783
1784
1785
1786
1787
1788
1789
1790 16 Amber Herold
1791 11 Amber Herold
h2.  Create Leginon and Leginon Client shortcut in Start menu menu under Leginon
1792
1793
1794 1 Eric Hou
1795 11 Amber Herold
This instruction refers to Windows XP
1796
1797
1798
1799
1800
* Go to C:\Documents and Settings\All Users\Start Menu\Programs\ and create a new
1801 1 Eric Hou
folder named Leginon.
1802 11 Amber Herold
1803
1804
* In another window, go
1805 17 Amber Herold
to<pre>C:\Python25\Lib\site-packages\Leginon</pre>
1806 11 Amber Herold
1807
1808 1 Eric Hou
* Create a shortcut from start-leginon.py as Leginon and a shortcut from
1809 11 Amber Herold
launcher.py as Leginon Client.
1810
1811
1812
* Move the two shortcuts into
1813 17 Amber Herold
<pre>C:\Documents and Settings\All Users\Start Menu\Programs\Leginon</pre>
1814 11 Amber Herold
1815
1816
1817
1818
1819
1820
1821
1822 16 Amber Herold
1823 11 Amber Herold
h2.  Additional Software (Optional):
1824 1 Eric Hou
1825 11 Amber Herold
1826
TightVNC (http://www.tightvnc.com)
1827
1828
1829
1830
1831
1832
1833
1834
1835 16 Amber Herold
1836 11 Amber Herold
h2. Database server Windows Installation
1837
1838
1839
1840 16 Amber Herold
1841 11 Amber Herold
h2. We do not do this at NRAMM. Please follow the instruction in Linux installation and
1842
modify it for Windows at your own risk.
1843
1844
1845
1846
For a good Windows specific instruction for general PHP configuration with MySQL for
1847
Apache 2 in Windows, try http://www.artfulsoftware.com/php_mysql_win.html.
1848
1849
1850
1851
1852
1853
1854
1855
1856 16 Amber Herold
1857 1 Eric Hou
h2.  Additional installation on the microscope computer:
1858 11 Amber Herold
1859
1860
1861
The full leginon and its supporting packages need to be installed on the Windows computer
1862
controlling the microscope. Additional programs are required for communication with the CCD
1863
and extra functions for Tecnai scripting not supported as default by FEI Tecnai
1864
microscopes
1865
1866
1867 16 Amber Herold
1868 11 Amber Herold
h2.  Required supporting programs for the CCD camera from camera makers
1869
1870
1871
1872
Install and register the following programs for CCD cameras from the two makes:
1873
1874 1 Eric Hou
1875
1876 14 Amber Herold
|_.Camera Make:|_.File:|
1877 11 Amber Herold
| Gatan|TecnaiCCD.dll|
1878
| Tietz|CAMC4.exe*|
1879
1880
1881
1882
1883
* We have experienced slowness of the CAMC4.exe comes with later version Tecnai TUI/TIA.
1884
Replacing it with an earlier version of CAMC4.exe resolved the problem.
1885
1886
1887
1888
1889 16 Amber Herold
1890 11 Amber Herold
h2. Additional Package required from NRAMM for Gatan camera or camera that uses TIA
1891
1892
1893
1894
For Gatan Camera or FEI Eagle Camera that uses TIA, comarray package need to be install
1895 1 Eric Hou
with python
1896 11 Amber Herold
1897 1 Eric Hou
1898
1899
|_.SVN Package Name|_.Installed Python Package Name|_.Reason for update:|
1900 11 Amber Herold
|comarray|comarray|com module output conversion to array|
1901
1902
1903
1904
1905
1906
1907
* <link linkend="window_svn">Check out the comarray package</link> from svn
1908
depository with your SVN client
1909
1910 17 Amber Herold
<pre>cd Leginon-1.6-ALL
1911 11 Amber Herold
svn co http://emg.nysbc.org/svn/comarray/branches/leg1.6 comarray</pre>
1912
1913
* Start a command line Window from Start Menu
1914 1 Eric Hou
1915 11 Amber Herold
1916
* Install the package with commands such as
1917
<pre>cd Your_Download_Place\Leginon-1.6-ALL\comarray
1918
c:\\python25\python.exe setup.py install</pre>
1919
1920
1921
1922
1923
1924
1925
1926
1927 16 Amber Herold
1928 11 Amber Herold
h2.  Supporting programs for film exposure
1929
1930
1931
1932
Install the following if you need film exposure on FEI Tecnai TEM through Leginon,
1933
available through FEI. Please contact Max Otten: <email>mto@feico.com</email> and request
1934 1 Eric Hou
for adaexp.exe that works with your version of Tecnai user interface program.
1935
1936 11 Amber Herold
1937 14 Amber Herold
1938 13 Amber Herold
|_.Name:|_.File:|
1939 11 Amber Herold
| exposure adaptor|adaexp.exe|
1940
1941
1942
1943
1944
1945
1946 16 Amber Herold
1947 11 Amber Herold
h2.  Register adaexep.exe
1948
1949
1950
1951
*  From the command prompt:
1952
1953
<pre>adaexp.exe /regserver</pre>
1954
1955
*  From the program start menu under pyScope or
1956
C:\Python25\Lib\site-packages\pyScope\
1957
1958
<pre>updatecom.py</pre>
1959
1960
1961
1962
1963
1964
1965
1966 16 Amber Herold
1967 11 Amber Herold
h2. Modify instruments.cfg:
1968
1969 1 Eric Hou
1970 11 Amber Herold
1971
Sinedon now have full control of database interaction, therefore, the database
1972
configuration in leginon.cfg is no longer needed.
1973
1974
1975
1976
1977
* Modify the file instruments.cfg in the installed pyScope directory to that for
1978
your microscope and camera. For example, if your microscope uses Tecnai Scripting
1979
Interface and you have a Gatan camera that you interface through
1980
DigitalMicrograph:
1981
1982
1983
1984
<pre>[tem]
1985
class: tecnai.Tecnai
1986
[camera]
1987
class: gatan.Gatan</pre>
1988
1989 1 Eric Hou
1990 11 Amber Herold
1991
The file contains other examples of microscope and camera drivers that we
1992
distribute from NRAMM.
1993
1994
1995
1996
1997
1998
1999
2000
2001 16 Amber Herold
2002 11 Amber Herold
h2. Run updatecom.py
2003
2004
2005
2006
From a command line window:
2007
2008
2009
2010
<pre>cd C:\python25\Lib\Site-Packages\pyScope
2011
C:\python25\python.exe updatecom.py</pre>
2012
2013
2014
2015
This should generate a few files, including tecnaicom.py, gatancom.py and tietzcom.py,
2016
in the same directory.
2017
2018
2019
2020
2021 16 Amber Herold
2022 11 Amber Herold
h2. Additional setup on Tietz PXL camera (optional)
2023
2024
2025
2026
The actual Tietz PXL camera dimension is slightly larger than 2048 x 2048. If you'd like
2027
to take the images using the maximum of 2048 x 2048 pixels, you may modify the function that
2028
get camera dimension in tietz.py of pyScope package.
2029
2030 1 Eric Hou
2031 11 Amber Herold
*  Go to C:\Python25\Lib\site-packages\pyScope\
2032
2033
2034
*  Edit tietz.py with a plain text editor
2035
2036
2037
*  Find the lines:
2038
2039
2040
2041
<pre> def getCameraSize(self):
2042
# {'type': dict, 'values': {'x': {'type': int}, 'y': {'type': int}}}}
2043
x = self._getParameterValue('cpTotalDimensionX')
2044 1 Eric Hou
y = self._getParameterValue('cpTotalDimensionY')
2045 11 Amber Herold
return {'x': x, 'y': y}
2046
</pre>
2047
2048
2049 1 Eric Hou
2050 11 Amber Herold
*  Change the last line to:
2051
2052
2053
2054
<pre>    return {'x': 2048, 'y': 2048}</pre>
2055
2056
2057
2058
2059 1 Eric Hou
2060 11 Amber Herold
2061
2062 16 Amber Herold
2063 11 Amber Herold
h2.  Additional Software (Optional):
2064
2065
2066
TightVNC (http://www.tightvnc.com) if you get tired of going into the microscope room
2067
just to open the column valves.
2068 1 Eric Hou
2069 11 Amber Herold
2070
2071
2072
2073
2074
2075
2076 16 Amber Herold
2077 11 Amber Herold
h2.  What is next
2078
2079 1 Eric Hou
2080 11 Amber Herold
2081 16 Amber Herold
2082 11 Amber Herold
h2.  Perform Administration Setup
2083
2084 1 Eric Hou
2085 11 Amber Herold
See Next chapter on <link linkend="addusers">Leginon Administration Tools</link>.
2086
2087
2088
2089 1 Eric Hou
2090 11 Amber Herold
2091 16 Amber Herold
2092 11 Amber Herold
h2.  Create a test project
2093
2094
2095
See the chapter on <link linkend="project">Project Management Tools</link>.
2096
2097
2098
2099 1 Eric Hou
2100 16 Amber Herold
2101 11 Amber Herold
h2.  Perform Microscope Setup and Test run
2102
2103
2104
See the chapter on <link linkend="runleg_chapter">Start Leginon</link>.
2105
2106
2107
2108
2109
2110 1 Eric Hou
2111 16 Amber Herold
2112 11 Amber Herold
h2.  Backup Practices
2113
2114
2115
2116 16 Amber Herold
2117 11 Amber Herold
h2.  Ideal setup:
2118
2119
2120
2121 16 Amber Herold
2122 11 Amber Herold
h2.  Regular computer backup
2123 1 Eric Hou
2124 11 Amber Herold
2125
2126
To protect your program installation.
2127
2128
2129
2130
2131 16 Amber Herold
2132 11 Amber Herold
h2.  Full backup of the databases every night
2133 1 Eric Hou
2134 11 Amber Herold
2135
2136
The database is the metadata generated by Leginon named "dbemdata" in the installation
2137
example.
2138
2139
2140
2141
2142 16 Amber Herold
2143 11 Amber Herold
h2. Full/Incremental/differential backup of the images
2144
2145
2146 1 Eric Hou
2147 11 Amber Herold
The image files located in [your storage disk] should be backup in full at least once
2148
a month, and nightly in differential or incremental mode.
2149
2150
2151
2152
2153 1 Eric Hou
2154 11 Amber Herold
2155 16 Amber Herold
2156 11 Amber Herold
h2.  Current NRAMM setup:
2157
2158
2159
2160
Because our tape backup system is not on-site and the data have to transfer over slow
2161
network to perform the task, we have not been able to backup in the ideal setting. Here is
2162
what we currently do:
2163
2164
2165 16 Amber Herold
2166 11 Amber Herold
h2.  Regular computer backup
2167
2168
2169
2170
TSRI Research Computing provides regular back up service that does full backup of the
2171
computers every four weeks and incremental backup every night. The full backup expires in
2172
4 months and the incremental backup one month. We do this for the computers running
2173
Leginon main program, the database server, as well as the web server.
2174
2175
2176
2177
2178 16 Amber Herold
2179 11 Amber Herold
h2.  Regular backup &amp; tar of the databases every night
2180
2181
2182
2183
On top of the TSRI backup, we also tar the database everynight and store it on the
2184
institutional tape library.
2185 16 Amber Herold
2186 11 Amber Herold
2187
2188
2189
2190
h2.  rsync to a tape library of the image storage disk
2191
2192
2193
2194
As our image storage disks total 15T byte, we currently use rsync function to update a
2195
copy on our institutional tape library. It creates new files, updates old files, but does
2196
not remove old files on the tape if it is removed on the storage disk. We do not think
2197
this is the best solution since it is not possible to retrieve old files that have been
2198
modified on a later time.