Project

General

Profile

Appion Home » History » Version 29

Amber Herold, 04/01/2010 11:06 AM

1 1 Eric Hou
2 11 Amber Herold
h1. Complete Installation
3 6 Amber Herold
4
5 1 Eric Hou
6 16 Amber Herold
7 11 Amber Herold
h2. Four Parts of Leginon System
8 1 Eric Hou
9 11 Amber Herold
10 16 Amber Herold
!http://emg.nysbc.org/software/leginon/images/images/Leg_Proj_parts.png!
11 1 Eric Hou
12 11 Amber Herold
Leginon as a system that we distribute can be divided into three parts:
13 1 Eric Hou
14 6 Amber Herold
15 16 Amber Herold
16 11 Amber Herold
h2. Processing Server (Python-side)
17 1 Eric Hou
18
19 11 Amber Herold
Python (and some c) scripts that handle instrument control, data acquisition, and
20
processing.
21 1 Eric Hou
22
23
24
25 16 Amber Herold
26 11 Amber Herold
h2. Database Server (MySQL-side)
27 1 Eric Hou
28 11 Amber Herold
29
a MySQL server that handles the database
30
31
32 1 Eric Hou
33 11 Amber Herold
34 16 Amber Herold
35 11 Amber Herold
h2. Web Server (PHP-side)
36
37
38
This includes php and Java scripts at a webserver that we will create to retrieve image
39
and metadata from the database and file-storage system.
40
41
42 1 Eric Hou
43 11 Amber Herold
44 16 Amber Herold
45 11 Amber Herold
h2. File Server
46
47
48
This is up to you to set up to store lots of data coming out of Leginon system.
49
50
51
52
53 1 Eric Hou
54 11 Amber Herold
55 16 Amber Herold
56 11 Amber Herold
h2. What is in this chapter
57
58
59
60
Leginon runs under both the Linux and Microsoft Windows Operating systems. However, the
61
current php-mrctool can not be installed on Windows which means mrc images can not be viewed
62
on the web. See the section on Possible Computer Set-up Configurations for details.
63
64
65
The many components for the Leginon system are divided into packages and need to be
66
installed separately. It has been integrated with several third party software packages that
67
are necessary for its operation. The installation documentation will describe how to set-up
68
Leginon and give hints as to how to install and set-up the necessary third-party software.
69
This documentation is not intended to support the additional, but necessary third-party
70
software.
71
72
73
There has been interest in installing Leginon under Mac OS X. While this is possible in
74
theory, we have not been successful in installing a fully functional system. The two main
75
problems are (1) compilation of php-mrctool from the pre-installed php and (2) wxPython on Mac
76
is unable to hide bitmap objects which makes Leginon graphical user interface difficult to
77
use.
78
79
80 29 Amber Herold
See <link linkend="InstT_install">Installation Troubleshooting</link> and "Leginon Bulletin Board":http://emg.nysbc.org/bb/viewforum.php?f=2 searching
81 11 Amber Herold
for "install" if you run into problems.
82
83
84 1 Eric Hou
85 11 Amber Herold
86 16 Amber Herold
87 11 Amber Herold
h2.  Possible Computer Set-up Configurations
88
89
90
91
Computer(s) used for the whole Leginon system need to support five functions:
92
*  TEM/Camera control and data output (by python-side packages of Leginon
93
system)
94
95
96
*The computer attached to the microscope/camera is used for this function.
97
Therefore, it is on a Windows computer.
98
99
100
*  General Leginon operation such as target selection and image processing (processing
101
server of Leginon system)
102
103
104
*  Image data storage.
105
106
107
*  Other meta-data storage through MySQL database (by database server of Leginon
108
system)
109
110
111
*  Web server (using PHP-side of Leginon system).
112
113
114
*Important* The MRC module that is used to load mrc formatted files to the web
115
pages is not compiled on Windows. Therefore, this function must be on a linux box unless
116
you don't need to see the images***
117
118
119
120
121
122
The five functions can be distributed to five different computers on a network or all on
123
one single computer. However, since the latter minimal setup requires the use of the computer
124
attached to the microscope for all functions, it is not advisable.
125
126
127
Processing server of Leginion is a multi-platform software, meaning it can run on both
128
Microsoft Windows and Linux. It is also inherently designed to utilize distributed systems on
129
a network. MySQL used in the database server is also supported by multiple platforms. This is
130
not true for the Web server because the custom MRC module used in the web viewer only compiles
131
on linux machines. Therefore, one of the computer not part of the microscope must be a linux.
132
The following examples show several arrangements that take advantage of distributed system.
133
Other combinations are possible, but will basically be variations on one of the following
134
themes.
135 1 Eric Hou
136 11 Amber Herold
137 16 Amber Herold
138 11 Amber Herold
h2.  A: 1 Windows computer (attached to the Microscope) &amp; 1 Linux computer (for all
139
other functions, separate from the Microscope) (Good for single user and small-scale
140
acquisition)
141
142
143
144
145
146
*  Install the latest processing server side of Leginon release (and supporting
147
packages) on the Windows computer controlling the microscope and on the Linux computer
148
which is to run Leginon image acquisition and processing.
149
150
151
*  Install the additional modules needed for CCD camera and film interaction on the
152
Windows computer controlling the microscope.
153
154
155
*  Install MySQL, PHP, the Apache Web Server, and dbem tools only on the Linux
156
computer that will perform these functions, (i.e. NOT the computer controlling the
157
microscope).
158
159
160
161
162
163
164 1 Eric Hou
165 11 Amber Herold
166 16 Amber Herold
167 11 Amber Herold
h2.  B: 1 Windows computer (attached to the Microscope) &amp; 4 Linux computers (one for
168
each function) (Good for multiple microsopes, users and large-scale acquisition)
169
170
171
172
This is the set-up at NRAMM. It has the processing server side of Leginon installed on
173
the Windows computer attached to the microscope and on the Linux computer that is used for
174
running Leginon. A second Linux machine is dedicated to the database while the web-based
175
viewer is hosted by another server.
176
177
178
179
180
*  Install the python-side Leginon (and supporting packages) on the Windows computer
181
controlling the microscope and on the Linux computer that is to run Leginon. You can
182
make the latter installation accessable to any numbers of linux computers on your
183
network.
184
185
186
*  Install MySQL, PHP, the Apache Web Server, and dbem tools on the second Linux
187
computer that will serve as the database/web server/PHP host.
188
189
190
*  The computer hosting the data storage should be accessable by all linux
191
boxes.
192
193
194
195
196
197
198 1 Eric Hou
199 11 Amber Herold
200 16 Amber Herold
201 11 Amber Herold
h2.  C: 1 Windows computer (attached to the Microscope), 1 Windows computer running python-
202
and MySQL-side functions and 1 Linux computer as web server (Has been applied
203
successfully)
204
205
206
207
The data storage may be on one of these computers that is accessable by other
208
computers.
209
210
211
212
213 1 Eric Hou
214 11 Amber Herold
215 16 Amber Herold
216 11 Amber Herold
h2.  Ports that Leginon system uses:
217
218
219
220
This information is useful if you want to open specific ports between computers with a
221
firewall in between.
222
223
224
225
226
* MySQL normally uses port 3306 (but configurable in /etc/my.cnf)
227
228
229
* Leginon Client running on TEM host: port 55555
230
231
232
* Leginon main program on the processing server: dynamically assigned ports between
233
49152 and 65535
234
235
236
There is no strict port assignment since we could potentially have more than one
237
Leginon process running on the same linux host talking to different TEM hosts. It is
238
probably good enough if you only worry about opening up the first few of those ports in
239
your firewall (maybe 49152 through 49160, or something like that).
240 1 Eric Hou
241 27 Amber Herold
242 29 Amber Herold
See more discussion at Leginon bulletin board thread on ""network problem, Leginon not seeing
243
tecnai host":http://emg.nysbc.org/bb/viewtopic.php?t=7 "
244 11 Amber Herold
245
246
247
248
249
250 1 Eric Hou
251 11 Amber Herold
252 15 Amber Herold
253 1 Eric Hou
h2. Where to Register and download Leginon system
254 11 Amber Herold
255
256
257 28 Amber Herold
"http://www.leginon.org/":http://www.leginon.org is the home. Please
258 11 Amber Herold
register if you want to recieve support from Leginon team.
259 1 Eric Hou
260 11 Amber Herold
261 1 Eric Hou
*  python-side Leginon packages
262
263 23 Amber Herold
264 29 Amber Herold
direct link: "http://emg.nysbc.org/software/leginon/":http://emg.nysbc.org/software/leginon/ *  mrctools: including the tools for reading mrc images on the web
265 1 Eric Hou
266 11 Amber Herold
267 29 Amber Herold
direct link: "http://emg.nysbc.org/software/mrctools/":http://emg.nysbc.org/software/mrctools/ *  dbemtools: including the tools for administration and webviewing
268 11 Amber Herold
269 23 Amber Herold
270 29 Amber Herold
direct link: "http://emg.nysbc.org/software/dbemtools/":http://emg.nysbc.org/software/dbemtools/ *  project tools (optional): For administration and webviewing of projects
271 25 Amber Herold
272
273 29 Amber Herold
direct link: "http://emg.nysbc.org/software/dbemtools/":http://emg.nysbc.org/software/dbemtools/ h2.  Processing server-side Leginon Installation
274 25 Amber Herold
275
276
277 11 Amber Herold
For the lack of better name, processing server-side Leginon refers to packages that
278 1 Eric Hou
performs the general functions of image acquisition and processing. The instructions here are
279 11 Amber Herold
for Linux. Refer to Windows Installation for Windows specific instructions.
280
281 1 Eric Hou
282
283 11 Amber Herold
h2. Download svn checkout script from Leginon website
284
285
286
287 28 Amber Herold
"http://www.leginon.org/":http://www.leginon.org/ Download
288 15 Amber Herold
svnget-1.6.sh to a convenient location such as your home directory.
289 11 Amber Herold
290
291
292
293 16 Amber Herold
294 11 Amber Herold
h2. Check out SVN Source Files from the depository
295
296
297
298
run svnget-1.6.sh. It does the
299 17 Amber Herold
following:<pre>mkdir Leginon-1.6-ALL
300 11 Amber Herold
cd Leginon-1.6-ALL
301
svn co http://emg.nysbc.org/svn/leginon/branches/1.6 Leginon
302 1 Eric Hou
svn co http://emg.nysbc.org/svn/pyami/branches/leg1.6 pyami
303 11 Amber Herold
svn co http://emg.nysbc.org/svn/pyScope/branches/leg1.6 pyScope
304
svn co http://emg.nysbc.org/svn/sinedon/branches/leg1.6 sinedon
305
svn co http://emg.nysbc.org/svn/numextension/branches/leg1.6 numextension
306
svn co http://emg.nysbc.org/svn/libcv/branches/leg1.6 libcv
307
svn co http://emg.nysbc.org/svn/ImageViewer/branches/leg1.6 ImageViewer</pre>
308
309
310
If the process fails, you probably do not have svn client installed. Skip to "Install
311
the supporting packages first if missing:", then try this again.
312
313
314
315
316 16 Amber Herold
317 11 Amber Herold
h2.  Perform system check:
318
319
320
321
In addition to the downloads from our svn depository, there are several other
322
requirements that you will get either from your OS installation source, or from its
323
respective website. The system check in the Leginon package checks your system to see if you
324
already have these requirements
325
326 1 Eric Hou
327 11 Amber Herold
328
<pre>cd your_download_area/Leginon-1.6-ALL/Leginon
329
python syscheck.py</pre>
330
331
332
333
If python is not installed, this, of course will not run. If you see any lines like "***
334
Failed...", then you have something missing. Otherwise, everything should result in
335
"OK".
336
337
338 1 Eric Hou
339
340
341
h2.  Install the supporting packages first if missing:
342
343
344
345
Follow the instruction for your specific Linux distribution.
346 24 Amber Herold
347
348
For example, SUSE users can use YaST to install them; RedHat and CentOS users can use
349
yum.
350
351
352
*SciPy may not build properly on some versions of SuSE due to an incompatible LAPACK
353
package that comes with SuSE. You can get scipy as well as a compatible LAPACK etc. from
354
http://repos.opensuse.org/science (need to specify your SuSE version and machine
355
etc.)
356
357
358 1 Eric Hou
359
360
361
h2.  Required supporting packages:
362
363
364 25 Amber Herold
365
366 26 Amber Herold
|_.Name:|_.Download site:|_.yum package name|_.SuSE rpm name|
367 25 Amber Herold
|Python 2.4 or newer|"http://www.python.org":http://www.python.org|python|python-devel|
368
|wxPython 2.5.2.8 or newer|"http://www.wxpython.org":http://www.wxpython.org|wxPython|python-wxGTK|
369 29 Amber Herold
|MySQL Python client 1.2 or newer|"http://sourceforge.net/projects/mysql-python":http://sourceforge.net/projects/mysql-python|MySQL-python|python-mysql|
370
|Python Imaging Library (PIL) 1.1.4 or newer|"http://www.pythonware.com/products/pil/":http://www.pythonware.com/products/pil/|python-imaging|python-imaging|
371 1 Eric Hou
|Python XML module 0.8.3 or newer|"http://pyxml.sourceforge.net":http://pyxml.sourceforge.net|PyXML|python-xml|
372 26 Amber Herold
|NumPy 1.0.1 or newer|"http://www.scipy.org":http://www.scipy.org|numpy|numpy|
373 29 Amber Herold
|SciPy 0.5.1 (tested, others may work)*|"http://www.scipy.org":http://www.scipy.org , "http://repos.opensuse.org/science":http://repos.opensuse.org/science *|scipy|python-scipy|
374 26 Amber Herold
375 1 Eric Hou
376
377 26 Amber Herold
378
*SciPy may not build properly on some versions of SuSE due to an incompatible LAPACK
379
package that comes with SuSE. You can get scipy as well as a compatible LAPACK etc. from
380
http://repos.opensuse.org/science (need to specify your SuSE version and machine
381
etc.)
382
383
384
385
386
387
h2.  Optional supporting packages:
388
389
390
391
392
393
*  For using Matlab to explore holefinding algorithm (Matlab HoleFinder)
394
395
396 29 Amber Herold
pymat: "http://sourceforge.net/projects/pymat":http://sourceforge.net/projects/pymat h2. Install the packages you downloaded from svn depository
397 26 Amber Herold
398
399
400
Here are the packages you need to install with python installer inside the
401
Leginon-1.6-ALL folder.
402
403
404
405 11 Amber Herold
|_.Name:|_.Purpose:|
406 26 Amber Herold
| Leginon|modular TEM image acquisition|
407 11 Amber Herold
| pyami|general functions|
408
| pyScope|microscope control and monitoring|
409
| sinedon|Leginon/database interaction|
410
| numExtension|c extension for numerical processing|
411
| libCV|small c library of algorithm from computer vision field|
412
| ImageViewer|image viewing for tomography|
413
414
415
416
417
418
419
*  run syscheck.py again to make sure you have everything.
420
421 1 Eric Hou
422 19 Amber Herold
*  Install the package in _each folder_ with commands
423 11 Amber Herold
like
424
these<pre>cd /your_download_area/Leginon-1.6-ALL/Leginon
425 1 Eric Hou
python setup.py install
426 11 Amber Herold
427
cd ../pyami
428
python setup.py install
429
cd ../pyScope
430
python setup.py install
431
cd ../sinedon
432
python setup.py install
433
cd ../numextension
434
python setup.py install
435
cd ../libcv
436
python setup.py install
437
cd ../ImageViewer
438
python setup.py install</pre>
439
440
441
442
443
444
445
446
447 16 Amber Herold
448 11 Amber Herold
h2.  python-site-package-path: where the installed python
449
packages went:
450
451
452
Python installer put the packages you installed to its site-packages directory. This
453
enable all users on the same computer to access them. The easiest way to find where your
454
installed package is called by python is to load a module from the package using interactive
455
python command lines like this:
456
457
458
459
460
*  Start python command line from shell
461
462
463
464
<pre>> python</pre>
465
466
467
468
*  Import a module from the package. Let's try sinedon here. All packages installed
469
through the above setup.py script should go to the same place.
470
471
472
473
<pre>python> import sinedon</pre>
474
475
476
477 1 Eric Hou
*  If the module is loaded successfully, call the module attribute __path__ (two
478 11 Amber Herold
underscrolls before "path" and two underscrolls after) will return the location of the
479
module it is loaded from
480
481
482
483
<pre>python> sinedon.__path__
484
485
RETURNS, For example,
486 21 Amber Herold
python> <module 'sinedon' from '/usr/lib/python2.5/site-packages/sinedon/__init__.pyc'> </pre>
487 11 Amber Herold
488
489
In this case, /usr/lib/python2.5/site-packages/ is your python-site-package-path.
490
If you go to that directory, you will find all the packages you just installed.
491
492
493
494
495
496
497
498
499 16 Amber Herold
500 11 Amber Herold
h2.  Configure leginon.cfg:
501
502
503
Leginon can be configured at three levels:
504
505
506
507
508
*  A skeleton (default) configuration file is available:
509
510 1 Eric Hou
511 11 Amber Herold
512 19 Amber Herold
<pre> _<link linkend="package_path">[python-site-package-path]</link>_/Leginon/config/default.cfg</pre>
513 11 Amber Herold
514
515
516
*  Configurations for all users
517 1 Eric Hou
518 11 Amber Herold
519
520 19 Amber Herold
<pre> _<link linkend="package_path">[python-site-package-path]</link>_/Leginon/config/leginon.cfg</pre>
521 11 Amber Herold
522
523
524 1 Eric Hou
Example of the configuration that you should set: (Here we assume your leginon
525 11 Amber Herold
database is named "dbemdata", and Leginon log into the database as a user named
526
"usr_object" with no password
527
set).<pre>[Images]
528 19 Amber Herold
path: _your_storage_disk_path_/leginon
529 11 Amber Herold
</pre>
530
531 1 Eric Hou
532 11 Amber Herold
*  Configurations for individual users that overrides the all-user configuration if
533
exists
534
535
536
537
<pre> ~/leginon.cfg</pre>
538
539 1 Eric Hou
540 11 Amber Herold
541
This configuration should include all the minimal configuration above and
542
additional configuration for individual user for
543
convenience<pre>[User]
544 19 Amber Herold
Fullname: _your_name_  #The name you entered in the leginon administration web page. Note that it is case sensitive</pre>
545 11 Amber Herold
546
547
548
549
550
551
552
553 16 Amber Herold
554 11 Amber Herold
h2.  Names used in the example installation
555
556
557
From this point on, you will need to enter database user names for different
558
configuration files and settings. These are NOT Leginon user names but what is required by
559
MySQL database interaction. Consistent names are essential. In the example shown here, the
560
leginon database is called "dbemdata", the project database is called "projectdata". The
561
MySQL user name are the same, "usr_object", and in this case without a password.
562
563
564
565
566 16 Amber Herold
567 11 Amber Herold
h2.  Configure sinedon.cfg:
568
569
570
Sinedon is designed to be able to interact with multiple databases. It can be
571
configured at global or user level.
572
573
574
575
576
*  An example configuration file is available at:
577
578
579
580
<pre>/your_download_area/Leginon-1.6-ALL/sinedon/examples/sinedon.cfg</pre>
581
582 1 Eric Hou
583 11 Amber Herold
584
*  Configurations for all users should be placed at the following path
585
586
587
588 19 Amber Herold
<pre>  _<link linkend="package_path">[python-site-package-path]</link>_/sinedon/sinedon.cfg</pre>
589 11 Amber Herold
590
591
592 1 Eric Hou
*  Modify host,db,user,passwd to what is used. Note that the user here is MySQL user
593 11 Amber Herold
Leginon uses to communicate with the database for all Leginon users. For Leginon that
594
uses Project database, set the following:
595
596
597 1 Eric Hou
598
<pre>[global]
599
host: _your_database_host_
600
user: usr_object
601
passwd:
602 25 Amber Herold
603
[projectdata]
604 1 Eric Hou
db: projectdata
605
606
[leginondata]
607
db: dbemdata
608 25 Amber Herold
</pre>
609
610
611
612 23 Amber Herold
*  Add database configuration if you intend to use grid-inserting robot. The Robot2
613 25 Amber Herold
module uses the database to communicate to the robot. Applications that carries the
614
name "Robot" requires this to be set. In general, using the same database as the
615
general leginon database is fine.
616
617
618
619
<pre>[robot2]
620
db: dbemdata</pre>
621
622
623
624
625
626
627
628
629
630
631
632
633
h2. Database Server-side Installation
634
635
636
637
The following is for the computer that hosts the databases. This involves installing MySQL
638
server and creation/configuration of the leginon and project databases.
639
640
641
642
643
644
|_.Name:|_.Download site:|
645 26 Amber Herold
|MySQL-Server 5.0 or higher|"http://www.mysql.com":http://www.mysql.com|
646 25 Amber Herold
|MySQL-Client 5.0 or higher|"http://www.mysql.com":http://www.mysql.com|
647
648
649
650
651
652 11 Amber Herold
h2.  MySQL
653
654
655
656
657
658
*  Install MySQL-Server
659
660
661
662
663 1 Eric Hou
*  Use your package installer (yum, zypper, YaST) if available.
664 11 Amber Herold
665 1 Eric Hou
666
OR
667
668
669 28 Amber Herold
*  Download the latest MySQL-server RPM for Linux from "www.mysql.com":http://www.mysql.com *  Install the MySQL-server rpm:
670 27 Amber Herold
671
672 11 Amber Herold
<pre> rpm -Uvh MySQL-server-5.0.xx-y.i386.rpm</pre>
673
(substitute correct version numbers)
674
675
676
677
678
679
680
*  Install MySQL-Client
681
682
683
684 1 Eric Hou
685
*  Use your package installer (yum, zypper, YaST) if available.
686 11 Amber Herold
687
688 1 Eric Hou
OR
689
690 11 Amber Herold
691 28 Amber Herold
*  Download the latest MySQL-client RPM for Linux from "www.mysql.com":http://www.mysql.com *  Install the MySQL-client rpm:
692 27 Amber Herold
693
694 11 Amber Herold
<pre> rpm -Uvh MySQL-client-5.0.xx-y.i386.rpm</pre>
695
(substitute correct version numbers)
696
697
698
699
700
701
702
*  MySQL configuration file is usually located in /usr/share/mysql. There are
703
several examples there:
704
705
706
707 22 Amber Herold
<pre>> ls /usr/share/mysql/my*
708 11 Amber Herold
/usr/share/mysql/my-huge.cnf
709
/usr/share/mysql/my-innodb-heavy-4G.cnf
710
/usr/share/mysql/my-large.cnf
711
/usr/share/mysql/my-medium.cnf
712
/usr/share/mysql/my-small.cnf
713 22 Amber Herold
> </pre>
714 11 Amber Herold
715
716
717
*  Configure my.cnf in /etc using my-huge.cnf as the template
718
719
720
721
722
* <pre>> cp /usr/share/mysql/my-huge.cnf /etc/my.cnf</pre>
723
724
725
726
*  Edit /etc/my.cnf to add or change query cache variables like these:
727
728
<pre>query_cache_type= 1
729
query_cache_size = 100M
730
query_cache_limit= 100M</pre>
731
732
733
734
735
736
737
738
*  start MySQL Server
739
740 22 Amber Herold
<pre> >/etc/init.d/mysqld start</pre>
741 11 Amber Herold
742
on some installation,
743
744 22 Amber Herold
<pre> >/etc/init.d/mysql start</pre>
745 11 Amber Herold
746
For future reference: start | stop | restart MySQL Server with similar
747
commands:
748
749 22 Amber Herold
<pre> >/etc/init.d/mysqld start
750
>/etc/init.d/mysqld stop
751
>/etc/init.d/mysqld restart</pre>
752 11 Amber Herold
753
If you want to start MySQL automatically at boot on SuSE
754
755 22 Amber Herold
<pre> SuSE >chkconfig mysql on
756 11 Amber Herold
</pre>
757
758
*  For future reference, the database location will be:
759
760
761
762 22 Amber Herold
<pre>> cd /var/lib/mysql
763 11 Amber Herold
Directory: /var/lib/mysql
764 22 Amber Herold
> ls
765 11 Amber Herold
yourdbserver.pid
766
ib_logfile0
767
mysql
768
mysql.sock
769
test
770 22 Amber Herold
> </pre>
771 11 Amber Herold
772
773
774
*  Create leginon database, here we call <link linkend="db_example_names"
775
>dbemdata</link>
776
777 22 Amber Herold
<pre> >mysqladmin create dbemdata</pre>
778 11 Amber Herold
779
780
781
*  Create project database, here we call <link linkend="db_example_names"
782
>projectdata</link> (optional)
783
784 22 Amber Herold
<pre> >mysqladmin create projectdata</pre>
785 11 Amber Herold
786
787
788
*  Connect to mysql db
789
790 22 Amber Herold
<pre>>mysql mysql
791 11 Amber Herold
792 22 Amber Herold
mysql> select user, password, host from user;
793 11 Amber Herold
+------+----------+-----------+
794
| user | password | host      |
795
+------+----------+-----------+
796
| root |          | localhost |
797
| root |          | host1     |
798
|      |          | host1     |
799
|      |          | localhost |
800
+------+----------+-----------+
801
4 rows in set (0.00 sec)</pre>
802
803
804
805
*  Create and grant privileges to a user called <link linkend="db_example_names"
806
>usr_object</link> for the databases on both the localhost and other hosts involved.
807
For example, use wild card '%' for all hosts. You may also set specific privilege to
808
the user. See MySQL Reference Manual for details
809
810 22 Amber Herold
<pre>mysql> create user usr_object@'localhost';
811
mysql> grant all privileges on dbemdata.* to usr_object@'localhost';
812
mysql> grant all privileges on projectdata.* to usr_object@'localhost';</pre>
813 11 Amber Herold
814
Similarly,
815 1 Eric Hou
816 22 Amber Herold
<pre>mysql> create usr_object@'%';
817
mysql> grant all privileges on dbemdata.* to usr_object@'%';
818
mysql> grant all privileges on projectdata.* to usr_object@'%';
819 11 Amber Herold
</pre>
820
821
*  Change Root passworld
822
823 22 Amber Herold
<pre>mysql> update user set password=password('_a passwd_') where user="root";
824 1 Eric Hou
Query OK, 2 rows affected (0.01 sec)
825 11 Amber Herold
Rows matched: 2  Changed: 2  Warnings: 0
826 1 Eric Hou
827 22 Amber Herold
mysql>^D or exit;</pre>
828 11 Amber Herold
From now on, you will need to specify the password to connect to the database as
829
root user like this:
830
831
<pre>>mysql -u root -p mysql</pre>
832
833 19 Amber Herold
_***If your password change does not take effect in your next
834 11 Amber Herold
database connection, you will need to flush the mysql pivileges cache as mysql root
835
and connecting with you old password (or no password in the above
836 19 Amber Herold
case):_
837 11 Amber Herold
838
<pre>>mysql -u root mysql</pre>
839 22 Amber Herold
<pre>mysql> flush privileges;
840
mysql>^D or exit;</pre>
841 11 Amber Herold
842
*  Check MySQL variables
843
844 22 Amber Herold
<pre>>mysql -u usr_object dbemdata
845 11 Amber Herold
846 22 Amber Herold
mysql> SHOW VARIABLES LIKE 'query%';
847 11 Amber Herold
+------------------------------+-----------+
848
| Variable_name                | Value     |
849
+------------------------------+-----------+
850
| ft_query_expansion_limit     | 20        |
851
| have_query_cache             | YES       |
852
| long_query_time              | 10        |
853
| query_alloc_block_size       | 8192      |
854 21 Amber Herold
| query_cache_limit            | 104857600 | <<---This should correspond to your change
855 11 Amber Herold
| query_cache_min_res_unit     | 4096      |
856 21 Amber Herold
| query_cache_size             | 104857600 | <<---This should correspond to your change
857
| query_cache_type             | ON        | <<---This should correspond to your change
858 11 Amber Herold
| query_cache_wlock_invalidate | OFF       |
859 1 Eric Hou
| query_prealloc_size          | 8192      |
860
+------------------------------+-----------+
861
10 rows in set (0.00 sec)
862
863
mysql> exit;</pre>
864 11 Amber Herold
865 22 Amber Herold
866 11 Amber Herold
867
*  Make sure MySQL is running
868
869
<pre>prompt:~> mysqlshow
870 1 Eric Hou
+--------------+
871 11 Amber Herold
| Databases    |
872
+--------------+
873
| mysql        |
874
| dbemdata     |
875
| projectdata  |
876
+--------------+</pre>
877 21 Amber Herold
878 1 Eric Hou
879 11 Amber Herold
880 22 Amber Herold
*  Or check with the following php script (if already installed)
881 11 Amber Herold
882
<pre><?
883
mysql_connect('your_host.your_institute.edu', 'usr_object', '','dbemdata');
884
echo mysql_stat();
885
?> </pre>
886
887
888
Output
889
890
<pre> Uptime: 1452562 Threads: 1 Questions: 618 Slow queries: 0 Opens: 117 Flush tables: 1 Open tables: 106 Queries per second avg: 0.000</pre>
891
892 1 Eric Hou
893
894
895
896 11 Amber Herold
897 1 Eric Hou
898 11 Amber Herold
899 1 Eric Hou
900
901
h2.  Web server set up and Installation
902
903
904
905 11 Amber Herold
The following applies to the computer that will host the web-accessable image viewers ( a
906 1 Eric Hou
part of dbem tools) and project manager (a part of project tools).
907
908
909 11 Amber Herold
910 1 Eric Hou
h2. Differences between Linux flavors
911 11 Amber Herold
912
913 1 Eric Hou
914 11 Amber Herold
Different Linux flavors often put web server and mysql-related files in different
915 1 Eric Hou
locations. This can be confusing. From experience, we found the equivalent on CentOS vs
916
SuSE. Here we list them for reference. If your system use a different naming and you are
917
willing to share your experience, please send us the list. We will add it here:
918 11 Amber Herold
919
920
921 1 Eric Hou
|_.File or Command Head|_.CentOS|_.SuSE|
922 11 Amber Herold
| php.ini|/etc/| /etc/php5/apache2/|
923 14 Amber Herold
| httpd.conf|/etc/httpd/conf/|/etc/php5/apache2/|
924 11 Amber Herold
| default document_root|/var/www/html/|/srv/www/htdocs/|
925
| apache start/stop/restart command head|/etc/init.d/httpd|/etc/init.d/apache2|
926
| mysql start/stop/restart command head|/etc/init.d/mysqld|/etc/init.d/mysql|
927
928
929
930
931 1 Eric Hou
932
933
934
h2. Prerequisites for dbemtools, projecttools, and mrctools extension
935 11 Amber Herold
936 1 Eric Hou
937
938 11 Amber Herold
The dbemtools are mostly php scripts that run at the web server. The followings are
939
required before installation of dbemtools and the mrc extension that handles mrc files to be
940
displayed. Some of these packages may be found on the SuSE Linux DVD or included in common
941
package repository. MySQL and the Apache Web Server can be downloaded from their respective
942
websites.
943 1 Eric Hou
944
945
946
|_.Name:|_.Download site:|_.yum package name|_.SuSE rpm name|
947
| Apache|"www.apache.org":http://www.apache.org| httpd| apache2|
948
| php|"www.php.net":http://www.php.net| php| php|
949 29 Amber Herold
| php-devel*|"rpmfind.net/linux/RPM/Development_Languages_PHP.html":http://rpmfind.net/linux/RPM/Development_Languages_PHP.html| php-devel| php-devel|
950 25 Amber Herold
| php-gd (including GD library, its development libraries and header *)|"www.php.ned/gd":http://www.libgd.org (Use gd2)| php-gd, gd-devel| php-gd,gd-devel|
951 28 Amber Herold
| fftw3 library (including development libraries and header *)|"www.fftw.org":http://www.fftw.org (Use fftw3.x)| fftw3-devel| fftw3-devel|
952
953 24 Amber Herold
954 15 Amber Herold
955 11 Amber Herold
956
* mrctools are compiled and added to php extension with php-devel package. Mrctools use
957
GD and FFTW3 that need to be compiled from their development libraries while the extension
958
is compiled. If GD and FFTW3 sources were downloaded and compiled directly on your computer,
959
these development files are included. If (as in most cases) GD and FFTW3 are installed from
960
rpm, they are not included. An error message will appear when you attempt to compile
961
mrctools. In this case, you will need separate download and installation of GD-devel and
962 29 Amber Herold
FFTW3-devel. Search "http://rpmfind.net/linux/rpm2html/":http://rpmfind.ned/linux/rpm2html/ for GD-devel and FFTW3-devel for the rpm
963 26 Amber Herold
distribution needed for your system.
964 25 Amber Herold
965
966 1 Eric Hou
967 11 Amber Herold
968
969
h2. Installation tools
970 1 Eric Hou
971 11 Amber Herold
972
973
Use the installation tools available for your linux distribution.
974 1 Eric Hou
975
976 11 Amber Herold
977
978
*  For example, Use yum utility to install.
979
980
981
*  For example, Use the SuSE Linux YaST2 utility or zypper (openSuSE 10.2 and above)
982
to install.
983
984
985 29 Amber Herold
* "A list of required CENTOS
986
rpms and instruction":http://emg.nysbc.org/bb/viewtopic.php?t=238 can be found at dbemtools bulletin board at
987 11 Amber Herold
leginon.org.
988 15 Amber Herold
989 11 Amber Herold
990
991
992
993
994
995
996 16 Amber Herold
997 11 Amber Herold
h2.  PHP, PHP-devel, gd, and fftw3
998
999
1000
Install the packages using your installation tools if available. For example, to
1001
install gd as php extension you may use
1002
1003
1004
1005 17 Amber Herold
<pre>CentOS> yum install php-gd
1006 11 Amber Herold
SuSE10.2 and above> zypper install php-gd</pre>
1007
1008
1009
1010
1011
1012 16 Amber Herold
1013 11 Amber Herold
h2.  Configure php.ini
1014
1015
1016
1017
Edit the following two sections in php.ini (found as /etc/php.ini on CentOS and
1018 1 Eric Hou
/etc/php5/apache2/php.ini on SuSE) so that they look like the following:
1019 11 Amber Herold
1020
1021
1022
<pre>register_argc_argv = On</pre>
1023
1024
1025
1026
1027
<pre>short_open_tag = On</pre>
1028
1029
1030
1031
1032
<pre>max_execution_time = 300     ; Maximum execution time of each script, in seconds
1033
max_input_time = 300     ; Maximum amount of time each script may spend parsing request data
1034
memory_limit = 256M      ; Maximum amount of memory a script may consume (8MB)</pre>
1035
1036
1037
1038
You may want to increase max_input_time and memory_limit if the server is heavily used.
1039
At NRAMM, max_input_time=600 and memory_limit=4000M.
1040
1041
1042
1043
1044 16 Amber Herold
1045 11 Amber Herold
h2.  Apache Web Server
1046
1047
1048
1049
1050
1051
*  Install the Apache Web Server with the YaST or yum utility.
1052
1053
1054
*  Find "httpd.conf".
1055
1056
1057
This is /etc/httpd/conf/httpd.conf on CentOS and /etc/php5/apache2/httpd.conf on
1058
SuSE
1059
1060
1061
*  Edit the "httpd.conf" configuration file to look like the following:
1062
1063
1064
1065 22 Amber Herold
<pre><IfModule mod_dir.c>
1066 11 Amber Herold
DirectoryIndex index.htm index.shtm index.html index.shtml index.php
1067 22 Amber Herold
</IfModule></pre>
1068 11 Amber Herold
1069
1070 1 Eric Hou
(Note: It may be possible to edit httpd.conf in YaST2 as well.)
1071 11 Amber Herold
1072
1073
*  Restart the web server.
1074
1075
1076
1077 1 Eric Hou
<pre>apachectl restart
1078
or
1079
/etc/init.d/httpd restart     (ON CentOS)
1080
or
1081
/etc/init.d/apache2 restart   (ON SuSE)</pre>
1082
1083 21 Amber Herold
1084 11 Amber Herold
1085 22 Amber Herold
If you want to start the web server automatically at boot on SuSE
1086 11 Amber Herold
1087 1 Eric Hou
<pre> SuSE >chkconfig apache2 on
1088 11 Amber Herold
</pre>
1089
1090
1091
1092
1093
1094
1095
1096
1097
h2. Check php information
1098
1099
1100 16 Amber Herold
Create the following info.php in your web server document root directory (/var/www/html
1101 11 Amber Herold
on CentOS. /srv/www/htdocs on SuSE. You can find its location in httpd.conf mentioned above
1102
under the line starting
1103 1 Eric Hou
DocumentRoot).<pre><?php
1104
phpinfo();
1105 11 Amber Herold
?></pre>
1106
1107 16 Amber Herold
1108 11 Amber Herold
Visit this page at http://yourhost/info.php
1109
1110
1111
You will see comprehensive tables of php and apache information, including the location
1112
of the addition .ini files, extension, include path, and what extension is enabled.
1113
1114
1115 1 Eric Hou
Here is an example screen shot of the part of the info.php page that tells you where
1116 16 Amber Herold
php.ini and other configuration files are.
1117 11 Amber Herold
1118 1 Eric Hou
1119
1120
!http://emg.nysbc.org/software/leginon/images/images/phpini.png!
1121
1122
1123
1124
1125
1126
1127
h2.  mrctools Installation
1128
1129
1130
1131
mrctools are installed as php extension and are required for displaying mrc files live
1132
on the web browser.
1133
1134
1135
1136
h2.  Install php-devel packages on the web server if missing:
1137
1138
1139
You can check whether php-devel is installed by
1140
typing<pre> >phpize</pre>
1141
1142 11 Amber Herold
1143
Follow the instruction for your specific Linux distribution.
1144
1145 22 Amber Herold
1146 11 Amber Herold
For example, SUSE users can use YaST or zypper to install them
1147
1148
1149
1150
1151
1152
h2.  php-GD/FFTW3-devel
1153
1154
1155
1156 16 Amber Herold
Follow instructions from the download site. These may be included already. If not,
1157 11 Amber Herold
mrctools installation will fail.
1158
1159
1160
1161
1162 16 Amber Herold
1163 11 Amber Herold
h2.  mrctools Installation
1164
1165
1166
1167
mrctools are installed from php devel directory. This is usually /usr/include/php/ext
1168
where you will find other php extension source such as gd. After the installation, mrc.so
1169
should be placed in php's extension directory (Look for "extension_dir" in
1170 1 Eric Hou
http://your_host/<link linkend="install_infophp">info.php</link>
1171 11 Amber Herold
1172 26 Amber Herold
1173
1174
1175 29 Amber Herold
*  Download dbem tools from "http://emg.nysbc.org/software/mrctools":http://emg.nysbc.org/software/mrctools .
1176 26 Amber Herold
1177
1178 29 Amber Herold
*  Follow the instruction at "http://emg.nysbc.org/software/mrctools/mrc_so.php":http://emg.nysbc.org/software/mrctools/mrc_so.php for installation and
1179 26 Amber Herold
testing. The next session about checking php information can also help debugging the
1180
installation
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
h2.  Check php information
1191
1192
1193
1194
Visit or refresh http://yourhost/info.php <link linkend="install_infophp">which you
1195 11 Amber Herold
created earlier</link>. It should have a section looking like this (The version should
1196
correspond to what you've just installed):
1197 16 Amber Herold
1198 11 Amber Herold
1199 1 Eric Hou
1200
!http://emg.nysbc.org/software/leginon/images/images/phpmrc.png!
1201
1202
1203
1204 11 Amber Herold
If mrc is not listed, the extension did not get added at the right order.
1205
1206
1207
1208
1209 16 Amber Herold
1210 11 Amber Herold
h2.  Alternative approach if mrc module does not show up in info.php output
1211
1212
1213 1 Eric Hou
1214 11 Amber Herold
* find in the <link linkend="install_infophp">info.php web page</link> the location
1215 1 Eric Hou
of "additional .ini files parsed" in the first table (such as
1216 11 Amber Herold
/etc/php.d/conf.d/*).
1217
1218
1219
*  Go to the directory and make a copy of any ini file to use as a template for
1220
mrc.ini
1221
1222
1223
1224
<pre>>cd [additional_ini_directory]
1225
>cp gd.ini mrc.ini</pre>
1226 22 Amber Herold
1227
1228 11 Amber Herold
1229
*  Edit mrc.ini to the following
1230
1231
1232
1233 1 Eric Hou
<pre>; comment out next line to disable mrc extension in php
1234 11 Amber Herold
extension=mrc.so</pre>
1235
1236
1237
1238 1 Eric Hou
*  Comment out mrc extension from php.ini (found in /etc/php.ini/ on a typical PHP
1239
installation)
1240
1241
1242
1243
<pre>;extension=mrc.so</pre>
1244 11 Amber Herold
1245 22 Amber Herold
1246 11 Amber Herold
1247
*  restart your webserver
1248
1249
1250
1251
<pre>> /etc/init.d/httpd restart</pre>
1252 25 Amber Herold
1253
1254
1255 16 Amber Herold
1256 25 Amber Herold
1257
1258
1259
1260
1261
1262
h2.  Web Viewing Tools (dbemtools) Linux
1263
Installation
1264
1265
1266
1267
php and java scripts for viewing images and Leginon information through the web
1268
server.
1269
1270
1271
1272
1273 29 Amber Herold
*  Download dbem tools from "http://emg.nysbc.org/software/dbemtools":http://emg.nysbc.org/software/dbemtools .
1274 25 Amber Herold
1275
1276 11 Amber Herold
*  Unpack the package in your [webdirectory]
1277
1278 1 Eric Hou
1279
1280
<pre>>cd [webdirectory]			#/var/www/html in this example
1281
[webdirectory>tar zxvf dbem_1_5_1.tgz</pre>
1282
1283
1284
1285
*  Copy config.php.template to config.php and edit the latter by adding these MySQL
1286
parameters:
1287 11 Amber Herold
1288 1 Eric Hou
"config.php" should be located in /var/www/html/dbem_1_5_1/ on CentOS and
1289 11 Amber Herold
/srv/www/htdocs/dbem_1_5_1/ on SuSE.
1290
1291
1292
1293
<pre><?
1294
// ---define dbem web tools base ---//
1295 21 Amber Herold
define('BASE_URL',"/dbem_1_5_1/";
1296 11 Amber Herold
// --- Leginon Viewer Configuration --- //
1297 1 Eric Hou
1298 11 Amber Herold
// --- Set your leginon MySQL database server parameters
1299
1300
$DB_HOST    = "[your database host]";
1301 1 Eric Hou
$DB_USER    = "<link linkend="db_example_names">usr_object</link>";
1302 11 Amber Herold
$DB_PASS    = "";
1303
$DB        = "<link linkend="db_example_names">dbemdata</link>";
1304
1305
// --- XML test dataset
1306
$XML_DATA = "test/viewerdata.xml";
1307
1308
// --- Project database config
1309
1310
$PROJECT_URL = "/project_1_2";
1311 1 Eric Hou
$PROJECT_DB_HOST = "[your database host]";
1312
$PROJECT_DB_USER = "<link linkend="db_example_names">usr_object</link>";
1313
$PROJECT_DB_PASS = "";
1314 11 Amber Herold
$PROJECT_DB = "<link linkend="db_example_names">projectdata</link>";
1315
</pre>
1316
1317 1 Eric Hou
1318
1319
*  Test the set-up by visiting http://yourhost/dbem_1_5_1
1320
1321
1322 11 Amber Herold
1323
1324 16 Amber Herold
1325 11 Amber Herold
1326
1327
1328
1329
h2.  Project management tools Linux Installation
1330
1331
1332
1333
Leginon sessions can be catagorized into different projects. The projects are created
1334
and viewed trhough a set of web tools. To access them, install the tools on the web server.
1335
The same project management tools are also used to create and track appion processing
1336 16 Amber Herold
databases. If you want to use our processing pipeline, Appion, in the future, you must
1337 11 Amber Herold
install this.
1338
1339
1340
1341 1 Eric Hou
h2. Prerequisites for project tools
1342
1343 11 Amber Herold
1344
1345
installed dbemtools and its prerequisites
1346
1347 1 Eric Hou
1348
1349
1350
1351 11 Amber Herold
h2. project tools
1352
1353 1 Eric Hou
1354
Current release: project 1.2.
1355
1356
1357 26 Amber Herold
1358
1359 29 Amber Herold
*  Download project tools from "http://emg.nysbc.org/software/dbemtools":http://emg.nysbc.org/software/dbemtools .
1360 26 Amber Herold
1361
1362
*  Unpack the package in your [webdirectory]
1363
1364
1365
1366
<pre>>cd [webdirectory]			#/var/www/html in this example
1367 1 Eric Hou
[webdirectory]>tar zxvf project_1_2.tgz</pre>
1368 24 Amber Herold
1369
1370
1371
*  Go to the uncompressed directory
1372
project_1_2.<pre>> cd project_1_2</pre>
1373
1374 22 Amber Herold
1375 11 Amber Herold
* Copy config.php.template to _config.php_
1376 25 Amber Herold
1377 11 Amber Herold
1378 1 Eric Hou
1379 17 Amber Herold
<pre>> cp config.php.template to config.php</pre>
1380 11 Amber Herold
1381
1382 19 Amber Herold
1383 11 Amber Herold
* Edit _config.php_: You need to insert these MySQL
1384
parameters in project tool's config.php
1385
1386
1387
1388
<pre>// --- Project Configuration --- //
1389
1390 19 Amber Herold
// --- Leginon database config
1391 11 Amber Herold
$DBEM_PATH="../dbem_1_5_1/";
1392
1393
$DB_HOST  = "[your database host]";
1394
$DB_USER  = "usr_object";
1395 1 Eric Hou
$DB_PASS  = "";
1396 11 Amber Herold
$DB   = "<link linkend="db_example_names">dbemdata</link>";
1397
1398
// --- Project database config
1399
$PROJECT_DB_HOST = "[your database host]";
1400
$PROJECT_DB_USER = "<link linkend="db_example_names">usr_object</link>";
1401
$PROJECT_DB_PASS = "";
1402 1 Eric Hou
$PROJECT_DB = "<link linkend="db_example_names">projectdata</link>";
1403 11 Amber Herold
</pre>
1404
1405
1406
1407 1 Eric Hou
*  Test the set-up by visiting http://yourhost/project_1_2
1408 11 Amber Herold
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418 16 Amber Herold
1419 11 Amber Herold
h2.  Other Tools
1420
1421
1422
This section includes tools that may help with Leginon development.
1423
1424
1425 16 Amber Herold
1426 11 Amber Herold
h2.  phpMyAdmin
1427 26 Amber Herold
1428 1 Eric Hou
1429
1430
1431
1432 29 Amber Herold
*  Download phpMyAdmin from Yast2 or "www.phpmyadmin.net":http://www.phpmyadmin.net *  Download phpMyAdmin into the directory where the web server has been
1433 26 Amber Herold
installed.
1434 24 Amber Herold
1435 26 Amber Herold
1436
1437
1438
1439
1440
*  Check phpMyAdmin installation:
1441
1442
1443
1444 11 Amber Herold
<pre>>rpm -qa |grep -i phpmyadmin </pre>
1445 22 Amber Herold
1446 1 Eric Hou
1447
1448
*  Install from tarball:
1449
1450
1451
1452
<pre>> cd [webdirectory]
1453
[webdirectory]>tar zxvf phpMyAdmin-2.x.tar.gz</pre>
1454 11 Amber Herold
1455
1456 1 Eric Hou
1457
or Install with yum if availabe:
1458
1459
1460
1461
<pre>> yum install phpMyAdmin</pre>
1462
1463
1464
1465
*  Create phpMyAdmin configuration file called config.inc.php from a
1466 11 Amber Herold
sample.
1467
1468 1 Eric Hou
1469
1470
<pre>[webdirectory]>cd phpMyAdmin
1471
(if you install with yum, the program directory is /usr/share/phpMyAdmin)
1472
1473
> cp config.sample.inc.php config.inc.php
1474
</pre>
1475
1476 11 Amber Herold
1477
1478 1 Eric Hou
*  Make sure these are set correctly in _config.inc.php_. Note that these are for MySQL which we set earlier.
1479 11 Amber Herold
You can set specific user, but it is safer just leave it blank.
1480 1 Eric Hou
1481
1482
1483
<pre>
1484
$cfg['Servers'][$i]['host']            = '[your host]';
1485
$cfg['Servers'][$i]['port']            = '';
1486
$cfg['Servers'][$i]['socket']          = '';
1487
$cfg['Servers'][$i]['connect_type']    = 'tcp';
1488
$cfg['Servers'][$i]['extension']       = 'mysql';
1489
$cfg['Servers'][$i]['compress']        = FALSE;
1490
$cfg['Servers'][$i]['controluser']     = '';
1491
$cfg['Servers'][$i]['auth_type']     = 'http';
1492
$cfg['Servers'][$i]['user']          = '';
1493
$cfg['Servers'][$i]['password']      = '';
1494
</pre>
1495
1496
1497 11 Amber Herold
1498
*  For additional security, you may restrict mysql root access
1499
1500
1501
1502
<pre>
1503
$cfg['Servers'][$i]['AllowRoot']     = FALSE;
1504
</pre>
1505
1506 1 Eric Hou
1507 11 Amber Herold
1508 1 Eric Hou
*  If you want to access phpMyAdmin from another computer, you can add it to its
1509 11 Amber Herold
web access configuration file found as /etc/httpd/conf.d/phpMyAdmin.conf in a
1510
typical installation
1511
1512
1513 1 Eric Hou
1514 11 Amber Herold
<pre>
1515
<Directory /usr/share/phpMyAdmin/>
1516
order deny,allow
1517
deny from all
1518 1 Eric Hou
allow from 127.0.0.1
1519 11 Amber Herold
allow from YOUR_IP_ADDRESS
1520 21 Amber Herold
</Directory>
1521 11 Amber Herold
</pre>
1522
1523
1524
1525 21 Amber Herold
1526 11 Amber Herold
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
h2.  Windows installation
1537
1538
1539 16 Amber Herold
1540 11 Amber Herold
1541
h2.  Processing-side Leginon Windows Installation
1542 16 Amber Herold
1543 11 Amber Herold
1544 23 Amber Herold
1545 25 Amber Herold
1546
h2.  Install Python and Support Packages (Note that python 2.5 must be used):
1547
1548
1549
1550
This list does not include pyton XML module because it is included in the python
1551
package for window.
1552
1553 26 Amber Herold
1554 25 Amber Herold
1555 26 Amber Herold
|_.Name:|_.Download site:|
1556
|Python 2.5*|"http://www.python.org":http://www.python.org|
1557 29 Amber Herold
|Python for Windows extension (pywin32)|"http://sourceforge.net/projects/pywin32/":http://sourceforge.net/projects/pywin32/|
1558 24 Amber Herold
|wxPython 2.5.2.8 or newer|"http://www.wxpython.org":http://www.wxpython.org|
1559 29 Amber Herold
|MySQL Python client 1.2 or newer|"http://sourceforge.net/projects/mysql-python":http://sourceforge.net/projects/mysql-python|
1560
|Python Imaging Library (PIL) 1.1.4 or newer|"http://www.pythonware.com/products/pil/":http://www.pythonware.com/products/pil/|
1561 25 Amber Herold
|NumPy 1.0b5 (tested, others may work)|"http://www.scipy.org":http://www.scipy.org|
1562 26 Amber Herold
|SciPy 0.5.1 or newer|"http://www.scipy.org":http://www.scipy.org|
1563 29 Amber Herold
|Tortoise SVN client|"http://tortoisesvn.tigris.org":http://tortoisesvn.tigris.org|
1564 25 Amber Herold
1565
1566 11 Amber Herold
1567
1568
*Python 2.5 is the only python version that we have compiled numExtension. libCV and
1569 23 Amber Herold
comarray in. Therefore no other python version works for now.
1570 25 Amber Herold
1571
1572
Execute the installer file and follow the directions.
1573
1574
1575 11 Amber Herold
1576
1577
1578 23 Amber Herold
h2. Packages required from NRAMM
1579 25 Amber Herold
1580
1581 1 Eric Hou
1582 23 Amber Herold
Here are the packages you need to install with python installer
1583 25 Amber Herold
1584 24 Amber Herold
1585 26 Amber Herold
1586
|_.Name:|_.Purpose:|
1587
| Leginon|modular TEM image acquisition|
1588 1 Eric Hou
| pyami|general functions|
1589
| sinedon|Leginon/database interaction|
1590 26 Amber Herold
| pyScope|microscope control and monitoring|
1591
| ImageViewer|image viewing for tomography|
1592
1593
1594
1595
1596
Because numextension and libCV requires extra compilers, we have created window
1597 27 Amber Herold
installer for them for python 2.5 and made them available through "http://www.leginon.org/":http://www.leginon.org/
1598
1599 11 Amber Herold
|_.Downloadfile Name|_.Installed Python Package File|_.Purpose:|
1600
|NumExtension-1.2.0.win32-py2.5.exe|numextension.pyd|c extension for numerical processing|
1601 1 Eric Hou
|libCV-0.2.win32-py2.5.exe|libCV.pyd|small c library of algorithm from computer vision field|
1602 11 Amber Herold
1603
1604
1605 16 Amber Herold
1606 11 Amber Herold
1607
1608
1609
h2. Check out SVN Source Files from the depository
1610
1611
1612
1613
Use your mouse to do the following
1614
* Create Leginon-1.6-ALL directory somewhere at your convenience
1615
1616 1 Eric Hou
1617 11 Amber Herold
* Change directory into Leginon-1.6-ALL
1618 17 Amber Herold
1619 1 Eric Hou
1620 11 Amber Herold
* Right-click the mouse botton in this directory window and select Tortoise svn
1621
Checkout in the menu:!http://emg.nysbc.org/software/leginon/images/images/svnmenu.png!
1622 17 Amber Herold
1623 11 Amber Herold
1624
* Set up svn checkout window like this for <link linkend="windows_packages">EACH
1625 1 Eric Hou
of the leginon packages</link>.!http://emg.nysbc.org/software/leginon/images/images/svnco.png!
1626 11 Amber Herold
1627
1628
1629
1630 16 Amber Herold
1631 11 Amber Herold
1632
1633 1 Eric Hou
1634 11 Amber Herold
h2. Install the packages you downloaded from NRAMM svn depository
1635
1636
1637
1638
1639
1640
* Start a command line Window from Start Menu
1641
1642
1643
* Install the package in each folder with commands such as
1644
<pre>cd Your_Download_Place\Leginon-1.6-ALL\leginon
1645 1 Eric Hou
c:\\python25\python.exe setup.py install</pre>
1646 11 Amber Herold
1647
1648 1 Eric Hou
1649
1650
1651
1652
1653 27 Amber Herold
1654
1655
h2. Download the two Window Installer Files from Leginon website
1656
1657
1658
1659 28 Amber Herold
"http://www.leginon.org/":http://www.leginon.org/ h2. Install individual packages
1660 11 Amber Herold
1661
1662
1663
Excute the installer files and follow the instruction.
1664
1665 16 Amber Herold
1666 11 Amber Herold
1667
1668
1669
h2.  Mapping Drives:
1670
1671
1672
1673
If you plan to run Leginon directly on the Windows machine, such as in <link
1674
linkend="config_C">configuration C</link>, and your data files are served through a
1675
Samba server on a Linux machine, you will need to map the network drive. For example, if
1676 1 Eric Hou
your Samba server has a hostname your_smbserver, and you have set up a share called
1677 11 Amber Herold
[your_share_point] which points to /your_data_path/ and leginon data will be saved under a
1678
folder in /your_data_path/leginon/
1679
1680
1681 1 Eric Hou
1682 11 Amber Herold
1683
*  Start, My Computer
1684 1 Eric Hou
1685 11 Amber Herold
1686
*  Tools menu, Map network drive
1687
1688 1 Eric Hou
1689 17 Amber Herold
*  Use an unmapped drive such as Z:
1690 11 Amber Herold
1691
Enter shared path in Windows format
1692
as<pre>\\your_smbserver\your_share_point</pre>
1693 17 Amber Herold
1694 11 Amber Herold
1695
*  Add the drive and the Linux path to leginon.cfg on the Windows machine
1696
as<pre>[Drive Mapping]
1697
Z:/your_data_path</pre>
1698 17 Amber Herold
1699 11 Amber Herold
1700
*  Add image path to leginon.cfg on the Windows machine in Linux format
1701
as<pre>[Images]
1702
path:/your_data_path/leginon</pre>
1703
1704
1705
1706
1707
1708 16 Amber Herold
1709 11 Amber Herold
1710
1711
1712
h2.  Configure leginon.cfg:
1713
1714
1715
Follow instruction in "<link linkend="leginon_cfg">Configure leginon.cfg</link>" in
1716
the section for Linux installation but note the location of the configuration files
1717
follows. In addition, if the storage disk is mapped onto the Windows PC as drive Z, this
1718
mapping should be included in leginon.cfg. See above.
1719
1720
1721
1722
1723
*  Configurations for all users
1724 22 Amber Herold
1725 11 Amber Herold
1726
1727
<pre> <Python directory>\Lib\site-packages\Leginon\config\leginon.cfg</pre>
1728
1729
1730
Example:
1731
<pre> C:\Python25\Lib\site-packages\Leginon\config\leginon.cfg</pre>
1732
1733
1734
*  Configurations for individual users
1735 22 Amber Herold
1736 11 Amber Herold
1737
1738
<pre> <Home directory>\leginon.cfg</pre>
1739
1740
1741
Example:
1742
<pre> C:\Documents and Settings\Leginon User\leginon.cfg </pre>
1743
1744
1745
*  A skeleton (default) configuration file is available:
1746
1747
1748
1749
<pre> C:\Python25\Lib\site-packages\Leginon\config\default.cfg</pre>
1750
1751
1752
1753
1754
1755 1 Eric Hou
1756 16 Amber Herold
1757 11 Amber Herold
1758
1759
1760
h2.  Configure sinedon.cfg:
1761
1762
1763
Sinedon is designed to be able to interact with multiple databases.
1764
1765
Follow instruction in "<link linkend="sinedon_cfg">Configure sinedon.cfg</link>" in
1766
the section for Linux installation but note the location of the configuration files
1767
follows.
1768
1769
1770
1771
1772
* For all users, put sinedon.cfg with the installed package
1773
as<pre>C:\Python25\Lib\site-packages\sinedon\sinedon.cfg</pre>
1774 22 Amber Herold
1775 11 Amber Herold
1776
*  Your home directory on
1777
Windows:<pre>C:\Documents and Settings\your_name></pre>
1778
1779
1780
* the skeleton sinedon configuration file
1781
is<pre>C:\Python25\Lib\site-packages\sinedon\examples\sinedon.cfg</pre>
1782
1783
1784
1785
1786
1787 16 Amber Herold
1788 11 Amber Herold
1789
1790
1791 1 Eric Hou
h2.  Create Leginon and Leginon Client shortcut in Start menu menu under Leginon
1792 11 Amber Herold
1793
1794
1795
This instruction refers to Windows XP
1796
1797
1798 1 Eric Hou
1799 11 Amber Herold
1800
* Go to C:\Documents and Settings\All Users\Start Menu\Programs\ and create a new
1801
folder named Leginon.
1802 17 Amber Herold
1803 11 Amber Herold
1804
* In another window, go
1805 1 Eric Hou
to<pre>C:\Python25\Lib\site-packages\Leginon</pre>
1806 11 Amber Herold
1807
1808
* Create a shortcut from start-leginon.py as Leginon and a shortcut from
1809
launcher.py as Leginon Client.
1810 17 Amber Herold
1811 11 Amber Herold
1812
* Move the two shortcuts into
1813
<pre>C:\Documents and Settings\All Users\Start Menu\Programs\Leginon</pre>
1814
1815
1816
1817
1818
1819 16 Amber Herold
1820 11 Amber Herold
1821 1 Eric Hou
1822 11 Amber Herold
1823
h2.  Additional Software (Optional):
1824
1825
1826
TightVNC (http://www.tightvnc.com)
1827
1828
1829
1830
1831
1832 16 Amber Herold
1833 11 Amber Herold
1834
1835
1836
h2. Database server Windows Installation
1837 16 Amber Herold
1838 11 Amber Herold
1839
1840
1841
h2. We do not do this at NRAMM. Please follow the instruction in Linux installation and
1842
modify it for Windows at your own risk.
1843
1844
1845
1846
For a good Windows specific instruction for general PHP configuration with MySQL for
1847
Apache 2 in Windows, try http://www.artfulsoftware.com/php_mysql_win.html.
1848
1849
1850
1851
1852
1853 16 Amber Herold
1854 1 Eric Hou
1855 11 Amber Herold
1856
1857
h2.  Additional installation on the microscope computer:
1858
1859
1860
1861
The full leginon and its supporting packages need to be installed on the Windows computer
1862
controlling the microscope. Additional programs are required for communication with the CCD
1863
and extra functions for Tecnai scripting not supported as default by FEI Tecnai
1864 16 Amber Herold
microscopes
1865 11 Amber Herold
1866
1867
1868
h2.  Required supporting programs for the CCD camera from camera makers
1869
1870
1871 1 Eric Hou
1872
Install and register the following programs for CCD cameras from the two makes:
1873 14 Amber Herold
1874 11 Amber Herold
1875
1876
|_.Camera Make:|_.File:|
1877
| Gatan|TecnaiCCD.dll|
1878
| Tietz|CAMC4.exe*|
1879
1880
1881
1882
1883
* We have experienced slowness of the CAMC4.exe comes with later version Tecnai TUI/TIA.
1884
Replacing it with an earlier version of CAMC4.exe resolved the problem.
1885
1886 16 Amber Herold
1887 11 Amber Herold
1888
1889
1890
h2. Additional Package required from NRAMM for Gatan camera or camera that uses TIA
1891
1892 1 Eric Hou
1893 11 Amber Herold
1894 1 Eric Hou
For Gatan Camera or FEI Eagle Camera that uses TIA, comarray package need to be install
1895
with python
1896
1897 11 Amber Herold
1898
1899
|_.SVN Package Name|_.Installed Python Package Name|_.Reason for update:|
1900
|comarray|comarray|com module output conversion to array|
1901
1902
1903
1904
1905
1906
1907 17 Amber Herold
* <link linkend="window_svn">Check out the comarray package</link> from svn
1908 11 Amber Herold
depository with your SVN client
1909
1910
<pre>cd Leginon-1.6-ALL
1911 1 Eric Hou
svn co http://emg.nysbc.org/svn/comarray/branches/leg1.6 comarray</pre>
1912 11 Amber Herold
1913
* Start a command line Window from Start Menu
1914
1915
1916
* Install the package with commands such as
1917
<pre>cd Your_Download_Place\Leginon-1.6-ALL\comarray
1918
c:\\python25\python.exe setup.py install</pre>
1919
1920
1921
1922
1923
1924 16 Amber Herold
1925 11 Amber Herold
1926
1927
1928
h2.  Supporting programs for film exposure
1929
1930
1931 1 Eric Hou
1932
Install the following if you need film exposure on FEI Tecnai TEM through Leginon,
1933 11 Amber Herold
available through FEI. Please contact Max Otten: <email>mto@feico.com</email> and request
1934 14 Amber Herold
for adaexp.exe that works with your version of Tecnai user interface program.
1935 13 Amber Herold
1936 11 Amber Herold
1937
1938
|_.Name:|_.File:|
1939
| exposure adaptor|adaexp.exe|
1940
1941
1942
1943 16 Amber Herold
1944 11 Amber Herold
1945
1946
1947
h2.  Register adaexep.exe
1948
1949
1950
1951
*  From the command prompt:
1952
1953
<pre>adaexp.exe /regserver</pre>
1954
1955
*  From the program start menu under pyScope or
1956
C:\Python25\Lib\site-packages\pyScope\
1957
1958
<pre>updatecom.py</pre>
1959
1960
1961
1962
1963 16 Amber Herold
1964 11 Amber Herold
1965
1966 1 Eric Hou
1967 11 Amber Herold
h2. Modify instruments.cfg:
1968
1969
1970
1971
Sinedon now have full control of database interaction, therefore, the database
1972
configuration in leginon.cfg is no longer needed.
1973
1974
1975
1976
1977
* Modify the file instruments.cfg in the installed pyScope directory to that for
1978
your microscope and camera. For example, if your microscope uses Tecnai Scripting
1979
Interface and you have a Gatan camera that you interface through
1980
DigitalMicrograph:
1981
1982
1983
1984
<pre>[tem]
1985
class: tecnai.Tecnai
1986 1 Eric Hou
[camera]
1987 11 Amber Herold
class: gatan.Gatan</pre>
1988
1989
1990
1991
The file contains other examples of microscope and camera drivers that we
1992
distribute from NRAMM.
1993
1994
1995
1996
1997
1998 16 Amber Herold
1999 11 Amber Herold
2000
2001
2002
h2. Run updatecom.py
2003
2004
2005
2006
From a command line window:
2007
2008
2009
2010
<pre>cd C:\python25\Lib\Site-Packages\pyScope
2011
C:\python25\python.exe updatecom.py</pre>
2012
2013
2014
2015
This should generate a few files, including tecnaicom.py, gatancom.py and tietzcom.py,
2016
in the same directory.
2017
2018 16 Amber Herold
2019 11 Amber Herold
2020
2021
2022
h2. Additional setup on Tietz PXL camera (optional)
2023
2024
2025
2026
The actual Tietz PXL camera dimension is slightly larger than 2048 x 2048. If you'd like
2027 1 Eric Hou
to take the images using the maximum of 2048 x 2048 pixels, you may modify the function that
2028 11 Amber Herold
get camera dimension in tietz.py of pyScope package.
2029
2030
2031
*  Go to C:\Python25\Lib\site-packages\pyScope\
2032
2033
2034
*  Edit tietz.py with a plain text editor
2035
2036
2037
*  Find the lines:
2038
2039
2040
2041 1 Eric Hou
<pre> def getCameraSize(self):
2042 11 Amber Herold
# {'type': dict, 'values': {'x': {'type': int}, 'y': {'type': int}}}}
2043
x = self._getParameterValue('cpTotalDimensionX')
2044
y = self._getParameterValue('cpTotalDimensionY')
2045
return {'x': x, 'y': y}
2046 1 Eric Hou
</pre>
2047 11 Amber Herold
2048
2049
2050
*  Change the last line to:
2051
2052
2053
2054
<pre>    return {'x': 2048, 'y': 2048}</pre>
2055
2056 1 Eric Hou
2057 11 Amber Herold
2058
2059 16 Amber Herold
2060 11 Amber Herold
2061
2062
2063
h2.  Additional Software (Optional):
2064
2065 1 Eric Hou
2066 11 Amber Herold
TightVNC (http://www.tightvnc.com) if you get tired of going into the microscope room
2067
just to open the column valves.
2068
2069
2070
2071
2072
2073 16 Amber Herold
2074 11 Amber Herold
2075
2076 1 Eric Hou
2077 11 Amber Herold
h2.  What is next
2078 16 Amber Herold
2079 11 Amber Herold
2080
2081 1 Eric Hou
2082 11 Amber Herold
h2.  Perform Administration Setup
2083
2084
2085
See Next chapter on <link linkend="addusers">Leginon Administration Tools</link>.
2086 1 Eric Hou
2087 11 Amber Herold
2088 16 Amber Herold
2089 11 Amber Herold
2090
2091
2092
h2.  Create a test project
2093
2094
2095
See the chapter on <link linkend="project">Project Management Tools</link>.
2096 1 Eric Hou
2097 16 Amber Herold
2098 11 Amber Herold
2099
2100
2101
h2.  Perform Microscope Setup and Test run
2102
2103
2104
See the chapter on <link linkend="runleg_chapter">Start Leginon</link>.
2105
2106
2107 1 Eric Hou
2108 16 Amber Herold
2109 11 Amber Herold
2110
2111
2112
h2.  Backup Practices
2113 16 Amber Herold
2114 11 Amber Herold
2115
2116
2117
h2.  Ideal setup:
2118 16 Amber Herold
2119 11 Amber Herold
2120 1 Eric Hou
2121 11 Amber Herold
2122
h2.  Regular computer backup
2123
2124
2125
2126
To protect your program installation.
2127
2128 16 Amber Herold
2129 11 Amber Herold
2130 1 Eric Hou
2131 11 Amber Herold
2132
h2.  Full backup of the databases every night
2133
2134
2135
2136
The database is the metadata generated by Leginon named "dbemdata" in the installation
2137
example.
2138
2139 16 Amber Herold
2140 11 Amber Herold
2141
2142
2143 1 Eric Hou
h2. Full/Incremental/differential backup of the images
2144 11 Amber Herold
2145
2146
2147
The image files located in [your storage disk] should be backup in full at least once
2148
a month, and nightly in differential or incremental mode.
2149
2150 1 Eric Hou
2151 11 Amber Herold
2152 16 Amber Herold
2153 11 Amber Herold
2154
2155
2156
h2.  Current NRAMM setup:
2157
2158
2159
2160
Because our tape backup system is not on-site and the data have to transfer over slow
2161
network to perform the task, we have not been able to backup in the ideal setting. Here is
2162 16 Amber Herold
what we currently do:
2163 11 Amber Herold
2164
2165
2166
h2.  Regular computer backup
2167
2168
2169
2170
TSRI Research Computing provides regular back up service that does full backup of the
2171
computers every four weeks and incremental backup every night. The full backup expires in
2172
4 months and the incremental backup one month. We do this for the computers running
2173
Leginon main program, the database server, as well as the web server.
2174
2175 16 Amber Herold
2176 11 Amber Herold
2177
2178
2179
h2.  Regular backup &amp; tar of the databases every night
2180
2181
2182 16 Amber Herold
2183 11 Amber Herold
On top of the TSRI backup, we also tar the database everynight and store it on the
2184
institutional tape library.
2185
2186
2187
2188
2189
2190
h2.  rsync to a tape library of the image storage disk
2191
2192
2193
2194
As our image storage disks total 15T byte, we currently use rsync function to update a
2195
copy on our institutional tape library. It creates new files, updates old files, but does
2196 1 Eric Hou
not remove old files on the tape if it is removed on the storage disk. We do not think
2197
this is the best solution since it is not possible to retrieve old files that have been
2198
modified on a later time.