Project

General

Profile

Appion Home » History » Version 25

Amber Herold, 04/01/2010 10:51 AM

1 1 Eric Hou
2 11 Amber Herold
h1. Complete Installation
3 6 Amber Herold
4
5 1 Eric Hou
6 16 Amber Herold
7 11 Amber Herold
h2. Four Parts of Leginon System
8 1 Eric Hou
9 11 Amber Herold
10 16 Amber Herold
!http://emg.nysbc.org/software/leginon/images/images/Leg_Proj_parts.png!
11 1 Eric Hou
12 11 Amber Herold
Leginon as a system that we distribute can be divided into three parts:
13 1 Eric Hou
14 6 Amber Herold
15 16 Amber Herold
16 11 Amber Herold
h2. Processing Server (Python-side)
17 1 Eric Hou
18
19 11 Amber Herold
Python (and some c) scripts that handle instrument control, data acquisition, and
20
processing.
21 1 Eric Hou
22
23
24
25 16 Amber Herold
26 11 Amber Herold
h2. Database Server (MySQL-side)
27 1 Eric Hou
28 11 Amber Herold
29
a MySQL server that handles the database
30
31
32 1 Eric Hou
33 11 Amber Herold
34 16 Amber Herold
35 11 Amber Herold
h2. Web Server (PHP-side)
36
37
38
This includes php and Java scripts at a webserver that we will create to retrieve image
39
and metadata from the database and file-storage system.
40
41
42 1 Eric Hou
43 11 Amber Herold
44 16 Amber Herold
45 11 Amber Herold
h2. File Server
46
47
48
This is up to you to set up to store lots of data coming out of Leginon system.
49
50
51
52
53 1 Eric Hou
54 11 Amber Herold
55 16 Amber Herold
56 11 Amber Herold
h2. What is in this chapter
57
58
59
60
Leginon runs under both the Linux and Microsoft Windows Operating systems. However, the
61
current php-mrctool can not be installed on Windows which means mrc images can not be viewed
62
on the web. See the section on Possible Computer Set-up Configurations for details.
63
64
65
The many components for the Leginon system are divided into packages and need to be
66
installed separately. It has been integrated with several third party software packages that
67
are necessary for its operation. The installation documentation will describe how to set-up
68
Leginon and give hints as to how to install and set-up the necessary third-party software.
69
This documentation is not intended to support the additional, but necessary third-party
70
software.
71
72
73
There has been interest in installing Leginon under Mac OS X. While this is possible in
74
theory, we have not been successful in installing a fully functional system. The two main
75
problems are (1) compilation of php-mrctool from the pre-installed php and (2) wxPython on Mac
76
is unable to hide bitmap objects which makes Leginon graphical user interface difficult to
77
use.
78
79
80 24 Amber Herold
See <link linkend="InstT_install">Installation Troubleshooting</link> and "Leginon Bulletin Board":http://emg.nysbc.org/bb/viewforum.php?f=2 searching
81 11 Amber Herold
for "install" if you run into problems.
82
83
84 1 Eric Hou
85 11 Amber Herold
86 16 Amber Herold
87 11 Amber Herold
h2.  Possible Computer Set-up Configurations
88
89
90
91
Computer(s) used for the whole Leginon system need to support five functions:
92
*  TEM/Camera control and data output (by python-side packages of Leginon
93
system)
94
95
96
*The computer attached to the microscope/camera is used for this function.
97
Therefore, it is on a Windows computer.
98
99
100
*  General Leginon operation such as target selection and image processing (processing
101
server of Leginon system)
102
103
104
*  Image data storage.
105
106
107
*  Other meta-data storage through MySQL database (by database server of Leginon
108
system)
109
110
111
*  Web server (using PHP-side of Leginon system).
112
113
114
*Important* The MRC module that is used to load mrc formatted files to the web
115
pages is not compiled on Windows. Therefore, this function must be on a linux box unless
116
you don't need to see the images***
117
118
119
120
121
122
The five functions can be distributed to five different computers on a network or all on
123
one single computer. However, since the latter minimal setup requires the use of the computer
124
attached to the microscope for all functions, it is not advisable.
125
126
127
Processing server of Leginion is a multi-platform software, meaning it can run on both
128
Microsoft Windows and Linux. It is also inherently designed to utilize distributed systems on
129
a network. MySQL used in the database server is also supported by multiple platforms. This is
130
not true for the Web server because the custom MRC module used in the web viewer only compiles
131
on linux machines. Therefore, one of the computer not part of the microscope must be a linux.
132
The following examples show several arrangements that take advantage of distributed system.
133
Other combinations are possible, but will basically be variations on one of the following
134
themes.
135 1 Eric Hou
136 11 Amber Herold
137 16 Amber Herold
138 11 Amber Herold
h2.  A: 1 Windows computer (attached to the Microscope) &amp; 1 Linux computer (for all
139
other functions, separate from the Microscope) (Good for single user and small-scale
140
acquisition)
141
142
143
144
145
146
*  Install the latest processing server side of Leginon release (and supporting
147
packages) on the Windows computer controlling the microscope and on the Linux computer
148
which is to run Leginon image acquisition and processing.
149
150
151
*  Install the additional modules needed for CCD camera and film interaction on the
152
Windows computer controlling the microscope.
153
154
155
*  Install MySQL, PHP, the Apache Web Server, and dbem tools only on the Linux
156
computer that will perform these functions, (i.e. NOT the computer controlling the
157
microscope).
158
159
160
161
162
163
164 1 Eric Hou
165 11 Amber Herold
166 16 Amber Herold
167 11 Amber Herold
h2.  B: 1 Windows computer (attached to the Microscope) &amp; 4 Linux computers (one for
168
each function) (Good for multiple microsopes, users and large-scale acquisition)
169
170
171
172
This is the set-up at NRAMM. It has the processing server side of Leginon installed on
173
the Windows computer attached to the microscope and on the Linux computer that is used for
174
running Leginon. A second Linux machine is dedicated to the database while the web-based
175
viewer is hosted by another server.
176
177
178
179
180
*  Install the python-side Leginon (and supporting packages) on the Windows computer
181
controlling the microscope and on the Linux computer that is to run Leginon. You can
182
make the latter installation accessable to any numbers of linux computers on your
183
network.
184
185
186
*  Install MySQL, PHP, the Apache Web Server, and dbem tools on the second Linux
187
computer that will serve as the database/web server/PHP host.
188
189
190
*  The computer hosting the data storage should be accessable by all linux
191
boxes.
192
193
194
195
196
197
198 1 Eric Hou
199 11 Amber Herold
200 16 Amber Herold
201 11 Amber Herold
h2.  C: 1 Windows computer (attached to the Microscope), 1 Windows computer running python-
202
and MySQL-side functions and 1 Linux computer as web server (Has been applied
203
successfully)
204
205
206
207
The data storage may be on one of these computers that is accessable by other
208
computers.
209
210
211
212
213 1 Eric Hou
214 11 Amber Herold
215 16 Amber Herold
216 11 Amber Herold
h2.  Ports that Leginon system uses:
217
218
219
220
This information is useful if you want to open specific ports between computers with a
221
firewall in between.
222
223
224
225
226
* MySQL normally uses port 3306 (but configurable in /etc/my.cnf)
227
228
229
* Leginon Client running on TEM host: port 55555
230
231
232
* Leginon main program on the processing server: dynamically assigned ports between
233
49152 and 65535
234
235
236
There is no strict port assignment since we could potentially have more than one
237
Leginon process running on the same linux host talking to different TEM hosts. It is
238
probably good enough if you only worry about opening up the first few of those ports in
239
your firewall (maybe 49152 through 49160, or something like that).
240
241
242
See more discussion at Leginon bulletin board thread on ""network problem, Leginon not seeing
243 24 Amber Herold
tecnai host":http://emg.nysbc.org/bb/viewtopic.php?t=7 "
244 11 Amber Herold
245
246
247
248
249
250 1 Eric Hou
251 11 Amber Herold
252 16 Amber Herold
253 15 Amber Herold
h2. Where to Register and download Leginon system
254 11 Amber Herold
255
256
257 24 Amber Herold
"http://www.leginon.org/":http://www.leginon.org is the home. Please
258 11 Amber Herold
register if you want to recieve support from Leginon team.
259
260 1 Eric Hou
261
*  python-side Leginon packages
262
263 23 Amber Herold
264 25 Amber Herold
direct link: "http://www.leginon.org/":http://emg.nysbc.org/software/leginon/"
265
>http://emg.nysbc.org/software/leginon/</ulink>
266 1 Eric Hou
267 11 Amber Herold
268 25 Amber Herold
*  mrctools: including the tools for reading mrc images on the web
269 1 Eric Hou
270 11 Amber Herold
271 25 Amber Herold
direct link: <ulink url="http://emg.nysbc.org/software/mrctools/"
272
>http://emg.nysbc.org/software/mrctools/</ulink>
273 23 Amber Herold
274 1 Eric Hou
275 25 Amber Herold
*  dbemtools: including the tools for administration and webviewing
276 23 Amber Herold
277 1 Eric Hou
278 25 Amber Herold
direct link: <ulink url="http://emg.nysbc.org/software/dbemtools/"
279
>http://emg.nysbc.org/software/dbemtools/</ulink>
280 1 Eric Hou
281 25 Amber Herold
282
*  project tools (optional): For administration and webviewing of projects
283
284
285
direct link: <ulink url="http://emg.nysbc.org/software/dbemtools/"
286
>http://emg.nysbc.org/software/dbemtools/</ulink>
287
288
289
290
291
292
293
294
h2.  Processing server-side Leginon Installation
295
296
297
298 11 Amber Herold
For the lack of better name, processing server-side Leginon refers to packages that
299 1 Eric Hou
performs the general functions of image acquisition and processing. The instructions here are
300 11 Amber Herold
for Linux. Refer to Windows Installation for Windows specific instructions.
301
302 16 Amber Herold
303 1 Eric Hou
304 11 Amber Herold
h2. Download svn checkout script from Leginon website
305
306
307
308 25 Amber Herold
<ulink url="http://www.leginon.org/ Download
309 15 Amber Herold
svnget-1.6.sh to a convenient location such as your home directory.
310 11 Amber Herold
311
312
313
314 16 Amber Herold
315 11 Amber Herold
h2. Check out SVN Source Files from the depository
316
317
318
319
run svnget-1.6.sh. It does the
320 17 Amber Herold
following:<pre>mkdir Leginon-1.6-ALL
321 11 Amber Herold
cd Leginon-1.6-ALL
322
svn co http://emg.nysbc.org/svn/leginon/branches/1.6 Leginon
323 1 Eric Hou
svn co http://emg.nysbc.org/svn/pyami/branches/leg1.6 pyami
324 11 Amber Herold
svn co http://emg.nysbc.org/svn/pyScope/branches/leg1.6 pyScope
325
svn co http://emg.nysbc.org/svn/sinedon/branches/leg1.6 sinedon
326
svn co http://emg.nysbc.org/svn/numextension/branches/leg1.6 numextension
327
svn co http://emg.nysbc.org/svn/libcv/branches/leg1.6 libcv
328
svn co http://emg.nysbc.org/svn/ImageViewer/branches/leg1.6 ImageViewer</pre>
329
330
331
If the process fails, you probably do not have svn client installed. Skip to "Install
332
the supporting packages first if missing:", then try this again.
333
334
335
336
337 16 Amber Herold
338 11 Amber Herold
h2.  Perform system check:
339
340
341
342
In addition to the downloads from our svn depository, there are several other
343
requirements that you will get either from your OS installation source, or from its
344
respective website. The system check in the Leginon package checks your system to see if you
345
already have these requirements
346
347 1 Eric Hou
348 11 Amber Herold
349
<pre>cd your_download_area/Leginon-1.6-ALL/Leginon
350
python syscheck.py</pre>
351
352
353
354
If python is not installed, this, of course will not run. If you see any lines like "***
355
Failed...", then you have something missing. Otherwise, everything should result in
356
"OK".
357
358
359 1 Eric Hou
360
361
362
h2.  Install the supporting packages first if missing:
363
364 24 Amber Herold
365
366
Follow the instruction for your specific Linux distribution.
367
368
369
For example, SUSE users can use YaST to install them; RedHat and CentOS users can use
370
yum.
371
372
373
*SciPy may not build properly on some versions of SuSE due to an incompatible LAPACK
374
package that comes with SuSE. You can get scipy as well as a compatible LAPACK etc. from
375
http://repos.opensuse.org/science (need to specify your SuSE version and machine
376 1 Eric Hou
etc.)
377
378
379
380
381
382
h2.  Required supporting packages:
383
384
385
386
387 24 Amber Herold
|_.Name:|_.Download site:|_.yum package name|_.SuSE rpm name|
388 25 Amber Herold
|Python 2.4 or newer|"http://www.python.org":http://www.python.org|python|python-devel|
389
|wxPython 2.5.2.8 or newer|"http://www.wxpython.org":http://www.wxpython.org|wxPython|python-wxGTK|
390
|MySQL Python client 1.2 or newer|"http://pyxml.sourceforge.net":http://sourceforge.net/projects/mysql-python"
391
>http://sourceforge.net/projects/mysql-python</ulink>|MySQL-python|python-mysql|
392
|Python Imaging Library (PIL) 1.1.4 or newer|<ulink url="http://www.pythonware.com/products/pil/"
393
>http://www.pythonware.com/products/pil/</ulink>|python-imaging|python-imaging|
394
|Python XML module 0.8.3 or newer|<ulink url="http://pyxml.sourceforge.net|PyXML|python-xml|
395
|NumPy 1.0.1 or newer|"http://www.scipy.org":http://www.scipy.org|numpy|numpy|
396
|SciPy 0.5.1 (tested, others may work)*|"http://www.scipy.org":http://www.scipy.org , "
397
<colspec colnum="1"/>
398
<colspec colnum="2"/>
399 24 Amber Herold
400
401 11 Amber Herold
|_.Name:|_.Purpose:|
402 25 Amber Herold
<entry> Leginon|modular TEM image acquisition|
403 11 Amber Herold
| pyami|general functions|
404
| pyScope|microscope control and monitoring|
405
| sinedon|Leginon/database interaction|
406
| numExtension|c extension for numerical processing|
407
| libCV|small c library of algorithm from computer vision field|
408
| ImageViewer|image viewing for tomography|
409
410
411
412
413
414
415
*  run syscheck.py again to make sure you have everything.
416
417 1 Eric Hou
418 19 Amber Herold
*  Install the package in _each folder_ with commands
419 11 Amber Herold
like
420
these<pre>cd /your_download_area/Leginon-1.6-ALL/Leginon
421 1 Eric Hou
python setup.py install
422 11 Amber Herold
423
cd ../pyami
424
python setup.py install
425
cd ../pyScope
426
python setup.py install
427
cd ../sinedon
428
python setup.py install
429
cd ../numextension
430
python setup.py install
431
cd ../libcv
432
python setup.py install
433
cd ../ImageViewer
434
python setup.py install</pre>
435
436
437
438
439
440
441
442
443 16 Amber Herold
444 11 Amber Herold
h2.  python-site-package-path: where the installed python
445
packages went:
446
447
448
Python installer put the packages you installed to its site-packages directory. This
449
enable all users on the same computer to access them. The easiest way to find where your
450
installed package is called by python is to load a module from the package using interactive
451
python command lines like this:
452
453
454
455
456
*  Start python command line from shell
457
458
459
460
<pre>> python</pre>
461
462
463
464
*  Import a module from the package. Let's try sinedon here. All packages installed
465
through the above setup.py script should go to the same place.
466
467
468
469
<pre>python> import sinedon</pre>
470
471
472
473 1 Eric Hou
*  If the module is loaded successfully, call the module attribute __path__ (two
474 11 Amber Herold
underscrolls before "path" and two underscrolls after) will return the location of the
475
module it is loaded from
476
477
478
479
<pre>python> sinedon.__path__
480
481
RETURNS, For example,
482 21 Amber Herold
python> <module 'sinedon' from '/usr/lib/python2.5/site-packages/sinedon/__init__.pyc'> </pre>
483 11 Amber Herold
484
485
In this case, /usr/lib/python2.5/site-packages/ is your python-site-package-path.
486
If you go to that directory, you will find all the packages you just installed.
487
488
489
490
491
492
493
494
495 16 Amber Herold
496 11 Amber Herold
h2.  Configure leginon.cfg:
497
498
499
Leginon can be configured at three levels:
500
501
502
503
504
*  A skeleton (default) configuration file is available:
505
506 1 Eric Hou
507 11 Amber Herold
508 19 Amber Herold
<pre> _<link linkend="package_path">[python-site-package-path]</link>_/Leginon/config/default.cfg</pre>
509 11 Amber Herold
510
511
512
*  Configurations for all users
513 1 Eric Hou
514 11 Amber Herold
515
516 19 Amber Herold
<pre> _<link linkend="package_path">[python-site-package-path]</link>_/Leginon/config/leginon.cfg</pre>
517 11 Amber Herold
518
519
520 1 Eric Hou
Example of the configuration that you should set: (Here we assume your leginon
521 11 Amber Herold
database is named "dbemdata", and Leginon log into the database as a user named
522
"usr_object" with no password
523
set).<pre>[Images]
524 19 Amber Herold
path: _your_storage_disk_path_/leginon
525 11 Amber Herold
</pre>
526
527 1 Eric Hou
528 11 Amber Herold
*  Configurations for individual users that overrides the all-user configuration if
529
exists
530
531
532
533
<pre> ~/leginon.cfg</pre>
534
535 1 Eric Hou
536 11 Amber Herold
537
This configuration should include all the minimal configuration above and
538
additional configuration for individual user for
539
convenience<pre>[User]
540 19 Amber Herold
Fullname: _your_name_  #The name you entered in the leginon administration web page. Note that it is case sensitive</pre>
541 11 Amber Herold
542
543
544
545
546
547
548
549 16 Amber Herold
550 11 Amber Herold
h2.  Names used in the example installation
551
552
553
From this point on, you will need to enter database user names for different
554
configuration files and settings. These are NOT Leginon user names but what is required by
555
MySQL database interaction. Consistent names are essential. In the example shown here, the
556
leginon database is called "dbemdata", the project database is called "projectdata". The
557
MySQL user name are the same, "usr_object", and in this case without a password.
558
559
560
561
562 16 Amber Herold
563 11 Amber Herold
h2.  Configure sinedon.cfg:
564
565
566
Sinedon is designed to be able to interact with multiple databases. It can be
567
configured at global or user level.
568
569
570
571
572
*  An example configuration file is available at:
573
574
575
576
<pre>/your_download_area/Leginon-1.6-ALL/sinedon/examples/sinedon.cfg</pre>
577
578 1 Eric Hou
579 11 Amber Herold
580
*  Configurations for all users should be placed at the following path
581
582
583
584 19 Amber Herold
<pre>  _<link linkend="package_path">[python-site-package-path]</link>_/sinedon/sinedon.cfg</pre>
585 11 Amber Herold
586
587
588 1 Eric Hou
*  Modify host,db,user,passwd to what is used. Note that the user here is MySQL user
589 11 Amber Herold
Leginon uses to communicate with the database for all Leginon users. For Leginon that
590
uses Project database, set the following:
591
592
593
594
<pre>[global]
595 19 Amber Herold
host: _your_database_host_
596 11 Amber Herold
user: usr_object
597
passwd:
598 1 Eric Hou
599
[projectdata]
600
db: projectdata
601
602 11 Amber Herold
[leginondata]
603 1 Eric Hou
db: dbemdata
604
</pre>
605
606
607 11 Amber Herold
608 1 Eric Hou
*  Add database configuration if you intend to use grid-inserting robot. The Robot2
609
module uses the database to communicate to the robot. Applications that carries the
610
name "Robot" requires this to be set. In general, using the same database as the
611
general leginon database is fine.
612
613
614
615
<pre>[robot2]
616
db: dbemdata</pre>
617
618
619
620
621
622
623
624
625
626
627
628
629
h2. Database Server-side Installation
630
631
632
633
The following is for the computer that hosts the databases. This involves installing MySQL
634
server and creation/configuration of the leginon and project databases.
635
636
637
638
639
640
|_.Name:|_.Download site:|
641 25 Amber Herold
|MySQL-Server 5.0 or higher|<ulink url="http://www.mysql.com">http://www.mysql.com":http://repos.opensuse.org/science"
642
>http://repos.opensuse.org/science</ulink>*|scipy|python-scipy|
643 1 Eric Hou
644
645
646
647 25 Amber Herold
*SciPy may not build properly on some versions of SuSE due to an incompatible LAPACK
648
package that comes with SuSE. You can get scipy as well as a compatible LAPACK etc. from
649
http://repos.opensuse.org/science (need to specify your SuSE version and machine
650
etc.)
651 23 Amber Herold
652 25 Amber Herold
653
654
655
656
h2.  Optional supporting packages:
657
658
659
660
661
662
*  For using Matlab to explore holefinding algorithm (Matlab HoleFinder)
663
664
665
pymat: <ulink url="http://sourceforge.net/projects/pymat"
666
>http://sourceforge.net/projects/pymat</ulink>
667
668
669
670
671
672
673
674
675
676
h2. Install the packages you downloaded from svn depository
677
678
679
680
Here are the packages you need to install with python installer inside the
681
Leginon-1.6-ALL folder.
682
683
684
685
|MySQL-Client 5.0 or higher|"http://www.mysql.com":http://www.mysql.com|
686
687
688
689
690
691 11 Amber Herold
h2.  MySQL
692
693
694
695
696
697
*  Install MySQL-Server
698
699
700
701
702
*  Use your package installer (yum, zypper, YaST) if available.
703
704 1 Eric Hou
705 11 Amber Herold
OR
706
707
708 24 Amber Herold
*  Download the latest MySQL-server RPM for Linux from "www.mysql.com":http://www.mysql.com *  Install the MySQL-server rpm:
709 11 Amber Herold
710
711
<pre> rpm -Uvh MySQL-server-5.0.xx-y.i386.rpm</pre>
712
(substitute correct version numbers)
713
714
715
716
717
718
719
*  Install MySQL-Client
720
721
722
723
724
*  Use your package installer (yum, zypper, YaST) if available.
725
726
727
OR
728
729
730 24 Amber Herold
*  Download the latest MySQL-client RPM for Linux from "www.mysql.com":http://www.mysql.com *  Install the MySQL-client rpm:
731 11 Amber Herold
732
733
<pre> rpm -Uvh MySQL-client-5.0.xx-y.i386.rpm</pre>
734
(substitute correct version numbers)
735
736
737
738
739
740
741
*  MySQL configuration file is usually located in /usr/share/mysql. There are
742
several examples there:
743
744
745
746 22 Amber Herold
<pre>> ls /usr/share/mysql/my*
747 11 Amber Herold
/usr/share/mysql/my-huge.cnf
748
/usr/share/mysql/my-innodb-heavy-4G.cnf
749
/usr/share/mysql/my-large.cnf
750
/usr/share/mysql/my-medium.cnf
751
/usr/share/mysql/my-small.cnf
752 22 Amber Herold
> </pre>
753 11 Amber Herold
754
755
756
*  Configure my.cnf in /etc using my-huge.cnf as the template
757
758
759
760
761
* <pre>> cp /usr/share/mysql/my-huge.cnf /etc/my.cnf</pre>
762
763
764
765
*  Edit /etc/my.cnf to add or change query cache variables like these:
766
767
<pre>query_cache_type= 1
768
query_cache_size = 100M
769
query_cache_limit= 100M</pre>
770
771
772
773
774
775
776
777
*  start MySQL Server
778
779 22 Amber Herold
<pre> >/etc/init.d/mysqld start</pre>
780 11 Amber Herold
781
on some installation,
782
783 22 Amber Herold
<pre> >/etc/init.d/mysql start</pre>
784 11 Amber Herold
785
For future reference: start | stop | restart MySQL Server with similar
786
commands:
787
788 22 Amber Herold
<pre> >/etc/init.d/mysqld start
789
>/etc/init.d/mysqld stop
790
>/etc/init.d/mysqld restart</pre>
791 11 Amber Herold
792
If you want to start MySQL automatically at boot on SuSE
793
794 22 Amber Herold
<pre> SuSE >chkconfig mysql on
795 11 Amber Herold
</pre>
796
797
*  For future reference, the database location will be:
798
799
800
801 22 Amber Herold
<pre>> cd /var/lib/mysql
802 11 Amber Herold
Directory: /var/lib/mysql
803 22 Amber Herold
> ls
804 11 Amber Herold
yourdbserver.pid
805
ib_logfile0
806
mysql
807
mysql.sock
808
test
809 22 Amber Herold
> </pre>
810 11 Amber Herold
811
812
813
*  Create leginon database, here we call <link linkend="db_example_names"
814
>dbemdata</link>
815
816 22 Amber Herold
<pre> >mysqladmin create dbemdata</pre>
817 11 Amber Herold
818
819
820
*  Create project database, here we call <link linkend="db_example_names"
821
>projectdata</link> (optional)
822
823 22 Amber Herold
<pre> >mysqladmin create projectdata</pre>
824 11 Amber Herold
825
826
827
*  Connect to mysql db
828
829 22 Amber Herold
<pre>>mysql mysql
830 11 Amber Herold
831 22 Amber Herold
mysql> select user, password, host from user;
832 11 Amber Herold
+------+----------+-----------+
833
| user | password | host      |
834
+------+----------+-----------+
835
| root |          | localhost |
836
| root |          | host1     |
837
|      |          | host1     |
838
|      |          | localhost |
839
+------+----------+-----------+
840
4 rows in set (0.00 sec)</pre>
841
842
843
844
*  Create and grant privileges to a user called <link linkend="db_example_names"
845
>usr_object</link> for the databases on both the localhost and other hosts involved.
846
For example, use wild card '%' for all hosts. You may also set specific privilege to
847
the user. See MySQL Reference Manual for details
848
849 22 Amber Herold
<pre>mysql> create user usr_object@'localhost';
850
mysql> grant all privileges on dbemdata.* to usr_object@'localhost';
851
mysql> grant all privileges on projectdata.* to usr_object@'localhost';</pre>
852 11 Amber Herold
853
Similarly,
854 1 Eric Hou
855 22 Amber Herold
<pre>mysql> create usr_object@'%';
856
mysql> grant all privileges on dbemdata.* to usr_object@'%';
857
mysql> grant all privileges on projectdata.* to usr_object@'%';
858 11 Amber Herold
</pre>
859
860
*  Change Root passworld
861
862 22 Amber Herold
<pre>mysql> update user set password=password('_a passwd_') where user="root";
863 1 Eric Hou
Query OK, 2 rows affected (0.01 sec)
864 11 Amber Herold
Rows matched: 2  Changed: 2  Warnings: 0
865 1 Eric Hou
866 22 Amber Herold
mysql>^D or exit;</pre>
867 11 Amber Herold
From now on, you will need to specify the password to connect to the database as
868
root user like this:
869
870
<pre>>mysql -u root -p mysql</pre>
871
872 19 Amber Herold
_***If your password change does not take effect in your next
873 11 Amber Herold
database connection, you will need to flush the mysql pivileges cache as mysql root
874
and connecting with you old password (or no password in the above
875 19 Amber Herold
case):_
876 11 Amber Herold
877
<pre>>mysql -u root mysql</pre>
878 22 Amber Herold
<pre>mysql> flush privileges;
879
mysql>^D or exit;</pre>
880 11 Amber Herold
881
*  Check MySQL variables
882
883 22 Amber Herold
<pre>>mysql -u usr_object dbemdata
884 11 Amber Herold
885 22 Amber Herold
mysql> SHOW VARIABLES LIKE 'query%';
886 11 Amber Herold
+------------------------------+-----------+
887
| Variable_name                | Value     |
888
+------------------------------+-----------+
889
| ft_query_expansion_limit     | 20        |
890
| have_query_cache             | YES       |
891
| long_query_time              | 10        |
892
| query_alloc_block_size       | 8192      |
893 21 Amber Herold
| query_cache_limit            | 104857600 | <<---This should correspond to your change
894 11 Amber Herold
| query_cache_min_res_unit     | 4096      |
895 21 Amber Herold
| query_cache_size             | 104857600 | <<---This should correspond to your change
896
| query_cache_type             | ON        | <<---This should correspond to your change
897 11 Amber Herold
| query_cache_wlock_invalidate | OFF       |
898 1 Eric Hou
| query_prealloc_size          | 8192      |
899
+------------------------------+-----------+
900
10 rows in set (0.00 sec)
901
902
mysql> exit;</pre>
903 11 Amber Herold
904 22 Amber Herold
905 11 Amber Herold
906
*  Make sure MySQL is running
907
908
<pre>prompt:~> mysqlshow
909 1 Eric Hou
+--------------+
910 11 Amber Herold
| Databases    |
911
+--------------+
912
| mysql        |
913
| dbemdata     |
914
| projectdata  |
915
+--------------+</pre>
916 21 Amber Herold
917 1 Eric Hou
918 11 Amber Herold
919 22 Amber Herold
*  Or check with the following php script (if already installed)
920 11 Amber Herold
921
<pre><?
922
mysql_connect('your_host.your_institute.edu', 'usr_object', '','dbemdata');
923
echo mysql_stat();
924
?> </pre>
925
926
927
Output
928
929
<pre> Uptime: 1452562 Threads: 1 Questions: 618 Slow queries: 0 Opens: 117 Flush tables: 1 Open tables: 106 Queries per second avg: 0.000</pre>
930
931 1 Eric Hou
932
933
934
935 11 Amber Herold
936
937
938 1 Eric Hou
939
940
h2.  Web server set up and Installation
941
942
943 11 Amber Herold
944 1 Eric Hou
The following applies to the computer that will host the web-accessable image viewers ( a
945
part of dbem tools) and project manager (a part of project tools).
946
947 11 Amber Herold
948
949
h2. Differences between Linux flavors
950
951
952
953 1 Eric Hou
Different Linux flavors often put web server and mysql-related files in different
954
locations. This can be confusing. From experience, we found the equivalent on CentOS vs
955
SuSE. Here we list them for reference. If your system use a different naming and you are
956 11 Amber Herold
willing to share your experience, please send us the list. We will add it here:
957
958
959 1 Eric Hou
960 11 Amber Herold
|_.File or Command Head|_.CentOS|_.SuSE|
961 14 Amber Herold
| php.ini|/etc/| /etc/php5/apache2/|
962 11 Amber Herold
| httpd.conf|/etc/httpd/conf/|/etc/php5/apache2/|
963
| default document_root|/var/www/html/|/srv/www/htdocs/|
964
| apache start/stop/restart command head|/etc/init.d/httpd|/etc/init.d/apache2|
965
| mysql start/stop/restart command head|/etc/init.d/mysqld|/etc/init.d/mysql|
966
967
968
969 1 Eric Hou
970
971 11 Amber Herold
972
973 16 Amber Herold
h2. Prerequisites for dbemtools, projecttools, and mrctools extension
974 11 Amber Herold
975
976 1 Eric Hou
977 11 Amber Herold
The dbemtools are mostly php scripts that run at the web server. The followings are
978
required before installation of dbemtools and the mrc extension that handles mrc files to be
979
displayed. Some of these packages may be found on the SuSE Linux DVD or included in common
980
package repository. MySQL and the Apache Web Server can be downloaded from their respective
981
websites.
982 1 Eric Hou
983
984 11 Amber Herold
985 1 Eric Hou
|_.Name:|_.Download site:|_.yum package name|_.SuSE rpm name|
986 25 Amber Herold
| Apache|"www.apache.org":http://www.apache.org| httpd| apache2|
987
| php|"www.php.net":http://www.php.net| php| php|
988
| php-devel*|"www.php.ned/gd":http://rpmfind.net/linux/RPM/Development_Languages_PHP.html"
989
>rpmfind.net/linux/RPM/Development_Languages_PHP.html</ulink>| php-devel| php-devel|
990
| php-gd (including GD library, its development libraries and header *)|<ulink url="http://www.libgd.org (Use gd2)| php-gd, gd-devel| php-gd,gd-devel|
991 24 Amber Herold
| fftw3 library (including development libraries and header *)|"www.fftw.org":http://www.fftw.org (Use fftw3.x)| fftw3-devel| fftw3-devel|
992
993 15 Amber Herold
994 11 Amber Herold
995
996
* mrctools are compiled and added to php extension with php-devel package. Mrctools use
997
GD and FFTW3 that need to be compiled from their development libraries while the extension
998
is compiled. If GD and FFTW3 sources were downloaded and compiled directly on your computer,
999
these development files are included. If (as in most cases) GD and FFTW3 are installed from
1000
rpm, they are not included. An error message will appear when you attempt to compile
1001
mrctools. In this case, you will need separate download and installation of GD-devel and
1002 25 Amber Herold
FFTW3-devel. Search "A list of required CENTOS
1003
rpms and instruction":http://rpmfind.ned/linux/rpm2html/"
1004
>http://rpmfind.net/linux/rpm2html/</ulink> for GD-devel and FFTW3-devel for the rpm
1005 1 Eric Hou
distribution needed for your system.
1006 11 Amber Herold
1007
1008
1009 16 Amber Herold
1010 11 Amber Herold
1011 1 Eric Hou
h2. Installation tools
1012 11 Amber Herold
1013
1014
1015
Use the installation tools available for your linux distribution.
1016
1017
1018
1019
1020
*  For example, Use yum utility to install.
1021
1022
1023
*  For example, Use the SuSE Linux YaST2 utility or zypper (openSuSE 10.2 and above)
1024
to install.
1025
1026 24 Amber Herold
1027 25 Amber Herold
* <ulink url="http://emg.nysbc.org/bb/viewtopic.php?t=238 can be found at dbemtools bulletin board at
1028 11 Amber Herold
leginon.org.
1029 15 Amber Herold
1030 11 Amber Herold
1031
1032
1033
1034
1035
1036
1037 16 Amber Herold
1038 11 Amber Herold
h2.  PHP, PHP-devel, gd, and fftw3
1039
1040
1041
Install the packages using your installation tools if available. For example, to
1042
install gd as php extension you may use
1043
1044
1045
1046 17 Amber Herold
<pre>CentOS> yum install php-gd
1047 11 Amber Herold
SuSE10.2 and above> zypper install php-gd</pre>
1048
1049
1050
1051
1052
1053 16 Amber Herold
1054 11 Amber Herold
h2.  Configure php.ini
1055
1056
1057
1058
Edit the following two sections in php.ini (found as /etc/php.ini on CentOS and
1059 1 Eric Hou
/etc/php5/apache2/php.ini on SuSE) so that they look like the following:
1060 11 Amber Herold
1061
1062
1063
<pre>register_argc_argv = On</pre>
1064
1065
1066
1067
1068
<pre>short_open_tag = On</pre>
1069
1070
1071
1072
1073
<pre>max_execution_time = 300     ; Maximum execution time of each script, in seconds
1074
max_input_time = 300     ; Maximum amount of time each script may spend parsing request data
1075
memory_limit = 256M      ; Maximum amount of memory a script may consume (8MB)</pre>
1076
1077
1078
1079
You may want to increase max_input_time and memory_limit if the server is heavily used.
1080
At NRAMM, max_input_time=600 and memory_limit=4000M.
1081
1082
1083
1084
1085 16 Amber Herold
1086 11 Amber Herold
h2.  Apache Web Server
1087
1088
1089
1090
1091
1092
*  Install the Apache Web Server with the YaST or yum utility.
1093
1094
1095
*  Find "httpd.conf".
1096
1097
1098
This is /etc/httpd/conf/httpd.conf on CentOS and /etc/php5/apache2/httpd.conf on
1099
SuSE
1100
1101
1102
*  Edit the "httpd.conf" configuration file to look like the following:
1103
1104
1105
1106 22 Amber Herold
<pre><IfModule mod_dir.c>
1107 11 Amber Herold
DirectoryIndex index.htm index.shtm index.html index.shtml index.php
1108 22 Amber Herold
</IfModule></pre>
1109 11 Amber Herold
1110
1111 1 Eric Hou
(Note: It may be possible to edit httpd.conf in YaST2 as well.)
1112 11 Amber Herold
1113
1114
*  Restart the web server.
1115
1116
1117
1118 1 Eric Hou
<pre>apachectl restart
1119
or
1120
/etc/init.d/httpd restart     (ON CentOS)
1121
or
1122
/etc/init.d/apache2 restart   (ON SuSE)</pre>
1123
1124 21 Amber Herold
1125 11 Amber Herold
1126 22 Amber Herold
If you want to start the web server automatically at boot on SuSE
1127 11 Amber Herold
1128 1 Eric Hou
<pre> SuSE >chkconfig apache2 on
1129 11 Amber Herold
</pre>
1130
1131
1132
1133
1134
1135
1136
1137
1138
h2. Check php information
1139
1140
1141 16 Amber Herold
Create the following info.php in your web server document root directory (/var/www/html
1142 11 Amber Herold
on CentOS. /srv/www/htdocs on SuSE. You can find its location in httpd.conf mentioned above
1143
under the line starting
1144 1 Eric Hou
DocumentRoot).<pre><?php
1145
phpinfo();
1146 11 Amber Herold
?></pre>
1147
1148 16 Amber Herold
1149 11 Amber Herold
Visit this page at http://yourhost/info.php
1150
1151
1152
You will see comprehensive tables of php and apache information, including the location
1153
of the addition .ini files, extension, include path, and what extension is enabled.
1154
1155
1156 1 Eric Hou
Here is an example screen shot of the part of the info.php page that tells you where
1157 16 Amber Herold
php.ini and other configuration files are.
1158 11 Amber Herold
1159
1160
1161
!http://emg.nysbc.org/software/leginon/images/images/phpini.png!
1162 22 Amber Herold
1163 11 Amber Herold
1164
1165
1166
1167
1168
h2.  mrctools Installation
1169
1170
1171
1172
mrctools are installed as php extension and are required for displaying mrc files live
1173 16 Amber Herold
on the web browser.
1174 11 Amber Herold
1175
1176
1177
h2.  Install php-devel packages on the web server if missing:
1178
1179
1180
You can check whether php-devel is installed by
1181
typing<pre> >phpize</pre>
1182
1183 1 Eric Hou
1184 16 Amber Herold
Follow the instruction for your specific Linux distribution.
1185 11 Amber Herold
1186
1187
For example, SUSE users can use YaST or zypper to install them
1188
1189
1190
1191
1192 1 Eric Hou
1193 11 Amber Herold
h2.  php-GD/FFTW3-devel
1194
1195
1196 1 Eric Hou
1197 24 Amber Herold
Follow instructions from the download site. These may be included already. If not,
1198
mrctools installation will fail.
1199
1200
1201
1202
1203
1204
h2.  mrctools Installation
1205
1206
1207
1208
mrctools are installed from php devel directory. This is usually /usr/include/php/ext
1209
where you will find other php extension source such as gd. After the installation, mrc.so
1210
should be placed in php's extension directory (Look for "extension_dir" in
1211
http://your_host/<link linkend="install_infophp">info.php</link>
1212
1213
1214
1215
1216 25 Amber Herold
*  Download dbem tools from "which you
1217 11 Amber Herold
created earlier</link>. It should have a section looking like this (The version should
1218
correspond to what you've just installed):
1219
1220
1221 16 Amber Herold
1222 11 Amber Herold
!http://emg.nysbc.org/software/leginon/images/images/phpmrc.png!
1223 1 Eric Hou
1224
1225
1226 11 Amber Herold
If mrc is not listed, the extension did not get added at the right order.
1227
1228 1 Eric Hou
1229
1230
1231
1232
h2.  Alternative approach if mrc module does not show up in info.php output
1233
1234
1235
1236
* find in the <link linkend="install_infophp">info.php web page</link> the location
1237
of "additional .ini files parsed" in the first table (such as
1238
/etc/php.d/conf.d/*).
1239
1240
1241
*  Go to the directory and make a copy of any ini file to use as a template for
1242
mrc.ini
1243
1244
1245
1246
<pre>>cd [additional_ini_directory]
1247
>cp gd.ini mrc.ini</pre>
1248
1249 11 Amber Herold
1250
1251
*  Edit mrc.ini to the following
1252
1253
1254 16 Amber Herold
1255 11 Amber Herold
<pre>; comment out next line to disable mrc extension in php
1256
extension=mrc.so</pre>
1257
1258
1259
1260
*  Comment out mrc extension from php.ini (found in /etc/php.ini/ on a typical PHP
1261
installation)
1262
1263
1264
1265
<pre>;extension=mrc.so</pre>
1266
1267
1268
1269 22 Amber Herold
*  restart your webserver
1270
1271 11 Amber Herold
1272
1273
<pre>> /etc/init.d/httpd restart</pre>
1274
1275
1276 1 Eric Hou
1277 11 Amber Herold
1278
1279
1280
1281 1 Eric Hou
1282
1283
1284
h2.  Web Viewing Tools (dbemtools) Linux
1285
Installation
1286
1287 11 Amber Herold
1288 22 Amber Herold
1289 11 Amber Herold
php and java scripts for viewing images and Leginon information through the web
1290
server.
1291
1292
1293
1294
1295 25 Amber Herold
*  Download dbem tools from <ulink url="http://emg.nysbc.org/software/dbemtools"
1296
>http://emg.nysbc.org/software/dbemtools":http://emg.nysbc.org/software/mrctools"
1297
>http://emg.nysbc.org/software/mrctools</ulink>.
1298 16 Amber Herold
1299 11 Amber Herold
1300 25 Amber Herold
*  Follow the instruction at <ulink
1301
url="http://emg.nysbc.org/software/mrctools/mrc_so.php"
1302
>http://emg.nysbc.org/software/mrctools/mrc_so.php</ulink> for installation and
1303
testing. The next session about checking php information can also help debugging the
1304
installation
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
h2.  Check php information
1315
1316
1317
1318
Visit or refresh http://yourhost/info.php <link linkend="install_infophp .
1319
1320
1321 11 Amber Herold
*  Unpack the package in your [webdirectory]
1322
1323
1324 1 Eric Hou
1325 11 Amber Herold
<pre>>cd [webdirectory]			#/var/www/html in this example
1326
[webdirectory>tar zxvf dbem_1_5_1.tgz</pre>
1327
1328
1329
1330
*  Copy config.php.template to config.php and edit the latter by adding these MySQL
1331 21 Amber Herold
parameters:
1332 11 Amber Herold
1333
"config.php" should be located in /var/www/html/dbem_1_5_1/ on CentOS and
1334
/srv/www/htdocs/dbem_1_5_1/ on SuSE.
1335
1336 1 Eric Hou
1337 11 Amber Herold
1338
<pre><?
1339
// ---define dbem web tools base ---//
1340
define('BASE_URL',"/dbem_1_5_1/";
1341
// --- Leginon Viewer Configuration --- //
1342
1343
// --- Set your leginon MySQL database server parameters
1344
1345
$DB_HOST    = "[your database host]";
1346 1 Eric Hou
$DB_USER    = "<link linkend="db_example_names">usr_object</link>";
1347
$DB_PASS    = "";
1348
$DB        = "<link linkend="db_example_names">dbemdata</link>";
1349 11 Amber Herold
1350
// --- XML test dataset
1351
$XML_DATA = "test/viewerdata.xml";
1352 1 Eric Hou
1353
// --- Project database config
1354
1355
$PROJECT_URL = "/project_1_2";
1356
$PROJECT_DB_HOST = "[your database host]";
1357 11 Amber Herold
$PROJECT_DB_USER = "<link linkend="db_example_names">usr_object</link>";
1358
$PROJECT_DB_PASS = "";
1359 16 Amber Herold
$PROJECT_DB = "<link linkend="db_example_names">projectdata</link>";
1360 11 Amber Herold
</pre>
1361
1362
1363
1364
*  Test the set-up by visiting http://yourhost/dbem_1_5_1
1365
1366
1367
1368
1369
1370
1371 16 Amber Herold
1372 11 Amber Herold
1373
1374
h2.  Project management tools Linux Installation
1375
1376
1377
1378 16 Amber Herold
Leginon sessions can be catagorized into different projects. The projects are created
1379 1 Eric Hou
and viewed trhough a set of web tools. To access them, install the tools on the web server.
1380
The same project management tools are also used to create and track appion processing
1381 11 Amber Herold
databases. If you want to use our processing pipeline, Appion, in the future, you must
1382
install this.
1383 1 Eric Hou
1384 11 Amber Herold
1385
1386
h2. Prerequisites for project tools
1387
1388 1 Eric Hou
1389
1390
installed dbemtools and its prerequisites
1391 11 Amber Herold
1392
1393 1 Eric Hou
1394 24 Amber Herold
1395
1396
h2. project tools
1397
1398
1399
Current release: project 1.2.
1400
1401
1402 22 Amber Herold
1403 11 Amber Herold
1404 25 Amber Herold
*  Download project tools from ">cd [webdirectory]			#/var/www/html in this example
1405 11 Amber Herold
[webdirectory]>tar zxvf project_1_2.tgz</pre>
1406 1 Eric Hou
1407 17 Amber Herold
1408 11 Amber Herold
1409
*  Go to the uncompressed directory
1410 19 Amber Herold
project_1_2.<pre>> cd project_1_2</pre>
1411 11 Amber Herold
1412
1413
* Copy config.php.template to _config.php_
1414
1415
1416
1417
<pre>> cp config.php.template to config.php</pre>
1418 19 Amber Herold
1419 11 Amber Herold
1420
1421
* Edit _config.php_: You need to insert these MySQL
1422
parameters in project tool's config.php
1423 1 Eric Hou
1424 11 Amber Herold
1425
1426
<pre>// --- Project Configuration --- //
1427
1428
// --- Leginon database config
1429
$DBEM_PATH="../dbem_1_5_1/";
1430 1 Eric Hou
1431 11 Amber Herold
$DB_HOST  = "[your database host]";
1432
$DB_USER  = "usr_object";
1433
$DB_PASS  = "";
1434
$DB   = "<link linkend="db_example_names">dbemdata</link>";
1435 1 Eric Hou
1436 11 Amber Herold
// --- Project database config
1437
$PROJECT_DB_HOST = "[your database host]";
1438
$PROJECT_DB_USER = "<link linkend="db_example_names">usr_object</link>";
1439
$PROJECT_DB_PASS = "";
1440
$PROJECT_DB = "<link linkend="db_example_names">projectdata</link>";
1441
</pre>
1442
1443
1444
1445
*  Test the set-up by visiting http://yourhost/project_1_2
1446
1447
1448
1449
1450
1451
1452
1453 16 Amber Herold
1454 11 Amber Herold
1455
1456
1457
h2.  Other Tools
1458
1459
1460 16 Amber Herold
This section includes tools that may help with Leginon development.
1461 11 Amber Herold
1462
1463 1 Eric Hou
1464
h2.  phpMyAdmin
1465
1466
1467 24 Amber Herold
1468
1469
1470 25 Amber Herold
*  Download phpMyAdmin from Yast2 or <ulink url="http://www.phpmyadmin.net"
1471
>www.phpmyadmin.net":http://emg.nysbc.org/software/dbemtools"
1472
>http://emg.nysbc.org/software/dbemtools</ulink>.
1473 23 Amber Herold
1474 22 Amber Herold
1475 25 Amber Herold
*  Unpack the package in your [webdirectory]
1476 11 Amber Herold
1477
1478
1479
<pre>>rpm -qa |grep -i phpmyadmin </pre>
1480 22 Amber Herold
1481 1 Eric Hou
1482
1483
*  Install from tarball:
1484
1485
1486
1487
<pre>> cd [webdirectory]
1488
[webdirectory]>tar zxvf phpMyAdmin-2.x.tar.gz</pre>
1489 11 Amber Herold
1490
1491 1 Eric Hou
1492
or Install with yum if availabe:
1493
1494
1495 11 Amber Herold
1496 1 Eric Hou
<pre>> yum install phpMyAdmin</pre>
1497
1498
1499
1500
*  Create phpMyAdmin configuration file called config.inc.php from a
1501
sample.
1502
1503
1504
1505
<pre>[webdirectory]>cd phpMyAdmin
1506
(if you install with yum, the program directory is /usr/share/phpMyAdmin)
1507
1508
> cp config.sample.inc.php config.inc.php
1509 11 Amber Herold
</pre>
1510 1 Eric Hou
1511
1512
1513
*  Make sure these are set correctly in _config.inc.php_. Note that these are for MySQL which we set earlier.
1514
You can set specific user, but it is safer just leave it blank.
1515
1516
1517
1518
<pre>
1519
$cfg['Servers'][$i]['host']            = '[your host]';
1520
$cfg['Servers'][$i]['port']            = '';
1521
$cfg['Servers'][$i]['socket']          = '';
1522
$cfg['Servers'][$i]['connect_type']    = 'tcp';
1523 11 Amber Herold
$cfg['Servers'][$i]['extension']       = 'mysql';
1524
$cfg['Servers'][$i]['compress']        = FALSE;
1525
$cfg['Servers'][$i]['controluser']     = '';
1526
$cfg['Servers'][$i]['auth_type']     = 'http';
1527 1 Eric Hou
$cfg['Servers'][$i]['user']          = '';
1528
$cfg['Servers'][$i]['password']      = '';
1529
</pre>
1530
1531 11 Amber Herold
1532
1533 1 Eric Hou
*  For additional security, you may restrict mysql root access
1534 11 Amber Herold
1535
1536
1537
<pre>
1538
$cfg['Servers'][$i]['AllowRoot']     = FALSE;
1539
</pre>
1540
1541
1542
1543
*  If you want to access phpMyAdmin from another computer, you can add it to its
1544 1 Eric Hou
web access configuration file found as /etc/httpd/conf.d/phpMyAdmin.conf in a
1545 11 Amber Herold
typical installation
1546 1 Eric Hou
1547 11 Amber Herold
1548
1549
<pre>
1550
<Directory /usr/share/phpMyAdmin/>
1551 1 Eric Hou
order deny,allow
1552 11 Amber Herold
deny from all
1553
allow from 127.0.0.1
1554
allow from YOUR_IP_ADDRESS
1555
</Directory>
1556 1 Eric Hou
</pre>
1557 11 Amber Herold
1558 21 Amber Herold
1559 11 Amber Herold
1560
1561
1562
1563 21 Amber Herold
1564 11 Amber Herold
1565
1566
1567
1568
1569
1570
1571
h2.  Windows installation
1572
1573
1574
1575
1576
h2.  Processing-side Leginon Windows Installation
1577
1578 16 Amber Herold
1579 11 Amber Herold
1580
1581
h2.  Install Python and Support Packages (Note that python 2.5 must be used):
1582
1583 16 Amber Herold
1584 11 Amber Herold
1585
This list does not include pyton XML module because it is included in the python
1586
package for window.
1587
1588 16 Amber Herold
1589 11 Amber Herold
1590 23 Amber Herold
|_.Name:|_.Download site:|
1591 25 Amber Herold
|Python 2.5*|"http://www.python.org":http://www.python.org|
1592
|Python for Windows extension (pywin32)|"http://www.wxpython.org":http://sourceforge.net/projects/pywin32/"
1593
>http://sourceforge.net/projects/pywin32/</ulink>|
1594
|wxPython 2.5.2.8 or newer|<ulink url="http://www.wxpython.org|
1595
|MySQL Python client 1.2 or newer|"http://www.scipy.org":http://sourceforge.net/projects/mysql-python"
1596
>http://sourceforge.net/projects/mysql-python</ulink>|
1597
|Python Imaging Library (PIL) 1.1.4 or newer|<ulink url="http://www.pythonware.com/products/pil/"
1598
>http://www.pythonware.com/products/pil/</ulink>|
1599
|NumPy 1.0b5 (tested, others may work)|<ulink url="http://www.scipy.org|
1600
|SciPy 0.5.1 or newer|"http://www.scipy.org":http://www.scipy.org|
1601
|Tortoise SVN client|"
1602
<colspec colnum="1"/>
1603
<colspec colnum="2"/>
1604 24 Amber Herold
1605
1606 25 Amber Herold
|_.Name:|_.Purpose:|
1607
<entry> Leginon|modular TEM image acquisition|
1608
| pyami|general functions|
1609
| sinedon|Leginon/database interaction|
1610
| pyScope|microscope control and monitoring|
1611
| ImageViewer|image viewing for tomography|
1612 11 Amber Herold
1613
1614
1615 23 Amber Herold
1616 25 Amber Herold
Because numextension and libCV requires extra compilers, we have created window
1617
installer for them for python 2.5 and made them available through <ulink
1618
url="http://www.leginon.org/">http://www.leginon.org/":http://tortoisesvn.tigris.org"
1619
>http://tortoisesvn.tigris.org</ulink></entry>
1620
|
1621 11 Amber Herold
1622
1623
1624 23 Amber Herold
1625 25 Amber Herold
*Python 2.5 is the only python version that we have compiled numExtension. libCV and
1626
comarray in. Therefore no other python version works for now.
1627 1 Eric Hou
1628 23 Amber Herold
1629 25 Amber Herold
Execute the installer file and follow the directions.
1630 24 Amber Herold
1631
1632
1633
1634
1635 25 Amber Herold
h2. Packages required from NRAMM
1636 24 Amber Herold
1637
1638
1639 25 Amber Herold
Here are the packages you need to install with python installer
1640 24 Amber Herold
1641 25 Amber Herold
1642 14 Amber Herold
1643 13 Amber Herold
|_.Downloadfile Name|_.Installed Python Package File|_.Purpose:|
1644 11 Amber Herold
|NumExtension-1.2.0.win32-py2.5.exe|numextension.pyd|c extension for numerical processing|
1645
|libCV-0.2.win32-py2.5.exe|libCV.pyd|small c library of algorithm from computer vision field|
1646
1647 1 Eric Hou
1648 11 Amber Herold
1649
1650
1651 16 Amber Herold
1652 11 Amber Herold
1653
h2. Check out SVN Source Files from the depository
1654
1655
1656
1657
Use your mouse to do the following
1658
* Create Leginon-1.6-ALL directory somewhere at your convenience
1659
1660
1661
* Change directory into Leginon-1.6-ALL
1662 1 Eric Hou
1663 11 Amber Herold
1664 17 Amber Herold
* Right-click the mouse botton in this directory window and select Tortoise svn
1665 1 Eric Hou
Checkout in the menu:!http://emg.nysbc.org/software/leginon/images/images/svnmenu.png!
1666 11 Amber Herold
1667
1668 17 Amber Herold
* Set up svn checkout window like this for <link linkend="windows_packages">EACH
1669 11 Amber Herold
of the leginon packages</link>.!http://emg.nysbc.org/software/leginon/images/images/svnco.png!
1670
1671 1 Eric Hou
1672 11 Amber Herold
1673
1674
1675
1676 16 Amber Herold
1677 11 Amber Herold
1678
h2. Install the packages you downloaded from NRAMM svn depository
1679 1 Eric Hou
1680 11 Amber Herold
1681
1682
1683
1684
* Start a command line Window from Start Menu
1685
1686
1687
* Install the package in each folder with commands such as
1688
<pre>cd Your_Download_Place\Leginon-1.6-ALL\leginon
1689
c:\\python25\python.exe setup.py install</pre>
1690
1691
1692
1693
1694
1695
1696
1697 16 Amber Herold
1698 11 Amber Herold
1699
h2. Download the two Window Installer Files from Leginon website
1700
1701
1702 24 Amber Herold
1703 15 Amber Herold
"http://www.leginon.org/":http://www.leginon.org/ h2. Install individual packages
1704 11 Amber Herold
1705
1706
1707
Excute the installer files and follow the instruction.
1708
1709
1710
1711 16 Amber Herold
1712 11 Amber Herold
1713
h2.  Mapping Drives:
1714
1715
1716
1717
If you plan to run Leginon directly on the Windows machine, such as in <link
1718
linkend="config_C">configuration C</link>, and your data files are served through a
1719
Samba server on a Linux machine, you will need to map the network drive. For example, if
1720
your Samba server has a hostname your_smbserver, and you have set up a share called
1721
[your_share_point] which points to /your_data_path/ and leginon data will be saved under a
1722 1 Eric Hou
folder in /your_data_path/leginon/
1723 11 Amber Herold
1724
1725
1726
1727 1 Eric Hou
*  Start, My Computer
1728 11 Amber Herold
1729
1730 1 Eric Hou
*  Tools menu, Map network drive
1731 11 Amber Herold
1732
1733
*  Use an unmapped drive such as Z:
1734 1 Eric Hou
1735 17 Amber Herold
Enter shared path in Windows format
1736 11 Amber Herold
as<pre>\\your_smbserver\your_share_point</pre>
1737
1738
1739 17 Amber Herold
*  Add the drive and the Linux path to leginon.cfg on the Windows machine
1740 11 Amber Herold
as<pre>[Drive Mapping]
1741
Z:/your_data_path</pre>
1742
1743
1744 17 Amber Herold
*  Add image path to leginon.cfg on the Windows machine in Linux format
1745 11 Amber Herold
as<pre>[Images]
1746
path:/your_data_path/leginon</pre>
1747
1748
1749
1750
1751
1752
1753
1754 16 Amber Herold
1755 11 Amber Herold
1756
h2.  Configure leginon.cfg:
1757
1758
1759
Follow instruction in "<link linkend="leginon_cfg">Configure leginon.cfg</link>" in
1760
the section for Linux installation but note the location of the configuration files
1761
follows. In addition, if the storage disk is mapped onto the Windows PC as drive Z, this
1762
mapping should be included in leginon.cfg. See above.
1763
1764
1765
1766
1767
*  Configurations for all users
1768
1769
1770 22 Amber Herold
1771 11 Amber Herold
<pre> <Python directory>\Lib\site-packages\Leginon\config\leginon.cfg</pre>
1772
1773
1774
Example:
1775
<pre> C:\Python25\Lib\site-packages\Leginon\config\leginon.cfg</pre>
1776
1777
1778
*  Configurations for individual users
1779
1780
1781 22 Amber Herold
1782 11 Amber Herold
<pre> <Home directory>\leginon.cfg</pre>
1783
1784
1785
Example:
1786
<pre> C:\Documents and Settings\Leginon User\leginon.cfg </pre>
1787
1788
1789
*  A skeleton (default) configuration file is available:
1790
1791
1792
1793
<pre> C:\Python25\Lib\site-packages\Leginon\config\default.cfg</pre>
1794
1795
1796
1797
1798
1799
1800
1801 1 Eric Hou
1802 16 Amber Herold
1803 11 Amber Herold
1804
h2.  Configure sinedon.cfg:
1805
1806
1807
Sinedon is designed to be able to interact with multiple databases.
1808
1809
Follow instruction in "<link linkend="sinedon_cfg">Configure sinedon.cfg</link>" in
1810
the section for Linux installation but note the location of the configuration files
1811
follows.
1812
1813
1814
1815
1816
* For all users, put sinedon.cfg with the installed package
1817
as<pre>C:\Python25\Lib\site-packages\sinedon\sinedon.cfg</pre>
1818
1819
1820 22 Amber Herold
*  Your home directory on
1821 11 Amber Herold
Windows:<pre>C:\Documents and Settings\your_name></pre>
1822
1823
1824
* the skeleton sinedon configuration file
1825
is<pre>C:\Python25\Lib\site-packages\sinedon\examples\sinedon.cfg</pre>
1826
1827
1828
1829
1830
1831
1832
1833 16 Amber Herold
1834 11 Amber Herold
1835
h2.  Create Leginon and Leginon Client shortcut in Start menu menu under Leginon
1836
1837 1 Eric Hou
1838 11 Amber Herold
1839
This instruction refers to Windows XP
1840
1841
1842
1843
1844 1 Eric Hou
* Go to C:\Documents and Settings\All Users\Start Menu\Programs\ and create a new
1845 11 Amber Herold
folder named Leginon.
1846
1847
1848 17 Amber Herold
* In another window, go
1849 11 Amber Herold
to<pre>C:\Python25\Lib\site-packages\Leginon</pre>
1850
1851 1 Eric Hou
1852 11 Amber Herold
* Create a shortcut from start-leginon.py as Leginon and a shortcut from
1853
launcher.py as Leginon Client.
1854
1855
1856 17 Amber Herold
* Move the two shortcuts into
1857 11 Amber Herold
<pre>C:\Documents and Settings\All Users\Start Menu\Programs\Leginon</pre>
1858
1859
1860
1861
1862
1863
1864
1865 16 Amber Herold
1866 11 Amber Herold
1867 1 Eric Hou
h2.  Additional Software (Optional):
1868 11 Amber Herold
1869
1870
TightVNC (http://www.tightvnc.com)
1871
1872
1873
1874
1875
1876
1877
1878 16 Amber Herold
1879 11 Amber Herold
1880
h2. Database server Windows Installation
1881
1882
1883 16 Amber Herold
1884 11 Amber Herold
1885
h2. We do not do this at NRAMM. Please follow the instruction in Linux installation and
1886
modify it for Windows at your own risk.
1887
1888
1889
1890
For a good Windows specific instruction for general PHP configuration with MySQL for
1891
Apache 2 in Windows, try http://www.artfulsoftware.com/php_mysql_win.html.
1892
1893
1894
1895
1896
1897
1898
1899 16 Amber Herold
1900 1 Eric Hou
1901 11 Amber Herold
h2.  Additional installation on the microscope computer:
1902
1903
1904
1905
The full leginon and its supporting packages need to be installed on the Windows computer
1906
controlling the microscope. Additional programs are required for communication with the CCD
1907
and extra functions for Tecnai scripting not supported as default by FEI Tecnai
1908
microscopes
1909
1910 16 Amber Herold
1911 11 Amber Herold
1912
h2.  Required supporting programs for the CCD camera from camera makers
1913
1914
1915
1916
Install and register the following programs for CCD cameras from the two makes:
1917 1 Eric Hou
1918
1919 14 Amber Herold
1920 11 Amber Herold
|_.Camera Make:|_.File:|
1921
| Gatan|TecnaiCCD.dll|
1922
| Tietz|CAMC4.exe*|
1923
1924
1925
1926
1927
* We have experienced slowness of the CAMC4.exe comes with later version Tecnai TUI/TIA.
1928
Replacing it with an earlier version of CAMC4.exe resolved the problem.
1929
1930
1931
1932 16 Amber Herold
1933 11 Amber Herold
1934
h2. Additional Package required from NRAMM for Gatan camera or camera that uses TIA
1935
1936
1937
1938 1 Eric Hou
For Gatan Camera or FEI Eagle Camera that uses TIA, comarray package need to be install
1939 11 Amber Herold
with python
1940 1 Eric Hou
1941
1942
1943 11 Amber Herold
|_.SVN Package Name|_.Installed Python Package Name|_.Reason for update:|
1944
|comarray|comarray|com module output conversion to array|
1945
1946
1947
1948
1949
1950
1951
* <link linkend="window_svn">Check out the comarray package</link> from svn
1952
depository with your SVN client
1953 17 Amber Herold
1954 11 Amber Herold
<pre>cd Leginon-1.6-ALL
1955
svn co http://emg.nysbc.org/svn/comarray/branches/leg1.6 comarray</pre>
1956
1957 1 Eric Hou
* Start a command line Window from Start Menu
1958 11 Amber Herold
1959
1960
* Install the package with commands such as
1961
<pre>cd Your_Download_Place\Leginon-1.6-ALL\comarray
1962
c:\\python25\python.exe setup.py install</pre>
1963
1964
1965
1966
1967
1968
1969
1970 16 Amber Herold
1971 11 Amber Herold
1972
h2.  Supporting programs for film exposure
1973
1974
1975
1976
Install the following if you need film exposure on FEI Tecnai TEM through Leginon,
1977 1 Eric Hou
available through FEI. Please contact Max Otten: <email>mto@feico.com</email> and request
1978
for adaexp.exe that works with your version of Tecnai user interface program.
1979 11 Amber Herold
1980 14 Amber Herold
1981 13 Amber Herold
1982 11 Amber Herold
|_.Name:|_.File:|
1983
| exposure adaptor|adaexp.exe|
1984
1985
1986
1987
1988
1989 16 Amber Herold
1990 11 Amber Herold
1991
h2.  Register adaexep.exe
1992
1993
1994
1995
*  From the command prompt:
1996
1997
<pre>adaexp.exe /regserver</pre>
1998
1999
*  From the program start menu under pyScope or
2000
C:\Python25\Lib\site-packages\pyScope\
2001
2002
<pre>updatecom.py</pre>
2003
2004
2005
2006
2007
2008
2009 16 Amber Herold
2010 11 Amber Herold
2011
h2. Modify instruments.cfg:
2012 1 Eric Hou
2013 11 Amber Herold
2014
2015
Sinedon now have full control of database interaction, therefore, the database
2016
configuration in leginon.cfg is no longer needed.
2017
2018
2019
2020
2021
* Modify the file instruments.cfg in the installed pyScope directory to that for
2022
your microscope and camera. For example, if your microscope uses Tecnai Scripting
2023
Interface and you have a Gatan camera that you interface through
2024
DigitalMicrograph:
2025
2026
2027
2028
<pre>[tem]
2029
class: tecnai.Tecnai
2030
[camera]
2031
class: gatan.Gatan</pre>
2032 1 Eric Hou
2033 11 Amber Herold
2034
2035
The file contains other examples of microscope and camera drivers that we
2036
distribute from NRAMM.
2037
2038
2039
2040
2041
2042
2043
2044 16 Amber Herold
2045 11 Amber Herold
2046
h2. Run updatecom.py
2047
2048
2049
2050
From a command line window:
2051
2052
2053
2054
<pre>cd C:\python25\Lib\Site-Packages\pyScope
2055
C:\python25\python.exe updatecom.py</pre>
2056
2057
2058
2059
This should generate a few files, including tecnaicom.py, gatancom.py and tietzcom.py,
2060
in the same directory.
2061
2062
2063
2064 16 Amber Herold
2065 11 Amber Herold
2066
h2. Additional setup on Tietz PXL camera (optional)
2067
2068
2069
2070
The actual Tietz PXL camera dimension is slightly larger than 2048 x 2048. If you'd like
2071
to take the images using the maximum of 2048 x 2048 pixels, you may modify the function that
2072
get camera dimension in tietz.py of pyScope package.
2073 1 Eric Hou
2074 11 Amber Herold
2075
*  Go to C:\Python25\Lib\site-packages\pyScope\
2076
2077
2078
*  Edit tietz.py with a plain text editor
2079
2080
2081
*  Find the lines:
2082
2083
2084
2085
<pre> def getCameraSize(self):
2086
# {'type': dict, 'values': {'x': {'type': int}, 'y': {'type': int}}}}
2087 1 Eric Hou
x = self._getParameterValue('cpTotalDimensionX')
2088 11 Amber Herold
y = self._getParameterValue('cpTotalDimensionY')
2089
return {'x': x, 'y': y}
2090
</pre>
2091
2092 1 Eric Hou
2093 11 Amber Herold
2094
*  Change the last line to:
2095
2096
2097
2098
<pre>    return {'x': 2048, 'y': 2048}</pre>
2099
2100
2101
2102 1 Eric Hou
2103 11 Amber Herold
2104
2105 16 Amber Herold
2106 11 Amber Herold
2107
h2.  Additional Software (Optional):
2108
2109
2110
TightVNC (http://www.tightvnc.com) if you get tired of going into the microscope room
2111 1 Eric Hou
just to open the column valves.
2112 11 Amber Herold
2113
2114
2115
2116
2117
2118
2119 16 Amber Herold
2120 11 Amber Herold
2121
h2.  What is next
2122 1 Eric Hou
2123 11 Amber Herold
2124 16 Amber Herold
2125 11 Amber Herold
2126
h2.  Perform Administration Setup
2127 1 Eric Hou
2128 11 Amber Herold
2129
See Next chapter on <link linkend="addusers">Leginon Administration Tools</link>.
2130
2131
2132 1 Eric Hou
2133 11 Amber Herold
2134 16 Amber Herold
2135 11 Amber Herold
2136
h2.  Create a test project
2137
2138
2139
See the chapter on <link linkend="project">Project Management Tools</link>.
2140
2141
2142 1 Eric Hou
2143 16 Amber Herold
2144 11 Amber Herold
2145
h2.  Perform Microscope Setup and Test run
2146
2147
2148
See the chapter on <link linkend="runleg_chapter">Start Leginon</link>.
2149
2150
2151
2152
2153 1 Eric Hou
2154 16 Amber Herold
2155 11 Amber Herold
2156
h2.  Backup Practices
2157
2158
2159 16 Amber Herold
2160 11 Amber Herold
2161
h2.  Ideal setup:
2162
2163
2164 16 Amber Herold
2165 11 Amber Herold
2166 1 Eric Hou
h2.  Regular computer backup
2167 11 Amber Herold
2168
2169
2170
To protect your program installation.
2171
2172
2173
2174 16 Amber Herold
2175 11 Amber Herold
2176 1 Eric Hou
h2.  Full backup of the databases every night
2177 11 Amber Herold
2178
2179
2180
The database is the metadata generated by Leginon named "dbemdata" in the installation
2181
example.
2182
2183
2184
2185 16 Amber Herold
2186 11 Amber Herold
2187
h2. Full/Incremental/differential backup of the images
2188
2189 1 Eric Hou
2190 11 Amber Herold
2191
The image files located in [your storage disk] should be backup in full at least once
2192
a month, and nightly in differential or incremental mode.
2193
2194
2195
2196 1 Eric Hou
2197 11 Amber Herold
2198 16 Amber Herold
2199 11 Amber Herold
2200
h2.  Current NRAMM setup:
2201
2202
2203
2204
Because our tape backup system is not on-site and the data have to transfer over slow
2205
network to perform the task, we have not been able to backup in the ideal setting. Here is
2206
what we currently do:
2207
2208 16 Amber Herold
2209 11 Amber Herold
2210
h2.  Regular computer backup
2211
2212
2213
2214
TSRI Research Computing provides regular back up service that does full backup of the
2215
computers every four weeks and incremental backup every night. The full backup expires in
2216
4 months and the incremental backup one month. We do this for the computers running
2217
Leginon main program, the database server, as well as the web server.
2218
2219
2220
2221 16 Amber Herold
2222 11 Amber Herold
2223
h2.  Regular backup &amp; tar of the databases every night
2224
2225
2226
2227
On top of the TSRI backup, we also tar the database everynight and store it on the
2228 16 Amber Herold
institutional tape library.
2229 11 Amber Herold
2230
2231
2232
2233
2234
h2.  rsync to a tape library of the image storage disk
2235
2236
2237
2238
As our image storage disks total 15T byte, we currently use rsync function to update a
2239
copy on our institutional tape library. It creates new files, updates old files, but does
2240
not remove old files on the tape if it is removed on the storage disk. We do not think
2241
this is the best solution since it is not possible to retrieve old files that have been
2242 1 Eric Hou
modified on a later time.