Using Redux to serve images on myamiweb » History » Version 56
Amber Herold, 12/16/2013 03:22 PM
1 | 1 | Anchi Cheng | h1. Using Redux to serve images on myamiweb |
---|---|---|---|
2 | |||
3 | 52 | Amber Herold | These instructions cover installation of Leginon and Appion web tools for viewing images and performing image processing through the web server using Redux as an image server. |
4 | 21 | Amber Herold | Redux is new for CentOS 6 (PHP 5.3). |
5 | |||
6 | 1 | Anchi Cheng | h2. Installation |
7 | |||
8 | 33 | Amber Herold | * Install python filesystem abstraction for redux caching: |
9 | 1 | Anchi Cheng | <pre> |
10 | sudo easy_install fs |
||
11 | </pre> |
||
12 | 33 | Amber Herold | * Install Myami packages |
13 | <pre> |
||
14 | cd myami |
||
15 | ./pysetup.sh install |
||
16 | </pre> |
||
17 | 1 | Anchi Cheng | |
18 | 47 | Amber Herold | h2. Copy the myami/myamiweb directory to your Apache web directory |
19 | |||
20 | Example: |
||
21 | <pre> |
||
22 | cd myami |
||
23 | |||
24 | #CentOS example |
||
25 | sudo cp -vr myamiweb /var/www/html/ |
||
26 | |||
27 | #this is temporary for setup, revert to 755 when finished with this page |
||
28 | sudo chmod 777 /var/www/html/myamiweb |
||
29 | |||
30 | #if you have SELinux enabled this command will help |
||
31 | sudo chcon -R --type=httpd_sys_content_t /var/www/html |
||
32 | </pre> |
||
33 | |||
34 | |||
35 | |||
36 | |||
37 | 34 | Amber Herold | h2. Configuration |
38 | 1 | Anchi Cheng | |
39 | 49 | Amber Herold | h3. Configure Redux |
40 | 17 | Anchi Cheng | |
41 | 1 | Anchi Cheng | <pre> |
42 | 17 | Anchi Cheng | > cd /YourMyamDownload/redux |
43 | 16 | Anchi Cheng | > cp redux.cfg.template redux.cfg |
44 | 1 | Anchi Cheng | </pre> |
45 | 44 | Amber Herold | |
46 | 17 | Anchi Cheng | * You can also copy it to /etc/myami/redux.cfg if you prefer. |
47 | 1 | Anchi Cheng | * Set a writable log path by the user starting it. In this example, we start redux as root and save the log in /var/log/redux.log |
48 | 34 | Amber Herold | * Edit the redux.cfg file as follows: |
49 | 44 | Amber Herold | |
50 | 31 | Anchi Cheng | <pre> |
51 | [log] |
||
52 | file: /var/log/redux.log |
||
53 | </pre> |
||
54 | 44 | Amber Herold | |
55 | 43 | Amber Herold | * Turn on redux caching if desired: |
56 | 44 | Amber Herold | |
57 | 1 | Anchi Cheng | <pre> |
58 | 16 | Anchi Cheng | [cache] |
59 | enable: yes |
||
60 | path: /var/cache/myami/redux |
||
61 | disksize: 500 |
||
62 | 40 | Amber Herold | memsize: 500 |
63 | 1 | Anchi Cheng | </pre> |
64 | 44 | Amber Herold | |
65 | ** *You need to make sure the cache path exists and writable by the user that starts the redux server (reduxd)* |
||
66 | 38 | Amber Herold | ** Input the desired disk_cache_path, disk_cache_size, and mem_cache_size in the next few lines |
67 | ** Create the disk_cache_path before running redux if cache will be used |
||
68 | 1 | Anchi Cheng | |
69 | 49 | Amber Herold | h3. Configure Web Interface (myamiweb) |
70 | |||
71 | There is a setup wizard available to help you set the configuration parameters for your installation. If you prefer not to use the wizard, there are instructions for manually editing the configuration file. If this is your first time creating the web tool configuration file, we recommend using the setup wizard. |
||
72 | |||
73 | |||
74 | +Configuration using the setup wizard+ |
||
75 | |||
76 | The setup wizard will check your database connection, create required database tables, and perform default data initialization. |
||
77 | |||
78 | * Run the online setup wizard by visiting http://yourhost/myamiweb/setup or http://localhost/myamiweb/setup to create the myami website’s config file. |
||
79 | |
||
80 | *Tips:* |
||
81 | # You need to know your database setup before you start. If you have been using the parameters in this instruction, here is a [[Explanation of Sample Names|summary]]. |
||
82 | # To discover what the Apache user is: |
||
83 | <pre> |
||
84 | sudo egrep -iw --color=auto '^(user|group)' /etc/httpd/conf/httpd.conf |
||
85 | </pre> |
||
86 | # You also need to decide whether you would like to enable the [[appion:What does User Authentication do to myamiweb|user management system]]. |
||
87 | |||
88 | +Manual configuration instructions (Advanced User)+ |
||
89 | |||
90 | Go to [[appion:Install the Web Interface Advanced]] for the advanced configuration. |
||
91 | |||
92 | h3. Revert permissions |
||
93 | |||
94 | <pre> |
||
95 | sudo chmod 755 /var/www/html/myamiweb |
||
96 | </pre> |
||
97 | 1 | Anchi Cheng | |
98 | 42 | Amber Herold | |
99 | 51 | Amber Herold | h2. Test Redux Installation |
100 | 1 | Anchi Cheng | |
101 | # Using Redux to do simple input output (simple client with no reduxd server needed):<pre>redux --filename=test.jpg --oformat=PNG > test.png</pre>Make sure the resulting test.png is in fact an image and not an error message. |
||
102 | # Run the server: run the following on a command line:<pre>reduxd</pre>leave it running for the following... |
||
103 | # Test command line client connecting to the server<pre>redux --server_host=localhost --filename=test.jpg --oformat=PNG > test.png</pre>Note: The input file name is from the perspective of the reduxd server, so be sure to give it an absolute path (unlike the first test, which was not accessing the server) |
||
104 | 31 | Anchi Cheng | # Check the redux.log created as configured in redux.cfg. This contains the port opened by redux server that will allow myamiweb to connect to. |
105 | 2 | Anchi Cheng | |
106 | 53 | Amber Herold | h2. Setup fftw to use wisdom |
107 | 25 | Anchi Cheng | |
108 | 27 | Anchi Cheng | *This applies only if you see the messge "*** Using custom copy of fftw3"* in the previous redux test. |
109 | |||
110 | 29 | Anchi Cheng | fftw runs much faster on odd image dimension if a wisdom is saved. It is best to create the wisdom ahead of time using the stand alone script pyami/fft/fftwsetup.py. It will store the wisdom you create in your home directory with a name like 'fftw3-wisdom-hostname'. If you only start reduxd as root and fftwsetup.py is run as root, this is enough. |
111 | 1 | Anchi Cheng | |
112 | 47 | Amber Herold | * Redux needs write permission at its installation location to write fftw wisdom file. |
113 | 29 | Anchi Cheng | * For developer only: you can copy the wisdom file to any other home directory that wants to use it, or copy it as root to a file called /etc/fftw/wisdom which any user can access. |
114 | |||
115 | 1 | Anchi Cheng | h3. Using fftwsetup.py: |
116 | 25 | Anchi Cheng | |
117 | 30 | Anchi Cheng | FIrst, you need to know the typical image dimension redux needs to process, especially the large ones and *ones that are not powers of 2 such as those produced by Gatan K2*. |
118 | 25 | Anchi Cheng | |
119 | <pre> |
||
120 | cd /YourMyamDownload/pyami/fft |
||
121 | ./fftwsetup.py 1 number_of_rows number_of_colums |
||
122 | </pre> |
||
123 | |||
124 | For example, for K2 summit super-resolution image there are 7676 rows and 7420 columns, therefore, the command line is |
||
125 | <pre> |
||
126 | ./fftwsetup.py 1 7676 7420 |
||
127 | </pre> |
||
128 | This may take a few minutes. |
||
129 | |||
130 | Run this for as many dimensions you know will be used and move it for general use as instructed above if preferred. |
||
131 | 1 | Anchi Cheng | |
132 | 54 | Amber Herold | h2. Start the Redux server for use with myamiweb |
133 | 14 | Jim Pulokas | |
134 | 55 | Amber Herold | Start reduxd server if not already running from the above test:<pre>reduxd</pre> |
135 | 11 | Neil Voss | |
136 | 55 | Amber Herold | h2. Starting reduxd at boot |
137 | 11 | Neil Voss | |
138 | <pre> |
||
139 | sudo cp -v myami/redux/init.d/reduxd /etc/init.d/ |
||
140 | 1 | Anchi Cheng | sudo service reduxd start |
141 | </pre> |
||
142 | |||
143 | 54 | Amber Herold | h2. Test the myamiweb and Redux installation |
144 | |||
145 | 51 | Amber Herold | # Visit http://yourhost/myamiweb or http://localhost/myamiweb to confirm functionality of the myamiweb website. |
146 | # Browse to the automatic web server troubleshooter at: http://localhost/myamiweb/test/checkwebserver.php |
||
147 | 54 | Amber Herold | # Click on "[test dataset]" on your main myamiweb home page, or go directly to the URL: myamiweb/viewerxml.php |
148 | # Test by accessing your own images from Leginon in myamiweb/imageviewer.php |
||
149 | 51 | Amber Herold | |
150 | 56 | Amber Herold | h2. Turn off error checking in php.ini |
151 | 51 | Amber Herold | |
152 | Once functionality is confirmed, you may turn off the display of website errors. |
||
153 | Edit the following items in php.ini (found as /etc/php.ini on CentOS and /etc/php5/apache2/php.ini on SuSE) so that they look like the following: |
||
154 | |||
155 | bq. display_errors = Off |
||
156 | 18 | Anchi Cheng | |
157 | 22 | Jim Pulokas | h2. Alternative reduxd installation on file server rather than web server |
158 | |||
159 | This approach is less tested, but has been found necessary in the case of a very high load system (multiple microscopes acquiring data, multiple processing jobs, multiple web clients and servers, etc). The problem has been that all of this demand for image I/O is occuring over NFS, and the NFS server is not coping with the high demand. One solution we have been investigating is to run reduxd directly on the file server rather than the web server. This means reduxd has direct access to the image files (not through NFS) and the web server now accesses reduxd over the network rather than locally. This has two potential benefits: 1) less load on the NFS server, 2) only transferring JPEGs over the network, not MRCs. |
||
160 | |||
161 | How to do it: |
||
162 | |||
163 | # Install required myami components on the file server, at least pyami, sinedon, numextension, redux, leginon, modules, and any other 3rd part packages required by those. |
||
164 | 23 | Jim Pulokas | # (optional) edit init.d/reduxd and use the optional high priority command line (see comment in that file) |
165 | 22 | Jim Pulokas | # Configure redux.cfg on file server and start reduxd |
166 | 23 | Jim Pulokas | # On web server, configure config.php: turn off all caching options, set redux server host to be the file server |
167 | 24 | Jim Pulokas | # skip starting reduxd on the web server |
168 | 23 | Jim Pulokas | # skip imcache as described below, since this also uses redux locally and accesses images over NFS |
169 | 22 | Jim Pulokas | |
170 | 20 | Amber Herold | {{include(leginon:Using imcache to cache mrc images as jpeg images of the default size on myamiweb)}} |