Project

General

Profile

Using Redux to serve images on myamiweb » History » Version 22

Jim Pulokas, 03/18/2013 11:47 AM

1 1 Anchi Cheng
h1. Using Redux to serve images on myamiweb
2
3 21 Amber Herold
*Before you begin:*
4
Redux is new for CentOS 6 (PHP 5.3). 
5
If you want to install *Appion/Leginon 2.2 on CentOS 5*, you do not need to install Redux. Instead, you must follow the directions to [[Install the MRC PHP Extension]].
6
7 1 Anchi Cheng
h2. Installation
8
9
# Follow [[Web_Server_Installation]] but ignore warning on not using php 5.3
10 17 Anchi Cheng
# Install myami-redux branch of myami on the webserver machine.
11
** Install redux prerequisites as found in its README file
12
** Redux needs write permission at its installation location to write fftw wisdom file.
13
** Install python filesystem abstraction for redux caching like this:
14 6 Neil Voss
<pre>
15
sudo easy_install fs
16
</pre>
17
18 1 Anchi Cheng
h2. Configurations:
19
20 16 Anchi Cheng
h3. configure redux
21 17 Anchi Cheng
22 1 Anchi Cheng
<pre>
23 17 Anchi Cheng
> cd /YourMyamDownload/redux
24 16 Anchi Cheng
> cp redux.cfg.template redux.cfg
25 1 Anchi Cheng
</pre>
26 17 Anchi Cheng
* You can also copy it to /etc/myami/redux.cfg if you prefer.
27 1 Anchi Cheng
28 17 Anchi Cheng
* Turn on redux caching if desired,
29 1 Anchi Cheng
<pre>
30 16 Anchi Cheng
[cache]
31
enable: yes
32
path: /var/cache/myami/redux
33
disksize:  500
34
memsize: 500
35 1 Anchi Cheng
</pre>
36
*You need to make sure the cache path exists and writable by the user that starts the redux server (reduxd)*
37 16 Anchi Cheng
** Input the desired disk_cache_path, disk_cache_size, and mem_cache_size in the next few lines
38 3 Anchi Cheng
** Create the disk_cache_path before running redux if cache will be used
39
40 16 Anchi Cheng
h3. Assign redux server in myamiweb
41 17 Anchi Cheng
42 16 Anchi Cheng
* At myamiweb/config.php
43 7 Neil Voss
<pre>
44 16 Anchi Cheng
define('SERVER_HOST',"localhost")
45 2 Anchi Cheng
</pre>
46
47 15 Jim Pulokas
h2. Test Installation
48
49
# Using Redux to do simple input output (simple client with no reduxd server needed):<pre>redux --filename=test.jpg --oformat=PNG > test.png</pre>Make sure the resulting test.png is in fact an image and not an error message.
50 2 Anchi Cheng
# Run the server:  run the following on a command line:<pre>reduxd</pre>leave it running for the following...
51 12 Gabriel Lander
# Test command line client connecting to the server<pre>redux --server_host=localhost --filename=test.jpg --oformat=PNG > test.png</pre>Note: The input file name is from the perspective of the reduxd server, so be sure to give it an absolute path (unlike the first test, which was not accessing the server)
52 2 Anchi Cheng
53 15 Jim Pulokas
h2. Using redux server with myamiweb
54 14 Jim Pulokas
55 15 Jim Pulokas
# start reduxd server if not already running from the above test:<pre>reduxd</pre>
56 10 Neil Voss
# click on "[test dataset]" on your main myamiweb home page, or go directly to the URL: myamiweb/viewerxml.php
57 15 Jim Pulokas
# test by accessing your own images from Leginon in myamiweb/imageviewer.php
58 11 Neil Voss
59
h2. starting reduxd at boot
60
61
<pre>
62
sudo cp -v myami/redux/init.d/reduxd /etc/init.d/
63 1 Anchi Cheng
sudo service reduxd start
64
</pre>
65 18 Anchi Cheng
66 22 Jim Pulokas
h2. Alternative reduxd installation on file server rather than web server
67
68
This approach is less tested, but has been found necessary in the case of a very high load system (multiple microscopes acquiring data, multiple processing jobs, multiple web clients and servers, etc).  The problem has been that all of this demand for image I/O is occuring over NFS, and the NFS server is not coping with the high demand.  One solution we have been investigating is to run reduxd directly on the file server rather than the web server.  This means reduxd has direct access to the image files (not through NFS) and the web server now accesses reduxd over the network rather than locally.  This has two potential benefits:  1) less load on the NFS server, 2) only transferring JPEGs over the network, not MRCs.
69
70
How to do it:
71
72
# Install required myami components on the file server, at least pyami, sinedon, numextension, redux, leginon, modules, and any other 3rd part packages required by those.
73
# Configure redux.cfg on file server and start reduxd
74
# On web server, configure config.php:  turn off all caching options, set redux server to be the file server
75
76
77
78 20 Amber Herold
{{include(leginon:Using imcache to cache mrc images as jpeg images of the default size on myamiweb)}}