Project

General

Profile

Actions

Bug #1271

closed

Using Redux server instead of PHP mrc module.

Added by Eric Hou over 13 years ago. Updated about 11 years ago.

Status:
Closed
Priority:
High
Assignee:
Category:
-
Target version:
-
Start date:
04/26/2011
Due date:
% Done:

0%

Estimated time:
Affected Version:
Appion/Leginon 2.2.0
Show in known bugs:
No
Workaround:

Description

convert current php mrc module usage to Redux.


Related issues 3 (0 open3 closed)

Related to Leginon - Bug #669: php MRC module does not work with php version greater than 5.2Duplicate06/16/2010

Actions
Related to Appion - Bug #179: viewstack.php bug, does not display last imageDuplicate03/04/2010

Actions
Related to Appion - Bug #2289: myamiweb/processing/download.php is not working to download mrc volumesClosedAmber Herold03/12/2013

Actions
Actions #1

Updated by Eric Hou over 13 years ago

There are still have things need to implement.
Caching, PadShape, Pad Position, Pad Value etc...

Actions #2

Updated by Eric Hou over 13 years ago

  • Assignee changed from Eric Hou to Amber Herold
Actions #3

Updated by Neil Voss over 13 years ago

Redux changes are all in myami-eric branch also see r15632, r15633, r15634, and r15635

Actions #4

Updated by Jim Pulokas about 13 years ago

  • Status changed from Assigned to In Code Review
  • Assignee changed from Amber Herold to Anchi Cheng

Anchi, I am adding in the support for reading a frame from a stack. starting with MRC:
r16148
You should be able to give one more argument "frame" in the redux request, which should be the frame number that you want (starting from 0 as usual)
Imagic support to follow soon...

Actions #5

Updated by Jim Pulokas about 13 years ago

None of this is even the least bit tested...

Imagic read support added to pyami. I realize there are already imagic functions in appion that should be moved to common area, but wanted something quick and simple without figuring that out
r16149

Imagic read support added to redux:
r16150

Actions #6

Updated by Jim Pulokas about 13 years ago

r16151 fixes dtype problem. I tested redux stand alone and it can read a frame from mrc or imagic, so only thing left is to make whatever changes in php code.

Actions #7

Updated by Jim Pulokas about 13 years ago

r16155
adds option to read image header, meta-data, etc and returns JSON data.
request parameters may look like this for example:
"filename=whatever.mrc&info=1&oformat=JSON"

NOTE: due to the use of json module, this is only compatible with python 2.6 and greater. If this is a problem, we can simplify it considerably to only return essential information (like image size) and write our own function to encode the JSON string. Right now it takes everything it can get from the header dict.

Actions #8

Updated by Jim Pulokas about 13 years ago

added several updates r16158 r16160 r16161 r16162 r16163 r16164

Adds the histogram pipe, for example: "histbins=100" will create a histogram with 100 bins. Then the Format pipe must be configured with oformat=JSON.
The result is a JSON string which represents the return value of the numpy.histogram function, so it will return two arrays, the first array is the histogram values, the second array is the boundary values of the bins.

Also adds the ability to customize the pipeline, for example: "pipes=r:Read,f:Format" will create a very small pipeline of only the Read and Format pipes. Notice each pipe instance has a name (the string before the colon). This will come in useful once I implement the ability to have more than one instance of the same pipe class. We can also store "preset" pipelines in the pipelines directory and then refer to them as "pipeline=myexample" instead of the long messy "pipes" declaration above. The default is the "standard" preset defined in pipelines/standard.py

example to get a histogram of an image in JSON format:
"pipes=r:Read,h:Histogram,f:Format&filename=test.mrc&histbins=100&oformat=JSON"

Actions #9

Updated by Jim Pulokas about 13 years ago

in r16165:
added option rgb option to the Format pipe. For example, "rgb=true" to make sure the output is RGB and not grayscale. Default is rgb=false.

Actions #10

Updated by Anchi Cheng about 13 years ago

r16193 and r16194 commit components needed for myamiweb.

Tested it on our webservers with reduxd started on various host and learned the following requirements:

  1. reduxd need either python 2.6 or demjson module for older versions of python.
  2. webserver need to be at least 5.2 to decode json.
Actions #11

Updated by Anchi Cheng about 13 years ago

Jim found a pecl extension that works with php 5.1.x that CentOS 5.3-5.6 have.
See:

http://www.php.net/manual/en/json.installation.php

Actions #12

Updated by Anchi Cheng about 13 years ago

  • Status changed from In Code Review to Assigned
  • Assignee changed from Anchi Cheng to Jim Pulokas

Latest testing result by doing it on cronus3: It can do fft and change intensity by mean/stdev, but the minmax method behaves funny.

Jim, do you remember why Eric and you decided to remove the absolute image value option but keep the relative one? It is now making it only allow an absolute range between 0 and 255 when most image mean is in the 1000 range.

Actions #13

Updated by Anchi Cheng about 13 years ago

After discussion with Jim, I got the answer: after manipulation by redux, the intensity scaling that is done at the end can be affected by many things such as binning. Using absolute scale makes it harder to go between images using the same settings even though it is still possible.

I've made php code to use the original intensity range to set absolute scale based on the relative ones to redux and therefore preserve the same scaling for images with different scale. This does not work with power spectrum since the range is not known unless it is transformed, the most time-consuming step. Therefore, it is disabled for fft displays.

The mean/stdev method still has scaling differences between the view in the viewer and the map from the binning difference. Bridget thought there is no need to get that perfect and like the better contrast of redux over the smoother php-mrc.

r16212, r16213, bring redux in sync with trunk
Jim made some change earlier to make frame reading from image stack uncached while caching when full image is read. This solves the problem of frame caching. r16214 does the myamiweb side change for it. The result is good.

Just spider file reading left.

Actions #14

Updated by Anchi Cheng almost 13 years ago

After discussion at appion meeting, we decide to concentrate on making CentOS 6 as a requirement for the next myami release that uses redux because of some difficulty of getting modules needed for python 2.4.

Actions #15

Updated by Anchi Cheng over 12 years ago

For preparation of the upcoming Appion Developer Workshop that uses newer dependency like python 2.6, we are going to merge redux branch to trunk in about an hour.

This means that for those of you with a sandbox myamiweb, it will stop working until redux is started as root on the webserver host Using_Redux_to_serve_images_on_myamiweb

If you work only on the processing side or part does not involve image display in myamiweb, you should still be able to continue as you were.

I will inform you regarding the revision number when this happens.

Actions #16

Updated by Anchi Cheng over 12 years ago

  • Affected Version changed from Appion/Leginon 2.1.0 to Appion/Leginon 2.2.0 (trunk)

It is done. r16995 is the one that with redux. Hope it is not a disaster.

Actions #17

Updated by Jim Pulokas about 11 years ago

  • Status changed from Assigned to In Test
  • Affected Version changed from Appion/Leginon 2.2.0 (trunk) to Appion/Leginon 2.2.0
Actions #18

Updated by Jim Pulokas about 11 years ago

  • Status changed from In Test to Closed
Actions

Also available in: Atom PDF