Project

General

Profile

Actions

Support #1224

closed

Request for DB statistics

Added by Bridget Carragher over 13 years ago. Updated over 13 years ago.

Status:
Closed
Priority:
High
Assignee:
Eric Hou
Category:
-
Target version:
-
Start date:
03/05/2011
Due date:
% Done:

0%

Estimated time:
Affected Version:
Appion/Leginon 2.1.0

Description

I am hoping to generate the usual set of statistics for the Legnon DB. I need: # projects, # experiments, # images, size of data, # DB records, size of DB]

and a plot of the change over time would be really nice as in: http://cronus3.scripps.edu/myamiweb/project/statistics.php although I would prefer to see the cumulative total rather then the month by month number.

It would be great if this could be generated automatically on a query page.

I would also very much like to request this information form our external Leignon users so if a query could be written that they could run really easily and report the results back to us that would be great!


Files

DB_Stats.docx (153 KB) DB_Stats.docx Bridget Carragher, 03/20/2011 01:54 PM
DBStatistics.docx (76.1 KB) DBStatistics.docx Bridget Carragher, 03/29/2011 11:21 PM

Related issues 3 (1 open2 closed)

Related to Leginon - Feature #1285: Additions to statistics report pageClosedBridget Carragher05/11/2011

Actions
Related to Leginon - Feature #1296: Write DB schema update scripts for new statistics tableClosedAmber Herold05/11/2011

Actions
Related to Appion - Bug #1349: Display lastest statistics report page while updating db queriesNew07/08/2011

Actions
Actions #1

Updated by Clint Potter over 13 years ago

We'd like to add a few more statistics (#stackes, class averages, maps) as well to these pages. When Eric returns, lets meet and discuss.

Actions #2

Updated by Eric Hou over 13 years ago

  • Status changed from New to Assigned
  • Priority changed from Normal to High

move up the priority.

Actions #3

Updated by Bridget Carragher over 13 years ago

How is this coming along? I really need this as son as possible as I would really like to send some subset of this query out to the Leginon user community to generate some external statistics. Let's discuss at automem meeting tomorrow.
I would like a page at NRAMM that looks something like the mockup in the attached doc file and a way of letting our users do something the same or similar and send the data back to us.
Bridget

Actions #4

Updated by Amber Herold over 13 years ago

Based on our last status meeting, the script is complete, however we were still deciding on how to distribute it. It takes about 20 minutes to run and brings the web server to a crawl, so we don't want people to run it when the web server is busy. We also don't want to run it in the middle of the night when a visitor is trying to collect data. So it seemed best not to have it run automatically, just when the user initiates it. This could be from a web page, but we will need to make it clear that it should only be run when the web server is relatively free. Eric has most likely made some progress since then.

Actions #5

Updated by Bridget Carragher over 13 years ago

OK thanks. But can I view the results please?
Are there some slower parts of it that can be left out expect for when absolutely neessary and other parts that can run routinely?
Wll it be slow on everyone's systems or jsut ours because we have so many projects?

Actions #6

Updated by Eric Hou over 13 years ago

I have the page in svn now. Before it update to betamyamiweb, you can see it at:
http://cronus3.scripps.edu/~erichou/myamiweb/project/runStat.php

There still some numbers not in the page yet according to Bridget's doc. The numbers in runStat.php is want I did when we was writing Appion proposal.

Thanks.

Eric

Actions #7

Updated by Bridget Carragher over 13 years ago

OK, is this still on your list to get me the rest of the numbers and graphs generated on a single page?
Also how do we send these to the other Leginon/Appion users? i.e. can we jsut send them a script to run that generates a file.
I would like tot get this finalized and shipped off this week.

Actions #8

Updated by Eric Hou over 13 years ago

Yes, I would keep adding the numbers base on the requested doc. But I need some help from others also. There are couple numbers I don't know where them stored at.

how do we send these to the other Leginon/Appion users?
- If the users use svn to setup their myamiweb, they can just do svn update. Otherwise, they need to get it at the next release. Or we need to teach them how to put the files in the right place in their file system. I am trying to put everything in one file, but sometimes generate some numbers required changes from other support file. If that's the case, users will need to add a new file and update couple different files (sounds dangerous).

The other way is to write a command line script, then we might be able to send them a file to generate these number, but it won't have any interface (no tables or graphic).

Thanks.

Eric

Actions #9

Updated by Eric Hou over 13 years ago

Add 7 days caching in to the report, so the page does not take too long to display back to user.
You can visit the page at: http://cronus3.scripps.edu/betamyamiweb/project/runStat.php

Thanks.

Eric

Actions #10

Updated by Bridget Carragher over 13 years ago

Hi Eric,
Very nice.
Caching is a great idea.
I have a few suggestions for the format of the output page. I have put this into a doc file that is attached. Also as a pdf in case the doc file is hard to read.
Basically I would like to have the whole output stats as one page. This will make is easier to assimilate in one look. Come chat is you want to discuss.
Bridget

Actions #11

Updated by Eric Hou over 13 years ago

No problem, I will get this fix this week.

Thanks.

Eric

Actions #12

Updated by Eric Hou over 13 years ago

Hi
Basically the DB statistic page is done, the only thing not there is the graphical summaries for Appion processed session and images.
To display the graphic for each ap database is pretty easy, but if we want to combine all of them it's a bit of work. First of all, each one created at different time. And they need a sql temporary table to hold the temporary query data before the finial calculation.
I am still investigating the strategy between using complex sql queries or the php mix with sql query.

But so far, you can visit the report page at:
http://cronus3.scripps.edu/betamyamiweb/project/runStat.php

Thanks.

Eric

Actions #13

Updated by Eric Hou over 13 years ago

updated database schema xml file for new myamiweb installation.
There still an database update script needs to be done before class this issue.

Eric

Actions #14

Updated by Eric Hou over 13 years ago

  • Status changed from Assigned to Closed

The status report page is done and working fine.

Actions

Also available in: Atom PDF