Project

General

Profile

Actions

Feature #1168

open

multiple database hosts

Added by Scott Stagg over 13 years ago. Updated over 13 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
01/31/2011
Due date:
% Done:

0%

Estimated time:

Description

We are trying to get things setup here at FSU where we have leginondb and projectdb on one host and the processing databases on a different host. As far as I can tell, this shouldn't be a problem at the leginon level, but it seems like it may be a problem for appion and definitely for myamiweb. Has anybody attempted anything like this before? If not would anyone be opposed if I added this functionality to myamiweb? Comments?

Actions #1

Updated by Amber Herold over 13 years ago

I have not attempted this, and I'm pretty sure it wont work with the current code. I'm not opposed to adding it, as long as it doesn't make installation and setting up the config file too complicated. Eric might have some tips for implementation.

Actions #2

Updated by Eric Hou over 13 years ago

Not so sure the Appion python part, but for the web part, you probably just need to create new constants for mysql object in particledata.inc.
Also, look at the code for create ap databases. You should also need to change the mysql object.

Thanks.

Eric

Actions #3

Updated by Anchi Cheng over 13 years ago

May I ask why you want to do this?

From python point of view, this might not be a problem but probably not tried. From myamiweb, there might be a number of mysql queries that currently only joining other databases assuming that they are on the same host. Go to myamiweb/processing/inc and search for DB_LEGINON in particledata.inc and you will see what I mean. You will have to specify hosts there, too.

Actions #4

Updated by Scott Stagg over 13 years ago

The reason to do this is that we want to take advantage of a new facility on campus that supports large volumes of disk space all under one partition. We want to write straight from the microscope to their disks over the network. We also want to take the database load off of the acquisition computer which is currently hosting all the databases and the webserver. The problem is that they worry that if their system goes down, we won't be able to collect data. Our proposed solution is to have the current acquisition computer host projectdb and leginondb and have the remote computer host all the processing databases and the webserver. The raw data will be written locally to the acquisition computer and simultaneously copied to the remote big volume (the local copies will be pruned once a week). In this scheme, if the remote storage goes down, we can still collect data - just not process it until the remote system comes back up.

Actions #5

Updated by Anchi Cheng over 13 years ago

Interesting idea. Let us know if it works well.

Actions #6

Updated by Neil Voss over 13 years ago

Anchi brings up a good question, can mysql 'LEFT JOIN' tables across different servers?

Actions #7

Updated by Eric Hou over 13 years ago

No, you can not join two tables from different host. You can fetch them first and compare from server side.
If it is possible, separate the web server and database server instead do the way you mention above. Web server is the one always give you more trouble. Database server is more reliable.

Thanks.

Eric

Actions

Also available in: Atom PDF