Forums » Software Installation »
error while trying to create a processing database
Added by Rogelio Hernandez-Lopez about 11 years ago
Hi
I am getting the following message when trying to access the processing plugging for any of my projects,
No processing database can be found for this project
click here to create one
If I click on the link I get the following error:
Your access to the page was denied
You are not an owner of the project nor a guest invited by the owner to the experiment session.
Use your browser to return to the last allowed page.
any advice about where to find or change permissions? I think I am missing something,
thanks a lot!
Replies (19)
RE: error while trying to create a processing database - Added by Anchi Cheng about 11 years ago
Rogelio,
Projects can only be created by someone in the administrator group. Unforturnately, during the creation process, it does not assign owner automatically, or at least insist on you to have one. Not knowing how you arrive at the error exactly, here are two of my guesses:
1. You have a project created by someone in the administrator group, such as administrator, no owner was assigned. An Leginon session was created with a user in the user group. Normally, what should have happen is that the user has no access to the session he/she made in myamiweb when logged in as him/her. Obviously you have got past that for some reason, likely you have changed login in the web browser session but left the image viewer page there. When you click on "processing" tool in this case, myamiweb checked the web browser loggin and found that you are now a user in the user group, not administrator group, trying to access a project administration page that does not belong to you, so it rejected you.
The solution to this case is to log in as an administrator again, add the user as an owner of the project, and then log in as the user. It should solve the problem.
2. You did not allow creation of processing database through webserver database user defined in config.php during database installation.
The procedure to do this is covered in Database Server Installation step 11 Give create and access privileges for the processing databases which begin with "ap". This is less likely the reason since that error should come when you are actually creating the processing database link, not when you access the project administration page.
If neither one of these is your problem, we will need more information on how you arrived there, specifically about whether you've activated login and the login group etc.
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
Hi,
how do I check whether I've activated login and the login group, and what the logged user is?
I did review the way I created the user: usr_object and the permissions granted to the user in the databases following the instruction in http://emg.nysbc.org/projects/leginon/wiki/Database_Server_Installation,
I used option 1 in step 10.
CREATE USER usr_object@'localhost' IDENTIFIED BY 'YOUR PASSWORD';
GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON leginondb.* TO usr_object@'localhost';
GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON projectdb.* TO usr_object@'localhost';
and granted access to the ap* databases as
GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON `ap%`.* to usr_object@localhost;
if I review the permission to usr_object I get:
Grants for usr_object@localhost |-----------------------------------------------------------------------------------------------------------------
| GRANT USAGE ON *.* TO 'usr_object'@'localhost' IDENTIFIED BY PASSWORD '*04FD2043EADE6511EFE52E7598FB32F90D0557F6' |
| GRANT SELECT, INSERT, UPDATE, CREATE, ALTER ON `projectdb`.* TO 'usr_object'@'localhost' |
| GRANT SELECT, INSERT, UPDATE, CREATE, ALTER ON `leginondb`.* TO 'usr_object'@'localhost' |
| GRANT SELECT, INSERT, UPDATE, CREATE, ALTER ON `ap%`.* TO 'usr_object'@'localhost' |-----------------------------------------------------------------------------------------------------------------
In the projects page I can create new projects without problems and upload images to them
http://localhost/myamiweb/project/project.php
the owners option of the project is left empty, but the problem comes when I go to the processing page from the Image viewer, at this point the page can't get correctly the ProjectId as always shows 0
thanks
Rogelio
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
when I setup the data base I used the web tools set up wizard and did not enable the login feature
These are the options I entered.
PROJECT_NAME : myamiweb
BASE_PATH : myamiweb
BASE_URL : /myamiweb/
PROJECT_URL : /myamiweb/project/
PROJECT_TITLE : Appion and Leginon Tools
ENABLE_LOGIN : false
DB_HOST : localhost
DB_USER : usr_object
DB_PASS : leginon
DB_LEGINON : leginondb
DB_PROJECT : projectdb
SERVER_HOST : localhost
SERVER_PORT : 55123
ENABLE_CACHE : true
CACHE_PATH : /srv/cache/myamiweb
PROCESSING : true
DEF_PROCESSING_PREFIX : ap
HIDE_IMAGIC : false
HIDE_MATLAB : false
HIDE_FEATURE : true
TEMP_IMAGES_DIR : /labdata/allab/leginondata
DEFAULT_APPION_PATH : /labdata/allab/leginondata/appion
RE: error while trying to create a processing database - Added by Amber Herold about 11 years ago
Just to be sure, when you did this step:
CREATE USER usr_object@'localhost' IDENTIFIED BY 'YOUR PASSWORD'; GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON leginondb.* TO usr_object@'localhost'; GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON projectdb.* TO usr_object@'localhost';
You replaced the words YOUR PASSWORD with leginon which is what the setup wizard is using as the db password? I don't think you would get as far as you have if they did not match, just making sure.
And have you tried to set an owner for the project?
We don't usually run with the login feature disabled here. I'll do some tests to see if I can reproduce this.
RE: error while trying to create a processing database - Added by Anchi Cheng about 11 years ago
The mysql user is always usr_object as in config.php while you run any query initiated from myamiweb.
Does the session show up when you display the project page that is supposed to contain your session? It looks like the screenshot in this wiki page
It is likely that the session has no link to a project.
You can check this with phpMyAdmin if you have installed it, or directly with mysql script
select * from projectdb.projects; select * from projectdb.projectexperiments;
The first line gives you the list of projects in your project database note the `DEF_id` field, the second line gives you the matching pair of the project in `REF|projects|project` and upload session (`REF|leginondata|SessionData|session`).
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
No, the sessions do not show up in the project page,
the two commands you ask me to check output the following:
select * from projectdb.projects;
---------------------------+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------------+------------------------+---------+-----------+--------+
| DEF_id | DEF_timestamp | name | short_description long_description | category | funding | leginondb | hidden |---------------------------+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------------+------------------------+---------+-----------+--------+
| 1 | 2013-10-04 19:18:12 | monomer K2 | Grids force:3 time:1.5 s, Polara 300 kV, px Size = 1.97 A/px | | NULL | 0 |
| 2 | 2013-10-08 19:52:42 | test | Test project | | NULL | 0 |
| 3 | 2013-10-14 13:57:34 | test2 | testing database | | | | NULL | 0 | ---------------------------+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------------+------------------------+---------+-----------+--------+
3 rows in set (0.00 sec)
select * from projectdb.projectexperiments;
Empty set (0.00 sec)
how could I fix this? thanks!
RE: error while trying to create a processing database - Added by Anchi Cheng about 11 years ago
Amber is looking into how this happens because we wrote it in the way that the data can only be uploaded through a specific project page.
First, check your sinedon.cfg to make sure both leginondata and projectdata are defined according to your database names.
Configure_sinedoncfg
If that is correct, for you to fix this, you can use the following python script. I am assuming that your session is the most recent session here
from leginon import leginondata # This gives the most recent session data object s = leginondata.SessionData().query(results=1)[0] # Print the name of the session so that you can check if it is the correct one before proceed print s['name'] from leginon import projectdata # This gives the project with DEF_id of 1 in your projects table as given in your post above p = projectdata.projects().direct_query(1) # This attempts to insert the row that links the session with the project pequery = projectdata.projectexperiments(project=p,session=s) pequery.insert()
The syntex here uses sinedon, our ORM from python to mysql. It should have done this during your upload. Hopefully this will either work, or we will find what the error it generates.
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
Hi,
thanks for your help,
I typed the above commands directly in my terminal and did not get any error, yet the session does not display in the project's page
Screenshot.png (63.3 KB) Screenshot.png |
RE: error while trying to create a processing database - Added by Anchi Cheng about 11 years ago
O.K.,
My script assumes that your project is the first of your project (monomerK2), but the page you are showing screen shot is for the project named test2 which has id of 3. Check the first project instead.
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
I checked all pages, none display the sessions,
Screenshot-1.png (73.1 KB) Screenshot-1.png |
RE: error while trying to create a processing database - Added by Amber Herold about 11 years ago
I just ran a fresh installation of the trunk with user login disabled, and I'm not seeing issues like this when creating a project and uploading images to a new session.
The only thing to note is that the session is hidden if there are not very many images in it.
It might help to know exactly what steps you are taking to create the project and session. Like "step 1: open myamiweb, step 2: Select Projet DB button". If you can tell me what you do from the beginning and where in the process things go wrong, I can narrow down the possible issues.
RE: error while trying to create a processing database - Added by Anchi Cheng about 11 years ago
My last guess. Since there is no error in sinedon insert but you have no association on the webviewer, I can only guess that the two are not pointing to the same project database.
Try this in shell to show all mysql databases.
mysql -u usr_object -p mysql> show databases;
You should get the whole list of the databases. See if there is anything odd. Compare what you have there with what you assigned in myamiweb and sinedon.cfg
Also, confirm that the insertion from my last python script has happened by repeating the script except the last two llines.
Instead of
pequery = projectdata.projectexperiments(project=p,session=s) pequery.insert()
Do
pequery = projectdata.projectexperiments(project=p,session=s) results = pequery.query() print results
This should give you a list with lots of stuff in it. If your insert did not happen, you get empty list [] back.
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
The way I am uploading the images is as follow:
1. "Open myamiweb http://localhost/myamiweb/"
2. Go to project database "http://localhost/myamiweb/project/project.php"
3. Click on Add a new project "http://localhost/myamiweb/project/updateproject.php"
4. Fill the form and click on add, I got a red message "Your update has been submitted"
5. Click on view projects
6. Click on the project that just created
7. click on create processing database
8. Click on upload images to new session and fill the form with you data (image attached)
9. Click on just show command (image attached)
10. copy and pasted command in a terminal
Got the following result:
[rogelio@aida appion]$ imageloader.py --projectid=5 --session=13oct16z --cs=2 --dir=/labdata/allab/rogelio/tempUpload/test2 --filetype=mrc --apix=1.97 --binx=1 --biny=1 --df=-3 --mag=25342 --kv=300 --description="my test" --jobtype=uploadimage
... Time stamp: 13oct16w31
... Function name: imageloader
... Appion directory: /usr/lib/python2.6/site-packages
... Load average is high 6.1
... New load average 6.09
... Using split database
Connected to database: 'ap5'
... Committing data to database
Creating a new session
... Found project id=5 for session 13oct16z in 2.8 msec
!!! WARNING: run directory not defined, automatically setting it
... Run directory: /labdata/allab/leginondata/13oct16z/rawdata
... Writing function log to: imageloader.log
... Running on host: aida
... Uploading ScriptData....
... Running on host: aida
... Found 8 processors on this machine
... Running on host: aida
... Looking up session, 13oct16z
... Running Appion version 'trunk'
... Creating new done dictionary: imageloader.donedict
Beginning Main Loop
... reading image info
Starting image 1 ( skip:0, remain:2 ) file: 13oct16z_Oct01_12.32.50_01
... Reading original image: /labdata/allab/rogelio/tempUpload/test2/Oct01_12.32.50_0001.mrc
==== Committing data to database ====
... Load average is high 6.16
SUMMARY: imageloader
------------------------------------------
TIME: 8.77 sec
------------------------------------------
... reading image info
Starting image 2 ( skip:0, remain:1 ) file: 13oct16z_Oct01_13.29.04_0020
... Reading original image: /labdata/allab/rogelio/tempUpload/test2/Oct01_13.29.04_0020.mrc
==== Committing data to database ====
... Load average is high 6.15
SUMMARY: imageloader
------------------------------------------
TIME: 7.76 sec
AVG TIME: 8.27 +/- 0.71 sec
(- REMAINING TIME: 8.97 sec for 1 images )
-----------------------------------------
... reading image info
Starting image 3 ( skip:0, remain:0 ) file: 13oct16z_Oct01_23.47.24_0237
... Reading original image: /labdata/allab/rogelio/tempUpload/test2/Oct01_23.47.24_0237.mrc
==== Committing data to database ====
... Load average is high 6.20
SUMMARY: imageloader
------------------------------------------
TIME: 7.83 sec
AVG TIME: 8.12 +/- 0.56 sec
------------------------------------------
COMPLETE LOOP: 24.36 sec for 3 images
... Load average is high 6.19
... Closing out function log: imageloader.log
... Ended at Wed, 16 Oct 2013 22:33:06
... Memory increase during run: 53.254 MB
Total run time: 1 min 12 sec
... Run directory:
/labdata/allab/leginondata/13oct16z/rawdata
11. Click on project 4 again, the session that just created is not there
Regarding mysql databases it looks normal to me,
[rogelio@aida monomerMTs-K2]$ mysql -u usr_object -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1157
Server version: 5.5.34-log MySQL Community Server (GPL) by Remi
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases
> ;
| Database |
-------------------------------------
| information_schema |
| ap1 |
| ap2 |
| ap3 |
| ap4 |
| leginondb |
| projectdb |
| test |------------------
8 rows in set (0.10 sec)
hmmm...
I did get an empty list [] back after the python commands though
Python 2.7.2 (default, Oct 27 2011, 16:10:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
from leginon import leginondata
- This gives the most recent session data object
... s = leginondata.SessionData().query(results=1)[0]
- Print the name of the session so that you can check if it is the correct one before proceed
... print s['name']
13oct14m
from leginon import projectdata
- This gives the project with DEF_id of 1 in your projects table as given in your post above
... p = projectdata.projects().direct_query(1)
pequery = projectdata.projectexperiments(project=p,session=s)
results = pequery.query()
print results
[]
print results
[]
results = pequery.query()
print results
[]
pequery = projectdata.projectexperiments(project=p,session=s)
results = pequery.query()
print results
[]
Screenshot-2.png (94.8 KB) Screenshot-2.png | |||
Screenshot-3.png (123 KB) Screenshot-3.png | |||
Screenshot-4.png (75.2 KB) Screenshot-4.png |
RE: error while trying to create a processing database - Added by Anchi Cheng about 11 years ago
For some reason, you can query and insert to projectdb from myamiweb php and query and insert to leginondb through sinedon o.k. However, you can not insert through sinedon.
First, please confirm that sinedon can query projectdb. I think it does. Do this in python:
from leginon import projectdata p = projectdata.projects().direct_query(5) print p['name']
This should give the project name that has id of 5. I believe it is 'test4' in your case.
Second, please find out the session id number of the session you have uploaded. Easiest is to click on [summary] tag of the session. It is shown in the url as expId assignment like the 8 below:
http://localhost/myamiweb/summary.php?expId=8
You can then try the straight mysql command with that number replace the 65 at the end of this example.
CREATE TABLE IF NOT EXISTS `projectdb`.`projectexperiments` (`DEF_id` int(16) NOT NULL AUTO_INCREMENT, `DEF_timestamp` timestamp NOT NULL, `REF|projects|project` INT(20) NULL DEFAULT NULL, `REF|leginondata|SessionData|session` INT(20) NULL DEFAULT NULL, PRIMARY KEY (`DEF_id`), KEY `DEF_timestamp` (`DEF_timestamp`), KEY `REF|projects|project` (`REF|projects|project`), KEY `REF|leginondata|SessionData|session` (`REF|leginondata|SessionData|session`)) ENGINE=MyISAM; ALTER TABLE `projectdb`.`projectexperiments` ADD `DEF_id` INT(16) NULL; ALTER TABLE `projectdb`.`projectexperiments` ADD `REF|projects|project` INT(20) NULL; ALTER TABLE `projectdb`.`projectexperiments` ADD `REF|leginondata|SessionData|session` INT(20) NULL; INSERT INTO `projectdb`.`projectexperiments` (`REF|projects|project`, `REF|leginondata|SessionData|session`) VALUES (5, 65);
After that make it show the entries of projectexperiments with mysql command
select * from `projectdb`.`projectexperiments`;
I got these directly from what sinedon sent to mysql. If the insertion gives no error and yet the projectexperiments table is still empty, it is beyond me.
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
That worked!I guess this means that somewhere in the code sinedon is not doing this for the other cases?
Python 2.7.2 (default, Oct 27 2011, 16:10:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
from leginon import projectdata
p = projectdata.projects().direct_query(5)
print p['name']
Test4
mysql> CREATE TABLE IF NOT EXISTS `projectdb`.`projectexperiments` (`DEF_id` int(16) NOT NULL AUTO_INCREMENT, `DEF_timestamp` timestamp NOT NULL, `REF|projects|project` INT NULL
-> DEFAULT NULL, `REF|leginondata|SessionData|session` INT NULL DEFAULT NULL, PRIMARY KEY (`DEF_id`), KEY `DEF_timestamp` (`DEF_timestamp`), KEY `REF|projects|project` (`REF|projects|project`), KEY `REF|leginondata|SessionData|session` (`REF|leginondata|SessionData|session`)) ENGINE=MyISAM;
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> ALTER TABLE `projectdb`.`projectexperiments` ADD `DEF_id` INT NULL;
ERROR 1060 (42S21): Duplicate column name 'DEF_id'
mysql> ALTER TABLE `projectdb`.`projectexperiments` ADD `REF|projects|project` INT NULL;
ERROR 1060 (42S21): Duplicate column name 'REF|projects|project'
mysql> ALTER TABLE `projectdb`.`projectexperiments` ADD `REF|leginondata|SessionData|session` INT NULL;
ERROR 1060 (42S21): Duplicate column name 'REF|leginondata|SessionData|session'
mysql> INSERT INTO `projectdb`.`projectexperiments` (`REF|projects|project`, `REF|leginondata|SessionData|session`) VALUES (5, 10);
Query OK, 1 row affected (0.28 sec)
mysql> select * from `projectdb`.`projectexperiments`;---------------------------+----------------------+-------------------------------------+
| DEF_id | DEF_timestamp | REF|projects|project | REF|leginondata|SessionData|session |---------------------------+----------------------+-------------------------------------+
| 4 | 2013-10-17 00:19:52 | 5 | 10 |---------------------------+----------------------+-------------------------------------+
1 row in set (0.00 sec)
RE: error while trying to create a processing database - Added by Anchi Cheng about 11 years ago
In mysql, check the table type by
show table status from projectdb where Name = 'projectexperiments';
If Engine is MyISAM then the table is normal. Don't see why sinedon can not insert. Could there be a write permission problem? We've never had problem with one single table nor single database when installation include granting permission to the database user.
You can turn on debug in sinedon to print out the queries like the one I created by changing in sinedon/sqldict.py at your installation location (python site-packages)
debug = True
There will be a lot of print out, but you can search for projectexperiments in the output.
Or maybe the python-mysql version you had to use is not working right. None of our local machine has your exact mysql and python-mysqldb version.
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
the engine is not MyISAM but rather InnoDB,
mysql> show table status from projectdb where Name = 'projectexperiments';--------------------------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+
| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |--------------------------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+
| projectexperiments | InnoDB | 10 | Compact | 2 | 8192 | 16384 | 0 | 49152 | 10485760 | 8 | 2013-10-04 15:34:03 | NULL | NULL | latin1_swedish_ci | NULL | | |--------------------------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+
1 row in set (0.00 sec)
how can I change it? I modified the my.cnf file stored in /etc/ according to http://emg.nysbc.org/projects/appion/wiki/Database_Server_Installation_Shared, is there any other location where I should make a change?
thanks!
RE: error while trying to create a processing database - Added by Anchi Cheng about 11 years ago
Rogelio,
I believe the reason for you to have some tables created in InnoDB is because you were halfway into myamiweb SetupWizard already before changing the default. It is unfortunate that sinedon is not compatible with InnoDB. We hope to fix this some time.
Here is a good answer from google search for how http://forums.mysql.com/read.php?21,26193,190902#msg-190902
If this looks crypted, and you have not accumulated data you want to keep, you can drop all tables in your leginondb and projectdb database and start from the myamiweb SetupWIzard again which should propagates tables in MyISAM this time.
RE: error while trying to create a processing database - Added by Rogelio Hernandez-Lopez about 11 years ago
Thanks a lot! I think dropping the tables and setting them up trough the wizard again solved the problem,
thanks again for all your help,