Leginon

Rogelio,

Projects can only be created by someone in the administrator group. Unforturnately, during the creation process, it does not assign owner automatically, or at least insist on you to have one. Not knowing how you arrive at the error exactly, here are two of my guesses:

1. You have a project created by someone in the administrator group, such as administrator, no owner was assigned. An Leginon session was created with a user in the user group. Normally, what should have happen is that the user has no access to the session he/she made in myamiweb when logged in as him/her. Obviously you have got past that for some reason, likely you have changed login in the web browser session but left the image viewer page there. When you click on "processing" tool in this case, myamiweb checked the web browser loggin and found that you are now a user in the user group, not administrator group, trying to access a project administration page that does not belong to you, so it rejected you.

The solution to this case is to log in as an administrator again, add the user as an owner of the project, and then log in as the user. It should solve the problem.

2. You did not allow creation of processing database through webserver database user defined in config.php during database installation.

The procedure to do this is covered in Database Server Installation step 11 Give create and access privileges for the processing databases which begin with "ap". This is less likely the reason since that error should come when you are actually creating the processing database link, not when you access the project administration page.

If neither one of these is your problem, we will need more information on how you arrived there, specifically about whether you've activated login and the login group etc.

Hi,

how do I check whether I've activated login and the login group, and what the logged user is?

I did review the way I created the user: usr_object and the permissions granted to the user in the databases following the instruction in http://emg.nysbc.org/projects/leginon/wiki/Database_Server_Installation,

I used option 1 in step 10.
CREATE USER usr_object@'localhost' IDENTIFIED BY 'YOUR PASSWORD';
GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON leginondb.* TO usr_object@'localhost';
GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON projectdb.* TO usr_object@'localhost';

and granted access to the ap* databases as

GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON `ap%`.* to usr_object@localhost;

if I review the permission to usr_object I get:

Grants for usr_object@localhost |
----------------------------------------------------------------------------------------------------------------- | GRANT USAGE ON *.* TO 'usr_object'@'localhost' IDENTIFIED BY PASSWORD '*04FD2043EADE6511EFE52E7598FB32F90D0557F6' | | GRANT SELECT, INSERT, UPDATE, CREATE, ALTER ON `projectdb`.* TO 'usr_object'@'localhost' | | GRANT SELECT, INSERT, UPDATE, CREATE, ALTER ON `leginondb`.* TO 'usr_object'@'localhost' | | GRANT SELECT, INSERT, UPDATE, CREATE, ALTER ON `ap%`.* TO 'usr_object'@'localhost' |
-----------------------------------------------------------------------------------------------------------------

In the projects page I can create new projects without problems and upload images to them
http://localhost/myamiweb/project/project.php

the owners option of the project is left empty, but the problem comes when I go to the processing page from the Image viewer, at this point the page can't get correctly the ProjectId as always shows 0

thanks

Rogelio

when I setup the data base I used the web tools set up wizard and did not enable the login feature

These are the options I entered.

PROJECT_NAME : myamiweb

BASE_PATH : myamiweb

BASE_URL : /myamiweb/

PROJECT_URL : /myamiweb/project/

PROJECT_TITLE : Appion and Leginon Tools

ENABLE_LOGIN : false

DB_HOST : localhost

DB_USER : usr_object

DB_PASS : leginon

DB_LEGINON : leginondb

DB_PROJECT : projectdb

SERVER_HOST : localhost

SERVER_PORT : 55123

ENABLE_CACHE : true

CACHE_PATH : /srv/cache/myamiweb

PROCESSING : true

DEF_PROCESSING_PREFIX : ap

HIDE_IMAGIC : false

HIDE_MATLAB : false

HIDE_FEATURE : true

TEMP_IMAGES_DIR : /labdata/allab/leginondata

DEFAULT_APPION_PATH : /labdata/allab/leginondata/appion

Just to be sure, when you did this step:

CREATE USER usr_object@'localhost' IDENTIFIED BY 'YOUR PASSWORD';
GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON leginondb.* TO usr_object@'localhost';
GRANT ALTER, CREATE, INSERT, SELECT, UPDATE ON projectdb.* TO usr_object@'localhost';

You replaced the words YOUR PASSWORD with leginon which is what the setup wizard is using as the db password? I don't think you would get as far as you have if they did not match, just making sure.

And have you tried to set an owner for the project?
We don't usually run with the login feature disabled here. I'll do some tests to see if I can reproduce this.

The mysql user is always usr_object as in config.php while you run any query initiated from myamiweb.

Does the session show up when you display the project page that is supposed to contain your session? It looks like the screenshot in this wiki page

It is likely that the session has no link to a project.

You can check this with phpMyAdmin if you have installed it, or directly with mysql script

select * from projectdb.projects;
select * from projectdb.projectexperiments;

The first line gives you the list of projects in your project database note the `DEF_id` field, the second line gives you the matching pair of the project in `REF|projects|project` and upload session (`REF|leginondata|SessionData|session`).

No, the sessions do not show up in the project page,

the two commands you ask me to check output the following:

select * from projectdb.projects;

---------------------------+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------------+------------------------+---------+-----------+--------+ | DEF_id | DEF_timestamp | name | short_description long_description | category | funding | leginondb | hidden |
---------------------------+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------------+------------------------+---------+-----------+--------+ | 1 | 2013-10-04 19:18:12 | monomer K2 | Grids force:3 time:1.5 s, Polara 300 kV, px Size = 1.97 A/px | | NULL | 0 | | 2 | 2013-10-08 19:52:42 | test | Test project | | NULL | 0 | | 3 | 2013-10-14 13:57:34 | test2 | testing database | | | | NULL | 0 | ---------------------------+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------+------------------+------------------------+---------+-----------+--------+
3 rows in set (0.00 sec)

select * from projectdb.projectexperiments;

Empty set (0.00 sec)

how could I fix this? thanks!

Amber is looking into how this happens because we wrote it in the way that the data can only be uploaded through a specific project page.

First, check your sinedon.cfg to make sure both leginondata and projectdata are defined according to your database names.
Configure_sinedoncfg

If that is correct, for you to fix this, you can use the following python script. I am assuming that your session is the most recent session here

from leginon import leginondata

# This gives the most recent session data object
s = leginondata.SessionData().query(results=1)[0]

# Print the name of the session so that you can check if it is the correct one before proceed
print s['name']

from leginon import projectdata

# This gives the project with DEF_id of 1 in your projects table as given in your post above
p = projectdata.projects().direct_query(1)

# This attempts to insert the row that links the session with the project
pequery = projectdata.projectexperiments(project=p,session=s)
pequery.insert()

The syntex here uses sinedon, our ORM from python to mysql. It should have done this during your upload. Hopefully this will either work, or we will find what the error it generates.

Hi,
thanks for your help,

I typed the above commands directly in my terminal and did not get any error, yet the session does not display in the project's page

O.K.,

My script assumes that your project is the first of your project (monomerK2), but the page you are showing screen shot is for the project named test2 which has id of 3. Check the first project instead.

I checked all pages, none display the sessions,

I just ran a fresh installation of the trunk with user login disabled, and I'm not seeing issues like this when creating a project and uploading images to a new session.
The only thing to note is that the session is hidden if there are not very many images in it.
It might help to know exactly what steps you are taking to create the project and session. Like "step 1: open myamiweb, step 2: Select Projet DB button". If you can tell me what you do from the beginning and where in the process things go wrong, I can narrow down the possible issues.

My last guess. Since there is no error in sinedon insert but you have no association on the webviewer, I can only guess that the two are not pointing to the same project database.

Try this in shell to show all mysql databases.

mysql -u usr_object -p
mysql> show databases;

You should get the whole list of the databases. See if there is anything odd. Compare what you have there with what you assigned in myamiweb and sinedon.cfg

Also, confirm that the insertion from my last python script has happened by repeating the script except the last two llines.

Instead of

pequery = projectdata.projectexperiments(project=p,session=s)
pequery.insert()

pequery = projectdata.projectexperiments(project=p,session=s)
results = pequery.query()
print results

The way I am uploading the images is as follow:

1. "Open myamiweb http://localhost/myamiweb/"
2. Go to project database "http://localhost/myamiweb/project/project.php"
3. Click on Add a new project "http://localhost/myamiweb/project/updateproject.php"
4. Fill the form and click on add, I got a red message "Your update has been submitted"
5. Click on view projects
6. Click on the project that just created
7. click on create processing database
8. Click on upload images to new session and fill the form with you data (image attached)
9. Click on just show command (image attached)
10. copy and pasted command in a terminal

Got the following result:
[rogelio@aida appion]$ imageloader.py --projectid=5 --session=13oct16z --cs=2 --dir=/labdata/allab/rogelio/tempUpload/test2 --filetype=mrc --apix=1.97 --binx=1 --biny=1 --df=-3 --mag=25342 --kv=300 --description="my test" --jobtype=uploadimage

... Time stamp: 13oct16w31
 ... Function name: imageloader
 ... Appion directory: /usr/lib/python2.6/site-packages
 ... Load average is high 6.1
 ... New load average 6.09
 ... Using split database
Connected to database: 'ap5'
 ... Committing data to database
Creating a new session
 ... Found project id=5 for session 13oct16z in 2.8 msec
!!! WARNING: run directory not defined, automatically setting it
 ... Run directory: /labdata/allab/leginondata/13oct16z/rawdata
 ... Writing function log to: imageloader.log
 ... Running on host: aida
 ... Uploading ScriptData....
 ... Running on host: aida
 ... Found 8 processors on this machine
 ... Running on host: aida
 ... Looking up session, 13oct16z
 ... Running Appion version 'trunk'
 ... Creating new done dictionary: imageloader.donedict

Beginning Main Loop
... reading image info

Starting image 1 ( skip:0, remain:2 ) file: 13oct16z_Oct01_12.32.50_01
... Reading original image: /labdata/allab/rogelio/tempUpload/test2/Oct01_12.32.50_0001.mrc ==== Committing data to database ====
... Load average is high 6.16

SUMMARY: imageloader
    ------------------------------------------
    TIME:         8.77 sec
    ------------------------------------------
 ... reading image info

Starting image 2 ( skip:0, remain:1 ) file: 13oct16z_Oct01_13.29.04_0020
... Reading original image: /labdata/allab/rogelio/tempUpload/test2/Oct01_13.29.04_0020.mrc ==== Committing data to database ====
... Load average is high 6.15

SUMMARY: imageloader
    ------------------------------------------
    TIME:         7.76 sec
    AVG TIME:     8.27 +/- 0.71 sec
    (- REMAINING TIME: 8.97 sec for 1 images )
    -----------------------------------------
 ... reading image info

Starting image 3 ( skip:0, remain:0 ) file: 13oct16z_Oct01_23.47.24_0237
... Reading original image: /labdata/allab/rogelio/tempUpload/test2/Oct01_23.47.24_0237.mrc ==== Committing data to database ====
... Load average is high 6.20

SUMMARY: imageloader
    ------------------------------------------
    TIME:         7.83 sec
    AVG TIME:     8.12 +/- 0.56 sec
    ------------------------------------------
COMPLETE LOOP:    24.36 sec for 3 images
 ... Load average is high 6.19
 ... Closing out function log: imageloader.log
 ... Ended at Wed, 16 Oct 2013 22:33:06
 ... Memory increase during run: 53.254 MB
Total run time:    1 min 12 sec
 ... Run directory:
 /labdata/allab/leginondata/13oct16z/rawdata

11. Click on project 4 again, the session that just created is not there

Regarding mysql databases it looks normal to me,

[rogelio@aida monomerMTs-K2]$ mysql -u usr_object -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1157
Server version: 5.5.34-log MySQL Community Server (GPL) by Remi

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases
> ;
------------------- | Database |
------------------ | information_schema | | ap1 | | ap2 | | ap3 | | ap4 | | leginondb | | projectdb | | test |
------------------
8 rows in set (0.10 sec)

hmmm...

I did get an empty list [] back after the python commands though

Python 2.7.2 (default, Oct 27 2011, 16:10:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

from leginon import leginondata

1. This gives the most recent session data object

... s = leginondata.SessionData().query(results=1)[0]

1. Print the name of the session so that you can check if it is the correct one before proceed

... print s['name']
13oct14m

from leginon import projectdata

1. This gives the project with DEF_id of 1 in your projects table as given in your post above

... p = projectdata.projects().direct_query(1)

pequery = projectdata.projectexperiments(project=p,session=s)
results = pequery.query()
print results

[]

print results

[]

results = pequery.query()
print results

[]

pequery = projectdata.projectexperiments(project=p,session=s)
results = pequery.query()
print results

[]

For some reason, you can query and insert to projectdb from myamiweb php and query and insert to leginondb through sinedon o.k. However, you can not insert through sinedon.

First, please confirm that sinedon can query projectdb. I think it does. Do this in python:

from leginon import projectdata
p = projectdata.projects().direct_query(5)
print p['name']

Second, please find out the session id number of the session you have uploaded. Easiest is to click on [summary] tag of the session. It is shown in the url as expId assignment like the 8 below:

http://localhost/myamiweb/summary.php?expId=8

You can then try the straight mysql command with that number replace the 65 at the end of this example.

CREATE TABLE IF NOT EXISTS `projectdb`.`projectexperiments` (`DEF_id` int(16) NOT NULL AUTO_INCREMENT, `DEF_timestamp` timestamp NOT NULL, `REF|projects|project` INT(20) NULL DEFAULT NULL, `REF|leginondata|SessionData|session` INT(20) NULL DEFAULT NULL, PRIMARY KEY (`DEF_id`),  KEY `DEF_timestamp` (`DEF_timestamp`),  KEY `REF|projects|project` (`REF|projects|project`),  KEY `REF|leginondata|SessionData|session` (`REF|leginondata|SessionData|session`)) ENGINE=MyISAM;
ALTER TABLE `projectdb`.`projectexperiments` ADD `DEF_id` INT(16)  NULL;
ALTER TABLE `projectdb`.`projectexperiments` ADD `REF|projects|project` INT(20)  NULL;
ALTER TABLE `projectdb`.`projectexperiments` ADD `REF|leginondata|SessionData|session` INT(20)  NULL;
INSERT INTO `projectdb`.`projectexperiments` (`REF|projects|project`, `REF|leginondata|SessionData|session`) VALUES (5, 65);

After that make it show the entries of projectexperiments with mysql command

select * from `projectdb`.`projectexperiments`;

I got these directly from what sinedon sent to mysql. If the insertion gives no error and yet the projectexperiments table is still empty, it is beyond me.

That worked!I guess this means that somewhere in the code sinedon is not doing this for the other cases?

Python 2.7.2 (default, Oct 27 2011, 16:10:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

from leginon import projectdata
p = projectdata.projects().direct_query(5)
print p['name']

Test4

mysql> CREATE TABLE IF NOT EXISTS `projectdb`.`projectexperiments` (`DEF_id` int(16) NOT NULL AUTO_INCREMENT, `DEF_timestamp` timestamp NOT NULL, `REF|projects|project` INT NULL
-> DEFAULT NULL, `REF|leginondata|SessionData|session` INT NULL DEFAULT NULL, PRIMARY KEY (`DEF_id`), KEY `DEF_timestamp` (`DEF_timestamp`), KEY `REF|projects|project` (`REF|projects|project`), KEY `REF|leginondata|SessionData|session` (`REF|leginondata|SessionData|session`)) ENGINE=MyISAM;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> ALTER TABLE `projectdb`.`projectexperiments` ADD `DEF_id` INT NULL;
ERROR 1060 (42S21): Duplicate column name 'DEF_id'
mysql> ALTER TABLE `projectdb`.`projectexperiments` ADD `REF|projects|project` INT NULL;
ERROR 1060 (42S21): Duplicate column name 'REF|projects|project'
mysql> ALTER TABLE `projectdb`.`projectexperiments` ADD `REF|leginondata|SessionData|session` INT NULL;
ERROR 1060 (42S21): Duplicate column name 'REF|leginondata|SessionData|session'
mysql> INSERT INTO `projectdb`.`projectexperiments` (`REF|projects|project`, `REF|leginondata|SessionData|session`) VALUES (5, 10);
Query OK, 1 row affected (0.28 sec)

mysql> select * from `projectdb`.`projectexperiments`;
---------------------------+----------------------+-------------------------------------+ | DEF_id | DEF_timestamp | REF|projects|project | REF|leginondata|SessionData|session |
---------------------------+----------------------+-------------------------------------+ | 4 | 2013-10-17 00:19:52 | 5 | 10 |
---------------------------+----------------------+-------------------------------------+
1 row in set (0.00 sec)

In mysql, check the table type by

show table status from projectdb where Name = 'projectexperiments';

If Engine is MyISAM then the table is normal. Don't see why sinedon can not insert. Could there be a write permission problem? We've never had problem with one single table nor single database when installation include granting permission to the database user.

You can turn on debug in sinedon to print out the queries like the one I created by changing in sinedon/sqldict.py at your installation location (python site-packages)

debug = True

There will be a lot of print out, but you can search for projectexperiments in the output.

Or maybe the python-mysql version you had to use is not working right. None of our local machine has your exact mysql and python-mysqldb version.

the engine is not MyISAM but rather InnoDB,

mysql> show table status from projectdb where Name = 'projectexperiments';
--------------------------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+ | Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |
--------------------------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+ | projectexperiments | InnoDB | 10 | Compact | 2 | 8192 | 16384 | 0 | 49152 | 10485760 | 8 | 2013-10-04 15:34:03 | NULL | NULL | latin1_swedish_ci | NULL | | |
--------------------------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+
1 row in set (0.00 sec)

how can I change it? I modified the my.cnf file stored in /etc/ according to http://emg.nysbc.org/projects/appion/wiki/Database_Server_Installation_Shared, is there any other location where I should make a change?

thanks!

Rogelio,

I believe the reason for you to have some tables created in InnoDB is because you were halfway into myamiweb SetupWizard already before changing the default. It is unfortunate that sinedon is not compatible with InnoDB. We hope to fix this some time.

Here is a good answer from google search for how http://forums.mysql.com/read.php?21,26193,190902#msg-190902

If this looks crypted, and you have not accumulated data you want to keep, you can drop all tables in your leginondb and projectdb database and start from the myamiweb SetupWIzard again which should propagates tables in MyISAM this time.

Thanks a lot! I think dropping the tables and setting them up trough the wizard again solved the problem,

thanks again for all your help,