Leginon

Hi, Patrick,

There are a few levels here.

image and frames acquired:¶

First, the location where the files are saved is in leginon.cfg. If you make it so that each user has their leginon.cfg in his/her home directory, then the session is saved according to their own
configuration by default. See Configure_leginoncfg

You can handle read restriction with LDAP user restriction. The redux server and rawtransfer.py are run as root so it should be possible to set to read-only by group but not by others.

One problem is the reference sessions that need to be readable by all users which is a bit tricky because files can accumulate in there over time so we have a mechanism to make a new one once it is 90 days old. See Reference_session I will need to improve on this to give it a configurable behavior.

Image viewing in the webserver:¶

These are taken care of with the log in to myamiweb. Users in users group can only view and process the project they own. See Recommandation_for_setup_at_a_new_institute
The image service, redux, runns as root, need read-access to all users' image directories, of course.

running leginon sessions:¶

start-leginon.py is in /usr/bin so every LDAP users have access. Each person just start their leginon session with their LDAP account. It is indeed a problem if a second person starts it when the first one is still running. However, our experience is that since microscope time is booked and controlled. It is not possible that two persons will be booked on the same system (meaning connect to Leginon client on the scope/camera). Only developer, such as me, causes problem occassionally.

Hi Anchi,

For Image viewing in the webserver, is it possible to group some regular users (say, users from the same lab) together to let them have the access to the same dataset or each other' data?
It seems to me that project ownership has that function, but I'm 100% sure whether it let different user to use the same project for different session (but not accessible to others) or all the sessions (even collected by different users) in an individual project are accessible to all owners of that project.
Thanks.

Sorry, typo.
"I'm not 100% sure whether it let different user to use the same project for different session (but not accessible to others) or all the sessions (even collected by different users) in an individual project are accessible to all owners of that project.".

Yes, a project will allow multiple owners, and the owners get access to all sessions in that project even if not collected by them.

A non-owner typically can not see sessions of the project, unless one of the owner of the project decide to share one particular session to him/her.

Say there are persons A, B, and C. A and B are owners of project1, not C. A and B see all sessions in project1 even though A is the one created the sessions.
C does not see any of these sessions. However, A and B may decide that they want C to see session1 in the project1, so they share session1 to C.

These assumes that all these users are in "Users" group whose access privilege is limited. Someone in "Administrators" group have access to all session from all projects all the time.

Now that I've been experimenting with this, I have some more focused questions.

What is the purpose of being able to add groups in the myamiweb interface?

• All at administration level
• View all but adminstrate owned
• Administrate/view only owned projects and view shared experiments
• View owned projects and shared experiments
• No privilege for anything

Our preferred configuration is to allow all members of a group be able to view every project that group has, but there doesn't seem to be any way to facilitate this. If we create a group and set the privileges to "Administrate/view only owned projects and view shared experiments" this seems to be exactly the same as "View owned projects and shared experiment"; namely a user can only view projects that s/he is on; i.e. group membership has no effect, so it's not clear why the ability to create new groups is included in the interface.

Next Questions

For testing purposes we only have /etc/myami/leginon.cfg, with relevant setting

[Images]
path: /cryodata/leginon

When images are first taken, the session goes into a Samba-mounted share from the Camera PC, and are owned by root with 777 permissions (probably nothing to be done about that, since Windows doesn't know about POSIX permissions).

Then rawtransfer.py moves the session to ... 2 different locations? I have destination_head=/cryodata, session = 18apr19a, and end up with:

cnsit@kraken:/cryodata/leginon$ ls -ld 18apr19a
drwxrwxr-x 4 yizhou yizhou 47 Apr 19 11:38 18apr19a
cnsit@kraken:/cryodata/leginon$ cd ../frames
cnsit@kraken:/cryodata/frames$ ls -ld 18apr19a
drwxrwxr-x 3 yizhou yizhou 29 Apr 19 11:38 18apr19a

The file ownership is set to whomever launched start-leginon.py, but I'm not sure about the file permissions -- is this determined by the user's umask setting? But in any case, the permissions on the mrc files are inconsistent:

cnsit@kraken:/cryodata/leginon/18apr19a$ ls -l rawdata
total 65536
-rw-rw-r-- 1 yizhou yizhou 56956944 Apr 19 11:38 18apr19a_00001ex.count.mrc
cnsit@kraken:/cryodata/leginon/18apr19a$ cd ../../frames/18apr19a
cnsit@kraken:/cryodata/frames/18apr19a$ ls -l rawdata/
total 556216
-rwxrwxrwx 1 yizhou yizhou 569561248 Apr 19 11:38 18apr19a_00001ex.count.frames.mrc
drwxrwxr-x 2 yizhou yizhou       297 Apr 19 11:38 references

One strange thing. When we run start-leginon.py, and if the user doesn't have a local leginon.cfg file, then it asks for a user/project. If we select a user different from yizhou, then even though yizhou owns the files in the filesystem, he won't be able to access the session from the web interface. Just confirming that this is the way it works.

OK, so now for the questions:

As mentioned, the PI of one of our research groups has requested that everyone in his group be able to access the raw data that is collected for them, but that only he have write permissions in order to guarantee data integrity. Of course it's generally going to be grad students and postdocs running data collection. We could set up a group user account that everyone from that groups uses to collect data, have a local leginon.cfg file for this user which puts the images into a different folder (say /cryodata/johns-lab) which has an appropriate guid bit set, but then how would the data be accessible from the web interface? I'm a bit confused about how the web interface finds the raw data, given that only the Appion folder is configured in ~config.php. Is the image path recorded in the database by start-leginon.py at the time of image collection?

One PI has requested that no one but members of his research group be able to see this group's raw data. Again, using a group microscope user account I can lock down the folder permissions on the folders that the mrc files are moved to, but I think this definitely means that they won't be accessible by the web interface? If I make the www-data user a member of his group, then possibly anyone will be able to access the raw data through the web interface, or is it safe to rely on the in-application permissions system?

Creating new group was there so you can separate users by institution, originally for accounting purpose. Regarding Administrate vs view privilege. Think of administrate privilege as the write access. You can add another owner if this is at the project level. When a user is in guest group which typically only allow to view, is not allowed to bring in another guest into the project.

Regarding file permissions,

When a session is created in leginon, the frames directory of the session is also created to ensure ownership. When running as root, rawtransfer.py sets the ownership and permission of the frames files transferred or moved according to ldap user that created the session. You can read the code of that file under leginon/rawtransfer.py. I don't see in there a place that it changes the permission, so it is just the default behavior of your umask. As far as mismatching ownership, It has no knowledge of leginon settings and web access users. Unless you require your users always creates only their own session, there is no way to sync them. We do that locally by setting up a leginon-side user and per user leginon.cfg when they sign up ldap account. For those who intend to collect data for another leginon-user, he/she should either also own the project or below to a privilege group with more power such as "power users".

Regarding web access of group-restricted files:

The images are found and served by redux image server which typically run as root so it should not have trouble reading images that have group restriction at file level. And Yes, image path is recorded in the database by start-leginon.py at the time of image collection.

The web pages that request these images have privilege checks in there so that you can not access someone else's session by changing the url. However, if someone login on the same web browser but not logged out, then the next person inherit his/her privilege, so it is up to the user to be careful about that.

Regarding the group-based access behavior of the viewer you want:

Group-based access is something we have not considered. The behavior you want calls for a "Project Group" more like the file system. With that, we can make use of the privilege of the "power user" within that "project group" but not outside the project group. What is needed is a link from project, not user to the "project group", then query can be written to do this. We need to brainstorm on this a bit. We have a case that need this, too.

Hope I have gotten everything