Leginon

Hi, Morgan,

I did a test upgrade, you should have fairly smooth update.

Transferring configuration/settings should be no problem. All features I added are back-compatible recently. You need to run dbschema/schema_update.py and follow those instruction. As usual, do make a backup of your database before you do that, just in case.

You will have to create some instrument configurations now. Configuration such as fei.cfg/dmsem.cfg/pyami.cfg would be new to you. There are template files for them with the behavior of mostly the old version, so it should not be difficult.

Most people have a bit of trouble with the new ip mapping configuration pyami.cfg since it only works if you set all computers properly. It is worth doing, though. It makes sure that they communicate through fixed network cards .

CentOS 7 is recommended now. The main change is the use of mysqli in the webserver, but that should be available in CentOS 6.5 as well. Python requests package is used for Slack notification and leginon-remote but both are optional. It is possible to install requests on CentOS 6, but it is included in python2.7 that comes with CentOS 7 so it is easier. My intuition is that CentOS 6.5 will still work, but I have not tested it on that for a while.

Hi Anchi,

Ok, thanks for this. I've made a note and will consult it when I move onto the install.

One major remaining question, however, is: how do I go about migrating my existing Leginon install to a new OS when I install CentOS 7? Will is simply be a matter of copying key directories/files? I can't find anything in the Leginon docs about this?

thanks,

Morgan

Migration mainly means database migration and image file migration.

Database contains the metadata of the image files such as mag/camera exposure time, and more importantly your user settings and calibrations. I created this page for you and others who want to have a clean start Database_migrating_exportimport_scripts I have been improving these over the years. I think it is good now.

If you are keeping all files and databases in this process but just need to change the session image path. Use myami/dbschema/tools/batch_leginon_path_change.py and its equivalent batch_appion_path_change.py

Hi Anchi,

Excellent. So if I understand correctly, I can do a fresh Leginon 3.5 install on my new CentOS 7 box, and just copy the mysqldump/restore. Presumably no need for a schema_update.py .

No need to reply unless I'm wrong about this!

Planning on ordering a new server in the coming days.

best,

Morgan

You should still run schema_update.py after restore. It will tell you what it needs to do.

Anchi,

We've received delivery of our new microscope so while it's being assembled I'm working on migrating from Leginon 3.1 to 3.5.

Apologies for an anxious message, but I wanted to make sure everything is OK. I've mysqldumped and imported my database, then I've run schema_update.py -- but it suggests nothing needed doing, which surprised me. Did I miss something...?

many thanks,

Morgan

@
[root@la-ruska ~]# /tmp/myami/dbschema/schema_update.py
connected to db 'projectdb' on server 'localhost'
Branch name ''
Commit count is 0
getDatabaseRevisionOLD()
update schema table in database from legacy version
getDatabaseRevisionOLD()
new branch: reset revision
0
Warning: git version is possibly too old for tag option --merged. Trying getAvailableTagsForBranchOLDGit()
Warning: git version is possibly too old for tag option --merged. Trying getAvailableTagsForBranchOLDGit()

list of schema tags to check:
[]

No database schema updates are needed@

Let me propogate databases with myami-3.1 to check this.

What is your git version ?

Also, is this on CentOS 7.x or 6.5 My guess is 7.

You might be lucky, but it seems that it will work as is. The only requirement for me to run 3.5 is to start new session since the tomography prediction model has changed. It could not load the old tilt data in the old session.

Most dbschema update has to do with intermediate database schema that we made correction to past data so that they are compatible. It seems that 3.1 is old enough that it skipped these intermediate versions. My test only asked me to the most recent one which would not be a problem for you since we have not started enforcing it and new sessions will add those column automatically for you through sinedon.

It is bad though that it could not determine even the branch name. Please let me know the followings:

1. git version
2. What does projectdb table install looks like.
3. Is there a table called schemaupdates generated during your effort in your projectdb.

If the 3rd item is not there, the update process failed to back-fill update records. We should try to fix that.

Hi Anchi,

Many thanks for looking into this. In answer to your questions:

1. git version

git version 1.8.3.1

2. What does projectdb table install looks like.

MariaDB [projectdb]> show tables;
+---------------------+
| Tables_in_projectdb |
+---------------------+
| boxtypes            |
| confirmauth         |
| dataStatusReport    |
| gridboxes           |
| gridlocations       |
| grids               |
| install             |
| privileges          |
| processingdb        |
| projectexperiments  |
| projectowners       |
| projects            |
| shareexperiments    |
| userdetails         |
+---------------------+
14 rows in set (0.00 sec)

3. Is there a table called schemaupdates generated during your effort in your projectdb.

As you can see, no schemaupdates table in the projectdb database.

thanks for your help,

Morgan

Regarding install table, I meant for you to run

select * from projectdb.install;

Another test I would like you to run is to run this in your /tmp/myami/dbschema

git branch | grep '\*' | cut -d' ' -f2-

If this is a git clone as it should be from autoinstaller, you should get the branch name myami-3.5 instead of empty string as your output has shown. I tried it on the same git version we have, and it worked right.

Hi Anchi,

This is what I get:

MariaDB [(none)]> select * from projectdb.install;
+-----------+-------+
| key       | value |
+-----------+-------+
| settable  | 1     |
| version   | 3.1   |
| revision  | 18535 |
| revision  | 18535 |
| resetfrom | 0     |
+-----------+-------+
5 rows in set (0.06 sec)

For

git branch | grep '\*' | cut -d' ' -f2-

I'm afraid I get...

fatal: Not a git repository (or any of the parent directories): .git

FYI I got the autoinstaller with:

wget https://emg.nysbc.org/projects/appion/repository/revisions/trunk/raw/install/centos7AutoInstallation.py

thanks...

Morgan