Leginon

Some more weirdness:

1. Instrument -> Tecnai is empty (no text boxes/labels displayed!) but refreshing information works. Instrument -> Camera works as usual.
   [/*:m:19q9t77g]
2. Changing between presets manually works but Presets -> Align does not:
   

   The application called an interface that was marshalled in a differnt thread.
   Preset change failed: unable to set instrument
   Acquire align image failed: cannot go to preset gr
   

   
   [/*:m:19q9t77g]
3. Beam shift calibration does not work. In Tecnai UI -> System Status, "User Shift" remains unchanged during the whole calibration procedure. But when I load a preset manually the beam shift is changed without problems, so this should be possible, in principle.
   [/*:m:19q9t77g]
4. Image shift calibration reports a 32% pixel size error at 59kx, although the pixel size is certainly correct within 2-5% (entered manually, we calculated this from a diffraction grating). If I set the tolerance to 40%, the calibration works but I am still puzzled by the discrepancy.
   [/*:m:19q9t77g]
5. Saving a camera sensitivity in the Calibration->Dose node is not possible. The text box is always empty after closing and opening the dialog and taking a dose image in the presets manager is not allowed.[/*:m:19q9t77g][/list:u:19q9t77g]

Could it be that I use the wrong version of Leginon (it reports version 1.6) - SVN revision 6237 from November? Would you recommend another version?

Mathias,

Your problems are very weird. Leginon 1.6 is a pretty good version. You can update to the latest on that branch which contains some very minor bug fixes unrelated to what you do. Jim will be better in answering threading problems. Leginon is obviously not talking to Tecnai properly.

For some of the GUI funnies, could you tell us what is the version of python, wxpython that you are running? What Linux distribution are you on?

About beam shift calibration, refresh instrument should give us the value that displayed in the box called Beam Shift on the instrument node panel at the Leginon client on your scope PC. If this is not a number in the range of 1e-7 or below but in the range of 0.02, your beam shift is not reading in a physical space value like Image Shift does. We just noticed such behavior at another place. In this case, you will not be able to do beam shift matrix calibration unless you can find the conversion and hard code it in. This calibration is optional, so it should not stop you.

About image shift calibration determines that the pixel error is much larger, what is the pixel size you put in for the particular microscope magnification? Remember that it should be that on the unbinned ccd. If you are sure it is right, this means your microsope image shift which FEI calibrated when it was made is very off.

About Dose calibration gui, it behaved right. After you enter a number for the sensitivity, you should click "Save". There would be message in the main Leginon message logger that says Camera sensitivity saved as xxxx counts/e at xxx kV. The number you enter is not part of the node settings and is not saved in the database in the way that it would redisplayed when you close/open. If it does not have the message in the log, it may have problem figuring out the instrument information as happens in your other cases.

"Grid Targeting" and "Save location" problem probably is related to your other problems in presets manager and the preset change failure.

Anchi

Anchi,

thanks for getting back to me.

"anchi" wrote: Mathias,
Your problems are very weird. Leginon 1.6 is a pretty good version. You can update to the latest on that branch which contains some very minor bug fixes unrelated to what you do. Jim will be better in answering threading problems. Leginon is obviously not talking to Tecnai properly.

I did a svn diff, there are no changes in pyScope and minor ones in Leginon.

"anchi" wrote:
For some of the GUI funnies, could you tell us what is the version of python, wxpython that you are running? What Linux distribution are you on?

I run the calibrations/manual application fully on a Windows machine to rule out network issues (WinXP SP 2), wxpython 2.8.10.1(ansi) for python 2.5, python 2.5.
I can now reproduce the problem - the dialog boxes vanish only if you open the instrument node when the Leginon window is sized to full screen (1600x1200) and only if the node has not been opened before during the session.
Now that I know this, it is easy to work around - should I file a bug report nevertheless?

"anchi" wrote:

About beam shift calibration, refresh instrument and give us the value that displayed in the box called Beam Shift. If this is not a number in the range of 1e-7 or below but in the range of 0.02, your beam shift is not reading in a physical space value like Image Shift. We just noticed that at another place. In this case, you will not be able to do such calibration unless you can find the conversion and hard code it in. This calibration is optional, so it should not stop you.

Unfortunately, it is in the latter range (0.006, 0.008) - the value is equal to what is displayed in Tecnai UI -> System Status -> User Shift. Do you know anything specific about this conversion?

"anchi" wrote:
About image shift calibration determines that the pixel error is much larger, what is the pixel size you put in for the particular microscope magnification? Remember that it should be that on the unbinned ccd. If you are sure it is right, this means your microsope image shift which FEI calibrated when it was made is very off.

It is the pixel size of an unbinned CCD - what FEI calibration do you mean (our EM facility manager was present during on-site construction/alignment and does not recall any image shift calibration)?

"anchi" wrote:

About Dose calibration gui, it behaved right. After you enter a number for the sensitivity, you should click "Save". There would be message in the main Leginon message logger that says Camera sensitivity saved as xxxx counts/e at xxx kV. The number you enter is not part of the node settings and is not saved in the database in the way that it would redisplayed when you close/open. If it does not have the message in the log, it may have problem figuring out the instrument information as happens in your other cases.

"Grid Targeting" and "Save location" problem probably is related to your other problems in presets manager and the preset change failure.

No message in the log, unfortunately.

How would you suggest to tackle these communication problems? Is there any test script or would you just recommend to use step-by-step debugging (which I had hoped to avoid)? Is there anyone with a similar setup who was successful?

Thanks again for your help!

Details on our setup:
Tecnai F30 Polara , Gatan US 4000
FEI TEMScripting/TecnaiScripting version 3.1.1
Digital Micrograph version 3.9.5 for GMS 1.4.5
WinXP SP2

I just wanted to add that at least the saving/recalling of stage positions and taking alignment images works as soon as I run leginon main on linux instead of running both main and scope on the Windows microscope PC...How can this be?

Atlas creation still does not work...Simulate target in the Grid node works (I had to add a preset in the properties - presets order).
But when I publish an atlas messages in the grid node are:

Acquisition.processData done
Continuing...
Grid done with target list ID:(('linux',49155(,1944018), status: success
9 targest will be passed to another node
Acquisition.processData

The Process ("Play") button remains deactivated, I can pause and stop the process but nothing happens...the stage does not move even though I set "modeled stage position" as "move to target". No images are acquired (in contrast to simulated target). Changing the Mover option also does not help...
Is there anything that I am missing here?

For the grid atlas problem the message "targets will be passed to another node" indicates that the Grid node is not configured to handle the type of targets that are being created by the Grid Targeting node. In the settings for Grid, check the setting "Process ...... targets" and make sure it is set to "acquisition". That should be the default, but maybe got switched by accident.

Still not sure about the problem with the align image shift, but I will be testing it this week. It seems to be something wrong with how we do multi-threading access to the microscope and camera. We do not usually do so many things with MSI completely on Tecnai, but I will test this case to see if I can reproduce. Are you ok using linux for the main component until this is fixed? This is our usual mode of operation, so it is well tested.

Jim

"pulokas" wrote:

Still not sure about the problem with the align image shift, but I will be testing it this week. It seems to be something wrong with how we do multi-threading access to the microscope and camera. We do not usually do so many things with MSI completely on Tecnai, but I will test this case to see if I can reproduce. Are you ok using linux for the main component until this is fixed? This is our usual mode of operation, so it is well tested.

Jim

Hi, Jim!

Switching the main process to Linux fixed some of the problems: Atlas creation, saving stage positions and saving a camera sensitivity (!) is now possible...I could not check yet for the other problems (like pixel size error)...What I also noted is that memory consumption of the python process is less for the same tasks in linux...

We will continue with this setup now (it just took until yesterday to change the network architecture so we can run main on linux ). Finally I can explore all of Leginon's functionality - still a lot of work ahead <img src="{SMILIES_PATH}/icon_cool.gif" alt="8)" title="Cool" />

Another question: When I select and submit a square in the square targeting node, I subsequently see the correct square in hole targetting - but the orange cross (position) in square targeting is way off (outside of the atlas even) - not at all reflecting the true stage position (Leginon managed to go to the right square but does not seem to take notice)

Thanks for your help!

About the "orange cross" that shows the current position. It is not refreshed automatically. You will need to click the "refresh" button (two green curved arrow chasing each other) or "show position" (red cross on an orange tilted atlas) in your "Square Targeting" node to have it refreshed. The "refresh" button retrieve also the targets you've submitted and display them on the atlas according to its current status and it is important that you click on it "BEFORE" you choose new targets again so that when you submit the targets, Leginon will not think the old ones need to be redone.

Anchi

"anchi" wrote: About the "orange cross" that shows the current position. It is not refreshed automatically. You will need to click the "refresh" button (two green curved arrow chasing each other) or "show position" (red cross on an orange tilted atlas) in your "Square Targeting" node to have it refreshed. The "refresh" button retrieve also the targets you've submitted and display them on the atlas according to its current status and it is important that you click on it "BEFORE" you choose new targets again so that when you submit the targets, Leginon will not think the old ones need to be redone.

Anchi

I both hit the refresh and the show position button. It is still way off, not within the image. But square targetting works nevertheless...

And, before I call it a day: focus targets are ignored although I set "process focus nodes" in "Focus" and have an enabled Focus sequence (simulate target works)...is there another checkbox I forgot?

All the best from Vienna!

If you have chosen the focus target in "Square Targeting" node, it would not do anything because we didn't bind the event to either of the Focuser nodes. The "Z Focus" accepts targets submitted from "Hole Targeting" node and the "Focus" accepts targets submitted from "Exposure Targeting" node.

Regarding what you said about "focus targets are ignored although I set "process focus nodes" in "Focus" " and the fact that you had a wrong type of target assigned in "Grid" node, it makes me wonder: Aren't these set correctly as default when you start the application the first time? If not, you probably missed the step of importing defaults during the installation. There are so many settings need to match right in the nodes to coordinate their actions, if you missed that step, many things do go wrong. To tell that you missed the step, open the settings of an acquistion or focus node you have NEVER opened, it should have selected an appropriate preset for you even if you have not. Some of the check boxes would have been checked. If you start making changes, and close the settings window, the defaults won't apply any more.

Anchi

"anchi" wrote: To tell that you missed the step, open the settings of an acquistion or focus node you have NEVER opened, it should have selected an appropriate preset for you even if you have not. Some of the check boxes would have been checked. If you start making changes, and close the settings window, the defaults won't apply any more.

Anchi

I think I missed loading the defaults because I had to change many things to get MSI working, for instance all nodes with a camera icon were set to process "focus nodes". When I installed Leginon no applications were available, therefore I imported the xml files from Leginon/applications...How do I get to the defaults?

-Matthias

default loading can be done any time before you use the applications the first time. In other words, it does not matter whether the applications are uploaded, yet.

The instruction is at
http://ami/documentation/leginon/bk02ch06s02.php

It is just a two step procedure.

Unfortunately, the damage is already done to you as a user. The easiest way without involving you go into the database to delete tables and tables of stuff is to create a new Leginon user and use that instead after you complete the default loading. You can rename the old user first from
your Leginon admin webpage "adduser.php". so that you can use your familiar name again. You need to change the field "full name" and the field "name". The former is what you have put in leginon.cfg. The latter is the "username" which need to be unique so that it won't look confusing in the Leginon admin webpage. Once that is done, you can create a new unique user.

Anchi

"anchi" wrote: default loading can be done any time before you use the applications the first time. In other words, it does not matter whether the applications are uploaded, yet.
The instruction is at
http://ami/documentation/leginon/bk02ch06s02.php
It is just a two step procedure.

Anchi,

thanks for your replies.

I loaded the defaults. I installed the MSI-Tomography application (which has not been there before). The nodes Square_Q, Hole_Q and Tomography, Tomography_Preview are used only by this application, AFAIK. All of them are set to process focus targets? This is not right, is it? (I tried this with the same and a new user)

If I execute the following:

DELETE FROM AcquisitionSettingsData;
DELETE FROM BeamFixerSettingsData;
DELETE FROM BlobFinderSettingsData;
DELETE FROM CenterTargetFilterSettingsData;
DELETE FROM ClickTargetFinderSettingsData;
DELETE FROM CorrectorSettingsData;
DELETE FROM DTFinderSettingsData;
DELETE FROM FocuserSettingsData;
DELETE FROM FocusSequenceData;
DELETE FROM FocusSettingData;
DELETE FROM HoleFinderSettingsData;
DELETE FROM JAHCFinderSettingsData;
DELETE FROM LowPassFilterSettingsData;
DELETE FROM MosaicClickTargetFinderSettingsData;
DELETE FROM MosaicTargetMakerSettingsData;
DELETE FROM RasterFinderSettingsData;
DELETE FROM RasterTargetFilterSettingsData;
DELETE FROM RCTAcquisitionSettingsData;
DELETE FROM TomographySettingsData;

and, after that, run load defaults php, some of these tables (AcquisitionSettings, TomographySettings, RCTAcquisitionSettings), remain empty...I am not so sure whether load defaults ever worked properly for me, specifically with all acquisition nodes (because I vaguely recall having loaded the defaults during installation).

I tracked this down, this seems to be a bug in setdefault.php. When I php echo the query it is going to submit, I get for instance the following:

INSERT INTO `AcquisitionSettingsData` (`correct image`, `SEQ|preset order`, `wait for rejects`, `move type`, `save image`, `pause time`, `display image`, `duplicate targets`, `wait for process`, `duplicate target type`, `REF|SessionData|session`, `name`, `wait time`, `iterations`, `adjust for drift`, `isdefault`, `mover`, `move precision`, `process target type`, `save integer`, `drift between`, `final image shift`, `accept precision`, `wait for reference`, `wait for transform`, `adjust for transform`) VALUES (1, '[u'gr']', 0, 'modeled stage position', 1, 1.5, 1, '', 0, '', 101, 'Grid', 0, 1, '', 1, 'presets manager', 0, 'acquisition', 0, 0, 0, 0.001, 0, '', 'no');

This causes a syntax error in mysql which is not reported at all! (I had to check it manually at the mysql prompt):

ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'gr']', 

This will likely fail for all acquisition nodes because the preset name is always escaped in the wrong way. ( '[u'gr']' )

sed -i "s/'/\\\\'/g" setdefault.php

Matthias,

If you do what you suggest, you will get backslash on all single-quotes and that would not work. The same adddsettings.php and inc/setdefault.php here works since there is an addslashes() function in inc/mysql.inc.

My guess is that you have magic_quotes_gpc ON while we have it OFF. Despite some php documentation says it is default to ON, it is actually default to OFF.
Check your info.php. The table on PHP Core Configuration should have the field and the value.
If this is the case, you can change that in your php.ini and then restart your webserver.

The more permanent solution is to fix this bug by removing the cause that checks for magic_quotes_gpc in inc/mysql.inc since we pass data directly.
Open inc/mysql.inc of your dbemtools, and search for get_magic_quotes_gpc().
Edit the surrounding lines to this:

    foreach($data as $k=>$v)

if (!is_numeric($v))

$data[$k] = "'".addslashes($v)."'";

$q = "INSERT INTO `$table` (".implode(', ',$fields).") VALUES (".implode(', ',$data).")";

Anchi

Thanks, turning off magic_quotes did the job!

Is the SQL script above (truncation of all tables that are mentioned in setdefault.php - I figure these are the node settings tables) safe to execute when I want to reset all node settings for all users?

Unfortunately, the default is loaded to the administrator user. You can not reset a user's settings this way because once a user makes schanges, Leginon reads that first. The defaults are only read when Leginon query for a node (an instance) of the same name of the same class yields no result on the particular user. Therefore, it is only useful for new user.

Unless...., you go into the xml file and make a series of changes so that instead loading the values to the administrator, it loads to the said user. That would be an advanced topic. <img src="{SMILIES_PATH}/icon_wink.gif" alt=":wink:" title="Wink" />

Anchi