Project

General

Profile

Minimum Requirements for frame-saving direct detectors » History » Version 4

Anchi Cheng, 07/02/2018 11:05 AM

1 3 Anchi Cheng
h1. Example all-in-one configuration for frame-saving direct detectors
2 1 Anchi Cheng
3 4 Anchi Cheng
This configuration has a strong hardware.  If frame and any other post-collection processing will be done else where, this can be reduced.
4 1 Anchi Cheng
5 2 Anchi Cheng
h2. Using gpu-based frame alignment:
6 1 Anchi Cheng
7 2 Anchi Cheng
A good gpu is needed for frame alignment using the program described in Li et. al. (2013) Nat. Method vol. 10 pp584-590 and variants of it.
8
If real-time speed is desired, parallel processing on multiple hosts may be needed.
9
10
Here is an example that we designed as a back up for our typically distributed system.  Good for a few days of data collection of Krios/K2 combination.
11
12 1 Anchi Cheng
- Advanced HPC Mercury GPU408 4U rack mountable workstation
13 2 Anchi Cheng
- Dual 2.1GHz 8-core CPUs (16 cores total can run database, webserver, leginon, and Appion pre-processing and very minimal post-processing)
14 1 Anchi Cheng
- 128 Gb RAM
15
- 1 Tb SSD scratch space
16
- GTX 1050 GPU for graphical display.
17
- Two GTX 1070 GPU cards (will support real-time frame alignment in counted mode)
18
- 12 Tb of storage in RAID configuration (will support 3-4 days of data collection)
19
- 10GB network card to support direct transfer from K2 camera and connection to the scope.
20 2 Anchi Cheng
21
To expand from this:
22
23
* Add more storage space or connect to a network storage solution.
24
* Add more gpus or have a over-flow frame processing computer to share.
25
26
h2. Recommendation:
27
28
Have one primary frame processing computer per direct detector to be used by the person currently using the scope.  And an over-flow frame processing computer to share among scopes and users not able to finish during their session.  Our experience is that primary frame processing computer can not keep up with super-resolution data collection if binning is not applied ahead of the time. The shared gpu can also be used for gCTF, gAutomatch and other light-weight gpu programs.  Counted mode movies can be processed fast enough with just primary frame processing computer.
29
30
h2. For CPU programs that do frame alignment:
31
32
A 12 core-computer with Torque scheduler can be used for DE-20 frame alignment.  The rest can be used for Leginon data collection if needed.  We have not tried it, though.