Project

General

Profile

Actions
Copy link

Bug #7812

open

Download particle data takes too much memory

Added by William Rice over 5 years ago. Updated over 5 years ago.

Status:
In Code Review
Priority:
Normal
Assignee:
Gabriel Lander
Category:
-
Target version:
Leginon - Appion/Leginon 3.5
Start date:
07/17/2019
Due date:
% Done:

0%

Estimated time:
Affected Version:
Appion/Leginon 3.4
Show in known bugs:
No
Workaround:

Description

Downloading particle data as csv file takes about 30X as much memory as the final file size. For example, 1 GB RAM was not enough for a 527,000 particle download.

Made changes to downloadparticledata.php to use standard php functions to create csv file.
Added inc/datatocsv.inc which includes this function.
Downloading particle data now can be done with standard 1 GB RAM allocation to php webserver.
One consequence is that I eliminated the special tratment for "ef" presets. Likely this is not used anymore.

Other csv downloads could be adjusted to use this file, but they are not so large so perhaps it is not worth doing,

Actions
Copy link

Also available in: Atom PDF