Project

General

Profile

Actions

Feature #11984

closed

transfer frames in batch of sessions

Added by Anchi Cheng over 2 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
02/02/2022
Due date:
% Done:

0%

Estimated time:
Deliverable:

Description

This feature is used in case when frames are accumulated on the camera host due to either network or permanent storage used for the detination is not available.

A few things can be made more efficient if we don't have to consider new files streaming in.

The live-streaming version need to do inefficient text search. It is much faster to match in python set.interception the frames name of the session query results that specify the camera host and image data.

This new feature also first move all frame files into a tmp directory on the source, and then do rsync or mv of the whole directory. rsync is faster with directory.

Actions #1

Updated by Anchi Cheng about 2 years ago

Usage: batchtransfer.py [options]

Options:
-h, --help show this help message and exit
--method=METHOD method to transfer, e.g. --method=mv
--source_path=PATH Mounted parent path to transfer, e.g.
--source_path=/mnt/ddframes
--camera_host=CAMERA_HOST
Camera computer hostname in leginondb, e.g.
--camera_host=gatank2
--destination_head=PATH
Specific head destination frame path to transfer if
multiple frame transfer is run for one source to frame
paths not all mounted on the same computer, e.g.
--destination_head=/data1
--path_mode=MODE_STR recursive session permission modification by chmod if
specified, default means not to modify e.g.
--path_mode=g-w,o-rw
--check_days=CHECK_DAYS
Number of days to query database

Actions #2

Updated by Anchi Cheng over 1 year ago

  • Status changed from Assigned to Closed

works and used in a few cases

Actions

Also available in: Atom PDF