freezer.engine.rsync.rsync module

Freezer rsync incremental engine

class freezer.engine.rsync.rsync.RsyncEngine(compression, symlinks, exclude, storage, max_segment_size, encrypt_key=None, dry_run=False, **kwargs)

Bases: freezer.engine.engine.BackupEngine

backup_data(backup_resource, manifest_path)

Execute backup using rsync algorithm.

If an existing rsync meta data is available the backup will be incremental, otherwise will be executed a level 0 backup

Parameters
  • backup_resource

  • manifest_path

Returns

static compute_checksums(rel_path, files_meta, reg_file=True)
compute_incrementals(rel_path, inode_str_struct, inode_dict_struct, files_meta, old_fs_meta_struct, write_queue, deleted=False)
static gen_file_header(file_path, inode_str_struct)

Generate file header for rsync binary data file

Parameters
  • file_path – file path

  • inode_str_struct – file binary string including inode data

Returns

chunk of binary data to be processed on the next iteration

gen_struct_for_deleted_files(files_meta, old_fs_meta_struct, rel_path, write_queue)
get_file_struct(fs_path, new_level=False)

Generate file meta data from file abs path.

Return the meta data as a dict structure and a binary string

Parameters
  • fs_path – file abs path

  • new_level

Returns

file data structure

static get_file_type(file_mode, fs_path)

Extract file type from the file mode retrieved from file abs path

Parameters
  • file_mode

  • fs_path

Returns

get_fs_meta_struct(fs_meta_path)
static get_old_file_meta(old_fs_meta_struct, rel_path)
get_sign_delta(fs_path, manifest_path, write_queue)

Compute the file or fs tree path signatures.

Parameters
  • fs_path

  • manifest_path

  • write_queue

Returns

static is_file_modified(old_file_meta, file_meta)

Check for changes on inode or file data

Parameters
  • old_file_meta – meta data of the previous backup execution

  • file_meta – meta data of the current backup execution

Returns

True if the file changed, False otherwise

static is_reg_file(file_type)
make_files(header_list, restore_abs_path, read_pipe, data_chunk, flushed, current_backup_level)

Header list binary structure:

header_len, file_abs_path, RSYNC_DATA_STRUCT_VERSION, file_mode, os_stat.st_uid, os_stat.st_gid, os_stat.st_size, mtime, ctime, uname, gname, file_type, linkname, rsync_block_size,

Parameters
  • header_list

  • restore_abs_path

  • read_pipe

  • data_chunk

  • flushed

  • current_backup_level

Returns

make_reg_file(size, file_path, read_pipe, data_chunk, flushed, level_id)

Create the regular file and write data on it.

Parameters
  • size

  • file_path

  • read_pipe

  • data_chunk

  • flushed

  • level_id

Returns

metadata(*args)
property name
Return type

str

Returns

Engine name

process_backup_data(data, do_compress=True)

Compresses and encrypts provided data according to args

process_file(file_path, fs_path, files_meta, old_fs_meta_struct, write_queue)
process_restore_data(data)

Decrypts and decompresses provided data according to args

restore_level(restore_resource, read_pipe, backup, except_queue)

Restore the provided file into restore_abs_path.

Decrypt the file if backup_opt_dict.encrypt_pass_file key is provided. Freezer rsync header data structure:

header_len, RSYNC_DATA_STRUCT_VERSION, file_mode, os_stat.st_uid, os_stat.st_gid, os_stat.st_size, mtime, ctime, uname, gname, file_type, linkname

Parameters
  • restore_resource

  • read_pipe

  • backup

  • except_queue

Returns

static rsync_gen_delta(file_path_fd, old_file_meta)

Get rsync delta for file descriptor provided as arg.

Parameters
  • file_path_fd

  • old_file_meta

Returns

set_inode(uname, gname, mtime, name)

Set the file inode fields according to the provided args.

Parameters
  • uname

  • gname

  • mtime

  • name

Returns

write_changes_in_file(fd_curr_file, data_chunk, read_pipe)
write_file(file_fd, size, data_chunk, read_pipe, flushed)