Create snapshot fails when file is too big

Description

Create snapshot fails when file size is bigger than 219 MB
We get the following error:
017-01-19T08:05:49 CFY <None> Starting 'create_snapshot' workflow execution
2017-01-19T08:06:04 LOG <None> INFO: Dumping elasticsearch data
2017-01-19T08:22:28 LOG <None> INFO: Dumping credentials data
2017-01-19T08:27:26 LOG <None> INFO: Preparing agents data
2017-01-19T08:30:22 LOG <None> INFO: Creating snapshot archive
2017-01-19T08:30:23 CFY <None> 'create_snapshot' workflow execution failed: LargeZipFile: Filesize would require ZIP64 extensions
Execution of workflow create_snapshot for deployment None failed. [error=Traceback (most recent call last):
File "/tmp/pip-build-3oyjhc/cloudify-plugins-common/cloudify/dispatch.py", line 472, in _remote_workflow_child_thread
File "/tmp/pip-build-3oyjhc/cloudify-plugins-common/cloudify/dispatch.py", line 504, in _execute_workflow_function
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify_system_workflows/snapshot.py", line 238, in create
_create(snapshot_id, config, **kwargs)
File "/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify_system_workflows/snapshot.py", line 226, in _create
tempdir
File "/usr/lib64/python2.7/shutil.py", line 549, in make_archive
filename = func(base_name, base_dir, **kwargs)
File "/usr/lib64/python2.7/shutil.py", line 456, in _make_zipfile
zip.write(path, path)
File "/usr/lib64/python2.7/zipfile.py", line 1137, in write
self._writecheck(zinfo)
File "/usr/lib64/python2.7/zipfile.py", line 1102, in _writecheck
raise LargeZipFile("Filesize would require ZIP64 extensions")
LargeZipFile: Filesize would require ZIP64 extensions

Activity

Show:
Javier Collado Cabeza
January 31, 2017, 2:42 PM
Edited

Jonathan, please find attached to the issue a patch that I've verified for 3.4patch1.

The procedure I've followed to make sure that the patch work has been basically index a lot of data into Elasticsearch, so that the snapshot files gets really big. Below you can see that the problem was reproduced before applying the patch and that the snapshot succeeded after applying the patch:

The procedure to apply the patch is as follows:

  • Log into the manager

  • Change to management worker directory
    cd /opt/mgmtworker/env/lib/python2.7/site-packages/

  • Make sure patch is installed:
    sudo yum install -y patch

  • Apply patch
    sudo patch -p2 < CFY-6333-enable-zip64-in-snapshots.patch

  • Restart the management worker
    sudo systemctl restart cloudify-mgmtworker

This will update:
/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify_system_workflows/snapshot.py
and copy the original file to:
/opt/mgmtworker/env/lib/python2.7/site-packages/cloudify_system_workflows/snapshot.py.orig

In case of any problem, copy the .py.orig file back to the .py file and restart the management worker to get to the state before applying the patch.

Fixed

Assignee

Javier Collado Cabeza

Reporter

Jonathan Abramsohn

Labels

Bug Type

None

Target Version

None

Severity

None

Sprint

None

Fix versions

Affects versions

Configure