inplace upgrade from 4.1.1 is broken

Description

`cfy agents install` fails after restoring a snapshot from version 4.1.1 with installed deployment.
We get an internal error and the execution is stuck on pending.

Reproduce:
1. Create a 4.1.1 manager (bootstrap or image).
2. Install a deployment (for example: hello-world-singlehost).
3. Create and download a snapshot.
4. Re-create a 4.2 manager on the same ip (teardown and bootstrap or rebuild instance).
5. Upload and restore the snapshot (with `--restore-certificates`).
6. Try to install agents (`cfy agents install` will probably fails).

Stack trace from manager log:

09/11/2017 11:33:22 [ERROR] [manager_rest.server] Internal Error
Traceback (most recent call last):
File "/opt/manager/env/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/opt/manager/env/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/manager/env/lib/python2.7/site-packages/flask_restful/_init_.py", line 261, in wrapper
resp = resource(*args, **kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/flask/views.py", line 84, in view
return self.dispatch_request(*args, **kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/flask_restful/_init_.py", line 341, in dispatch_request
resp = meth(*args, **kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/security/secured_resource.py", line 31, in wrapper
return func(*args, **kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/rest/rest_decorators.py", line 95, in wrapper
return func(*args, **kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/security/authorization.py", line 69, in wrapper
return func(*args, **kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/rest/rest_decorators.py", line 135, in wrapper
response = f(*args, **kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/rest/resources_v1/executions.py", line 118, in post
bypass_maintenance=bypass_maintenance)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/resource_manager.py", line 396, in execute_workflow
bypass_maintenance=bypass_maintenance)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/workflow_executor.py", line 55, in execute_workflow
context=context)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/workflow_executor.py", line 102, in _execute_task
kwargs=execution_parameters)
File "/opt/manager/env/lib/python2.7/site-packages/manager_rest/celery_client.py", line 69, in execute_task
kwargs=kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/celery/app/base.py", line 353, in send_task
reply_to=reply_to or self.oid, **options
File "/opt/manager/env/lib/python2.7/site-packages/celery/app/amqp.py", line 305, in publish_task
**kwargs
File "/opt/manager/env/lib/python2.7/site-packages/kombu/messaging.py", line 172, in publish
routing_key, mandatory, immediate, exchange, declare)
File "/opt/manager/env/lib/python2.7/site-packages/kombu/connection.py", line 470, in _ensured
interval_max)
File "/opt/manager/env/lib/python2.7/site-packages/kombu/connection.py", line 382, in ensure_connection
interval_start, interval_step, interval_max, callback)
File "/opt/manager/env/lib/python2.7/site-packages/kombu/utils/_init_.py", line 246, in retry_over_time
return fun(*args, **kwargs)
File "/opt/manager/env/lib/python2.7/site-packages/kombu/connection.py", line 250, in connect
return self.connection
File "/opt/manager/env/lib/python2.7/site-packages/kombu/connection.py", line 756, in connection
self._connection = self._establish_connection()
File "/opt/manager/env/lib/python2.7/site-packages/kombu/connection.py", line 711, in _establish_connection
conn = self.transport.establish_connection()
File "/opt/manager/env/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 116, in establish_connection
conn = self.Connection(**opts)
File "/opt/manager/env/lib/python2.7/site-packages/amqp/connection.py", line 165, in _init_
self.transport = self.Transport(host, connect_timeout, ssl)
File "/opt/manager/env/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
return create_transport(host, connect_timeout, ssl)
File "/opt/manager/env/lib/python2.7/site-packages/amqp/transport.py", line 297, in create_transport
return SSLTransport(host, connect_timeout, ssl)
File "/opt/manager/env/lib/python2.7/site-packages/amqp/transport.py", line 199, in _init_
super(SSLTransport, self)._init_(host, connect_timeout)
File "/opt/manager/env/lib/python2.7/site-packages/amqp/transport.py", line 102, in _init_
self._setup_transport()
File "/opt/manager/env/lib/python2.7/site-packages/amqp/transport.py", line 204, in _setup_transport
self.sock = ssl.wrap_socket(self.sock, **self.sslopts)
File "/usr/lib64/python2.7/ssl.py", line 936, in wrap_socket
ciphers=ciphers)
File "/usr/lib64/python2.7/ssl.py", line 611, in _init_
self.do_handshake()
File "/usr/lib64/python2.7/ssl.py", line 833, in do_handshake
self._sslobj.do_handshake()
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)

Status

Assignee

Unassigned

Reporter

Yariv Hamamy

Labels

None

Bug Type

None

Target Version

None

Severity

None

Epic Link

Sprint

None

Fix versions

Affects versions

Configure