Common Resource Properties and Behaviors

Description

We should add the base properties for all resource types:

  • use_external_resource, boolean, this resource is meant to already exist (Cloudify should not try to create it). Default: `true` for all base types`, should be over-written by plugin types.

  • resource_id, string: this should be the name of the resource. Default: None, must be provided by users or over-written by plugins.

Today we reimplement this for nearly every plugin. Most of the time, the logic is the same, but usually it is up to the developer (internal or community) which of the following behaviors will be used:

  • The resource is new, but what if the name (resource_id) is already taken? (Fail? Create variation? Other options?)

  • The resource is new, but what if the user did not provide a resource_id?

  • The resource is expected to exist, but what if it does not?

  • The resource is expected to exist, but what if another resource wants to connect to it (via a relationship).

  • The resource is expected to exist, but what if it wants to use another resource (via a relationship).

  • The resource already exists, but what if we want to scale it (add instances)?

  • The resource existed, but what if we need to heal it (delete existing and create new)?

These and other similar considerations need to be reimplemented time and time again, and over time, because of changing developers, changing developer guide, changing use cases, we have a pretty big mess of what needs to happen.

The sad thing is that this should really be part of core. Cloudify should decide how to handle each of these situations, not the plugin.

Cloudify 5 is the time to set this right.

In order to add the checking for external resources to the cloudify common the following changes must be added:

  • Update types.yaml by adding new operation called validation under Root node and adding properties for use_external_resource & resource_id as the following:

  • Update install workflow in order to add logic for checking the existing resources by triggering validation operation to check if the resource exists or not (validation task operation must be implemented by plugin developer since it is not possible to have this inside the cloudify common)

  • Have the ability to check if the resource exists or not by calling validation operation task

  • If the resource is new, but (resource_id) is already taken then usually this will fail whenever the create operation task executed, which something that we do not need really to take care

  • If the resource is new, but the user did not provide resource_id, then one of the solution is to let the cloudify common tries to generate the resource_id based on the node instance id but this will cause some issues

  • Some Plugins such as GCP needs to have resource_id in specific format and need to be normalized before set them

  • Most of the plugins (AWS, GCP, Azure, Openstack) has a separate property called name and this
    somehow need to be mapped between the cloudify resource_id and the API resource name

  • If the resource is expected to exist, and in fact it was not, then this should be handled as part of the changes made to the cloudify common which is part of the install workflow

  • If the resource is expected to exist, but what if another resource wants to connect to it (via a relationship). In order to solve this issue I think the plugins should handle that since each plugin has a lot of resources and each resource connect to another one in different way and I’m not sure if this is possible to move to cloudify common (Maybe need to be discussed more in order to come up with a solution for this)

  • The resource is expected to exist, but what if it wants to use another resource (via a relationship). Same for above

  • We need to ask ourself if there should be a mapping between the remote resource_id for the node and the cloudify resource_id and in case user provider resource_id, is this consider to be a cloudify resource_id or the remote resource_id

Activity

Show:
Trammell -
January 30, 2019, 9:55 AM

Maybe there are more such property keys? For example, `mac_address`, `cpus` or `cores`, `memory`, etc?

Trammell -
January 30, 2019, 12:43 PM


I also think maybe we could satisfy the `use_external_resource` functionality by adding a function to ctx.
Here are some implementations of `use_external_resource`:
https://github.com/cloudify-incubator/cloudify-azure-plugin/blob/master/cloudify_azure/utils.py#L91
https://github.com/cloudify-incubator/cloudify-openstacksdk-plugin/blob/dev/openstacksdk_plugin/decorators.py#L95
https://github.com/cloudify-cosmo/cloudify-aws-plugin/blob/master/cloudify_aws/common/decorators.py#L184

The question is what are the advantages of each and what are the disadvantages?

I personally think the OpenstackSDK one is the coolest.

Mohammed Abuaisha
January 31, 2019, 8:00 AM

For adding `use_external_resource` I think this will be a good idea to add this to cloudify common and I agree with you about openstacksdk is the best for me we can add it to the cloudify common and have unified way of defining resource object for how to call API from cloud, which requires from us to update all other plugins to match the one that should be added to the cloudify context which I think it is a good idea

Mohammed Abuaisha
January 31, 2019, 8:03 AM

I think it would be more consistent to have all common properties by plugins such ip/public ip address also shared between all plugins which will avoid any confusions to the use since any object can expose public ip should have the same runtime properties

Trammell -
March 3, 2019, 9:56 AM

Assignee

Unassigned

Reporter

Trammell -

Labels

Target Version

future

QA Owner

None

Premium Only

no

Documentation Required

None

Why Blocked?

None

Release Notes

yes

Priority

None

Sprint

None

Priority

Unprioritized
Configure