Skip to content

Apache ARIA Service Proxy Plugin

Jump to bottom
DeWayne Filppi edited this page Feb 5, 2018 · 17 revisions

Introduction

The Apache ARIA project, is an OASIS TOSCA Simple YAML 1.0 reference implementation that includes a CLI and orchestration/execution engine. With it, TOSCA YAML service templates can be parsed and validated, instantiated, and have workflows run against them. Like Cloudify, the orchestration engine is extensible via plugins. This post is about the implementation and features of one such plugin which has a similar counterpart in Cloudify, the ARIA Service Proxy Plugin.

Rationale

TOSCA concerns itself with defining service templates, which contain a model of a given deployment structure or stack. TOSCA also defines normative operations (e.g. create, configure, start..) for template components (nodes) which an orchestrator can call to produce desired effects (e.g. install, uninstall). TOSCA also define a form of template composition known as substitution mapping. Substitution mapping permits the construction of service templates that can expose a simple interface for reference by other service templates. For example, a database service template, including compute and networking configuration, can be consumed by another template as a single TOSCA node.

TOSCA provides no means for composing services with independent life cycles. In the substitution mapping discussion above, the inclusion of the database template in the consuming template means that when the consuming template is realized, the database will also be realized. Conversely, when the consuming template is unrealized, the database will be destroyed. It is desirable to have a convenient means of orchestrating services with independent lifecycles so, for example, stacks that are clients can be created, destroyed, scaled, and updated at will without impacting an associated running service. One obvious approach to this problem is to define custom TOSCA types for each target service. A more convenient approach is to exploit the outputs feature of template services. Outputs are optionally exposed data that a service can provide (for example a public IP or URL). Outputs can be used as a means to allow the construction of templates that have proxy references to independent, TOSCA managed, running services.

Features

The service proxy needs to identify a target service, gather the outputs from the service, and expose them for use by the enclosing service. Since a service's existence doesn't mean it is realized (or installed), the proxy needs to be able to detect the status of the target service. Also, since the target service is completely independent, and outputs are dynamic, the proxy needs to be flexible regarding gathering of the service outputs. Given these requirements, the service proxy plugin provides following features:

  • A service_name property to identify the target service. This is the name of the service given when instantiated.
  • An outputs property defining a list of output names, to constrain what is retrieved from the target service. If not defined, the service existence equates to success.
  • A wait_config complex property, that defines whether to wait, what to wait for, and how long to wait for the service and/or outputs to become available.
  • A last_updated attribute, which indicates the time the outputs were retrieved.
  • A service_outputs attribute, which contains the list of retrieved outputs.

Detailed Comments

Output Names

The outputs list defines the outputs that will be copied to the service_outputs attribute when available. If no timeout is defined, a fatal exception will be thrown if all listed outputs are not available. A defined timeout will wait for all to be available before throwing the exception.

Wait Configuration

Wait configuration includes the following subconfiguration options:

  • wait_for_service - a boolean that determines, in coordination with the wait_time property, whether to retry fetching outputs. It will cause retries if either the service itself is not available or the desired outputs are not available.
  • wait_time - the number of seconds to wait. Ignored if wait_for_service is false.
  • wait_expression - if defined, and the service and outputs are available, the wait_expression serves as a final barrier to copying the outputs. wait_expression is a string value that is a Python boolean expression. The expression can use Python built-ins and reference requested outputs. For example, if a target service exposes a boolean output called available, a wait_expression can be as simple as available. On the other hand, something more complex like output1 > output2 and len(output3) < 5 is possible. It should be noted that outputs that have names that collide with the Python namespace will make this feature unusable.

Implementation Details

Native plugins for ARIA are a work in progress at this point. On the other hand, Cloudify plugins can be used with ARIA by means of the Cloudify ARIA extensions. Using the extension, Cloudify plugins can be used with ARIA. The only caveat is that the Cloudify DSL definitions (i.e. typically plugin.yaml) need to be translated to TOSCA DSL. So the approach taken for the service proxy plugin was to write a Cloudify native plugin, but putting TOSCA DSL in the plugin.yaml.

Cloudify already has a proxy plugin, which is included in the Cloudify Utilities Plugin, so why not just translate the relevant DSL and avoid coding something new? A couple reasons. First, the Cloudify proxy plugin relies on the Cloudify REST API, so it's useless for ARIA. Secondly, many features of the Cloudify proxy plugin supply functionality provided by substitution mapping in TOSCA, and so are irrelevant.

The definition, skipping the properties discussed above, is very simple:

node_types:

  aria.serviceproxy.ServiceProxy:
    derived_from: tosca.nodes.Root
...
    interfaces:
      Standard:
        type: tosca.interfaces.node.lifecycle.Standard
        create:
          implementation: aria-service-proxy-plugin > aria_service_proxy.tasks.proxy_connect

This shows that there is only a single lifecycle method in the ServiceProxy type that is implemented. There is no need for a symmetrical delete implementation, because there is no "connection" to unwind; just a copy done during instantiation. The implementation of proxy_connect uses the ARIA Python API to discover the target service and gather the outputs. Note the rather exotic (by Cloudify standards) implementation definition. This is a syntax required by the Cloudify ARIA extension to reference Cloudify plugin code.

The implementation of the features is straightforward. Retries are accomplished in the usual Cloudify manner: by calling for a retry on the Cloudify context object, which also track the retry number. No state is maintained between retries; the complete discovery and output copying process is repeated to capture any changes.

Areas For Improvement

  • Use an existing substition mapping definition. Rather than define the list of desired outputs, use the TOCSA substitution mapping concept at the service level. Since substitution mapping is meant for defining an interface to a service, extending it to be available as a runtime export is an good next step. This would require a TOSCA extension.
  • Proxy workflows. The proxy could expose public workflows on the target service to participate in day 2 operation orchestrations.
  • Sync updates - Outputs in TOSCA are dynamic, yet the plugin makes a copy at instantiation. The proxy could define an update operation to update it's state. This is somewhat useless unless there is a mechanism for reacting to changes (see Aync updates below).
  • Async updates - Outputs in TOSCA are dynamic, yet the plugin makes a copy at instantiation. Ideally, dependent nodes would receive events notifying them of changes in the target service status. This would require some architecture changes on the orchestrator side at a minimum.
  • True proxy capability - Ideally, a copy of target system properties would never be made, but rather proxying would occur at the attribute level. This would require orchestrator modification, which currently maps node attributes directly into database tables. If an attribute access could trigger plugin code, rather than be simple scalar values, the proxying could occur in lazy fashion upon request.

Conclusion

The ARIA Service Proxy Plugin fills a void in TOSCA orchestration by allowing a generic form of inter-service coordination based on the standard TOSCA outputs mechanism. This capability has applications for large scale operations that deploy many individually orchestrated subsystems that depend on each other. The topic of inter-service orchestration is a big one, and this implementation just scratches the surface. Code is available here, and comments always welcome.

Clone this wiki locally