dependencies:
- type: olm.package
value:
packageName: prometheus
version: ">0.27.0"
- type: olm.gvk
value:
group: etcd.database.coreos.com
kind: etcdCluster
version: v1beta2
This guide outlines dependency resolution and custom resource definition (CRD) upgrade lifecycles with Operator Lifecycle Manager (OLM) in OKD.
OLM manages the dependency resolution and upgrade lifecycle of running Operators. In many ways, the problems OLM faces are similar to other operating system package managers like yum
and rpm
.
However, there is one constraint that similar systems do not generally have that OLM does: because Operators are always running, OLM attempts to ensure that you are never left with a set of Operators that do not work with each other.
This means that OLM must never do the following:
Install a set of Operators that require APIs that cannot be provided.
Update an Operator in a way that breaks another that depends upon it.
The dependencies of an Operator are listed in a dependencies.yaml
file in the metadata/
folder of a bundle. This file is optional and currently only used to specify explicit Operator-version dependencies.
The dependency list contains a type
field for each item to specify what kind of dependency this is. There are two supported types of Operator dependencies:
olm.package
: This type indicates a dependency for a specific Operator version. The dependency information must include the package name and the version of the package in semver format. For example, you can specify an exact version such as 0.5.2
or a range of versions such as >0.5.1
.
olm.gvk
: With a gvk
type, the author can specify a dependency with group/version/kind (GVK) information, similar to existing CRD and API-based usage in a CSV. This is a path to enable Operator authors to consolidate all dependencies, API or explicit versions, to be in the same place.
In the following example, dependencies are specified for a Prometheus Operator and etcd CRDs:
dependencies.yaml
filedependencies:
- type: olm.package
value:
packageName: prometheus
version: ">0.27.0"
- type: olm.gvk
value:
group: etcd.database.coreos.com
kind: etcdCluster
version: v1beta2
There can be many options that equally satisfy a dependency of an Operator. The dependency resolver in Operator Lifecycle Manager (OLM) determines which option best fits the requirements of the requested Operator. As an Operator author or user, it can be important to understand how these choices are made so that dependency resolution is clear.
On OKD cluster, OLM reads catalog sources to know which Operators are available for installation.
CatalogSource
objectapiVersion: "operators.coreos.com/v1alpha1"
kind: "CatalogSource"
metadata:
name: "my-operators"
namespace: "operators"
spec:
sourceType: grpc
image: example.com/my/operator-index:v1
displayName: "My Operators"
priority: 100
A CatalogSource
object has a priority
field, which is used by the resolver to know how to prefer options for a dependency.
There are two rules that govern catalog preference:
Options in higher-priority catalogs are preferred to options in lower-priority catalogs.
Options in the same catalog as the dependent are preferred to any other catalogs.
An Operator package in a catalog is a collection of update channels that a user can subscribe to in a OKD cluster. Channels can be used to provide a particular stream of updates for a minor release (1.2
, 1.3
) or a release frequency (stable
, fast
).
It is likely that a dependency might be satisfied by Operators in the same package, but different channels. For example, version 1.2
of an Operator might exist in both the stable
and fast
channels.
Each package has a default channel, which is always preferred to non-default channels. If no option in the default channel can satisfy a dependency, options are considered from the remaining channels in lexicographic order of the channel name.
There are almost always multiple options to satisfy a dependency within a single channel. For example, Operators in one package and channel provide the same set of APIs.
When a user creates a subscription, they indicate which channel to receive updates from. This immediately reduces the search to just that one channel. But within the channel, it is likely that many Operators satisfy a dependency.
Within a channel, newer Operators that are higher up in the update graph are preferred. If the head of a channel satisfies a dependency, it will be tried first.
In addition to the constraints supplied by package dependencies, OLM includes additional constraints to represent the desired user state and enforce resolution invariants.
A subscription constraint filters the set of Operators that can satisfy a subscription. Subscriptions are user-supplied constraints for the dependency resolver. They declare the intent to either install a new Operator if it is not already on the cluster, or to keep an existing Operator updated.
OLM upgrades a custom resource definition (CRD) immediately if it is owned by a singular cluster service version (CSV). If a CRD is owned by multiple CSVs, then the CRD is upgraded when it has satisfied all of the following backward compatible conditions:
All existing serving versions in the current CRD are present in the new CRD.
All existing instances, or custom resources, that are associated with the serving versions of the CRD are valid when validated against the validation schema of the new CRD.
When specifying dependencies, there are best practices you should consider.
Operators can add or remove APIs at any time; always specify an olm.gvk
dependency on any APIs your Operators requires. The exception to this is if you are specifying olm.package
constraints instead.
The Kubernetes documentation on API changes describes what changes are allowed for Kubernetes-style Operators. These versioning conventions allow an Operator to update an API without bumping the API version, as long as the API is backwards-compatible.
For Operator dependencies, this means that knowing the API version of a dependency might not be enough to ensure the dependent Operator works as intended.
For example:
TestOperator v1.0.0 provides v1alpha1 API version of the MyObject
resource.
TestOperator v1.0.1 adds a new field spec.newfield
to MyObject
, but still at v1alpha1.
Your Operator might require the ability to write spec.newfield
into the MyObject
resource. An olm.gvk
constraint alone is not enough for OLM to determine that you need TestOperator v1.0.1 and not TestOperator v1.0.0.
Whenever possible, if a specific Operator that provides an API is known ahead of time, specify an additional olm.package
constraint to set a minimum.
Because Operators provide cluster-scoped resources such as API services and CRDs, an Operator that specifies a small window for a dependency might unnecessarily constrain updates for other consumers of that dependency.
Whenever possible, do not set a maximum version. Alternatively, set a very wide semantic range to prevent conflicts with other Operators. For example, >1.0.0 <2.0.0
.
Unlike with conventional package managers, Operator authors explicitly encode that updates are safe through channels in OLM. If an update is available for an existing subscription, it is assumed that the Operator author is indicating that it can update from the previous version. Setting a maximum version for a dependency overrides the update stream of the author by unnecessarily truncating it at a particular upper bound.
Cluster administrators cannot override dependencies set by an Operator author. |
However, maximum versions can and should be set if there are known incompatibilities that must be avoided. Specific versions can be omitted with the version range syntax, for example > 1.0.0 !1.2.1
.
Kubernetes documentation: Changing the API
When specifying dependencies, there are caveats you should consider.
There is currently no method for specifying an AND relationship between constraints. In other words, there is no way to specify that one Operator depends on another Operator that both provides a given API and has version >1.1.0
.
This means that when specifying a dependency such as:
dependencies:
- type: olm.package
value:
packageName: etcd
version: ">3.1.0"
- type: olm.gvk
value:
group: etcd.database.coreos.com
kind: etcdCluster
version: v1beta2
It would be possible for OLM to satisfy this with two Operators: one that provides etcdCluster and one that has version >3.1.0
. Whether that happens, or whether an Operator is selected that satisfies both constraints, depends on the ordering that potential options are visited. Dependency preferences and ordering options are well-defined and can be reasoned about, but to exercise caution, Operators should stick to one mechanism or the other.
OLM performs dependency resolution at the namespace scope. It is possible to get into an update deadlock if updating an Operator in one namespace would be an issue for an Operator in another namespace, and vice-versa.
In the following examples, a provider is an Operator which "owns" a CRD or API service.
A and B are APIs (CRDs):
The provider of A depends on B.
The provider of B has a subscription.
The provider of B updates to provide C but deprecates B.
This results in:
B no longer has a provider.
A no longer works.
This is a case OLM prevents with its upgrade strategy.
A and B are APIs:
The provider of A requires B.
The provider of B requires A.
The provider of A updates to (provide A2, require B2) and deprecate A.
The provider of B updates to (provide B2, require A2) and deprecate B.
If OLM attempts to update A without simultaneously updating B, or vice-versa, it is unable to progress to new versions of the Operators, even though a new compatible set can be found.
This is another case OLM prevents with its upgrade strategy.