apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
gateways:
openshiftRoute:
enabled: true
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.
This release adds improvements related to the following components and concepts.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.
The most current version of the Red Hat OpenShift Service Mesh Operator can be used with all supported versions of Service Mesh. The version of Service Mesh is specified using the ServiceMeshControlPlane
.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.
This release of Red Hat OpenShift Service Mesh adds new features, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.
This release ends maintenance support for OpenShift Service Mesh version 2.2. If you are using OpenShift Service Mesh version 2.2, you should update to a supported version.
Component | Version |
---|---|
Istio |
1.18.7 |
Envoy Proxy |
1.26.8 |
Kiali |
1.73.4 |
Service Mesh 2.5 is based on Istio 1.18, which brings in new features and product enhancements. While Red Hat OpenShift Service Mesh supports many Istio 1.18 features, the following exceptions should be noted:
Ambient mesh is not supported
QuickAssist Technology (QAT) PrivateKeyProvider in Istio is not supported
This release adds documentation for migrating from a multitenant mesh to a cluster-wide mesh. For more information, see the following documentation:
"About migrating to a cluster-wide mesh"
"Excluding namespaces from a cluster-wide mesh"
"Defining which namespaces receive sidecar injection in a cluster-wide mesh"
"Excluding individual pods from a cluster-wide mesh"
This release provides the Red Hat OpenShift Service Mesh Operator on ARM-based clusters as a generally available feature.
This release introduces a generally available integration of the tracing extension provider(s). You can expose tracing data to the Red Hat OpenShift distributed tracing platform (Tempo) stack by appending a named element and the zipkin
provider to the spec.meshConfig.extensionProviders
specification. Then, a telemetry custom resource configures Istio proxies to collect trace spans and send them to the Tempo distributor service endpoint.
Red Hat OpenShift distributed tracing platform (Tempo) Stack is not supported on IBM Z. |
This release introduces a generally available version of the OpenShift Service Mesh Console (OSSMC) plugin.
The OSSMC plugin is an extension to the OpenShift Console that provides visibility into your Service Mesh. With the OSSMC plugin installed, a new Service Mesh menu option is available on the navigation pane of the web console, as well as new Service Mesh tabs that enhance existing Workloads and Service console pages.
The features of the OSSMC plugin are very similar to those of the standalone Kiali Console. The OSSMC plugin does not replace the Kiali Console, and after installing the OSSMC plugin, you can still access the standalone Kiali Console.
The default setting for Istio OpenShift Routing (IOR) has changed. Starting with this release, automatic routes are disabled by default for new instances of the ServiceMeshControlPlane
resource.
For new instances of the ServiceMeshControlPlane
resources, you can use automatic routes by setting the enabled
field to true
in the gateways.openshiftRoute
specification of the ServiceMeshControlPlane
resource.
ServiceMeshControlPlane
resourceapiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
gateways:
openshiftRoute:
enabled: true
When updating existing instances of the ServiceMeshControlPlane
resource to Red Hat OpenShift Service Mesh version 2.5, automatic routes remain enabled by default.
The concurrency
parameter in the networking.istio
API configures how many worker threads the Istio proxy runs.
For consistency across deployments, Istio now configures the concurrency
parameter based upon the CPU limit allocated to the proxy container. For example, a limit of 2500m would set the concurrency
parameter to 3
. If you set the concurrency
parameter to a different value, then Istio uses that value to configure how many threads the proxy runs instead of using the CPU limit.
Previously, the default setting for the parameter was 2
.
OpenShift Container Platform Gateway API support is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
A new version of the Gateway API custom resource definition (CRD) is now available. Refer to the following table to determine which Gateway API version should be installed with the OpenShift Service Mesh version you are using:
Service Mesh Version | Istio Version | Gateway API Version | Notes |
---|---|---|---|
2.5.x |
1.18.x |
0.6.2 |
Use the experimental branch because |
2.4.x |
1.16.x |
0.5.1 |
For multitenant mesh deployment, all Gateway API CRDs must be present. Use the experimental branch. |
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.
The most current version of the Red Hat OpenShift Service Mesh Operator can be used with all supported versions of Service Mesh. The version of Service Mesh is specified using the ServiceMeshControlPlane
.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.
The Red Hat OpenShift Service Mesh Operator is now available on ARM-based clusters as a Technology Preview feature.
The envoyExtAuthzGrpc
field has been added, which is used to configure an external authorization provider using the gRPC API.
Common Vulnerabilities and Exposures (CVEs) have been addressed.
This release is supported on OpenShift Container Platform 4.10 and newer versions.
Component | Version |
---|---|
Istio |
1.16.7 |
Envoy Proxy |
1.24.10 |
Jaeger |
1.42.0 |
Kiali |
1.65.8 |
Red Hat OpenShift Service Mesh operator to ARM based clusters is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
This release makes the Red Hat OpenShift Service Mesh Operator available on ARM-based clusters as a Technology Preview feature. Images are available for Istio, Envoy, Prometheus, Kiali, and Grafana. Images are not available for Jaeger, so Jaeger must be disabled as a Service Mesh add-on.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
Component | Version |
---|---|
Istio |
1.16.5 |
Envoy Proxy |
1.24.8 |
Jaeger |
1.42.0 |
Kiali |
1.65.6 |
This enhancement introduces a generally available version of cluster-wide deployments. A cluster-wide deployment contains a service mesh control plane that monitors resources for an entire cluster. The control plane uses a single query across all namespaces to monitor each Istio or Kubernetes resource that affects the mesh configuration. Reducing the number of queries the control plane performs in a cluster-wide deployment improves performance.
This enhancement introduces a generally available version of the meshConfig.discoverySelectors
field, which can be used in cluster-wide deployments to limit the services the service mesh control plane can discover.
spec:
meshConfig
discoverySelectors:
- matchLabels:
env: prod
region: us-east1
- matchExpressions:
- key: app
operator: In
values:
- cassandra
- spark
With this update, Red Hat OpenShift Service Mesh integrates with the cert-manager
controller and the istio-csr
agent. cert-manager
adds certificates and certificate issuers as resource types in Kubernetes clusters, and simplifies the process of obtaining, renewing, and using those certificates. cert-manager
provides and rotates an intermediate CA certificate for Istio. Integration with istio-csr
enables users to delegate signing certificate requests from Istio proxies to cert-manager
. ServiceMeshControlPlane
v2.4 accepts CA certificates provided by cert-manager
as cacerts
secret.
Integration with |
This enhancement introduces a generally available method of integrating Red Hat OpenShift Service Mesh with external authorization systems by using the action: CUSTOM
field of the AuthorizationPolicy
resource. Use the envoyExtAuthzHttp
field to delegate the access control to an external authorization system.
This enhancement introduces a generally available version of the Prometheus extension provider. You can expose metrics to the OpenShift Container Platform monitoring stack or a custom Prometheus installation by setting the value of the extensionProviders
field to prometheus
in the spec.meshConfig
specification. The telemetry object configures Istio proxies to collect traffic metrics. Service Mesh only supports the Telemetry API for Prometheus metrics.
spec:
meshConfig:
extensionProviders:
- name: prometheus
prometheus: {}
---
apiVersion: telemetry.istio.io/v1alpha1
kind: Telemetry
metadata:
name: enable-prometheus-metrics
spec:
metrics:
- providers:
- name: prometheus
This enhancement introduces generally available support for single stack IPv6 clusters, providing access to a broader range of IP addresses. Dual stack IPv4 or IPv6 cluster is not supported.
Single stack IPv6 support is not available on IBM Power, IBM Z, and IBM® LinuxONE. |
OpenShift Container Platform Gateway API support is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
This enhancement introduces an updated Technology Preview version of the OpenShift Container Platform Gateway API. By default, the OpenShift Container Platform Gateway API is disabled.
To enable the OpenShift Container Platform Gateway API, set the value of the enabled
field to true
in the techPreview.gatewayAPI
specification of the ServiceMeshControlPlane
resource.
spec:
techPreview:
gatewayAPI:
enabled: true
Previously, environment variables were used to enable the Gateway API.
spec:
runtime:
components:
pilot:
container:
env:
PILOT_ENABLE_GATEWAY_API: "true"
PILOT_ENABLE_GATEWAY_API_STATUS: "true"
PILOT_ENABLE_GATEWAY_API_DEPLOYMENT_CONTROLLER: "true"
Service Mesh control plane deployment is now supported and documented on OpenShift infrastructure nodes. For more information, see the following documentation:
Configuring all Service Mesh control plane components to run on infrastructure nodes
Configuring individual Service Mesh control plane components to run on infrastructure nodes
Service Mesh 2.4 is based on Istio 1.16, which brings in new features and product enhancements. While many Istio 1.16 features are supported, the following exceptions should be noted:
HBONE protocol for sidecars is an experimental feature that is not supported.
Service Mesh on ARM64 architecture is not supported.
OpenTelemetry API remains a Technology Preview feature.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.
The most current version of the Red Hat OpenShift Service Mesh Operator can be used with all supported versions of Service Mesh. The version of Service Mesh is specified using the ServiceMeshControlPlane
.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.12 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh introduces new features, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh introduces new features, addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
Component | Version |
---|---|
Istio |
1.14.3 |
Envoy Proxy |
1.22.4 |
Jaeger |
1.38 |
Kiali |
1.57.3 |
The openshift-operators
namespace includes a new istio CNI DaemonSet istio-cni-node-v2-3
and a new ConfigMap
resource, istio-cni-config-v2-3
.
When upgrading to Service Mesh Control Plane 2.3, the existing istio-cni-node
DaemonSet is not changed, and a new istio-cni-node-v2-3
DaemonSet is created.
This name change does not affect previous releases or any istio-cni-node
CNI DaemonSet associated with a Service Mesh Control Plane deployed using a previous release.
This release introduces generally available support for Gateway injection. Gateway configurations are applied to standalone Envoy proxies that are running at the edge of the mesh, rather than the sidecar Envoy proxies running alongside your service workloads. This enables the ability to customize gateway options. When using gateway injection, you must create the following resources in the namespace where you want to run your gateway proxy: Service
, Deployment
, Role
, and RoleBinding
.
Service Mesh 2.3 is based on Istio 1.14, which brings in new features and product enhancements. While many Istio 1.14 features are supported, the following exceptions should be noted:
ProxyConfig API is supported with the exception of the image field.
Telemetry API is a Technology Preview feature.
SPIRE runtime is not a supported feature.
OpenShift Service Mesh Console is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
This release introduces a Technology Preview version of the OpenShift Container Platform Service Mesh Console, which integrates the Kiali interface directly into the OpenShift web console. For additional information, see Introducing the OpenShift Service Mesh Console (A Technology Preview)
Cluster-wide deployment is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
This release introduces cluster-wide deployment as a Technology Preview feature. A cluster-wide deployment contains a Service Mesh Control Plane that monitors resources for an entire cluster. The control plane uses a single query across all namespaces to monitor each Istio or Kubernetes resource kind that affects the mesh configuration. In contrast, the multitenant approach uses a query per namespace for each resource kind. Reducing the number of queries the control plane performs in a cluster-wide deployment improves performance.
This cluster-wide deployment documentation is only applicable for control planes deployed using SMCP v2.3. cluster-wide deployments created using SMCP v2.3 are not compatible with cluster-wide deployments created using SMCP v2.4. |
The following example ServiceMeshControlPlane
object configures a cluster-wide deployment.
To create an SMCP for cluster-wide deployment, a user must belong to the cluster-admin
ClusterRole. If the SMCP is configured for cluster-wide deployment, it must be the only SMCP in the cluster. You cannot change the control plane mode from multitenant to cluster-wide (or from cluster-wide to multitenant). If a multitenant control plane already exists, delete it and create a new one.
This example configures the SMCP for cluster-wide deployment.
apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
name: cluster-wide
namespace: istio-system
spec:
version: v2.3
techPreview:
controlPlaneMode: ClusterScoped (1)
1 | Enables Istiod to monitor resources at the cluster level rather than monitor each individual namespace. |
Additionally, the SMMR must also be configured for cluster-wide deployment. This example configures the SMMR for cluster-wide deployment.
apiVersion: maistra.io/v1
kind: ServiceMeshMemberRoll
metadata:
name: default
spec:
members:
- '*' (1)
1 | Adds all namespaces to the mesh, including any namespaces you subsequently create. The following namespaces are not part of the mesh: kube, openshift, kube-* and openshift-*. |
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.11 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.10 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
Component | Version |
---|---|
Istio |
1.12.7 |
Envoy Proxy |
1.20.6 |
Jaeger |
1.36 |
Kiali |
1.48.2-1 |
With this enhancement, in addition to copying annotations, you can copy specific labels for an OpenShift route. Red Hat OpenShift Service Mesh copies all labels and annotations present in the Istio Gateway resource (with the exception of annotations starting with kubectl.kubernetes.io) into the managed OpenShift Route resource.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh adds new features and enhancements, and is supported on OpenShift Container Platform 4.9 and later versions.
Component | Version |
---|---|
Istio |
1.12.7 |
Envoy Proxy |
1.20.4 |
Jaeger |
1.34.1 |
Kiali |
1.48.0.16 |
WasmPlugin
APIThis release adds support for the WasmPlugin
API and deprecates the ServiceMeshExtension
API.
This release introduces service mesh support for Red Hat OpenShift on AWS (ROSA), including multi-cluster federation.
istio-node
DaemonSet renamedThis release, the istio-node
DaemonSet is renamed to istio-cni-node
to match the name in upstream Istio.
Istio 1.10 updated Envoy to send traffic to the application container using eth0
rather than lo
by default.
This release marks the end of support for Service Mesh Control Planes based on Service Mesh 1.1 for all platforms.
Service Mesh 2.2 is based on Istio 1.12, which brings in new features and product enhancements. While many Istio 1.12 features are supported, the following unsupported features should be noted:
AuthPolicy Dry Run is a tech preview feature.
gRPC Proxyless Service Mesh is a tech preview feature.
Telemetry API is a tech preview feature.
Discovery selectors is not a supported feature.
External control plane is not a supported feature.
Gateway injection is not a supported feature.
Kubernetes Gateway API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
Kubernetes Gateway API is a technology preview feature that is disabled by default. If the Kubernetes API deployment controller is disabled, you must manually deploy and link an ingress gateway to the created Gateway object.
If the Kubernetes API deployment controller is enabled, then an ingress gateway automatically deploys when a Gateway object is created.
The Gateway API CRDs do not come preinstalled by default on OpenShift clusters. Install the CRDs prior to enabling Gateway API support in the SMCP.
$ kubectl get crd gateways.gateway.networking.k8s.io || { kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=v0.4.0" | kubectl apply -f -; }
To enable the feature, set the following environment variables for the Istiod
container in ServiceMeshControlPlane
:
spec:
runtime:
components:
pilot:
container:
env:
PILOT_ENABLE_GATEWAY_API: "true"
PILOT_ENABLE_GATEWAY_API_STATUS: "true"
# and optionally, for the deployment controller
PILOT_ENABLE_GATEWAY_API_DEPLOYMENT_CONTROLLER: "true"
Restricting route attachment on Gateway API listeners is possible using the SameNamespace
or All
settings. Istio ignores usage of label selectors in listeners.allowedRoutes.namespaces
and reverts to the default behavior (SameNamespace
).
If the Kubernetes API deployment controller is disabled, you must manually deploy and then link an ingress gateway to the created Gateway resource.
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: Gateway
metadata:
name: gateway
spec:
addresses:
- value: ingress.istio-gateways.svc.cluster.local
type: Hostname
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), contains bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 and later versions.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
With this release, the Red Hat OpenShift distributed tracing platform (Jaeger) Operator is now installed to the openshift-distributed-tracing
namespace by default. Previously the default installation had been in the openshift-operator
namespace.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release also adds the ability to disable the automatic creation of network policies.
Component | Version |
---|---|
Istio |
1.9.9 |
Envoy Proxy |
1.17.1 |
Jaeger |
1.24.1 |
Kiali |
1.36.7 |
Red Hat OpenShift Service Mesh automatically creates and manages a number of NetworkPolicies
resources in the Service Mesh control plane and application namespaces. This is to ensure that applications and the control plane can communicate with each other.
If you want to disable the automatic creation and management of NetworkPolicies
resources, for example to enforce company security policies, you can do so. You can edit the ServiceMeshControlPlane
to set the spec.security.manageNetworkPolicy
setting to false
When you disable |
In the OpenShift Container Platform web console, click Operators → Installed Operators.
Select the project where you installed the Service Mesh control plane, for example istio-system
, from the Project menu.
Click the Red Hat OpenShift Service Mesh Operator. In the Istio Service Mesh Control Plane column, click the name of your ServiceMeshControlPlane
, for example basic-install
.
On the Create ServiceMeshControlPlane Details page, click YAML
to modify your configuration.
Set the ServiceMeshControlPlane
field spec.security.manageNetworkPolicy
to false
, as shown in this example.
apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
security:
trust:
manageNetworkPolicy: false
Click Save.
This release of Red Hat OpenShift Service Mesh adds support for Istio 1.9.8, Envoy Proxy 1.17.1, Jaeger 1.24.1, and Kiali 1.36.5 on OpenShift Container Platform 4.6 EUS, 4.7, 4.8, 4.9, along with new features and enhancements.
Component | Version |
---|---|
Istio |
1.9.6 |
Envoy Proxy |
1.17.1 |
Jaeger |
1.24.1 |
Kiali |
1.36.5 |
New Custom Resource Definitions (CRDs) have been added to support federating service meshes. Service meshes may be federated both within the same cluster or across different OpenShift clusters. These new resources include:
ServiceMeshPeer
- Defines a federation with a separate service mesh, including gateway configuration, root trust certificate configuration, and status fields. In a pair of federated meshes, each mesh will define its own separate ServiceMeshPeer
resource.
ExportedServiceMeshSet
- Defines which services for a given ServiceMeshPeer
are available for the peer mesh to import.
ImportedServiceSet
- Defines which services for a given ServiceMeshPeer
are imported from the peer mesh. These services must also be made available by the peer’s ExportedServiceMeshSet
resource.
Service Mesh Federation is not supported between clusters on Red Hat OpenShift Service on AWS (ROSA), Azure Red Hat OpenShift (ARO), or OpenShift Dedicated (OSD).
The OVN-Kubernetes Container Network Interface (CNI) was previously introduced as a Technology Preview feature in Red Hat OpenShift Service Mesh 2.0.1 and is now generally available in Red Hat OpenShift Service Mesh 2.1 and 2.0.x for use on OpenShift Container Platform 4.7.32, OpenShift Container Platform 4.8.12, and OpenShift Container Platform 4.9.
The ServiceMeshExtensions
Custom Resource Definition (CRD), first introduced in 2.0 as Technology Preview, is now generally available. You can use CRD to build your own plugins, but Red Hat does not provide support for the plugins you create.
Mixer has been completely removed in Service Mesh 2.1. Upgrading from a Service Mesh 2.0.x release to 2.1 will be blocked if Mixer is enabled. Mixer plugins will need to be ported to WebAssembly Extensions.
With Mixer now officially removed, OpenShift Service Mesh 2.1 does not support the 3scale mixer adapter. Before upgrading to Service Mesh 2.1, remove the Mixer-based 3scale adapter and any additional Mixer plugins. Then, manually install and configure the new 3scale WebAssembly adapter with Service Mesh 2.1+ using a ServiceMeshExtension
resource.
3scale 2.11 introduces an updated Service Mesh integration based on WebAssembly
.
Service Mesh 2.1 is based on Istio 1.9, which brings in a large number of new features and product enhancements. While the majority of Istio 1.9 features are supported, the following exceptions should be noted:
Virtual Machine integration is not yet supported
Kubernetes Gateway API is not yet supported
Remote fetch and load of WebAssembly HTTP filters are not yet supported
Custom CA Integration using the Kubernetes CSR API is not yet supported
Request Classification for monitoring traffic is a tech preview feature
Integration with external authorization systems via Authorization policy’s CUSTOM action is a tech preview feature
The amount of time Red Hat OpenShift Service Mesh uses to prune old resources at the end of every ServiceMeshControlPlane
reconciliation has been reduced. This results in faster ServiceMeshControlPlane
deployments, and allows changes applied to existing SMCPs to take effect more quickly.
Kiali 1.36 includes the following features and enhancements:
Service Mesh troubleshooting functionality
Control plane and gateway monitoring
Proxy sync statuses
Envoy configuration views
Unified view showing Envoy proxy and application logs interleaved
Namespace and cluster boxing to support federated service mesh views
New validations, wizards, and distributed tracing enhancements
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs), bug fixes, and is supported on OpenShift Container Platform 4.9 or later.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs).
Red Hat OpenShift Service Mesh contains a remotely exploitable vulnerability, CVE-2021-39156, where an HTTP request with a fragment (a section in the end of a URI that begins with a # character) in the URI path could bypass the Istio URI path-based authorization policies. For instance, an Istio authorization policy denies requests sent to the URI path /user/profile
. In the vulnerable versions, a request with URI path /user/profile#section1
bypasses the deny policy and routes to the backend (with the normalized URI path /user/profile%23section1
), possibly leading to a security incident.
You are impacted by this vulnerability if you use authorization policies with DENY actions and operation.paths
, or ALLOW actions and operation.notPaths
.
With the mitigation, the fragment part of the request’s URI is removed before the authorization and routing. This prevents a request with a fragment in its URI from bypassing authorization policies which are based on the URI without the fragment part.
To opt-out from the new behavior in the mitigation, the fragment section in the URI will be kept. You can configure your ServiceMeshControlPlane
to keep URI fragments.
Disabling the new behavior will normalize your paths as described above and is considered unsafe. Ensure that you have accommodated for this in any security policies before opting to keep URI fragments. |
ServiceMeshControlPlane
modificationapiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
name: basic
spec:
techPreview:
meshConfig:
defaultConfig:
proxyMetadata: HTTP_STRIP_FRAGMENT_FROM_PATH_UNSAFE_IF_DISABLED: "false"
Istio generates hostnames for both the hostname itself and all matching ports. For instance, a virtual service or Gateway for a host of "httpbin.foo" generates a config matching "httpbin.foo and httpbin.foo:*". However, exact match authorization policies only match the exact string given for the hosts
or notHosts
fields.
Your cluster is impacted if you have AuthorizationPolicy
resources using exact string comparison for the rule to determine hosts or notHosts.
You must update your authorization policy rules to use prefix match instead of exact match. For example, replacing hosts: ["httpbin.com"]
with hosts: ["httpbin.com:*"]
in the first AuthorizationPolicy
example.
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin
namespace: foo
spec:
action: DENY
rules:
- from:
- source:
namespaces: ["dev"]
to:
- operation:
hosts: [“httpbin.com”,"httpbin.com:*"]
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: httpbin
namespace: default
spec:
action: DENY
rules:
- to:
- operation:
hosts: ["httpbin.example.com:*"]
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
Red Hat OpenShift Service Mesh is now supported through Red Hat OpenShift Dedicated and Microsoft Azure Red Hat OpenShift.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
There are manual steps that must be completed to address CVE-2021-29492 and CVE-2021-31920. |
Istio contains a remotely exploitable vulnerability where an HTTP request path with multiple slashes or escaped slash characters (%2F
or %5C
) could potentially bypass an Istio authorization policy when path-based authorization rules are used.
For example, assume an Istio cluster administrator defines an authorization DENY policy to reject the request at path /admin
. A request sent to the URL path //admin
will NOT be rejected by the authorization policy.
According to RFC 3986, the path //admin
with multiple slashes should technically be treated as a different path from the /admin
. However, some backend services choose to normalize the URL paths by merging multiple slashes into a single slash. This can result in a bypass of the authorization policy (//admin
does not match /admin
), and a user can access the resource at path /admin
in the backend; this would represent a security incident.
Your cluster is impacted by this vulnerability if you have authorization policies using ALLOW action + notPaths
field or DENY action + paths field
patterns. These patterns are vulnerable to unexpected policy bypasses.
Your cluster is NOT impacted by this vulnerability if:
You don’t have authorization policies.
Your authorization policies don’t define paths
or notPaths
fields.
Your authorization policies use ALLOW action + paths
field or DENY action + notPaths
field patterns. These patterns could only cause unexpected rejection instead of policy bypasses. The upgrade is optional for these cases.
The Red Hat OpenShift Service Mesh configuration location for path normalization is different from the Istio configuration. |
Istio authorization policies can be based on the URL paths in the HTTP request. Path normalization, also known as URI normalization, modifies and standardizes the incoming requests' paths so that the normalized paths can be processed in a standard way. Syntactically different paths may be equivalent after path normalization.
Istio supports the following normalization schemes on the request paths before evaluating against the authorization policies and routing the requests:
Option | Description | Example | Notes |
---|---|---|---|
|
No normalization is done. Anything received by Envoy will be forwarded exactly as-is to any backend service. |
|
This setting is vulnerable to CVE-2021-31920. |
|
This is currently the option used in the default installation of Istio. This applies the |
|
This setting is vulnerable to CVE-2021-31920. |
|
Slashes are merged after the BASE normalization. |
|
Update to this setting to mitigate CVE-2021-31920. |
|
The strictest setting when you allow all traffic by default. This setting is recommended, with the caveat that you must thoroughly test your authorization policies routes. Percent-encoded slash and backslash characters ( |
|
Update to this setting to mitigate CVE-2021-31920. This setting is more secure, but also has the potential to break applications. Test your applications before deploying to production. |
The normalization algorithms are conducted in the following order:
Percent-decode %2F
, %2f
, %5C
and %5c
.
The RFC 3986 and other normalization implemented by the normalize_path
option in Envoy.
Merge slashes.
While these normalization options represent recommendations from HTTP standards and common industry practices, applications may interpret a URL in any way it chooses to. When using denial policies, ensure that you understand how your application behaves. |
Ensuring Envoy normalizes request paths to match your backend services' expectations is critical to the security of your system.
The following examples can be used as a reference for you to configure your system.
The normalized URL paths, or the original URL paths if NONE
is selected, will be:
Used to check against the authorization policies.
Forwarded to the backend application.
If your application… | Choose… |
---|---|
Relies on the proxy to do normalization |
|
Normalizes request paths based on RFC 3986 and does not merge slashes. |
|
Normalizes request paths based on RFC 3986 and merges slashes, but does not decode percent-encoded slashes. |
|
Normalizes request paths based on RFC 3986, decodes percent-encoded slashes, and merges slashes. |
|
Processes request paths in a way that is incompatible with RFC 3986. |
|
To configure path normalization for Red Hat OpenShift Service Mesh, specify the following in your ServiceMeshControlPlane
. Use the configuration examples to help determine the settings for your system.
spec:
techPreview:
global:
pathNormalization: <option>
In some environments, it may be useful to have paths in authorization policies compared in a case insensitive manner.
For example, treating https://myurl/get
and https://myurl/GeT
as equivalent.
In those cases, you can use the EnvoyFilter
shown below.
This filter will change both the path used for comparison and the path presented to the application. In this example, istio-system
is the name of the Service Mesh control plane project.
Save the EnvoyFilter
to a file and run the following command:
$ oc create -f <myEnvoyFilterFile>
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: ingress-case-insensitive
namespace: istio-system
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
subFilter:
name: "envoy.filters.http.router"
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
inlineCode: |
function envoy_on_request(request_handle)
local path = request_handle:headers():get(":path")
request_handle:headers():replace(":path", string.lower(path))
end
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
In addition, this release has the following new features:
Added an option to the must-gather
data collection tool that gathers information from a specified Service Mesh control plane namespace. For more information, see OSSM-351.
Improved performance for Service Mesh control planes with hundreds of namespaces
This release of Red Hat OpenShift Service Mesh adds support for IBM Z and IBM Power Systems. It also addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh adds support for Istio 1.6.5, Jaeger 1.20.0, Kiali 1.24.2, and the 3scale Istio Adapter 2.0 and OpenShift Container Platform 4.6.
In addition, this release has the following new features:
Simplifies installation, upgrades, and management of the Service Mesh control plane.
Reduces the Service Mesh control plane’s resource usage and startup time.
Improves performance by reducing inter-control plane communication over networking.
Adds support for Envoy’s Secret Discovery Service (SDS). SDS is a more secure and efficient mechanism for delivering secrets to Envoy side car proxies.
Removes the need to use Kubernetes Secrets, which have well known security risks.
Improves performance during certificate rotation, as proxies no longer require a restart to recognize new certificates.
Adds support for Istio’s Telemetry v2 architecture, which is built using WebAssembly extensions. This new architecture brings significant performance improvements.
Updates the ServiceMeshControlPlane resource to v2 with a streamlined configuration to make it easier to manage the Service Mesh Control Plane.
Introduces WebAssembly extensions as a Technology Preview feature.
Some features in this release are currently in Technology Preview. These experimental features are not intended for production use.
Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
Some features available in previous releases have been deprecated or removed.
Deprecated functionality is still included in OpenShift Container Platform and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.
Removed functionality no longer exists in the product.
The v2.2 ServiceMeshControlPlane
resource is no longer supported. Customers should update their mesh deployments to use a later version of the ServiceMeshControlPlane
resource.
Support for the Red Hat OpenShift distributed tracing platform (Jaeger) Operator is deprecated. To collect trace spans, use the Red Hat OpenShift distributed tracing platform (Tempo) Stack.
Support for the OpenShift Elasticsearch Operator is deprecated.
Istio will remove support for first-party JSON Web Tokens (JWTs). Istio will still support third-Party JWTs.
The v2.1 ServiceMeshControlPlane
resource is no longer supported. Customers should upgrade their mesh deployments to use a later version of the ServiceMeshControlPlane
resource.
Support for Istio OpenShift Routing (IOR) is deprecated and will be removed in a future release.
Support for Grafana is deprecated and will be removed in a future release.
Support for the following cipher suites, which were deprecated in Red Hat OpenShift Service Mesh 2.3, has been removed from the default list of ciphers used in TLS negotiations on both the client and server sides. Applications that require access to services requiring one of these cipher suites will fail to connect when a TLS connection is initiated from the proxy.
ECDHE-ECDSA-AES128-SHA
ECDHE-RSA-AES128-SHA
AES128-GCM-SHA256
AES128-SHA
ECDHE-ECDSA-AES256-SHA
ECDHE-RSA-AES256-SHA
AES256-GCM-SHA384
AES256-SHA
Support for the following cipher suites has been deprecated. In a future release, they will be removed from the default list of ciphers used in TLS negotiations on both the client and server sides.
ECDHE-ECDSA-AES128-SHA
ECDHE-RSA-AES128-SHA
AES128-GCM-SHA256
AES128-SHA
ECDHE-ECDSA-AES256-SHA
ECDHE-RSA-AES256-SHA
AES256-GCM-SHA384
AES256-SHA
The ServiceMeshExtension
API, which was deprecated in Red Hat OpenShift Service Mesh version 2.2, was removed in Red Hat OpenShift Service Mesh version 2.3. If you are using the ServiceMeshExtension
API, you must migrate to the WasmPlugin
API to continue using your WebAssembly extensions.
The ServiceMeshExtension
API is deprecated as of release 2.2 and will be removed in a future release. While ServiceMeshExtension
API is still supported in release 2.2, customers should start moving to the new WasmPlugin
API.
This release marks the end of support for Service Mesh control planes based on Service Mesh 1.1 for all platforms.
In Service Mesh 2.1, the Mixer component is removed. Bug fixes and support is provided through the end of the Service Mesh 2.0 life cycle.
Upgrading from a Service Mesh 2.0.x release to 2.1 will not proceed if Mixer plugins are enabled. Mixer plugins must be ported to WebAssembly Extensions.
The Mixer component was deprecated in release 2.0 and will be removed in release 2.1. While using Mixer for implementing extensions was still supported in release 2.0, extensions should have been migrated to the new WebAssembly mechanism.
The following resource types are no longer supported in Red Hat OpenShift Service Mesh 2.0:
Policy
(authentication.istio.io/v1alpha1) is no longer supported. Depending on the specific configuration in your Policy resource, you may have to configure multiple resources to achieve the same effect.
Use RequestAuthentication
(security.istio.io/v1beta1)
Use PeerAuthentication
(security.istio.io/v1beta1)
ServiceMeshPolicy
(maistra.io/v1) is no longer supported.
Use RequestAuthentication
or PeerAuthentication
, as mentioned above, but place in the Service Mesh control plane namespace.
RbacConfig
(rbac.istio.io/v1alpha1) is no longer supported.
Replaced by AuthorizationPolicy
(security.istio.io/v1beta1), which encompasses behavior of RbacConfig
, ServiceRole
, and ServiceRoleBinding
.
ServiceMeshRbacConfig
(maistra.io/v1) is no longer supported.
Use AuthorizationPolicy
as above, but place in Service Mesh control plane namespace.
ServiceRole
(rbac.istio.io/v1alpha1) is no longer supported.
ServiceRoleBinding
(rbac.istio.io/v1alpha1) is no longer supported.
In Kiali, the login
and LDAP
strategies are deprecated. A future version will introduce authentication using OpenID providers.
These limitations exist in Red Hat OpenShift Service Mesh:
Red Hat OpenShift Service Mesh does not yet fully support IPv6. As a result, Red Hat OpenShift Service Mesh does not support dual-stack clusters.
Graph layout - The layout for the Kiali graph can render differently, depending on your application architecture and the data to display (number of graph nodes and their interactions). Because it is difficult if not impossible to create a single layout that renders nicely for every situation, Kiali offers a choice of several different layouts. To choose a different layout, you can choose a different Layout Schema from the Graph Settings menu.
The first time you access related services such as distributed tracing platform (Jaeger) and Grafana, from the Kiali console, you must accept the certificate and re-authenticate using your OpenShift Container Platform login credentials. This happens due to an issue with how the framework displays embedded pages in the console.
The Bookinfo sample application cannot be installed on IBM Power, IBM Z, and IBM® LinuxONE.
WebAssembly extensions are not supported on IBM Power, IBM Z, and IBM® LinuxONE.
LuaJIT is not supported on IBM Power, IBM Z, and IBM® LinuxONE.
Single stack IPv6 support is not available on IBM Power, IBM Z, and IBM® LinuxONE.
These are the known issues in Red Hat OpenShift Service Mesh:
OSSM-6267 After a data source is configured correctly in the Grafana, a data query returns authentication error. Users are not able to view data in the Istio service and Istio workload dashboards. Currently, no workaround exists for this issue.
OSSM-5556 Gateways are skipped when istio-system labels do not match discovery selectors.
Workaround: Label the control plane namespace to match discovery selectors to avoid skipping the Gateway configurations.
ServiceMeshControlPlane
resourceapiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
name: basic
namespace: istio-system
spec:
mode: ClusterWide
meshConfig:
discoverySelectors:
- matchLabels:
istio-discovery: enabled
gateways:
ingress:
enabled: true
Then, run the following command at the command line:
oc label namespace istio-system istio-discovery=enabled
OSSM-3890 Attempting to use the Gateway API in a multitenant mesh deployment generates an error message similar to the following:
2023-05-02T15:20:42.541034Z error watch error in cluster Kubernetes: failed to list *v1alpha2.TLSRoute: the server could not find the requested resource (get tlsroutes.gateway.networking.k8s.io)
2023-05-02T15:20:42.616450Z info kube controller "gateway.networking.k8s.io/v1alpha2/TCPRoute" is syncing...
To support Gateway API in a multitenant mesh deployment, all Gateway API Custom Resource Definition (CRD) files must be present in the cluster.
In a multitenant mesh deployment, CRD scan is disabled, and Istio has no way to discover which CRDs are present in a cluster. As a result, Istio attempts to watch all supported Gateway API CRDs, but generates errors if some of those CRDs are not present.
Service Mesh 2.3.1 and later versions support both v1alpha2
and v1beta1
CRDs. Therefore, both CRD versions must be present for a multitenant mesh deployment to support the Gateway API.
Workaround: In the following example, the kubectl get
operation installs the v1alpha2
and v1beta1
CRDs. Note the URL contains the additional experimental
segment and updates any of your existing scripts accordingly:
$ kubectl get crd gateways.gateway.networking.k8s.io || { kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd/experimental?ref=v0.5.1" | kubectl apply -f -; }
OSSM-2042 Deployment of SMCP named default
fails. If you are creating an SMCP object, and set its version field to v2.3, the name of the object cannot be default
. If the name is default
, then the control plane fails to deploy, and OpenShift generates a Warning
event with the following message:
Error processing component mesh-config: error: [mesh-config/templates/telemetryv2_1.6.yaml: Internal error occurred: failed calling webhook "rev.validation.istio.io": Post "https://istiod-default.istio-system.svc:443/validate?timeout=10s": x509: certificate is valid for istiod.istio-system.svc, istiod-remote.istio-system.svc, istio-pilot.istio-system.svc, not istiod-default.istio-system.svc, mesh-config/templates/enable-mesh-permissive.yaml
OSSM-1655 Kiali dashboard shows error after enabling mTLS in SMCP
.
After enabling the spec.security.controlPlane.mtls
setting in the SMCP, the Kiali console displays the following error message No subsets defined
.
OSSM-1505 This issue only occurs when using the ServiceMeshExtension
resource on OpenShift Container Platform 4.11. When you use ServiceMeshExtension
on OpenShift Container Platform 4.11 the resource never becomes ready. If you inspect the issue using oc describe ServiceMeshExtension
you will see the following error: stderr: Error creating mount namespace before pivot: function not implemented
.
Workaround: ServiceMeshExtension
was deprecated in Service Mesh 2.2. Migrate from ServiceMeshExtension
to the WasmPlugin
resource.
For more information, see Migrating from ServiceMeshExtension
to WasmPlugin
resources.
OSSM-1396 If a gateway resource contains the spec.externalIPs
setting, instead of being recreated when the ServiceMeshControlPlane
is updated, the gateway is removed and never recreated.
OSSM-1168 When service mesh resources are created as a single YAML file, the Envoy proxy sidecar is not reliably injected into pods. When the SMCP, SMMR, and Deployment resources are created individually, the deployment works as expected.
OSSM-1115 The concurrency
field of the spec.proxy
API did not propagate to the istio-proxy. The concurrency
field works when set with ProxyConfig
. The concurrency
field specifies the number of worker threads to run. If the field is set to 0
, then the number of worker threads available is equal to the number of CPU cores. If the field is not set, then the number of worker threads available defaults to 2
.
In the following example, the concurrency
field is set to 0
.
apiVersion: networking.istio.io/v1beta1
kind: ProxyConfig
metadata:
name: mesh-wide-concurrency
namespace: <istiod-namespace>
spec:
concurrency: 0
OSSM-1052 When configuring a Service ExternalIP
for the ingressgateway in the Service Mesh control plane, the service is not created. The schema for the SMCP is missing the parameter for the service.
Workaround: Disable the gateway creation in the SMCP spec and manage the gateway deployment entirely manually (including Service, Role and RoleBinding).
OSSM-882 This applies for Service Mesh 2.1 and earlier. Namespace is in the accessible_namespace list but does not appear in Kiali UI. By default, Kiali will not show any namespaces that start with "kube" because these namespaces are typically internal-use only and not part of a mesh.
For example, if you create a namespace called 'akube-a' and add it to the Service Mesh member roll, then the Kiali UI does not display the namespace. For defined exclusion patterns, the software excludes namespaces that start with or contain the pattern.
Workaround: Change the Kiali Custom Resource setting so it prefixes the setting with a carat (^). For example:
api:
namespaces:
exclude:
- "^istio-operator"
- "^kube-.*"
- "^openshift.*"
- "^ibm.*"
- "^kiali-operator"
MAISTRA-2692 With Mixer removed, custom metrics that have been defined in Service Mesh 2.0.x cannot be used in 2.1. Custom metrics can be configured using EnvoyFilter
. Red Hat is unable to support EnvoyFilter
configuration except where explicitly documented. This is due to tight coupling with the underlying Envoy APIs, meaning that backward compatibility cannot be maintained.
MAISTRA-2648 Service mesh extensions are currently not compatible with meshes deployed on IBM Z.
MAISTRA-1959 Migration to 2.0 Prometheus scraping (spec.addons.prometheus.scrape
set to true
) does not work when mTLS is enabled. Additionally, Kiali displays extraneous graph data when mTLS is disabled.
This problem can be addressed by excluding port 15020 from proxy configuration, for example,
spec:
proxy:
networking:
trafficControl:
inbound:
excludedPorts:
- 15020
MAISTRA-453 If you create a new project and deploy pods immediately, sidecar injection does not occur. The operator fails to add the maistra.io/member-of
before the pods are created, therefore the pods must be deleted and recreated for sidecar injection to occur.
MAISTRA-158 Applying multiple gateways referencing the same hostname will cause all gateways to stop functioning.
New issues for Kiali should be created in the OpenShift Service Mesh project with the |
These are the known issues in Kiali:
KIALI-2206 When you are accessing the Kiali console for the first time, and there is no cached browser data for Kiali, the “View in Grafana” link on the Metrics tab of the Kiali Service Details page redirects to the wrong location. The only way you would encounter this issue is if you are accessing Kiali for the first time.
KIALI-507 Kiali does not support Internet Explorer 11. This is because the underlying frameworks do not support Internet Explorer. To access the Kiali console, use one of the two most recent versions of the Chrome, Edge, Firefox or Safari browser.
The following issues have been resolved in the current release:
OSSM-6331 Previously, the smcp.general.logging.componentLevels
spec accepted invalid LogLevel
values, and the ServiceMeshControlPlane
resource was still created. Now, the terminal shows an error message if an invalid value is used, and the control plane is not created.
OSSM-6290 Previously, the Project filter of the Istio Config list page did not work correctly. All istio config
items were displayed from all namespaces even if you selected a specific project from the drop-down menu. Now, only the istio config
items that belong to the selected project in the filter dropdown are displayed.
OSSM-6298 Previously, when you clicked an item reference within the OpenShift Service Mesh Console (OSSMC) plugin, the console sometimes performed multiple redirects before opening the desired page. As a result, navigating back to the previous page that was open in the console caused your web browser to open the wrong page. Now, these redirects do not occur, and clicking Back in a web browser brings you to the correct page.
OSSM-6299 Previously, in OpenShift Container Platform 4.15, when you clicked the Node graph menu option of any node menu within the traffic graph, the node graph was not displayed. Instead, the page refreshed with the same traffic graph. Now, clicking the Node graph menu option correctly displays the node graph.
The following issues have been resolved in previous releases:
OSSM-6177 Previously, when validation messages were enabled in the ServiceMeshControlPlane
(SMCP), the istiod
crashed continuously unless GatewayAPI
support was enabled. Now, when validation messages are enabled but GatewayAPI
support is not, the istiod
does not continuously crash.
OSSM-6163 Resolves the following issues:
Previously, an unstable Prometheus image was included in the Service Mesh control plane (SMCP) v2.5, and users were not able to access the Prometheus dashboard. Now, in the Service Mesh operator 2.5.1, the Prometheus image has been updated.
Previously, in the Service Mesh control plane (SMCP), a Grafana data source was not able to set Basic authentication password automatically and users were not able to view metrics from Prometheus in Grafana mesh dashboards. Now, a Grafana data source password is configured under the secureJsonData
field. Metrics are displayed correctly in dashboards.
OSSM-6148 Previously, the OpenShift Service Mesh Console (OSSMC) plugin did not respond when the user clicked any option in the menu of any node on the Traffic Graph page. Now, the plugin responds to the selected option in the menu by redirecting to the corresponding details page.
OSSM-6099 Previously, the OpenShift Service Mesh Console (OSSMC) plugin failed to load correctly in an IPv6 cluster. Now, the OSSMC plugin configuration has been modified to ensure proper loading in an IPv6 cluster.
OSSM-5960 Previously, the OpenShift Service Mesh Console (OSSMC) plugin did not display notification messages such as backend errors or Istio validations. Now, these notifications are displayed correctly at the top of the plugin page.
OSSM-5959 Previously, the OpenShift Service Mesh Console (OSSMC) plugin did not display TLS and Istio certification information in the Overview page. Now, this information is displayed correctly.
OSSM-5902 Previously, the OpenShift Service Mesh Console (OSSMC) plugin redirected to a "Not Found Page" error when the user clicked the Istio config health symbol on the Overview page. Now, the plugin redirects to the correct Istio config details page.
OSSM-5541 Previously, an Istio operator pod might keep waiting for the leader lease in some restart conditions. Now, the leader election implementation has been enhanced to avoid this issue.
OSSM-1397 Previously, if you removed the maistra.io/member-of
label from a namespace, the Service Mesh Operator did not automatically reapply the label to the namespace. As a result, sidecar injection did not work in the namespace.
The Operator would reapply the label to the namespace when you made changes to the ServiceMeshMember
object, which triggered the reconciliation of this member object.
Now, any change to the namespace also triggers the member object reconciliation.
OSSM-3647 Previously, in the Service Mesh control plane (SMCP) v2.2 (Istio 1.12), WasmPlugins were applied only to inbound listeners. Since SMCP v2.3 (Istio 1.14), WasmPlugins have been applied to inbound and outbound listeners by default, which introduced regression for users of the 3scale WasmPlugin. Now, the environment variable APPLY_WASM_PLUGINS_TO_INBOUND_ONLY
is added, which allows safe migration from SMCP v2.2 to v2.3 and v2.4.
The following setting should be added to the SMCP config:
spec:
runtime:
components:
pilot:
container:
env:
APPLY_WASM_PLUGINS_TO_INBOUND_ONLY: "true"
To ensure safe migration, perform the following steps:
Set APPLY_WASM_PLUGINS_TO_INBOUND_ONLY
in SMCP v2.2.
Upgrade to 2.4.
Set spec.match[].mode: SERVER
in WasmPlugins.
Remove the previously-added environment variable.
OSSM-4851 Previously, an error occurred in the operator deploying new pods in a namespace scoped inside the mesh when runAsGroup
, runAsUser
, or fsGroup
parameters were nil
. Now, a yaml validation has been added to avoid the nil
value.
OSSM-3771 Previously, OpenShift routes could not be disabled for additional ingress gateways defined in a Service Mesh Control Plane (SMCP). Now, a routeConfig
block can be added to each additionalingress
gateway so the creation of OpenShift routes can be enabled or disabled for each gateway.
OSSM-4197 Previously, if you deployed a v2.2 or v2.1 of the 'ServiceMeshControlPlane' resource, the /etc/cni/multus/net.d/
directory was not created. As a result, the istio-cni
pod failed to become ready, and the istio-cni
pods log contained the following message:
$ error Installer exits with open /host/etc/cni/multus/net.d/v2-2-istio-cni.kubeconfig.tmp.841118073: no such file or directory
Now, if you deploy a v2.2 or v2.1 of the 'ServiceMeshControlPlane' resource, the /etc/cni/multus/net.d/
directory is created, and the istio-cni
pod becomes ready.
OSSM-3993 Previously, Kiali only supported OpenShift OAuth via a proxy on the standard HTTPS port of 443
. Now, Kiali supports OpenShift OAuth over a non-standard HTTPS port. To enable the port, you must set the spec.server.web_port
field to the proxy’s non-standard HTTPS port in the Kiali CR.
OSSM-3936 Previously, the values for the injection_label_rev
and injection_label_name
attributes were hardcoded. This prevented custom configurations from taking effect in the Kiali Custom Resource Definition (CRD). Now, the attribute values are not hardcoded. You can customize the values for the injection_label_rev
and injection_label_name
attributes in the spec.istio_labels
specification.
OSSM-3644 Previously, the federation egress-gateway received the wrong update of network gateway endpoints, causing extra endpoint entries. Now, the federation-egress gateway has been updated on the server side so it receives the correct network gateway endpoints.
OSSM-3595 Previously, the istio-cni
plugin sometimes failed on RHEL because SELinux did not allow the utility iptables-restore
to open files in the /tmp
directory. Now, SELinux passes iptables-restore
via stdin
input stream instead of via a file.
OSSM-3586 Previously, Istio proxies were slow to start when Google Cloud Platform (GCP) metadata servers were not available. When you upgrade to Istio 1.14.6, Istio proxies start as expected on GCP, even if metadata servers are not available.
OSSM-3025 Istiod sometimes fails to become ready. Sometimes, when a mesh contained many member namespaces, the Istiod pod did not become ready due to a deadlock within Istiod. The deadlock is now resolved and the pod now starts as expected.
OSSM-2493 Default nodeSelector
and tolerations
in SMCP not passed to Kiali. The nodeSelector
and tolerations
you add to SMCP.spec.runtime.defaults
are now passed to the Kiali resource.
OSSM-2492 Default tolerations in SMCP not passed to Jaeger. The nodeSelector
and tolerations
you add to SMCP.spec.runtime.defaults
are now passed to the Jaeger resource.
OSSM-2374 If you deleted one of the ServiceMeshMember
resources, then the Service Mesh operator deleted the ServiceMeshMemberRoll
. While this is expected behavior when you delete the last ServiceMeshMember
, the operator should not delete the ServiceMeshMemberRoll
if it contains any members in addition to the one that was deleted. This issue is now fixed and the operator only deletes the ServiceMeshMemberRoll when the last ServiceMeshMember
resource is deleted.
OSSM-2373 Error trying to get OAuth metadata when logging in. To fetch the cluster version, the system:anonymous
account is used. With the cluster’s default bundled ClusterRoles and ClusterRoleBinding, the anonymous account can fetch the version correctly. If the system:anonymous
account loses its privileges to fetch the cluster version, OpenShift authentication becomes unusable.
This is fixed by using the Kiali SA to fetch the cluster version. This also allows for improved security on the cluster.
OSSM-2371 Despite Kiali being configured as "view-only," a user can change the proxy logging level via the Workload details' Logs tab’s kebab menu. This issue has been fixed so the options under "Set Proxy Log Level" are disabled when Kiali is configured as "view-only."
OSSM-2344 Restarting Istiod causes Kiali to flood CRI-O with port-forward requests. This issue occurred when Kiali could not connect to Istiod and Kiali simultaneously issued a large number of requests to istiod. Kiali now limits the number of requests it sends to istiod.
OSSM-2335 Dragging the mouse pointer over the Traces scatterchart plot sometimes caused the Kiali console to stop responding due to concurrent backend requests.
OSSM-2221 Previously, gateway injection in the ServiceMeshControlPlane
namespace was not possible because the ignore-namespace
label was applied to the namespace by default.
When creating a v2.4 control plane, the namespace no longer has the ignore-namespace
label applied, and gateway injection is possible.
In the following example, the oc label
command removes the ignore-namespace
label from a namespace in an existing deployment:
$ oc label namespace istio-system maistra.io/ignore-namespace-
where:
Specified the name of the ServiceMeshControlPlane
namespace.
OSSM-2053 Using Red Hat OpenShift Service Mesh Operator 2.2 or 2.3, during SMCP reconciliation, the SMMR controller removed the member namespaces from SMMR.status.configuredMembers
. This caused the services in the member namespaces to become unavailable for a few moments.
Using Red Hat OpenShift Service Mesh Operator 2.2 or 2.3, the SMMR controller no longer removes the namespaces from SMMR.status.configuredMembers
. Instead, the controller adds the namespaces to SMMR.status.pendingMembers
to indicate that they are not up-to-date. During reconciliation, as each namespace synchronizes with the SMCP, the namespace is automatically removed from SMMR.status.pendingMembers
.
OSSM-1962 Use EndpointSlices
in federation controller. The federation controller now uses EndpointSlices
, which improves scalability and performance in large deployments. The PILOT_USE_ENDPOINT_SLICE flag is enabled by default. Disabling the flag prevents use of federation deployments.
OSSM-1668 A new field spec.security.jwksResolverCA
was added to the Version 2.1 SMCP
but was missing in the 2.2.0 and 2.2.1 releases. When upgrading from an Operator version where this field was present to an Operator version that was missing this field, the .spec.security.jwksResolverCA
field was not available in the SMCP
.
OSSM-1325 istiod pod crashes and displays the following error message: fatal error: concurrent map iteration and map write
.
OSSM-1211 Configuring Federated service meshes for failover does not work as expected.
The Istiod pilot log displays the following error: envoy connection [C289] TLS error: 337047686:SSL routines:tls_process_server_certificate:certificate verify failed
OSSM-1099
The Kiali console displayed the message Sorry, there was a problem. Try a refresh or navigate to a different page.
OSSM-1074 Pod annotations defined in SMCP are not injected in the pods.
OSSM-999 Kiali retention did not work as expected. Calendar times were greyed out in the dashboard graph.
OSSM-797 Kiali Operator pod generates CreateContainerConfigError
while installing or updating the operator.
OSSM-722
Namespace starting with kube
is hidden from Kiali.
OSSM-569 There is no CPU memory limit for the Prometheus istio-proxy
container. The Prometheus istio-proxy
sidecar now uses the resource limits defined in spec.proxy.runtime.container
.
OSSM-535 Support validationMessages in SMCP. The ValidationMessages
field in the Service Mesh Control Plane can now be set to True
. This writes a log for the status of the resources, which can be helpful when troubleshooting problems.
OSSM-449 VirtualService and Service causes an error "Only unique values for domains are permitted. Duplicate entry of domain."
OSSM-419 Namespaces with similar names will all show in Kiali namespace list, even though namespaces may not be defined in Service Mesh Member Role.
OSSM-296 When adding health configuration to the Kiali custom resource (CR) is it not being replicated to the Kiali configmap.
OSSM-291 In the Kiali console, on the Applications, Services, and Workloads pages, the "Remove Label from Filters" function is not working.
OSSM-289 In the Kiali console, on the Service Details pages for the 'istio-ingressgateway' and 'jaeger-query' services there are no Traces being displayed. The traces exist in Jaeger.
OSSM-287 In the Kiali console there are no traces being displayed on the Graph Service.
OSSM-285 When trying to access the Kiali console, receive the following error message "Error trying to get OAuth Metadata".
Workaround: Restart the Kiali pod.
MAISTRA-2735 The resources that the Service Mesh Operator deletes when reconciling the SMCP changed in Red Hat OpenShift Service Mesh version 2.1. Previously, the Operator deleted a resource with the following labels:
maistra.io/owner
app.kubernetes.io/version
Now, the Operator ignores resources that does not also include the app.kubernetes.io/managed-by=maistra-istio-operator
label. If you create your own resources, you should not add the app.kubernetes.io/managed-by=maistra-istio-operator
label to them.
MAISTRA-2687 Red Hat OpenShift Service Mesh 2.1 federation gateway does not send the full certificate chain when using external certificates. The Service Mesh federation egress gateway only sends the client certificate. Because the federation ingress gateway only knows about the root certificate, it cannot verify the client certificate unless you add the root certificate to the federation import ConfigMap
.
MAISTRA-2635 Replace deprecated Kubernetes API. To remain compatible with OpenShift Container Platform 4.8, the apiextensions.k8s.io/v1beta1
API was deprecated as of Red Hat OpenShift Service Mesh 2.0.8.
MAISTRA-2631 The WASM feature is not working because podman is failing due to nsenter binary not being present. Red Hat OpenShift Service Mesh generates the following error message: Error: error configuring CNI network plugin exec: "nsenter": executable file not found in $PATH
. The container image now contains nsenter and WASM works as expected.
MAISTRA-2534 When istiod attempted to fetch the JWKS for an issuer specified in a JWT rule, the issuer service responded with a 502. This prevented the proxy container from becoming ready and caused deployments to hang. The fix for the community bug has been included in the Service Mesh 2.0.7 release.
MAISTRA-2411 When the Operator creates a new ingress gateway using spec.gateways.additionaingress
in the ServiceMeshControlPlane
, Operator is not creating a NetworkPolicy
for the additional ingress gateway like it does for the default istio-ingressgateway. This is causing a 503 response from the route of the new gateway.
Workaround: Manually create the NetworkPolicy
in the istio-system
namespace.
MAISTRA-2401 CVE-2021-3586 servicemesh-operator: NetworkPolicy resources incorrectly specified ports for ingress resources. The NetworkPolicy resources installed for Red Hat OpenShift Service Mesh did not properly specify which ports could be accessed. This allowed access to all ports on these resources from any pod. Network policies applied to the following resources are affected:
Galley
Grafana
Istiod
Jaeger
Kiali
Prometheus
Sidecar injector
MAISTRA-2378 When the cluster is configured to use OpenShift SDN with ovs-multitenant
and the mesh contains a large number of namespaces (200+), the OpenShift Container Platform networking plugin is unable to configure the namespaces quickly. Service Mesh times out causing namespaces to be continuously dropped from the service mesh and then reenlisted.
MAISTRA-2370 Handle tombstones in listerInformer. The updated cache codebase was not handling tombstones when translating the events from the namespace caches to the aggregated cache, leading to a panic in the go routine.
MAISTRA-2117 Add optional ConfigMap
mount to operator. The CSV now contains an optional ConfigMap
volume mount, which mounts the smcp-templates
ConfigMap
if it exists. If the smcp-templates
ConfigMap
does not exist, the mounted directory is empty. When you create the ConfigMap
, the directory is populated with the entries from the ConfigMap
and can be referenced in SMCP.spec.profiles
. No restart of the Service Mesh operator is required.
Customers using the 2.0 operator with a modified CSV to mount the smcp-templates ConfigMap can upgrade to Red Hat OpenShift Service Mesh 2.1. After upgrading, you can continue using an existing ConfigMap, and the profiles it contains, without editing the CSV. Customers that previously used ConfigMap with a different name will either have to rename the ConfigMap or update the CSV after upgrading.
MAISTRA-2010 AuthorizationPolicy does not support request.regex.headers
field. The validatingwebhook
rejects any AuthorizationPolicy with the field, and even if you disable that, Pilot tries to validate it using the same code, and it does not work.
MAISTRA-1979 Migration to 2.0 The conversion webhook drops the following important fields when converting SMCP.status
from v2 to v1:
conditions
components
observedGeneration
annotations
Upgrading the operator to 2.0 might break client tools that read the SMCP status using the maistra.io/v1 version of the resource.
This also causes the READY and STATUS columns to be empty when you run oc get servicemeshcontrolplanes.v1.maistra.io
.
MAISTRA-1947 Technology Preview Updates to ServiceMeshExtensions are not applied.
Workaround: Remove and recreate the ServiceMeshExtensions
.
MAISTRA-1983 Migration to 2.0 Upgrading to 2.0.0 with an existing invalid ServiceMeshControlPlane
cannot easily be repaired. The invalid items in the ServiceMeshControlPlane
resource caused an unrecoverable error. The fix makes the errors recoverable. You can delete the invalid resource and replace it with a new one or edit the resource to fix the errors. For more information about editing your resource, see [Configuring the Red Hat OpenShift Service Mesh installation].
MAISTRA-1502 As a result of CVEs fixes in version 1.0.10, the Istio dashboards are not available from the Home Dashboard menu in Grafana. To access the Istio dashboards, click the Dashboard menu in the navigation panel and select the Manage tab.
MAISTRA-1399 Red Hat OpenShift Service Mesh no longer prevents you from installing unsupported CNI protocols. The supported network configurations has not changed.
MAISTRA-1089 Migration to 2.0 Gateways created in a non-control plane namespace are automatically deleted. After removing the gateway definition from the SMCP spec, you need to manually delete these resources.
MAISTRA-858 The following Envoy log messages describing deprecated options and configurations associated with Istio 1.1.x are expected:
[2019-06-03 07:03:28.943][19][warning][misc] [external/envoy/source/common/protobuf/utility.cc:129] Using deprecated option 'envoy.api.v2.listener.Filter.config'. This configuration will be removed from Envoy soon.
[2019-08-12 22:12:59.001][13][warning][misc] [external/envoy/source/common/protobuf/utility.cc:174] Using deprecated option 'envoy.api.v2.Listener.use_original_dst' from file lds.proto. This configuration will be removed from Envoy soon.
MAISTRA-806 Evicted Istio Operator Pod causes mesh and CNI not to deploy.
Workaround: If the istio-operator
pod is evicted while deploying the control pane, delete the evicted istio-operator
pod.
MAISTRA-681 When the Service Mesh control plane has many namespaces, it can lead to performance issues.
MAISTRA-193 Unexpected console info messages are visible when health checking is enabled for citadel.
Bugzilla 1821432 The toggle controls in OpenShift Container Platform Custom Resource details page does not update the CR correctly. UI Toggle controls in the Service Mesh Control Plane (SMCP) Overview page in the OpenShift Container Platform web console sometimes updates the wrong field in the resource. To update a SMCP, edit the YAML content directly or update the resource from the command line instead of clicking the toggle controls.