OpenShift Container Platform 4.14 release notes

RHCOS now uses Red Hat Enterprise Linux (RHEL) 9.2 packages in OpenShift Container Platform 4.14. These packages ensure that your OpenShift Container Platform instance receives the latest fixes, features, enhancements, hardware support, and driver updates. Excluded from this change, OpenShift Container Platform 4.12 is an Extended Update Support (EUS) release that will continue to use RHEL 8.6 EUS packages for the entirety of its lifecycle.

In OpenShift Container Platform 4.14, you can install a cluster on AWS that uses a shared Virtual Private Cloud (VPC), with a private hosted zone in a different account than the cluster. For more information, see Installing a cluster on AWS into an existing VPC.

With this update, you can opt out of the automatic deletion of the S3 bucket during the cluster bootstrap on AWS. This option is useful when you have security policies that prevent the deletion of S3 buckets.

In OpenShift Container Platform 4.14, you can install a cluster that uses a NAT gateway for outbound networking. This is available as a Technology Preview (TP). For more information, see Additional Azure configuration parameters.

In OpenShift Container Platform 4.14, you can install a cluster on GCP using the pd-balanced disk type. This disk type is only available for compute nodes and cannot be used for control plane nodes. For more information, see Additional GCP configuration parameters.

In OpenShift Container Platform 4.14, you can disable the Build, DeploymentConfig, ImageRegistry and MachineAPI capabilities during installation. You can only disable the MachineAPI capability if you install a cluster with user-provisioned infrastructure. For more information, see Cluster capabilities.

During installation, you can now configure a Microsoft Azure cluster to use Azure AD Workload Identity. With Azure AD Workload Identity, cluster components use short-term security credentials that are managed outside the cluster.

For more information about the short-term credentials implementation for OpenShift Container Platform clusters on Azure, see Azure AD Workload Identity.

To learn how to configure this credentials management strategy during installation, see Configuring an Azure cluster to use short-term credentials.

The user-defined tags feature for Microsoft Azure was previously introduced as Technology Preview in OpenShift Container Platform 4.13 and is now generally available in OpenShift Container Platform 4.14. For more information, see Configuring the user-defined tags for Azure.

You can enable confidential VMs when you install your cluster on Azure. You can use confidential computing to encrypt the virtual machine guest state storage during installation. This feature is in Technology Preview due to known issues which are listed in the Known Issues section of this page. For more information, see Enabling confidential VMs.

You can enable trusted launch features when you install your cluster on Azure as a Technology Preview. These features include secure boot and virtualized Trusted Platform Modules. For more information, see Enabling trusted launch for Azure VMs.

You can now configure user-defined labels and tags in Google Cloud Platform (GCP) for grouping resources and for managing resource access and cost. User-defined labels can be applied only to resources created with the OpenShift Container Platform installation program and its core components. User-defined tags can be applied only to resources created with the OpenShift Container Platform Image Registry Operator. For more information, see Managing the user-defined labels and tags for GCP.

In OpenShift Container Platform 4.14, you can install a cluster on Microsoft Azure in a restricted network for installer-provisioned infrastructure (IPI) and user-provisioned infrastructure (UPI). For IPI, you can create an internal mirror of the installation release content on an existing Azure Virtual Network (VNet). For UPI, you can install a cluster on Microsoft Azure by using infrastructure that you provide. For more information, see Installing a cluster on Azure in a restricted network and Installing a cluster on Azure in a restricted network with user-provisioned infrastructure.

You can now specify the installation disk using a by-path device alias, such as deviceName: "/dev/disk/by-path/pci-0000:01:00.0-scsi-0:0:0:0", when you install a cluster on bare metal with installer-provisioned infrastructure. You can also specify this parameter during Agent-based installations. This type of disk alias persists across reboots. For more information, see Configuring the install-config.yaml file for bare metal or About root device hints for Agent-based installations.

By default, the installation program creates and attaches security groups to control plane and compute machines. The rules associated with the default security groups cannot be modified.

With OpenShift Container Platform 4.14, if you deploy a cluster to an existing Amazon Virtual Private Cloud (VPC), you can apply additional existing AWS security groups to control plane and compute machines. These security groups must be associated with the VPC that you are deploying the cluster to. Applying custom security groups can help you meet the security needs of your organization, in such cases where you must control the incoming or outgoing traffic of these machines. For more information, see Applying existing AWS security groups to the cluster.

OpenShift Container Platform 4.14 uses Kubernetes 1.27, which removed a deprecated API.

A cluster administrator must provide a manual acknowledgment before the cluster can be updated from OpenShift Container Platform 4.13 to 4.14. This is to help prevent issues after updating to OpenShift Container Platform 4.14, where APIs that have been removed are still in use by workloads, tools, or other components running on or interacting with the cluster. Administrators must evaluate their cluster for any APIs in use that will be removed and migrate the affected components to use the appropriate new API version. After this is done, the administrator can provide the administrator acknowledgment.

All OpenShift Container Platform 4.13 clusters require this administrator acknowledgment before they can be updated to OpenShift Container Platform 4.14.

For more information, see Preparing to update to OpenShift Container Platform 4.14.

Deploying a three-node cluster is supported on Nutanix as of OpenShift Container Platform 4.14. This type of OpenShift Container Platform cluster is a more resource efficient cluster. It consists of only three control plane machines, which also act as compute machines. For more information, see Installing a three-node cluster on Nutanix.

In OpenShift Container Platform 4.14, using Confidential VMs when installing your cluster is generally available. Confidential VMs are currently not supported on 64-bit ARM architectures. For more information, see Enabling Confidential VMs.

You can now specify one or more root volume types in RHOSP, by using the rootVolume.types parameter. This parameter is available for both control plane and compute machines.

You can provision bootstrap, control plane, and compute nodes with static IP addresses in environments where Dynamic Host Configuration Protocol (DHCP) does not exist.

After you have deployed your cluster to run nodes with static IP addresses, you can scale a machine to use one of these static IP addresses. Additionally, you can use a machine set to configure a machine to use one of the configured static IP addresses.

For more information, see the "Static IP addresses for vSphere nodes" section in the Installing a cluster on vSphere document.

The Bare Metal Host Custom Resource (CR) now contains the ValidatingWebhooks parameter. With this parameter, the Bare Metal Operator now catches any configuration errors before accepting the CR, and returns a message with the configuration errors to the user.

For OpenShift Container Platform 4.14, you can quickly install a cluster on Amazon Web Services (AWS) to extend compute nodes to Local Zone locations. After you add zone names to the installation configuration file, the installation program fully automates the creation of required resources, network and compute, on each Local Zone. For more information, see Intall a cluster quickly in AWS Local Zones.

This release includes changes that improve the experience of installing and updating clusters that use the Cloud Credential Operator (CCO) in manual mode for cloud provider authentication. The following parameters for the oc adm release extract command simplify the manual configuration of cloud credentials:

To understand how to use these parameters, see the installation procedure for your configuration and the procedures in Preparing to update a cluster with manually maintained credentials.

OpenShift Container Platform 4.14 includes a new VMware vSphere configuration parameter for use on installer-provisioned infrastructure: template. By using this parameter, you can now specify the absolute path to a pre-existing Red Hat Enterprise Linux CoreOS (RHCOS) image template or virtual machine in the installation configuration file. The installation program can then use the image template or virtual machine to quickly install RHCOS on vSphere hosts.

This installation method is an alternative to uploading an RHCOS image on vSphere hosts.

OpenShift Container Platform 4.14 is now supported on 64-bit ARM architecture-based Google Cloud Platform installer-provisioned and user-provisioned infrastructures. You can also now use the oc mirror CLI plug-in disconnected environments on 64-bit ARM clusters. For more information about instance availability and installation documentation, see Supported installation methods for different platforms.

By default, the installation program downloads and installs the Red Hat Enterprise Linux CoreOS (RHCOS) image that is used to boot control plane and compute machines. With this enhancement, you can now override the default behavior by modifying the installation configuration file (install-config.yaml) to specify a custom RHCOS image. Before you deploy the cluster, you can modify the following installation parameters:

For more information about these parameters, see Additional Azure configuration parameters.

OpenShift Container Platform 4.14 expands support for installing single-node OpenShift on cloud providers. Installation options for single-node OpenShift include Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure. For more information about the supported platforms, see Supported cloud providers for single node openshift.

OpenShift Container Platform 4.14 clusters with multi-architecture compute machines are now supported on Google Cloud Platform (GCP) as a Day 2 operation. OpenShift Container Platform clusters with multi-architecture compute machines on bare metal installations are now generally available. For more information on clusters with multi-architecture compute machines and supported platforms, see About clusters with multi-architecture compute machines.

With this release, there are several updates to the Administrator perspective of the web console. You can now perform the following actions:

With this release, there are several updates to the Developer perspective of the web console. You can now perform the following actions:

With OpenShift Container Platform 4.14, oc-mirror supports multi-arch OCI local images for catalogs.

OCI layouts consist of an index.json file that identifies the images held within them on disk. This index.json file can reference any number of single or multi-arch images. However, oc-mirror only references a single image at a time in a given OCI layout. The image stored in the OCI layout can be a single-arch image, that is, an image manifest or a multi-arch image, that is, a manifest list.

The ImageSetConfiguration stores the OCI images. After processing the catalog, the catalog content adds new layers representing the content of all images in the layout. The ImageBuilder is modified to handle image updates for both single-arch and multi-arch images.

With OpenShift Container Platform 4.14, a new oc command-line interface (CLI) flag, --web is now available for the oc login command.

With this enhancement, you can log in by using a web browser, so that you do not need to insert your access token into the command line.

For more information, see Logging in to the OpenShift CLI using a web browser.

A new oc CLI flag, --import-mode, has been added to the oc new-build command. With this enhancement, you can set the --import-mode flag to Legacy or PreserverOriginal, so that you trigger builds by using a single sub-manifest or all manifests.

A new oc CLI flag, --import-mode, has been added to the oc new-app command. With this enhancement, you can set the --import-mode flag to Legacy or PreserverOriginal, and then create new applications by using a single sub-manifest or all manifests.

For more information, see Setting the import mode.

Starting in OpenShift Container Platform 4.14, Extended Update Support (EUS) is extended to the IBM Z® platform. For more information, see the OpenShift EUS Overview.

The IBM Z® and IBM® LinuxONE release on OpenShift Container Platform 4.14 adds improvements and new capabilities to OpenShift Container Platform components and concepts.

This release introduces support for the following features on IBM Z® and IBM® LinuxONE:

OpenShift Container Platform now supports configuring Red Hat Enterprise Linux CoreOS (RHCOS) nodes for IBM Secure Execution on IBM Z® and IBM® LinuxONE (s390x architecture).

For installation instructions, see the following documentation:

Starting in OpenShift Container Platform 4.14, Extended Update Support (EUS) is extended to the IBM Power® platform. For more information, see the OpenShift EUS Overview.

The IBM Power® release on OpenShift Container Platform 4.14 adds improvements and new capabilities to OpenShift Container Platform components.

This release introduces support for the following features on IBM Power®:

With this release, you can now require your workloads to use a specific security context constraint (SCC). By setting a specific SCC, you can prevent the SCC that you want from being preempted by another SCC in the cluster. For more information, see Configuring a workload to require a specific SCC.

With this release, the following system namespaces are always set to the privileged pod security admission profile:

For more information, see Privileged namespaces.

With this release, if a user manually modifies a pod security admission label from the automatically labeled value on a label-synchronized namespace, synchronization is disabled for that label. Users can enable synchronization again, if necessary. For more information, see Pod security admission synchronization namespace exclusions.

With this release, some Operators managed by Operator Lifecycle Manager (OLM) on Amazon Web Services (AWS) clusters can use the Cloud Credential Operator (CCO) in manual mode with the Security Token Service (STS). These Operators authenticate with limited-privilege, short-term credentials that are managed outside the cluster. For more information, see Token authentication for Operators on cloud providers.

With this release, if the noProxy field is set and the route is reachable without the cluster-wide proxy, the Authentication Operator will bypass the proxy and perform connection checks directly through the configured ingress route. Previously, the Authentication Operator always performed connection checks through the cluster-wide proxy, regardless of the noProxy setting. For more information, see Configuring the cluster-wide proxy.

During cluster installation on vSphere, you can configure IPv6 as the primary IP address family on a dual-stack cluster. To enable this feature when installing a new cluster, specify an IPv6 address family before an IPv4 address family for the machine network, cluster network, service network, API VIPs, and ingress VIPs.

The OVN-Kubernetes network plugin supports defining additional default gateways for specific workloads. Both IPv4 and IPv6 address families are supported. You define each default gateway by using the AdminPolicyBasedExternalRoute object, in which you can specify two types of next hops, static and dynamic:

The next hops that you define are scoped by a namespace selector that you specify. You can then use the external gateway for specific workloads that match the namespace selector.

For more information, refer to Configure an external gateway through a secondary network interface.

Ingress Node Firewall Operator was designated a Technology Preview feature in OpenShift Container Platform 4.12. With this release, Ingress Node Firewall Operator is generally available. You can now configure firewall rules at the node level. For more information, see Ingress Node Firewall Operator.

With this release, the Open vSwitch (OVS) networking stack can dynamically use non-reserved CPUs. This dynamic use of non-reserved CPUs occurs by default in nodes in a machine config pool that has a performance profile applied to it. The dynamic use of available, non-reserved CPUs maximizes compute resources for OVS and minimizes network latency for workloads during periods of high demand. OVS remains unable to dynamically use isolated CPUs assigned to containers in Guaranteed QoS pods. This separation avoids disruption to critical application workloads.

In previous releases of the Whereabouts IPAM CNI plugin, only one IP address could be assigned per network interface.

Now, Whereabouts supports the assignment of an arbitrary number of IP addresses to support dual-stack IPv4/IPv6 functionality. See Creating a configuration for assignment of dual-stack IP addresses dynamically.

With this release, you can exclude advertising the Non-Uniform Memory Access (NUMA) node for the SR-IOV network to the Topology Manager. By not advertising the NUMA node for the SR-IOV network, you can permit more flexible SR-IOV network deployments during NUMA-aware pod scheduling.

For example, in some scenarios, it is a priority to maximize CPU and memory resources for a pod on a single NUMA node. By not providing a hint to the Topology Manager about the NUMA node for the pod’s SR-IOV network resource, the Topology Manager can deploy the SR-IOV network resource and the pod CPU and memory resources to different NUMA nodes. In earlier OpenShift Container Platform releases, the Topology Manager attempted to place all resources on the same NUMA node only.

For more information about this more flexible SR-IOV network deployment during NUMA-aware pod scheduling, see Exclude the SR-IOV network topology for NUMA-aware scheduling.

With this release, OpenShift Container Platform is updated to HAProxy 2.6.

Previously, the maximum length of the syslog message in the Ingress Controller was 1024 bytes. Now, the maximum value can be increased. For more information, see Allow the Ingress Controller to modify the HAProxy log length when using a sidecar.

With this release, you can access the NMstate Operator and resources such as the NodeNetworkState (NNS), NodeNetworkConfigurationPolicy (NNCP), and NodeNetworkConfigurationEnhancement (NNCE) from the web console. In the Administrator perspective of the console from the Networking page you can access NNCP, NNCE from the NodeNetworkConfigurationPolicy page, and NNS on the NodeNetworkState page. For more information about NMState resources and how to update them in the console, see Updating node network configuration.

IPsec is now supported on the IBM Cloud platform for clusters that use the OVN-Kubernetes network plugin, which is the default in OpenShift Container Platform 4.14. For more information, see Configuring IPsec encryption.

OpenShift Container Platform now supports encryption of external traffic, also known as north-south traffic. IPsec already supports encryption of network traffic between pods, known as east-west traffic. You can use both features in conjunction to provide full in-transit encryption for OpenShift Container Platform clusters. This is available as a Technology Preview feature.

To use this feature, you need to define an IPsec configuration tuned for your network infrastructure. For more information, refer to Enabling IPsec encryption for external IPsec endpoints.

With this release, you can use Kubernetes NMState Operator in single-stack IPv6 clusters.

With this update, you can use an egressService custom resource (CR) to manage egress traffic for pods behind a load balancer service. This is available as a Technology Preview feature.

You can use the egressService CR to manage egress traffic in the following ways:

For more information, see Configuring an egress service.

With this update, you can specify a Virtual Routing and Forwarding (VRF) instance in a BGPPeer custom resource. MetalLB can advertise services through the interfaces belonging to the VRF. This is available as a Technology Preview feature. For more information, see Exposing a service through a network VRF.

With this update, you can associate a Virtual Routing and Forwarding (VRF) instance with a network interface by using a NodeNetworkConfigurationPolicy custom resource. By associating a VRF instance with a network interface, you can support traffic isolation, independent routing decisions, and the logical separation of network resources. This feature is available as a Technology Preview feature. For more information, see Example: Network interface with a VRF instance node network configuration policy.

Support for the Broadcom BCM57504 network interface controller is now available for the SR-IOV Network Operator.For more information, see Supported devices.

With this release, the Red Hat OpenShift Networking OVN-Kubernetes network plugin allows the configuration of secondary network interfaces for pods. As a secondary network, OVN-Kubernetes supports both layer 2 switched and localnet switched topology networks. For more information about OVN-Kubernetes as a secondary network, see Configuration for an OVN-Kubernetes additional network.

Starting with this release, global IP address forwarding is disabled on OVN-Kubernetes based cluster deployments to prevent undesirable effects for cluster administrators with nodes acting as routers. OVN-Kubernetes now enables and restricts forwarding on a per-managed interface basis.

You can control IP forwarding for all traffic on OVN-Kubernetes managed interfaces by using the gatewayConfig.ipForwarding specification in the Network resource. Specify Restricted to forward all traffic related to OVN-Kubernetes only. Specify Global to allow forwarding of all IP traffic. For new installations, the default is Restricted. Clusters upgraded to 4.14 are not affected by this change; the IP forwarding behavior remains unchanged and continues to be globally enabled. For more information about enabling IP forwarding globally, see Enabling IP forwarding globally.

Admin Network Policy is available as a Technology Preview feature. You can enable AdminNetworkPolicy and BaselineAdminNetworkPolicy resources, which are part of the Network Policy V2 API, in clusters running the OVN-Kubernetes CNI plugin. Cluster administrators can apply cluster-scoped policies and safeguards for an entire cluster before namespaces are created. Network administrators can secure clusters by enforcing network traffic controls that cannot be overridden by users. Network administrators can enforce optional baseline network traffic controls that can be overridden by users in the cluster, if necessary. Currently, these APIs support only expressing policies for intra-cluster traffic.

With this release, the ability to create a MAC-VLAN, IP-VLAN, and VLAN subinterface based on a master interface in a container namespace is generally available. You can use this feature to create the master interfaces as part of the pod network configuration in a separate network attachment definition. You can then base the VLAN, MACVLAN or IPVLAN on this interface without knowing the network configuration of the node. For more information, see About configuring the master interface in the container network namespace.

OpenShift Container Platform release now supports configuring the all-multicast mode by using the tuning CNI plugin. This update eliminates the need to grant the NET_ADMIN capability to the pod’s Security Context Constraints (SCC), enhancing security by minimizing potential vulnerabilities for your pods.

For further information about all-multicast mode, see About all-multicast mode.

This release introduces a new Container Network Interface (CNI) network plugin type: the Tanzu Application Platform (TAP) device plugin. You can use this plugin to create TAP devices within containers, which enables user-space programs to handle network frames and act as an interface that receives frames from and that sends frames to user-space applications instead of through traditional network interfaces. For more information, see Configuration for a TAP additional network.

In OpenShift Container Platform version 4.14 and later, DPDK applications that need to inject traffic to the kernel can run in non-privileged pods with the help of the TAP CNI plugin. For more information, see Using the TAP CNI to run a rootless DPDK workload with kernel access.

Certain HTTP request and response headers can now be set or deleted either globally by using an Ingress Controller or for specific routes. You can set or delete the following headers:

For more information, see Setting or deleting HTTP request and response headers in an Ingress Controller and Setting or deleting HTTP request and response headers in a route.

You can use egress IPs addresses on additional network interfaces is generally available. This feature provides OpenShift Container Platform administrators with a greater level of control over networking aspects such as routing, addressing, segmentation, and security policies. You can also route workload traffic over specific network interfaces for purposes such as traffic segmentation or meeting specialized requirements.

For more information, see Considerations for using an egress IP on additional network interfaces.

With this update, you can configure the SR-IOV Network Operator to drain nodes in parallel during network policy updates. The option to drain nodes in parallel enables faster rollouts of SR-IOV network configurations. You can use the SriovNetworkPoolConfig custom resource to configure parallel node draining and define the maximum number of nodes in the pool that the Operator can drain in parallel.

For more information, see Configuring parallel node draining during SR-IOV network policy updates.

In the OpenShift Container Platform 4,13 release, the 168.254.0.0/16 IP address range was the reserved IP address range that the Open Virtual Network Infrastructure Controller used for the transit switch subnet. For OpenShift Container Platform 4.14, the Open Virtual Network Infrastructure Controller uses 100.88.0.0/16 as the reserved IP address range. The 100.88.0.0/16 range must not conflict with any connected networks. See OCPBUGS-20178 and CIDR range definitions.

With this release, the Image Registry Operator is now an optional component. This feature helps reduce the overall resources footprint of OpenShift Container Platform in Telco environments when the Image Registry Operator is not needed. For more information about disabling the Image Registry Operator, see Selecting cluster capabilities.

With this release, the logical volume manager (LVM) cluster custom resource (CR) provides OR logic in the deviceSelector setting. In previous releases, specifying the paths setting for device paths used AND logic only. With this release, you can also specify the optionalPaths setting, which supports OR logic. For more information, see the CR examples in Persistent storage using logical volume manager storage.

With this release, the logical volume manager (LVM) cluster custom resource (CR) provides support for the ext4 filesystem with the fstype setting under deviceClasses. The default filesystem is xfs. For more information, see the CR examples in Persistent storage using logical volume manager storage.

OpenShift Container Platform 4.14 provides a streamlined and standardized procedure to configure Security Token Service (STS) with the AWS Elastic File Storage (EFS) Container Storage Interface (CSI) Driver Operator.

For more information, see Obtaining a role Amazon Resource Name for Security Token Service.

OpenShift Container Platform 4.14 introduces a new access mode for persistent volumes (PVs) and persistent volume claims (PVCs) called ReadWriteOncePod (RWOP), which can be used only in a single pod on a single node. This is compared to the existing ReadWriteOnce access mode where a PV or PVC can be used on a single node by many pods. This is available as a Technology Preview feature.

For more information, see Access modes.

OpenShift Container Platform is capable of provisioning persistent volumes (PVs) using the Container Storage Interface (CSI) driver for Google Compute Platform (GCP) Filestore Storage. The GCP Filestore CSI Driver Operator was introduced in OpenShift Container Platform 4.12 with Technology Preview support. The GCP Filestore CSI Driver Operator is now generally available. For more information, see Google Compute Platform Filestore CSI Driver Operator.

The Automatic CSI migration for VMware vSphere feature automatically translates in-tree objects to their counterpart CSI representations and, ideally, must be completely transparent to users. Although storage class referencing to the in-tree storage plug-in continues to work, consider switching the default storage class to the CSI storage class.

In OpenShift Container Platform 4.14, CSI migration for vSphere is enabled by default under all circumstances and requires no action by an administrator.

If you are using vSphere in-tree persistent volumes (PVs) and want to upgrade from OpenShift Container Platform 4.12 or 4.13 to 4.14, update vSphere vCenter and ESXI host to 7.0 Update 3L or 8.0 Update 2, otherwise the OpenShift Container Platform upgrade is blocked. If you do not want to update vSphere, you can proceed with an OpenShift Container Platform upgrade by performing an administrator acknowledgment. However, with using the administrator acknowledgment, known issues can occur. Before proceeding with the administrator acknowledgement, carefully read the Knowledge Base article.

For more information, see CSI automatic migration.

The Secrets Store Container Storage Interface (CSI) Driver Operator, secrets-store.csi.k8s.io, allows OpenShift Container Platform to mount multiple secrets, keys, and certificates stored in enterprise-grade external secrets stores into pods as an inline ephemeral volume. The Secrets Store CSI Driver Operator communicates with the provider using gRPC to fetch the mount contents from the specified external secrets store. After the volume is attached, the data in it is mounted into the container’s file system. This is available as a Technology Preview feature. For more information about the Secrets Store CSI driver, see Secrets Store CSI driver.

For information about using the Secrets Store CSI Driver Operator to mount secrets from an external secrets store to a CSI volume, see Providing sensitive data to pods by using an external secrets store.

OpenShift Container Platform 4.14 supports Azure File Container Storage Interface (CSI) Driver Operator with Network File System (NFS) as generally available.

For more information, see NFS support.

Operator Lifecycle Manager (OLM) has been included with OpenShift Container Platform 4 since its initial release. OpenShift Container Platform 4.14 introduces components for a next-generation iteration of OLM as a Technology Preview feature, known during this phase as OLM 1.0. This updated framework evolves many of the concepts that have been part of previous versions of OLM and adds new capabilities.

During this Technology Preview phase of OLM 1.0 in OpenShift Container Platform 4.14, administrators can explore the following features:

For more information, see About Operator Lifecycle Manager 1.0.

With this release, Operators managed by Operator Lifecycle Manager (OLM) can support token authentication when running on Amazon Web Services (AWS) clusters that use the Security Token Service (STS). The Cloud Credential Operator (CCO) is updated to semi-automate provisioning certain limited-privilege, short-term credentials, provided that the Operator author has enabled their Operator to support AWS STS. For more information about enabling OLM-based Operators to support CCO-based workflows with AWS STS, see Token authentication for Operators on cloud providers.

With this release, Operator authors can configure their Operator projects with support for multiple architectures and operating systems, or platforms. Operator authors can configure support for multiple platforms by performing the following actions:

For more information, see Configuring Operator projects for multi-platform support.

The Machine Config Operator now handles distributing certificate authorities for image registries. This change does not affect end users.

With this release, you can query additional metrics to more closely monitor the state of your machines and machine config pools.

For more information on how to use Prometheus, see Viewing a list of available metrics.

With this release, you can now provision an OpenShift Container Platform cluster with Tang-enforced, network-bound disk encryption (NBDE) by using Tang servers that are unreachable during first boot.

For more information, see Configuring an encryption threshold and Configuring disk encryption and mirroring.

In previous OpenShift Container Platform versions, the MCO read and handled certificates directly from machine configuration files. This led to rotation issues and created unwanted situations, such as certificates getting stuck behind a paused machine config pool.

With this release, certificates are no longer templated from bootstrap into machine configuration files. Instead, they are put directly into the Ignition object, written onto a disk using the controller config, and handled by the Machine Config Daemon (MCD) during regular cluster operation. The certs are then visible by using the ControllerConfig resource.

The Machine Config Controller (MCC) holds the following certificate data:

The MCC also handles the image registry certificates and its associated user bundle certificate. This means that certificates are not bound by the machine config pool status and are more timely in their rotation. The previously listed CAs stored in machine configuration files are removed, and the templated files found during cluster installation no longer exist. For more information on how to access these certificates, see Viewing and interacting with certificates.

With this release, control plane machine sets are supported for Nutanix clusters. For more information, see Getting started with the Control Plane Machine Set Operator.

With this release, control plane machine sets are supported for clusters that run on RHOSP.

For more information, see Getting started with the Control Plane Machine Set Operator.

With this release, you can configure a machine set to deploy machines within an existing AWS placement group. You can use this feature with Elastic Fabric Adapter (EFA) instances to improve network performance for machines within the specified placement group. You can use this feature with compute and control plane machine sets.

With this release, you can configure a machine set to deploy machines that use Azure confidential VMs, trusted launch, or both. These machines can use Unified Extensible Firmware Interface (UEFI) security features such as Secure Boot or a dedicated virtual Trusted Platform Module (vTPM) instance.

You can use this feature with compute and control plane machine sets.

With this release, the resource limits for the descheduler operand are removed. This enables the descheduler to be used for large clusters with many nodes and pods without failing due to out-of-memory errors.

The matchLabelKeys parameter for configuring pod topology spread constraints is now generally available in OpenShift Container Platform 4.14. Previously, the parameter was available as a Technology Preview feature by enabling the TechPreviewNoUpgrade feature set. The matchLabelKeys parameter takes a list of pod label keys to select the pods to calculate spreading over.

For more information, see Controlling pod placement by using pod topology spread constraints.

With this release, the MaxUnavailableStatefulSet featureSet configuration parameter is available as Technology Preview feature by enabling the TechPreviewNoUpgrade feature set. You can now define the maximum number of StatefulSet pods that can be unavailable during updates, thereby reducing application downtime when upgrading.

For more information, see Understanding feature gates.

With this release, specifying an unhealthy pod eviction policy for pod disruption budgets (PDBs) is Generally Available in OpenShift Container Platform and has been removed from the TechPreviewNoUpgrade featureSet. This helps evict malfunctioning applications during a node drain.

For more information, see Specifying the eviction policy for unhealthy pods.

Beginning with OpenShift Container Platform 4.14, new installs use Control Groups version 2 by default, also known as cgroup v2, cgroup2, or cgroupsv2. This enhancement includes many bug fixes, performance improvements, and the ability to integrate with new features. cgroup v1 is still used in upgraded clusters that have initial installation dates prior to OpenShift Container Platform 4.14. cgroup v1 can still be used by changing the cgroupMode field in the node.config object to v1.

For more information, see Configuring the Linux cgroup version on your nodes.

Setting a time zone for a cron job schedule is now generally available. If a time zone is not specified, the Kubernetes controller manager interprets the schedule relative to its local time zone.

For more information, see Creating cron jobs.

This release includes the following version updates for monitoring stack components and dependencies:

With this release, administrators can create new alerting rules based on core platform metrics. You can now modify settings for existing platform alerting rules by adjusting thresholds and by changing labels. You can also define and add new custom alerting rules by constructing a query expression based on core platform metrics in the openshift-monitoring namespace. This feature was included as a Technology Preview feature in the OpenShift Container Platform 4.12 release, and the feature is now generally available in OpenShift Container Platform 4.14. For more information, see Managing alerting rules for core platform monitoring.

With this release, you can now specify resource requests and limits for all monitoring components, including the following:

In previous versions of OpenShift Container Platform, you could only set options for Prometheus, Alertmanager, Thanos Querier, and Thanos Ruler.

With this release, you can customize Cluster Monitoring Operator (CMO) config map settings for additional node-exporter collectors. The following node-exporter collectors are now optional, and you can enable or disable each one individually in the config map settings:

In addition, you can now exclude network devices from the relevant collector configuration for the netdev and netclass collectors. You can also now use the maxProcs option to set the maximum number of processes that can run node-exporter.

With this release, the monitoring pages in the Observe section of the OpenShift Container Platform web console are deployed as a dynamic plugin. With this change, the Cluster Monitoring Operator (CMO) is now the component that deploys the OpenShift Container Platform web console monitoring plugin resources. You can now use CMO settings to configure the following features of the console monitoring plugin resource:

With this release, the Performance Addon Operator (PAO) must-gather image is no longer required as an argument for the must-gather command to capture debugging data related to low-latency tuning. The functions of the PAO must-gather image are now under the default plugin image used by the must-gather command without any image arguments. For further information about gathering debugging information relating to low-latency tuning, see Collecting low latency tuning debugging data for Red Hat Support.

In this release, the must-gather tool is updated to collect the data of the NUMA Resources Operator with the must-gather image of the Operator. For further information about gathering debugging information for the NUMA Resources Operator, see Collecting NUMA Resources Operator data.

With this release, you have more control over the C-states for your pods. Now, instead of disabling C-states completely, you can specify a maximum latency in microseconds for C-states. You can configure this option in the cpu-c-states.crio.io annotation. This helps to optimize power savings in high-priority applications by enabling some of the shallower C-states instead of disabling them completely. For further information about controlling pod C-states, see Disabling power saving mode for high priority pods.

With this update, you can provision IPv6 address spoke clusters from dual-stack hub clusters. In a zero touch provisioning (ZTP) environment, the HTTP server on the hub cluster that hosts the boot ISO now listens on both IPv4 and IPv6 networks. The provisioning service also checks the baseboard management controller (BMC) address scheme on the target spoke cluster and provides a matching URL for the installation media. These updates offer the ability to provision single-stack, IPv6 spoke clusters from a dual-stack hub cluster.

Dual-stack network configuration is now available for clusters that run on RHOSP. This is a Technology Preview feature. You can configure dual-stack networking during the deployment of a cluster on installer-provisioned infrastructure.

For more information, see Configuring a cluster with dual-stack networking.

In OpenShift Container Platform 4.14, security for clusters that run on RHOSP is enhanced. By default, the OpenStack cloud provider now sets the manage-security-groups option for load balancers to true, ensuring that only node ports that are required for cluster operation are open. Previously, security groups for both compute and control plane machines were configured to open a wide range of node ports for all incoming traffic.

You can opt to use the previous configuration by setting the manage-security-groups option to false in the configuration of a load balancer and ensuring that the security group rules permit traffic from 0.0.0.0/0 on the node ports range 30000 through 32767.

For clusters that are upgraded to 4.14, you must manually remove permissive security group rules that open the deployment to all traffic. For example, you must remove a rule that permits traffic from 0.0.0.0/0 on the node ports range 30000 through 32767.

You can now use GitOps ZTP to include custom CRs in addition to the base source CRs provided by the GitOps ZTP plugin in the ztp-site-generate container. For more information, see Adding custom content to the GitOps ZTP pipeline.

You can now use GitOps ZTP to provision managed clusters that are running different versions of OpenShift Container Platform. This means that the hub cluster and the GitOps ZTP plugin version can be independent of the version of OpenShift Container Platform running on the managed clusters. For more information, see Preparing the GitOps ZTP site configuration repository for version independence.

With this release, you can precache your application workload images before upgrading your applications on single-node OpenShift clusters with Topology Aware Lifecycle Manager. For more information, see Pre-caching user-specified images with TALM on single-node OpenShift clusters.

With this release, you can remove the partitioning table before installation by using the automatedCleaningMode field in the SiteConfig CR. For more information, see Single-node OpenShift SiteConfig CR installation reference.

With this update, you can add the nodeLabels field in the SiteConfig CR to create custom roles for nodes in managed clusters. For more information about how to add custom labels, see Deploying a managed cluster with SiteConfig and GitOps ZTP, Generating GitOps ZTP installation and configuration CRs manually, and single-node OpenShift SiteConfig CR installation reference.

With this release, you can set the control plane hardware speed to one of "Standard", "Slower", or the default, "", which allows the system to decide which speed to use. This is a Technology Preview feature. For more information, see Setting tuning parameters for etcd.

With this update, the apiVIP and ingressVIP fields in the SiteConfig custom resource definition (CRD) are deprecated in favor of the plural forms, apiVIPs and ingressVIPs.

Hosted control planes for OpenShift Container Platform is now Generally Available on the bare-metal and OpenShift Virtualization platforms. Hosted control planes on AWS remains a Technology Preview feature.

In this release, you can schedule application workloads on 64-bit ARM and AMD64 from the same hosted control plane. For more information, see Creating ARM NodePool objects on AWS hosted clusters.

In this release, hosted control planes is available as a Technology Preview feature on IBM Z. For more information, see Configuring the hosting cluster on 64-bit x84 bare metal for IBM Z compute nodes (Technology Preview).

In this release, hosted control planes is available as a Technology Preview feature on IBM Power. For more information, see Configuring the hosting cluster on a 64-bit x86 OpenShift Container Platform cluster to create hosted control planes for IBM Power compute nodes (Technology Preview).

In OpenShift Container Platform 4.14, Insights Operator can now run gather operations on demand. For more information about running gather operations on demand, see Running an Insights Operator gather operation.

In OpenShift Container Platform 4.14 Technology Preview clusters, Insights Operator runs gather operations in individual pods. This supports the new on demand data gathering feature.

OpenShift SDN CNI is deprecated as of OpenShift Container Platform 4.14. It is currently planned that the network plugin will not be an option for new installations in the next minor release of OpenShift Container Platform. In a subsequent future release, the OpenShift SDN network plugin is planned to be be removed and no longer supported. Red Hat will provide bug fixes and support for this feature until removed, but this feature will no longer receive enhancements. As an alternative to OpenShift SDN CNI, you can use OVN Kubernetes CNI instead.

The Service Binding Operator is deprecated and will be removed with the OpenShift Container Platform 4.16 release. Red Hat will provide critical bug fixes and support for this component during the current release lifecycle, but this component will no longer receive feature enhancements.

As of OpenShift Container Platform 4.14, DeploymentConfig objects are deprecated. DeploymentConfig objects are still supported, but are not recommended for new installations. Only security-related and critical issues will be fixed.

Instead, use Deployment objects or another alternative to provide declarative updates for pods.

From OpenShift Container Platform 4.14, you must only use the DefaultCatSrc.yaml CatalogSource CR when updating Operators with Topology Aware Lifecycle Manager (TALM). All other CatalogSource CRs are deprecated and are planned to be removed in a future release. Red Hat will provide bug fixes and support for this feature during the current release lifecycle, but this feature will no longer receive enhancements and will be removed. For more information about DefaultCatSrc CR, see Performing an Operator update.

As of OpenShift Container Platform 4.14, the --cloud parameter for the oc adm release extract command is deprecated. The introduction of the --included and --install-config parameters make the --cloud parameter unnecessary.

For more information, see Simplified installation and update experience for clusters with manually maintained cloud credentials.

Red Hat Virtualization (RHV) as a host platform for OpenShift Container Platform was deprecated and is no longer supported. This platform will be removed from OpenShift Container Platform in a future OpenShift Container Platform release.

Using the REGISTRY_AUTH_PREFERENCE environment variable to specify your preferred location to obtain registry credentials for OpenShift CLI (oc) commands is now deprecated.

OpenShift CLI (oc) commands now default to obtaining the credentials from Podman configuration locations first, but will fall back to checking the deprecated Docker configuration file locations.

Starting in OpenShift Container Platform 4.14, the operators.openshift.io/infrastructure-features group of annotations are deprecated by the group of annotations with the features.operators.openshift.io namespace.

See Deprecated infrastructure feature annotations for the earlier group of annotations, and see Infrastructure features annotations for the latest group.

Kubernetes 1.27 removed the following deprecated API, so you must migrate manifests and API clients to use the appropriate API version. For more information about migrating removed APIs, see the Kubernetes documentation.

As of OpenShift Container Platform 4.14, support for the LatencySensitive feature set is removed.

As of OpenShift Container Platform 4.14, the oc registry login command no longer stores registry credentials in the Docker file locations, such as ~/.docker/config.json. The oc registry login command now stores credentials in the Podman configuration file locations, such as ${XDG_RUNTIME_DIR}/containers/auth.json.