OKD 4.6 release notes

Operator SDK now aligns with the OKD-wide switch to using UBI-8 and Python 3. Downstream base images now use UBI-8 and include Python 3.

The command run --local is deprecated in favor of run local.

The commands run --olm and --kubeconfig are deprecated in favor of run packagemanifests.

The default CRD version changed from apiextensions.k8s.io/v1beta1 to apiextensions.k8s.io/v1 for commands that create or generate CRDs.

The --kubeconfig flag is added to the <run|cleanup> packagemanifests command.

The Ansible Operator is now available as a supported release.

The Ansible Operator now includes a healthz endpoint and liveness probe.

Helm Operators can watch and reconcile when cluster-scoped release resources are changed.

Helm Operators can now reconcile logic by using three-way strategic merge patches for native Kubernetes objects so that array patch strategies are correctly honored and applied.

Helm Operators have the default API version changed to helm.operator-sdk/v1alpha1.

GA: General Availability

DEP: Deprecated

REM: Removed

Previously, certain conditions caused the ingress Operator to not push the CA certificate to its router-certs secret, so the Cluster Authentication Operator could not construct a trust chain to the certificate in its health checks, causing it to go Degraded and prevent an upgrade. The CA is now always included from the default-ingress-cert config map during the default router CA check, so the Cluster Authentication Operator no longer blocks upgrades. (BZ#1866818)

Previously, the Cluster Authentication Operator failed to parse HTML pages that were returned from OIDC servers that ignore Accept: application/json when a login flow that they do not support is requested, because the Operator was expecting a JSON response. As a result, the Operator failed to honor the IDP configuration. The Cluster Authentication Operator no longer fails with an error when an HTML page is returned from an OIDC server because it does not support the requested flow. (BZ#1877803)

Previously, config maps and secrets were not properly validated by the Cluster Authentication Operator, which could cause a new deployment of the OAuth server to roll out with invalid or missing files, causing the pods to crash. Config maps and secrets are now properly validated by the Cluster Authentication Operator, so a new deployment should not roll out when the config maps or secrets referenced contain invalid data. (BZ#1777137)

Previously, the ironic-image container configuration was missing the setting to enable the idrac-redfish-virtual-media boot driver. Because of this, users were unable to select the idrac-virtual-media boot URL for Metal3. The missing ironic-image container configuration is now included, so users are able to select the idrac-virtual-media URL for Metal3. (BZ#1858019)

Previously, certain Dell firmware versions dropped support for configuring persistent boot using Redfish. Updating Dell iDRAC firmware to version 4.20.20.20 resolves the issue. (BZ#1828885)

In this release, an issue that resulted in inspection timeouts if many nodes were inspected at the same time has been fixed. (BZ#1830256)

Previously, the ironic-image container configuration was missing the setting to enable the idrac-redfish-virtual-media boot driver. Because of this, users were unable to select the idrac-virtual-media boot URL for Metal3. The missing ironic-image container configuration is now included, so users are able to select the idrac-virtual-media URL for Metal3. (BZ#1853302)

Previously, the HTTPd container in the metal3 pod in the openshift-machine-api namespace that is used for serving bare metal ironic images allowed directory listings. With this release, directory listings are no longer allowed in this container. (BZ#1859334)"

Errors in buildah libraries could ignore certain HTTP errors. Builds could fail to push images due to temporary issues with the target registry. This bug fix updates buildah to respect these errors when pushing image blobs. As a result, buildah now fails to push an image if the upstream registry is temporarily unavailable. (BZ#1816578)

Previously, the container image signature policy used in OpenShift Container Platform builds did not contain any configuration for local images. When only allowing images from specific registries, postCommit scripts in builds failed because it was not allowed to use local images. The container image signature policy has been updated to always allow images that reference local storage layers directly. Now, builds can successfully complete if they contain a postCommit hook. (BZ#1838372)

Previously, if a Dockerfile used in Docker strategy builds used the ARG instruction to define build arguments before the first FROM instruction occurred in the Dockerfile, that instruction was dropped when the Dockerfile was preprocessed to incorporate any overrides that were specified in the Build or BuildConfig resource. References to those arguments were subsequently not resolved properly while building an image using the preprocessed Dockerfile. The preprocessing logic has been modified to preserve ARG instructions which are encountered before the first FROM instruction when generating the updated Dockerfile contents, so this problem no longer occurs. (BZ#1842982)

Previously, Buildah erased image architecture and OS fields on images. This caused common container tools to fail because the resulting images could not identify their architecture and OS. This bug fix prevents Buildah from overwriting the image and architecture unless there are explicit overrides. This ensures that images always have architecture and OS fields, and the image mismatch warning does not appear. (BZ#1858779)

Previously, Dockerfile builds failed because they did not expand build arguments correctly in some situations. This update fixes the Dockerfile build argument processing, and thus Dockerfile builds now succeed. (BZ#1839683)

Previously, Buildah made an extraneous call to read an image from its blob cache, which caused Source-to-Image (S2I) builds to fail. This issue was fixed in Buildah v1.14.11, which was vendored into OKD builds in 4.6. (BZ#1844469)

Previously, Buildah could not reference images in COPY –from Dockerfile instructions. As a result, multistage Dockerfile builds that contained COPY –from=<image> failed. Buildah has been updated to a version that supports COPY –from instructions. Builds that contain these instructions can now succeed. (BZ#1844596)

Previously, if the replicas field on a machine set was set to a nil value, the autoscaler could not determine the current number of replicas within the machine set and therefore could not perform scaling operations. With this release, the autoscaler uses the last number of observed replicas in the machine set as reported by the replicas field in the status if a nil value is set. (BZ#1852061)

Previously, the autoscaler did not balance workloads across different failure domains if a memory discrepancy of more than 128 MB existed between nodes of the same type. With this release, the maximum memory discrepancy is increased to 256 MB. (BZ#1824215)

Previously, the machine set replicas field did not have a default value. As a result, if the field were not present, the machine set controller failed silently. The replicas field now has a default value. A default of one replica is used if the replicas field is not set. (BZ#1844596)

Previously, the machine health check controller did not check if a machine had been deleted previously before it attempted to delete it. As a result, the controller could send multiple deletion requests, resulting in spurious logging and event reports. The machine health check controller now checks if a machine has been deleted before attempting to delete it. As a result, duplicate logs and events are reduced. (BZ#1844986)

Previously, the machine API Operator updated the cluster Operator machine API when it was in a stable state. As a result, the resource cycled rapidly between states. The resource’s status now changes only after changes are rolled out. The status remains stable. (BZ#1855839)

Previously, setting the ClusterAutoscaler resource values of balanceSimilarNodeGroups, ignoreDaemonSetsUtilization, or skipNodesWithLocalStorage to false did not register when the cluster autoscaler was deployed. These values are now read properly when the cluster autoscaler is deployed. (BZ#1854907)

Rarely, duplicate machine API controller instances could be deployed. As a result, clusters could leak machines that would become inaccessible. Leader election mechanisms are now added to all machine API components to ensure that duplicate instances are not created. Machine API controllers only run the prescribed number of instances. (BZ#1861896)

On oVirt clusters, manual machine scaling could fail. Scaling machines from the web console or CLI now works. (BZ#1817853)

Previously, must-gather did not collect BareMetalHost records. As a result, debugging information could be incomplete. BareMetalHost records are now collected by must-gather. (BZ#1841886)

Previously, on clusters that run on Azure, compute machines converted into a Failed stage at installation. As a result, VMs were not recognized after being created. Attempts to contact the machines flooded logs with errors, and VMs could fail after starting correctly. As a fix, machines in the Creating state are identified as being created already. Logs contain fewer errors, and machines are less likely to fail. (BZ#1836141)

Previously, machine health check could accept negative values for spec.maxUnhealthy. As a result, at negative values, numerous events were produced for each reconciliation attempt. Negative values for spec.maxUnhealthy. Are now treated as 0, which reduces spurious log messages. (BZ#1862556)

Previously, when upgrading from OpenShift Container Platform version 4.5 to version 4.6, some fields were updated to the 4.6 default values. This affected the ability to downgrade from 4.6 to 4.5 because the 4.5 field values were not preserved. Rather than leaving the fields unspecified in 4.5, this bug fix explicitly preserves the 4.5 values so they can be specified as default values again on a downgrade attempt. Now, downgrading from 4.6 to 4.5 can succeed. (BZ#1868376)

Previously, the Cloud Credential Operator leader election used the default values from controller-runtime and as a result, wrote to etcd every two seconds. This release implements custom leader election, which now writes only every 90 seconds and releases the lock immediately on normal shutdown. (BZ#1858403)

The Cluster Version Operator served metrics using HTTP instead of HTTPS, and was subject to man-in-the-middle attacks as a result of unencrypted data. Now, The Cluster Version Operator serves metrics using HTTPS and data is encrypted. (BZ#1809195)

When cluster administrators had configured cluster version overrides, the upgrade process would get stuck. Now, the upgrades are blocked when the override is set. Upgrades will not begin until an administrator removes the overrides. (BZ#1822844)

The Cluster Version Operator used to load trusted CAs from the config map referenced by the proxy configuration’s trustedCA property. The referenced config map is user-maintained, and a user setting corrupted certificates would interrupt the Operator’s access to the Proxy. Now, the Operator loads the trustedCA configuration from the openshift-config-managed/trusted-ca-bundle, which the Network Operator populates when the proxy configuration’s referenced trustedCA config map is valid. (BZ#1797123)

HTTPS signatures retrieved serialized stores, causing potential timeouts before the Cluster Version Operator could complete the tasks. Now, external HTTPS signature retrieval is parallel and all stores will be attempted. (BZ#1840343)

Previously, during z-stream cluster upgrades using the option --to-image, for example oc adm upgrade --to-image, the Cluster Version Operator was using the cluster version being upgraded to, rather than the current cluster version for validation purposes. This caused z-stream upgrades to fail. Now, z-stream cluster upgrades using the option --to-image are allowed even when Cluster Version Operator has Upgradeable=false. (BZ#1822513)

Previously, the Cluster Version Operator (CVO) was not syncing the shareProcessNamespace parameter in the pod spec, which caused the Registry Operator to not update the shareProcessNamespace setting. The CVO now syncs the shareProcessNamespace, DNSPolicy, and TerminationGracePeriodSeconds parameters, fixing the Registry Operator update issues. (BZ#1866554)

Previously, NICs that had the same NIC profile could not be imported successfully, or the wrong network was chosen. The web console now forces users to select the same network, which is not the pod network, for such NICs. (BZ#1852473)

Virtual machines were not displayed when logged in as a non-administrative user, due to the VM list waiting for the virtualmachineimports data to render. Now, the VM list is rendered correctly. (BZ#1843780)

The Create VM wizard Edit Disk modal did not respect cloned PVC namespaces. It was not possible to edit datavolume disks from a different namespace. Now, the disk modal properly registers the correct namespace of datavolume disks. (BZ#1859518)

Virtual machines and templates could not have the same name when referring to a URL source because the datavolume name was hard-coded. Now, an automatically generated unique string is added to the datavolume name and new virtual machines and templates can have the same name. (BZ#1860936)

Network utilization data would show Not Available due to an empty array of data. Now, a check has been implemented and empty arrays are interpreted as no data. (BZ#1850438)

With this release, the Import VM function is removed from the Developer perspective of the web console. (BZ#1876377)

The user interface did not detect older node maintenance CRDs because the EI was searching for the latest version. As a result, the Node Maintenance action would miss the older NodeMaintenance CR if present. Now, the web console observes both NodeMaintenance CRs. (BZ#1837156)

The user interface did not correctly evaluate graceful shutdowns, causing incorrect warnings to appear when shutting down the console. Now, the interface waits for the node pods to load and the correct warnings are displayed upon shutdown. (BZ#1872893)

Previously, the logic that handled COPY or ADD instructions for copying content from the build context did not efficiently filter when a .dockerignore file was present. COPY and ADD would be noticeably slowed by the cumulative overhead of evaluating whether each item in the source location needed to be copied to the destination. This bug fix rewrote the logic and the presence of a .dockerignore file will no longer noticeably slow the speed at which COPY and ADD instructions are handled during a build. (BZ#1828119, BZ#1813258)

Previously, image builds and pushes would fail with the error reading blob from source error message because the builder logic would compute the contents of new layers twice. The logic, which cached layer contents, depended on the products of those calculations remaining consistent. If a new layer’s contents changed between the two computations, the cache was unable to supply the layers contents when they were needed. The contents of new layers are no longer computed twice and image builds and pushes will no longer fail. (BZ#1720730)

Previously, when you tried to delete a Knative application through the Topology view, a false positive error about a non-existing Knative route was reported. This issue is now fixed and the error is no longer displayed. (BZ#1866214)

Previously, the Developer perspective of the web console did not allow images from insecure registries to be imported. This bug fix adds a checkbox that allows users to use the insecure registries in the Deploy image form. (BZ#1826740)

When a user selected the From Catalog option to create an application, the Developer Catalog displayed a blank page instead of a list of templates to create an application. This was caused when the 1.18.0 Jaeger Operator was installed. This issue has now been fixed and the templates are displayed as expected. (BZ#1845279)

When deleting a parallel task in a Pipeline through the Pipeline Builder in the Developer perspective of the web console, the interface was rearranging the tasks connected to the parallel task incorrectly, creating orphan tasks. With this fix, the tasks connected to the deleted parallel task are reconnected with the original Pipeline. (BZ#1856155)

The web console was crashing with a JavaScript exception when the user canceled the creation of a Pipeline through the web console with a side panel opened at the same time. This was fixed by improving the internal state handling. (BZ#1856267)

A user with the required permissions was unable to retrieve and deploy an image from another project. The required role bindings have now been created to fix this issue. (BZ#1843222)

When you tried to deploy an application from a Git repository with the Import from Git function, the Developer perspective of the web console reported a false positive error Git repository is not reachable for private repositories reachable by the cluster. This was fixed by adding information on making the private repository available to the cluster in the error message. (BZ#1877739)

When a Go application was created through the Developer perspective of the web console, a route to the application was not created. This was caused by a bug in build-tools and incorrectly configured ports. The issue has been fixed by picking either the user-provided port or the default port 8080 as the target port. (BZ#1874817)

When you created an application with the Import from Git function, a subsequent change of the application’s Git repository from the web console was not possible. This was caused by changing the application name in subsequent editing of the Git repository URL. This was fixed by making the application name read-only when editing the application Git repository URL. (BZ#1873095)

Previously, a user without administrative or project listing privileges could not see the metrics of any projects. This bug fix removes the checks for user privileges when accessing the cluster metrics. (BZ#1842875)

Users with the @ character in their user names, like user@example.com, could not start a Pipeline from the Developer perspective of the web console. This was caused by a limitation in Kubernetes labels. The issue was fixed by moving the Started by metadata from a Kubernetes label to a Kubernetes annotation. (BZ#1868653)

Previously, when a user selected a metric the QueryEditor would show the query. However, if the user deleted or modified the query and selected the same metric again, the QueryEditor would not update. With this fix, if a query is cleared by the user and they tried to select the same query again, the query input text area will show the query. (BZ#1843387)

The Che Workspace Operator removed support for the Workspace resource and replaced it with the DevWorkspace CRD. As a result, the command line terminal was not enabled with the latest Che Workspace Operator. With this fix, the OpenShift command line terminal was moved to use the DevWorkspace resource. The command line terminal will now be enabled in the OpenShift Console when the Che Workspace Operator is installed. (BZ#1844938)

Previously, the route decorator for Knative services redirected users to a revision specific route if traffic was distributed across multiple revisions. The route decorator has been updated to always point to the Knative base service route. (BZ#1860768)

Previously, the pod did not start when the user added a secret for an external private container registry and imported the container image from the registry. As a result, the deployment was stuck until the service account or the deployment was updated manually. This bug fix allows the new deployments to use the internal container registry to start their pods. With this bug fix, the user can import a container image from an external private container registry without additional changes on the service account or the deployment. (BZ#1926340)

The console used a prior version of the KafkaSource object that used the resources and serviceAccountName fields in their specification. The v1beta1 version of the KafkaSource object removed these fields, due to which the user was unable to create the KafkaSource object with the v1beta1 version. This issue has been fixed now and the user is able to create the KafkaSource object with the v1beta1 version. (BZ#1892695)

Previously, the relative chart URL for downloading the chart to instantiate a helm release was unreachable. This happened because the index.yaml file from the remote repository, referenced in the Helm chart repository, was fetched and used as is. Some of these index files contained relative chart URLs. This issue is fixed by translating relative chart URLs to absolute URLs, which makes the chart URL reachable. (BZ#1916406)

Previously, the users were not able to see any Knative services and sources on the Topology view. This was because the triggers had both a Knative service and an In-Memory Channel as subscribers due to a Null Pointer Exception (NPE). This issue is resolved by fixing the Null Pointer Exception, so that the Knative data model returns proper data and the Knative resources are properly shown on the Topology view. (BZ#1907827)

Previously, the API server could fail to create a resource and return a 409 status code due to a conflict while updating a resource quota resource. This issue is fixed and the {product title} web console will attempt to retry the request while receiving a 409 status code. At most three attempts are made which is often sufficient to complete the request. In the event that a 409 continues to occur, an error will be displayed in the console. (BZ#1928230)

Previously, no helm charts were displayed in the developer catalog because the chart repository httpClient() did not consider any proxy environment variable. This issue is fixed and the helm charts are now being displayed in the developer catalog. (BZ#1919138)

Technology preview badges are being displayed on the Eventing user interface although it would be going for a GA release. The Technology preview badges are now removed from Eventing user interface. (BZ#1899382)

Previously, the application crashed when the correlated pod data was not available for the deployment config. This was because the console deployment fetched two sets of data with the pod status donut shown as soon as deployment config is loaded. If the API returns more than 250 pods, some of the information is skipped and is not available. This issue is fixed and the pod data is available even when the project contains more than 250 pods, thus ensuring that the DeploymentConfig detail page does not crash anymore. (BZ#1921603)

Previously, the static models corresponding to the trigger, subscription, channel, and IMC eventing sources used the beta API version. With the Serverless 0.10 release, the latest supported versions for eventing sources were updated to the v1 version. This bug fix updates the user interface models to point to the latest supported version. (BZ#1896625)

Previously, when conditional tasks failed, the completed pipeline runs showed a permanent pending task for each failed conditional task. This issue has been fixed by disabling the failed conditional tasks and by adding skipped icons to them. This gives a better picture of the state of the pipeline run. (BZ#1916378)

Previously, the user was denied access to pull images from other projects due to insufficient user permissions. This bug fix removes all the user interface checks for role bindings and shows the oc command alert to help users use the command line. With this bug fix, the user is no longer blocked from creating images from different namespaces and is now able to deploy images from other projects. (BZ#1933727)

While creating a sample application, the Developer perspective creates multiple resources that depend on each other and must be completed in a specific order. Previously, the admission plug-in sometimes could not check one of these resources, preventing the Developer perspective from generating the sample application. This issue has been fixed. The code creates the resources in the required order, so creating a sample application is more stable. (BZ#1933666)

Previously, users were not able to create a Knative service as a private service from the Developer perspective. This issue has now been fixed, by updating the label 'networking.knative.dev/visibility': 'cluster-local'. (BZ#1978159)

Previously, the topology URLs created for deployments using a Bitbucket repository in the OKD web console did not work if they included a branch name that contained a slash character. This was due to an issue with the Bitbucket API BCLOUD-9969. The current release mitigates this issue: if a branch name contains a slash, the topology URLs point to the default branch page for the repository. (BZ#1972694)

Previously, pipelines created in the git import flow for private repositories failed to run. This issue has now been fixed by adding the secret name to the pipeline’s ServiceAccount object annotations, and by adding pipeline-specific annotations to the provided secret. (BZ#1970984)

Previously, intermittent invalid memory address or nil pointer dereference errors occurred and were followed by timeouts for Kube API access when running CoreDNS 1.6.6. This is now fixed by correctly handling errors with Endpoint Tombstones. Now, CoreDNS behaves as intended without intermittent panics. (BZ#1868315)

Previously, the DNS Operator repeatedly attempted to update DNS and Service objects in response to default values that were set by the API. With this update, the DNS Operator now considers values that are left unspecified by the DNS Operator to be equal to values in DNS and Service objects. Thus, the DNS Operator no longer updates a DNS or Service object in response to API default values. (BZ#1842741)

Previously, the bootstrap endpoint in ETCDCTL_ENDPOINTS was not removed after the bootstrap node was removed, so etcdctl commands showed unexpected errors. The bootstrap endpoint is no longer added to ETCDCTL_ENDPOINTS, so etcdctl commands do not show errors related to the bootstrap endpoint. (BZ#1835238)

Previously, image imports using digests on manifests lists failed. This update fixes the conversion from manifest list to manifest by using the digest of the selected manifest inside the manifest list. Thus, imports by digest of manifest lists now work as expected. (BZ#1751258)

Previously, some internal packages used an internal error structure, causing a null pointer issue. Now, the internal error interface is returned and nil errors are converted correctly. (BZ#1815562)

The Operator did not generate httpSecrets when empty, causing the value to not set correctly. Now, the Operator generates the httpSecret and uses it for all replicas when the configuration file is created. (BZ#1824834)

Previously, an invalid alert would appear when the image pruner was disabled. This alert has now been removed. (BZ#1845642)

Registry Operator type assertions were made twice for a variable and the second time the result was not checked. This caused false assertions and created panic conditions. Now, checked type assertions are used and the Operator does not panic. (BZ#1879426)

Previously, the Operator bootstrap storageManaged setting was set to true, causing conflicts if the user manually updated the configuration file. Now, an additional configuration field, spec.storage.storageManagementState has been created. Users can indicate Managed or Unmanaged and the Operator will respect the setting. (BZ#1833109)

Removing the Image Registry on OpenStack when content was written to storage resulted in storage for the Image Registry not being removed and a 409 HTTP return code error being logged. This bug fix removes the storage content before removing the storage. Now, when the Image Registry Operator is removed, its storage is also removed. (BZ#1835770)

During installation on OpenStack, if there was a failure to access Swift storage during the Image Registry Operator bootstrap process, it caused an incomplete bootstrap. This resulted in a failure to create the Image Registry configuration resource, which blocked fixes or changes its configuration. This bug fix prevents failure during bootstrap in case of a problem when accessing Swift storage. If there is an error, it is logged allowing the bootstrap to finish and configuration resources to be created. Now, the Image Registry Operator is now more flexible, and if it cannot access Swift storage, it bootstraps the internal Image Registry using a PVC. (BZ#1846263)

The Image Registry Operator avoids calling Azure endpoints too many times, because Azure enforces quotas and previously the Operator was constantly querying for storage account keys. This bug fix caches the keys locally for a set period of time to avoid going remotely to get the keys every time they are needed. (BZ#1853734)

Previously, the Operator interpreted a running job as successful when reporting on the Operator status, even though the job could potentially result in a failed state. Now, the running jobs are ignored when reporting the Operator status. (BZ#1857684)

The image registry purging process failed when running over s3 storage because it was listing directories twice. Now, directories are listed once and the image purging process completes successfully. (BZ#1861304)

Previously, if the Image Registry Operator was removed, the pruner job failed because it could not reach a non-existent registry. Now, the pruner job removes etcd objects only and does not attempt to ping the registry if it is removed. (BZ#1867792)

If a user manually added a bucket name, the Operator did not create the bucket. Now, the Operator creates a bucket successfully based on the user-provided name. (BZ31875013)

Previously, the Image Registry Operator could not get events from the cluster when it encountered Too large resource version error messages. With this release, the client-go library is updated to fix the reflector so that the Operator can recover from Too large resource version error messages. (BZ#1880354)

Previously, the value provided for spec.requests.read/write.maxWaitInQueue in the Image Registry Operator configuration file was not validated. If the provided value was a string that could not be parsed as a duration, the change was not applied, and a message informing about the incorrect value was repeatedly logged. This release adds validation so that a user cannot provide invalid values for this field. (BZ#1833245)

Previously, dependency tracking between image objects was slow when pruning images. Image pruning would sometimes take a long time to complete. The underlying image pruning mechanism has been redesigned. Now, image pruning is faster and has improved parallelism. (BZ#1860163)

Previously, when a machine tried to get the resourcePoolPath, it found multiple resource pools and was unable to resolve the correct one. With this release, a property added to the machine sets with the resourcePoolPath information helps resolve the correct one. (BZ#1852545)

Previously, a hard-coded value was set when calculating the end of the DHCP allocation pool when provisioning the nodes subnet. This caused an error when deploying on an OpenShift Container Platform cluster on OpenStack with a machine CIDR smaller than 18. This bug fix removes hard-coding the number of nodes and, instead, dynamically calculates the end of the DHCP allocation pool. Now, it is possible to deploy a cluster on OpenStack with a machine CIDR of any length, provided it is large enough for all required nodes. (BZ#1871048)

Previously, some available networks were not shown during cluster installation because of an ovirt-engine-sdk-go API error that affected oVirt network parsing. The issue is now resolved. (BZ#1838559)

Previously, in the vSphere web console wizard, only networks of type Network and DistributedVirtualPortgroup were displayed even though OpaqueNetwork is also a valid option. OpaqueNetwork is now an option in the wizard, so that type of network can be selected. (BZ#1844103)

Previously, the Manila Operator did not support custom self-signed certificates, so the Manila Operator failed to deploy Manila CSI driver on some environments that used self-signed certificates. Now, the Operator fetches the user-provided CA certificate from the system config map, mounts it to the driver’s containers, and update the driver’s configuration. As a result, the Manila Operator can deploy and manage the Manila CSI driver on environments with self-signed certificates. (BZ#1839226)

Previously, configuring the platform.aws.userTags parameter to add name or kubernetes.io/cluster/ tags to resources that the installation program creates caused the machine API to fail to identify existing control plane machines and create another set of control plane hosts, which created problems with etcd cluster membership. Now, you can no longer set error-prone tags in the platform.aws.userTags parameter, and your cluster is less likely to have extra control plane hosts and broken etcd clusters. (BZ#1862209)

Previously, health check probes were not defined on load balancers deployed in Azure clusters on user-provisioned infrastructure. Because the load balancers were not defined, they did not detect when an API endpoints were no longer available and still directed traffic to them, which caused client failures. Now, the health check probes are used on the load balancers, and they correctly detect when API endpoints are not available and stop routing traffic to offline nodes. (BZ#1836016)

Previously, the installation program did not accept * as a valid value for the proxy.noProxy field, so you could not create a cluster with no proxy set to * during installation. Now, the installation program accepts * as a valid value for that parameter, so you can set no proxy to * during installation. (BZ#1877486)

Previously, when you installed a cluster in GCP, US was always used as the location, and it was not possible to install a cluster in some regions outside of US. Now, the installation program sets the right location for the region that you specify, and installations succeed in other locations. (BZ#1871030)

Previously, when you installed a cluster on vSphere with installer-provisioned infrastructure, it was possible to assign the same IP address to ingress and the API, which caused the bootstrap machine and one of the control plane machines to have the same IP address. Now, the installation program validates that the IP addresses are different, and the control plane and bootstrap machines have unique IP addresses. (BZ#1853859)

Previously, when an interface obtained a new DHCP4 or DHCP6 lease, the local-dns-prepender did not update the resolv.conf file to include all the required resolvers for the cluster. Now, the dhcp4-change and dhcp6-change action types always make the local-dns-prepender start updates.(BZ#1866534)

Previously, you could not deploy clusters to the following GCP regions: asia-northeast3, asia-southeast-2, us-west3, and us-west4. You can now use these regions. (BZ#1847549)

Previously, the OpenStack installation program used inconsistent output format for the InstanceID() function. It obtained the instance ID from either metadata or by sending requests to the server. In the latter case, the result always had '/' prefix, which is the correct format. If the instance ID came from the metadata, the system failed to verify its node existence and failed because of the error. Because, the metadata format now also contains the '/' prefix there, the ID format is always correct, and the system can always successfully verify node existence. (BZ#1850149)

Previously, provisioning services for the bare metal installer-provisioned infrastructure platform failed when FIPS was enabled for a cluster. With this update, provisioning services run as expected when FIPS is enabled and installation successfully completes. (BZ#1804232)

Previously, you could configure the DHCP range so that the provisioning network consumed the entire subnet, including the cluster provisioning VIP. Thus, installation would fail because a bootstrap VM IP and cluster provisioning IP could not be assigned. This fix validates VIPs to ensure that they do not overlap with the DHCP range. (BZ#1843587)

Previously, installation failed when a certificate was followed by two end-of-line character sequences. This update fixes the certificate authority (CA) certificate trust bundle parser, which now ignores invisible characters. Thus, the CA certificate trust bundle can now feature an arbitrary number of end-of-line character sequences before, between, and after certificates. (BZ#1813354)

Previously, when the interactive installer prompt requested the ExternalNetwork resource for the cluster, all possible network choices were listed, which included invalid options. With this update, the interactive installer now filters the possible options and lists only external networks. (BZ#1881532)

Previously, the bare metal kube-apiserver health check probe used a hard-coded IPv4 address to communicate with the load balancer. This update fixes the health check to use localhost, which covers both IPv4 and IPv6 cases. Additionally, the readyz endpoint for the API server is checked rather than healthz. (BZ#1847082)

Due to a Terraform step that did not list all of its dependent steps, clusters on Red Hat OpenStack Platform (RHOSP) experienced a race condition that sometimes caused the Terraform job to fail with a Resource not found error message. The step is now listed as depends_on to avoid the race condition. (BZ#1734460)

Previously, the cluster API did not validate RHOSP flavors before updating machines. As a result, machines that had invalid flavors failed to boot. Flavors are now validated before updating machines, and machines that have invalid flavors return errors immediately. (BZ#1820421)

Previously, clusters on RHOSP that had periods (.) in their names failed at the bootstrap phase in installation. Periods are no longer allowed in the names of clusters on RHOSP. If a cluster name contains a period, an error message is generated early in the installation process. (BZ#1857158)

Previously, health check probes were not defined for load balancers in user-provisioned infrastructure deployed clusters on AWS. The load balancers did not detect when API endpoints became unavailable, which caused client failures. In this update, health check probes have been added and the load balancers do not route traffic to offline nodes. (BZ#1836018)

Previously, the DiskPostCloneOperation function in the Terraform vSphere provider checked the thin_provisioned and eagerly_scrub properties of virtual machines cloned from Red Hat CoreOS OVA images. The check failed because the provisioning type changed during cloning and did not match the source provisioning type. The DiskPostCloneOperation function now ignores those properties and the Red Hat CoreOS OVA cloning succeeds. (BZ#1862290)

When running ./openshift-install destroy cluster, the installer attempted to remove installer tags before the resources using those tags were removed. The installer subsequently failed to remove the tags. With this update, the installer removes the tags after the corresponding resources are deleted. (BZ#1846125)

Previously, the Cluster Version Operator (CVO) marked a cluster as not upgradeable when the LatencySensitive Feature Gate was in use. With this update, the CVO no longer considers LatencySensitive Feature Gate as a block for cluster upgrades. To resolve the issue, force the upgrade to the latest 4.5.z or stable version, which include the solution. (BZ#1861431)

Previously, the daemon set controller did not clear expectations when recreating a daemon set. As a result, the daemon set might get stuck for five minutes until the expectations expired. Now, the daemon set controller correctly clears the expectations. (BZ#1843319)

UID range allocation is not updated when a project is removed, and thus the UID range can be exhausted on a cluster that has high namespace creation and removal turnover. To resolve the issue, the kube-controller-manager pod is periodically restarted, which triggers the repair procedure and clears the UID range. (BZ#1808588)

Previously, the endpoints controller sent large quantities of API requests at every informer resync interval, which caused degradation in large clusters with many endpoints. Various inconsistencies in storage and the comparison of endpoints caused the endpoints controller to incorrectly assume that there were many additional updates that needed to be made to a cluster than were actually necessary. This fix updates storage and comparison functions for endpoints so that they are more consistent. Thus, the endpoints controller now only sends updates when necessary. (BZ#1854434)

Previously, NotFound errors were mishandled by controller logic. This caused controllers such as deployment, daemon set, and replica set controllers to not be aware of missing pods. This fix updates controllers to correctly react to a pod NotFound event, which indicates that the pod was previously removed. (BZ#1843187)

Previously, when evicting a pod in dry run mode, certain descheduler strategies logged the eviction twice. Now, only a single log entry is created for the eviction. (BZ#1841187)

Previously, the Elasticsearch index metrics were collected in Prometheus by default. As a consequence, Prometheus would run out of storage quickly, due to the size of the index-level metrics, and the user would need to intervene, for example, reducing the Prometheus retention time, to keep Prometheus functioning. The default behavior was changed to not collect the Elasticsearch index metrics. (BZ#1858249)

Because the OpenShift Elasticsearch Operator was not updating the secret hash for the Kibana deployment, Kibana pods would not be restarted if the secret gets updated. The code was changed to correctly update the hash for the deployment, which triggers a the pods to redeploy, as expected. (BZ#1834576)

Because of the introduction of Fluentd init containers, Fluentd could not be deployed in a cluster without deploying the OpenShift Elasticsearch Operator (EO). The code was changed to allow Fluentd without the EO being present. As a result, in a cluster without Elasticsearch, Fluentd works as expected. (BZ#1849188)

Because the ability to open Kibana in an iframe was not expressly denied, it opened Kibana to the possibility of attack, such as clickjacking. The code was changed to explicitly set x-frame-options:deny, which blocks the use of iframes. (BZ#1832783)

In bare metal environments, an infra-dns container runs on each host to support node name resolutions and other internal DNS records. A NetworkManager script also updates the /etc/resolv.conf on the host to point to the infra-dns container. Additionally, when pods are created, they receive their DNS configuration file (the /etc/resolv.conf file) from their hosts.

Keepalived is used to provide high availability (HA) for both the API and default router. The Keepalived instance in each node monitors local health by curling the health endpoint of the local entity, for example the local kube-apiserver. Previously, the curl command failed only when the TCP connection failed, and not on HTTP non-200 errors. This caused Keepalived to sometimes not fail over to another healthy node, even when the local entity was unhealthy, which lead to errors in API requests.

In bare metal environments, some DNS records were hard-coded for IPv4. This caused some records to not be served correctly in IPv6 environments, which might necessitate creating those records in an external DNS server. This bug fix updates the Machine Config Operator (MCO) so that DNS records are now populated correctly based on the Internet Protocol version in use. As a result, internal records are now served correctly in both IPv4 and IPv6. (BZ#1820785)

Previously, kernel arguments specified in machine configs needed to be split out into individual argument strings in the array. These arguments were not validated before being concatenated into an rpm-ostree command. Multiple kernel arguments concatenated using a space, as allowed in a single line in the kernel command line, would create an invalid rpm-ostree command. This bug fix updates the machine config controller to parse each kernelArgument item in a similar manner as the kernel. As a result, users can supply multiple arguments concatenated using a space without errors. (BZ#1812649)

Previously, the control plane would always be schedulable for user workloads in bare metal environments. This bug fix updates the Machine Config Operator (MCO) so that control plane nodes are now correctly configured as NoSchedule in a typical deployment with workers. (BZ#1828250)

Previously, with the Machine Config Operator (MCO), unnecessary API VIP moves could cause client connection errors. This bug fix updates API VIP health checks to limit the number of times it moves. As a result, there are now fewer errors caused by API VIP moves. (BZ#1823950)

The operand’s tab for operand list view was missing. With this bug fix, the issue is resolved. (BZ#1842965)

When IPv6 was disabled, the downloads pod socket could not bind and the downloads pod crashed. If IPv6 is not enabled, IPv4 is now used for the socket. The downloads pod now works regardless of enabling IPv4 and IPv6. (BZ#1846922)

The Operator status display values did not account for manual approval strategy. Therefore, the upgrade available status was displayed, which did not convey that further action was required in order to upgrade. A new status message was added for Operators that are waiting for manual approval to upgrade. You can now clearly tell when an Operator upgrade requires further action.(BZ#1826481)

The catch logic for a failed operand form submission was attempting to access a deep property in the resulting error object that is not always defined. For specific failed request types, this would cause a runtime error. With this bug fix, the issue is resolved. (BZ#1846863)

Victory does not handle all-zero data sets well. Y-axis tick marks for all-zero data sets were repeated zeros. The area chart logic was updated to force a Y-domain of [0,1] when all data sets are all-zero. Now, Y-axis tick marks for all-zero data sets are 0 and 1. (BZ#1856352)

The currently installed version was not shown in the Operator details pane on OperatorHub. Therefore, a user could not determine if the currently installed version was latest. When the currently installed Operator version is not the latest, the installed version is now shown in the Operator details pane on OperatorHub. (BZ#1856353)

The create role binding links were inconsistently namespaced or used cluster links, so the create role binding page was incorrect. This bug fix updates the links to use the namespace, where available, and the cluster in other cases. The correct page is now used. (BZ#1871996)

Previously, the web console did not support Grafana value maps for singlestat panels, so singlestat panels were unable to display Grafana value maps. Support is now added and singlestat panels can display Grafana value maps. (BZ#1866928)

oc debug was updated with BZ#1812813 to create a new debug project with an empty node selector in order to work around a problem where oc debug only works against worker nodes. The web console avoided this problem by asking users to choose a namespace when visiting the Node → Terminal page before the terminal is opened, resulting in inconsistency in the user experience compared to oc. The web console now creates a new debug project with an empty node selector upon visiting the Node → Terminal page. The behavior of the web console Node → Terminal page now aligns with the behavior of oc debug. (BZ#1881953)

When all receivers were deleted, the web console would experience a runtime error and the user was presented with a blank screen. Null checks are now added in the code and users ate presented with a No Receivers empty state screen instead. (BZ#1849556)

In some cases, the value passed to the resource requirements widget in the legacy operand creation form might not be an immutablejs map instance. A runtime error was thrown when trying to reference the immutablejs Map.getIn function on the resource requirement widget current value. Use optional chaining when referencing the immutablejs Map.getIn function. No runtime error is thrown and the widget is rendered without a value. (BZ#1883679)

A blank page appeared when using search on the imagemanifestvuln resource. The component was using props.match.params.ns, which was sometimes undefined. As a result, there was a runtime error. This issue is now resolved. (BZ#1859256)

Previously, the action menu to the right of operand lists could close immediately after opening. This could be seen on the Installed Operators details page when clicking one of the tabs for the Operator-provided APIs. The menu now functions correctly and will no longer close without user interaction. (BZ#1840706)

There was a missing keyword field in the API and users were unable to search by keyword in OperatorHub. With this bug fix, the issue is resolved. (BZ#1840786)

Previously, the OperatorHub in the web console sometimes showed the incorrect icon for an Operator. The issue has been resolved in this release. (BZ#1844125)

In this release, a broken link to the cluster monitoring documentation from the web console OperatorHub install page is corrected. (BZ#1856803)

Previously, clicking Create EtcdRestore on the EtcdRestores page caused the web console to stop responding. With this release, the Create EtcdRestore form view workflow loads correctly. (BZ#1845815)

Previously, the operand form array and object fields did not have logic to retrieve and show field descriptions on the form. As a result, descriptions were not rendered for array or object type fields. This bug fix adds logic to now display array and object field descriptions on the operand creation form. (BZ#1854198)

Previously, the Deployment Configuration Overview page sometimes crashed with the error e is undefined when a new pod was starting up. The issue has been resolved in this release. (BZ#1853705)

Previously, the cluster upgrade interface in the web console was visible on OpenShift Dedicated, even though OpenShift Dedicated users cannot perform cluster upgrades. The cluster upgrade interface is now hidden for OpenShift Dedicated. (BZ#1874257)

Previously, empty objects in an operand template were pruned when submitting the form or switching between the form and YAML view. With this release, template data is used as a mask when pruning empty structures from the form data, and only values that are not defined in the template are pruned. Empty values that are defined in a template remain. (BZ#1847921)

The tooltip details were sometimes truncated when hovering over the Cluster Logging donut chart. This bug fix enables the entire tooltip description to display for this chart. (BZ#1842408)

For the Registry field on the Create Instance page, some of the text in the schema property descriptions matched the format used to create fuzzy hyperlinks in Linkify. This resulted in unintended hyperlinks. This bug fix disables the fuzzy link feature in Linkify. Now, only URL strings using a proper protocol scheme are rendered as hyperlinks. (BZ#1841025)

The AWS Secret field always displayed the loading icon even when the ListDropdown component was not in a loading state. This bug fixes the component logic so that the loading icon only displays when the drop-down list is in a loading state. A placeholder displays if the drop-down list is not in a loading state. (BZ#1845817)

Previously, Resource Log pages in the web console became slow or unresponsive when the logs contained long lines. This bug fixes the performance issues by limiting the number of characters per log line that displays on the Resource Log page and providing an alternate method to view the full log content. (BZ#1874558)

Previously, the node file system calculations for Used and Total were incorrect due to an incorrect query. Now, the query is updated and calculates the data correctly. (BZ31874028)

The Overview page in the web console incorrectly included the container level usage metric to display network utilization data. This bug fix replaces the incorrect metric with the node level usage metric to accurately display network utilization data. (BZ#1855556)

Previously, the Created At timestamp format for Operators did not require a specific format, resulting in the timestamp displaying inconsistently on the web console. Now, if the Created At timestamp value is entered as a valid date for an Operator, the timestamp displays in a consistent manner with other timestamps in the console. If the value is not entered using a valid date format, the Created At timestamp displays as an invalid string. (BZ#1855378)

The web console displayed an old logo for OperatorHub. This bug fix replaces the old logo with the current logo. (BZ#1810046)

Previously, the Operator resource field names displayed inconsistently in the web console as either camelCase or Start Case. Now, the operand form generation logic labels the form fields using Start Case by default. The default can be overridden by CSV or CRD objects. (BZ#1854196)

Previously, the Resource Log view did not show single line logs and did not show the last line of a pod log if it was missing a newline control character (/n). Now, the log view has been updated to show the entire pod log content for single line logs and final lines that are not terminated with a newline character. (BZ#1876853)

Previously, the OKD web console YAML editor allowed metadata.namespace entries for all resources. An error was returned when namespace: <namespace> was added for resources that did not take a namespace value. metadata.namespace entries are now removed when a configuration is saved, if a resource does not take a namespace value. (BZ#1846894)

Previously, the delete icon (-) for the name value editor did not provide a tooltip, so it was not clear to users what this icon did. The the delete icon (-) now provides a tooltip to make its action easier to understand. (BZ#1853706)

Previously, resource names with special characters, such as ( and ), could prevent resource details from being displayed in the OKD console. Resource details are now displayed properly when resource names have special characters. (BZ#1845624)

Previously, the configuration reload for Prometheus was sometimes triggered while the configuration was not fully generated, which triggered the PrometheusNotIngestingSamples and PrometheusNotConnectedToAlertmanagers alerts because there were no scrape or alerting targets. The configuration reload process now ensures that the configuration on disk is valid before reloading Prometheus, and the alerts are not fired. (BZ#1845561)

Previously, the AlertmanagerConfigInconsistent alert could fire during an upgrade because some of the Alertmanager pods were temporarily not running due to a rolling update of the stateful set. Even though the alert resolved itself, this could cause confusion for cluster administrators. The AlertmanagerConfigInconsistent no longer considers the number of running Alertmanager pods, so it no longer fires during upgrades when some of the Alertmanager pods are in a not-running, transient state. (BZ#1846397)

Previously, some alerts did not have the correct severity set or were incorrect, which caused upgrade issues. The severity level for many alerts were changed from critical to warning, the KubeStatefulSetUpdateNotRolledOut and KubeDaemonSetRolloutStuck alerts were adjusted, and the KubeAPILatencyHigh and KubeAPIErrorsHigh alerts were removed. These alerts are now correct and should not cause upgrade issues. (BZ#1824988)

Previously, KubeTooManyPods alert used the kubelet_running_pod_count, which includes completed pods, so was incorrect for the KubeTooManyPods alert. Now, container_memory_rss is leveraged to find the actual number of pods running on a node for the KubeTooManyPods alert. (BZ#1846805)

Previously, the node_exporter daemon set defaulted to a maxUnavailable value of 1, so the rollout was entirely serialized and slow on large clusters. Since the node_exporter daemon set does not affect workload availability, the maxUnavailable value now scales with the cluster size, which allows for a faster rollout. (BZ#1867603)

With this release, Kuryr-Kubernetes now attempts to detect the bridge interface for pod subports on the Container Network Interface (CNI) level, instead of using a value set in kuryr.conf. This approach supports cases when VMs call the interface without using a value set in kuryr.conf. (BZ#1829517)

Previously, OpenShift SDN exposed metrics unencrypted over HTTP. Now, OpenShift SDN exposes metrics over TLS. (BZ#1809205)

Previously, when idling a service, OpenShift SDN did not always delete a service and its endpoints in the correct order. As a result, sometimes the node port for a service was not deleted. When the service scaled up again, it was therefore unreachable. Now, OpenShift SDN ensures that the node port is always deleted correctly. (BZ#1857743)

Previously, an egress router pod failed to initialize because of a missing dependency on legacy iptables binaries. Now, an egress router pod successfully initializes. (BZ#1822945)

Previously, when deleting network policies on a cluster that is using the OVN-Kubernetes cluster networking provider, a race condition prevented network policies from being reliably deleted when deleting the associated namespace. Now, network policy objects are always correctly deleted. (BZ#1859682)

Previously, when using the OpenShift SDN pod network provider in multitenant isolation mode, pods were unable to reach services configured with externalIPs set. Now, pods can reach services with a service external IP address configured. (BZ#1762580)

Previously, OVN-Kubernetes exposed metrics unencrypted over HTTP. Now, OVN-Kubernetes exposes metrics over TLS. (BZ#1822720)

Previously, on OKD on Red Hat OpenStack Platform, when using the OVN-Octavia driver, it was not possible to attach listeners to different protocols on the same port. Now, it is possible to expose several protocols on the same port. (BZ#1846396)

Previously, a Kubelet failed to start if a soft eviction threshold and grace period were not specified. With this release, the presence of these values is verified during Kubelet configuration. (BZ#1805019)

Because the user could enter invalid characters, such as negative values, non-numeric characters, and so forth, in the CPU and memory requests for a kubeconfig object, the kubelet would not start. The code has been changed to verify that the kubelet config memory request values are valid. As a result, invalid values are rejected. (BZ#1745919)

Previously, if a system were using a device mapper for the root device, a number of key host level IO metrics returned by cadvisor were incorrectly set to zero. cadvisor was fixed to report these metrics if the device mapper is used for root. (BZ#1831908)

Previously, when the Authentication Operator received an HTML payload from an OpenID Connect Authentication (OIDC) server that ignores the Accept: application/json header, the Operator logged an error about the payload. Now, the Operator includes the URL of the page that it requested to aid in troubleshooting the OIDC server response. (BZ#1861789)

Previously, the oc project command required the privileges of a self-provisioner role to switch projects, which meant that some users could not switch projects if they did not have that role. The self-provisioner role requirement was removed, so anyone with access to a project can now switch projects using oc project. (BZ#1849983)

Previously, sorting events by lastTimestamp could cause an error when sorting an event that has an empty lastTimestamp. Sorting is now resistant to empty elements, and works properly when sorting by lastTimestamp. (BZ#1880283)

Previously, the oc create job command was missing logic for the --save-config flag, so the --save-config option did not work as expected. Logic was added for the --save-logic flag, and it now works properly. (BZ#1844998)

Operator Lifecycle Manager (OLM) exposes the subscription.spec.config.nodeSelector field in the subscription CRD, but previously did not apply nodeSelectors labels to the deployments defined in the ClusterServiceVersion object (CSV). This made users unable to set nodeSelectors on their CSV deployments. This bug fix updates OLM to now propagate nodeSelector labels defined in the subscription.spec.config.nodeSelector field to deployments in the CSV. As a result, the field now works as expected. (BZ#1860035)

Previously, Operator Lifecycle Manager (OLM) did not reuse existing valid CA certificates when installing a ClusterServiceVersion object (CSV) that entered the installing phase multiple times. OLM applied a new webhook hash to the deployment, causing a new replica set to be created. The running Operator then redeployed, possibly many times during an installation. This bug fix updates OLM to now check if the CA already exists and reuses it if valid. As a result, if OLM detects existing valid CAs, OLM now reuses the CAs. (BZ#1868712)

Operator Lifecycle Manager (OLM) appends OwnerReferences metadata to service resources installed for Operators that provide API services. Previously, whenever an Operator of this class was redeployed by OLM, for example during a certificate rotation, a duplicate OwnerReference was appended to the related service, causing the number of OwnerReferences to grow unbounded. With this bug fix, when adding OwnerReferences, OLM now updates an existing OwnerReference if found. As a result, the number of OwnerReferences appended to a service by OLM is bounded. (BZ#1842399)

Operator Lifecycle Manager (OLM) previously did not pull bundle images before attempting to unpack them, causing the opm alpha bundle validate command to fail with image not found or similar error messages. This bug fix updates OLM to now pull bundle images before attempting to unpack in the bundle validator. As a result, the opm alpha bundle validate command successfully pulls and unpacks images before performing validation. (BZ#1857502)

Previously, the web console would choose Operator icons to display in OperatorHub by returning the icon from the first channel declared in the package. This sometimes caused the displayed icon to be different than the latest icon published to the package. This has been fixed by choosing the icon from the default channel, which ensures the latest icon is displayed. (BZ#1843652)

Previously, whiteout files appeared in unpacked content when using podman or docker tooling options. With this release, whiteout files are no longer present after unpacking with podman and docker tooling options. (BZ#1841178)

Previously, intermittent availability issues with the API server could lead to intermittent issues with the OpenShift Controller Manager Operator retrieving deployments. Failure to retrieve a deployment sometimes caused the Operator to panic. With this release, checks have been added to handle and report this error condition, and to retry the operation. The Operator now properly handles intermittent issues retrieving deployments from the API server. (BZ#1852964)

Previously, machines with a large number of NICs on networks without DHCP took a long time to boot. This was caused by the initramfs using legacy network scripts that attempted to bring up DHCP on every interface on the machine, one interface at a time. Now, initramfs uses NetworkManager instead of the legacy scripts. NetworkManager does not attempt DHCP on any interface that does not have a physical connection. The NetworkManager also attempts DHCP on interfaces in parallel, instead of one-at-a-time. These changes reduce the waiting time for DHCP timeouts. (BZ#1836248)

Previously, it was not possible to modify kernel arguments during installation. Now, kernel arguments can be modified on an installed system using the coreos-installer command. For example, you can configure the installed system to use a different serial console argument using:

$ coreos-installer install ... ＼
 --delete-karg console=ttyS0,115200n8 ＼
 --append-karg console=ttyS1,115200n8

When MCO was being used to deploy a worker node, it failed to load the file because the user-configured iSCSI initiator name in the Ignition configuration was automatically replaced by a dynamically generated name. Now, the iSCSI initiator name is generated dynamically only when a name has not been specified in the Ignition configuration. (BZ#1868174)

Manual changes to Azure VMs during provisioning altered the incarnation number, which then caused the afterburn read state reporting to fail due to the incarnation numbers not matching. Afterburn now fetches a fresh incarnation number just before posting the ready state. (BZ#1853068)

Some interfaces, such as bond interfaces, were not visible in the console. The NetworkManager dispatch scripts replaced the previously used Udev rules. This fix enables network interfaces that have either permanent hardware addresses or that are backed by devices with permanent hardware addresses to display in the console. (BZ#1866048)

Previously, the HAProxy router 503 page was not compliant with the standards used by some web application firewalls. The 503 page has been updated to resolve this issue. (BZ#1852728)

When the ingress Operator reconciles an ingressController object configured for the NodePortService endpoint publishing strategy type, the Operator gets the ingress controller’s NodePort service from the API to determine whether the Operator needs to create or update the service. If the service does not exist, the Operator creates it, with an empty value for the spec.sessionAffinity field. If the service does exist, the Operator compares it with what the ingress Operator expects in order to determine whether an update is needed for that service. In this comparison, if the API has set the default value, None, for the service spec.sessionAffinity field, the Operator detects the update and tries to set the spec.sessionAffinity field back to the empty value.

Previously, if you updated a cluster with improper routes, HAProxy would initialize into a defunct state. However, the update would not trigger any alerts and the cluster would incorrectly report the ingress Controller as available. HAProxy initial sync logic was fixed so that upgrades with broken routes fail. As a result, upgrading a cluster with improper routes is not successful and the HAProxyReloadFail or HAProxyDown alerts are reported. (BZ#1861455)

Because of the risk of connection re-use/coalescing when using HTTP/2 ALPN, a warning message was added to the output of the ingress Controller in the CLI to use to enable HTTP/2 ALPN only on a route that uses custom (non-wildcard) certificate. As a result, routes that do not have their own custom certificate will not be HTTP/2 ALPN-enabled on either the frontend or the backend. (BZ#1827364)

Previously, when HAProxy was reloaded, HAProxy Prometheus counter metrics would decrease in value, which explicitly violates the definition of a counter metric. The router code was fixed to note the time of the last metrics scrape. This prevents scraping beyond the preserved counter values during a reload. As a result the counter metrics do not show a sudden increase followed by a decrease when the router is reloading. (BZ#1752814)

Previously, an alert rule for the Samples Operator misspelled the registry.redhat.io host name. The rule now uses the correct host name in the alert message. (BZ#1863014)

Previously, when upgrading OKD, the Samples Operator might block the upgrade if the API server is intermittently unavailable. Now, the Operator handles intermittent connectivity gracefully and upgrades are no longer blocked. (BZ#1854857)

Error message content for Local Storage Operator logging was too generic to help with debugging. Additional details are now provided when a LocalVolume object is created and the specified device is either not found or invalid. (BZ#1840127)

Storage Operator stopped reconciling when Upgradable=False on v1alpha1 CRDs. As a result, the Cluster Storage Operator could not perform z-stream upgrades when these CRDs were detected on the cluster. This fix changed the order of the reconciliation loop. Now, z-stream upgrades complete successfully and v1alpha1 CRDs are detected without errors. (BZ#1873299)

Manila volumes could not be unmounted after the NFS driver pod restarted because the restart process assigned a new IP address to the pod. Now, the pod uses the host network and host IP address to mount and unmount volumes, even after restarting the driver pod. (BZ#1867152)

This bug fix reduces log noise when changing fsGroup of a small volume. (BZ#1877001)

A condition in the vSphere cloud provider could cause failures in persistent volume provisioning in rare circumstances due to a heavy load. This bug fixes the condition and allows vSphere volumes to provision reliably. (BZ#1806034)

Upgrades from OCP 4.3.z to 4.4.0 fail when clusters have installed v1alpha1 VolumeSnapshot CRDs manually or by an independent CSI driver. OpenShift Container Platform 4.4 introduced v1beta1 VolumeSnapshot CRDs, which are not compatible with v1alpha1 VolumeSnapshot CRDs. Now, the Cluster Storage Operator checks for the presence of v1alpha1 VolumeSnapshot CRDs. If detected, a message displays stating that the v1alpha1 VolumeSnapshot CRDs must be removed for the upgrade to proceed. (BZ#1835869)

When VolumeSnapshotContent object deletion policies were modified, the VolumeSnapshot resource instances could not be deleted because the finalizers associated with those instances were not updated. This bug fix removes the finalizers when the VolumeSnapshotContent object deletion policies are modified, and allows the VolumeSnapshot resource instances to be deleted after the associated resource objects are removed. (BZ#1842964)

Previously, the default OpenShift RBAC rules did not allow regular users to access or create VolumeSnapshot and VolumeSnapshotClass resource instances. Now, the default OpenShift RBAC rules allow basic-users to read/write VolumeSnapshot resources and read VolumeSnapshotClass resources. Additionally, storage-admins can read/write VolumeSnapshotContent objects by default. (BZ#1842408)

Previously, there was no validation to prevent the Local Storage Operator from creating one device to multiple PVs. This release adds validation for this scenario so that trying to create a PV on a block device where one is already provisioned by the Local Storage Operator will fail. (BZ#1744385)

Previously, the Insights Operator could gather data for an unlimited number of certificate signing requests (CSRs) in a single report. This resulted in excessive data collection for clusters with many CSRs. The Insights Operator now gathers data for a maximum of 5000 CSRs in a single report. (BZ#1881044)

TP: Technology Preview

GA: General Availability

-: Not Available

When powering on a virtual machine on vSphere with user-provisioned infrastructure, the process of scaling up a node might not work as expected. A known issue in the hypervisor configuration causes machines to be created within the hypervisor but not powered on. If a node appears to be stuck in the Provisioning state after scaling up a machine set, you can investigate the status of the virtual machine in the vSphere instance itself. Use the VMware commands govc tasks and govc events to determine the status of the virtual machine. Check for a similar error message to the following:

[Invalid memory setting: memory reservation (sched.mem.min) should be equal to memsize(8192). ]

In OKD 4.6.8, a bug fix was introduced that changed how certificates were regenerated by the Cluster Logging Operator (CLO). This fix causes an issue where the CLO could possibly regenerate certificates while the OpenShift Elasticsearch Operator (EO) is trying to restart the cluster. This results in communication issues between the EO and the cluster, causing EO nodes to have mismatched certificates.

$ oc delete pod -l component=es

Currently, upgrading from OKD 4.5 to 4.6 with the OVN-Kubernetes cluster networking provider will not work. This will be resolved in a future 4.6.z release. (BZ#1880591)

Currently, when scaling to greater than 75 nodes in a cluster, the OVN-Kubernetes cluster networking provider database might become corrupt, leaving the cluster in an unusable state. (BZ#1887585)

Currently, scaling up Fedora worker nodes on a cluster with the OVN-Kubernetes cluster networking provider will not work. This will be resolved in a future Fedora 7.8.z and Fedora 7.9.z release. (BZ#1884323, BZ#1871935)

Currently, when scaling up a worker node that is running on Fedora 7.8, the OVN-Kubernetes cluster networking provider fails to initialize on the new node. (BZ#1884323)

Downgrading from OKD 4.6 to 4.5 will be fixed in a future 4.5.z release. (BZ#1882394, BZ#1886148, BZ#1886127)

Currently, upgrading from OKD 4.5 to 4.6 with Fedora worker nodes does not work. This will be resolved in a future 4.6.z release. First, upgrade Fedora, then upgrade the cluster, and then run the normal Fedora upgrade playbook again. (BZ#1887607)

OKD 4.5 to 4.6 upgrade fails when an external network is configured on a bond device; the ovs-configuration service fails and nodes becomes unreachable. This will be resolved in a future 4.6.z release. (BZ#1887545)

Currently, huge pages are not detected properly when requested in several Non-Uniform Memory Access (NUMA) nodes. This is caused by the cnf-tests suite reporting errors when clusters contain several NUMA nodes because the tests compare the number of huge pages on one NUMA with the total number of huge pages on the entire node. (BZ#1889633)

The Data Plane Development Kit (DPDK) test used to check packet forwarding and receiving always fails. (BZ#1889631)

The Stream Control Transmission Protocol (SCTP) validation phase fails when there is at least one machine configuration with no raw configuration. For example, this would include machine configurations containing only kernel arguments. (BZ#1889275)

The Precision Time Protocol (PTP) validation phase fails because the cnf-tests suite does not properly detect the number of nodes running PTP. (BZ#1889741)

The Network Interface Card (NIC) validation phase fails because it does not wait for the devices on the node to become available. The waiting time for a pod to start running on the node is too short, so a pod could still have the status Pending and be incorrectly tested. (BZ#1890088)

The ose-egress-dns-proxy image has a known defect that prevents the container from starting. This image is also broken in earlier releases, so this is not considered a regression in 4.6. (BZ#1888024)

In OKD 4.1, anonymous users could access discovery endpoints. Later releases revoked this access to reduce the possible attack surface for security exploits because some discovery endpoints are forwarded to aggregated API servers. However, unauthenticated access is preserved in upgraded clusters so that existing use cases are not broken.

## Snippet to remove unauthenticated group from all the cluster role bindings
$ for clusterrolebinding in cluster-status-binding discovery system:basic-user system:discovery system:openshift:discovery ;
do
### Find the index of unauthenticated group in list of subjects
index=$(oc get clusterrolebinding ${clusterrolebinding} -o json | jq 'select(.subjects!=null) | .subjects | map(.name=="system:unauthenticated") | index(true)');
### Remove the element at index from subjects array
oc patch clusterrolebinding ${clusterrolebinding} --type=json --patch "[{'op': 'remove','path': '/subjects/$index'}]";
done

Running the operator-sdk new or operator-sdk create api commands without the --helm-chart flag builds a Helm-based Operator that uses the default boilerplate Nginx chart. While this example chart works correctly on upstream Kubernetes, it fails to deploy successfully on OKD.

$ operator-sdk new <operator_name> --type=helm \
  --helm-chart=<repo>/<name>

When installing OKD on bare metal nodes with the Redfish Virtual Media feature, a failure occurs when the Baseboard Management Controller (BMC) attempts to load the virtual media image from the provisioning network. This happens if the BMC is not using the provisioning network, or its network does not have routing set up to the provisioning network. As a workaround, when using virtual media, the provisioning network must be turned off, or the BMCs must be routed to the provisioning network as a prerequisite. (BZ#1872787)

The OKD installation program does not support the Manual mode configuration by using the install-config.yaml file on GCP and Azure due to a known issue. Instead, you must manually insert a config map into the manifest directory during the manifest generation stage of the cluster installation process as documented in Manually creating IAM for Azure and Manually creating IAM for GCP. (BZ#1884691)

On a power environment, when a pod is created using the FC persistent volume claim and the targetWWN as a parameter, the FC volume attach fails with no fc disk found error message and the pod remains in ContainerCreating state. (BZ#1887026)

When a node providing an egress IP is shut down, the pods hosted on that node are not moved to another node providing an egress IP. This causes the outgoing traffic of the pods to always fail when a node providing an egress IP is shut down. (BZ#1877273)

Private, disconnected cluster installations are not supported for AWS GovCloud when installing in the us-gov-east-1 region due to a known issue. (BZ#1881262).

Firewall rules used by machines not prefixed with the infrastructure ID are preserved when destroying a cluster running on Google Cloud Platform (GCP) with installer-provisioned infrastructure. This causes the destroy process of the installation program to fail. As a workaround, you must manually delete the firewall rule of the machine in the GCP web console:

$ gcloud compute firewall-rules delete <firewall_rule_name>

The opm alpha bundle build command fails on Windows 10. (BZ#1883773)

In OKD 4.6, the resource metrics API server provides support for custom metrics. The resource metrics API server does not implement the OpenAPI specification, and the following messages are sent to the kube-apiserver logs:

controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.

Rules API back-ends are sometimes not detected if Store API stores are discovered before Rules API stores. When this occurs, a store reference is created without a Rules API client, and the Rules API endpoint from Thanos Querier does not return any rules. (BZ#1870287)

If an AWS account is configured to use AWS Organizations service control policies (SCPs) that use a global condition to deny all actions or require a specific permission, the AWS policy simulator API that validates permissions produces a false negative. When the permissions cannot be validated, OKD AWS installations fail, even if the provided credentials have the required permissions for installation.

apiVersion: v1
baseDomain: cluster1.example.com
credentialsMode: Mint (1)
compute:
- architecture: amd64
  hyperthreading: Enabled
...

Clusters that run on RHOSP and use Kuryr create unnecessary Neutron ports for each hostNetworking pod. You can delete these ports safely. Automatic port deletion is planned for a future release of OKD. (BZ#1888318)

Deployments on RHOSP configured with Kuryr might experience kuryr-cni pods going into a crash loop, which reports a NetlinkError: (17, 'File exists') error message. As a workaround, you must reboot the node. A fix is planned to resolve this issue in a future release of OKD. (BZ#1869606)

When deploying an egress router pod in DNS proxy mode, the pod fails to initialize. (BZ#1888024)

FCOS real time (RT) kernels are currently only supported on compute nodes, not control plane nodes. Compact clusters are not supported with RT kernels in OKD 4.6. (BZ#1887007)

To increase security, the NET_RAW and SYS_CHROOT capabilities are no longer available in the default list of CRI-O capabilities.

The OKD Machine API bare metal actuator currently deletes Machine objects when the underlying bare metal host is deleted. This behavior does not align with other cloud provider actuators, which move a Machine object to the failed phase rather than removing it altogether if the underlying cloud provider resource is deleted. (BZ#1868104)

When upgrading a cluster from version 4.5 to 4.6 that was installed with installer-provisioned infrastructure on vSphere, the upgrade fails if the control plane node IP addresses change during the upgrade. As a workaround, you must reserve the control plane node IP addresses before upgrading to version 4.6. Review your DHCP server’s documentation for the configuration of reservations. (BZ#1883521)

For oc commands that require TLS verification, if the certificates do not set a Subject Alternative Name, verification does not fall back to the Common Name field and the command fails with the following error:

x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0

When you install Agones using Helm package manager and try to examine the chart resources in your namespace using the Developer perspective, you see an error message instead of the resource details. (BZ#1866087)

When you select a deployment in the Topology view, click Actions → Edit <deployment_name>, and then modify it; the modified Deployment YAML file overwrites or removes the volume mounts in the Pod Template spec. (BZ#1867965)

No success or failure message is displayed in the Developer perspective when you use the Add → From Catalog option, filter by Template, select a template, and then instantiate a template. (BZ#1876535)

PipelineRuns with skipped tasks incorrectly show the tasks as Failed. (BZ#1880389)

The Application Details page, in the Application Stages view, provides inaccurate links for the projects in the application environment. (BZ#1889348)

Under the condition of heavy pod creation, creation fails with the message error reserving pod name …​: name is reserved. CRI-O’s context for the CNI executable ends and it kills the process. Pod creation succeeds eventually, but it takes a lot of time. Therefore, the kubelet thinks that CRI-O did not create the pod. The kubelet sends the request again and a name conflict occurs. This issue is currently under investigation. (BZ#1785399)

If the cluster networking provider is OVN-Kubernetes, when using a service external IP address that is not assigned to any node in the cluster, network traffic to the external IP address is not routable. As a workaround, ensure that a service external IP address is always assigned to a node in the cluster. (BZ#1890270)

The oc annotate command does not work for LDAP group names that contain an equal sign (=), because the command uses the equal sign as a delimiter between the annotation name and value. As a workaround, use oc patch or oc edit to add the annotation. (BZ#1917280)

The OVN-Kubernetes network provider does not support the externalTrafficPolicy feature for NodePort- and LoadBalancer-type services. The service.spec.externalTrafficPolicy field determines whether traffic for a service is routed to node-local or cluster-wide endpoints. Currently, such traffic is routed by default to cluster-wide endpoints, and there is no way to limit traffic to node-local endpoints. This will be resolved in a future release. (BZ#1903408)

Currently, a Kubernetes port collision issue can cause a breakdown in pod-to-pod communication, even after pods are redeployed. For detailed information and a workaround, see the Red Hat Knowledge Base solution Port collisions between pod and cluster IPs on OpenShift 4 with OVN-Kubernetes. (BZ#1939676, BZ#1939045)