OpenShift Enterprise 3.2 Release Notes

Kubernetes has been updated to v1.2.0-36.

etcd has been updated to v2.2.5.

OpenShift Enterprise 3.2 requires Docker 1.9.1.

Admission controllers are now included, which intercept requests to the master API prior to persistence of a resource, but after the request is authenticated and authorized. The following admission control plug-ins can be configured:

Multiple web login providers can now be configured at the same time.

The oc adm diagnostics command can now launch a diagnostic pod that reports on more potential issues with pod networking, DNS configuration, and registry authentication.

Support for security context constraints (SCCs) has been added to the oc describe command.

The NO_PROXY environment variable will now accept a CIDR in a number of places in the code for controlling which IP ranges bypass the default HTTP proxy settings. See Configuring Hosts for Proxies for details.

Masters are now taken down one at a time during upgrades through rolling restarts. The openshift_rolling_restart_mode parameter can now be used in Ansible inventories to control this behavior: services for service restarts or system for full system restarts. See Configuring Cluster Variables for details.

The new Volumes field in SCCs allows an administrator full control over which volume plug-ins may be specified.

The new ReadOnlyRootFilesystem field in SCCs allows an administrator to force containers to run with a read-only root file system.

By default, the restricted and anyuid SCCs drop Linux capabilities that could be used to escalate container privileges. Administrators can change the list of default or enforced capabilities.

A constant-time string comparison is now used on webhooks.

Only users authenticated via OAuth can request projects.

A GitLab server can now be used as an identity provider. See Configuring Authentication for details.

The SETUID and SETGID capabilities have been added back to the anyuid SCC, which ensures that programs that start as root and then drop to a lower permission level will work by default.

Quota support has been added for emptydir. When the quota is enabled on an XFS system, nodes will limit the amount of space any given project can use on a node to a fixed upper bound. The quota is tied to the FSGroup of the project. Administrators can control this value by editing the project directly or allowing users to set FSGroup via SCCs.

The DaemonSet object is now limited to cluster administrators because pods running under a DaemonSet are considered to have higher priority than regular pods, and for regular users on the cluster this could be a security issue.

Administrators can prevent clients from accessing the API by their User-Agent header the new userAgentMatching configuration setting.

A readiness probe and health check have been added to the integrated registry to ensure new instances do not serve traffic until they are fully initialized.

You can limit the frequency of router reloads using the --interval=DURATION flag or RELOAD_INTERVAL environment variable to the router. This can minimize the memory and CPU used by the router while reloading, at the cost of delaying when the route is exposed via the router.

Routers now report back status to the master about whether routes are accepted, rejected, or conflict with other users. The CLI will now display that error information, allowing users to know that the route is not being served.

Using router sharding, you can specify a selection criteria for either namespaces (projects) or labels on routes. This enables you to select the routes a router would expose, and you can use this functionality to distribute routes across a set of routers, or shards.

The NoDiskConflicts scheduling predicate can be added to the scheduler configuration to ensure that pods using the same Ceph RBD device are not placed on the same node. See Scheduler for details.

The administrative commands are now exposed via oc adm so you have access to them in a client context. The oadm commands will still work, but will be a symlink to the openshift binary.

The help output of the oadm policy command has been improved.

Service accounts are now supported for the router and registry:

Administrators can pass the --all-namespaces flag to oc status to see status information across all namespaces and projects.

Users can now be presented with a customized, branded page before continuing on to a login identity provider. This allows users to see your branding up front instead of immediately redirecting to identity providers like GitHub and Google. See Customizing the Login Page for details.

CLI download URLs and documentation URLs are now customizable through web console extensions. See Adding or Changing Links to Download the CLI for details.

The web console uses a brand new theme that changes the look and feel of the navigation, tabs, and other page elements. See Project Overviews for details.

A new About page provides developers with information about the product version, oc CLI download locations, and a quick access to their current token to login using oc login. See CLI Downloads for details.

You can now add or edit resource constraints for your containers during Add to Project or later from the deployment configuration.

A form-based editor for build configurations has been added for modifying commonly edited fields directly from the web console.

All Browse resource pages (e.g, viewing a particular pod) now have a tab for Events related to that pod.

Limits, quotas, and quota scopes are now displayed.

More error and warning information is now displayed about routes, their configuration, and their use in the system.

Support has been added for filtering and sorting on all Events pages.

You can now edit a project’s display name and description from the Settings page.

Existing persistent volume claims (PVCs) can now be listed and attached to deployments and deployment configurations.

More detailed pod status is now provided on all pages.

Better status and alert messages are now provided.

Improved Dockerfile build keyword highlighting has been added when editing builds.

More accurate information is now displayed about routes based on which addresses the router exposed them under.

The layout and display of logs have been improved.

The following commands have been added to oc create, allowing more objects to be created directly using the CLI (instead of passing it a file or JSON/YAML):

Display more information about the application being created by the oc new-app command, including any display name or description set on the image as a label, or whether the image may require running as root.

If you have set up the latest tag in an image stream to point to another tag in the same image stream, the oc new-app command will follow that reference and create the application using the referenced tag, not latest. This allows administrators to ensure applications are created on stable tags (like php:5.6). The default image streams created in the openshift project follow this pattern.

You can view the logs of the oldest pod in a deployment or build configuration with:

$ oc logs dc/<name>

The oc env and oc volume commands have been moved to oc set env and oc set volume, and future commands that modify aspects of existing resources will be located under this command.

When a pod is crash-looping, meaning it is starting and exiting repeatedly, an error is now displayed in oc status output and provides more information about possible causes.

The new oc debug command makes it easy to obtain shell access in a misbehaving pod. It clones the exact environment of the running deployment configuration, replication controller, or pod, but replaces the run command with a shell.

The new oc set trigger command can be used to update deployment and build configuration triggers.

More information is displayed about liveness and readiness probes in the oc status and oc describe commands.

Builds can now be supplied with input files from unrelated images. Previously, all input to a build had to come from the builder image itself, or a Git repository. It is now possible to specify additional images and paths within those images to use as an input to a build for things like external dependencies.

apiVersion: v1
kind: BuildConfig
metadata:
  name: imagedockerbuild
spec:
  source:
    images:
    - from:
        kind: ImageStreamTag
        name: jenkins:latest
      paths:
      - destinationDir: installed-apps/
        sourcePath: /usr/lib/jenkins/jenkins.war

Builds can now be supplied with secrets for use during the build process. Previously, secrets could be used for Git cloning but now secrets can also be made available to the build process itself so that build operations such as Maven packaging can use a secret for credentials. See Using Secrets During a Build for details.

Builds now properly use Git submodules when checking out the source repository. When a build configuration is deleted (via oc delete), all associated builds are now deleted as well. To prevent this behavior, specify --cascade=false.

Custom build configurations can now specify the API version to use. This API version will determine the schema version used for the serialized build configuration supplied to the custom build pod in the BUILD environment variable.

Resource limits are now enforced on the container launched by S2I builds, and also on the operations performed within containers as part of a docker build of a Dockerfile. Previously, the resource limit only applied to the build pod itself and not the containers spawned by the build process.

You can now provide a command to be triggered after a build succeeds but before the push. You can set shell (to run a shell script), command, or args to run a command in the working directory of the built image. All S2I builders set the user’s source repository as the working directory, so commands like bundle exec rake test should work. See Build Hooks for details.

You can now import images from Docker v2 registries that are authenticated via Basic or Token credentials. To import, create a secret in your project based on a .docker/config.json or .dockercfg file:

$ oc secrets new hub .dockerconfigjson=$HOME/.docker/config.json
Created secret/hub

$ oc import-image auth-protected/image-from-dockerhub
The import completed successfully.

Name:       image-from-dockerhub
Created:    Less than a second ago

Tag     Spec                                 Created
latest  default/image-from-dockerhub:latest  Less than a second ago ...

$ oc annotate secret/hub openshift.io/image.excludeSecret=true

Image stream tags can be set to be automatically imported from remote repositories when they change (public or private). OpenShift Enterprise will periodically query the remote registry and check for updates depending on the configuration the administrator sets. By default, images will be checked every 15 minutes.

$ oc tag --source=docker redis:latest myredis:latest --scheduled
Tag myredis:latest set to import redis:latest periodically.

$ oc describe is myredis

The integrated Docker registry now supports image pullthrough, allowing you to tag a remote image into OpenShift Enterprise and directly pull it from the integrated registry as if it were already pushed to the OpenShift Enterprise registry. If the remote registry is configured to use content-offload (sending back a temporary redirect URL to the actual binary contents), that value will be passed through the OpenShift Enterprise registry and down to the Docker daemon, avoiding the need to proxy the binary contents.

$ oc tag --source=docker redis:latest redis:local
$ oc get is redis
NAME      DOCKER REPO                     TAGS     UPDATED
mysql     172.30.1.5:5000/default/redis   local    Less than a second ago

$ docker pull 127.30.1.5:5000/default/redis:local
Using default tag: local
Trying to pull repository 127.30.1.5:5000/default/redis ... latest: Pulling from 127.30.1.5:5000/default/redis
47d44cb6f252: Pull complete
838c1c5c4f83: Pull complete
5764f0a31317: Pull complete
60e65a8e4030: Pull complete
449f8db3c25a: Pull complete
a6b6487c42f6: Pull complete
Digest: sha256:c541c66a86b0715bfbb89c5515929268196b642551beccf8fbd452bb00170cde
Status: Downloaded newer image for 127.30.1.5:5000/default/redis:local

The oc describe command now reports overall image size for imported images as well as the individual layers and size of each layer.

When importing an entire remote repository, only the first five tags are imported by default. OpenShift Enterprise preferentially imports the latest tag and the highest semantically versioned tag (i.e., tags in the form v5, 5.0, or 5.0.1). You can import the remaining tags directly. Lists of tags will be sorted with the latest tag on top, followed by the highest major semantic tags, in descending order.

The Recreate deployment strategy now supports mid hooks, which run while all old pods have been scaled down and before any new pods are scaled up; use it to run migrations or configuration changes that can only happen while the application is completely shut down.

The Recreate deployment strategy now has the same behavior as the Rolling strategy, requiring the pod to be "Ready" before continuing with the deployment. A new field timeoutSeconds was added to the strategy that is the maximum allowed interval between pods becoming ready; it defaults to 120s.

The new Kubernetes 1.2 ConfigMap resource is now usable.

Pods being pulled or terminating are now distinguished in the pod status output, and the size of images is now shown with other pod information.

The Jenkins image can now be used as an S2I-compatible build image. See Using Jenkins as a Source-to-Image Builder for details.

Have the pod.spec.securityContext.fsGroup field populated to a namespace-wide allocated value automatically.

Have their emptyDir-derived (emptyDir, gitRepo, secret, configMap, and downwardAPI) and block device volumes (basically every network volume except ceph and nfs) owned by the FSGroup.

Run with the FSGroup in each container’s list of supplemental groups.

By default, new nodes installed with OpenShift Enterprise 3.2 will have Dnsmasq installed and configured as the default nameserver for both the host and pods.

By default, new masters installed with OpenShift Enterprise 3.2 will run SkyDNS on port 8053 rather than 53. Network access controls must allow nodes to connect to masters on port 8053. This is necessary so that Dnsmasq may be configured on all nodes.

Due to a change in the upstream JSON serialization path used in Kubernetes, some fields that were previously accepted case-insensitively are no longer accepted. Please validate that your API objects have the correct case for all attributes.

When creating a deployment configuration, omitting the spec.selector field will default that value to the pod template labels.

ImageStreamTag objects now return the spec tag tag, the current status conditions, and latest status generation generation, so clients can get an accurate view of the current tag.

ImageStreamTag objects can be updated via PUT to set their spec tag in a single call.

Deployment configuration hooks now default the container name if there is only a single container in the deployment configuration.

Registry pushes using AWS S3 storage considerably slower with Docker 1.10:

Images from Docker Hub fail due to v2 image schema:

A change in Ansible 2.2.1.0 caused problems with the OpenShift Enterprise 3.2 plabooks. This bug fix works around the change. ensuring that playbooks run correctly on Ansible 2.2.1.0 and newer. (BZ#1419533)

An Ansible 2.2.1.0 compatibility issue has been fixed in the quick installer. (BZ#1421059)

When executing the installer on a remote host that is also included in the inventory, the firewall configuration could potentially cause the installer to hang. A 10 second delay has been added after reseting the firewall to avoid this problem from occurring. (BZ#1379189)

Network Manager previously reset the net.ipv4.ip_forward parameter, causing OpenShift Enterprise to lose certain functionality. This bug fix ensures the installer sets the sysctl parameter on the system level, and as a result Network Manager restarts no longer interfere with the installation process. (BZ#1415067)

A certificate expiry checker has been added to the installer tools. (BZ#1417680)

This bug fix addresses an issue where the OpenShift Container Platform node logs a panic with a nil deference during volume teardown. (BZ#1425301)

The Heapster password was being set via a property value, so the password could be leaked by processes such as ps. This bug fix ensures the password is now being set via a system property. As a result, the password is no longer leaked by such processes. (BZ#1427544)

The passwords for Hawkular Metrics were being set via a property value, so the passwords could be leaked by processes such as ps. This bug fix ensures the passwords are now being set via a property file. As a result, the passwords are no longer leaked by such processes. (BZ#1417652)

Previously, installation would fail in multi-master environments in which the load balanced API was listening on a different port than that of the OpenShift Enterprise API and web console. This bug fix accounts for this difference and ensures the master loopback client configuration is configured to interact with the local master. (BZ#1462283)

During certificate expiration checking or redeployment, certificiates with large serial numbers could not be parsed using the existing manual parser workaround on hosts that were missing the OpenSSL python library. This bug fix updates the manual parser to account for the format of certificates with large serial numbers. As a result, these certificates can now be parsed. (BZ#1464546)

The OpenShift CA redeployment playbook (playbooks/byo/openshift-cluster/redeploy-openshift-ca.yml) would fail to restart services if certificates were previously expired. This bug fix ensures that service restarts are now skipped within the OpenShift CA redeployment playbook when expired certificates are detected. Expired cluster certificates may be replaced with the certificate redeployment playbook (playbooks/byo/openshift-cluster/redeploy-certificates.yml) after the OpenShift CA certificate has been replaced via the OpenShift CA redeployment playbook. (BZ#1460972)

If etcd 3.x or later was running on the host, a v3 snapshot database must be backed up as part of the backup process. If this directory is not included in the backup, then etcd failed to restore the backup even though v3 data was not used. This bug fix amends the etcd backup steps to ensure that the v3 snapshot database is included in backups. (BZ#1440303)

Previously, it was only possible to redeploy the etcd CA certificate by also redeploying the OpenShift CA certificate, which was unnecessary maintenance. With this enhancement, the etcd CA certificate may now be replaced independent of the OpenShift CA certificate using the etcd CA certificate redeployment playbook (playbooks/byo/openshift-cluster/redeploy-etcd-ca.yml). Note that the OpenShift CA redeployment playbook (playbooks/byo/openshift-cluster/redeploy-openshift-ca.yml) now only replaces the OpenShift CA certificate. Similarly, the etcd CA redeployment playbook only redeploys the etcd CA certificate. (BZ#1463775)