Configuring the SDN

The OpenShift SDN enables communication between pods across the OKD cluster, establishing a pod network. Three SDN plug-ins are currently available (ovs-subnet, ovs-multitenant, and ovs-networkpolicy), which provide different methods for configuring the pod network.

The upstream Kubernetes project does not come with a default network solution. Instead, Kubernetes has developed a Container Network Interface (CNI) to allow network providers for integration with their own SDN solutions.

There are several OpenShift SDN plug-ins available out of the box from Red Hat, as well as third-party plug-ins.

Red Hat has worked with a number of SDN providers to certify their SDN network solution on OKD via the Kubernetes CNI interface, including a support process for their SDN plug-in through their product’s entitlement process. Should you open a support case with OpenShift, Red Hat can facilitate an exchange process so that both companies are involved in meeting your needs.

The following SDN solutions are validated and supported on OKD directly by the third-party vendor:

VMware NSX-T (™) provides an SDN and security infrastructure to build cloud-native application environments. In addition to vSphere hypervisors (ESX), these environments include KVM and native public clouds.

The current integration requires a new install of both NSX-T and OKD. Currently, NSX-T version 2.4 is supported, and only supports the use of ESXi and KVM hypervisors at this time.

See the NSX-T Container Plug-in for OpenShift - Installation and Administration Guide for more information.

For initial cluster installations, the ovs-subnet plug-in is installed and configured by default, though it can be overridden during installation using the os_sdn_network_plugin_name parameter, which is configurable in the Ansible inventory file.

For example, to override the standard ovs-subnet plug-in and use the ovs-multitenant plug-in instead:

See Configuring Cluster Variables for descriptions of networking-related Ansible variables that can be set in your inventory file.

The cluster administrators can control pod network settings on master hosts by modifying parameters in the networkConfig section of the master configuration file (located at /etc/origin/master/master-config.yaml by default):

Alternatively, you can create a pod network with multiple CIDR ranges by adding separate ranges into the clusterNetworks field with the range and the hostSubnetLength.

Multiple ranges can be used at once, and the range can be expanded or contracted. Nodes can be moved from one range to another by evacuating a node, then deleting and re-creating the node. See the Managing Nodes section for more information. Node allocations occur in the order listed, then when the range is full, move to the next on the list.

You can add elements to the clusterNetworks value, or remove them if no node is using that CIDR range.

Ensure that you restart the API and master services for any changes to take effect:

After the master services have been restarted the configuration must be propagated to the nodes. On each node the atomic-openshift-node service and ovs pod must be restarted. To avoid downtime follow the steps defined in Managing Nodes and described in the following procedure for each node or group of nodes at a time:

As a cluster administrator, you can change the VXLAN port the system uses.

Because you cannot change the VXLAN port of a running clusternetwork object, you must delete any existing network configurations and create a new configuration by editing the vxlanPort variable in the master configuration file.

The cluster administrators can control pod network settings on nodes by modifying parameters in the networkConfig section of the appropriate node configuration map:

If you are running low on addresses in your service network, you can expand the range as long as you ensure that the current range is at the beginning of the new range.

For each node in the cluster, complete the following steps:

If you are already using one SDN plug-in and want to switch to another:

If a host that is external to OKD requires access to the cluster network, you have two options:

Both options are presented as part of a practical use-case in the documentation for configuring routing from an edge load-balancer to containers within OpenShift SDN.

As an alternate to the default SDN, OKD also provides Ansible playbooks for installing flannel-based networking. This is useful if running OKD within a cloud provider platform that also relies on SDN, such as Red Hat OpenStack Platform, and you want to avoid encapsulating packets twice through both platforms.

Flannel uses a single IP network space for all of the containers allocating a contiguous subset of the space to each instance. Consequently, nothing prevents a container from attempting to contact any IP address in the same network space. This hinders multi-tenancy because the network cannot be used to isolate containers in one application from another.

Depending on whether you prefer mutli-tenancy isolation or performance, you should determine the appropriate choice when deciding between OpenShift SDN (multi-tenancy) and flannel (performance) for internal networks.

To enable flannel within your OKD cluster:

To isolate container traffic from the rest of the OKD traffic, Red Hat recommends creating an isolated tenant network and attaching all the nodes to it. If you are using a different network interface (eth1), remember to configure the interface to start at boot time through the /etc/sysconfig/network-scripts/ifcfg-eth1 file: