Assigning Unique External IPs for <strong>ingress</strong> Traffic | Cluster Administration

Overview
Restrictions
Configuring the Cluster to Use Unique External IPs
- Configuring an ingress IP for a Service
Routing the ingress CIDR for Development or Testing
- Service externalIPs

Overview

One approach to getting external traffic into the cluster is by using ExternalIP or ingressIP addresses.

This feature is only supported in non-cloud deployments. For cloud (GCE, AWS, and OpenStack) deployments, use the Load Balancer services for automatic deployment of a cloud load balancer to target the service’s endpoints.

OKD supports two pools of IP addresses:

ingressIP uses by the Loadbalancer when choosing an external IP address for the service.
ExternalIP is used when the user selects a specific IP from the configured pool.

Both have to be configured to a device on an OKD host to be used, whether with network interface controller (NIC) or virtual ethernet, as well as external routing. Ipfailover is recommended for this, because it selects the host and configures the NIC.

ingressIP and ExternalIP both allow external traffic access to the cluster, and, if routed correctly, external traffic can reach that service’s endpoints via any TCP/UDP port the service exposes. This can be simpler than having to manage the port space of a limited number of shared IP addresses when manually assigning external IPs to services. Also, these addresses can be used as virtual IPs (VIPs) when configuring high availability.

OKD supports both the automatic and manual assignment of IP addresses, and each address is guaranteed to be assigned to a maximum of one service. This ensures that each service can expose its chosen ports regardless of the ports exposed by other services.

Restrictions

To use an ExternalIP, you can:

Select an IP address from the externalIPNetworkCIDRs range.

Have an IP address assigned from the ingressIPNetworkCIDR pool in the master configuration file. In this case, OKD implements a non-cloud version of the load balancer service type and assigns IP addresses to the services.

You must ensure that the IP address pool you assign terminates at one or more nodes in your cluster. You can use the existing oc adm ipfailover to ensure that the external IPs are highly available.

For manually-configured external IPs, potential port clashes are handled on a first-come, first-served basis. If you request a port, it is only available if it has not yet been assigned for that IP address. For example:

Port clash example for manually-configured external IPs

Two services have been manually configured with the same external IP address of 172.7.7.7.

MongoDB service A requests port 27017, and then MongoDB service B requests the same port; the first request gets the port.

However, port clashes are not an issue for external IPs assigned by the ingress controller, because the controller assigns each service a unique address.

Configuring the Cluster to Use Unique External IPs

In non-cloud clusters, ingressIPNetworkCIDR is set by default to 172.29.0.0/16. If your cluster environment is not already using this private range, you can use the default. However, if you want to use a different range, then you must set ingressIPNetworkCIDR in the /etc/origin/master/master-config.yaml file before you assign an ingress IP. Then, restart the master service.

External IPs assigned to services of type LoadBalancer will always be in the range of ingressIPNetworkCIDR. If ingressIPNetworkCIDR is changed such that the assigned external IPs are no longer in range, the affected services will be assigned new external IPs compatible with the new range.

If you are using high availibility, then this range must be less than 255 IP addresses.

Sample /etc/origin/master/master-config.yaml

networkConfig:
  ingressIPNetworkCIDR: 172.29.0.0/16

Configuring an ingress IP for a Service

To assign an ingress IP:

Create a YAML file for a LoadBalancer service that requests a specific IP via the loadBalancerIP setting:

Sample LoadBalancer Configuration

apiVersion: v1
kind: Service
metadata:
  name: egress-1
spec:
  ports:
  - name: db
    port: 3306
  loadBalancerIP: 172.29.0.1
  type: LoadBalancer
  selector:
    name: my-db-selector

Create a LoadBalancer service on your pod:
```
$ oc create -f loadbalancer.yaml
```
Check the service for an external IP. For example, for a service named myservice:
```
$ oc get svc myservice
```
When your LoadBalancer-type service has an external IP assigned, the output displays the IP:
```
NAME         CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
myservice    172.30.74.106   172.29.0.1    3306/TCP    30s
```

Routing the ingress CIDR for Development or Testing

Add a static route directing traffic for the ingress CIDR to a node in the cluster. For example:

# route add -net 172.29.0.0/16 gw 10.66.140.17 eth0

In the example above, 172.29.0.0/16 is the ingressIPNetworkCIDR, and 10.66.140.17 is the node IP.

Service externalIPs

In addition to the cluster’s internal IP addresses, the application developer can configure IP addresses that are external to the cluster. As the OKD administrator, you are responsible for ensuring that traffic arrives at a node with this IP.

The externalIPs must be selected by the administrator from the externalIPNetworkCIDRs range configured in the master-config.yaml file. When master-config.yaml changes, the master services must be restarted.

# systemctl restart origin-master-api origin-master-controllers

Sample externalIPNetworkCIDR /etc/origin/master/master-config.yaml

networkConfig:
  externalIPNetworkCIDR: 172.47.0.0/24

Service externalIPs Definition (JSON)

{
    "kind": "Service",
    "apiVersion": "v1",
    "metadata": {
        "name": "my-service"
    },
    "spec": {
        "selector": {
            "app": "MyApp"
        },
        "ports": [
            {
                "name": "http",
                "protocol": "TCP",
                "port": 80,
                "targetPort": 9376
            }
        ],
        "externalIPs" : [
            "80.11.12.10"         (1)
        ]
    }
}

1	List of External IP addresses on which the port is exposed. In addition to the internal IP addresses)

Assigning Unique External IPs for ingress Traffic