Routes - Networking | Architecture | OpenShift Dedicated 3

Overview
Route Host Names
Route Types
- Path Based Routes
- Secured Routes
Alternate Backends and Weights
Route-specific Annotations
Route-specific IP Whitelists

Overview

An OpenShift Dedicated route exposes a service at a host name, such as www.example.com, so that external clients can reach it by name.

DNS resolution for a host name is handled separately from routing. Your administrator may have configured a DNS wildcard entry that will resolve to the OpenShift Dedicated node that is running the OpenShift Dedicated router. If you are using a different host name you may need to modify its DNS records independently to resolve to the node that is running the router.

Each route consists of a name (limited to 63 characters), a service selector, and an optional security configuration.

Wildcard routes are disabled in OpenShift Dedicated.

Route Host Names

In order for services to be exposed externally, an OpenShift Dedicated route allows you to associate a service with an externally-reachable host name. This edge host name is then used to route traffic to the service.

When multiple routes from different namespaces claim the same host, the oldest route wins and claims it for the namespace. If additional routes with different path fields are defined in the same namespace, those paths are added. If multiple routes with the same path are used, the oldest takes priority.

A consequence of this behavior is that if you have two routes for a host name: an older one and a newer one. If someone else has a route for the same host name that they created between when you created the other two routes, then if you delete your older route, your claim to the host name will no longer be in effect. The other namespace now claims the host name and your claim is lost.

A Route with a Specified Host:

apiVersion: v1
kind: Route
metadata:
  name: host-route
spec:
  host: www.example.com  (1)
  to:
    kind: service
    name: service-name

1	Specifies the externally-reachable host name used to expose a service.

A Route Without a Host:

apiVersion: v1
kind: Route
metadata:
  name: no-route-hostname
spec:
  to:
    kind: service
    name: service-name

If a host name is not provided as part of the route definition, then OpenShift Dedicated automatically generates one for you. The generated host name is of the form:

<route-name>[-<namespace>].<suffix>

The following example shows the OpenShift Dedicated-generated host name for the above configuration of a route without a host added to a namespace mynamespace:

Generated Host Name

no-route-hostname-mynamespace.router.default.svc.cluster.local (1)

1	The generated host name suffix is the default routing subdomain router.default.svc.cluster.local.

A cluster administrator can also customize the suffix used as the default routing subdomain for their environment.

Route Types

Routes can be either secured or unsecured. Secure routes provide the ability to use several types of TLS termination to serve certificates to the client. Routers support edge, passthrough, and re-encryption termination.

Unsecured Route Object YAML Definition

apiVersion: v1
kind: Route
metadata:
  name: route-unsecured
spec:
  host: www.example.com
  to:
    kind: service
    name: service-name

Unsecured routes are simplest to configure, as they require no key or certificates, but secured routes offer security for connections to remain private.

A secured route is one that specifies the TLS termination of the route. The available types of termination are described below.

Path Based Routes

Path based routes specify a path component that can be compared against a URL (which requires that the traffic for the route be HTTP based) such that multiple routes can be served using the same host name, each with a different path. Routers should match routes based on the most specific path to the least; however, this depends on the router implementation. The host name and path are passed through to the backend server so it should be able to successfully answer requests for them. For example: a request to http://example.com/foo/ that goes to the router will result in a pod seeing a request to http://example.com/foo/.

The following table shows example routes and their accessibility:

Table 1. Route Availability
Route	When Compared to	Accessible
www.example.com/test	www.example.com/test	Yes
www.example.com/test	www.example.com	No
www.example.com/test and www.example.com	www.example.com/test	Yes
www.example.com/test and www.example.com	www.example.com	Yes
www.example.com	www.example.com/test	Yes (Matched by the host, not the route)
www.example.com	www.example.com	Yes

An Unsecured Route with a Path:

apiVersion: v1
kind: Route
metadata:
  name: route-unsecured
spec:
  host: www.example.com
  path: "/test"   (1)
  to:
    kind: service
    name: service-name

1	The path is the only added attribute for a path-based route.

Path-based routing is not available when using passthrough TLS, as the router does not terminate TLS in that case and cannot read the contents of the request.

Secured Routes

Secured routes specify the TLS termination of the route and, optionally, provide a key and certificate(s).

TLS termination in OpenShift Dedicated relies on SNI for serving custom certificates. Any non-SNI traffic received on port 443 is handled with TLS termination and a default certificate (which may not match the requested host name, resulting in validation errors).

Secured routes can use any of the following three types of secure TLS termination.

Edge Termination

With edge termination, TLS termination occurs at the router, prior to proxying traffic to its destination. TLS certificates are served by the front end of the router, so they must be configured into the route, otherwise the router’s default certificate will be used for TLS termination.

A Secured Route Using Edge Termination

apiVersion: v1
kind: Route
metadata:
  name: route-edge-secured (1)
spec:
  host: www.example.com
  to:
    kind: service
    name: service-name (1)
  tls:
    termination: edge            (2)
    key: |-                      (3)
      -----BEGIN PRIVATE KEY-----
      [...]
      -----END PRIVATE KEY-----
    certificate: |-              (4)
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    caCertificate: |-            (5)
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----

1	The name of the object, which is limited to 63 characters.
2	The `termination` field is `edge` for edge termination.
3	The `key` field is the contents of the PEM format key file.
4	The `certificate` field is the contents of the PEM format certificate file.
5	An optional CA certificate may be required to establish a certificate chain for validation.

Because TLS is terminated at the router, connections from the router to the endpoints over the internal network are not encrypted.

Edge-terminated routes can specify an insecureEdgeTerminationPolicy that enables traffic on insecure schemes (HTTP) to be disabled, allowed or redirected. The allowed values for insecureEdgeTerminationPolicy are: None or empty (for disabled), Allow or Redirect. The default insecureEdgeTerminationPolicy is to disable traffic on the insecure scheme. A common use case is to allow content to be served via a secure scheme but serve the assets (example images, stylesheets and javascript) via the insecure scheme.

A Secured Route Using Edge Termination Allowing HTTP Traffic

apiVersion: v1
kind: Route
metadata:
  name: route-edge-secured-allow-insecure (1)
spec:
  host: www.example.com
  to:
    kind: service
    name: service-name (1)
  tls:
    termination:                   edge   (2)
    insecureEdgeTerminationPolicy: Allow  (3)
    [ ... ]

1	The name of the object, which is limited to 63 characters.
2	The `termination` field is `edge` for edge termination.
3	The insecure policy to allow requests sent on an insecure scheme `HTTP`.

A Secured Route Using Edge Termination Redirecting HTTP Traffic to HTTPS

apiVersion: v1
kind: Route
metadata:
  name: route-edge-secured-redirect-insecure (1)
spec:
  host: www.example.com
  to:
    kind: service
    name: service-name (1)
  tls:
    termination:                   edge      (2)
    insecureEdgeTerminationPolicy: Redirect  (3)
    [ ... ]

1	The name of the object, which is limited to 63 characters.
2	The `termination` field is `edge` for edge termination.
3	The insecure policy to redirect requests sent on an insecure scheme `HTTP` to a secure scheme `HTTPS`.

Passthrough Termination

With passthrough termination, encrypted traffic is sent straight to the destination without the router providing TLS termination. Therefore no key or certificate is required.

A Secured Route Using Passthrough Termination

apiVersion: v1
kind: Route
metadata:
  name: route-passthrough-secured (1)
spec:
  host: www.example.com
  to:
    kind: service
    name: service-name (1)
  tls:
    termination: passthrough     (2)

1	The name of the object, which is limited to 63 characters.
2	The `termination` field is set to `passthrough`. No other encryption fields are needed.

The destination pod is responsible for serving certificates for the traffic at the endpoint. This is currently the only method that can support requiring client certificates (also known as two-way authentication).

Passthrough routes can also have an insecureEdgeTerminationPolicy. The only valid values are None (or empty, for disabled) or Redirect.

Re-encryption Termination

Re-encryption is a variation on edge termination where the router terminates TLS with a certificate, then re-encrypts its connection to the endpoint which may have a different certificate. Therefore the full path of the connection is encrypted, even over the internal network. The router uses health checks to determine the authenticity of the host.

A Secured Route Using Re-Encrypt Termination

apiVersion: v1
kind: Route
metadata:
  name: route-pt-secured (1)
spec:
  host: www.example.com
  to:
    kind: service
    name: service-name (1)
  tls:
    termination: reencrypt        (2)
    key: [as in edge termination]
    certificate: [as in edge termination]
    caCertificate: [as in edge termination]
    destinationCACertificate: |-  (3)
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----

1	The name of the object, which is limited to 63 characters.
2	The `termination` field is set to `reencrypt`. Other fields are as in edge termination.
3	Required for re-encryption. `destinationCACertificate` specifies a CA certificate to validate the endpoint certificate, securing the connection from the router to the destination pods. If the service is using a service signing certificate, or the administrator has specified a default CA certificate for the router and the service has a certificate signed by that CA, this field can be omitted.

If the destinationCACertificate field is left empty, the router automatically leverages the certificate authority that is generated for service serving certificates, and is injected into every pod as /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt. This allows new routes that leverage end-to-end encryption without having to generate a certificate for the route. This is useful for custom routers or the F5 router, which might not allow the destinationCACertificate unless the administrator has allowed it.

Re-encrypt routes can have an insecureEdgeTerminationPolicy with all of the same values as edge-terminated routes.

Alternate Backends and Weights

A route is usually associated with one service through the to: token with kind: service. All of the requests to the route are handled by endpoints in the service based on the load balancing strategy

It is possible to have as many as four services supporting the route. The portion of requests that are handled by each service is governed by the service weight.

The first service is entered using the to: token as before, and up to three additional services can be entered using the alternateBackend: token. Each service must be kind: service which is the default.

Each service has a weight associated with it. The portion of requests handled by the service is weight / sum_of_all_weights. When a service has more than one endpoint, the service’s weight is distributed among the endpoints with each endpoint getting at least 1. If the service weight is 0 each of the service’s endpoints will get 0.

The weight must be in the range 0-256. The default is 100. When the weight is 0, the service does not participate in load-balancing but continues to serve existing persistent connections.

When using alternateBackends also use the roundrobin load balancing strategy to ensure requests are distributed as expected to the services based on weight. roundrobin can be set for a route using a route annotation, or for the router in general using an environment variable.

The following is an example route configuration using alternate backends for A/B deployments.

A Route with alternateBackends and weights:

apiVersion: v1
kind: Route
metadata:
  name: route-alternate-service
  annotations:
    haproxy.router.openshift.io/balance: roundrobin  (1)
spec:
  host: www.example.com
  to:
    kind: service
    name: service-name  (2)
    weight: 20          (4)
  alternateBackends:
  - kind: service
    name: service-name2 (3)
    weight: 10          (4)
  - kind: service
    name: service-name3 (3)
    weight: 10          (4)

1	This route uses `roundrobin` load balancing strategy
2	The first service name is `service-name` which may have 0 or more pods
3	The alternateBackend services may also have 0 or more pods
4	The total `weight` is 40. `service-name` will get 20/40 or 1/2 of the requests, `service-name2` and `service-name3` will each get 1/4 of the requests, assuming each service has 1 or more endpoints.

Route-specific Annotations

Using environment variables, a router can set the default options for all the routes it exposes. An individual route can override some of these defaults by providing specific configurations in its annotations.

Route Annotations

For all the items outlined in this section, you can set annotations on the route definition for the route to alter its configuration

Table 2. Route Annotations
Variable	Description	Environment Variable Used as Default
`haproxy.router.openshift.io/balance`	Sets the load-balancing algorithm. Available options are `source`, `roundrobin`, and `leastconn`.	`ROUTER_TCP_BALANCE_SCHEME` for passthrough routes. Otherwise, use `ROUTER_LOAD_BALANCE_ALGORITHM`.
`haproxy.router.openshift.io/disable_cookies`	Disables the use of cookies to track related connections. If set to `true` or `TRUE`, the balance algorithm is used to choose which back-end serves connections for each incoming HTTP request.
`router.openshift.io/cookie_name`	Specifies an optional cookie to use for this route. The name must consist of any combination of upper and lower case letters, digits, "_", and "-". The default is the hashed internal key name for the route.
`haproxy.router.openshift.io/pod-concurrent-connections`	Sets the maximum number of connections that are allowed to a backing pod from a router. Note: if there are multiple pods, each can have this many connections. But if you have multiple routers, there is no coordination among them, each may connect this many times. If not set, or set to 0, there is no limit.
`haproxy.router.openshift.io/rate-limit-connections`	Setting `true` or `TRUE` to enables rate limiting functionality.
`haproxy.router.openshift.io/rate-limit-connections.concurrent-tcp`	Limits the number of concurrent TCP connections shared by an IP address.
`haproxy.router.openshift.io/rate-limit-connections.rate-http`	Limits the rate at which an IP address can make HTTP requests.
`haproxy.router.openshift.io/rate-limit-connections.rate-tcp`	Limits the rate at which an IP address can make TCP connections.
`haproxy.router.openshift.io/timeout`	Sets a server-side timeout for the route. (TimeUnits)	`ROUTER_DEFAULT_SERVER_TIMEOUT`
`router.openshift.io/haproxy.health.check.interval`	Sets the interval for the back-end health checks. (TimeUnits)	`ROUTER_BACKEND_CHECK_INTERVAL`
`haproxy.router.openshift.io/ip_whitelist`	Sets a whitelist for the route.
`haproxy.router.openshift.io/hsts_header`	Sets a Strict-Transport-Security header for the edge terminated or re-encrypt route.
`router.openshift.io/cookie-same-site`	Sets a value to restrict cookies. The values are: `Lax`: cookies are transferred between the visited site and third-party sites. `Strict`: cookies are restricted to the visited site. `None`: cookies are restricted to the visited site. This value is applicable to re-encrypt and edge routes only. For more information, see the SameSite cookies documentation.

A Route Setting Custom Timeout

apiVersion: v1
kind: Route
metadata:
  annotations:
    haproxy.router.openshift.io/timeout: 5500ms (1)
[...]

1	Specifies the new timeout with HAProxy supported units (us, ms, s, m, h, d). If unit not provided, ms is the default.

Setting a server-side timeout value for passthrough routes too low can cause WebSocket connections to timeout frequently on that route.

Route-specific IP Whitelists

You can restrict access to a route to a select set of IP addresses by adding the haproxy.router.openshift.io/ip_whitelist annotation on the route. The whitelist is a space-separated list of IP addresses and/or CIDRs for the approved source addresses. Requests from IP addresses that are not in the whitelist are dropped.

Some examples:

When editing a route, add the following annotation to define the desired source IP’s. Alternatively, use oc annotate route <name>.

Allow only one specific IP address:

metadata:
  annotations:
    haproxy.router.openshift.io/ip_whitelist: 192.168.1.10

Allow several IP addresses:

metadata:
  annotations:
    haproxy.router.openshift.io/ip_whitelist: 192.168.1.10 192.168.1.11 192.168.1.12

Allow an IP CIDR network:

metadata:
  annotations:
    haproxy.router.openshift.io/ip_whitelist: 192.168.1.0/24

Allow mixed IP addresses and IP CIDR networks:

metadata:
  annotations:
    haproxy.router.openshift.io/ip_whitelist: 180.5.61.153 192.168.1.0/24 10.0.0.0/8