Security - Service Mesh 2.x | Service Mesh | OpenShift Container Platform 4.6

About mutual Transport Layer Security (mTLS)
- Enabling strict mTLS across the service mesh
- Validating encryption with Kiali
Configuring Role Based Access Control (RBAC)
Configuring cipher suites and ECDH curves
Adding an external certificate authority key and certificate

If your service mesh application is constructed with a complex array of microservices, you can use Red Hat OpenShift Service Mesh to customize the security of the communication between those services. The infrastructure of OpenShift Container Platform along with the traffic management features of Service Mesh help you manage the complexity of your applications and secure microservices.

Before you begin

If you have a project, add your project to the ServiceMeshMemberRoll resource.

If you don’t have a project, install the Bookinfo sample application and add it to the ServiceMeshMemberRoll resource. The sample application helps illustrate security concepts.

About mutual Transport Layer Security (mTLS)

Mutual Transport Layer Security (mTLS) is a protocol that enables two parties to authenticate each other. It is the default mode of authentication in some protocols (IKE, SSH) and optional in others (TLS). You can use mTLS without changes to the application or service code. The TLS is handled entirely by the service mesh infrastructure and between the two sidecar proxies.

By default, mTLS in Red Hat OpenShift Service Mesh is enabled and set to permissive mode, where the sidecars in Service Mesh accept both plain-text traffic and connections that are encrypted using mTLS. If a service in your mesh is communicating with a service outside the mesh, strict mTLS could break communication between those services. Use permissive mode while you migrate your workloads to Service Mesh. Then, you can enable strict mTLS across your mesh, namespace, or application.

Enabling mTLS across your mesh at the Service Mesh control plane level secures all the traffic in your service mesh without rewriting your applications and workloads. You can secure namespaces in your mesh at the data plane level in the ServiceMeshControlPlane resource. To customize traffic encryption connections, configure namespaces at the application level with PeerAuthentication and DestinationRule resources.

Enabling strict mTLS across the service mesh

If your workloads do not communicate with outside services, you can quickly enable mTLS across your mesh without communication interruptions. You can enable it by setting spec.security.dataPlane.mtls to true in the ServiceMeshControlPlane resource. The Operator creates the required resources.

apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
  version: v2.2
  security:
    dataPlane:
      mtls: true

You can also enable mTLS by using the OpenShift Container Platform web console.

Procedure

Log in to the web console.
Click the Project menu and select the project where you installed the Service Mesh control plane, for example istio-system.
Click Operators → Installed Operators.
Click Service Mesh Control Plane under Provided APIs.
Click the name of your ServiceMeshControlPlane resource, for example, basic.
On the Details page, click the toggle in the Security section for Data Plane Security.

Configuring sidecars for incoming connections for specific services

You can also configure mTLS for individual services by creating a policy.

Procedure

Create a YAML file using the following example.

PeerAuthentication Policy example policy.yaml

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: <namespace>
spec:
  mtls:
    mode: STRICT

Replace <namespace> with the namespace where the service is located.

Run the following command to create the resource in the namespace where the service is located. It must match the namespace field in the Policy resource you just created.
```
$ oc create -n <namespace> -f <policy.yaml>
```

If you are not using automatic mTLS and you are setting PeerAuthentication to STRICT, you must create a DestinationRule resource for your service.

Configuring sidecars for outgoing connections

Create a destination rule to configure Service Mesh to use mTLS when sending requests to other services in the mesh.

Procedure

Create a YAML file using the following example.

DestinationRule example destination-rule.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: default
  namespace: <namespace>
spec:
  host: "*.<namespace>.svc.cluster.local"
  trafficPolicy:
   tls:
    mode: ISTIO_MUTUAL

Replace <namespace> with the namespace where the service is located.

Run the following command to create the resource in the namespace where the service is located. It must match the namespace field in the DestinationRule resource you just created.
```
$ oc create -n <namespace> -f <destination-rule.yaml>
```

Setting the minimum and maximum protocol versions

If your environment has specific requirements for encrypted traffic in your service mesh, you can control the cryptographic functions that are allowed by setting the spec.security.controlPlane.tls.minProtocolVersion or spec.security.controlPlane.tls.maxProtocolVersion in your ServiceMeshControlPlane resource. Those values, configured in your Service Mesh control plane resource, define the minimum and maximum TLS version used by mesh components when communicating securely over TLS.

The default is TLS_AUTO and does not specify a version of TLS.

Table 1. Valid values
Value	Description
`TLS_AUTO`	default
`TLSv1_0`	TLS version 1.0
`TLSv1_1`	TLS version 1.1
`TLSv1_2`	TLS version 1.2
`TLSv1_3`	TLS version 1.3

Procedure

Log in to the web console.
Click the Project menu and select the project where you installed the Service Mesh control plane, for example istio-system.
Click Operators → Installed Operators.
Click Service Mesh Control Plane under Provided APIs.
Click the name of your ServiceMeshControlPlane resource, for example, basic.
Click the YAML tab.
Insert the following code snippet in the YAML editor. Replace the value in the minProtocolVersion with the TLS version value. In this example, the minimum TLS version is set to TLSv1_2.
ServiceMeshControlPlane snippet
```
kind: ServiceMeshControlPlane
spec:
  security:
    controlPlane:
      tls:
        minProtocolVersion: TLSv1_2
```
Click Save.
Click Refresh to verify that the changes updated correctly.

Validating encryption with Kiali

The Kiali console offers several ways to validate whether or not your applications, services, and workloads have mTLS encryption enabled.

Figure 1. Masthead icon mesh-wide mTLS enabled

At the right side of the masthead, Kiali shows a lock icon when the mesh has strictly enabled mTLS for the whole service mesh. It means that all communications in the mesh use mTLS.

Figure 2. Masthead icon mesh-wide mTLS partially enabled

Kiali displays a hollow lock icon when either the mesh is configured in PERMISSIVE mode or there is a error in the mesh-wide mTLS configuration.

Figure 3. Security badge

The Graph page has the option to display a Security badge on the graph edges to indicate that mTLS is enabled. To enable security badges on the graph, from the Display menu, under Show Badges, select the Security checkbox. When an edge shows a lock icon, it means at least one request with mTLS enabled is present. In case there are both mTLS and non-mTLS requests, the side-panel will show the percentage of requests that use mTLS.

The Applications Detail Overview page displays a Security icon on the graph edges where at least one request with mTLS enabled is present.

The Workloads Detail Overview page displays a Security icon on the graph edges where at least one request with mTLS enabled is present.

The Services Detail Overview page displays a Security icon on the graph edges where at least one request with mTLS enabled is present. Also note that Kiali displays a lock icon in the Network section next to ports that are configured for mTLS.

Configuring Role Based Access Control (RBAC)

Role-based access control (RBAC) objects determine whether a user or service is allowed to perform a given action within a project. You can define mesh-, namespace-, and workload-wide access control for your workloads in the mesh.

To configure RBAC, create an AuthorizationPolicy resource in the namespace for which you are configuring access. If you are configuring mesh-wide access, use the project where you installed the Service Mesh control plane, for example istio-system.

For example, with RBAC, you can create policies that:

Configure intra-project communication.
Allow or deny full access to all workloads in the default namespace.
Allow or deny ingress gateway access.
Require a token for access.

An authorization policy includes a selector, an action, and a list of rules:

The selector field specifies the target of the policy.
The action field specifies whether to allow or deny the request.
The rules field specifies when to trigger the action.
- The from field specifies constraints on the request origin.
- The to field specifies constraints on request target and parameters.
- The when field specifies additional conditions that to apply the rule.

Procedure

Create your AuthorizationPolicy resource. The following example shows a resource that updates the ingress-policy AuthorizationPolicy to deny an IP address from accessing the ingress gateway.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ingress-policy
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
  action: DENY
  rules:
  - from:
    - source:
      ipBlocks: ["1.2.3.4"]

Run the following command after you write your resource to create your resource in your namespace. The namespace must match your metadata.namespace field in your AuthorizationPolicy resource.
```
$ oc create -n istio-system -f <filename>
```

Next steps

Consider the following examples for other common configurations.

Configure intra-project communication

You can use AuthorizationPolicy to configure your Service Mesh control plane to allow or deny the traffic communicating with your mesh or services in your mesh.

Restrict access to services outside a namespace

You can deny requests from any source that is not in the bookinfo namespace with the following AuthorizationPolicy resource example.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: httpbin-deny
 namespace: bookinfo
spec:
 selector:
   matchLabels:
     app: httpbin
     version: v1
 action: DENY
 rules:
 - from:
   - source:
       notNamespaces: ["bookinfo"]

Creating allow-all and default deny-all authorization policies

The following example shows an allow-all authorization policy that allows full access to all workloads in the bookinfo namespace.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: allow-all
  namespace: bookinfo
spec:
  action: ALLOW
  rules:
  - {}

The following example shows a policy that denies any access to all workloads in the bookinfo namespace.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: deny-all
  namespace: bookinfo
spec:
  {}

Allow or deny access to the ingress gateway

You can set an authorization policy to add allow or deny lists based on IP addresses.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ingress-policy
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
  action: ALLOW
  rules:
  - from:
    - source:
       ipBlocks: ["1.2.3.4", "5.6.7.0/24"]

Restrict access with JSON Web Token

You can restrict what can access your mesh with a JSON Web Token (JWT). After authentication, a user or service can access routes, services that are associated with that token.

Create a RequestAuthentication resource, which defines the authentication methods that are supported by a workload. The following example accepts a JWT issued by http://localhost:8080/auth/realms/master.

apiVersion: "security.istio.io/v1beta1"
kind: "RequestAuthentication"
metadata:
  name: "jwt-example"
  namespace: bookinfo
spec:
  selector:
    matchLabels:
      app: httpbin
  jwtRules:
  - issuer: "http://localhost:8080/auth/realms/master"
    jwksUri: "http://keycloak.default.svc:8080/auth/realms/master/protocol/openid-connect/certs"

Then, create an AuthorizationPolicy resource in the same namespace to work with RequestAuthentication resource you created. The following example requires a JWT to be present in the Authorization header when sending a request to httpbin workloads.

apiVersion: "security.istio.io/v1beta1"
kind: "AuthorizationPolicy"
metadata:
  name: "frontend-ingress"
  namespace: bookinfo
spec:
  selector:
    matchLabels:
      app: httpbin
  action: DENY
  rules:
  - from:
    - source:
        notRequestPrincipals: ["*"]

Configuring cipher suites and ECDH curves

Cipher suites and Elliptic-curve Diffie–Hellman (ECDH curves) can help you secure your service mesh. You can define a comma separated list of cipher suites using spec.security.controlplane.tls.cipherSuites and ECDH curves using spec.security.controlplane.tls.ecdhCurves in your ServiceMeshControlPlane resource. If either of these attributes are empty, then the default values are used.

The cipherSuites setting is effective if your service mesh uses TLS 1.2 or earlier. It has no effect when negotiating with TLS 1.3.

Set your cipher suites in the comma separated list in order of priority. For example, ecdhCurves: CurveP256, CurveP384 sets CurveP256 as a higher priority than CurveP384.

You must include either TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 or TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 when you configure the cipher suite. HTTP/2 support requires at least one of these cipher suites.

The supported cipher suites are:

TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_GCM_SHA256
TLS_RSA_WITH_AES_256_GCM_SHA384
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA

The supported ECDH Curves are:

CurveP256
CurveP384
CurveP521
X25519

Adding an external certificate authority key and certificate

By default, Red Hat OpenShift Service Mesh generates a self-signed root certificate and key and uses them to sign the workload certificates. You can also use the user-defined certificate and key to sign workload certificates with user-defined root certificate. This task demonstrates an example to plug certificates and key into Service Mesh.

Prerequisites

Install Red Hat OpenShift Service Mesh with mutual TLS enabled to configure certificates.
This example uses the certificates from the Maistra repository. For production, use your own certificates from your certificate authority.
Deploy the Bookinfo sample application to verify the results with these instructions.
OpenSSL is required to verify certificates.

Adding an existing certificate and key

To use an existing signing (CA) certificate and key, you must create a chain of trust file that includes the CA certificate, key, and root certificate. You must use the following exact file names for each of the corresponding certificates. The CA certificate is named ca-cert.pem, the key is ca-key.pem, and the root certificate, which signs ca-cert.pem, is named root-cert.pem. If your workload uses intermediate certificates, you must specify them in a cert-chain.pem file.

Save the example certificates from the Maistra repository locally and replace <path> with the path to your certificates.

Create a secret named cacert that includes the input files ca-cert.pem, ca-key.pem, root-cert.pem and cert-chain.pem.

$ oc create secret generic cacerts -n istio-system --from-file=<path>/ca-cert.pem \
    --from-file=<path>/ca-key.pem --from-file=<path>/root-cert.pem \
    --from-file=<path>/cert-chain.pem

In the ServiceMeshControlPlane resource set spec.security.dataPlane.mtls true to true and configure the certificateAuthority field as shown in the following example. The default rootCADir is /etc/cacerts. You do not need to set the privateKey if the key and certs are mounted in the default location. Service Mesh reads the certificates and key from the secret-mount files.
```
apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
  security:
    dataPlane:
      mtls: true
    certificateAuthority:
      type: Istiod
      istiod:
        type: PrivateKey
        privateKey:
          rootCADir: /etc/cacerts
```
After creating/changing/deleting the cacert secret, the Service Mesh control plane istiod and gateway pods must be restarted so the changes go into effect. Use the following command to restart the pods:
```
$ oc -n istio-system delete pods -l 'app in (istiod,istio-ingressgateway, istio-egressgateway)'
```
The Operator will automatically recreate the pods after they have been deleted.

Restart the bookinfo application pods so that the sidecar proxies pick up the secret changes. Use the following command to restart the pods:

$ oc -n bookinfo delete pods --all

You should see output similar to the following:

pod "details-v1-6cd699df8c-j54nh" deleted
pod "productpage-v1-5ddcb4b84f-mtmf2" deleted
pod "ratings-v1-bdbcc68bc-kmng4" deleted
pod "reviews-v1-754ddd7b6f-lqhsv" deleted
pod "reviews-v2-675679877f-q67r2" deleted
pod "reviews-v3-79d7549c7-c2gjs" deleted

Verify that the pods were created and are ready with the following command:
```
$ oc get pods -n bookinfo
```

Verifying your certificates

Use the Bookinfo sample application to verify that the workload certificates are signed by the certificates that were plugged into the CA. This requires you have openssl installed on your machine

To extract certificates from bookinfo workloads use the following command:

$ sleep 60
$ oc -n bookinfo exec "$(oc -n bookinfo get pod -l app=productpage -o jsonpath={.items..metadata.name})" -c istio-proxy -- openssl s_client -showcerts -connect details:9080 > bookinfo-proxy-cert.txt
$ sed -n '/-----BEGIN CERTIFICATE-----/{:start /-----END CERTIFICATE-----/!{N;b start};/.*/p}' bookinfo-proxy-cert.txt > certs.pem
$ awk 'BEGIN {counter=0;} /BEGIN CERT/{counter++} { print > "proxy-cert-" counter ".pem"}' < certs.pem

After running the command, you should have three files in your working directory: proxy-cert-1.pem, proxy-cert-2.pem and proxy-cert-3.pem.

Verify that the root certificate is the same as the one specified by the administrator. Replace <path> with the path to your certificates.
```
$ openssl x509 -in <path>/root-cert.pem -text -noout > /tmp/root-cert.crt.txt
```
Run the following syntax at the terminal window.
```
$ openssl x509 -in ./proxy-cert-3.pem -text -noout > /tmp/pod-root-cert.crt.txt
```
Compare the certificates by running the following syntax at the terminal window.
```
$ diff -s /tmp/root-cert.crt.txt /tmp/pod-root-cert.crt.txt
```
You should see the following result: Files /tmp/root-cert.crt.txt and /tmp/pod-root-cert.crt.txt are identical
Verify that the CA certificate is the same as the one specified by the administrator. Replace <path> with the path to your certificates.
```
$ openssl x509 -in <path>/ca-cert.pem -text -noout > /tmp/ca-cert.crt.txt
```
Run the following syntax at the terminal window.
```
$ openssl x509 -in ./proxy-cert-2.pem -text -noout > /tmp/pod-cert-chain-ca.crt.txt
```
Compare the certificates by running the following syntax at the terminal window.
```
$ diff -s /tmp/ca-cert.crt.txt /tmp/pod-cert-chain-ca.crt.txt
```
You should see the following result: Files /tmp/ca-cert.crt.txt and /tmp/pod-cert-chain-ca.crt.txt are identical.
Verify the certificate chain from the root certificate to the workload certificate. Replace <path> with the path to your certificates.
```
$ openssl verify -CAfile <(cat <path>/ca-cert.pem <path>/root-cert.pem) ./proxy-cert-1.pem
```
You should see the following result: ./proxy-cert-1.pem: OK

Removing the certificates

To remove the certificates you added, follow these steps.

Remove the secret cacerts. In this example, istio-system is the name of the Service Mesh control plane project.
```
$ oc delete secret cacerts -n istio-system
```

Redeploy Service Mesh with a self-signed root certificate in the ServiceMeshControlPlane resource.

apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
  security:
    dataPlane:
      mtls: true