Resources - Azure | Architecture | Azure Red Hat OpenShift 3

Important

Azure Red Hat OpenShift 3.11 will be retired 30 June 2022. Support for creation of new Azure Red Hat OpenShift 3.11 clusters continues through 30 November 2020. Following retirement, remaining Azure Red Hat OpenShift 3.11 clusters will be shut down to prevent security vulnerabilities.

Follow this guide to create an Azure Red Hat OpenShift 4 cluster. If you have specific questions, please contact us

Virtual machine scale sets
Disks
Public IP addresses
Load Balancers
Network security groups
dns configuration
Key vault
Storage accounts
Pods

There are several Azure resources that are used to build clusters.

Virtual machine scale sets

Resource Type Description

Resource Type	Description
`ss-masters`	A scale set of three virtual machines that help run core Services. Some of the core Services that run on this set: An etcd daemon The Azure Red Hat OpenShift API server The Azure Red Hat OpenShift controller The Azure Red Hat OpenShift (ARO) sync Pod also runs on one of the master virtual machines.
`ss-infra-TIMESTAMP`	A scale set of three virtual machines that help run infrastructure Pods. Some of the Services that run on this scale set: The Docker registry Routers for applications that run on Azure Red Hat OpenShift The Azure Red Hat OpenShift web console The template service broker Prometheus cluster monitoring, which is used by ARO site reliability engineers to monitor cluster health
`ss-compute-TIMESTAMP`	A scale set of virtual machines that customers' Pods leverage. All customer application and build Pods are scheduled on nodes within this scale set. The customer can adjust the number of virtual machines.

ss-masters

A scale set of three virtual machines that help run core Services.

Some of the core Services that run on this set:

An etcd daemon
The Azure Red Hat OpenShift API server
The Azure Red Hat OpenShift controller

The Azure Red Hat OpenShift (ARO) sync Pod also runs on one of the master virtual machines.

ss-infra-TIMESTAMP

A scale set of three virtual machines that help run infrastructure Pods.

Some of the Services that run on this scale set:

The Docker registry
Routers for applications that run on Azure Red Hat OpenShift
The Azure Red Hat OpenShift web console
The template service broker
Prometheus cluster monitoring, which is used by ARO site reliability engineers to monitor cluster health

ss-compute-TIMESTAMP

A scale set of virtual machines that customers' Pods leverage. All customer application and build Pods are scheduled on nodes within this scale set. The customer can adjust the number of virtual machines.

Disks

All kubernetes-dynamic-pvc-* disks are attached to their respective virtual machines for Azure-disk-class persistent volumes (PVs).

Persistent volumes are not backed up or replicated. You cannot use them for long-term storage.

Public IP addresses

Resource Type Description

Resource Type	Description
`ip-apiserver`	OpenShift API and web console access. This address points to `lb-apiserver`. Name resolution is described in dns configuration.
`ip-outbound`	Resource for outbound traffic from customers' and infrastructure Pods. This is configured in the `kubernetes` load balancer.
`kubernetes-*`	Resource for inbound traffic to applications that run in the customers' namespace. This is configured in the `kubernetes` load balancer.

ip-apiserver

OpenShift API and web console access. This address points to lb-apiserver. Name resolution is described in dns configuration.

ip-outbound

Resource for outbound traffic from customers' and infrastructure Pods. This is configured in the kubernetes load balancer.

kubernetes-*

Resource for inbound traffic to applications that run in the customers' namespace. This is configured in the kubernetes load balancer.

Load Balancers

Resource Type Description

Resource Type	Description
`lb-apiserver`	See Public IP addresses.
`kubernetes`	For all other traffic that `lb-apiserver` does not handle: Connections inbound to customers' Services that run on the cluster Connections outbound from customers' Pods

lb-apiserver

See Public IP addresses.

kubernetes

For all other traffic that lb-apiserver does not handle:

Connections inbound to customers' Services that run on the cluster
Connections outbound from customers' Pods

Network security groups

Resource Type Description

Resource Type	Description
`nsg-master`	Applies to master nodes. This permits API access and SSH management access by ARO back-end management systems.
`nsg-worker`	Applies to infrastructure and compute nodes. This permits access to the Services that the `kubernetes` load balancer exposes.

nsg-master

Applies to master nodes. This permits API access and SSH management access by ARO back-end management systems.

nsg-worker

Applies to infrastructure and compute nodes. This permits access to the Services that the kubernetes load balancer exposes.

dns configuration

Two dns zones jointly compose a dns configuration.

Resource Type Description

Resource Type	Description
`openshift.ID.REGION.azmosa.io`	A dns name that provides API and web console access from the `ip-apiserver` IP address.
`*.apps.ID.REGION.azmosa.io`	An alias that, from CNAME, points to the `kubernetes-*` IP address for inbound requests to customers' Services.

openshift.ID.REGION.azmosa.io

A dns name that provides API and web console access from the ip-apiserver IP address.

*.apps.ID.REGION.azmosa.io

An alias that, from CNAME, points to the kubernetes-* IP address for inbound requests to customers' Services.

Key vault

The key vault kv-* stores the cluster’s keys and certificates.

Storage accounts

Resource Type Description

Resource Type	Description
`sacfg`	The account that begins with `sacfg` stores the master node startup config, scale set hashes, and etcd backups.
`sareg`	Docker registry storage for customers' applications.
`safil`	Data storage for Azure-file storage-class PVs.

sacfg

The account that begins with sacfg stores the master node startup config, scale set hashes, and etcd backups.

sareg

Docker registry storage for customers' applications.

safil

Data storage for Azure-file storage-class PVs.

Pods

A number of management Pods run in ARO clusters.

Resource Type	Description
Sync Pod	The sync Pod runs on a single master node. Its deployment settings ensure that one instance runs at all times. The sync Pod’s role guarantees that managed OpenShift resources are synchronized with the desired values.
Customer admin controller	This controller synchronizes the contents of the designated Azure Active Directory group to the cluster’s RBAC customer-admin group. It also ensures that required RBAC roles are granted in customer-created namespaces.
Monitoring related	A cluster monitoring Prometheus instance gathers monitoring data for Azure Red Hat OpenShift site reliability engineering teams. Customers cannot access this instance. Customers can configure clusters with Azure Monitor for containers to get extensive telemetry and container logs.

Resource Type

Description

Sync Pod

The sync Pod runs on a single master node. Its deployment settings ensure that one instance runs at all times. The sync Pod’s role guarantees that managed OpenShift resources are synchronized with the desired values.

Customer admin controller

This controller synchronizes the contents of the designated Azure Active Directory group to the cluster’s RBAC customer-admin group. It also ensures that required RBAC roles are granted in customer-created namespaces.

Monitoring related

A cluster monitoring Prometheus instance gathers monitoring data for Azure Red Hat OpenShift site reliability engineering teams. Customers cannot access this instance. Customers can configure clusters with Azure Monitor for containers to get extensive telemetry and container logs.