Placing pods on specific nodes using node selectors - Working with pods | Nodes

Using node selectors to control pod placement

A node selector specifies a map of key-value pairs. The rules are defined using custom labels on nodes and selectors specified in pods.

For the pod to be eligible to run on a node, the pod must have the indicated key-value pairs as the label on the node.

If you are using node affinity and node selectors in the same pod configuration, see the important considerations below.

Using node selectors to control pod placement

You can use node selector labels on pods to control where the pod is scheduled.

With node selectors, OpenShift Container Platform schedules the pods on nodes that contain matching labels.

To add node selectors to an existing pod, add a node selector to the controlling object for that node, such as a ReplicaSet, Daemonset, or StatefulSet. Any existing pods under that controlling object are recreated on a node with a matching label. If you are creating a new pod, you can add the node selector directly to the pod spec.

You can add labels to a node or MachineConfig, but the labels will not persist if the node or machine goes down. Adding the label to the MachineSet ensures that new nodes or machines will have the label.

You cannot add a node selector directly to an existing scheduled pod.

Prerequisite

To add a node selector to existing pods, determine the controlling object for that pod. For example, the router-default-66d5cf9464-m2g75 pod is controlled by the router-default-66d5cf9464 ReplicaSet:

$ oc describe pod router-default-66d5cf9464-7pwkc

Name:               router-default-66d5cf9464-7pwkc
Namespace:          openshift-ingress

....

Controlled By:      ReplicaSet/router-default-66d5cf9464

The web console lists the controlling object under ownerReferences in the pod YAML:

  ownerReferences:
    - apiVersion: apps/v1
      kind: ReplicaSet
      name: router-default-66d5cf9464
      uid: d81dd094-da26-11e9-a48a-128e7edf0312
      controller: true
      blockOwnerDeletion: true

Procedure

Add labels to a node by using a MachineSet or editing the node directly:

Use a MachineSet to add labels to nodes managed by the MachineSet when a node is created:

Run the following command to add a node selector to a MachineSet:

$ oc patch MachineSet <name> --type='json' -p='[{"op":"add","path":"/spec/template/spec/metadata/labels", "value":{"<key>"="<value>","<key>"="<value>"}}]'  -n openshift-machine-api

For example:

$ oc patch MachineSet abc612-msrtw-worker-us-east-1c  --type='json' -p='[{"op":"add","path":"/spec/template/spec/metadata/labels", "value":{"type":"user-node","region":"east"}}]'  -n openshift-machine-api

Verify that the label is added to the MachineSet by using the oc edit command:

For example:

$ oc edit MachineSet abc612-msrtw-worker-us-east-1c -n openshift-machine-api

Example MachineSet object

apiVersion: machine.openshift.io/v1beta1
kind: MachineSet

....

spec:
...
  template:
    metadata:
...
    spec:
      metadata:
        labels:
          region: east
          type: user-node
....

Add labels directly to a node:

Edit the Node object for the node:

$ oc label nodes <name> <key>=<value>

For example, to label a node:

$ oc label nodes ip-10-0-142-25.ec2.internal type=user-node region=east

Verify that the label is added to the node:

$ oc get nodes -l type=user-node,region=east

Example output

NAME                          STATUS   ROLES    AGE   VERSION
ip-10-0-142-25.ec2.internal   Ready    worker   17m   v1.18.3+002a51f

Add the matching node selector a pod:

To add a node selector to existing and future pods, add a node selector to the controlling object for the pods:

Example ReplicaSet object

kind: ReplicaSet

....

spec:

....

  template:
    metadata:
      creationTimestamp: null
      labels:
        ingresscontroller.operator.openshift.io/deployment-ingresscontroller: default
        pod-template-hash: 66d5cf9464
    spec:
      nodeSelector:
        beta.kubernetes.io/os: linux
        node-role.kubernetes.io/worker: ''
        type: user-node (1)

1	Add the node selector.

To add a node selector to a specific pod, add the selector to the Pod object directly:

Example Pod object

apiVersion: v1
kind: Pod

...

spec:
  nodeSelector:
    <key>: <value>

...

For example:

Example Pod object with a node selector

apiVersion: v1
kind: Pod

....

spec:
  nodeSelector:
    region: east
    type: user-node