Persistent Storage Using <strong>nfs</strong> | Administration

Overview
Provisioning
- Enforcing Disk Quotas
- Volume Security
Reclaiming Resources
Automation
SELinux and nfs Export Settings

Overview

You can provision your OpenShift cluster with persistent storage using nfs. Some familiarity with Kubernetes and nfs is assumed.

The Kubernetes persistent volume framework allows administrators to provision a cluster with persistent storage and gives users a way to request those resources without having any knowledge of the underlying infrastructure.

For a detailed example, see the guide for WordPress and MySQL using nfs.

High-availability of storage in the infrastructure is left to the underlying storage provider.

Provisioning

Storage must exist in the underlying infrastructure before it can be mounted as a volume in OpenShift. All that is required for nfs is a distinct list of servers and paths and the PersistentVolume API.

Example 1. Persistent Volume Object Definition

{
  "apiVersion": "v1",
  "kind": "PersistentVolume",
  "metadata": {
    "name": "pv0001"
  },
  "spec": {
    "capacity": {
        "storage": "5Gi"
    },
    "accessModes": [ "ReadWriteOnce" ],
    "nfs": {
        "path": "/tmp",
        "server": "172.17.0.2"
    },
    "persistentVolumeReclaimPolicy": "Recycle"
  }
}

Enforcing Disk Quotas

Use disk partitions to enforce disk quotas and size constraints. Each partition can be its own export. Each export is one persistent volume. Kubernetes enforces unique names for persistent volumes, but the uniqueness of the nfs volume’s server and path is up to the administrator.

Enforcing quotas in this way allows the end user to request persistent storage by a specific amount (e.g, 10Gi) and be matched with a corresponding volume of equal or greater capacity.

Volume Security

Users request storage with a PersistentVolumeClaim. This claim only lives in the user’s namespace and can only be referenced by a pod within that same namespace. Any attempt to access a persistent volume across a namespace causes the pod to fail.

Each nfs volume must be mountable by all nodes in the cluster.

Reclaiming Resources

nfs implements the Kubernetes Recyclable plug-in interface. Automatic processes handle reclamation tasks based on policies set on each persistent volume.

By default, persistent volumes are set to Retain. nfs volumes which are set to Recycle are scrubbed (i.e., rm -rf is run on the volume) after being released from their claim (i.e, after the user’s PersistentVolumeClaim bound to the volume is deleted). Once recycled, the nfs volume can be bound to a new claim.

Automation

As discussed, clusters can be provisioned with persistent storage using nfs in the following way:

Disk partitions can be used to enforce storage quotas.
Security can be enforced by restricting volumes to the namespace that has a claim to them.
Reclamation of discarded resources can be configured for each persistent volume.

They are many ways that you can use scripts to automate the above tasks. You can use an example Ansible playbook to help you get started.

SELinux and nfs Export Settings

By default, SELinux does not allow writing from a pod to a remote nfs server. The nfs volume mounts correctly, but is read-only.

To enable writing in SELinux on each node:

# setsebool -P virt_use_nfs 1

The -P option makes the bool persistent between reboots.

Additionally, in order to enable arbitrary container users to read and write the volume, each exported volume on the nfs server itself should conform to the following:

Each export must be:

/<example_fs> *(rw,all_squash)

Each export must be owned by nfsnobody:

chown -R nfsnobody:nfsnobody /<example_fs>

Each export must have the following permissions:

chmod 777 /<example_fs>

The export definition above allows arbitrary network clients to mount this volume. Exports can be restricted to a range of IP addresses for hosts that will access the volume. See man exports for more information.

Starting in OpenShift Enterprise 3.1, the export values have changed. See the OpenShift Enterprise 3.1 documentation for instructions on ensuring proper security for nfs in 3.1.

Persistent Storage Using nfs