The For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
Kubernetes offers two features that users can use to enforce network security. One feature that allows users to enforce network policy is the NetworkPolicy
API that is designed mainly for application developers and namespace tenants to protect their namespaces by creating namespace-scoped policies. For more information, see About network policy.
The second feature is AdminNetworkPolicy
which is comprised of two API: the AdminNetworkPolicy
(ANP) API and the BaselineAdminNetworkPolicy
(BANP) API. ANP and BANP are designed for cluster and network administrators to protect their entire cluster by creating cluster-scoped policies. Cluster administrators can use ANPs to enforce non-overridable policies that take precedence over NetworkPolicy
objects. Administrators can use BANP to setup and enforce optional cluster-scoped network policy rules that are overridable by users using NetworkPolicy
objects if need be. When used together ANP and BANP can create multi-tenancy policy that administrators can use to secure their cluster.
OVN-Kubernetes CNI in OKD implements these network policies using Access Control List (ACLs) Tiers to evaluate and apply them. ACLs are evaluated in descending order from Tier 1 to Tier 3.
Tier 1 evaluates AdminNetworkPolicy
(ANP) objects. Tier 2 evaluates NetworkPolicy
objects. Tier 3 evaluates BaselineAdminNetworkPolicy
(BANP) objects.
If traffic matches an ANP rule, the rules in that ANP will be evaluated first. If the match is an ANP allow
or deny
rule, any existing NetworkPolicies
and BaselineAdminNetworkPolicy
(BANP) in the cluster will be intentionally skipped from evaluation. If the match is an ANP pass
rule, then evaluation moves from tier 1 of the ACLs to tier 2 where the NetworkPolicy
policy is evaluated.
An AdminNetworkPolicy
(ANP) is a cluster-scoped custom resource definition (CRD). As a OKD administrator, you can use ANP to secure your network by creating network policies before creating namespaces. Additionally, you can create network policies on a cluster-scoped level that is non-overridable by NetworkPolicy
objects.
The key difference between AdminNetworkPolicy
and NetworkPolicy
objects are that the former is for administrators and is cluster scoped while the latter is for tenant owners and is namespace scoped.
An ANP allows administrators to specify the following:
A priority
value that determines the order of its evaluation. The lower the value the higher the precedence.
A subject
that consists of a set of namespaces or namespace..
A list of ingress rules to be applied for all ingress traffic towards the subject
.
A list of egress rules to be applied for all egress traffic from the subject
.
The |
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: sample-anp-deny-pass-rules (1)
spec:
priority: 50 (2)
subject:
namespaces:
matchLabels:
kubernetes.io/metadata.name: example.name (3)
ingress: (4)
- name: "deny-all-ingress-tenant-1" (5)
action: "Deny"
from:
- pods:
namespaces: (6)
namespaceSelector:
matchLabels:
custom-anp: tenant-1
podSelector:
matchLabels:
custom-anp: tenant-1 (7)
egress:(8)
- name: "pass-all-egress-to-tenant-1"
action: "Pass"
to:
- pods:
namespaces:
namespaceSelector:
matchLabels:
custom-anp: tenant-1
podSelector:
matchLabels:
custom-anp: tenant-1
1 | Specify a name for your ANP. |
2 | The spec.priority field supports a maximum of 100 ANP in the values of 0-99 in a cluster. The lower the value the higher the precedence. Creating AdminNetworkPolicy with the same priority creates a nondeterministic outcome. |
3 | Specify the namespace to apply the ANP resource. |
4 | ANP have both ingress and egress rules. ANP rules for spec.ingress field accepts values of Pass , Deny , and Allow for the action field. |
5 | Specify a name for the ingress.name . |
6 | Specify the namespaces to select the pods from to apply the ANP resource. |
7 | Specify podSelector.matchLabels name of the pods to apply the ANP resource. |
8 | ANP have both ingress and egress rules. ANP rules for spec.egress field accepts values of Pass , Deny , and Allow for the action field. |
As an administrator, you can set Allow
, Deny
, or Pass
as the action
field for your AdminNetworkPolicy
rules. Because OVN-Kubernetes uses a tiered ACLs to evaluate network traffic rules, ANP allow you to set very strong policy rules that can only be changed by an administrator modifying them, deleting the rule, or overriding them by setting a higher priority rule.
The following ANP that is defined at priority 9 ensures all ingress traffic is allowed from the monitoring
namespace towards any tenant (all other namespaces) in the cluster.
Allow
ANPapiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: allow-monitoring
spec:
priority: 9
subject:
namespaces: {}
ingress:
- name: "allow-ingress-from-monitoring"
action: "Allow"
from:
- namespaces:
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
This is an example of a strong Allow
ANP because it is non-overridable by all the parties involved. No tenants can block themselves from being monitored using NetworkPolicy
objects and the monitoring tenant also has no say in what it can or cannot monitor.
The following ANP that is defined at priority 5 ensures all ingress traffic from the monitoring
namespace is blocked towards restricted tenants (namespaces that have labels security: restricted
).
Deny
ANPapiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: block-monitoring
spec:
priority: 5
subject:
namespaces:
matchLabels:
security: restricted
ingress:
- name: "deny-ingress-from-monitoring"
action: "Deny"
from:
- namespaces:
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
This is a strong Deny
ANP that is non-overridable by all the parties involved. The restricted tenant owners cannot authorize themselves to allow monitoring traffic, and the infrastructure’s monitoring service cannot scrape anything from these sensitive namespaces.
When combined with the strong Allow
example, the block-monitoring
ANP has a lower priority value giving it higher precedence, which ensures restricted tenants are never monitored.
TThe following ANP that is defined at priority 7 ensures all ingress traffic from the monitoring
namespace towards internal infrastructure tenants (namespaces that have labels security: internal
) are passed on to tier 2 of the ACLs and evaluated by the namespaces’ NetworkPolicy
objects.
Pass
ANPapiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
name: pass-monitoring
spec:
priority: 7
subject:
namespaces:
matchLabels:
security: internal
ingress:
- name: "pass-ingress-from-monitoring"
action: "Pass"
from:
- namespaces:
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
This example is a strong Pass
action ANP because it delegates the decision to NetworkPolicy
objects defined by tenant owners. This pass-monitoring
ANP allows all tenant owners grouped at security level internal
to choose if their metrics should be scraped by the infrastructures' monitoring service using namespace scoped NetworkPolicy
objects.
BaselineAdminNetworkPolicy
(BANP) is a cluster-scoped custom resource definition (CRD). As a OKD administrator, you can use BANP to setup and enforce optional baseline network policy rules that are overridable by users using NetworkPolicy
objects if need be. Rule actions for BANP are allow
or deny
.
The BaselineAdminNetworkPolicy
resource is a cluster singleton object that can be used as a guardrail policy incase a passed traffic policy does not match any NetworkPolicy
objects in the cluster. A BANP can also be used as a default security model that provides guardrails that intra-cluster traffic is blocked by default and a user will need to use NetworkPolicy
objects to allow known traffic. You must use default
as the name when creating a BANP resource.
A BANP allows administrators to specify:
A subject
that consists of a set of namespaces or namespace.
A list of ingress rules to be applied for all ingress traffic towards the subject
.
A list of egress rules to be applied for all egress traffic from the subject
.
|
apiVersion: policy.networking.k8s.io/v1alpha1
kind: BaselineAdminNetworkPolicy
metadata:
name: default (1)
spec:
subject:
namespaces:
matchLabels:
kubernetes.io/metadata.name: example.name (2)
ingress: (3)
- name: "deny-all-ingress-from-tenant-1" (4)
action: "Deny"
from:
- pods:
namespaces:
namespaceSelector:
matchLabels:
custom-banp: tenant-1 (5)
podSelector:
matchLabels:
custom-banp: tenant-1 (6)
egress:
- name: "allow-all-egress-to-tenant-1"
action: "Allow"
to:
- pods:
namespaces:
namespaceSelector:
matchLabels:
custom-banp: tenant-1
podSelector:
matchLabels:
custom-banp: tenant-1
1 | The policy name must be default because BANP is a singleton object. |
2 | Specify the namespace to apply the ANP to. |
3 | BANP have both ingress and egress rules. BANP rules for spec.ingress and spec.egress fields accepts values of Deny and Allow for the action field. |
4 | Specify a name for the ingress.name |
5 | Specify the namespaces to select the pods from to apply the BANP resource. |
6 | Specify podSelector.matchLabels name of the pods to apply the BANP resource. |
The following BANP singleton ensures that the administrator has set up a default deny policy for all ingress monitoring traffic coming into the tenants at internal
security level. When combined with the "AdminNetworkPolicy Pass example", this deny policy acts as a guardrail policy for all ingress traffic that is passed by the ANP pass-monitoring
policy.
Deny
ruleapiVersion: policy.networking.k8s.io/v1alpha1
kind: BaselineAdminNetworkPolicy
metadata:
name: default
spec:
subject:
namespaces:
matchLabels:
security: internal
ingress:
- name: "deny-ingress-from-monitoring"
action: "Deny"
from:
- namespaces:
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
You can use an AdminNetworkPolicy
resource with a Pass
value for the action
field in conjunction with the BaselineAdminNetworkPolicy
resource to create a multi-tenant policy. This multi-tenant policy allows one tenant to collect monitoring data on their application while simultaneously not collecting data from a second tenant.
As an administrator, if you apply both the "AdminNetworkPolicy Pass
action example" and the "BaselineAdminNetwork Policy Deny
example", tenants are then left with the ability to choose to create a NetworkPolicy
resource that will be evaluated before the BANP.
For example, Tenant 1 can set up the following NetworkPolicy
resource to monitor ingress traffic:
NetworkPolicy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-monitoring
namespace: tenant 1
spec:
podSelector:
policyTypes:
- ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: monitoring
# ...
In this scenario, Tenant 1’s policy would be evaluated after the "AdminNetworkPolicy Pass
action example" and before the "BaselineAdminNetwork Policy Deny
example", which denies all ingress monitoring traffic coming into tenants with security
level internal
. With Tenant 1’s NetworkPolicy
object in place, they will be able to collect data on their application. Tenant 2, however, who does not have any NetworkPolicy
objects in place, will not be able to collect data. As an administrator, you have not by default monitored internal tenants, but instead, you created a BANP that allows tenants to use NetworkPolicy
objects to override the default behavior of your BANP.