RHACM Governance Policy framework

As we mentioned in the ZTP Workflow section, the Red Hat Advanced Cluster Management (RHACM) for Kubernetes allows us to provision and configure multiple clusters in a declarative way and can be easily coupled with GitOps. ZTP and ongoing configuration updates makes intensive use of RHACM Policies for managing the configuration of deployed clusters. In our telco use case, creation of these Policies is simplified with the policyGen kustomize plugin. This plugin translates the vRAN DU profile based on PolicyGenTemplates into multiple Policies that are understood by the hub cluster and applied to the managed clusters.

In this section, we provide an introduction to the concepts behind the RHACM "Governance" pillar. Furthermore, we are going to introduce the concept of a policy, which is key for managing the configuration lifecycle of our fleet of clusters.

In the RHACM console you can find the governance section in the bottom left part of the menu bar. As you can see in the picture, there are multiple policies applied, some are disabled while most of them are enabled (see Status column) with remediation set to inform. We will get into the details later on.

Policy framework

Policies are comprised into the policy framework. The policy framework provides governance capability to gain visibility, and drive remediation for various security and configuration aspects to help meet such enterprise standards. Basically, it provides an extensible policy framework for enterprises to harden security for software engineering, secure engineering, and resiliency.

Below are the functions of the policy framework:

Distributes policies to managed clusters from hub cluster.
Collects policy execution results from managed cluster to hub cluster.
Supports multiple policy engines and policy languages.
Provides an extensible mechanism to bring your own policy.

In the next picture you can see the communication flow and the multiple components involved in creating, propagating and applying policies:

At a glance, once we create a policy in the hub cluster, the Governance Policy framework is responsible for propagating it to the set of target managed clusters based on a set of binding rules for the policy. Then, the policy rules are evaluated against the current state of the managed cluster. The results of this assessment are sent back to the hub cluster and depending on the needs, we can remediate the drift or just be informed.

Policies

Policy CRs, as defined by the Policy CRD, are created on the hub cluster and act as a vehicle to deliver policies to managed cluster and collect results to send to the hub cluster.

Each Policy can contain one or multiple policy rules that will be evaluated against cluster(s) that they are bound to. Policy binding is determined by the RHACM placement APIs which include the PlacementBinding and the PlacementRule CRs.

Note that the PlacementBinding connects the Policy to the PlacementRule, and the PlacementRule defines where (on what clusters) the Policy should be active. View the following example specification of a Policy object:

apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  annotations:
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
    policy.open-cluster-management.io/standards: NIST SP 800-53
  labels:
    app.kubernetes.io/instance: policies
  name: sno-worker-0-performance-policy
  namespace: ztp-sno-worker-0-policies
spec:
  remediationAction: inform         # [inform/enforce] If set, it defines the remediationAction globally.
  disabled: false                   # [true/false] If true, the policy will not be distributed to the managed cluster.
  policy-templates:
  - objectDefinition: # Use `objectDefinition` to wrap the policy resource to be distributed to the managed cluster
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: sno-worker-0-performance-policy-config
      spec:
        evaluationInterval:
          compliant: 10m
          noncompliant: 10s
        namespaceselector:
          exclude:
          - kube-*
          include:
          - '*'
        object-templates:
        - complianceType: musthave
          objectDefinition:
            apiVersion: performance.openshift.io/v2
            kind: PerformanceProfile
            metadata:
              name: openshift-node-performance-profile
            spec:
              additionalKernelArgs:
                - rcupdate.rcu_normal_after_boot=0
                - efi=runtime
              cpu:
                isolated: 2-39,42-79
                reserved: 0-1,40-41
              hugepages:
                defaultHugepagesSize: 1G
                pages:
                  - count: 32
                    size: 1G
              machineConfigPoolSelector:
                pools.operator.machineconfiguration.openshift.io/master: ""
              nodeSelector:
                node-role.kubernetes.io/master: ""
              numa:
                topologyPolicy: restricted
              realTimeKernel:
                enabled: true
        remediationAction: inform
        severity: low

The PlacementBinding CRD is used to bind the Policy with a PlacementRule. Only a bound Policy is distributed to a managed cluster by the policy framework. View the following example specification of a PlacementBinding object:

apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: sno-worker-0-placementbinding
  namespace: ztp-sno-worker-0-policies
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: sno-worker-0-placementrules
subjects:
- apiGroup: policy.open-cluster-management.io
  kind: Policy
  name: sno-worker-0-performance-policy

The PlacementRule CRD is used to determine the target clusters to distribute policies to. The PlacementRule can match clusters based on labels applied to the ManagedCluster CR on the hub cluster. Each target (spoke) cluster under management by RHACM on the hub cluster will have a corresponding ManagedCluster CR. View the following example specification of a PlacementRule object:

apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: sno-worker-0-placementrules
  namespace: ztp-sno-worker-0-policies
spec:
  clusterSelector:
    matchExpressions:
    - key: name
      operator: In
      values:
      - sno-worker-0
  clusterConditions:
  - status: "True"
    type: ManagedClusterConditionAvailable

Policy Status

Policies can be enabled or disabled by setting the disabled parameter value to true or false. By default, they are enabled.

Policy Remediation

As we will detail in the following section "Managing at Scale", for the solution presented in this lab the`PolicyGenTemplates` are by default set to generate Policies in inform mode.

Policy controllers periodically (as set by evaluationInterval) check policies against the current state of CRs on the target clusters. This check sets the Policy to either the Compliant or NonCompliant state. When the policy is set to inform, a report of the violations is created. However, when the controller is set to enforce a report is created and violations are remediated as specified in the policy.

The difference in behavior between inform and enforce is used by the GitOps ZTP flow to provide users continuous visibility into the state of clusters in their fleet (inform mode). When a violation (drift) is observed Policies may be converted to enforce mode (more on this in later sections) to bring the cluster(s) into compliance. This enforcement is used to apply initial configuration such as installing and configuring an operator as well as "day-n" changes to the configuration.

Takeaways

With that, we have covered the basics of what a Policy in RHACM is, and how it works.

A more in-detail explanation of the RHACM policies is covered in this training recording GitOps and policy manifests - How policies work.

[1] https://github.com/stolostron/policy-collection