Machine Configs and Tuned

If you are familiar with MachineConfigs you may have seen that this API resource is not available in the hosted clusters. That’s like that by design, there is a way to use MachineConfigs in hosted clusters, but this action is limited to the Cluster Service Consumer persona. Same goes for Tuned configurations, we can use Tuned to tune node configs like Hugepages, real time kernel, etc. but only from the management cluster.

Creating a Machine Config

In this example we’re going to add a test file in /var/tmp/, in order to do that we need to create a ConfigMap with the MachineConfig embedded on it.

Create the MachineConfig on the ConfigMap.

cat <<EOF | oc --kubeconfig ~/hypershift-lab/mgmt-kubeconfig -n hosted apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: hosted-cluster-example-mc
  namespace: hosted
data:
  config: |
    apiVersion: machineconfiguration.openshift.io/v1
    kind: MachineConfig
    metadata:
      labels:
        machineconfiguration.openshift.io/role: worker
      name: worker-test-file
    spec:
      config:
        ignition:
          version: 3.2.0
        storage:
          files:
          - contents:
              source: data:text/plain;charset=utf-8;base64,dGVzdCBmaWxl
            mode: 420
            overwrite: true
            path: /var/tmp/testfile.txt
EOF

configmap/hosted-cluster-example-mc created

Once created, we need to tell the NodePool to apply this configuration.

oc --kubeconfig ~/hypershift-lab/mgmt-kubeconfig -n hosted patch nodepool nodepool-hosted-1 \
    -p '{"spec":{"config":[{"name":"hosted-cluster-example-mc"}]}}' \
    --type merge

nodepool.hypershift.openshift.io/nodepool-hosted-1 patched

After a few minutes, the nodes will start applying the config.

Most MachineConfigs will require nodes to reboot.

oc --insecure-skip-tls-verify=true --kubeconfig ~/hypershift-lab/hosted-kubeconfig \
    get nodes

NAME             STATUS                     ROLES    AGE    VERSION
hosted-worker1   Ready,SchedulingDisabled   worker   5h50m  v1.31.7
hosted-worker2   Ready                      worker   5h50m  v1.31.7

You can also check the NodePool’s UPDATINGCONFIG field that reports if the config is currently being updated.

oc --kubeconfig ~/hypershift-lab/mgmt-kubeconfig -n hosted get nodepool nodepool-hosted-1

NAME                CLUSTER   DESIRED NODES   CURRENT NODES   AUTOSCALING   AUTOREPAIR   VERSION   UPDATINGVERSION   UPDATINGCONFIG   MESSAGE
nodepool-hosted-1   hosted    2               2               False         False        4.18.8                      True

Once finished, the UPDATINGCONFIG column value will be empty.

oc --kubeconfig ~/hypershift-lab/mgmt-kubeconfig -n hosted get nodepool nodepool-hosted-1

NAME                CLUSTER   DESIRED NODES   CURRENT NODES   AUTOSCALING   AUTOREPAIR   VERSION   UPDATINGVERSION   UPDATINGCONFIG   MESSAGE
nodepool-hosted-1   hosted    2               2               False         False        4.18.8

The nodes will have the test file.

oc --insecure-skip-tls-verify=true --kubeconfig ~/hypershift-lab/hosted-kubeconfig \
    debug $(oc --insecure-skip-tls-verify=true --kubeconfig ~/hypershift-lab/hosted-kubeconfig \
    get node -l hypershift.openshift.io/nodePool=nodepool-hosted-1 --no-headers -o name | tail -1) \
    -- ls -l /host/var/tmp/testfile.txt

<OMITTED>
-rw-r--r--. 1 root root 9 Jun 15 14:49 /host/var/tmp/testfile.txt
<OMITTED>

Creating a Tuned Config

In this example we’re going to make the nodes in our NodePool configure a few hugepages.

Create the Tuned config on the ConfigMap.

cat <<EOF | oc --kubeconfig ~/hypershift-lab/mgmt-kubeconfig -n hosted apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: hosted-cluster-example-tuned
  namespace: hosted
data:
  tuning: |
    apiVersion: tuned.openshift.io/v1
    kind: Tuned
    metadata:
      name: hugepages
      namespace: openshift-cluster-node-tuning-operator
    spec:
      profile:
      - data: |
          [main]
          summary=Boot time configuration for hugepages
          include=openshift-node
          [bootloader]
          cmdline_openshift_node_hugepages=hugepagesz=2M hugepages=50
        name: openshift-node-hugepages
      recommend:
      - priority: 20
        profile: openshift-node-hugepages
EOF

configmap/hosted-cluster-example-tuned created

Once created, we need to tell the NodePool to apply this configuration.

oc --kubeconfig ~/hypershift-lab/mgmt-kubeconfig -n hosted patch nodepool nodepool-hosted-1 \
    -p '{"spec":{"tuningConfig":[{"name":"hosted-cluster-example-tuned"}]}}' \
    --type merge

nodepool.hypershift.openshift.io/nodepool-hosted-1 patched

After a few minutes, the nodes will start applying the config.

Most Tuned configs will require nodes to reboot.

oc --insecure-skip-tls-verify=true --kubeconfig ~/hypershift-lab/hosted-kubeconfig \
    get nodes

NAME             STATUS                     ROLES    AGE    VERSION
hosted-worker1   Ready,SchedulingDisabled   worker   5h59m   v1.31.7
hosted-worker2   Ready                      worker   5h59m   v1.31.7

You can also check the NodePool’s UPDATINGCONFIG field.

oc --kubeconfig ~/hypershift-lab/mgmt-kubeconfig -n hosted get nodepool nodepool-hosted-1

NAME                CLUSTER   DESIRED NODES   CURRENT NODES   AUTOSCALING   AUTOREPAIR   VERSION   UPDATINGVERSION   UPDATINGCONFIG   MESSAGE
nodepool-hosted-1   hosted    2               2               False         False        4.18.8                      True

Once finished, the UPDATINGCONFIG column value will be empty.

oc --kubeconfig ~/hypershift-lab/mgmt-kubeconfig -n hosted get nodepool nodepool-hosted-1

NAME                CLUSTER   DESIRED NODES   CURRENT NODES   AUTOSCALING   AUTOREPAIR   VERSION   UPDATINGVERSION   UPDATINGCONFIG   MESSAGE
nodepool-hosted-1   hosted    2               2               False         False        4.18.8

The nodes will have the HugePages configured.

oc --insecure-skip-tls-verify=true --kubeconfig ~/hypershift-lab/hosted-kubeconfig \
    debug $(oc --insecure-skip-tls-verify=true --kubeconfig ~/hypershift-lab/hosted-kubeconfig \
    get node -l hypershift.openshift.io/nodePool=nodepool-hosted-1 --no-headers -o name | tail -1) \
    -- grep HugePages_ /proc/meminfo

<OMITTED>
HugePages_Total:      50
HugePages_Free:       50
HugePages_Rsvd:        0
HugePages_Surp:        0
<OMITTED>