---
title: Managing a cluster with multi-architecture compute machines
---

# Managing a cluster with multi-architecture compute machines {#multi-architecture-compute-managing}

Managing a cluster that has nodes with multiple architectures requires you to consider node architecture as you monitor the cluster and manage your workloads. This requires you to take additional considerations into account when you configure cluster resource requirements and behaviors, or schedule workloads in a multi-architecture cluster.

## Scheduled workloads on clusters with multi-architecture compute machines {#multi-architecture-scheduling_multi-architecture-compute-managing}

When you deploy workloads on a cluster with compute nodes that use different architectures, you must align pod architecture with the architecture of the underlying node. Your workload might also require additional configuration to particular resources depending on the underlying node architecture.

You can use the Multiarch Tuning Operator to enable architecture-aware scheduling of workloads on clusters with multi-architecture compute machines. The Multiarch Tuning Operator implements additional scheduler predicates in the pod specifications based on the architectures that the pods can support at creation time.

**Additional resources**
{._additional-resources}

- [Managing workloads on multi-architecture clusters by using the Multiarch Tuning Operator](https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html/postinstallation_configuration/configuring-multi-architecture-compute-machines-on-an-openshift-cluster#multiarch-tuning-operator)

### Sample multi-architecture node workload deployments {#multi-architecture-scheduling-examples_multi-architecture-compute-managing}

Scheduling a workload to an appropriate node based on architecture works in the same way as scheduling based on any other node characteristic. Consider the following options when determining how to schedule your workloads.

Using `nodeAffinity` to schedule nodes with specific architectures
:   You can allow a workload to be scheduled on only a set of nodes with architectures supported by its images. You can set the `spec.affinity.nodeAffinity` field in your pod’s template specification.

    ```yaml {title="Example deployment with node affinity set"}
    apiVersion: apps/v1
    kind: Deployment
    metadata: # ...
    spec:
       # ...
      template:
         # ...
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                - matchExpressions:
                  - key: kubernetes.io/arch
                    operator: In
                    values:
                    - amd64
                    - arm64
    # ...
    ```

    - The `values` parameter specifies the supported architectures. Valid values include `amd64`, `arm64`, or both values.

Tainting each node for a specific architecture
:   You can taint a node to avoid the node scheduling workloads that are incompatible with its architecture. When your cluster uses a `MachineSet` object, you can add parameters to the `.spec.template.spec.taints` field to avoid workloads being scheduled on nodes with non-supported architectures.

    Before you add a taint to a node, you must scale down the `MachineSet` object or remove existing available machines. For more information, see *Modifying a compute machine set*.

    ```yaml {title="Example machine set with taint set"}
    apiVersion: machine.openshift.io/v1beta1
    kind: MachineSet
    metadata: # ...
    spec:
      # ...
      template:
        # ...
        spec:
          # ...
          taints:
          - effect: NoSchedule
            key: multiarch.openshift.io/arch
            value: arm64
    ```

    You can also set a taint on a specific node by running the following command:

    ```terminal
    $ oc adm taint nodes <node-name> multiarch.openshift.io/arch=arm64:NoSchedule
    ```

Creating a default toleration in a namespace
:   When a node or machine set has a taint, only workloads that tolerate that taint can be scheduled. You can annotate a namespace so all of the workloads get the same default toleration by running the following command:

    ```terminal {title="Example default toleration set on a namespace"}
    $ oc annotate namespace my-namespace \
      'scheduler.alpha.kubernetes.io/defaultTolerations'='[{"operator": "Exists", "effect": "NoSchedule", "key": "multiarch.openshift.io/arch"}]'
    ```

Tolerating architecture taints in workloads
:   When a node or machine set has a taint, only workloads that tolerate that taint can be scheduled. You can configure your workload with a `toleration` so that it is scheduled on nodes with specific architecture taints.

    ```yaml {title="Example deployment with toleration set"}
    apiVersion: apps/v1
    kind: Deployment
    metadata: # ...
    spec:
      # ...
      template:
        # ...
        spec:
          tolerations:
          - key: "multiarch.openshift.io/arch"
            value: "arm64"
            operator: "Equal"
            effect: "NoSchedule"
    ```

    This example deployment can be scheduled on nodes and machine sets that have the `multiarch.openshift.io/arch=arm64` taint specified.

Using node affinity with taints and tolerations
:   When a scheduler computes the set of nodes to schedule a pod, tolerations can broaden the set while node affinity restricts the set. If you set a taint on nodes that have a specific architecture, you must also add a toleration to workloads that you want to be scheduled there.

    ```yaml {title="Example deployment with node affinity and toleration set"}
    apiVersion: apps/v1
    kind: Deployment
    metadata: # ...
    spec:
      # ...
      template:
        # ...
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                - matchExpressions:
                  - key: kubernetes.io/arch
                    operator: In
                    values:
                    - amd64
                    - arm64
          tolerations:
          - key: "multiarch.openshift.io/arch"
            value: "arm64"
            operator: "Equal"
            effect: "NoSchedule"
    ```

**Additional resources**
{._additional-resources}

- [Managing workloads on multi-architecture clusters by using the Multiarch Tuning Operator](/openshift-docs-markdown/post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator#multiarch-tuning-operator)
- [Controlling pod placement using node taints](/openshift-docs-markdown/nodes/scheduling/nodes-scheduler-taints-tolerations#nodes-scheduler-taints-tolerations)
- [Controlling pod placement on nodes using node affinity](/openshift-docs-markdown/nodes/scheduling/nodes-scheduler-node-affinity#nodes-scheduler-node-affinity)
- [Controlling pod placement using the scheduler](/openshift-docs-markdown/nodes/scheduling/nodes-scheduler-about#nodes-scheduler-about)
- [Modifying a compute machine set](/openshift-docs-markdown/machine_management/modifying-machineset#machineset-modifying_modifying-machineset)

## Enabling 64k pages on the Red Hat Enterprise Linux CoreOS (RHCOS) kernel {#multi-architecture-enabling-64k-pages_multi-architecture-compute-managing}

You can enable the 64k memory page in the Red Hat Enterprise Linux CoreOS (RHCOS) kernel on the 64-bit ARM compute machines in your cluster. The 64k page size kernel specification can be used for large GPU or high memory workloads.

This configuration is possible by using the Machine Config Operator (MCO), which uses a machine config pool to update the kernel. To enable 64k page sizes on ARM64, create a dedicated `MachineConfigPool` and apply the 64k kernel configuration to it.

> [!IMPORTANT]
> Using 64k pages is exclusive to 64-bit ARM architecture compute nodes or clusters installed on 64-bit ARM machines. If you configure the 64k pages kernel on a machine config pool using 64-bit x86 machines, the machine config pool and the MCO degrades.

**Prerequisites**

- You installed the OpenShift CLI (`oc`).
- You created a cluster with compute nodes of different architecture on one of the supported platforms.

**Procedure**

1. Label the nodes where you want to run the 64k page size kernel:

   ```terminal
   $ oc label node <node_name> <label>
   ```

   ```terminal {title="Example command"}
   $ oc label node worker-arm64-01 node-role.kubernetes.io/worker-64k-pages=
   ```
2. Create a machine config pool that contains the worker role that uses the ARM64 architecture and the `worker-64k-pages` role:

   ```yaml
   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfigPool
   metadata:
     name: worker-64k-pages
   spec:
     machineConfigSelector:
       matchExpressions:
         - key: machineconfiguration.openshift.io/role
           operator: In
           values:
           - worker
           - worker-64k-pages
     nodeSelector:
       matchLabels:
         node-role.kubernetes.io/worker-64k-pages: ""
         kubernetes.io/arch: arm64
   ```
3. Create a machine config on your compute node to enable `64k-pages` with the `64k-pages` parameter.

   ```terminal
   $ oc create -f <filename>.yaml
   ```

   ```yaml {title="Example MachineConfig"}
   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfig
   metadata:
     labels:
       machineconfiguration.openshift.io/role: "worker-64k-pages"
     name: 99-worker-64kpages
   spec:
     kernelType: 64k-pages
   ```

   where:

   `metadata.labels.machineconfiguration.openshift.io/role`
   :   Specifies the value of the `machineconfiguration.openshift.io/role` label in the custom machine config pool. The example MachineConfig uses the `worker-64k-pages` label to enable 64k pages in the `worker-64k-pages` pool.

   `spec.kernelType`
   :   Specifies your desired kernel type. Valid values are `64k-pages` and `default`.

   > [!NOTE]
   > The `64k-pages` type is supported on only 64-bit ARM architecture based compute nodes. The `realtime` type is supported on only 64-bit x86 architecture based compute nodes.

**Verification**

- To view your new `worker-64k-pages` machine config pool, run the following command:

  ```terminal
  $ oc get mcp
  ```

  ```terminal {title="Example output"}
  NAME     CONFIG                                                                UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
  master   rendered-master-9d55ac9a91127c36314e1efe7d77fbf8                      True      False      False      3              3                   3                     0                      361d
  worker   rendered-worker-e7b61751c4a5b7ff995d64b967c421ff                      True      False      False      7              7                   7                     0                      361d
  worker-64k-pages  rendered-worker-64k-pages-e7b61751c4a5b7ff995d64b967c421ff   True      False      False      2              2                   2                     0                      35m
  ```

## Importing manifest lists in image streams on your multi-architecture compute machines {#multi-architecture-import-imagestreams_multi-architecture-compute-managing}

On an OpenShift Container Platform 4.22 cluster with multi-architecture compute machines, the image streams in the cluster do not import manifest lists automatically. You must manually change the default `importMode` option to the `PreserveOriginal` option to import the manifest list.

**Prerequisites**

- You installed the OpenShift CLI (`oc`).

**Procedure**

- Enter a command similar to the following example command to patch the `ImageStream` cli-artifacts so that the `cli-artifacts:latest` image stream tag is imported as a manifest list:

  ```terminal
  $ oc patch is/cli-artifacts -n openshift -p '{"spec":{"tags":[{"name":"latest","importPolicy":{"importMode":"PreserveOriginal"}}]}}'
  ```

**Verification**

- You can check that the manifest lists imported properly by inspecting the image stream tag. The following command lists the individual architecture manifests for a particular tag.

  ```terminal
  $ oc get istag cli-artifacts:latest -n openshift -oyaml
  ```

  If the `dockerImageManifests` object is present, the manifest list imported successfully.

  ```yaml {title="Example output of the dockerImageManifests object"}
  dockerImageManifests:
    - architecture: amd64
      digest: sha256:16d4c96c52923a9968fbfa69425ec703aff711f1db822e4e9788bf5d2bee5d77
      manifestSize: 1252
      mediaType: application/vnd.docker.distribution.manifest.v2+json
      os: linux
    - architecture: arm64
      digest: sha256:6ec8ad0d897bcdf727531f7d0b716931728999492709d19d8b09f0d90d57f626
      manifestSize: 1252
      mediaType: application/vnd.docker.distribution.manifest.v2+json
      os: linux
    - architecture: ppc64le
      digest: sha256:65949e3a80349cdc42acd8c5b34cde6ebc3241eae8daaeea458498fedb359a6a
      manifestSize: 1252
      mediaType: application/vnd.docker.distribution.manifest.v2+json
      os: linux
    - architecture: s390x
      digest: sha256:75f4fa21224b5d5d511bea8f92dfa8e1c00231e5c81ab95e83c3013d245d1719
      manifestSize: 1252
      mediaType: application/vnd.docker.distribution.manifest.v2+json
      os: linux
  ```