--- title: Creating a cluster with multi-architecture compute machines on Azure --- # Creating a cluster with multi-architecture compute machines on Azure {#creating-multi-arch-compute-nodes-azure} To deploy a cluster on Microsoft Azure with multi-architecture compute machines, you must first create a single-architecture installer-provisioned cluster that uses the multi-architecture installer binary. You can also migrate your current cluster with single-architecture compute machines to a cluster with multi-architecture compute machines. After creating a multi-architecture cluster, you can add nodes with different architectures to the cluster. ## Creating a 64-bit ARM boot image using the Azure image gallery {#multi-architecture-creating-bootimage_creating-multi-arch-compute-nodes-azure} You can generate a 64-bit x86 boot image or a 64-bit ARM boot image by using the Azure image gallery. The procedure example describes how to manually generate a 64-bit ARM boot image. If you want to generate a 64-bit x86 boot image, replace `aarch64` with `x86_64`; Additionally, replace any instances of `rhcos-arm64` with `rhcos-x86_64`. **Prerequisites** - You installed the Azure CLI (`az`). - You created a single-architecture Azure installer-provisioned cluster with the multi-architecture installer binary. **Procedure** 1. Log in to your Azure account by running the following command: ```terminal $ az login ``` 2. Create a storage account and upload the `aarch64` virtual hard drive (VHD) to your storage account. The OpenShift Container Platform installation program creates a resource group, however, the boot image can also be uploaded to a custom named resource group: ```terminal $ az storage account create -n ${STORAGE_ACCOUNT_NAME} -g ${RESOURCE_GROUP} -l westus --sku Standard_LRS ``` - The `westus` object is an example region. 3. Create a storage container using the storage account you generated by entering the following command: ```terminal $ az storage container create -n ${CONTAINER_NAME} --account-name ${STORAGE_ACCOUNT_NAME} ``` 4. You must use the OpenShift Container Platform installation program JSON file to extract the URL and `aarch64` VHD name: 1. Extract the `URL` field and set it to `RHCOS_VHD_ORIGIN_URL` as the file name by running the following command: ```terminal $ RHCOS_VHD_ORIGIN_URL=$(oc -n openshift-machine-config-operator get configmap/coreos-bootimages -o jsonpath='{.data.stream}' | jq -r '.architectures.aarch64."rhel-coreos-extensions"."azure-disk".url') ``` - For a 64-bit x86 boot image, replace `architectures.aarch64` with `architectures.x86_64`. 2. Extract the `aarch64` VHD name and set it to `BLOB_NAME` as the file name by running the following command: ```terminal $ BLOB_NAME=rhcos-$(oc -n openshift-machine-config-operator get configmap/coreos-bootimages -o jsonpath='{.data.stream}' | jq -r '.architectures.aarch64."rhel-coreos-extensions"."azure-disk".release')-azure.aarch64.vhd ``` - For a 64-bit x86 boot image, replace `architectures.aarch64` with `architectures.x86_64` and replace `aarch64.vhd` with `x86_64.vhd`. 5. Generate a shared access signature (SAS) token. Use this token to upload the RHCOS VHD to your storage container with the following commands: ```terminal $ end=`date -u -d "30 minutes" '+%Y-%m-%dT%H:%MZ'` ``` ```terminal $ sas=`az storage container generate-sas -n ${CONTAINER_NAME} --account-name ${STORAGE_ACCOUNT_NAME} --https-only --permissions dlrw --expiry $end -o tsv` ``` 6. Copy the RHCOS VHD into the storage container: ```terminal $ az storage blob copy start --account-name ${STORAGE_ACCOUNT_NAME} --sas-token "$sas" \ --source-uri "${RHCOS_VHD_ORIGIN_URL}" \ --destination-blob "${BLOB_NAME}" --destination-container ${CONTAINER_NAME} ``` You can check the status of the copying process with the following command: ```terminal $ az storage blob show -c ${CONTAINER_NAME} -n ${BLOB_NAME} --account-name ${STORAGE_ACCOUNT_NAME} | jq .properties.copy ``` ```terminal {title="Example output"} { "completionTime": null, "destinationSnapshot": null, "id": "1fd97630-03ca-489a-8c4e-cfe839c9627d", "incrementalCopy": null, "progress": "17179869696/17179869696", "source": "https://rhcos.blob.core.windows.net/imagebucket/rhcos-411.86.202207130959-0-azure.aarch64.vhd", "status": "success", "statusDescription": null } ``` If the status parameter displays the `success` object, the copying process is complete. 7. Create an image gallery using the following command: ```terminal $ az sig create --resource-group ${RESOURCE_GROUP} --gallery-name ${GALLERY_NAME} ``` Use the image gallery to create an image definition. In the following example command, `rhcos-arm64` is the name of the image definition. ```terminal $ az sig image-definition create --resource-group ${RESOURCE_GROUP} --gallery-name ${GALLERY_NAME} --gallery-image-definition rhcos-arm64 --publisher RedHat --offer arm --sku arm64 --os-type linux --architecture Arm64 --hyper-v-generation V2 ``` - For a 64-bit x86 boot image, replace `--offer arm --sku arm64` with `--offer x86_64 --sku x86_64 `. 8. To get the URL of the VHD and set it to `RHCOS_VHD_URL` as the file name, run the following command: ```terminal $ RHCOS_VHD_URL=$(az storage blob url --account-name ${STORAGE_ACCOUNT_NAME} -c ${CONTAINER_NAME} -n "${BLOB_NAME}" -o tsv) ``` 9. Use the `RHCOS_VHD_URL` file, your storage account, resource group, and image gallery to create an image version. In the following example, `1.0.0` is the image version. ```terminal $ az sig image-version create --resource-group ${RESOURCE_GROUP} --gallery-name ${GALLERY_NAME} --gallery-image-definition rhcos-arm64 --gallery-image-version 1.0.0 --os-vhd-storage-account ${STORAGE_ACCOUNT_NAME} --os-vhd-uri ${RHCOS_VHD_URL} ``` 10. Optional: Now that your `arm64` boot image is now generated, you can access the ID of your image with the following command: ```terminal $ az sig image-version show -r $GALLERY_NAME -g $RESOURCE_GROUP -i rhcos-arm64 -e 1.0.0 ``` The following example image ID is used in the `recourseID` parameter of the compute machine set: ```terminal {title="Example resourceID"} /resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.Compute/galleries/${GALLERY_NAME}/images/rhcos-arm64/versions/1.0.0 ``` ## Adding a multi-architecture compute machine set to your Azure cluster {#multi-architecture-modify-machine-set-azure_creating-multi-arch-compute-nodes-azure} After creating a multi-architecture cluster, you can add nodes with different architectures. You can add multi-architecture compute machines to a multi-architecture cluster in the following ways: - Adding 64-bit x86 compute machines to a cluster that uses 64-bit ARM control plane machines and already includes 64-bit ARM compute machines. In this case, 64-bit x86 is considered the secondary architecture. - Adding 64-bit ARM compute machines to a cluster that uses 64-bit x86 control plane machines and already includes 64-bit x86 compute machines. In this case, 64-bit ARM is considered the secondary architecture. To create a custom compute machine set on Azure, see "Creating a compute machine set on Azure". > [!NOTE] > Before adding a secondary architecture node to your cluster, it is recommended to install the Multiarch Tuning Operator, and deploy a `ClusterPodPlacementConfig` custom resource. For more information, see "Managing workloads on multi-architecture clusters by using the Multiarch Tuning Operator". **Prerequisites** - You installed the OpenShift CLI (`oc`). - You created a 64-bit ARM or 64-bit x86 boot image. - You used the installation program to create a 64-bit ARM or 64-bit x86 single-architecture cluster with the multi-architecture installer binary. **Procedure** 1. Log in to the OpenShift CLI (`oc`). 2. Create a YAML file and add the configuration to create a compute machine set to control the 64-bit ARM or 64-bit x86 compute nodes in your cluster. ```yaml {title="Example MachineSet object for an Azure 64-bit ARM or x86 compute node"} apiVersion: machine.openshift.io/v1beta1 kind: MachineSet metadata: labels: machine.openshift.io/cluster-api-cluster: machine.openshift.io/cluster-api-machine-role: worker machine.openshift.io/cluster-api-machine-type: worker name: -machine-set-0 namespace: openshift-machine-api spec: replicas: 2 selector: matchLabels: machine.openshift.io/cluster-api-cluster: machine.openshift.io/cluster-api-machineset: -machine-set-0 template: metadata: labels: machine.openshift.io/cluster-api-cluster: machine.openshift.io/cluster-api-machine-role: worker machine.openshift.io/cluster-api-machine-type: worker machine.openshift.io/cluster-api-machineset: -machine-set-0 spec: lifecycleHooks: {} metadata: {} providerSpec: value: acceleratedNetworking: true apiVersion: machine.openshift.io/v1beta1 credentialsSecret: name: azure-cloud-credentials namespace: openshift-machine-api image: offer: "" publisher: "" resourceID: /resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.Compute/galleries/${GALLERY_NAME}/images/rhcos-arm64/versions/1.0.0 sku: "" version: "" kind: AzureMachineProviderSpec location: managedIdentity: -identity networkResourceGroup: -rg osDisk: diskSettings: {} diskSizeGB: 128 managedDisk: storageAccountType: Premium_LRS osType: Linux publicIP: false publicLoadBalancer: resourceGroup: -rg subnet: -worker-subnet userDataSecret: name: worker-user-data vmSize: Standard_D4ps_v5 vnet: -vnet zone: "" ``` where: `image.resourceID` : Specifies the boot image, such as `arm64` or `amd64`. `vmSize` : Specifies the instance type used in your installation. Some example instance types are `Standard_D4ps_v5` or `D8ps`. 3. Create the compute machine set by running the following command: ```terminal $ oc create -f ``` - Replace `` with the name of the YAML file with compute machine set configuration. For example: `arm64-machine-set-0.yaml`, or `amd64-machine-set-0.yaml`. **Verification** 1. Verify that the new machines are running by running the following command: ```terminal $ oc get machineset -n openshift-machine-api ``` The output must include the machine set that you created. ```terminal {title="Example output"} NAME DESIRED CURRENT READY AVAILABLE AGE -machine-set-0 2 2 2 2 10m ``` 2. You can check if the nodes are ready and schedulable by running the following command: ```terminal $ oc get nodes ``` **Additional resources** {._additional-resources} - [Installing a cluster on Azure with customizations](/openshift-docs-markdown/installing/installing_azure/ipi/installing-azure-customizations#installing-azure-customizations) - [Migrating to a cluster with multi-architecture compute machines](/openshift-docs-markdown/updating/updating_a_cluster/migrating-to-multi-payload#migrating-to-multi-payload) - [Creating a compute machine set on Azure](/openshift-docs-markdown/machine_management/creating_machinesets/creating-machineset-azure#creating-machineset-azure) - [Managing workloads on multi-architecture clusters by using the Multiarch Tuning Operator](/openshift-docs-markdown/post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator#multiarch-tuning-operator)