--- title: Creating a compute machine set on vSphere --- # Creating a compute machine set on vSphere {#creating-machineset-vsphere} You can define and create a OpenShift Container Platform compute machine set on VMware vSphere to enable the Machine API to automatically scale and manage compute nodes in vSphere. You can create a different compute machine set to serve a specific purpose in your OpenShift Container Platform cluster on vSphere. For example, you might create infrastructure machine sets and related machines so that you can move supporting workloads to the new machines. > [!IMPORTANT] > You can use the advanced machine management and scaling capabilities only in clusters where the Machine API is operational. Clusters with user-provisioned infrastructure require additional validation and configuration to use the Machine API. > > Clusters with the infrastructure platform type `none` cannot use the Machine API. This limitation applies even if the compute machines that are attached to the cluster are installed on a platform that supports the feature. This parameter cannot be changed after installation. > > To view the platform type for your cluster, run the following command: > > ```terminal > $ oc get infrastructure cluster -o jsonpath='{.status.platform}' > ``` ## Sample YAML for a compute machine set custom resource on vSphere {#machineset-yaml-vsphere_creating-machineset-vsphere} To enable the Machine API to automate node provisioning on VMware vSphere infrastructure, define a `MachineSet` resource with parameters that are specific to vSphere, for example data center, resource pool, and template. The sample YAML file defines a compute machine set that runs on vSphere and creates nodes that are labeled with `node-role.kubernetes.io/: ""`. In this sample, `` is the infrastructure ID label that is based on the cluster ID that you set when you provisioned the cluster, and `` is the node label to add. ```yaml apiVersion: machine.openshift.io/v1beta1 kind: MachineSet metadata: creationTimestamp: null labels: machine.openshift.io/cluster-api-cluster: name: - namespace: openshift-machine-api spec: replicas: 1 selector: matchLabels: machine.openshift.io/cluster-api-cluster: machine.openshift.io/cluster-api-machineset: - template: metadata: creationTimestamp: null labels: machine.openshift.io/cluster-api-cluster: machine.openshift.io/cluster-api-machine-role: machine.openshift.io/cluster-api-machine-type: machine.openshift.io/cluster-api-machineset: - spec: metadata: creationTimestamp: null labels: node-role.kubernetes.io/: "" providerSpec: value: apiVersion: machine.openshift.io/v1beta1 credentialsSecret: name: vsphere-cloud-credentials dataDisks: - name: "" provisioningMode: "" sizeGiB: 20 diskGiB: 120 kind: VSphereMachineProviderSpec memoryMiB: 8192 metadata: creationTimestamp: null network: devices: - networkName: "" numCPUs: 4 numCoresPerSocket: 1 snapshot: "" template: userDataSecret: name: worker-user-data workspace: datacenter: datastore: folder: resourcepool: server: ``` where `` : Specifies the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. If you have the OpenShift CLI (`oc`) installed, you can obtain the infrastructure ID by running the following command: ```terminal $ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster ``` `-` : Specifies the infrastructure ID and node label. `` : Specifies the node label to add. `` : Specifies one or more data disk definitions. For more information, see "Configuring data disks by using machine sets". `` : Specifies the image to use as a boot image for your nodes. `` : Specifies the vSphere VM network to deploy the compute machine set to. This VM network must be where other compute machines reside in the cluster. `` : Specifies the vSphere VM template to use as a boot image for your nodes, such as `user-5ddjd-rhcos`. You should use a template with the latest OpenShift Container Platform image when adding a new machine set. `` : Specifies the vCenter datacenter to deploy the compute machine set on. `` : Specifies the vCenter datastore to deploy the compute machine set on. `` : Specifies the path to the vSphere VM folder in vCenter, such as `/dc1/vm/user-inst-5ddjd`. `` : Specifies the vSphere resource pool for your VMs. `` : Specifies the vCenter server IP or fully qualified domain name. **Additional resources** {._additional-resources} - [Manually updating the boot image](/openshift-docs-markdown/machine_configuration/mco-update-boot-images-manual#mco-update-boot-images-manual) ## Minimum required vCenter privileges for compute machine set management {#machineset-vsphere-requirements-user-provisioned-machine-sets_creating-machineset-vsphere} To manage compute machine sets in an OpenShift Container Platform cluster on vCenter, you must use an account with privileges to read, create, and delete the required resources. Using an account that has global administrative privileges is the simplest way to access all of the necessary permissions. If you cannot use an account with global administrative privileges, you must create roles to grant the minimum required privileges. The following table lists the minimum vCenter roles and privileges that are required to create, scale, and delete compute machine sets and to delete machines in your OpenShift Container Platform cluster. **Minimum vCenter roles and privileges required for compute machine set management**
vSphere object for role When required Required privileges
vSphere vCenter Always InventoryService.Tagging.AttachTag InventoryService.Tagging.CreateCategory InventoryService.Tagging.CreateTag InventoryService.Tagging.DeleteCategory InventoryService.Tagging.DeleteTag InventoryService.Tagging.EditCategory InventoryService.Tagging.EditTag Sessions.ValidateSession StorageProfile.Update^1^ StorageProfile.View^1^
vSphere vCenter Cluster Always Resource.AssignVMToPool
vSphere datastore Always Datastore.AllocateSpace Datastore.Browse
vSphere Port Group Always Network.Assign
Virtual Machine Folder Always VirtualMachine.Config.AddRemoveDevice VirtualMachine.Config.AdvancedConfig VirtualMachine.Config.Annotation VirtualMachine.Config.CPUCount VirtualMachine.Config.DiskExtend VirtualMachine.Config.Memory VirtualMachine.Config.Settings VirtualMachine.Interact.PowerOff VirtualMachine.Interact.PowerOn VirtualMachine.Inventory.CreateFromExisting VirtualMachine.Inventory.Delete VirtualMachine.Provisioning.Clone
vSphere vCenter data center If the installation program creates the virtual machine folder. Resource.AssignVMToPool VirtualMachine.Provisioning.DeployTemplate
^1^ The StorageProfile.Update and StorageProfile.View permissions are required only for storage backends that use the Container Storage Interface (CSI).
The following table details the permissions and propagation settings that are required for compute machine set management. **Required permissions and propagation settings**
vSphere object Folder type Propagate to children Permissions required
vSphere vCenter Always Not required Listed required privileges
vSphere vCenter data center Existing folder Not required ReadOnly permission
Installation program creates the folder Required Listed required privileges
vSphere vCenter Cluster Always Required Listed required privileges
vSphere vCenter datastore Always Not required Listed required privileges
vSphere Switch Always Not required ReadOnly permission
vSphere Port Group Always Not required Listed required privileges
vSphere vCenter Virtual Machine Folder Existing folder Required Listed required privileges
For more information about creating an account with only the required privileges, see [vSphere Permissions and User Management Tasks](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.security.doc/GUID-5372F580-5C23-4E9C-8A4E-EF1B4DD9033E.html) in the vSphere documentation. ## Requirements for clusters with user-provisioned infrastructure to use compute machine sets {#compute-machineset-upi-reqs_creating-machineset-vsphere} To enable the Machine API to manage and scale compute nodes on user-provisioned infrastructure, you can configure a `MachineSet` YAML file with specific vSphere parameters, for example data center and disk image. To use compute machine sets on clusters that have user-provisioned infrastructure, you must ensure that you cluster configuration supports using the Machine API. ### Obtaining the infrastructure ID {#machineset-upi-reqs-infra-id_creating-machineset-vsphere} To ensure the Machine API correctly identifies and manages virtual machines (VMs) that belong to a specific cluster, you must add the unique infrastructure ID to the `MachineSet` YAML file to label and link resources. To create compute machine sets, you must be able to supply the infrastructure ID for your cluster. **Procedure** - To obtain the infrastructure ID for your cluster, run the following command: ```terminal $ oc get infrastructure cluster -o jsonpath='{.status.infrastructureName}' ``` ### Satisfying vSphere credentials requirements {#machineset-upi-reqs-vsphere-creds_creating-machineset-vsphere} To use compute machine sets and manage virtual machine (VM) resources, the Machine API must be able to interact with vCenter. Credentials that authorize the Machine API components to interact with vCenter must exist in a secret in the `openshift-machine-api` namespace. **Procedure** 1. To determine whether the required credentials exist, run the following command: ```terminal $ oc get secret \ -n openshift-machine-api vsphere-cloud-credentials \ -o go-template='{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}' ``` ```terminal {title="Sample output"} .password= .username= ``` where `` : Specifies the IP address or fully qualified domain name (FQDN) of the vCenter server and `` and `` are the OpenShift Container Platform administrator credentials to use. 2. If the secret does not exist, create it by running the following command: ```terminal $ oc create secret generic vsphere-cloud-credentials \ -n openshift-machine-api \ --from-literal=.username= --from-literal=.password= ``` ### Satisfying Ignition configuration requirements {#machineset-upi-reqs-ignition-config_creating-machineset-vsphere} For the Machine API to provision virtual machines (VMs) with the correct initial configuration using Ignition, a valid Ignition configuration is required. The Ignition configuration contains the `machine-config-server` address and a system trust bundle for obtaining further Ignition configurations from the Machine Config Operator. By default, this configuration is stored in the `worker-user-data` secret in the `machine-api-operator` namespace. Compute machine sets reference the secret during the machine creation process. **Procedure** 1. To determine whether the required secret exists, run the following command: ```terminal $ oc get secret \ -n openshift-machine-api worker-user-data \ -o go-template='{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}' ``` ```terminal {title="Sample output"} disableTemplating: false userData: { "ignition": { ... }, ... } ``` The full output is omitted here, but this is the format to use. 2. If the secret does not exist, create it by running the following command: ```terminal $ oc create secret generic worker-user-data \ -n openshift-machine-api \ --from-file=/worker.ign ``` Specifies the directory that was used to store your installation assets during cluster installation. **Additional resources** {._additional-resources} - [Understanding the Machine Config Operator](/openshift-docs-markdown/machine_configuration/index#machine-config-operator_machine-config-overview) - [Installing RHCOS and starting the OpenShift Container Platform bootstrap process](/openshift-docs-markdown/installing/installing_vsphere/upi/installing-vsphere#installation-vsphere-machines_installing-vsphere) ## Creating a compute machine set {#machineset-creating_creating-machineset-vsphere} To dynamically manage machine compute resources, you can create your own compute machine sets in addition to the compute machine sets created by the installation program. Use the OpenShift Container Platform CLI to automate node provisioning. > [!NOTE] > Clusters that are installed with user-provisioned infrastructure have a different networking stack than clusters with infrastructure that is provisioned by the installation program. As a result of this difference, automatic load balancer management is unsupported on clusters that have user-provisioned infrastructure. For these clusters, a compute machine set can only create `worker` and `infra` type machines. **Prerequisites** - Deploy an OpenShift Container Platform cluster. - Install the OpenShift CLI (`oc`). - Log in to `oc` as a user with `cluster-admin` permission. - Have the necessary permissions to deploy VMs in your vCenter instance and have the required access to the datastore specified. - If your cluster uses user-provisioned infrastructure, you have satisfied the specific Machine API requirements for that configuration. **Procedure** 1. Create a new YAML file that contains the compute machine set custom resource (CR) sample and is named `.yaml`. Ensure that you set the `` and `` parameter values. 2. Optional: If you are not sure which value to set for a specific field, you can check an existing compute machine set from your cluster. 1. To list the compute machine sets in your cluster, run the following command: ```terminal $ oc get machinesets -n openshift-machine-api ``` The following is example output: ```terminal NAME DESIRED CURRENT READY AVAILABLE AGE agl030519-vplxk-worker-us-east-1a 1 1 1 1 55m agl030519-vplxk-worker-us-east-1b 1 1 1 1 55m agl030519-vplxk-worker-us-east-1c 1 1 1 1 55m agl030519-vplxk-worker-us-east-1d 0 0 55m agl030519-vplxk-worker-us-east-1e 0 0 55m agl030519-vplxk-worker-us-east-1f 0 0 55m ``` 2. To view values of a specific compute machine set custom resource (CR), run the following command: ```terminal $ oc get machineset \ -n openshift-machine-api -o yaml ``` The following is example output: ```yaml apiVersion: machine.openshift.io/v1beta1 kind: MachineSet metadata: labels: machine.openshift.io/cluster-api-cluster: name: - namespace: openshift-machine-api spec: replicas: 1 selector: matchLabels: machine.openshift.io/cluster-api-cluster: machine.openshift.io/cluster-api-machineset: - template: metadata: labels: machine.openshift.io/cluster-api-cluster: machine.openshift.io/cluster-api-machine-role: machine.openshift.io/cluster-api-machine-type: machine.openshift.io/cluster-api-machineset: - spec: providerSpec: ... ``` where: `metadata.labels.machine.openshift.io/cluster-api-cluster` : Specifies the cluster infrastructure ID. `metadata.labels.name` : Specifies a default node label. > [!NOTE] > For clusters that have user-provisioned infrastructure, a compute machine set can only create `worker` and `infra` type machines. `spec.template.metadata.spec.providerSpec` : Specifies the values of the compute machine set CR. The values are platform-specific. For more information about `` parameters in the CR, see the sample compute machine set CR configuration for your provider. 3. If you are creating a compute machine set for a cluster that has user-provisioned infrastructure, note the following important values: ```yaml {title="Example vSphere providerSpec values"} apiVersion: machine.openshift.io/v1beta1 kind: MachineSet ... template: ... spec: providerSpec: value: apiVersion: machine.openshift.io/v1beta1 credentialsSecret: name: vsphere-cloud-credentials dataDisks: - name: provisioningMode: sizeGiB: 10 diskGiB: 120 kind: VSphereMachineProviderSpec memoryMiB: 16384 network: devices: - networkName: "" numCPUs: 4 numCoresPerSocket: 4 snapshot: "" template: userDataSecret: name: worker-user-data workspace: datacenter: datastore: folder: resourcepool: server: ``` where: `vsphere-cloud-credentials` : Specifies the name of the secret in the `openshift-machine-api` namespace that contains the required vCenter credentials. `` : Specifies the collection of data disk definitions. For more information, see "Configuring data disks by using machine sets". `` : Specifies the name of the RHCOS VM template for your cluster that was created during installation. `worker-user-data` : Specifies the name of the secret in the `openshift-machine-api` namespace that contains the required Ignition configuration credentials. `` : Specifies the IP address or fully qualified domain name (FQDN) of the vCenter server. 3. Create a `MachineSet` CR by running the following command: ```terminal $ oc create -f .yaml ``` **Verification** - View the list of compute machine sets by running the following command: ```terminal $ oc get machineset -n openshift-machine-api ``` The following is example output: ```terminal NAME DESIRED CURRENT READY AVAILABLE AGE agl030519-vplxk-infra-us-east-1a 1 1 1 1 11m agl030519-vplxk-worker-us-east-1a 1 1 1 1 55m agl030519-vplxk-worker-us-east-1b 1 1 1 1 55m agl030519-vplxk-worker-us-east-1c 1 1 1 1 55m agl030519-vplxk-worker-us-east-1d 0 0 55m agl030519-vplxk-worker-us-east-1e 0 0 55m agl030519-vplxk-worker-us-east-1f 0 0 55m ``` When the new compute machine set is available, the `DESIRED` and `CURRENT` values match. If the compute machine set is not available, wait a few minutes and run the command again. ## Labeling GPU machine sets for the cluster autoscaler {#machineset-label-gpu-autoscaler_creating-machineset-vsphere} Label your machine sets to indicate which machines the cluster autoscaler can use for GPU-enabled nodes. Applying the accelerator label helps ensure that the autoscaler deploys the correct resources for your GPU workloads. **Prerequisites** - Your cluster uses a cluster autoscaler. **Procedure** - On the machine set that you want to create machines for the cluster autoscaler to use to deploy GPU-enabled nodes, add a `cluster-api/accelerator` label: ```yaml apiVersion: machine.openshift.io/v1beta1 kind: MachineSet metadata: name: machine-set-name spec: template: spec: metadata: labels: cluster-api/accelerator: ``` where: `` : Specifies a label of your choice that consists of alphanumeric characters, `-`, `_`, or `.` and starts and ends with an alphanumeric character. For example, you might use `nvidia-t4` to represent Nvidia T4 GPUs, or `nvidia-a10g` for A10G GPUs. > [!NOTE] > You must specify the value of this label for the `spec.resourceLimits.gpus.type` parameter in your `ClusterAutoscaler` CR. For more information, see "Cluster autoscaler resource definition". **Additional resources** {._additional-resources} - [Cluster autoscaler resource definition](/openshift-docs-markdown/machine_management/applying-autoscaling#cluster-autoscaler-cr_applying-autoscaling) ## Adding tags to machines by using machine sets {#machine-api-vmw-add-tags_creating-machineset-vsphere} To ensure that your cluster remains scalable and resilient, you can use a `MachineSet` object and machine health checks to automate the provisioning and repair of nodes. OpenShift Container Platform adds a cluster-specific tag to each virtual machine (VM) that it creates. The installation program uses these tags to select the VMs to delete when uninstalling a cluster. In addition to the cluster-specific tags assigned to VMs, you can configure a machine set to add up to 10 additional vSphere tags to the VMs it provisions. **Prerequisites** - You have access to an OpenShift Container Platform cluster installed on vSphere using an account with `cluster-admin` permissions. - You have access to the VMware vCenter console associated with your cluster. - You have created a tag in the vCenter console. - You have installed the OpenShift CLI (`oc`). **Procedure** 1. Use the vCenter console to find the tag ID for any tag that you want to add to your machines: 1. Log in to the vCenter console. 2. From the **Home** menu, click **Tags & Custom Attributes**. 3. Select a tag that you want to add to your machines. 4. Use the browser URL for the tag that you select to identify the tag ID. ```text {title="Example tag URL"} https://vcenter.example.com/ui/app/tags/tag/urn:vmomi:InventoryServiceTag:208e713c-cae3-4b7f-918e-4051ca7d1f97:GLOBAL/permissions ``` ```text {title="Example tag ID"} urn:vmomi:InventoryServiceTag:208e713c-cae3-4b7f-918e-4051ca7d1f97:GLOBAL ``` 2. In a text editor, open the YAML file for an existing machine set or create a new one. 3. Edit the following lines under the `providerSpec` field: ```yaml tag::compute[] apiVersion: machine.openshift.io/v1beta1 kind: MachineSet end::compute[] tag::controlplane[] apiVersion: machine.openshift.io/v1 kind: ControlPlaneMachineSet end::controlplane[] # ... spec: template: spec: providerSpec: value: tagIDs: - # ... ``` where `spec.template.spec.providerSpec.value.tagIDs` : Specifies a list of up to 10 tags to add to the machines that this machine set provisions. Replace `` with the tag that you want to add to your machines. For example, `urn:vmomi:InventoryServiceTag:208e713c-cae3-4b7f-918e-4051ca7d1f97:GLOBAL`. ## Configuring multiple network interface controllers by using machine sets {#machineset-vsphere-multiple-nics_creating-machineset-vsphere} By configuring multiple network interface controllers (NICs), you can provide dedicated network links in the node virtual machines (VMs) for uses such as storage or databases. OpenShift Container Platform clusters on VMware vSphere support connecting up to 10 network NICs to a node. You can use machine sets to manage this configuration. - If you want to use multiple NICs in a vSphere cluster that was not configured to do so during installation, you can use machine sets to implement this configuration. - If your cluster was set up during installation to use multiple NICs, machine sets that you create can use your existing failure domain configuration. - If your failure domain configuration changes, you can use machine sets to make updates that reflect those changes. tag:controlplane[]\[\] > [!NOTE] > This feature is not compatible with a control plane machine set that uses more than one failure domain. end:controlplane[]\[\] **Prerequisites** - You have administrator access to OpenShift CLI (`oc`) for an OpenShift Container Platform cluster on vSphere. **Procedure** 1. For a cluster that already uses multiple NICs, obtain the following values from the `Infrastructure` resource by running the following command: ```terminal $ oc get infrastructure cluster -o=jsonpath={.spec.platformSpec.vsphere.failureDomains} ``` **Required network interface controller values** | `Infrastructure` resource value | Placeholder value for sample machine set | Description | | --- | --- | --- | | `failureDomain.topology.networks[0]` | `` | The name of the first NIC to use. | | `failureDomain.topology.networks[1]` | `` | The name of the second NIC to use. | | `failureDomain.topology.networks[]` | `` | The name of the \__n__th NIC to use. Collect the name of each NIC in the `Infrastructure` resource. | | `failureDomain.topology.template` | `` | The vSphere VM template to use. | | `failureDomain.topology.datacenter` | `` | The vCenter data center to deploy the machine set on. | | `failureDomain.topology.datastore` | `` | The vCenter datastore to deploy the machine set on. | | `failureDomain.topology.folder` | `` | The path to the vSphere VM folder in vCenter, such as `/dc1/vm/user-inst-5ddjd`. | | `failureDomain.topology.computeCluster` + `/Resources` | `` | The vSphere resource pool for your VMs. | | `failureDomain.server` | `` | The vCenter server IP or fully qualified domain name (FQDN). | 2. In a text editor, open the YAML file for an existing machine set or create a new one. 3. Use a machine set configuration formatted like the following example. - For a cluster that currently uses multiple NICs, use the values from the `Infrastructure` resource to populate the values in the machine set custom resource. - For a cluster that is not using multiple NICs, populate the values you want to use in the machine set custom resource. ```yaml {title="Sample machine set"} tag::compute[] apiVersion: machine.openshift.io/v1beta1 kind: MachineSet # ... spec: template: spec: providerSpec: value: network: devices: - networkName: "" - networkName: "" template: workspace: datacenter: datastore: folder: resourcepool: server: # ... end::compute[] tag::controlplane[] apiVersion: machine.openshift.io/v1 kind: ControlPlaneMachineSet # ... spec: template: machines_v1beta1_machine_openshift_io: spec: providerSpec: value: network: devices: - networkName: "" - networkName: "" template: workspace: datacenter: datastore: folder: resourcepool: server: # ... end::controlplane[] ``` where: tag:compute[]\[\] `spec.template.spec.providerSpec.value.network.devices` : Specifies a list of up to 10 NICs to use. `spec.template.spec.providerSpec.value.network.template` : Specifies the vSphere VM template to use, such as `user-5ddjd-rhcos`. `spec.template.spec.providerSpec.value.network.workspace.datacenter` : Specifies the vCenter data center to deploy the machine set on. `spec.template.spec.providerSpec.value.network.workspace.datastore` : Specifies the vCenter datastore to deploy the machine set on. `spec.template.spec.providerSpec.value.network.workspace.folder` : Specifies the path to the vSphere VM folder in vCenter, such as `/dc1/vm/user-inst-5ddjd`. `spec.template.spec.providerSpec.value.network.workspace.resourcepool` : Specifies the vSphere resource pool for your VMs. `spec.template.spec.providerSpec.value.network.workspace.server` : Specifies the vCenter server IP or fully qualified domain name (FQDN). end:compute[]\[\] tag:controlplane[]\[\] `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.network.devices` : Specifies a list of up to 10 NICs to use. `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.network.template` : Specifies the vSphere VM template to use, such as `user-5ddjd-rhcos`. `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.network.workspace.datacenter` : Specifies the vCenter data center to deploy the machine set on. `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.network.workspace.datastore` : Specifies the vCenter datastore to deploy the machine set on. `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.network.workspace.folder` : Specifies the path to the vSphere VM folder in vCenter, such as `/dc1/vm/user-inst-5ddjd`. `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.network.workspace.resourcepool` : Specifies the vSphere resource pool for your VMs. `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.network.workspace.server` : Specifies the vCenter server IP or fully qualified domain name (FQDN). end:controlplane[]\[\] ## Configuring data disks by using machine sets {#machineset-vsphere-data-disks_creating-machineset-vsphere} To provide persistent storage beyond the root volume for specialized application workloads, define a `dataDisks` array in the `MachineSet` YAML file to specify disk size and storage policy. OpenShift Container Platform clusters on VMware vSphere support adding up to 29 disks to the virtual machine (VM) controller. > [!IMPORTANT] > Configuring vSphere data disks is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. > > For more information about the support scope of Red Hat Technology Preview features, see [Technology Preview Features Support Scope](https://access.redhat.com/support/offerings/techpreview/). By configuring data disks, you can attach disks to VMs and use them to store data for etcd, container images, and other uses. Separating data can help avoid filling the primary disk so that important activities such as upgrades have the resources that they require. > [!NOTE] > Adding data disks attaches them to the VM and mounts them to the location that RHCOS designates. **Prerequisites** - You have administrator access to OpenShift CLI (`oc`) for an OpenShift Container Platform cluster on vSphere. **Procedure** 1. In a text editor, open the YAML file for an existing machine set or create a new one. 2. Edit the following lines under the `providerSpec` field: ```yaml tag::compute[] apiVersion: machine.openshift.io/v1beta1 kind: MachineSet end::compute[] tag::controlplane[] apiVersion: machine.openshift.io/v1 kind: ControlPlaneMachineSet end::controlplane[] # ... spec: template: tag::compute[] spec: providerSpec: value: dataDisks: - name: "" provisioningMode: "" sizeGiB: 20 - name: "" provisioningMode: "" sizeGiB: 20 end::compute[] tag::controlplane[] machines_v1beta1_machine_openshift_io: spec: providerSpec: value: dataDisks: - name: "" provisioningMode: "" sizeGiB: 20 - name: "" provisioningMode: "" sizeGiB: 20 end::controlplane[] # ... ``` where tag:compute[]\[\] `spec.template.spec.providerSpec.value.dataDisks` : Specifies a collection of 1-29 data disk definitions. This sample configuration shows the formatting to include two data disk definitions. `spec.template.spec.providerSpec.value.dataDisks.name` : Specifies the name of the data disk. The name must meet the following requirements: - Start and end with an alphanumeric character - Consist only of alphanumeric characters, hyphens (`-`), and underscores (`_`) - Have a maximum length of 80 characters `spec.template.spec.providerSpec.value.dataDisks.provisioningMode` : Specifies the data disk provisioning method. This value defaults to the vSphere default storage policy if not set. Valid values are `Thin`, `Thick`, and `EagerlyZeroed`. `spec.template.spec.providerSpec.value.dataDisks.sizeGiB` : Specifies the size of the data disk in GiB. The maximum size is 16,384 GiB. end:compute[]\[\] tag:controlplane[]\[\] `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.dataDisks` : Specifies a collection of 1-29 data disk definitions. This sample configuration shows the formatting to include two data disk definitions. `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.dataDisks.name` : Specifies the name of the data disk. The name must meet the following requirements: - Start and end with an alphanumeric character - Consist only of alphanumeric characters, hyphens (`-`), and underscores (`_`) - Have a maximum length of 80 characters `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.dataDisks.provisioningMode` : Specifies the data disk provisioning method. This value defaults to the vSphere default storage policy if not set. Valid values are `Thin`, `Thick`, and `EagerlyZeroed`. `spec.template.machines_v1beta1_machine_openshift_io.spec.providerSpec.value.dataDisks.sizeGiB` : Specifies the size of the data disk in GiB. The maximum size is 16,384 GiB. end:controlplane[]\[\]