--- title: Updating OpenShift Virtualization --- # Updating OpenShift Virtualization {#upgrading-virt} Learn how to keep OpenShift Virtualization updated and compatible with OpenShift Container Platform. ## About updating OpenShift Virtualization {#virt-about-upgrading-virt_upgrading-virt} When you install OpenShift Virtualization, you select an update channel and an approval strategy. Both settings affect supportability. The update channel determines the version of OpenShift Virtualization that you use. The approval strategy determines whether updates occur automatically or require manual approval. ### Recommended settings {#recommended-settings_upgrading-virt} To keep a supportable environment, use the following settings: - Update channel: **stable** - Approval strategy: **Automatic** Most OpenShift Virtualization installations use the **stable** release channel and the **Automatic** approval strategy. Use other settings only if you understand the risks. With these settings, the update process starts automatically when a new Operator version is available in the **stable** channel. This keeps OpenShift Virtualization and OpenShift Container Platform versions compatible and ensures that OpenShift Virtualization is suitable for production environments. > [!NOTE] > Each minor version of OpenShift Virtualization is supported only with the corresponding OpenShift Container Platform version. For example, you must run OpenShift Virtualization 4.22 on OpenShift Container Platform 4.22. ### What to expect {#what-to-expect_upgrading-virt} You can expect consistent update behavior in OpenShift Virtualization, including duration, automation, and data preservation. - The time required to complete an update depends on your network connection. Most automatic updates complete within fifteen minutes. - Updating OpenShift Virtualization does not interrupt network connections. - An update preserves data volumes and their associated persistent volume claims. > [!IMPORTANT] > If virtual machines use hostpath provisioner storage, they cannot be live migrated and might block an OpenShift Container Platform cluster update. > > As a workaround, reconfigure the virtual machines so they can power off automatically during a cluster update. Set the `evictionStrategy` field to `None` and the `runStrategy` field to `Always`. ### How updates work {#how-updates-work_upgrading-virt} Learn how Operator Lifecycle Manager (OLM) updates the OpenShift Virtualization Operator and how update channels and approval strategies affect upgrade behavior. - Operator Lifecycle Manager (OLM) manages the lifecycle of the OpenShift Virtualization Operator. The Marketplace Operator, deployed during OpenShift Container Platform installation, makes external Operators available to your cluster. - OLM provides z-stream and minor version updates for OpenShift Virtualization. Minor version updates become available when you update OpenShift Container Platform to the next minor version. You cannot update OpenShift Virtualization to the next minor version without first updating OpenShift Container Platform. ### Changing update settings {#virt-changing-update-settings_upgrading-virt} You can control how and when updates are installed by changing the update channel and approval strategy for the OpenShift Virtualization Operator subscription. **Prerequisites** - You have installed the OpenShift Virtualization Operator. - You have logged in to the OpenShift Container Platform web console as a cluster administrator. **Procedure** 1. Click **Ecosystem** → **Installed Operators**. 2. Select **OpenShift Virtualization** from the list. 3. Click the **Subscription** tab. 4. In the **Subscription details** section, click the setting that you want to change. For example, to change the approval strategy from **Manual** to **Automatic**, click **Manual**. 5. In the window that opens, select the new update channel or approval strategy. 6. Click **Save**. ### Manual approval strategy {#virt-manual-approval-strategy_upgrading-virt} If you use the **Manual** approval strategy, you must approve every pending update. If OpenShift Container Platform and OpenShift Virtualization updates are out of sync, your cluster becomes unsupported. To avoid risk to cluster supportability and functionality, use the **Automatic** approval strategy. If you must use the **Manual** approval strategy, approve pending Operator updates as soon as they become available. ### Approving a pending Operator update manually {#olm-approving-pending-upgrade_upgrading-virt} If an installed Operator has the approval strategy in its subscription set to **Manual**, you must manually approve the update before installation can begin. Manual approval reviews the changes and control when updates are applied to prevent unexpected downtime. **Prerequisites** - An Operator previously installed using Operator Lifecycle Manager (OLM). **Procedure** 1. In the OpenShift Container Platform web console, navigate to **Ecosystem** → **Installed Operators**. 2. Operators that have a pending update display a status with **Upgrade available**. Click the name of the Operator you want to update. 3. Click the **Subscription** tab. Any updates requiring approval are displayed next to **Upgrade status**. For example, it might display **1 requires approval**. 4. Click **1 requires approval**, then click **Preview Install Plan**. 5. Review the resources that are listed as available for update. When satisfied, click **Approve**. 6. Navigate back to the **Ecosystem** → **Installed Operators** page to monitor the progress of the update. When complete, the status changes to **Succeeded** and **Up to date**. ## Remove unused Operators and resources {#virt-update-removing-unused_upgrading-virt} When updating OpenShift Virtualization to version 4.22, you can remove certain Operators and resources that are no longer required. This helps reclaim space and resources on your cluster. ## RHEL 9 compatibility {#virt-rhel-9_upgrading-virt} OpenShift Virtualization 4.22 is based on Red Hat Enterprise Linux (RHEL) 9. ### RHEL 9 machine type {#rhel-9-machine-type_upgrading-virt} All VM templates that are included with OpenShift Virtualization now use the RHEL 9 machine type by default: `machineType: pc-q35-rhel9..0`, where `` is a single digit corresponding to the latest minor version of RHEL 9. For example, the value `pc-q35-rhel9.2.0` is used for RHEL 9.2. Updating OpenShift Virtualization does not change the `machineType` value of any existing VMs. These VMs continue to function as they did before the update. You can optionally change a VM’s machine type so that it can benefit from RHEL 9 improvements. > [!IMPORTANT] > Before you change a VM’s `machineType` value, you must shut down the VM. ## Monitoring update status {#virt-monitoring-upgrade-status_upgrading-virt} To monitor the status of a OpenShift Virtualization Operator update, watch the cluster service version (CSV) `PHASE`. You can also monitor the CSV conditions in the web console or by using the CLI. > [!NOTE] > The `PHASE` and conditions values are approximations that are based on available information. **Prerequisites** - You have logged in to the OpenShift Container Platform cluster as a cluster administrator. - You have installed the OpenShift CLI (`oc`). **Procedure** 1. Run the following command: ```terminal $ oc get csv -n openshift-cnv ``` 2. Review the output, checking the `PHASE` field. For example: ```terminal VERSION REPLACES PHASE 4.9.0 kubevirt-hyperconverged-operator.v4.8.2 Installing 4.9.0 kubevirt-hyperconverged-operator.v4.9.0 Replacing ``` 3. Optional: Monitor the aggregated status of all OpenShift Virtualization component conditions by running the following command: ```terminal $ oc get hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv \ -o=jsonpath='{range .status.conditions[*]}{.type}{"\t"}{.status}{"\t"}{.message}{"\n"}{end}' ``` A successful upgrade results in the following output: ```terminal ReconcileComplete True Reconcile completed successfully Available True Reconcile completed successfully Progressing False Reconcile completed successfully Degraded False Reconcile completed successfully Upgradeable True Reconcile completed successfully ``` ## VM workload updates {#virt-about-workload-updates_upgrading-virt} When you update OpenShift Virtualization, virtual machine workloads, including `libvirt`, `virt-launcher`, and `qemu`, update automatically if they support live migration. > [!NOTE] > Each virtual machine has a `virt-launcher` pod that runs the virtual machine instance (VMI). The `virt-launcher` pod runs an instance of `libvirt`, which is used to manage the virtual machine (VM) process. You can configure how workloads are updated by editing the `spec.workloadUpdateStrategy` stanza of the `HyperConverged` custom resource (CR). There are two available workload update methods: `LiveMigrate` and `Evict`. Because the `Evict` method shuts down VMI pods, only the `LiveMigrate` update strategy is enabled by default. When `LiveMigrate` is the only update strategy enabled: - VMIs that support live migration are migrated during the update process. The VM guest moves into a new pod with the updated components enabled. - VMIs that do not support live migration are not disrupted or updated. - If a VMI has the `LiveMigrate` eviction strategy but does not support live migration, it is not updated. If you enable both `LiveMigrate` and `Evict`: - VMIs that support live migration use the `LiveMigrate` update strategy. - VMIs that do not support live migration use the `Evict` update strategy. If a VMI is controlled by a `VirtualMachine` object that has `runStrategy: Always` set, a new VMI is created in a new pod with updated components. ### Migration attempts and timeouts {#migration-attempts-timeouts_upgrading-virt} When updating workloads, live migration fails if a pod is in the `Pending` state for the following periods: 5 minutes : If the pod is pending because it is `Unschedulable`. 15 minutes : If the pod is stuck in the pending state for any reason. When a VMI fails to migrate, the `virt-controller` tries to migrate it again. It repeats this process until all migratable VMIs are running on new `virt-launcher` pods. If a VMI is improperly configured, however, these attempts can repeat indefinitely. > [!NOTE] > Each attempt corresponds to a migration object. Only the five most recent attempts are held in a buffer. This prevents migration objects from accumulating on the system while retaining information for debugging. ### Configuring workload update methods {#virt-configuring-workload-update-methods_upgrading-virt} You can configure how virtual machine workloads are updated during cluster upgrades by editing the `HyperConverged` custom resource (CR). **Prerequisites** - You have enabled live migration in the cluster. > [!NOTE] > If a `VirtualMachineInstance` CR contains `evictionStrategy: LiveMigrate` and the virtual machine instance (VMI) does not support live migration, the VMI will not update. - You have installed the OpenShift CLI (`oc`). **Procedure** 1. To open the `HyperConverged` CR in your default editor, run the following command: ```terminal $ oc edit hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv ``` 2. Edit the `workloadUpdateStrategy` stanza of the `HyperConverged` CR. For example: ```yaml apiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged spec: workloadUpdateStrategy: workloadUpdateMethods: - LiveMigrate - Evict batchEvictionSize: 10 batchEvictionInterval: "1m0s" # ... ``` - `spec.workloadUpdateStrategy.workloadUpdateMethods` defines the methods that can be used to perform automated workload updates. The available values are `LiveMigrate` and `Evict`. If you enable both options as shown in this example, updates use `LiveMigrate` for VMIs that support live migration and `Evict` for any VMIs that do not support live migration. To disable automatic workload updates, you can either remove the `workloadUpdateStrategy` stanza or set `workloadUpdateMethods: []` to leave the array empty. - `LiveMigrate` is the least disruptive update method. VMIs that support live migration are updated by migrating the virtual machine (VM) guest into a new pod with the updated components enabled. If `LiveMigrate` is the only workload update method listed, VMIs that do not support live migration are not disrupted or updated. - `Evict` is a disruptive method that shuts down VMI pods during upgrade. `Evict` is the only update method available if live migration is not enabled in the cluster. If a VMI is controlled by a `VirtualMachine` object that has `runStrategy: Always` configured, a new VMI is created in a new pod with updated components. - `spec.workloadUpdateStrategy.batchEvictionSize` defines the number of VMIs that can be forced to be updated at a time by using the `Evict` method. This does not apply to the `LiveMigrate` method. - `spec.workloadUpdateStrategy.batchEvictionInterval` defines the interval to wait before evicting the next batch of workloads. This does not apply to the `LiveMigrate` method. > [!NOTE] > You can configure live migration limits and timeouts by editing the `spec.liveMigrationConfig` stanza of the `HyperConverged` CR. 3. To apply your changes, save and exit the editor. ### Viewing outdated VM workloads {#virt-viewing-outdated-workloads_upgrading-virt} You can view a list of outdated virtual machine (VM) workloads by using the CLI. > [!NOTE] > If there are outdated virtualization pods in your cluster, the `OutdatedVirtualMachineInstanceWorkloads` alert fires. **Prerequisites** - You have installed the OpenShift CLI (`oc`). **Procedure** - To view a list of outdated virtual machine instances (VMIs), run the following command: ```terminal $ oc get vmi -l kubevirt.io/outdatedLauncherImage --all-namespaces ``` ## Control Plane Only updates {#virt-about-control-plane-only-updates_upgrading-virt} You can use a Control Plane Only update to move between Extended Update Support (EUS) versions of OpenShift Container Platform. This prevents virtual machine workloads from updating during the intermediate upgrade. Every even-numbered minor version of OpenShift Container Platform is an Extended Update Support (EUS) version. Kubernetes requires minor version updates to occur in sequence. You cannot update directly from one EUS version to the next. To move between EUS versions, first update OpenShift Virtualization to the latest z-stream release of the next odd-numbered minor version. Then update the cluster to the target EUS version of OpenShift Container Platform. After the cluster update, update OpenShift Virtualization to the target EUS version. > [!NOTE] > You can update OpenShift Virtualization directly to the latest z-stream release of your current minor version without applying each intermediate z-stream update. For more information about EUS versions, see the "OpenShift Container Platform Life Cycle Policy". ### Preventing workload updates during a Control Plane Only update {#virt-preventing-workload-updates-during-control-plane-only-update_upgrading-virt} When updating between Extended Update Support (EUS) versions, temporarily disable automatic workload updates. This prevents OpenShift Virtualization from migrating or evicting virtual machines during the upgrade. > [!IMPORTANT] > In OpenShift Container Platform 4.16, the underlying Red Hat Enterprise Linux CoreOS (RHCOS) upgraded to version 9.4 of Red Hat Enterprise Linux (RHEL). All `virt-launcher` pods in the cluster must use the same RHEL version. > > After upgrading to OpenShift Container Platform 4.16, re-enable workload updates in OpenShift Virtualization. This allows `virt-launcher` pods to update. Before upgrading to the next OpenShift Container Platform version, verify that all VMIs use up-to-date workloads: > > ```terminal > $ oc get kv kubevirt-kubevirt-hyperconverged -o json -n openshift-cnv | jq .status.outdatedVirtualMachineInstanceWorkloads > ``` > > If the command returns a value greater than `0`, list VMIs with outdated `virt-launcher` pods and start live migration: > > ```terminal > $ oc get vmi -l kubevirt.io/outdatedLauncherImage --all-namespaces > ``` > > For supported OpenShift Container Platform releases and their RHEL versions, see [RHEL Versions Utilized by RHCOS and OpenShift Container Platform](https://access.redhat.com/articles/6907891). **Prerequisites** - You have installed the OpenShift CLI (`oc`). - You are running an EUS version of OpenShift Container Platform and plan to update to the next EUS version. - You have not yet updated to the intermediate odd-numbered minor version. - You paused the worker nodes' machine config pools as described in the OpenShift Container Platform documentation. - Use the default **Automatic** approval strategy. If you use the **Manual** approval strategy, you must approve all pending updates in the web console. For more details, see "Manually approving a pending Operator update". **Procedure** 1. Run the following command and record the `workloadUpdateMethods` value: ```terminal $ oc get kv kubevirt-kubevirt-hyperconverged \ -n openshift-cnv -o jsonpath='{.spec.workloadUpdateStrategy.workloadUpdateMethods}' ``` 2. Disable workload update methods by running the following command: ```terminal $ oc patch hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv \ --type json -p '[{"op":"replace","path":"/spec/workloadUpdateStrategy/workloadUpdateMethods", "value":[]}]' ``` 3. Ensure that the `HyperConverged` Operator is `Upgradeable`: ```terminal $ oc get hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.conditions" ``` 4. Update your cluster from the source EUS version to the next minor version of OpenShift Container Platform: ```terminal $ oc adm upgrade ``` 5. Verify the current cluster version: ```terminal $ oc get clusterversion ``` > [!NOTE] > Updating OpenShift Container Platform to the next version is a prerequisite for updating OpenShift Virtualization. For more details, see the "Updating clusters" section of the OpenShift Container Platform documentation. 6. Update OpenShift Virtualization. - With the default **Automatic** approval strategy, OpenShift Virtualization automatically updates after the OpenShift Container Platform update completes. - If you use the **Manual** approval strategy, approve the pending update in the web console. 7. Monitor the OpenShift Virtualization update: ```terminal $ oc get csv -n openshift-cnv ``` 8. Confirm that OpenShift Virtualization updated to the latest z-stream release of the intermediate version: ```terminal $ oc get hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.versions" ``` 9. Wait until the `HyperConverged` Operator again reports the `Upgradeable` condition. 10. Update OpenShift Container Platform to the target EUS version. 11. Verify the cluster version: ```terminal $ oc get clusterversion ``` 12. Update OpenShift Virtualization to the target EUS version. - With the default **Automatic** approval strategy, OpenShift Virtualization updates automatically. - If you use the **Manual** approval strategy, approve the pending update in the web console. 13. Monitor the update: ```terminal $ oc get csv -n openshift-cnv ``` The update completes when the `VERSION` field matches the target EUS version and the `PHASE` field reads `Succeeded`. 14. Restore the `workloadUpdateMethods` configuration recorded in step 1: ```terminal $ oc patch hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv --type json -p \ "[{\"op\":\"add\",\"path\":\"/spec/workloadUpdateStrategy/workloadUpdateMethods\", \"value\":{WorkloadUpdateMethodConfig}}]" ``` **Verification** - Check the status of VM migrations: ```terminal $ oc get vmim -A ``` **Next steps** - Unpause the machine config pools for each compute node. ## Early access releases {#virt-early-access-releases_upgrading-virt} You can access development builds by subscribing to the **candidate** update channel for your version of OpenShift Virtualization. These releases are not fully tested by Red Hat and are not supported. Use them only on non-production clusters to test new capabilities and bug fixes. The **stable** channel matches the underlying OpenShift Container Platform version and is fully tested. It is suitable for production systems. You can switch between the **stable** and **candidate** channels in OperatorHub. Updating from a **candidate** release to a **stable** release is not tested by Red Hat. Red Hat promotes some candidate releases to the **stable** channel. Other candidate releases might not include all GA features. Red Hat might remove some features from candidate builds before GA. Candidate releases might not offer update paths to later GA releases. > [!IMPORTANT] > Use the candidate channel only for testing where you can delete and re-create the cluster. **Additional resources** {._additional-resources} - [OpenShift Container Platform Life Cycle Policy](https://access.redhat.com/support/policy/updates/openshift) - [Performing a Control Plane Only update](/openshift-docs-markdown/updating/updating_a_cluster/control-plane-only-update#control-plane-only-update) - [What are Operators?](/openshift-docs-markdown/operators/understanding/olm-what-operators-are#olm-what-operators-are) - [Operator Lifecycle Manager concepts and resources](/openshift-docs-markdown/operators/understanding/olm/olm-understanding-olm#olm-understanding-olm) - [Cluster service versions (CSVs)](/openshift-docs-markdown/operators/understanding/olm/olm-understanding-olm#olm-csv_olm-understanding-olm) - [About live migration](/openshift-docs-markdown/virt/live_migration/virt-about-live-migration#virt-about-live-migration) - [Configure eviction and run strategies](/openshift-docs-markdown/virt/nodes/virt-eviction-strategies#virt-eviction-strategies) - [Configuring live migration limits and timeouts](/openshift-docs-markdown/virt/live_migration/virt-configuring-live-migration#virt-configuring-live-migration)