Enabling descheduler evictions on virtual machines¶

You can use the descheduler to evict pods so that the pods can be rescheduled onto more appropriate nodes. If the pod is a virtual machine, the pod eviction causes the virtual machine to be live migrated to another node.

Descheduler profiles¶

Use descheduler profiles to enable specific eviction strategies that rebalance your cluster based on criteria such as pod lifecycle or node utilization.

Use the KubeVirtRelieveAndMigrate profile to enable the descheduler on a virtual machine.

KubeVirtRelieveAndMigrate

The KubeVirtRelieveAndMigrate profile evicts pods from high-cost nodes to reduce overall resource expenses and enable workload migration. It also periodically rebalances workloads to help maintain similar spare capacity across nodes, which supports better handling of sudden workload spikes. Nodes can experience the following costs:

• Resource utilization: Increased resource pressure raises the overhead for running applications.
• Node maintenance: A higher number of containers on a node increases resource consumption and maintenance costs.

The profile enables the LowNodeUtilization strategy with the alpha-level EvictionsInBackground feature. By default, the profile uses the PrometheusCPUMemoryCombinedProfile utilization metric. This metric combines CPU and memory utilization with pressure stall information (PSI) for both dimensions for comprehensive node load balancing.

The profile also exposes the following customization fields:

• devActualUtilizationProfile: Enables load-aware descheduling. You can configure the following utilization profiles:
  • PrometheusCPUMemoryCombinedProfile (default): Balances nodes based on CPU utilization, CPU PSI pressure, memory utilization, and memory PSI pressure. This profile is ideal for environments with memory overcommit enabled, as it spreads the load and prevents resource contention.
  • PrometheusCPUCombined: Balances nodes based on CPU utilization and CPU PSI pressure only. Use this profile in environments without memory overcommit, where memory allocations are strictly guaranteed and CPU pressure is the primary driver for workload distribution.
• devLowNodeUtilizationThresholds: Sets experimental thresholds for the LowNodeUtilization strategy. Do not use this field with devDeviationThresholds.
• devDeviationThresholds: Treats nodes with below-average resource usage as underutilized to help redistribute workloads from overutilized nodes. Do not use this field with devLowNodeUtilizationThresholds. Supported values are: Low (10%:10%), Medium (20%:20%), High (30%:30%), AsymmetricLow (0%:10%), AsymmetricMedium (0%:20%), AsymmetricHigh (0%:30%).
• devEnableSoftTainter: Enables the soft-tainting component to dynamically apply or remove soft taints as scheduling hints.

Example configuration:

yaml

apiVersion: operator.openshift.io/v1
kind: KubeDescheduler
metadata:
  name: cluster
  namespace: openshift-kube-descheduler-operator
spec:
  managementState: Managed
  deschedulingIntervalSeconds: 30
  mode: "Automatic"
  profiles:
    - KubeVirtRelieveAndMigrate
  profileCustomizations:
    devEnableSoftTainter: true
    devDeviationThresholds: AsymmetricLow
    devActualUtilizationProfile: PrometheusCPUMemoryCombinedProfile

The KubeVirtRelieveAndMigrate profile requires PSI metrics to be enabled on all worker nodes. You can enable this by applying the following MachineConfig custom resource (CR):

Example MachineConfig CR:

yaml

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 99-openshift-machineconfig-worker-psi-karg
spec:
  kernelArguments:
    - psi=1

Note

The name of the MachineConfig object is significant because machine configs are processed in lexicographical order. By default, a config that starts with 98- disables PSI. To ensure that PSI is enabled, name your config with a higher prefix, such as 99-openshift-machineconfig-worker-psi-karg.

You can use this profile with the SoftTopologyAndDuplicates profile to also rebalance pods based on soft topology constraints, which can be useful in hosted control plane environments.

Installing the descheduler¶

The descheduler is not available by default. To enable the descheduler, you must install the Kube Descheduler Operator from the software catalog and enable one or more descheduler profiles.

By default, the descheduler runs in predictive mode, which means that it only simulates pod evictions. You must change the mode to automatic for the descheduler to perform the pod evictions.

Prerequisites

• You are logged in to OpenShift Container Platform as a user with the cluster-admin role.
• Access to the OpenShift Container Platform web console.

Procedure

1. Log in to the OpenShift Container Platform web console.
2. Create the required namespace for the Kube Descheduler Operator.
   1. Navigate to Administration → Namespaces and click Create Namespace.
   2. Enter openshift-kube-descheduler-operator in the Name field, enter openshift.io/cluster-monitoring=true in the Labels field to enable descheduler metrics, and click Create.
3. Install the Kube Descheduler Operator.
   1. Navigate to Ecosystem → Software Catalog.
   2. Type Kube Descheduler Operator into the filter box.
   3. Select the Kube Descheduler Operator and click Install.
   4. On the Install Operator page, select A specific namespace on the cluster. Select openshift-kube-descheduler-operator from the drop-down menu.
   5. Adjust the values for the Update Channel and Approval Strategy to the desired values.
   6. Click Install.
4. Create a descheduler instance.
   1. From the Ecosystem → Installed Operators page, click the Kube Descheduler Operator.
   2. Select the Kube Descheduler tab and click Create KubeDescheduler.
   3. Edit the settings as necessary.
      1. To evict pods instead of simulating the evictions, change the Mode field to Automatic.
      2. Expand the Profiles section and select KubeVirtRelieveAndMigrate. The AffinityAndTaints profile is enabled by default.
         Note

         You can also configure the profiles and settings for the descheduler later using the OpenShift CLI (oc).

Configuring descheduler evictions for virtual machines¶

After the descheduler is installed and configured, all migratable virtual machines (VMs) are eligible for eviction by default. You can configure the descheduler to manage VM evictions across the cluster and optionally exclude specific VMs from eviction.

Prerequisites

• Install the descheduler in the OpenShift Container Platform web console or OpenShift CLI (oc).

Procedure

1. Stop the VM.
2. Configure the KubeDescheduler object with the KubeVirtRelieveAndMigrate profile and enable background evictions for improved VM eviction stability during live migration:
   yaml

   apiVersion: operator.openshift.io/v1
   kind: KubeDescheduler
   metadata:
     name: cluster
     namespace: openshift-kube-descheduler-operator
   spec:
     deschedulingIntervalSeconds: 60
     profiles:
     - KubeVirtRelieveAndMigrate
     mode: Automatic

3. Optional: To evict pods, set the mode field value to Automatic. By default, the descheduler does not evict pods.
4. Optional: Configure limits for the number of parallel evictions to improve stability in large clusters.

   The descheduler can limit the number of concurrent evictions per node and across the cluster by using the evictionLimits field. Set these limits to match the migration limits configured in the HyperConverged custom resource (CR).

   yaml

   spec:
     evictionLimits:
       node: 2
       total: 5

   Set values that correspond to the migration limits in the HyperConverged CR:

   yaml

   spec:
     liveMigrationConfig:
       parallelMigrationsPerCluster: 5
       parallelOutboundMigrationsPerNode: 2

5. Optional: To exclude the VM from eviction, add the descheduler.alpha.kubernetes.io/prefer-no-eviction annotation to the spec.template.metadata.annotations field. The change is applied dynamically and is propagated to the VirtualMachineInstance (VMI) object and the virt-launcher pod.

   Only the presence of the annotation is checked. The value is not evaluated, so "true" and "false" have the same effect.

   yaml

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   spec:
     template:
       metadata:
         annotations:
           descheduler.alpha.kubernetes.io/prefer-no-eviction: "true"

6. Start the VM.

Result

The VM is now configured according to the descheduler settings.