---
title: Configuring a cross-cluster live migration network
---

# Configuring a cross-cluster live migration network {#virt-configuring-cross-cluster-live-migration-network}

Cross-cluster live migration requires that the clusters be connected in the same network. Specifically, `virt-handler` pods must be able to communicate.

## Configuration for a bridge secondary network {#nw-multus-bridge-object_virt-configuring-cross-cluster-live-migration-network}

The Bridge CNI plugin JSON configuration object describes the configuration parameters for the Bridge CNI plugin.

The following table details the configuration parameters:

<table>
<thead>
<tr>
  <th>Field</th>
  <th>Type</th>
  <th>Description</th>
</tr>
</thead>
<tbody>
<tr>
  <td><code>cniVersion</code></td>
  <td><code>string</code></td>
  <td>The CNI specification version. A minimum version of <code>0.3.1</code> is required.</td>
</tr>
<tr>
  <td><code>name</code></td>
  <td><code>string</code></td>
  <td>The mandatory, unique identifier assigned to this CNI network attachment definition. It is used by the container runtime to select the correct network configuration and serves as the key for persistent resource state management, such as IP address allocations.</td>
</tr>
<tr>
  <td><code>type</code></td>
  <td><code>string</code></td>
  <td>The name of the CNI plugin to configure: <code>bridge</code>.</td>
</tr>
<tr>
  <td><code>ipam</code></td>
  <td><code>object</code></td>
  <td>The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition.</td>
</tr>
<tr>
  <td><code>bridge</code></td>
  <td><code>string</code></td>
  <td>Optional: Specify the name of the virtual bridge to use. If the bridge interface does not exist on the host, the bridge interface gets created. The default value is <code>cni0</code>.</td>
</tr>
<tr>
  <td><code>ipMasq</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Set to <code>true</code> to enable IP masquerading for traffic that leaves the virtual network. The source IP address for all traffic is rewritten to the bridge's IP address. If the bridge does not have an IP address, this setting has no effect. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>disableContainerInterface</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Controls the container interface (<code>veth</code> peer inside the <code>netns</code> container). When set to <code>true</code>, the container interface link-state is set to <code>down</code>, you cannot use the IPAM CNI plugin. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>isGateway</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Set to <code>true</code> to assign an IP address to the bridge. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>isDefaultGateway</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Set to <code>true</code> to configure the bridge as the default gateway for the virtual network. The assigned IP address of the bridge is used as the default route. If <code>isDefaultGateway</code> is set to <code>true</code>, <code>isGateway</code> is also set to <code>true</code> automatically. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>forceAddress</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Set to <code>true</code> to allow assignment of a previously assigned IP address to the virtual bridge. When set to <code>false</code>, if an IPv4 address or an IPv6 address from overlapping subsets is assigned to the virtual bridge, an error occurs. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>hairpinMode</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Set to <code>true</code> to allow the virtual bridge to send an Ethernet frame back through the virtual port it was received on. This mode is also known as <em>reflective relay</em>. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>promiscMode</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Set to <code>true</code> to enable promiscuous mode on the bridge. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>vlan</code></td>
  <td><code>integer</code></td>
  <td>Optional: Specify a virtual LAN (VLAN) tag as an integer value. By default, no VLAN tag is assigned.</td>
</tr>
<tr>
  <td><code>preserveDefaultVlan</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Indicates whether the default VLAN must be preserved on the <code>veth</code> end connected to the bridge. Defaults to <code>false</code>.</td>
</tr>
<tr>
  <td><code>portIsolation</code></td>
  <td><code>boolean</code></td>
  <td>Optional: If <code>true</code>, prevents containers on the same bridge from communicating with each other. A container can still reach non-isolated ports. For example, a bridge interface that allows access to the host or an optional uplink that allows access outside the host. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>vlanTrunk</code></td>
  <td><code>list</code></td>
  <td>Optional: Assign a VLAN trunk tag. The default value is <code>none</code>.</td>
</tr>
<tr>
  <td><code>mtu</code></td>
  <td><code>integer</code></td>
  <td>Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.</td>
</tr>
<tr>
  <td><code>enabledad</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Enables duplicate address detection for the container side <code>veth</code>. The default value is <code>false</code>.</td>
</tr>
<tr>
  <td><code>macspoofchk</code></td>
  <td><code>boolean</code></td>
  <td>Optional: Enables mac spoof check, limiting the traffic originating from the container to the mac address of the interface. The default value is <code>false</code>.</td>
</tr>
</tbody>
</table>

> [!NOTE]
> The VLAN parameter configures the VLAN tag on the host end of the `veth` and also enables the `vlan_filtering` feature on the bridge interface.

> [!NOTE]
> To configure an uplink for an L2 network, you must allow the VLAN on the uplink interface by using the following command:
>
> ```terminal
> $  bridge vlan add vid VLAN_ID dev DEV
> ```

### Bridge CNI plugin configuration example {#nw-multus-bridge-config-example_virt-configuring-cross-cluster-live-migration-network}

The following example configures a secondary network named `bridge-net`:

```json
{
  "cniVersion": "0.3.1",
  "name": "bridge-net",
  "type": "bridge",
  "isGateway": true,
  "vlan": 2,
  "ipam": {
    "type": "dhcp"
    }
}
```

## Configuring a dedicated secondary network for live migration {#virt-configuring-secondary-network-vm-live-migration_virt-configuring-cross-cluster-live-migration-network}

After you have configured a Linux bridge network, you can configure a dedicated network for live migration. A dedicated network minimizes the effects of network saturation on tenant workloads during live migration.

To configure a dedicated secondary network for live migration, you must first create a bridge network attachment definition (NAD) by using the CLI. You can then add the name of the `NetworkAttachmentDefinition` object to the `HyperConverged` custom resource (CR).

**Prerequisites**

- You installed the OpenShift CLI (`oc`).
- You logged in to the cluster as a user with the `cluster-admin` role.
- Each node has at least two Network Interface Cards (NICs).
- The NICs for live migration are connected to the same VLAN.

**Procedure**

1. Create a `NetworkAttachmentDefinition` manifest according to the following example:

   ```yaml
   apiVersion: "k8s.cni.cncf.io/v1"
   kind: NetworkAttachmentDefinition
   metadata:
     name: my-secondary-network
     namespace: openshift-cnv
   spec:
     config: '{
       "cniVersion": "0.3.1",
       "name": "migration-bridge",
       "type": "macvlan",
       "master": "eth1",
       "mode": "bridge",
       "ipam": {
         "type": "whereabouts",
         "range": "10.200.5.0/24"
       }
     }'
   ```

   - `metadata.name` defines the name of the `NetworkAttachmentDefinition` object.
   - `config.master` defines the name of the NIC to be used for live migration.
   - `config.type` defines the name of the CNI plugin that provides the network for the NAD.
   - `config.range` defines an IP address range for the secondary network. This range must not overlap the IP addresses of the main network.
2. Open the `HyperConverged` CR in your default editor by running the following command:

   ```terminal
   $ oc edit hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv
   ```
3. Add the name of the `NetworkAttachmentDefinition` object to the `spec.liveMigrationConfig` stanza of the `HyperConverged` CR.

   Example `HyperConverged` manifest:

   ```yaml
   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
     name: kubevirt-hyperconverged
     namespace: openshift-cnv
   spec:
     liveMigrationConfig:
       completionTimeoutPerGiB: 800
       network: <network>
       parallelMigrationsPerCluster: 5
       parallelOutboundMigrationsPerNode: 2
       progressTimeout: 150
   # ...
   ```

   - `spec.liveMigrationConfig.network` defines the name of the Multus `NetworkAttachmentDefinition` object to be used for live migrations.
4. Save your changes and exit the editor. The `virt-handler` pods restart and connect to the secondary network.

**Verification**

- When the node that the virtual machine runs on is placed into maintenance mode, the VM automatically migrates to another node in the cluster. You can verify that the migration occurred over the secondary network and not the default pod network by checking the target IP address in the virtual machine instance (VMI) metadata.

  ```terminal
  $ oc get vmi <vmi_name> -o jsonpath='{.status.migrationState.targetNodeAddress}'
  ```