--- title: Managing hosted control planes on RHOSP --- # Managing hosted control planes on RHOSP {#hcp-manage-openstack} After you deploy hosted control planes on Red Hat OpenStack Platform (RHOSP) agent machines, you can manage a hosted cluster. ## Accessing the hosted cluster {#hcp-openstack-accessing_hcp-manage-openstack} You can access hosted clusters on Red Hat OpenStack Platform (RHOSP) by extracting the kubeconfig secret directly from resources by using the `oc` CLI. The *hosted cluster (hosting)* namespace has hosted cluster resources and the access secrets. The *hosted control plane* namespace is where the hosted control plane runs. The secret name formats are as follows: - `kubeconfig` secret: `--admin-kubeconfig`. For example, `clusters-hypershift-demo-admin-kubeconfig`. - `kubeadmin` password secret: `--kubeadmin-password`. For example, `clusters-hypershift-demo-kubeadmin-password`. The `kubeconfig` secret has a Base64-encoded `kubeconfig` field. The `kubeadmin` password secret is also Base64-encoded; you can extract it and then use the password to log in to the API server or console of the hosted cluster. **Prerequisites** - The `oc` CLI is installed. **Procedure** 1. Extract the `admin-kubeconfig` secret by entering the following command: ```terminal $ oc extract -n \ secret/-admin-kubeconfig \ --to=./hostedcluster-secrets --confirm ``` ```text {title="Example output"} hostedcluster-secrets/kubeconfig ``` 2. View a list of nodes of the hosted cluster to verify your access by entering the following command: ```terminal $ oc --kubeconfig ./hostedcluster-secrets/kubeconfig get nodes ``` ## Enabling node auto-scaling for the hosted cluster {#hcp-openstack-autoscale_hcp-manage-openstack} When you need more capacity in your hosted cluster on Red Hat OpenStack Platform (RHOSP) and spare agents are available, you can enable auto-scaling to install new worker nodes. **Procedure** 1. To enable auto-scaling, enter the following command: ```terminal $ oc -n patch nodepool \ --type=json \ -p '[{"op": "remove", "path": "/spec/replicas"},{"op":"add", "path": "/spec/autoScaling", "value": { "max": 5, "min": 2 }}]' ``` 2. Create a workload that requires a new node. 1. Create a YAML file that has the workload configuration, by using the following example: ```yaml apiVersion: apps/v1 kind: Deployment metadata: labels: app: reversewords name: reversewords namespace: default spec: replicas: 40 selector: matchLabels: app: reversewords template: metadata: labels: app: reversewords spec: containers: - image: quay.io/mavazque/reversewords:latest name: reversewords resources: requests: memory: 2Gi ``` 2. Save the file with the name `workload-config.yaml`. 3. Apply the YAML by entering the following command: ```terminal $ oc apply -f workload-config.yaml ``` 3. Extract the `admin-kubeconfig` secret by entering the following command: ```terminal $ oc extract -n \ secret/-admin-kubeconfig \ --to=./hostedcluster-secrets --confirm ``` ```text {title="Example output"} hostedcluster-secrets/kubeconfig ``` 4. You can check if new nodes are in the `Ready` status by entering the following command: ```terminal $ oc --kubeconfig ./hostedcluster-secrets get nodes ``` 5. To remove the node, delete the workload by entering the following command: ```terminal $ oc --kubeconfig ./hostedcluster-secrets -n \ delete deployment ``` 6. Wait for several minutes to pass without requiring the additional capacity. You can confirm that the node was removed by entering the following command: ```terminal $ oc --kubeconfig ./hostedcluster-secrets get nodes ``` ## Configuring node pools for availability zones {#hcp-manage-openstack-az_hcp-manage-openstack} To improve the high availability of your hosted cluster, you can distribute node pools across multiple Red Hat OpenStack Platform (RHOSP) Nova availability zones. > [!NOTE] > Availability zones do not necessarily correspond to fault domains and do not inherently provide high availability benefits. **Prerequisites** - You created a hosted cluster. - You have access to the management cluster. - The `hcp` and `oc` command-line interfaces (CLIs) are installed. **Procedure** 1. Set environment variables that are appropriate for your needs. For example, if you want to create two additional machines in the `az1` availability zone, you could enter: ```terminal $ export NODEPOOL_NAME="${CLUSTER_NAME}-az1" \ && export WORKER_COUNT="2" \ && export FLAVOR="m1.xlarge" \ && export AZ="az1" ``` 2. Create the node pool by using your environment variables by entering the following command: ```terminal $ hcp create nodepool openstack \ --cluster-name \ --name $NODEPOOL_NAME \ --replicas $WORKER_COUNT \ --openstack-node-flavor $FLAVOR \ --openstack-node-availability-zone $AZ ``` where: `` : Specifies the name of your hosted cluster. 3. Check the status of the node pool by listing `nodepool` resources in the clusters namespace by running the following command: ```terminal $ oc get nodepools --namespace clusters ``` ```terminal {title="Example output"} NAME CLUSTER DESIRED NODES CURRENT NODES AUTOSCALING AUTOREPAIR VERSION UPDATINGVERSION UPDATINGCONFIG MESSAGE example example 5 5 False False 4.17.0 example-az1 example 2 False False True True Minimum availability requires 2 replicas, current 0 available ``` 4. Observe the notes as they start on your hosted cluster by running the following command: ```terminal $ oc --kubeconfig $CLUSTER_NAME-kubeconfig get nodes ``` ```terminal {title="Example output"} NAME STATUS ROLES AGE VERSION ... example-extra-az-zh9l5 Ready worker 2m6s v1.27.4+18eadca example-extra-az-zr8mj Ready worker 102s v1.27.4+18eadca ... ``` 5. Verify that the node pool is created by running the following command: ```terminal $ oc get nodepools --namespace clusters ``` ```terminal {title="Example output"} NAME CLUSTER DESIRED CURRENT AVAILABLE PROGRESSING MESSAGE 2 2 2 False All replicas are available ``` ## Additional ports for node pools {#hosted-clusters-openstack-addl-ports-cases_hcp-manage-openstack} You can configure additional ports for node pools to support advanced networking scenarios, such as SR-IOV or multiple networks. Common reasons to configure additional ports for node pools include the following: SR-IOV (Single Root I/O Virtualization) : Enables a physical network device to appear as multiple virtual functions (VFs). By attaching additional ports to node pools, workloads can use SR-IOV interfaces to achieve low-latency, high-performance networking. DPDK (Data Plane Development Kit) : Provides fast packet processing in user space, bypassing the kernel. Node pools with additional ports can expose interfaces for workloads that use DPDK to improve network performance. Manila RWX volumes on NFS : Supports `ReadWriteMany` (RWX) volumes over NFS, allowing multiple nodes to access shared storage. Attaching additional ports to node pools enables workloads to reach the NFS network used by Manila. Multus CNI : Enables pods to connect to multiple network interfaces. Node pools with additional ports support use cases that require secondary network interfaces, including dual-stack connectivity and traffic separation. ### Options for additional ports for node pools {#hosted-clusters-openstack-addl-ports-options_hcp-manage-openstack} To support advanced networking scenarios, you can use the `--openstack-node-additional-port` flag to attach additional ports to nodes in a hosted cluster on OpenStack. The flag takes a list of comma-separated parameters that can be used multiple times to attach multiple additional ports to the nodes. The parameters are as follows: | Parameter | Description | Required | Default | | --- | --- | --- | --- | | `network-id` | The ID of the network to attach to the node. | Yes | N/A | | `vnic-type` | The vNIC type to use for the port. If not specified, Neutron uses the default type `normal`. | No | N/A | | `disable-port-security` | Whether to disable port security for the port. If not specified, Neutron enables port security unless it is explicitly disabled at the network level. | No | N/A | | `address-pairs` | A list of IP address pairs to assign to the port. The format is `ip_address=mac_address`. Multiple pairs can be provided, separated by a hyphen (`-`). The `mac_address` portion is optional. | No | N/A | ### Creating additional ports for node pools {#hosted-clusters-openstack-addl-ports_hcp-manage-openstack} You can configure additional ports for node pools for hosted clusters that run on Red Hat OpenStack Platform (RHOSP). **Prerequisites** - You created a hosted cluster. - You have access to the management cluster. - The `hcp` CLI is installed. - Additional networks are created in RHOSP. - The project that is used by the hosted cluster must have access to the additional networks. - You reviewed the options that are listed in "Options for additional ports for node pools". **Procedure** - Create a hosted cluster with additional ports attached to it by running the `hcp create nodepool openstack` command with the `--openstack-node-additional-port` options. For example: ```terminal $ hcp create nodepool openstack \ --cluster-name \ --name \ --replicas \ --openstack-node-flavor \ --openstack-node-additional-port "network-id=,vnic-type=direct,disable-port-security=true" \ --openstack-node-additional-port "network-id=,address-pairs:192.168.0.1-192.168.0.2" ``` where: `` : Specifies the name of the hosted cluster. `` : Specifies the name of the node pool. `` : Specifies the number of replicas that you need. `` : Specifies the RHOSP flavor to use. `` : Specifies a SR-IOV network ID. `` : Specifies a load balancer network ID. ## Node tuning for hosted clusters on RHOSP {#hosted-clusters-openstack-performance_hcp-manage-openstack} You can tune hosted cluster node performance on RHOSP for high-performance workloads, such as cloud-native network functions (CNFs). Performance tuning includes configuring RHOSP resources, creating a performance profile, deploying a tuned `NodePool` resource, and enabling SR-IOV device support. CNFs are designed to run in cloud-native environments. They can provide network services such as routing, firewalling, and load balancing. You can configure the node pool to use high-performance computing and networking devices to run CNFs. ### Tuning performance for hosted cluster nodes {#hosted-clusters-openstack-performance-tuning_hcp-manage-openstack} To run high-performance workloads on hosted control planes on Red Hat OpenStack Platform (RHOSP), you can create a performance profile and deploy a tuned `NodePool` resource. **Prerequisites** - You have RHOSP flavor that has the necessary resources to run your workload, including dedicated CPU, memory, and host aggregate information. - You have an RHOSP network that is attached to SR-IOV or DPDK-capable NICs. The network must be available to the project used by hosted clusters. **Procedure** 1. Create a performance profile that meets your requirements in a file called `perfprofile.yaml`. For example: ```yaml {title="Example performance profile in a config map"} apiVersion: v1 kind: ConfigMap metadata: name: perfprof-1 namespace: clusters data: tuning: | apiVersion: v1 kind: ConfigMap metadata: name: cnf-performanceprofile namespace: "${HYPERSHIFT_NAMESPACE}" data: tuning: | apiVersion: performance.openshift.io/v2 kind: PerformanceProfile metadata: name: cnf-performanceprofile spec: additionalKernelArgs: - nmi_watchdog=0 - audit=0 - mce=off - processor.max_cstate=1 - idle=poll - intel_idle.max_cstate=0 - amd_iommu=on cpu: isolated: "${CPU_ISOLATED}" reserved: "${CPU_RESERVED}" hugepages: defaultHugepagesSize: "1G" pages: - count: ${HUGEPAGES} node: 0 size: 1G nodeSelector: node-role.kubernetes.io/worker: '' realTimeKernel: enabled: false globallyDisableIrqLoadBalancing: true ``` > [!IMPORTANT] > If you do not already have environment variables set for the HyperShift Operator namespace, isolated and reserved CPUs, and huge pages count, create them before applying the performance profile. 2. Apply the performance profile configuration by running the following command: ```terminal $ oc apply -f perfprof.yaml ``` 3. If you do not already have a `CLUSTER_NAME` environment variable set for the name of your cluster, define it. 4. Set a node pool name environment variable by running the following command: ```terminal $ export NODEPOOL_NAME=$CLUSTER_NAME-cnf ``` 5. Set a flavor environment variable by running the following command: ```terminal $ export FLAVOR="m1.xlarge.nfv" ``` 6. Create a node pool that uses the performance profile by running the following command: ```terminal $ hcp create nodepool openstack \ --cluster-name $CLUSTER_NAME \ --name $NODEPOOL_NAME \ --node-count 0 \ --openstack-node-flavor $FLAVOR ``` 7. Patch the node pool to reference the `PerformanceProfile` resource by running the following command: ```terminal $ oc patch nodepool -n ${HYPERSHIFT_NAMESPACE} ${CLUSTER_NAME} \ -p '{"spec":{"tuningConfig":[{"name":"cnf-performanceprofile"}]}}' --type=merge ``` 8. Scale the node pool by running the following command: ```terminal $ oc scale nodepool/$CLUSTER_NAME --namespace ${HYPERSHIFT_NAMESPACE} --replicas=1 ``` 9. Wait for the nodes to be ready: 1. Wait for the nodes to be ready by running the following command: ```terminal $ oc wait --for=condition=UpdatingConfig=True nodepool \ -n ${HYPERSHIFT_NAMESPACE} ${CLUSTER_NAME} \ --timeout=5m ``` 2. Wait for the configuration update to finish by running the following command: ```terminal $ oc wait --for=condition=UpdatingConfig=False nodepool \ -n ${HYPERSHIFT_NAMESPACE} ${CLUSTER_NAME} \ --timeout=30m ``` 3. Wait until all nodes are healthy by running the following command: ```terminal $ oc wait --for=condition=AllNodesHealthy nodepool \ -n ${HYPERSHIFT_NAMESPACE} ${CLUSTER_NAME} \ --timeout=5m ``` > [!NOTE] > You can make an SSH connection into the nodes or use the `oc debug` command to verify performance configurations. ### Enabling the SR-IOV Network Operator in a hosted cluster {#hosted-clusters-openstack-performance-enabling_hcp-manage-openstack} To manage the SR-IOV-capable devices on nodes deployed by the `NodePool` resource, you can enable the SR-IOV Network Operator. The Operator runs in the hosted cluster and requires labeled worker nodes. **Procedure** 1. Generate a `kubeconfig` file for the hosted cluster by running the following command: ```terminal $ hcp create kubeconfig --name $CLUSTER_NAME > $CLUSTER_NAME-kubeconfig ``` 2. Create a `kubeconfig` resource environment variable by running the following command: ```terminal $ export KUBECONFIG=$CLUSTER_NAME-kubeconfig ``` 3. Label each worker node to indicate SR-IOV capability by running the following command: ```terminal $ oc label node feature.node.kubernetes.io/network-sriov.capable=true ``` where: `` : Specifies the name of a worker node in the hosted cluster. 4. Install the SR-IOV Network Operator in the hosted cluster by following the instructions in "Installing the SR-IOV Network Operator". 5. After installation, configure SR-IOV workloads in the hosted cluster by using the same process as for a standalone cluster. **Additional resources** {._additional-resources} - [Installing the SR-IOV Network Operator](/openshift-docs-markdown/networking/networking_operators/sr-iov-operator/installing-sriov-operator#installing-sr-iov-operator_installing-sriov-operator)