--- title: Deploying hosted control planes on bare metal in a disconnected environment --- # Deploying hosted control planes on bare metal in a disconnected environment {#hcp-deploy-dc-bm} In the context of hosted control planes, a disconnected environment is an OpenShift Container Platform deployment that is not connected to the internet and that uses hosted control planes as a base. You can deploy hosted control planes in a disconnected environment on bare metal. When you provision hosted control planes on bare metal, you use the Agent platform. The Agent platform and multicluster engine for Kubernetes Operator work together to enable disconnected deployments. The Agent platform uses the central infrastructure management service to add worker nodes to a hosted cluster. For an introduction to the central infrastructure management service, see "Enabling the central infrastructure management service". ## Disconnected environment architecture for bare metal {#hcp-dc-bm-arch_hcp-deploy-dc-bm} Get familiar with the architecture for a deployment of hosted control planes on bare metal in a disconnected environment. The following diagram illustrates an example architecture of a disconnected environment: ![Disconnected architecture diagram](/openshift-docs-markdown/images/489_RHACM_HyperShift_on_bare_metal_1223.png) 1. Configure infrastructure services, including the registry certificate deployment with TLS support, web server, and DNS, to ensure that the disconnected deployment works. 2. Create a config map in the `openshift-config` namespace. In this example, the config map is named `registry-config`. The content of the config map is the Registry CA certificate. The data field of the config map must contain the following key/value: - Key: `..`, for example, `registry.hypershiftdomain.lab..5000:`. Ensure that you place `..` after the registry DNS domain name when you specify a port. - Value: The certificate content For more information about creating a config map, see "Adding the registry CA to the management cluster" and "Adding the registry CA to the compute nodes for the hosted cluster". 3. Modify the `images.config.openshift.io` custom resource (CR) specification and adds a new field named `additionalTrustedCA` with a value of `name: registry-config`. 4. Create a config map that contains two data fields. One field contains the `registries.conf` file in `RAW` format, and the other field contains the Registry CA and is named `ca-bundle.crt`. The config map belongs to the `multicluster-engine` namespace, and the config map name is referenced in other objects. For an example of a config map, see the following sample configuration: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: custom-registries namespace: multicluster-engine labels: app: assisted-service data: ca-bundle.crt: | -----BEGIN CERTIFICATE----- # ... -----END CERTIFICATE----- registries.conf: | unqualified-search-registries = ["registry.access.redhat.com", "docker.io"] [[registry]] prefix = "" location = "registry.redhat.io/openshift4" mirror-by-digest-only = true [[registry.mirror]] location = "registry.ocp-edge-cluster-0.qe.lab.redhat.com:5000/openshift4" [[registry]] prefix = "" location = "registry.redhat.io/rhacm2" mirror-by-digest-only = true # ... # ... ``` 5. In the multicluster engine Operator namespace, you create the `multiclusterengine` CR, which enables both the Agent and `hypershift-addon` add-ons. The multicluster engine Operator namespace must contain the config maps to modify behavior in a disconnected deployment. The namespace also contains the `multicluster-engine`, `assisted-service`, and `hypershift-addon-manager` pods. 6. Create the following objects that are necessary to deploy the hosted cluster: - Secrets: Secrets contain the pull secret, SSH key, and etcd encryption key. - Config map: The config map contains the CA certificate of the private registry. - `HostedCluster`: The `HostedCluster` resource defines the configuration of the cluster that the user intends to create. - `NodePool`: The `NodePool` resource identifies the node pool that references the machines to use for the data plane. 7. After you create the hosted cluster objects, the HyperShift Operator establishes the `HostedControlPlane` namespace to accommodate control plane pods. The namespace also hosts components such as Agents, bare metal hosts (BMHs), and the `InfraEnv` resource. Later, you create the `InfraEnv` resource, and after ISO creation, you create the BMHs and their secrets that contain baseboard management controller (BMC) credentials. 8. The Metal3 Operator in the `openshift-machine-api` namespace inspects the new BMHs. Then, the Metal3 Operator tries to connect to the BMCs to start them by using the configured `LiveISO` and `RootFS` values that are specified through the `AgentServiceConfig` CR in the multicluster engine Operator namespace. 9. After the worker nodes of the `HostedCluster` resource are started, an Agent container is started. This agent establishes contact with the Assisted Service, which orchestrates the actions to complete the deployment. Initially, you need to scale the `NodePool` resource to the number of worker nodes for the `HostedCluster` resource. The Assisted Service manages the remaining tasks. 10. At this point, you wait for the deployment process to be completed. **Additional resources** {._additional-resources} - [Adding the registry CA to the management cluster](/openshift-docs-markdown/hosted_control_planes/hcp-disconnected/hcp-deploy-dc-bm#hcp-dc-tls-mgmt_hcp-deploy-dc-bm) - [Adding the registry CA to the compute nodes for the hosted cluster](/openshift-docs-markdown/hosted_control_planes/hcp-disconnected/hcp-deploy-dc-bm#hcp-dc-tls-hosted_hcp-deploy-dc-bm) ## Requirements to deploy hosted control planes on bare metal in a disconnected environment {#hcp-dc-bm-reqs_hcp-deploy-dc-bm} To configure hosted control planes in a disconnected environment, you must meet several prerequisites. - CPU: The number of CPUs provided determines how many hosted clusters can run concurrently. In general, use 16 CPUs for each node for 3 nodes. For minimal development, you can use 12 CPUs for each node for 3 nodes. - Memory: The amount of RAM affects how many hosted clusters can be hosted. Use 48 GB of RAM for each node. For minimal development, 18 GB of RAM might be sufficient. - Storage: Use SSD storage for multicluster engine Operator. - Management cluster: 250 GB. - Registry: The storage needed depends on the number of releases, operators, and images that are hosted. An acceptable number might be 500 GB, preferably separated from the disk that hosts the hosted cluster. - Web server: The storage needed depends on the number of ISOs and images that are hosted. An acceptable number might be 500 GB. - Production: For a production environment, separate the management cluster, the registry, and the web server on different disks. This example illustrates a possible configuration for production: - Registry: 2 TB - Management cluster: 500 GB - Web server: 2 TB ## Extracting the release image digest {#hcp-dc-extract_hcp-deploy-dc-bm} To deploy hosted control planes on bare metal in a disconnected environment, you need the OpenShift Container Platform release image. You can extract the release image digest by using the tagged image. **Procedure** - Obtain the image digest by running the following command: ```terminal $ oc adm release info | grep "Pull From" ``` Replace `` with the tagged image for the supported OpenShift Container Platform version, for example, `quay.io/openshift-release-dev/ocp-release:4.14.0-x8_64`. ```terminal {title="Example output"} Pull From: quay.io/openshift-release-dev/ocp-release@sha256:69d1292f64a2b67227c5592c1a7d499c7d00376e498634ff8e1946bc9ccdddfe ``` ## DNS configurations on bare metal {#hcp-bm-dns_hcp-deploy-dc-bm} The API Server for the hosted cluster is exposed as a `NodePort` service. A DNS entry must exist for `api..` that points to destination where the API Server can be reached. The DNS entry can be as simple as a record that points to one of the nodes in the management cluster that is running the hosted control plane. The entry can also point to a load balancer that is deployed to redirect incoming traffic to the ingress pods. ```terminal {title="Example DNS configuration"} api.example.krnl.es. IN A 192.168.122.20 api.example.krnl.es. IN A 192.168.122.21 api.example.krnl.es. IN A 192.168.122.22 api-int.example.krnl.es. IN A 192.168.122.20 api-int.example.krnl.es. IN A 192.168.122.21 api-int.example.krnl.es. IN A 192.168.122.22 `*`.apps.example.krnl.es. IN A 192.168.122.23 ``` > [!NOTE] > In the previous example, `*.apps.example.krnl.es. IN A 192.168.122.23` is either a node in the hosted cluster or a load balancer, if one has been configured. If you are configuring DNS for a disconnected environment on an IPv6 network, the configuration looks like the following example. ```terminal {title="Example DNS configuration for an IPv6 network"} api.example.krnl.es. IN A 2620:52:0:1306::5 api.example.krnl.es. IN A 2620:52:0:1306::6 api.example.krnl.es. IN A 2620:52:0:1306::7 api-int.example.krnl.es. IN A 2620:52:0:1306::5 api-int.example.krnl.es. IN A 2620:52:0:1306::6 api-int.example.krnl.es. IN A 2620:52:0:1306::7 `*`.apps.example.krnl.es. IN A 2620:52:0:1306::10 ``` If you are configuring DNS for a disconnected environment on a dual stack network, be sure to include DNS entries for both IPv4 and IPv6. ```terminal {title="Example DNS configuration for a dual stack network"} host-record=api-int.hub-dual.dns.base.domain.name,192.168.126.10 host-record=api.hub-dual.dns.base.domain.name,192.168.126.10 address=/apps.hub-dual.dns.base.domain.name/192.168.126.11 dhcp-host=aa:aa:aa:aa:10:01,ocp-control-plane-0,192.168.126.20 dhcp-host=aa:aa:aa:aa:10:02,ocp-control-plane-1,192.168.126.21 dhcp-host=aa:aa:aa:aa:10:03,ocp-control-plane-2,192.168.126.22 dhcp-host=aa:aa:aa:aa:10:06,ocp-installer,192.168.126.25 dhcp-host=aa:aa:aa:aa:10:07,ocp-bootstrap,192.168.126.26 host-record=api-int.hub-dual.dns.base.domain.name,2620:52:0:1306::2 host-record=api.hub-dual.dns.base.domain.name,2620:52:0:1306::2 address=/apps.hub-dual.dns.base.domain.name/2620:52:0:1306::3 dhcp-host=aa:aa:aa:aa:10:01,ocp-control-plane-0,[2620:52:0:1306::5] dhcp-host=aa:aa:aa:aa:10:02,ocp-control-plane-1,[2620:52:0:1306::6] dhcp-host=aa:aa:aa:aa:10:03,ocp-control-plane-2,[2620:52:0:1306::7] dhcp-host=aa:aa:aa:aa:10:06,ocp-installer,[2620:52:0:1306::8] dhcp-host=aa:aa:aa:aa:10:07,ocp-bootstrap,[2620:52:0:1306::9] ``` ## Deploying a registry for hosted control planes in a disconnected environment {#hcp-dc-registry_hcp-deploy-dc-bm} For development environments, deploy a small, self-hosted registry by using a Podman container. For production environments, deploy an enterprise-hosted registry, such as Red Hat Quay, Nexus, or Artifactory. To deploy a small registry by using Podman, complete the following steps. **Procedure** 1. As a privileged user, access the `${HOME}` directory and create the following script: ```bash #!/usr/bin/env bash set -euo pipefail PRIMARY_NIC=$(ls -1 /sys/class/net | grep -v podman | head -1) export PATH=/root/bin:$PATH export PULL_SECRET="/root/baremetal/hub/openshift_pull.json" if [[ ! -f $PULL_SECRET ]];then echo "Pull Secret not found, exiting..." exit 1 fi dnf -y install podman httpd httpd-tools jq skopeo libseccomp-devel export IP=$(ip -o addr show $PRIMARY_NIC | head -1 | awk '{print $4}' | cut -d'/' -f1) REGISTRY_NAME=registry.$(hostname --long) REGISTRY_USER=dummy REGISTRY_PASSWORD=dummy KEY=$(echo -n $REGISTRY_USER:$REGISTRY_PASSWORD | base64) echo "{\"auths\": {\"$REGISTRY_NAME:5000\": {\"auth\": \"$KEY\", \"email\": \"sample-email@domain.ltd\"}}}" > /root/disconnected_pull.json mv ${PULL_SECRET} /root/openshift_pull.json.old jq ".auths += {\"$REGISTRY_NAME:5000\": {\"auth\": \"$KEY\",\"email\": \"sample-email@domain.ltd\"}}" < /root/openshift_pull.json.old > $PULL_SECRET mkdir -p /opt/registry/{auth,certs,data,conf} cat < /opt/registry/conf/config.yml version: 0.1 log: fields: service: registry storage: cache: blobdescriptor: inmemory filesystem: rootdirectory: /var/lib/registry delete: enabled: true http: addr: :5000 headers: X-Content-Type-Options: [nosniff] health: storagedriver: enabled: true interval: 10s threshold: 3 compatibility: schema1: enabled: true EOF openssl req -newkey rsa:4096 -nodes -sha256 -keyout /opt/registry/certs/domain.key -x509 -days 3650 -out /opt/registry/certs/domain.crt -subj "/C=US/ST=Madrid/L=San Bernardo/O=Karmalabs/OU=Guitar/CN=$REGISTRY_NAME" -addext "subjectAltName=DNS:$REGISTRY_NAME" cp /opt/registry/certs/domain.crt /etc/pki/ca-trust/source/anchors/ update-ca-trust extract htpasswd -bBc /opt/registry/auth/htpasswd $REGISTRY_USER $REGISTRY_PASSWORD podman create --name registry --net host --security-opt label=disable --replace -v /opt/registry/data:/var/lib/registry:z -v /opt/registry/auth:/auth:z -v /opt/registry/conf/config.yml:/etc/docker/registry/config.yml -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry" -e "REGISTRY_HTTP_SECRET=ALongRandomSecretForRegistry" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd -v /opt/registry/certs:/certs:z -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key docker.io/library/registry:latest [ "$?" == "0" ] || !! systemctl enable --now registry ``` Replace the location of the `PULL_SECRET` with the appropriate location for your setup. 2. Name the script file `registry.sh` and save it. When you run the script, it pulls in the following information: - The registry name, based on the hypervisor hostname - The necessary credentials and user access details 3. Adjust permissions by adding the execution flag as follows: ```terminal $ chmod u+x ${HOME}/registry.sh ``` 4. To run the script without any parameters, enter the following command: ```terminal $ ${HOME}/registry.sh ``` The script starts the server. The script uses a `systemd` service for management purposes. 5. If you need to manage the script, you can use the following commands. 1. To view the status, enter the following command: ```terminal $ systemctl status ``` 2. To start the script, enter the following command: ```terminal $ systemctl start ``` 3. To stop the script, enter the following command: ```terminal $ systemctl stop ``` The root folder for the registry is in the `/opt/registry` directory and contains the following subdirectories: - `certs` contains the TLS certificates. - `auth` contains the credentials. - `data` contains the registry images. - `conf` contains the registry configuration. ## Setting up a management cluster for hosted control planes in a disconnected environment {#hcp-dc-mgmt-cluster_hcp-deploy-dc-bm} An important part of a hosted control planes deployment is the OpenShift Container Platform management cluster. To set up an management cluster for a disconnected environment, you must install multicluster engine for Kubernetes Operator on it. The multicluster engine Operator plays a crucial role in deploying clusters across providers. **Prerequisites** - There must be bidirectional connectivity between the management cluster and the Baseboard Management Controller (BMC) of the target Bare Metal Host (BMH). As an alternative, you follow a Boot It Yourself approach through the Agent provider. - The hosted cluster must be able to resolve and reach the API hostname of the management cluster hostname and `*.apps` hostname. Here is an example of the API hostname of the management cluster and `*.apps` hostname: - `api.management-cluster.internal.domain.com` - `console-openshift-console.apps.management-cluster.internal.domain.com` - The management cluster must be able to resolve and reach the API and `*.apps` hostname of the hosted cluster. Here is an example of the API hostname of the hosted cluster and `*.apps` hostname: - `api.sno-hosted-cluster-1.internal.domain.com` - `console-openshift-console.apps.sno-hosted-cluster-1.internal.domain.com` **Procedure** 1. Install multicluster engine Operator 2.7 or later on an OpenShift Container Platform cluster. You can install multicluster engine Operator as an Operator from the OpenShift Container Platform software catalog. The HyperShift Operator is included with multicluster engine Operator. For more information about installing multicluster engine Operator, see "Installing and upgrading multicluster engine operator" in the Red Hat Advanced Cluster Management documentation. 2. Ensure that the HyperShift Operator is installed. The HyperShift Operator is automatically included with multicluster engine Operator, but if you need to manually install it, follow the steps in "Manually enabling the hypershift-addon managed cluster add-on for local-cluster". **Next steps** Next, configure the web server. **Additional resources** {._additional-resources} - [Installing and upgrading multicluster engine operator](https://docs.redhat.com/en/documentation/red_hat_advanced_cluster_management_for_kubernetes/latest/html/clusters/cluster_mce_overview#mce-install-intro) - [Manually enabling the hypershift-addon managed cluster add-on for local-cluster](/openshift-docs-markdown/hosted_control_planes/hcp-prepare/hcp-enable-disable#hcp-enable-manual-addon_hcp-enable-disable) - [Cluster lifecycle with multicluster engine operator overview](https://docs.redhat.com/en/documentation/red_hat_advanced_cluster_management_for_kubernetes/latest/html/clusters/cluster_mce_overview#mce-intro) ## Configuring the web server for hosted control planes in a disconnected environment {#hcp-dc-web-server_hcp-deploy-dc-bm} You must configure an additional web server to host the Red Hat Enterprise Linux CoreOS (RHCOS) images that are associated with the OpenShift Container Platform release that you are deploying as a hosted cluster. **Procedure** 1. Extract the `openshift-install` binary from the OpenShift Container Platform release that you want to use by entering the following command: ```terminal $ oc adm -a ${LOCAL_SECRET_JSON} release extract --command=openshift-install \ "${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}" ``` 2. Run the following script. The script creates a folder in the `/opt/srv` directory. The folder contains the RHCOS images to provision the worker nodes. ```bash #!/bin/bash WEBSRV_FOLDER=/opt/srv ROOTFS_IMG_URL="$(./openshift-install coreos print-stream-json | jq -r '.architectures.x86_64.artifacts.metal.formats.pxe.rootfs.location')" LIVE_ISO_URL="$(./openshift-install coreos print-stream-json | jq -r '.architectures.x86_64.artifacts.metal.formats.iso.disk.location')" mkdir -p ${WEBSRV_FOLDER}/images curl -Lk ${ROOTFS_IMG_URL} -o ${WEBSRV_FOLDER}/images/${ROOTFS_IMG_URL##*/} curl -Lk ${LIVE_ISO_URL} -o ${WEBSRV_FOLDER}/images/${LIVE_ISO_URL##*/} chmod -R 755 ${WEBSRV_FOLDER}/* ## Run Webserver podman ps --noheading | grep -q websrv-ai if [[ $? == 0 ]];then echo "Launching Registry pod..." /usr/bin/podman run --name websrv-ai --net host -v /opt/srv:/usr/local/apache2/htdocs:z quay.io/alosadag/httpd:p8080 fi ``` - You can find the `ROOTFS_IMG_URL` value on the OpenShift CI Release page. - You can find the `LIVE_ISO_URL` value on the OpenShift CI Release page. After the download is completed, a container runs to host the images on a web server. The container uses a variation of the official HTTPd image, which also enables it to work with IPv6 networks. ## Configuring image mirroring for hosted control planes in a disconnected environment {#hcp-dc-image-mirror_hcp-deploy-dc-bm} Image mirroring is the process of fetching images from external registries, such as `registry.redhat.com` or `quay.io`, and storing them in your private registry. In the following procedure, the `oc-mirror` tool is used, which is a binary that uses the `ImageSetConfiguration` object. In the file, you can specify the following information: - The OpenShift Container Platform versions to mirror. The versions are in `quay.io`. - The additional Operators to mirror. Select packages individually. - The extra images that you want to add to the repository. **Prerequisites** - Ensure that the registry server is running before you start the mirroring process. **Procedure** 1. Ensure that your `${HOME}/.docker/config.json` file is updated with the registries that you are going to mirror from and with the private registry that you plan to push the images to. 2. By using the following example, create an `ImageSetConfiguration` object to use for mirroring. Replace values as needed to match your environment: ```yaml apiVersion: mirror.openshift.io/v2alpha1 kind: ImageSetConfiguration mirror: platform: channels: - name: candidate-4.22 minVersion: <4.x.y-build> maxVersion: <4.x.y-build> type: ocp kubeVirtContainer: true graph: true operators: - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.22 packages: - name: lvms-operator - name: local-storage-operator - name: odf-csi-addons-operator - name: odf-operator - name: mcg-operator - name: ocs-operator - name: metallb-operator - name: kubevirt-hyperconverged ``` - `mirror.platform.channels.minVersion` specifies the supported OpenShift Container Platform version you want to use. - `mirror.platform.channels.maxVersion` specifies the supported (product-title) version you want to use. - `kubeVirtContainer` specifies whether you want to also mirror to the container disk image for the Red Hat Enterprise Linux CoreOS (RHCOS) boot image for the KubeVirt provider. This flag is optional. It is available with oc-mirror v2 only. - `mirror.operators.packages.name: kubevirt-hyperconverged` must be included for deployments that use the KubeVirt provider. 3. Start the mirroring process by entering the following command: ```terminal $ oc-mirror --v2 --config imagesetconfig.yaml \ --workspace file://mirror-file docker:// ``` After the mirroring process is finished, you have a new folder named `mirror-file`, which contains the `ImageDigestMirrorSet` (IDMS), `ImageTagMirrorSet` (ITMS), and the catalog sources to apply on the hosted cluster. 4. Mirror the nightly or CI versions of OpenShift Container Platform by configuring the `imagesetconfig.yaml` file as follows: ```yaml apiVersion: mirror.openshift.io/v2alpha1 kind: ImageSetConfiguration mirror: platform: graph: true release: registry.ci.openshift.org/ocp/release:<4.x.y-build> kubeVirtContainer: true # ... ``` - `mirror.platform.release` specifies the supported OpenShift Container Platform version you want to use. - `mirror.platform.kubeVirtContainer` specifies that you want to also mirror the container disk image for the Red Hat Enterprise Linux CoreOS (RHCOS) boot image for the KubeVirt provider. This flag is available with oc-mirror v2 only. 5. If you have a partially disconnected environment, mirror the images from the image set configuration to a registry by entering the following command: ```terminal $ oc mirror -c imagesetconfig.yaml \ --workspace file:// docker:// --v2 ``` For more information, see "Mirroring an image set in a partially disconnected environment". 6. If you have a fully disconnected environment, complete the following steps: 1. Mirror the images from the specified image set configuration to the disk by entering the following command: ```terminal $ oc mirror -c imagesetconfig.yaml file:// --v2 ``` For more information, see "Mirroring an image set in a fully disconnected environment". 2. Process the image set file on the disk and mirror the contents to a target mirror registry by entering the following command: ```terminal $ oc mirror -c imagesetconfig.yaml \ --from file:// docker:// --v2 ``` 7. Mirror the latest multicluster engine Operator images by following the steps in "Install on disconnected networks". **Additional resources** {._additional-resources} - [Mirroring an image set in a partially disconnected environment](/openshift-docs-markdown/disconnected/about-installing-oc-mirror-v2#oc-mirror-workflows-partially-disconnected-v2_about-installing-oc-mirror-v2) - [Mirroring an image set in a fully disconnected environment](/openshift-docs-markdown/disconnected/about-installing-oc-mirror-v2#oc-mirror-workflows-fully-disconnected-v2_about-installing-oc-mirror-v2) - [Install on disconnected networks](https://docs.redhat.com/en/documentation/red_hat_advanced_cluster_management_for_kubernetes/latest/html/clusters/cluster_mce_overview#install-on-disconnected-networks) ## Applying objects in the management cluster {#hcp-dc-apply-objects_hcp-deploy-dc-bm} After the mirroring process is complete, you must apply two objects required for mirroring in the management cluster. You apply the following objects: - `ImageContentSourcePolicy` (ICSP) or `ImageDigestMirrorSet` (IDMS) - Catalog sources When you use the `oc-mirror` tool, the output artifacts are in a folder named `oc-mirror-workspace/results-XXXXXX/`. The `oc mirror` mirroring file initiates a `MachineConfig` change that does not restart your nodes but restarts the kubelet on each of them. After the nodes are marked as `READY`, you need to apply the newly generated catalog sources. The catalog sources initiate actions in the `openshift-marketplace` Operator, such as downloading the catalog image and processing it to retrieve all the `PackageManifests` that are included in that image. **Procedure** 1. To check the new sources, run the following command by using the new `CatalogSource` as a source: ```terminal $ oc get packagemanifest ``` 2. To apply the artifacts, complete the following steps: 1. Create the ICSP or IDMS artifacts by entering the following command: ```terminal $ oc apply -f oc-mirror-workspace/results-XXXXXX/imageContentSourcePolicy.yaml ``` 2. Wait for the nodes to become ready, and then enter the following command: ```terminal $ oc apply -f catalogSource-XXXXXXXX-index.yaml ``` 3. Mirror the OLM catalogs and configure the hosted cluster to point to the mirror. When you use the `management` (default) OLMCatalogPlacement mode, the image stream that is used for OLM catalogs is not automatically amended with override information from the ICSP on the management cluster. 1. If the OLM catalogs are properly mirrored to an internal registry by using the original name and tag, add the `hypershift.openshift.io/olm-catalogs-is-registry-overrides` annotation to the `HostedCluster` resource. The format is `"sr1=dr1,sr2=dr2"`, where the source registry string is a key and the destination registry is a value. 2. To bypass the OLM catalog image stream mechanism, use the following four annotations on the `HostedCluster` resource to directly specify the addresses of the four images to use for OLM Operator catalogs: - `hypershift.openshift.io/certified-operators-catalog-image` - `hypershift.openshift.io/community-operators-catalog-image` - `hypershift.openshift.io/redhat-marketplace-catalog-image` - `hypershift.openshift.io/redhat-operators-catalog-image` In this case, the image stream is not created, and you must update the value of the annotations when the internal mirror is refreshed to pull in Operator updates. **Next steps** Deploy the multicluster engine Operator by completing the steps in "Deploying multicluster engine Operator for a disconnected installation of hosted control planes". **Additional resources** {._additional-resources} - [Mirroring images for a disconnected installation by using the oc-mirror plugin v2](/openshift-docs-markdown/disconnected/about-installing-oc-mirror-v2#about-installing-oc-mirror-v2) ## Deploying AgentServiceConfig resources {#hcp-agentserviceconfig_hcp-deploy-dc-bm} The `AgentServiceConfig` custom resource (CR) is an essential component of the Assisted Service add-on that is part of multicluster engine Operator. The CR is responsible for bare-metal cluster deployment. When the add-on is enabled, you deploy the `AgentServiceConfig` resource to configure the add-on. In addition to configuring the `AgentServiceConfig` resource, you need to include additional config maps to ensure that multicluster engine Operator functions properly in a disconnected environment. **Procedure** 1. Configure the custom registries by adding the following config map, which has the disconnected details to customize the deployment: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: custom-registries namespace: multicluster-engine labels: app: assisted-service data: ca-bundle.crt: | -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- registries.conf: | unqualified-search-registries = ["registry.access.redhat.com", "docker.io"] [[registry]] prefix = "" location = "registry.redhat.io/openshift4" mirror-by-digest-only = true [[registry.mirror]] location = "registry.dns.base.domain.name:5000/openshift4" [[registry]] prefix = "" location = "registry.redhat.io/rhacm2" mirror-by-digest-only = true # ... # ... ``` - The `ca-bundle.crt` field has the Certificate Authorities (CAs) that are loaded into the various processes of the deployment. - The `registries.conf` field has information about images and namespaces that need to be consumed from a mirror registry rather than the original source registry. - Replace the `dns.base.domain.name` in the `registry.mirror` container reference with the DNS base domain name. 2. Configure the Assisted Service by adding the `AssistedServiceConfig` object, as shown in the following example: ```yaml apiVersion: agent-install.openshift.io/v1beta1 kind: AgentServiceConfig metadata: annotations: unsupported.agent-install.openshift.io/assisted-service-configmap: assisted-service-config name: agent namespace: multicluster-engine spec: mirrorRegistryRef: name: custom-registries databaseStorage: storageClassName: lvms-vg1 accessModes: - ReadWriteOnce resources: requests: storage: 10Gi filesystemStorage: storageClassName: lvms-vg1 accessModes: - ReadWriteOnce resources: requests: storage: 20Gi osImages: - cpuArchitecture: x86_64 openshiftVersion: "4.14" rootFSUrl: http://registry.dns.base.domain.name:8080/images/rhcos-414.92.202308281054-0-live-rootfs.x86_64.img url: http://registry.dns.base.domain.name:8080/images/rhcos-414.92.202308281054-0-live.x86_64.iso version: 414.92.202308281054-0 - cpuArchitecture: x86_64 openshiftVersion: "4.15" rootFSUrl: http://registry.dns.base.domain.name:8080/images/rhcos-415.92.202403270524-0-live-rootfs.x86_64.img url: http://registry.dns.base.domain.name:8080/images/rhcos-415.92.202403270524-0-live.x86_64.iso version: 415.92.202403270524-0 ``` - The `metadata.annotations["unsupported.agent-install.openshift.io/assisted-service-configmap"]` annotation references the config map name that the Operator consumes to customize behavior. - The `spec.mirrorRegistryRef.name` field points to the `ConfigMap` CR that has the disconnected registry information that the Assisted Service Operator consumes. This config map adds those resources during the deployment process. - The `spec.osImages` field contains different versions available for deployment by this Operator. This field is mandatory. This example assumes that you already downloaded the `RootFS` and `LiveISO` files. - The `cpuArchitecture` field is added for every OpenShift Container Platform release that you want to deploy. In this example, `cpuArchitecture` subsections are included for 4.14 and 4.15. - The `osImages.rootFSUrl` field includes `dns.base.domain.name`. Replace that value with the DNS base domain name. - The `osImages.url` field includes `dns.base.domain.name`. Replace that value with the DNS base domain name. 3. Deploy all of the objects by concatenating them into a single file and applying them to the management cluster. To do so, enter the following command: ```terminal $ oc apply -f agentServiceConfig.yaml ``` The command triggers two pods. ```terminal {title="Example output"} assisted-image-service-0 1/1 Running 2 11d assisted-service-668b49548-9m7xw 2/2 Running 5 11d ``` - The `assisted-image-service` pod is responsible for creating the Red Hat Enterprise Linux CoreOS (RHCOS) boot image template, which is customized for each cluster that you deploy. - The `assisted-service` refers to the Operator. **Next steps** Configure TLS certificates. ## Adding the registry CA to the management cluster {#hcp-dc-tls-mgmt_hcp-deploy-dc-bm} To ensure proper function in a disconnected deployment, you need to configure the registry CA certificates in the management cluster. To add the registry CA to the management cluster, complete the following steps. **Procedure** 1. Create a config map that resembles the following example: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: namespace: data: ..: | -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- ..: | -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- ..: | -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- ``` - `metadata.name` specifies the name of the config map. - `metadata.namespace` specifies the namespace for the config map. - `data` specifies the registry names and the registry certificate content. Replace `` with the port where the registry server is running; for example, `5000`. Ensure that the data in the config map is defined by using `|` only instead of other methods, such as `| -`. If you use other methods, issues can occur when the pod reads the certificates. 2. Patch the cluster-wide object, `image.config.openshift.io` to include the following specification: ```yaml spec: additionalTrustedCA: name: registry-config ``` As a result of this patch, the control plane nodes can retrieve images from the private registry and the HyperShift Operator can extract the OpenShift Container Platform payload for hosted cluster deployments. The process to patch the object might take several minutes to be completed. ## Adding the registry CA to the compute nodes for the hosted cluster {#hcp-dc-tls-hosted_hcp-deploy-dc-bm} To ensure that the data plane compute nodes in the hosted cluster can retrieve images from the private registry, you must add the registry certificate authority (CA) to the compute nodes. **Procedure** 1. In the `hc.spec.additionalTrustBundle` file, add the following specification: ```yaml spec: additionalTrustBundle: name: user-ca-bundle ``` The `user-ca-bundle` entry is a config map that you create in the next step. 2. In the same namespace where the `HostedCluster` object is created, create the `user-ca-bundle` config map. The config map resembles the following example: ```yaml apiVersion: v1 data: ca-bundle.crt: | // Registry1 CA -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- // Registry2 CA -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- // Registry3 CA -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- kind: ConfigMap metadata: name: user-ca-bundle namespace: ``` Specify the namespace where the `HostedCluster` object is created. ## Hosted clusters on bare metal in a disconnected environment {#hcp-dc-bm-hosted_hcp-deploy-dc-bm} In a disconnected environment, creating a hosted cluster involves deploying hosted cluster objects, creating node pools, creating an `InfraEnv` resource, creating bare-metal hosts, and scaling the node pools as needed. A hosted cluster is an OpenShift Container Platform cluster with its control plane and API endpoint hosted on a management cluster. The hosted cluster includes the corresponding data plane. ### Deploying hosted cluster objects {#hcp-hc-objects_hcp-deploy-dc-bm} Typically, the HyperShift Operator creates the `HostedControlPlane` namespace. However, you might want to include all the objects before the HyperShift Operator begins to reconcile the `HostedCluster` object. Then, when the Operator starts the reconciliation process, it can find all of the objects in place. **Procedure** 1. Create a YAML file with the following information about the namespaces: ```yaml --- apiVersion: v1 kind: Namespace metadata: creationTimestamp: null name: - spec: {} status: {} --- apiVersion: v1 kind: Namespace metadata: creationTimestamp: null name: spec: {} status: {} ``` - `` is the name of your hosted cluster. - `` is the name of your hosted cluster namespace. 2. Create a YAML file with the following information about the config maps and secrets to include in the `HostedCluster` deployment: ```yaml --- apiVersion: v1 data: ca-bundle.crt: | -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- kind: ConfigMap metadata: name: user-ca-bundle namespace: --- apiVersion: v1 data: .dockerconfigjson: xxxxxxxxx kind: Secret metadata: creationTimestamp: null name: -pull-secret namespace: --- apiVersion: v1 kind: Secret metadata: name: sshkey-cluster- namespace: stringData: id_rsa.pub: ssh-rsa xxxxxxxxx --- apiVersion: v1 data: key: nTPtVBEt03owkrKhIdmSW8jrWRxU57KO/fnZa8oaG0Y= kind: Secret metadata: creationTimestamp: null name: -etcd-encryption-key namespace: type: Opaque ``` - `` is the name of your hosted cluster namespace. - `` is the name of your hosted cluster. 3. Create a YAML file that contains the RBAC roles so that Assisted Service agents can be in the same `HostedControlPlane` namespace as the hosted control plane and still be managed by the cluster API: ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: creationTimestamp: null name: capi-provider-role namespace: - rules: - apiGroups: - agent-install.openshift.io resources: - agents verbs: - '*' ``` - `` is the name of your hosted cluster namespace. - `` is the name of your hosted cluster. 4. Create a YAML file with information about the `HostedCluster` object, replacing values as necessary: ```yaml apiVersion: hypershift.openshift.io/v1beta1 kind: HostedCluster metadata: name: namespace: spec: additionalTrustBundle: name: "user-ca-bundle" olmCatalogPlacement: guest configuration: operatorhub: disableAllDefaultSources: true imageContentSources: - source: quay.io/openshift-release-dev/ocp-v4.0-art-dev mirrors: - registry.:5000/openshift/release - source: quay.io/openshift-release-dev/ocp-release mirrors: - registry.:5000/openshift/release-images - mirrors: - registry.:5000/openshift/release-images - source: registry.redhat.io/multicluster-engine mirrors: - registry.:5000/openshift/multicluster-engine # ... autoscaling: {} controllerAvailabilityPolicy: SingleReplica dns: baseDomain: etcd: managed: storage: persistentVolume: size: 8Gi restoreSnapshotURL: null type: PersistentVolume managementType: Managed fips: false networking: clusterNetwork: - cidr: 10.132.0.0/14 - cidr: fd01::/48 networkType: OVNKubernetes serviceNetwork: - cidr: 172.31.0.0/16 - cidr: fd02::/112 platform: agent: agentNamespace: type: Agent pullSecret: name: -pull-secret release: image: registry.:5000/openshift/release-images:<4.x.y>-x86_64 secretEncryption: aescbc: activeKey: name: -etcd-encryption-key type: aescbc services: - service: APIServer servicePublishingStrategy: type: LoadBalancer - service: OAuthServer servicePublishingStrategy: type: Route - service: OIDC servicePublishingStrategy: type: Route - service: Konnectivity servicePublishingStrategy: type: Route - service: Ignition servicePublishingStrategy: type: Route sshKey: name: sshkey-cluster- status: controlPlaneEndpoint: host: "" port: 0 ``` - `` is the name of your hosted cluster. - `` is the name of your hosted cluster namespace. - `disableAllDefaultSources` is `true` if you want to disable all default OLM catalog resources. The default value is `false`, which enables all default OLM catalog resources. - `imageContentSources` contains mirror references for user workloads within the hosted cluster. - `` is the DNS base domain name. - `` is the namespace where the Bare Metal Host (BMH) and `InfraEnv` resources are created. - `<4.x.y>` is the supported OpenShift Container Platform version you want to use. 5. Create all of the objects that you defined in the YAML files by concatenating them into a file and applying them against the management cluster. To do so, enter the following command: ```terminal $ oc apply -f 01-4.14-hosted_cluster-nodeport.yaml ``` The following example shows the output of the command: ```terminal NAME READY STATUS RESTARTS AGE capi-provider-5b57dbd6d5-pxlqc 1/1 Running 0 3m57s catalog-operator-9694884dd-m7zzv 2/2 Running 0 93s cluster-api-f98b9467c-9hfrq 1/1 Running 0 3m57s cluster-autoscaler-d7f95dd5-d8m5d 1/1 Running 0 93s cluster-image-registry-operator-5ff5944b4b-648ht 1/2 Running 0 93s cluster-network-operator-77b896ddc-wpkq8 1/1 Running 0 94s cluster-node-tuning-operator-84956cd484-4hfgf 1/1 Running 0 94s cluster-policy-controller-5fd8595d97-rhbwf 1/1 Running 0 95s cluster-storage-operator-54dcf584b5-xrnts 1/1 Running 0 93s cluster-version-operator-9c554b999-l22s7 1/1 Running 0 95s control-plane-operator-6fdc9c569-t7hr4 1/1 Running 0 3m57s csi-snapshot-controller-785c6dc77c-8ljmr 1/1 Running 0 77s csi-snapshot-controller-operator-7c6674bc5b-d9dtp 1/1 Running 0 93s csi-snapshot-webhook-5b8584875f-2492j 1/1 Running 0 77s dns-operator-6874b577f-9tc6b 1/1 Running 0 94s etcd-0 3/3 Running 0 3m39s hosted-cluster-config-operator-f5cf5c464-4nmbh 1/1 Running 0 93s ignition-server-6b689748fc-zdqzk 1/1 Running 0 95s ignition-server-proxy-54d4bb9b9b-6zkg7 1/1 Running 0 95s ingress-operator-6548dc758b-f9gtg 1/2 Running 0 94s konnectivity-agent-7767cdc6f5-tw782 1/1 Running 0 95s kube-apiserver-7b5799b6c8-9f5bp 4/4 Running 0 3m7s kube-controller-manager-5465bc4dd6-zpdlk 1/1 Running 0 44s kube-scheduler-5dd5f78b94-bbbck 1/1 Running 0 2m36s machine-approver-846c69f56-jxvfr 1/1 Running 0 92s oauth-openshift-79c7bf44bf-j975g 2/2 Running 0 62s olm-operator-767f9584c-4lcl2 2/2 Running 0 93s openshift-apiserver-5d469778c6-pl8tj 3/3 Running 0 2m36s openshift-controller-manager-6475fdff58-hl4f7 1/1 Running 0 95s openshift-oauth-apiserver-dbbc5cc5f-98574 2/2 Running 0 95s openshift-route-controller-manager-5f6997b48f-s9vdc 1/1 Running 0 95s packageserver-67c87d4d4f-kl7qh 2/2 Running 0 93s ``` When the hosted cluster is available, the output looks like the following example: ```terminal NAMESPACE NAME VERSION KUBECONFIG PROGRESS AVAILABLE PROGRESSING MESSAGE clusters hosted-dual hosted-admin-kubeconfig Partial True False The hosted control plane is available ``` ### Creating a NodePool object for the hosted cluster {#hcp-nodepool-hc_hcp-deploy-dc-bm} A `NodePool` object is a scalable set of compute nodes that is associated with a hosted cluster. `NodePool` machine architectures remain consistent within a specific pool and are independent of the machine architecture of the control plane. **Procedure** 1. Create a YAML file with the following information about the `NodePool` object, replacing values as necessary: ```yaml apiVersion: hypershift.openshift.io/v1beta1 kind: NodePool metadata: creationTimestamp: null name: namespace: spec: arch: amd64 clusterName: management: autoRepair: false upgradeType: InPlace nodeDrainTimeout: 0s platform: type: Agent release: image: registry.:5000/openshift/release-images:4.x.y-x86_64 replicas: 2 status: replicas: 2 ``` - `metadata.name` specifies the name of your hosted cluster. - `metadata.namespace` specifies the name of your hosted cluster namespace. - `spec.management.autoRepair` specifies whether the node will be re-created if it is removed. In this example, the `autoRepair` field is set to `false` because the node will not be re-created if it is removed. - `spec.management.upgradeType` specifies the upgrade type. In this example, the `upgradeType` field is set to `InPlace`, which indicates that the same bare-metal node is reused during an upgrade. - `spec.release.image` specifies which OpenShift Container Platform version the nodes in the `NodePool` object are based on. Replace the `` value with your DNS base domain name and the `4.x.y` value with the supported OpenShift Container Platform version you want to use. - `spec.replicas` specifies the number of node pool replicas to create. In this example, the `replicas` value is `2` to create two node pool replicas in your hosted cluster. 2. Create the `NodePool` object by entering the following command: ```terminal $ oc apply -f 02-nodepool.yaml ``` ```terminal {title="Example output"} NAMESPACE NAME CLUSTER DESIRED NODES CURRENT NODES AUTOSCALING AUTOREPAIR VERSION UPDATINGVERSION UPDATINGCONFIG MESSAGE clusters hosted-dual hosted 0 False False 4.x.y-x86_64 ``` ### Creating an InfraEnv resource for the hosted cluster {#hcp-dc-infraenv_hcp-deploy-dc-bm} The `InfraEnv` resource is an Assisted Service object that includes essential details that are used to create the Red Hat Enterprise Linux CoreOS (RHCOS) boot image that is customized for the hosted cluster. You can host more than one `InfraEnv` resource, and each one can adopt certain types of hosts. For example, you might want to divide your server farm between a host that has greater RAM capacity. **Procedure** 1. Create a YAML file with the following information about the `InfraEnv` resource, replacing values as necessary: ```yaml apiVersion: agent-install.openshift.io/v1beta1 kind: InfraEnv metadata: name: namespace: spec: pullSecretRef: name: pull-secret sshAuthorizedKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk7ICaUE+/k4zTpxLk4+xFdHi4ZuDi5qjeF52afsNkw0w/glILHhwpL5gnp5WkRuL8GwJuZ1VqLC9EKrdmegn4MrmUlq7WTsP0VFOZFBfq2XRUxo1wrRdor2z0Bbh93ytR+ZsDbbLlGngXaMa0Vbt+z74FqlcajbHTZ6zBmTpBVq5RHtDPgKITdpE1fongp7+ZXQNBlkaavaqv8bnyrP4BWahLP4iO9/xJF9lQYboYwEEDzmnKLMW1VtCE6nJzEgWCufACTbxpNS7GvKtoHT/OVzw8ArEXhZXQUS1UY8zKsX2iXwmyhw5Sj6YboA8WICs4z+TrFP89LmxXY0j6536TQFyRz1iB4WWvCbH5n6W+ABV2e8ssJB1AmEy8QYNwpJQJNpSxzoKBjI73XxvPYYC/IjPFMySwZqrSZCkJYqQ023ySkaQxWZT7in4KeMu7eS2tC+Kn4deJ7KwwUycx8n6RHMeD8Qg9flTHCv3gmab8JKZJqN3hW1D378JuvmIX4V0= ``` - `metadata.name` specifies the name of your hosted cluster namespace. - `spec.pullSecretRef` specifies the config map reference in the same namespace as the `InfraEnv`, where the pull secret is used. - `spec.sshAuthorizedKey` specifies the SSH public key that is placed in the boot image. The SSH key allows access to the worker nodes as the `core` user. 2. Create the `InfraEnv` resource by entering the following command: ```terminal $ oc apply -f 03-infraenv.yaml ``` ```text {title="Example output"} NAMESPACE NAME ISO CREATED AT clusters-hosted-dual hosted 2023-09-11T15:14:10Z ``` ### Creating bare-metal hosts for the hosted cluster {#hcp-bm-hosts_hcp-deploy-dc-bm} A *bare-metal host* is an `openshift-machine-api` object that encompasses physical and logical details so that it can be identified by a Metal3 Operator. Those details are associated with other Assisted Service objects, known as *agents*. **Prerequisites** - Before you create the bare-metal host and destination nodes, you must have the destination machines ready. - You have installed a Red Hat Enterprise Linux CoreOS (RHCOS) compute machine on bare-metal infrastructure for use in the cluster. **Procedure** 1. Create a YAML file with the following information. For more information about what details to enter for the bare-metal host, see "Provisioning new hosts in a user-provisioned cluster by using the BMO". Because you have at least one secret that holds the bare-metal host credentials, you need to create at least two objects for each compute node. ```yaml apiVersion: v1 kind: Secret metadata: name: -worker0-bmc-secret namespace: data: password: YWRtaW4= username: YWRtaW4= type: Opaque # ... apiVersion: metal3.io/v1alpha1 kind: BareMetalHost metadata: name: -worker0 namespace: labels: infraenvs.agent-install.openshift.io: annotations: inspect.metal3.io: disabled bmac.agent-install.openshift.io/hostname: -worker0 spec: automatedCleaningMode: disabled bmc: disableCertificateVerification: true address: redfish-virtualmedia://[192.168.126.1]:9000/redfish/v1/Systems/local/-worker0 credentialsName: -worker0-bmc-secret bootMACAddress: aa:aa:aa:aa:02:11 online: true ``` - `metadata.name` specifies the name of your hosted cluster. - `metadata.namespace` specifies the name of your hosted cluster namespace. - `data.password` specifies the password of the baseboard management controller (BMC) in Base64 format. - `data.username` specifies the user name of the BMC in Base64 format. - `metadata.labels.infraenvs.agent-install.openshift.io` serves as the link between the Assisted Installer and the `BareMetalHost` objects. - `metadata.annotations.bmac.agent-install.openshift.io/hostname` represents the node name that is adopted during deployment. - `spec.automatedCleaningMode` prevents the node from being erased by the Metal3 Operator. - `spec.bmc.disableCertificateVerification` is set to `true` to bypass certificate validation from the client. - `spec.bmc.address` denotes the BMC address of the compute node. - `spec.bmc.credentialsName` points to the secret where the user and password credentials are stored. - `spec.bootMACAddress` indicates the interface MAC address that the node starts from. - `spec.online` defines the state of the node after the `BareMetalHost` object is created. 2. Deploy the `BareMetalHost` object by entering the following command: ```terminal $ oc apply -f 04-bmh.yaml ``` During the process, the statuses of the bare-metal hosts change from `Registering` to `Provisioning` to `Provisioned`. The nodes start with the `LiveISO` of the agent and a default pod that is named `agent`. That agent is responsible for receiving instructions from the Assisted Service Operator to install the OpenShift Container Platform payload. - This output indicates that the process is trying to reach the nodes: ```terminal {title="Example output"} NAMESPACE NAME STATE CONSUMER ONLINE ERROR AGE clusters-hosted hosted-worker0 registering true 2s clusters-hosted hosted-worker1 registering true 2s clusters-hosted hosted-worker2 registering true 2s ``` - This output indicates that the nodes are starting: ```terminal {title="Example output"} NAMESPACE NAME STATE CONSUMER ONLINE ERROR AGE clusters-hosted hosted-worker0 provisioning true 16s clusters-hosted hosted-worker1 provisioning true 16s clusters-hosted hosted-worker2 provisioning true 16s ``` - This output indicates that the nodes started successfully: ```terminal {title="Example output"} NAMESPACE NAME STATE CONSUMER ONLINE ERROR AGE clusters-hosted hosted-worker0 provisioned true 67s clusters-hosted hosted-worker1 provisioned true 67s clusters-hosted hosted-worker2 provisioned true 67s ``` 3. After the nodes start, notice the agents in the namespace, as shown in this example: ```terminal {title="Example output"} NAMESPACE NAME CLUSTER APPROVED ROLE STAGE clusters-hosted aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaa0411 true auto-assign clusters-hosted aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaa0412 true auto-assign clusters-hosted aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaa0413 true auto-assign ``` The agents represent nodes that are available for installation. To assign the nodes to a hosted cluster, scale up the node pool. **Additional resources** {._additional-resources} - [Provisioning new hosts in a user-provisioned cluster by using the BMO](/openshift-docs-markdown/installing/installing_bare_metal/upi/scaling-a-user-provisioned-cluster-with-the-bare-metal-operator#upi-provisioning-new-hosts-in-a-upi-cluster_scaling-a-user-provisioned-cluster-with-the-bare-metal-operator) - [Understanding secrets](/openshift-docs-markdown/nodes/pods/nodes-pods-secrets#nodes-pods-secrets-about_nodes-pods-secrets) ### Scaling up the node pool {#hcp-dc-scale-np_hcp-deploy-dc-bm} After you create bare-metal hosts, agents are added to your namespace. The agents represent nodes that are available for installation. To assign the nodes to a hosted cluster, scale up the node pool. **Prerequisites** - You created bare-metal hosts for your hosted cluster. **Procedure** 1. To scale up the node pool, enter the following command: ```terminal $ oc -n scale nodepool \ --replicas 3 ``` - `` is the name of the hosted cluster namespace. - `` is the name of the hosted cluster. 2. After the scaling process is complete, notice that the agents are assigned to a hosted cluster: ```terminal {title="Example output"} NAMESPACE NAME CLUSTER APPROVED ROLE STAGE clusters-hosted aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaa0411 hosted true auto-assign clusters-hosted aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaa0412 hosted true auto-assign clusters-hosted aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaa0413 hosted true auto-assign ``` 3. Also notice that the node pool replicas are set, as shown in the following example: ```terminal {title="Example output"} NAMESPACE NAME CLUSTER DESIRED NODES CURRENT NODES AUTOSCALING AUTOREPAIR VERSION UPDATINGVERSION UPDATINGCONFIG MESSAGE clusters hosted hosted 3 False False <4.x.y>-x86_64 Minimum availability requires 3 replicas, current 0 available ``` In your output, `<4.x.y>` is replaced with the supported OpenShift Container Platform version that you are using. 4. Wait until the nodes join the cluster. During the process, the agents provide updates on their stage and status. **Additional resources** {._additional-resources} - [Enabling the central infrastructure management service](https://docs.redhat.com/en/documentation/red_hat_advanced_cluster_management_for_kubernetes/latest/html/clusters/cluster_mce_overview#enable-cim)