Prerequisites for installing a cluster on IBM Cloud(R) Bare Metal (Classic)

You can use installer-provisioned installation to install OpenShift Container Platform on IBM Cloud(R) Bare Metal (Classic) nodes. Review the prerequisites and requirements before you begin an installer-provisioned installation on IBM Cloud(R) nodes.

Warning

Red Hat supports Intelligent Platform Management Interface (IPMI) and PXE on the provisioning network only. Red Hat has not tested Red Fish, virtual media, or other complementary technologies such as Secure Boot on IBM Cloud(R) deployments. You must configure a provisioning network.

Installer-provisioned installation of OpenShift Container Platform requires:

• One node with Red Hat Enterprise Linux CoreOS (RHCOS) 8.x installed, for running the provisioner
• Three control plane nodes
• One routable network
• One provisioning network

About setting up IBM Cloud Bare Metal (Classic) infrastructure

To deploy an OpenShift Container Platform cluster on IBM Cloud(R) Bare Metal (Classic) infrastructure, you must provision the IBM Cloud(R) nodes and configure the supporting infrastructure. This includes setting up virtual networking and creating the bare-metal servers that make up the cluster.

Warning

Red Hat supports Intelligent Platform Management Interface (IPMI) and PXE on the provisioning network only. Red Hat has not tested Red Fish, virtual media, or other complementary technologies such as Secure Boot on IBM Cloud(R) deployments. You must configure a provisioning network.

You can customize IBM Cloud(R) nodes by using the IBM Cloud(R) API. When creating IBM Cloud(R) nodes, you must consider the following requirements.

Use one data center per cluster

All nodes in the OpenShift Container Platform cluster must run in the same IBM Cloud(R) data center.

Create public and private VLANs

Create all nodes with a single public virtual LAN (VLAN) and a single private VLAN.

Ensure subnets have enough IP addresses

IBM Cloud(R) public VLAN subnets use a /28 prefix by default, which provides 16 IP addresses. That is enough for a cluster consisting of three control plane nodes, four compute nodes, and two IP addresses for the API virtual IP (VIP) and Ingress VIP on the baremetal network. For larger clusters, you might need a smaller prefix.

IBM Cloud(R) private VLAN subnets use a /26 prefix by default, which provides 64 IP addresses. IBM Cloud(R) Bare Metal (Classic) uses private network IP addresses to access the Baseboard Management Controller (BMC) of each node. OpenShift Container Platform creates an additional subnet for the provisioning network. Network traffic for the provisioning network subnet routes through the private VLAN. For larger clusters, you might need a smaller prefix.

IP addresses per prefix

IP addresses	Prefix
32	/27
64	/26
128	/25
256	/24

Configuring NICs

OpenShift Container Platform deploys with two networks:

• provisioning: The provisioning network is a non-routable network used for provisioning the underlying operating system on each node that is a part of the OpenShift Container Platform cluster.
• baremetal: The baremetal network is a routable network. You can use any NIC order to interface with the baremetal network, provided it is not the NIC specified in the provisioningNetworkInterface configuration setting or the NIC associated to a node’s bootMACAddress configuration setting for the provisioning network.

While the cluster nodes can contain more than two NICs, the installation process only focuses on the first two NICs. For example:

NIC	Network	VLAN
NIC1	provisioning	<provisioning_vlan>
NIC2	baremetal	<baremetal_vlan>

In the earlier example, NIC1 on all control plane and worker nodes connects to the non-routable network (provisioning) that is only used for the installation of the OpenShift Container Platform cluster. NIC2 on all control plane and worker nodes connects to the baremetal network.

PXE	Boot order
NIC1 PXE-enabled provisioning network	1
NIC2 baremetal network.	2

Note

Enable PXE on the NIC used for the provisioning network and disable it on all other NICs.

Configure canonical names

Clients access the OpenShift Container Platform cluster nodes over the baremetal network. Configure IBM Cloud(R) subdomains or subzones where the canonical name extension is the cluster name.

<cluster_name>.<domain>

For example:

test-cluster.example.com

Create DNS entries

You must create DNS A record entries resolving to unused IP addresses on the public subnet for the following:

Usage	Hostname	IP
API	api.<cluster_name>.<domain>	<ip>
Ingress LB (apps)	*.apps.<cluster_name>.<domain>	<ip>

Control plane and worker nodes already have DNS entries after provisioning.

The following table provides an example of fully qualified domain names. The API and name server addresses begin with canonical name extensions. The hostnames of the control plane and worker nodes are examples, so you can use any host naming convention you prefer.

Usage	Hostname	IP
API	api.<cluster_name>.<domain>	<ip>
Ingress LB (apps)	*.apps.<cluster_name>.<domain>	<ip>
Provisioner node	provisioner.<cluster_name>.<domain>	<ip>
Control plane 0	openshift-control-0.<cluster_name>.<domain>	<ip>
Control plane 1	openshift-control-1.<cluster_name>.<domain>	<ip>
Control plane 2	openshift-control-2.<cluster_name>.<domain>	<ip>
Compute 0	openshift-compute-0.<cluster_name>.<domain>	<ip>
Compute 1	openshift-compute-1.<cluster_name>.<domain>	<ip>
Compute n	openshift-compute-n.<cluster_name>.<domain>	<ip>

OpenShift Container Platform includes functionality that uses cluster membership information to generate A records. This resolves the node names to their IP addresses. After the nodes register with the API, the cluster can disperse node information without using CoreDNS-mDNS. This eliminates the network traffic associated with multicast DNS.

Warning

After provisioning the IBM Cloud(R) nodes, you must create a DNS entry for the api.<cluster_name>.<domain> domain name on the external DNS because removing CoreDNS causes the local entry to disappear. Failure to create a DNS record for the api.<cluster_name>.<domain> domain name in the external DNS server prevents worker nodes from joining the cluster.

Network Time Protocol (NTP)

Each OpenShift Container Platform node in the cluster must have access to an NTP server. OpenShift Container Platform nodes use NTP to synchronize their clocks. For example, cluster nodes use SSL/TLS certificates that require validation, which might fail if the date and time between the nodes are not in sync.

Warning

Define a consistent clock date and time format in each cluster node’s firmware settings, or installation might fail.

Configure a DHCP server

IBM Cloud(R) Bare Metal (Classic) does not run DHCP on the public or private VLANs. After provisioning IBM Cloud(R) nodes, you must set up a DHCP server for the public VLAN, which corresponds to OpenShift Container Platform's baremetal network.

Note

The IP addresses allocated to each node do not need to match the IP addresses allocated by the IBM Cloud(R) Bare Metal (Classic) provisioning system.

See the "Configuring the public subnet" section for details.

Ensure BMC access privileges

The Remote management page for each node on the dashboard has the node’s IPMI credentials. The default IPMI privileges prevent the user from making certain boot target changes. You must change the privilege level to OPERATOR so that Ironic can make those changes.

In the install-config.yaml file, add the privilegelevel parameter to the URLs used to configure each BMC. See the "Configuring the install-config.yaml file" section for additional details. For example:

ipmi://<IP>:<port>?privilegelevel=OPERATOR

Contact IBM Cloud(R) support to request that they increase the IPMI privileges to ADMINISTRATOR for each node.

Create bare-metal servers

Create bare-metal servers in the IBM Cloud(R) dashboard by navigating to Create resource → Bare Metal Servers for Classic.

You can also create bare-metal servers with the ibmcloud CLI utility. For example:

$ ibmcloud sl hardware create --hostname <SERVERNAME> \
                            --domain <DOMAIN> \
                            --size <SIZE> \
                            --os <OS-TYPE> \
                            --datacenter <DC-NAME> \
                            --port-speed <SPEED> \
                            --billing <BILLING>

Note

IBM Cloud(R) servers might take 3-5 hours to become available.

Additional resources

• IBM Cloud(R) dashboard
• Installing the standalone IBM Cloud(R) CLI