Provisioning Baremetal OpenShift clusters using RHACM with GitOps leveraging on-premise Assisted Installer

Recently, I published the blog Provisioning OpenShift clusters using GitOps with ACM explaining how to  create OpenShift clusters with RHACM using Gitops with ArgoCD. The OpenShift installation type was IPI, and valid for most of the platforms: Azure, AWS, GCP, vSphere… but not for baremetal. If you’ve ever installed an OpenShift cluster in baremetal and disconnected, you know how different it is from any other installation.

In this blog, I’ll explain how to deploy a baremetal OpenShift cluster with Assisted Installer using RHACM and GitOps with ArgoCD. If you are not familiar with deploying OpenShift clusters with RHACM and Gitops, I recommend reading the article I wrote: GitOps for organizations: provisioning and configuring OpenShift clusters automatically. I also highly recommend reading the blog Managing OCP Infrastructures Using GitOps, which I used the first time I deployed this solution.

Solution Overview

We’ll use OpenShift Gitops and RHACM in the same way as we did in Provisioning OpenShift clusters using GitOps with ACM: we’ll upload the Kubernetes objects to our git repository, ArgoCD will synchronize these object to our OpenShift cluster, and RHACM will deploy the cluster leveraging Baremetal Operator, Ironic and Assisted Installer.

We’ll create the BareMetalHosts (BMH), which are Metal³ Custom Resource Definitions (CRDs) that define a physical host and its properties. The BMHs will connect to the baseboard management controller (BMC) physical nodes using the Redfish protocol. The node’s network will be statically configured using NMstateConfig. The OpenShift cluster will be deployed with Assisted Installer using the BareMetalHosts created.

Don’t start creating all the objects. As there are many resources involved, we recommend creating them one at a time and checking their status. Start checking the prerequisites in the RHACM documentation before creating an infrastructure environment, and enable the Central Infrastructure Management service, which is provided with the Multicluster Engine. 

Once prerequisites have been fulfilled, move to the RHACM console. In the Infrastructure Environment, create and connect the Baremetal Hosts to the host’s BMC with Redfish. And then, deploy a cluster using the Baremetal Hosts (existing discovered hosts) following Creating your cluster in ACM with the console. Check the objects created in the deployment, move to the command line, and deploy another cluster creating the same objects with other parameters following Creating your cluster in ACM with the command line. After that, upload the yaml files to your git repo, and create an ArgoCD Application to sync the objects. 

The last step would be to create a Helm chart with all the objects as templates, and an ApplicationSet to create an Application per cluster as we did in Provisioning OpenShift clusters using GitOps with ACM.

Baremetal Operator, Ironic and Assisted Installer in RHACM

Bare Metal Operator is the main component that interfaces with the Ironic API for all operations needed to provision bare-metal hosts, such as hardware capabilities inspection, operating system installation, and re-initialization when restoring a bare-metal machine to its original status.

Ironic is a service for automating provisioning and lifecycle management of bare metal machines.

The OpenShift Assisted Installer provides for easy provisioning of new bare metal machines and creation of OpenShift clusters.  The Assisted Installer ensures that all the hosts meet the requirements and triggers the OpenShift Container Platform cluster deployment. All the nodes (BMHs) have the Red Hat Enterprise Linux CoreOS (RHCOS) image written to the disk. 

The Assisted Image Service customizes and serves RHCOS images for the Assisted Installer Service. It downloads a set of RHCOS images on startup based on config .

The assisted installer provisioning workflow:

Objects

In the RHACM documentation chapter Creating your cluster with the command line, you can get all the objects needed to deploy the cluster with Assisted Installer: 

• Namespace
• ClusterImageSet
• ClusterDeployment
• AgentClusterInstall
• NMStateConfig
• BaremetalHost
• InfraEnv
• KlusterletAddonConfig
• ManagedCluster

Useful Tips

As I said before, start checking the prerequisites, and specially the connectivity between out-of-band management host IP addresses and RHACM. For virtualmedia, you’ll need to open the port 6183. The troubleshooting with virtualmedia, assisted-agent and ignition can be difficult, so try to get access to the host ILO and a virtual terminal. This will make it much easier to troubleshoot.

Don’t create all the objects at the same time, create them one at a time and checking their status

And the last tip: you can contact Red Hat Professional services for assistance. Although we openly share knowledge publishing blogs, solutions and articles like this one, each environment has its own customizations and challenges, and we have wide experience with a lot of customers all over the world.

Summary

We’ve deployed a baremetal OpenShift cluster using RHACM and on-premise assisted installer.

First, we need to check the prerequisites. Once we’ve the prerequisites in place, we move to the RHACM console, and create the Baremetal hosts and check their status. After that, we can deploy an Openshift cluster using Assisted Installer and the Baremetal hosts. If the deployment is successful, we can move to the command line, and create the objects one at a time checking their status. At last, we can create the objects in our Git repository, and synchronize the objects to our RHACM cluster using ArgoCD.

If we want to automate baremetal OpenShift cluster deployments like a self-service, you can create a Helm Chart with all the objects, and an ArgoCD ApplicationSet to deploy each cluster using the Helm chart as we did in Provisioning OpenShift clusters using GitOps with ACM.