--- title: Driver Toolkit --- # Driver Toolkit {#driver-toolkit} The Driver Toolkit is a base container image for building driver containers that enable specialized hardware and software devices on OpenShift Container Platform clusters. ## About the Driver Toolkit {#about-driver-toolkit_driver-toolkit} The Driver Toolkit is a container image in the OpenShift Container Platform release that provides kernel packages and tools for building driver containers. Its package versions match the kernel running on Red Hat Enterprise Linux CoreOS (RHCOS) nodes in each release. Driver containers are container images used for building and deploying out-of-tree kernel modules and drivers on container operating systems like RHCOS. Kernel modules and drivers are software libraries running with a high level of privilege in the operating system kernel. They extend the kernel functionalities or provide the hardware-specific code required to control new devices. Examples include hardware devices like Field Programmable Gate Arrays (FPGA) or GPUs, and software-defined storage (SDS) solutions, such as Lustre parallel file systems, which require kernel modules on client machines. Driver containers are the first layer of the software stack used to enable these technologies on Kubernetes. The list of kernel packages in the Driver Toolkit includes the following and their dependencies: - `kernel-core` - `kernel-devel` - `kernel-headers` - `kernel-modules` - `kernel-modules-extra` In addition, the Driver Toolkit also includes the corresponding real-time kernel packages: - `kernel-rt-core` - `kernel-rt-devel` - `kernel-rt-modules` - `kernel-rt-modules-extra` The Driver Toolkit also has several tools that are commonly needed to build and install kernel modules, including: - `elfutils-libelf-devel` - `kmod` - `binutilskabi-dw` - `kernel-abi-whitelists` - dependencies for the above Before the Driver Toolkit’s existence, users would install kernel packages in a pod or build config on OpenShift Container Platform using entitled builds or by installing from the kernel RPMs in the host’s `machine-os-content`. The Driver Toolkit simplifies the process by removing the entitlement step, and avoids the privileged operation of accessing the machine-os-content in a pod. The Driver Toolkit can also be used by partners who have access to pre-released OpenShift Container Platform versions to prebuild driver-containers for their hardware devices for future OpenShift Container Platform releases. The Driver Toolkit is also used by the Kernel Module Management (KMM), which is currently available as a community Operator in the software catalog. KMM supports out-of-tree and third-party kernel drivers and the support software for the underlying operating system. Users can create modules for KMM to build and deploy a driver container, as well as support software like a device plugin, or metrics. Modules can include a build config to build a driver container-based on the Driver Toolkit, or KMM can deploy a prebuilt driver container. **Additional resources** {._additional-resources} - [How to use entitled image builds to build DriverContainers with UBI on OpenShift](https://www.redhat.com/en/blog/how-to-use-entitled-image-builds-to-build-drivercontainers-with-ubi-on-openshift) ## Pulling the Driver Toolkit container image {#pulling-the-driver-toolkit_driver-toolkit} You can pull the `driver-toolkit` image from the Red Hat Ecosystem Catalog or extract its URL from the OpenShift Container Platform release payload by using the `oc adm` CLI. The `driver-toolkit` image is available from the [Container images section of the Red Hat Ecosystem Catalog](https://registry.redhat.io/) and in the OpenShift Container Platform release payload. The image corresponding to the most recent minor release of OpenShift Container Platform will be tagged with the version number in the catalog. The image URL for a specific release can be found using the `oc adm` CLI command. Instructions for pulling the `driver-toolkit` image from `registry.redhat.io` with `podman` or in OpenShift Container Platform can be found on the [Red Hat Ecosystem Catalog](https://catalog.redhat.com/software/containers/openshift4/driver-toolkit-rhel8/604009d6122bd89307e00865?container-tabs=gti). The driver-toolkit image for the latest minor release is tagged with the minor release version on `registry.redhat.io`, for example: `registry.redhat.io/openshift4/driver-toolkit-rhel8:v4.22`. **Prerequisites** - You obtained the image [pull secret from Red Hat OpenShift Cluster Manager](https://console.redhat.com/openshift/install/pull-secret). - You installed the OpenShift CLI (`oc`). **Procedure** 1. Use the `oc adm` command to extract the image URL of the `driver-toolkit` corresponding to a certain release: - For an x86 image, the command is as follows: ```terminal $ oc adm release info quay.io/openshift-release-dev/ocp-release:4.22.z-x86_64 --image-for=driver-toolkit ``` - For an ARM image, the command is as follows: ```terminal $ oc adm release info quay.io/openshift-release-dev/ocp-release:4.22.z-aarch64 --image-for=driver-toolkit ``` ```terminal {title="Example output"} quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b53883ca2bac5925857148c4a1abc300ced96c222498e3bc134fe7ce3a1dd404 ``` 2. Obtain this image using a valid pull secret, such as the pull secret required to install OpenShift Container Platform: ```terminal $ podman pull --authfile=path/to/pullsecret.json quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256: ``` ## Using the Driver Toolkit {#using-the-driver-toolkit_driver-toolkit} Build and deploy the `simple-kmod` example kernel module by using the Driver Toolkit as a base image to learn the driver container workflow on an OpenShift Container Platform cluster. > [!NOTE] > The Driver Toolkit includes the necessary dependencies, `openssl`, `mokutil`, and `keyutils`, needed to sign a kernel module. However, in this example, the `simple-kmod` kernel module is not signed and therefore cannot be loaded on systems with `Secure Boot` enabled. **Prerequisites** - You have a running OpenShift Container Platform cluster. - You set the Image Registry Operator state to `Managed` for your cluster. - You installed the OpenShift CLI (`oc`). - You are logged into the OpenShift CLI as a user with `cluster-admin` privileges. **Procedure** 1. Create a namespace. For example: ```terminal $ oc new-project simple-kmod-demo ``` 2. The YAML defines an `ImageStream` for storing the `simple-kmod` driver container image, and a `BuildConfig` for building the container. Save this YAML as `0000-buildconfig.yaml.template`. ```yaml apiVersion: image.openshift.io/v1 kind: ImageStream metadata: labels: app: simple-kmod-driver-container name: simple-kmod-driver-container namespace: simple-kmod-demo spec: {} --- apiVersion: build.openshift.io/v1 kind: BuildConfig metadata: labels: app: simple-kmod-driver-build name: simple-kmod-driver-build namespace: simple-kmod-demo spec: nodeSelector: node-role.kubernetes.io/worker: "" runPolicy: "Serial" triggers: - type: "ConfigChange" - type: "ImageChange" source: dockerfile: | ARG DTK FROM ${DTK} as builder ARG KVER WORKDIR /build/ RUN git clone https://github.com/openshift-psap/simple-kmod.git WORKDIR /build/simple-kmod RUN make all install KVER=${KVER} FROM registry.redhat.io/ubi8/ubi-minimal ARG KVER # Required for installing `modprobe` RUN microdnf install kmod COPY --from=builder /lib/modules/${KVER}/simple-kmod.ko /lib/modules/${KVER}/ COPY --from=builder /lib/modules/${KVER}/simple-procfs-kmod.ko /lib/modules/${KVER}/ RUN depmod ${KVER} strategy: dockerStrategy: buildArgs: - name: KMODVER value: DEMO # $ oc adm release info quay.io/openshift-release-dev/ocp-release:-x86_64 --image-for=driver-toolkit - name: DTK value: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:34864ccd2f4b6e385705a730864c04a40908e57acede44457a783d739e377cae - name: KVER value: 4.18.0-372.26.1.el8_6.x86_64 output: to: kind: ImageStreamTag name: simple-kmod-driver-container:demo ``` 3. Substitute the correct driver toolkit image for the OpenShift Container Platform version you are running in place of “DRIVER_TOOLKIT_IMAGE” with the following commands. ```terminal $ OCP_VERSION=$(oc get clusterversion/version -ojsonpath={.status.desired.version}) ``` ```terminal $ DRIVER_TOOLKIT_IMAGE=$(oc adm release info $OCP_VERSION --image-for=driver-toolkit) ``` ```terminal $ sed "s#DRIVER_TOOLKIT_IMAGE#${DRIVER_TOOLKIT_IMAGE}#" 0000-buildconfig.yaml.template > 0000-buildconfig.yaml ``` 4. Create the image stream and build config with ```terminal $ oc create -f 0000-buildconfig.yaml ``` 5. After the builder pod completes successfully, deploy the driver container image as a `DaemonSet`. 1. The driver container must run with the privileged security context in order to load the kernel modules on the host. The following YAML file contains the RBAC rules and the `DaemonSet` for running the driver container. Save this YAML as `1000-drivercontainer.yaml`. ```yaml apiVersion: v1 kind: ServiceAccount metadata: name: simple-kmod-driver-container --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: simple-kmod-driver-container rules: - apiGroups: - security.openshift.io resources: - securitycontextconstraints verbs: - use resourceNames: - privileged --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: simple-kmod-driver-container roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: simple-kmod-driver-container subjects: - kind: ServiceAccount name: simple-kmod-driver-container userNames: - system:serviceaccount:simple-kmod-demo:simple-kmod-driver-container --- apiVersion: apps/v1 kind: DaemonSet metadata: name: simple-kmod-driver-container spec: selector: matchLabels: app: simple-kmod-driver-container template: metadata: labels: app: simple-kmod-driver-container spec: serviceAccount: simple-kmod-driver-container serviceAccountName: simple-kmod-driver-container containers: - image: image-registry.openshift-image-registry.svc:5000/simple-kmod-demo/simple-kmod-driver-container:demo name: simple-kmod-driver-container imagePullPolicy: Always command: [sleep, infinity] lifecycle: postStart: exec: command: ["modprobe", "-v", "-a" , "simple-kmod", "simple-procfs-kmod"] preStop: exec: command: ["modprobe", "-r", "-a" , "simple-kmod", "simple-procfs-kmod"] securityContext: privileged: true nodeSelector: node-role.kubernetes.io/worker: "" ``` 2. Create the RBAC rules and daemon set: ```terminal $ oc create -f 1000-drivercontainer.yaml ``` 6. After the pods are running on the worker nodes, verify that the `simple_kmod` kernel module is loaded successfully on the host machines with `lsmod`. 1. Verify that the pods are running: ```terminal $ oc get pod -n simple-kmod-demo ``` ```terminal {title="Example output"} NAME READY STATUS RESTARTS AGE simple-kmod-driver-build-1-build 0/1 Completed 0 6m simple-kmod-driver-container-b22fd 1/1 Running 0 40s simple-kmod-driver-container-jz9vn 1/1 Running 0 40s simple-kmod-driver-container-p45cc 1/1 Running 0 40s ``` 2. Execute the `lsmod` command in the driver container pod: ```terminal $ oc exec -it pod/simple-kmod-driver-container-p45cc -- lsmod | grep simple ``` ```terminal {title="Example output"} simple_procfs_kmod 16384 0 simple_kmod 16384 0 ``` **Additional resources** {._additional-resources} - [Image Registry Operator in OpenShift Container Platform](/openshift-docs-markdown/registry/configuring-registry-operator#registry-removed_configuring-registry-operator)