---
title: Configuring the registry for vSphere
---

# Configuring the registry for vSphere {#configuring-registry-storage-vsphere}

Configure image registry storage for vSphere clusters after installation. Because vSphere installations do not automatically provision storage, you must change the registry management state from `Removed` to `Managed` and configure persistent storage or use Red Hat OpenShift Data Foundation before the registry can store container images.

## Image registry removed during installation {#registry-removed_configuring-registry-storage-vsphere}

On platforms that do not provide shareable object storage, the OpenShift Image Registry Operator bootstraps itself as `Removed`. This allows `openshift-installer` to complete installations on these platform types.

After installation, you must edit the Image Registry Operator configuration to switch the `managementState` from `Removed` to `Managed`. When this has completed, you must configure storage.

## Changing the image registry’s management state {#registry-change-management-state_configuring-registry-storage-vsphere}

To start the image registry, you must change the Image Registry Operator configuration’s `managementState` from `Removed` to `Managed`.

**Procedure**

- Change `managementState` Image Registry Operator configuration from `Removed` to `Managed`. For example:

  ```terminal
  $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'
  ```

## Image registry storage configuration {#installation-registry-storage-config_configuring-registry-storage-vsphere}

The Image Registry Operator is not initially available for platforms that do not provide default storage. After installation, you must configure your registry to use storage so that the Registry Operator is made available.

Configure a persistent volume, which is required for production clusters. Where applicable, you can configure an empty directory as the storage location for non-production clusters.

You can also allow the image registry to use block storage types by using the `Recreate` rollout strategy during upgrades.

### Configuring registry storage for VMware vSphere {#registry-configuring-storage-vsphere_configuring-registry-storage-vsphere}

As a cluster administrator, following installation you must configure your registry to use storage.

**Prerequisites**

- Cluster administrator permissions.
- A cluster on VMware vSphere.
- Persistent storage provisioned for your cluster, such as Red Hat OpenShift Data Foundation.

  > [!IMPORTANT]
  > OpenShift Container Platform supports `ReadWriteOnce` access for image registry storage when you have only one replica. `ReadWriteOnce` access also requires that the registry uses the `Recreate` rollout strategy. To deploy an image registry that supports high availability with two or more replicas, `ReadWriteMany` access is required.
- Must have "100Gi" capacity.

> [!IMPORTANT]
> Testing shows issues with using the NFS server on RHEL as storage backend for core services. This includes the OpenShift Container Registry and Quay, Prometheus for monitoring storage, and Elasticsearch for logging storage. Therefore, using RHEL NFS to back PVs used by core services is not recommended.
>
> Other NFS implementations on the marketplace might not have these issues. Contact the individual NFS implementation vendor for more information on any testing that was possibly completed against these OpenShift Container Platform core components.

**Procedure**

1. Change the `spec.storage.pvc` field in the `configs.imageregistry/cluster` resource.

   > [!NOTE]
   > When you use shared storage, review your security settings to prevent outside access.
2. Verify that you do not have a registry pod by running the following command:

   ```terminal
   $ oc get pod -n openshift-image-registry -l docker-registry=default
   ```

   ```terminal {title="Example output"}
   No resourses found in openshift-image-registry namespace
   ```

   > [!NOTE]
   > If you do have a registry pod in your output, you do not need to continue with this procedure.
3. Check the registry configuration by running the following command:

   ```terminal
   $ oc edit configs.imageregistry.operator.openshift.io
   ```

   ```yaml {title="Example output"}
   storage:
     pvc:
       claim:
   ```

   Leave the `claim` field blank to allow the automatic creation of an `image-registry-storage` persistent volume claim (PVC). The PVC is generated based on the default storage class. However, be aware that the default storage class might provide ReadWriteOnce (RWO) volumes, such as a RADOS Block Device (RBD), which can cause issues when you replicate to more than one replica.
4. Check the `clusteroperator` status by running the following command:

   ```terminal
   $ oc get clusteroperator image-registry
   ```

   ```terminal {title="Example output"}
   NAME             VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
   image-registry   4.7       True        False         False      6h50m
   ```

### Configuring storage for the image registry in non-production clusters {#installation-registry-storage-non-production_configuring-registry-storage-vsphere}

You must configure storage for the Image Registry Operator. For non-production clusters, you can set the image registry to an empty directory, but you lose all images if you restart the registry.

**Procedure**

- To set the image registry storage to an empty directory:

  ```terminal
  $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'
  ```

  > [!WARNING]
  > Configure this option only for non-production clusters.

  If you run this command before the Image Registry Operator initializes its components, the `oc patch` command fails with the following error:

  ```terminal {title="Example output"}
  Error from server (NotFound): configs.imageregistry.operator.openshift.io "cluster" not found
  ```

  Wait a few minutes and run the command again.

### Configuring block registry storage for VMware vSphere {#installation-registry-storage-block-recreate-rollout_configuring-registry-storage-vsphere}

To allow the image registry to use block storage types such as vSphere Virtual Machine Disk (VMDK) during upgrades as a cluster administrator, you can use the `Recreate` rollout strategy.

> [!IMPORTANT]
> Block storage volumes are supported but not recommended for use with image registry on production clusters. An installation where the registry is configured on block storage is not highly available because the registry cannot have more than one replica.

**Procedure**

1. Enter the following command to set the image registry storage as a block storage type, patch the registry so that it uses the `Recreate` rollout strategy, and runs with only `1` replica:

   ```terminal
   $ oc patch config.imageregistry.operator.openshift.io/cluster --type=merge -p '{"spec":{"rolloutStrategy":"Recreate","replicas":1}}'
   ```
2. Provision the persistent volume (PV) for the block storage device, and create a persistent volume claim (PVC) for that volume. The requested block volume uses the ReadWriteOnce (RWO) access mode.

   1. Create a `pvc.yaml` file with the following contents to define a VMware vSphere `PersistentVolumeClaim` object:

      ```yaml
      kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: image-registry-storage
        namespace: openshift-image-registry
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 100Gi
      ```

      where:

`metadata.name`
:   Specifies a unique name that represents the `PersistentVolumeClaim` object.

`metadata.namespace`
:   Specifies the `namespace` for the `PersistentVolumeClaim` object, which is `openshift-image-registry`.

`spec.accessModes`
:   Specifies the access mode of the persistent volume claim. With `ReadWriteOnce`, the volume can be mounted with read and write permissions by a single node.

`spec.resources.requests.storage`
:   Specifies the size of the persistent volume claim.

1. Enter the following command to create the `PersistentVolumeClaim` object from the file:

   ```terminal
   $ oc create -f pvc.yaml -n openshift-image-registry
   ```

   1. Enter the following command to edit the registry configuration so that it references the correct PVC:

      ```terminal
      $ oc edit config.imageregistry.operator.openshift.io -o yaml
      ```

      ```yaml {title="Example output"}
      storage:
        pvc:
          claim:
      ```

      By creating a custom PVC, you can leave the `claim` field blank for the default automatic creation of an `image-registry-storage` PVC.

### Configuring the Image Registry Operator to use Ceph RGW storage with Red Hat OpenShift Data Foundation {#registry-configuring-registry-storage-rhodf-cephrgw_configuring-registry-storage-vsphere}

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

- Ceph, a shared and distributed file system and on-premise object storage
- NooBaa, providing a Multicloud Object Gateway

Use the following, procedure to configure the image registry to use Ceph RGW storage.

**Prerequisites**

- You have access to the cluster as a user with the `cluster-admin` role.
- You have access to the OpenShift Container Platform web console.
- You installed the `oc` CLI.
- You installed the [OpenShift Data Foundation Operator](https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/latest) to provide object storage and Ceph RGW object storage.

**Procedure**

1. Create the object bucket claim using the `ocs-storagecluster-ceph-rgw` storage class. For example:

   ```terminal
   cat <<EOF | oc apply -f -
   apiVersion: objectbucket.io/v1alpha1
   kind: ObjectBucketClaim
   metadata:
     name: rgwbucket
     namespace: openshift-storage
   spec:
     storageClassName: ocs-storagecluster-ceph-rgw
     generateBucketName: rgwbucket
   EOF
   ```

   Alternatively, you can use the `openshift-image-registry` for the `namespace` value.
2. Get the bucket name by entering the following command:

   ```terminal
   $ bucket_name=$(oc get obc -n openshift-storage rgwbucket -o jsonpath='{.spec.bucketName}')
   ```
3. Get the AWS credentials by entering the following commands:

   ```terminal
   $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 --decode)
   ```

   ```terminal
   $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode)
   ```
4. Create the secret `image-registry-private-configuration-user` with the AWS credentials for the new bucket under `openshift-image-registry project` by entering the following command:

   ```terminal
   $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
   ```
5. Get the `route` host by entering the following command:

   ```terminal
   $ route_host=$(oc get route ocs-storagecluster-cephobjectstore -n openshift-storage --template='{{ .spec.host }}')
   ```
6. Create a config map that uses an ingress certificate by entering the following commands:

   ```terminal
   $ oc extract secret/$(oc get ingresscontroller -n openshift-ingress-operator default -o json | jq '.spec.defaultCertificate.name // "router-certs-default"' -r) -n openshift-ingress --confirm
   ```

   ```terminal
   $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
   ```
7. Configure the image registry to use the Ceph RGW object storage by entering the following command:

   ```terminal
   $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge
   ```

### Configuring the Image Registry Operator to use Noobaa storage with Red Hat OpenShift Data Foundation {#registry-configuring-registry-storage-rhodf-nooba_configuring-registry-storage-vsphere}

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

- Ceph, a shared and distributed file system and on-premise object storage
- NooBaa, providing a Multicloud Object Gateway

Use the following the procedure to configure the image registry to use Noobaa storage.

**Prerequisites**

- You have access to the cluster as a user with the `cluster-admin` role.
- You have access to the OpenShift Container Platform web console.
- You installed the `oc` CLI.
- You installed the [OpenShift Data Foundation Operator](https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/latest) to provide object storage and Noobaa object storage.

**Procedure**

1. Create the object bucket claim using the `openshift-storage.noobaa.io` storage class. For example:

   ```terminal
   cat <<EOF | oc apply -f -
   apiVersion: objectbucket.io/v1alpha1
   kind: ObjectBucketClaim
   metadata:
     name: noobaatest
     namespace: openshift-storage
   spec:
     storageClassName: openshift-storage.noobaa.io
     generateBucketName: noobaatest
   EOF
   ```

   Alternatively, you can use the `openshift-image-registry` for the `namespace` value.
2. Get the bucket name by entering the following command:

   ```terminal
   $ bucket_name=$(oc get obc -n openshift-storage noobaatest -o jsonpath='{.spec.bucketName}')
   ```
3. Get the AWS credentials by entering the following commands:

   ```terminal
   $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_ACCESS_KEY_ID:" | head -n1 | awk '{print $2}' | base64 --decode)
   ```

   ```terminal
   $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_SECRET_ACCESS_KEY:" | head -n1 | awk '{print $2}' | base64 --decode)
   ```
4. Create the secret `image-registry-private-configuration-user` with the AWS credentials for the new bucket under `openshift-image-registry project` by entering the following command:

   ```terminal
   $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry
   ```
5. Get the route host by entering the following command:

   ```terminal
   $ route_host=$(oc get route s3 -n openshift-storage -o=jsonpath='{.spec.host}')
   ```
6. Create a config map that uses an ingress certificate by entering the following commands:

   ```terminal
   $ oc extract secret/$(oc get ingresscontroller -n openshift-ingress-operator default -o json | jq '.spec.defaultCertificate.name // "router-certs-default"' -r) -n openshift-ingress --confirm
   ```

   ```terminal
   $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config
   ```
7. Configure the image registry to use the Nooba object storage by entering the following command:

   ```terminal
   $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge
   ```

## Configuring the Image Registry Operator to use CephFS storage with Red Hat OpenShift Data Foundation {#registry-configuring-registry-storage-rhodf-cephfs_configuring-registry-storage-vsphere}

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

- Ceph, a shared and distributed file system and on-premise object storage
- NooBaa, providing a Multicloud Object Gateway

Use the following procedure to configure the image registry to use CephFS storage.

> [!NOTE]
> CephFS uses persistent volume claim (PVC) storage. It is not recommended to use PVCs for image registry storage if there are other options are available, such as Ceph RGW or Noobaa.

**Prerequisites**

- You have access to the cluster as a user with the `cluster-admin` role.
- You have access to the OpenShift Container Platform web console.
- You installed the `oc` CLI.
- You installed the [OpenShift Data Foundation Operator](https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/latest) to provide object storage and CephFS file storage.

**Procedure**

1. Create a PVC to use the `cephfs` storage class. For example:

   ```terminal
   cat <<EOF | oc apply -f -
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
    name: registry-storage-pvc
    namespace: openshift-image-registry
   spec:
    accessModes:
    - ReadWriteMany
    resources:
      requests:
        storage: 100Gi
    storageClassName: ocs-storagecluster-cephfs
   EOF
   ```
2. Configure the image registry to use the CephFS file system storage by entering the following command:

   ```terminal
   $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","pvc":{"claim":"registry-storage-pvc"}}}}' --type=merge
   ```

**Additional resources**
{._additional-resources}

- [Configuring the registry for vSphere](/openshift-docs-markdown/registry/configuring_registry_storage/configuring-registry-storage-vsphere#registry-configuring-storage-vsphere_configuring-registry-storage-vsphere)
- [Recommended configurable storage technology](/openshift-docs-markdown/scalability_and_performance/optimization/optimizing-storage#optimizing-storage)
- [Configuring Image Registry to use OpenShift Data Foundation](https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/latest/html-single/managing_and_allocating_storage_resources/index#configuring-image-registry-to-use-openshift-data-foundation_rhodf)