--- title: Changing the cloud provider credentials configuration --- # Changing the cloud provider credentials configuration {#changing-cloud-credentials-configuration} You can change your cluster’s cloud provider credentials configuration to meet security and authentication requirements. You can rotate or remove credentials, or enable supported short-term credential methods. For supported configurations, you can change how OpenShift Container Platform authenticates with your cloud provider. To determine which cloud credentials strategy your cluster uses, see "Determining the Cloud Credential Operator mode". ## Rotating cloud provider service keys with the Cloud Credential Operator utility {#ccoctl-rotate-cloud-creds_changing-cloud-credentials-configuration} Some organizations require the rotation of the service keys that authenticate the cluster. You can use the Cloud Credential Operator (CCO) utility (`ccoctl`) to update keys for clusters installed on the following cloud providers: - Amazon Web Services (AWS) with Security Token Service (STS) - Google Cloud with GCP Workload Identity - Microsoft Azure with Workload ID - IBM Cloud ### Rotating AWS OIDC bound service account signer keys {#rotating-bound-service-keys_key-rotation-aws} You can rotate the bound service account signer key for an OpenShift Container Platform cluster on Amazon Web Services (AWS) that uses the Cloud Credential Operator (CCO) in manual mode with STS. To rotate the key, you delete the existing key on your cluster, which causes the Kubernetes API server to create a new key. To reduce authentication failures during this process, you must immediately add the new public key to the existing issuer file. After the cluster is using the new key for authentication, you can remove any remaining keys. > [!IMPORTANT] > The process to rotate OIDC bound service account signer keys is disruptive and takes a significant amount of time. Some steps are time-sensitive. Before proceeding, observe the following considerations: > > - Read the following steps and ensure that you understand and accept the time requirement. The exact time requirement varies depending on the individual cluster, but it is likely to require at least one hour. > - To reduce the risk of authentication failures, ensure that you understand and prepare for the time-sensitive steps. > - During this process, you must refresh all service accounts and restart all pods on the cluster. These actions are disruptive to workloads. To mitigate this impact, you can temporarily halt these services and then redeploy them when the cluster is ready. **Prerequisites** - You have access to the OpenShift CLI (`oc`) as a user with the `cluster-admin` role. - You have created an AWS account for the `ccoctl` utility to use with the following permissions: - `s3:GetObject` - `s3:PutObject` - `s3:PutObjectTagging` - For clusters that store the OIDC configuration in a private S3 bucket that is accessed by the IAM identity provider through a public CloudFront distribution URL, the AWS account that runs the `ccoctl` utility requires the `cloudfront:ListDistributions` permission. - You have configured the `ccoctl` utility. - Your cluster is in a stable state. You can confirm that the cluster is stable by running the following command: ```terminal $ oc adm wait-for-stable-cluster --minimum-stable-period=5s ``` **Procedure** 1. Configure the following environment variables: ```text INFRA_ID=$(oc get infrastructures cluster -o jsonpath='{.status.infrastructureName}') CLUSTER_NAME=${INFRA_ID%-*} ``` where: `CLUSTER_NAME` : This value should match the name of the cluster that was specified in the `metadata.name` field of the `install-config.yaml` file during installation. > [!NOTE] > Your cluster might differ from this example, and the resource names might not be derived identically from the cluster name. Ensure that you specify the correct corresponding resource names for your cluster. - For AWS clusters that store the OIDC configuration in a public S3 bucket, configure the following environment variable: ```text AWS_BUCKET=$(oc get authentication cluster -o jsonpath={'.spec.serviceAccountIssuer'} | awk -F'://' '{print$2}' |awk -F'.' '{print$1}') ``` - For AWS clusters that store the OIDC configuration in a private S3 bucket that is accessed by the IAM identity provider through a public CloudFront distribution URL, complete the following steps: 1. Extract the public CloudFront distribution URL by running the following command: ```terminal $ basename $(oc get authentication cluster -o jsonpath={'.spec.serviceAccountIssuer'} ) ``` ```text {title="Example output"} .cloudfront.net ``` where `` is an alphanumeric string. 2. Determine the private S3 bucket name by running the following command: ```terminal $ aws cloudfront list-distributions --query "DistributionList.Items[].{DomainName: DomainName, OriginDomainName: Origins.Items[0].DomainName}[?contains(DomainName, '.cloudfront.net')]" ``` ```text {title="Example output"} [ { "DomainName": ".cloudfront.net", "OriginDomainName": ".s3.us-east-2.amazonaws.com" } ] ``` where `` is the private S3 bucket name for your cluster. 3. Configure the following environment variable: ```text AWS_BUCKET=$ ``` where `` is the private S3 bucket name for your cluster. 2. Create a temporary directory to use and assign it an environment variable by running the following command: ```terminal $ TEMPDIR=$(mktemp -d) ``` 3. To cause the Kubernetes API server to create a new bound service account signing key, you delete the next bound service account signing key. > [!IMPORTANT] > After you complete this step, the Kubernetes API server starts to roll out a new key. To reduce the risk of authentication failures, complete the remaining steps as quickly as possible. The remaining steps might be disruptive to workloads. When you are ready, delete the next bound service account signing key by running the following command: ```terminal $ oc delete secrets/next-bound-service-account-signing-key \ -n openshift-kube-apiserver-operator ``` 4. Download the public key from the service account signing key secret that the Kubernetes API server created by running the following command: ```terminal $ oc get secret/next-bound-service-account-signing-key \ -n openshift-kube-apiserver-operator \ -ojsonpath='{ .data.service-account\.pub }' | base64 \ -d > ${TEMPDIR}/serviceaccount-signer.public ``` 5. Use the public key to create a `keys.json` file by running the following command: ```terminal $ ccoctl aws create-identity-provider \ --dry-run \ --output-dir ${TEMPDIR} \ --public-key-file=${TEMPDIR}/serviceaccount-signer.public \ --name fake \ --region us-east-1 ``` where: `--dry-run` : The dry run mode outputs files, including the new `keys.json` file, to the disk without making API calls. `--public-key-file` : The path to the public key that you downloaded in the previous step. `--name` : Some parameters do not require real values because the `--dry-run` option does not make any API calls. `--region` : Any valid AWS region, such as `us-east-1`. This value does not need to match the region the cluster is in. 6. Rename the `keys.json` file by running the following command: ```terminal $ cp ${TEMPDIR}/-keys.json ${TEMPDIR}/jwks.new.json ``` where `` is a two-digit numerical value that varies depending on your environment. 7. Download the existing `keys.json` file from the cloud provider by running the following command: ```terminal $ aws s3api get-object \ --bucket ${AWS_BUCKET} \ --key keys.json ${TEMPDIR}/jwks.current.json ``` 8. Combine the two `keys.json` files by running the following command: ```terminal $ jq -s '{ keys: map(.keys[])}' ${TEMPDIR}/jwks.current.json ${TEMPDIR}/jwks.new.json > ${TEMPDIR}/jwks.combined.json ``` 9. To enable authentication for the old and new keys during the rotation, upload the combined `keys.json` file to the cloud provider by running the following command: ```terminal $ aws s3api put-object \ --bucket ${AWS_BUCKET} \ --tagging "openshift.io/cloud-credential-operator/${CLUSTER_NAME}=owned" \ --key keys.json \ --body ${TEMPDIR}/jwks.combined.json ``` 10. Wait for the Kubernetes API server to update and use the new key. You can monitor the update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 11. To ensure that all pods on the cluster use the new key, you must restart them. > [!IMPORTANT] > This step maintains uptime for services that are configured for high availability across multiple nodes, but might cause downtime for any services that are not. Restart all of the pods in the cluster by running the following command: ```terminal $ oc adm reboot-machine-config-pool mcp/worker mcp/master ``` 12. Monitor the restart and update process by running the following command: ```terminal $ oc adm wait-for-node-reboot nodes --all ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All nodes rebooted ``` 13. Monitor the update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 14. Replace the combined `keys.json` file with the updated `keys.json` file on the cloud provider by running the following command: ```terminal $ aws s3api put-object \ --bucket ${AWS_BUCKET} \ --tagging "openshift.io/cloud-credential-operator/${CLUSTER_NAME}=owned" \ --key keys.json \ --body ${TEMPDIR}/jwks.new.json ``` ### Rotating Google Cloud OIDC bound service account signer keys {#_rotating_gcp_short_oidc_bound_service_account_signer_keys} You can rotate the bound service account signer key for an OpenShift Container Platform cluster on Google Cloud that uses the Cloud Credential Operator (CCO) in manual mode with GCP Workload Identity. To rotate the key, you delete the existing key on your cluster, which causes the Kubernetes API server to create a new key. To reduce authentication failures during this process, you must immediately add the new public key to the existing issuer file. After the cluster is using the new key for authentication, you can remove any remaining keys. > [!IMPORTANT] > The process to rotate OIDC bound service account signer keys is disruptive and takes a significant amount of time. Some steps are time-sensitive. Before proceeding, observe the following considerations: > > - Read the following steps and ensure that you understand and accept the time requirement. The exact time requirement varies depending on the individual cluster, but it is likely to require at least one hour. > - To reduce the risk of authentication failures, ensure that you understand and prepare for the time-sensitive steps. > - During this process, you must refresh all service accounts and restart all pods on the cluster. These actions are disruptive to workloads. To mitigate this impact, you can temporarily halt these services and then redeploy them when the cluster is ready. **Prerequisites** - You have access to the OpenShift CLI (`oc`) as a user with the `cluster-admin` role. - You have added one of the following authentication options to the Google Cloud account that the `ccoctl` utility uses: - The **IAM Workload Identity Pool Admin** role - The following granular permissions: - `storage.objects.create` - `storage.objects.delete` - You have configured the `ccoctl` utility. - Your cluster is in a stable state. You can confirm that the cluster is stable by running the following command: ```terminal $ oc adm wait-for-stable-cluster --minimum-stable-period=5s ``` **Procedure** 1. Configure the following environment variables: ```text CURRENT_ISSUER=$(oc get authentication cluster -o jsonpath='{.spec.serviceAccountIssuer}') GCP_BUCKET=$(echo ${CURRENT_ISSUER} | cut -d "/" -f4) CLUSTER_NAME=${GCP_BUCKET%-*} ``` > [!NOTE] > Your cluster might differ from this example, and the resource names might not be derived identically from the cluster name. Ensure that you specify the correct corresponding resource names for your cluster. 2. Create a temporary directory to use and assign it an environment variable by running the following command: ```terminal $ TEMPDIR=$(mktemp -d) ``` 3. To cause the Kubernetes API server to create a new bound service account signing key, you delete the next bound service account signing key. > [!IMPORTANT] > After you complete this step, the Kubernetes API server starts to roll out a new key. To reduce the risk of authentication failures, complete the remaining steps as quickly as possible. The remaining steps might be disruptive to workloads. When you are ready, delete the next bound service account signing key by running the following command: ```terminal $ oc delete secrets/next-bound-service-account-signing-key \ -n openshift-kube-apiserver-operator ``` 4. Download the public key from the service account signing key secret that the Kubernetes API server created by running the following command: ```terminal $ oc get secret/next-bound-service-account-signing-key \ -n openshift-kube-apiserver-operator \ -ojsonpath='{ .data.service-account\.pub }' | base64 \ -d > ${TEMPDIR}/serviceaccount-signer.public ``` 5. Use the public key to create a `keys.json` file by running the following command: ```terminal $ ccoctl gcp create-workload-identity-provider \ --dry-run \ --output-dir=${TEMPDIR} \ --public-key-file=${TEMPDIR}/serviceaccount-signer.public \ --name fake \ --project fake \ --workload-identity-pool fake ``` where: `--dry-run` : The dry run mode outputs files, including the new `keys.json` file, to the disk without making API calls. `--public-key-file` : The path to the public key that you downloaded in the previous step. `--name` : Some parameters do not require real values because the `--dry-run` option does not make any API calls. 6. Rename the `keys.json` file by running the following command: ```terminal $ cp ${TEMPDIR}/-keys.json ${TEMPDIR}/jwks.new.json ``` where `` is a two-digit numerical value that varies depending on your environment. 7. Download the existing `keys.json` file from the cloud provider by running the following command: - For Google Cloud clusters that store OIDC keys in a public bucket, run the following command: ```terminal $ gcloud storage cp gs://${GCP_BUCKET}/keys.json ${TEMPDIR}/jwks.current.json ``` - For Google Cloud clusters that attach OIDC keys directly to the workload identity pool, run the following command: ```terminal $ gcloud iam workload-identity-pools providers describe \ --format json \ --location global \ --workload-identity-pool ${CLUSTER_NAME} ${CLUSTER_NAME} \ | jq -r ".oidc.jwksJson" > ${TEMPDIR}/jwks.current.json ``` 8. Combine the two `keys.json` files by running the following command: ```terminal $ jq -s '{ keys: map(.keys[])}' ${TEMPDIR}/jwks.current.json ${TEMPDIR}/jwks.new.json > ${TEMPDIR}/jwks.combined.json ``` 9. To enable authentication for the old and new keys during the rotation, upload the combined `keys.json` file to the cloud provider by running the following command: - For Google Cloud clusters that store OIDC keys in a public bucket, run the following command: ```terminal $ gcloud storage cp ${TEMPDIR}/jwks.combined.json gs://${GCP_BUCKET}/keys.json ``` - For Google Cloud clusters that attach OIDC keys directly to the workload identity pool, run the following command: ```terminal $ gcloud iam workload-identity-pools providers update-oidc ${CLUSTER_NAME} \ --location=global \ --workload-identity-pool=${CLUSTER_NAME} \ --jwk-json-path=${TEMPDIR}/jwks.combined.json ``` 10. Wait for the Kubernetes API server to update and use the new key. You can monitor the update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 11. To ensure that all pods on the cluster use the new key, you must restart them. > [!IMPORTANT] > This step maintains uptime for services that are configured for high availability across multiple nodes, but might cause downtime for any services that are not. Restart all of the pods in the cluster by running the following command: ```terminal $ oc adm reboot-machine-config-pool mcp/worker mcp/master ``` 12. Monitor the restart and update process by running the following command: ```terminal $ oc adm wait-for-node-reboot nodes --all ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All nodes rebooted ``` 13. Monitor the update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 14. Replace the combined `keys.json` file with the updated `keys.json` file on the cloud provider by running the following command: - For Google Cloud clusters that store OIDC keys in a public bucket, run the following command: ```terminal $ gcloud storage cp ${TEMPDIR}/jwks.new.json gs://${GCP_BUCKET}/keys.json ``` - For Google Cloud clusters that attach OIDC keys directly to the workload identity pool, run the following command: ```terminal $ gcloud iam workload-identity-pools providers update-oidc ${CLUSTER_NAME} \ --location=global \ --workload-identity-pool=${CLUSTER_NAME} \ --jwk-json-path=${TEMPDIR}/jwks.new.json ``` ### Rotating Azure OIDC bound service account signer keys {#_rotating_azure_short_oidc_bound_service_account_signer_keys} You can rotate the bound service account signer key for an OpenShift Container Platform cluster on Microsoft Azure that uses the Cloud Credential Operator (CCO) in manual mode with Microsoft Entra Workload ID. To rotate the key, you delete the existing key on your cluster, which causes the Kubernetes API server to create a new key. To reduce authentication failures during this process, you must immediately add the new public key to the existing issuer file. After the cluster is using the new key for authentication, you can remove any remaining keys. > [!IMPORTANT] > The process to rotate OIDC bound service account signer keys is disruptive and takes a significant amount of time. Some steps are time-sensitive. Before proceeding, observe the following considerations: > > - Read the following steps and ensure that you understand and accept the time requirement. The exact time requirement varies depending on the individual cluster, but it is likely to require at least one hour. > - To reduce the risk of authentication failures, ensure that you understand and prepare for the time-sensitive steps. > - During this process, you must refresh all service accounts and restart all pods on the cluster. These actions are disruptive to workloads. To mitigate this impact, you can temporarily halt these services and then redeploy them when the cluster is ready. **Prerequisites** - You have access to the OpenShift CLI (`oc`) as a user with the `cluster-admin` role. - You have created a global Azure account for the `ccoctl` utility to use with the following permissions: - `Microsoft.Storage/storageAccounts/listkeys/action` - `Microsoft.Storage/storageAccounts/read` - `Microsoft.Storage/storageAccounts/write` - `Microsoft.Storage/storageAccounts/blobServices/containers/read` - `Microsoft.Storage/storageAccounts/blobServices/containers/write` - You have configured the `ccoctl` utility. - Your cluster is in a stable state. You can confirm that the cluster is stable by running the following command: ```terminal $ oc adm wait-for-stable-cluster --minimum-stable-period=5s ``` **Procedure** 1. Configure the following environment variables: ```text CURRENT_ISSUER=$(oc get authentication cluster -o jsonpath='{.spec.serviceAccountIssuer}') AZURE_STORAGE_ACCOUNT=$(echo ${CURRENT_ISSUER} | cut -d "/" -f3 | cut -d "." -f1) AZURE_STORAGE_CONTAINER=$(echo ${CURRENT_ISSUER} | cut -d "/" -f4) ``` > [!NOTE] > Your cluster might differ from this example, and the resource names might not be derived identically from the cluster name. Ensure that you specify the correct corresponding resource names for your cluster. 2. Create a temporary directory to use and assign it an environment variable by running the following command: ```terminal $ TEMPDIR=$(mktemp -d) ``` 3. To cause the Kubernetes API server to create a new bound service account signing key, you delete the next bound service account signing key. > [!IMPORTANT] > After you complete this step, the Kubernetes API server starts to roll out a new key. To reduce the risk of authentication failures, complete the remaining steps as quickly as possible. The remaining steps might be disruptive to workloads. When you are ready, delete the next bound service account signing key by running the following command: ```terminal $ oc delete secrets/next-bound-service-account-signing-key \ -n openshift-kube-apiserver-operator ``` 4. Download the public key from the service account signing key secret that the Kubernetes API server created by running the following command: ```terminal $ oc get secret/next-bound-service-account-signing-key \ -n openshift-kube-apiserver-operator \ -ojsonpath='{ .data.service-account\.pub }' | base64 \ -d > ${TEMPDIR}/serviceaccount-signer.public ``` 5. Use the public key to create a `keys.json` file by running the following command: ```terminal $ ccoctl aws create-identity-provider \ --dry-run \ --output-dir ${TEMPDIR} \ --public-key-file=${TEMPDIR}/serviceaccount-signer.public \ --name fake \ --region us-east-1 ``` where: `ccoctl aws` : The command does not include a `--dry-run` option. To use the `--dry-run` option, you must specify `aws` for an Azure cluster. `--dry-run` : The dry run mode outputs files, including the new `keys.json` file, to the disk without making API calls. `--public-key-file` : The path to the public key that you downloaded in the previous step. `--name` : Some parameters do not require real values because the `--dry-run` option does not make any API calls. `--region` : Any valid AWS region, such as `us-east-1`. This value does not need to match the region the cluster is in. 6. Rename the `keys.json` file by running the following command: ```terminal $ cp ${TEMPDIR}/-keys.json ${TEMPDIR}/jwks.new.json ``` where `` is a two-digit numerical value that varies depending on your environment. 7. Download the existing `keys.json` file from the cloud provider by running the following command: ```terminal $ az storage blob download \ --container-name ${AZURE_STORAGE_CONTAINER} \ --account-name ${AZURE_STORAGE_ACCOUNT} \ --name 'openid/v1/jwks' \ -f ${TEMPDIR}/jwks.current.json ``` 8. Combine the two `keys.json` files by running the following command: ```terminal $ jq -s '{ keys: map(.keys[])}' ${TEMPDIR}/jwks.current.json ${TEMPDIR}/jwks.new.json > ${TEMPDIR}/jwks.combined.json ``` 9. To enable authentication for the old and new keys during the rotation, upload the combined `keys.json` file to the cloud provider by running the following command: ```terminal $ az storage blob upload \ --overwrite \ --account-name ${AZURE_STORAGE_ACCOUNT} \ --container-name ${AZURE_STORAGE_CONTAINER} \ --name 'openid/v1/jwks' \ -f ${TEMPDIR}/jwks.combined.json ``` 10. Wait for the Kubernetes API server to update and use the new key. You can monitor the update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 11. To ensure that all pods on the cluster use the new key, you must restart them. > [!IMPORTANT] > This step maintains uptime for services that are configured for high availability across multiple nodes, but might cause downtime for any services that are not. Restart all of the pods in the cluster by running the following command: ```terminal $ oc adm reboot-machine-config-pool mcp/worker mcp/master ``` 12. Monitor the restart and update process by running the following command: ```terminal $ oc adm wait-for-node-reboot nodes --all ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All nodes rebooted ``` 13. Monitor the update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 14. Replace the combined `keys.json` file with the updated `keys.json` file on the cloud provider by running the following command: ```terminal $ az storage blob upload \ --overwrite \ --account-name ${AZURE_STORAGE_ACCOUNT} \ --container-name ${AZURE_STORAGE_CONTAINER} \ --name 'openid/v1/jwks' \ -f ${TEMPDIR}/jwks.new.json ``` ### Rotating IBM Cloud credentials {#refreshing-service-ids-ibm-cloud_changing-cloud-credentials-configuration} You can rotate API keys for existing IBM Cloud service IDs and update the corresponding cluster secrets to maintain valid cloud provider credentials. You can rotate API keys for your existing service IDs and update the corresponding secrets. **Prerequisites** - You have configured the `ccoctl` utility. - You have existing service IDs in a live OpenShift Container Platform cluster installed. **Procedure** - Use the `ccoctl` utility to rotate your API keys for the service IDs and update the secrets by running the following command: ```terminal $ ccoctl refresh-keys \ --kubeconfig \ --credentials-requests-dir \ --name ``` where: `` : The name of the provider. For example: `ibmcloud` or `powervs`. `` : The `kubeconfig` file associated with the cluster. For example, `/auth/kubeconfig`. `` : The directory where the credential requests are stored. `` : The name of the OpenShift Container Platform cluster. > [!NOTE] > If your cluster uses Technology Preview features that are enabled by the `TechPreviewNoUpgrade` feature set, you must include the `--enable-tech-preview` parameter. **Additional resources** {._additional-resources} - [Amazon Web Services (AWS) with Security Token Service (STS)](/openshift-docs-markdown/post_installation_configuration/changing-cloud-credentials-configuration#rotating-bound-service-keys_key-rotation-aws) - [Google Cloud with GCP Workload Identity](/openshift-docs-markdown/post_installation_configuration/changing-cloud-credentials-configuration#rotating-bound-service-keys_key-rotation-gcp) - [Microsoft Azure with Workload ID](/openshift-docs-markdown/post_installation_configuration/changing-cloud-credentials-configuration#rotating-bound-service-keys_key-rotation-azure) - [IBM Cloud](/openshift-docs-markdown/post_installation_configuration/changing-cloud-credentials-configuration#refreshing-service-ids-ibm-cloud_changing-cloud-credentials-configuration) ## Rotating cloud provider credentials {#post-install-rotate-cloud-creds_changing-cloud-credentials-configuration} Some organizations require the rotation of the cloud provider credentials. To allow the cluster to use the new credentials, you must update the secrets that the Cloud Credential Operator (CCO) uses to manage cloud provider credentials. ### Rotating cloud provider credentials manually {#manually-rotating-cloud-creds_changing-cloud-credentials-configuration} If your cloud provider credentials are changed for any reason, you must manually update the secret that the Cloud Credential Operator (CCO) uses to manage cloud provider credentials. The process for rotating cloud credentials depends on the mode that the CCO is configured to use. After you rotate credentials for a cluster that is using mint mode, you must manually remove the component credentials that were created by the removed credential. **Prerequisites** - Your cluster is installed on a platform that supports rotating cloud credentials manually with the CCO mode that you are using: - For mint mode, Amazon Web Services (AWS) and Google Cloud are supported. - For passthrough mode, Amazon Web Services (AWS), Microsoft Azure, Google Cloud, Red Hat OpenStack Platform (RHOSP), and VMware vSphere are supported. - You have changed the credentials that are used to interface with your cloud provider. - The new credentials have sufficient permissions for the mode CCO is configured to use in your cluster. **Procedure** 1. In the **Administrator** perspective of the web console, navigate to **Workloads** → **Secrets**. 2. In the table on the **Secrets** page, find the root secret for your cloud provider.
Platform Secret name
AWS aws-creds
Azure azure-credentials
Google Cloud gcp-credentials
RHOSP openstack-credentials
VMware vSphere vsphere-creds
1. Click the Options menu ![](/openshift-docs-markdown/images/kebab.png "Options menu") in the same row as the secret and select **Edit Secret**. 2. Record the contents of the **Value** field or fields. You can use this information to verify that the value is different after updating the credentials. 3. Update the text in the **Value** field or fields with the new authentication information for your cloud provider, and then click **Save**. 4. If you are updating the credentials for a vSphere cluster that does not have the vSphere CSI Driver Operator enabled, you must force a rollout of the Kubernetes controller manager to apply the updated credentials. > [!NOTE] > If the vSphere CSI Driver Operator is enabled, this step is not required. To apply the updated vSphere credentials, log in to the OpenShift Container Platform CLI as a user with the `cluster-admin` role and run the following command: ```terminal $ oc patch kubecontrollermanager cluster \ -p='{"spec": {"forceRedeploymentReason": "recovery-'"$( date )"'"}}' \ --type=merge ``` While the credentials are rolling out, the status of the Kubernetes Controller Manager Operator reports `Progressing=true`. To view the status, run the following command: ```terminal $ oc get co kube-controller-manager ``` 5. If the CCO for your cluster is configured to use mint mode, delete each component secret that is referenced by the individual `CredentialsRequest` objects. 1. Log in to the OpenShift Container Platform CLI as a user with the `cluster-admin` role. 2. Get the names and namespaces of all referenced component secrets: ```terminal $ oc -n openshift-cloud-credential-operator get CredentialsRequest \ -o json | jq -r '.items[] | select (.spec.providerSpec.kind=="") | .spec.secretRef' ``` where `` is the corresponding value for your cloud provider: - AWS: `AWSProviderSpec` - Google Cloud: `GCPProviderSpec` The following example is partial output for the command on an AWS cluster: ```json { "name": "ebs-cloud-credentials", "namespace": "openshift-cluster-csi-drivers" } { "name": "cloud-credential-operator-iam-ro-creds", "namespace": "openshift-cloud-credential-operator" } ``` 3. Delete each of the referenced component secrets: ```terminal $ oc delete secret \ (1) -n (2) ``` where: `` : Specifies the name of a secret. `` : Specifies the namespace that contains the secret. The following example is a command to delete an AWS secret: ```terminal $ oc delete secret ebs-cloud-credentials -n openshift-cluster-csi-drivers ``` You do not need to manually delete the credentials from your provider console. Deleting the referenced component secrets will cause the CCO to delete the existing credentials from the platform and create new ones. **Verification** 1. In the **Administrator** perspective of the web console, navigate to **Workloads** → **Secrets**. 2. Verify that the contents of the **Value** field or fields have changed. **Additional resources** {._additional-resources} - [The Cloud Credential Operator in mint mode](/openshift-docs-markdown/authentication/managing_cloud_provider_credentials/cco-mode-mint#cco-mode-mint) - [The Cloud Credential Operator in passthrough mode](/openshift-docs-markdown/authentication/managing_cloud_provider_credentials/cco-mode-passthrough#cco-mode-passthrough) - [vSphere CSI Driver Operator](/openshift-docs-markdown/storage/container_storage_interface/persistent-storage-csi-vsphere#persistent-storage-csi-vsphere) - [Cloud Credential Operator (CCO)](/openshift-docs-markdown/operators/operator-reference#cloud-credential-operator_cluster-operators-ref) ## Removing cloud provider credentials {#post-install-remove-cloud-creds_changing-cloud-credentials-configuration} After installing OpenShift Container Platform, some organizations require the removal of the cloud provider credentials that were used during the initial installation. To allow the cluster to use the new credentials, you must update the secrets that the Cloud Credential Operator (CCO) uses to manage cloud provider credentials. ### Removing cloud provider credentials {#manually-removing-cloud-creds_changing-cloud-credentials-configuration} You can remove administrator-level cloud provider credentials from a cluster that uses the Cloud Credential Operator in mint mode to reduce the risk of credential exposure after installation. For clusters that use the Cloud Credential Operator (CCO) in mint mode, the administrator-level credential is stored in the `kube-system` namespace. The CCO uses the `admin` credential to process the `CredentialsRequest` objects in the cluster and create users for components with limited permissions. After installing an OpenShift Container Platform cluster with the CCO in mint mode, you can remove the administrator-level credential secret from the `kube-system` namespace in the cluster. The CCO only requires the administrator-level credential during changes that require reconciling new or modified `CredentialsRequest` custom resources, such as minor cluster version updates. > [!NOTE] > Before performing a minor version cluster update (for example, updating from OpenShift Container Platform 4.21 to 4.22), you must reinstate the credential secret with the administrator-level credential. If the credential is not present, the update might be blocked. **Prerequisites** - Your cluster is installed on a platform that supports removing cloud credentials from the CCO. Supported platforms are AWS and Google Cloud. **Procedure** 1. In the **Administrator** perspective of the web console, navigate to **Workloads** → **Secrets**. 2. In the table on the **Secrets** page, find the root secret for your cloud provider. | Platform | Secret name | | --- | --- | | AWS | `aws-creds` | | Google Cloud | `gcp-credentials` | 3. Click the Options menu ![](/openshift-docs-markdown/images/kebab.png "Options menu") in the same row as the secret and select **Delete Secret**. **Additional resources** {._additional-resources} - [The Cloud Credential Operator in mint mode](/openshift-docs-markdown/authentication/managing_cloud_provider_credentials/cco-mode-mint#cco-mode-mint) - [Cloud Credential Operator (CCO)](/openshift-docs-markdown/operators/operator-reference#cloud-credential-operator_cluster-operators-ref) ## Enabling token-based authentication {#post-install-enable-token-auth_changing-cloud-credentials-configuration} After installing an OpenShift Container Platform cluster on Microsoft Azure or Amazon Web Services (AWS), you can enable Microsoft Entra Workload ID or Security Token Service (STS) to use short-term credentials. ### Configuring the Cloud Credential Operator utility {#cco-ccoctl-configuring_changing-cloud-credentials-configuration} To configure an existing cluster to create and manage cloud credentials from outside of the cluster, extract and prepare the Cloud Credential Operator utility (`ccoctl`) binary. > [!NOTE] > The `ccoctl` utility is a Linux binary that must run in a Linux environment. **Prerequisites** - You have access to an OpenShift Container Platform account with cluster administrator access. - You have installed the OpenShift CLI (`oc`). **Procedure** 1. Set a variable for the OpenShift Container Platform release image by running the following command: ``` $ RELEASE_IMAGE=$(oc get clusterversion -o jsonpath={..desired.image}) ``` 2. Obtain the CCO container image from the OpenShift Container Platform release image by running the following command: ```terminal $ CCO_IMAGE=$(oc adm release info --image-for='cloud-credential-operator' $RELEASE_IMAGE -a ~/.pull-secret) ``` > [!NOTE] > Ensure that the architecture of the `$RELEASE_IMAGE` matches the architecture of the environment in which you will use the `ccoctl` tool. 3. Extract the `ccoctl` binary from the CCO container image within the OpenShift Container Platform release image by running the following command: ```terminal $ oc image extract $CCO_IMAGE \ --file="/usr/bin/ccoctl." \ -a ~/.pull-secret ``` For ``, specify the value that corresponds to the version of Red Hat Enterprise Linux (RHEL) that the host uses. If no value is specified, `ccoctl.rhel8` is used by default. The following values are valid: - `rhel8`: Specify this value for hosts that use RHEL 8. - `rhel9`: Specify this value for hosts that use RHEL 9. > [!NOTE] > The `ccoctl` binary is created in the directory from where you executed the command and not in `/usr/bin/`. You must rename the directory or move the `ccoctl.` binary to `ccoctl`. 4. Change the permissions to make `ccoctl` executable by running the following command: ```terminal $ chmod 775 ccoctl ``` **Verification** - To verify that `ccoctl` is ready to use, display the help file. Use a relative file name when you run the command, for example: ```terminal $ ./ccoctl ``` ```terminal {title="Example output"} OpenShift credentials provisioning tool Usage: ccoctl [command] Available Commands: aws Manage credentials objects for AWS cloud azure Manage credentials objects for Azure gcp Manage credentials objects for Google cloud help Help about any command ibmcloud Manage credentials objects for IBM Cloud nutanix Manage credentials objects for Nutanix Flags: -h, --help help for ccoctl Use "ccoctl [command] --help" for more information about a command. ``` ### Enabling Microsoft Entra Workload ID on an existing cluster {#enabling-entra-workload-id-existing-cluster_changing-cloud-credentials-configuration} Enable Microsoft Entra Workload ID on an existing Microsoft Azure OpenShift Container Platform cluster. If you did not configure your cluster to use Microsoft Entra Workload ID during installation, you can enable this authentication method post-installation. > [!IMPORTANT] > The process to enable Workload ID on an existing cluster is disruptive and takes a significant amount of time. Before proceeding, observe the following considerations: > > - Read the following steps and ensure that you understand and accept the time requirement. The exact time requirement varies depending on the individual cluster, but it is likely to require at least one hour. > - During this process, you must refresh all service accounts and restart all pods on the cluster. These actions are disruptive to workloads. To mitigate this impact, you can temporarily halt these services and then redeploy them when the cluster is ready. > - After starting this process, do not attempt to update the cluster until it is complete. If an update is triggered, the process to enable Workload ID on an existing cluster fails. **Prerequisites** - You have installed an OpenShift Container Platform cluster on Microsoft Azure. - You have access to the cluster using an account with `cluster-admin` permissions. - You have installed the OpenShift CLI (`oc`). - You have extracted and prepared the Cloud Credential Operator utility (`ccoctl`) binary. - You have access to your Azure account by using the Azure CLI (`az`). **Procedure** 1. Create an output directory for the manifests that the `ccoctl` utility generates. This procedure uses `./output_dir` as an example. 2. Extract the service account public signing key for the cluster to the output directory by running the following command: ```terminal $ oc get secret/next-bound-service-account-signing-key \ -n openshift-kube-apiserver-operator \ -ojsonpath='{ .data.service-account\.pub }' | base64 -d \ > output_dir/serviceaccount-signer.public ``` This procedure uses a file named `serviceaccount-signer.public` as an example. 3. Use the extracted service account public signing key to create an OpenID Connect (OIDC) issuer and Azure blob storage container with OIDC configuration files by running the following command: ```terminal $ ./ccoctl azure create-oidc-issuer \ --name \ --output-dir ./output_dir \ --region \ --subscription-id \ --tenant-id \ --public-key-file ./output_dir/serviceaccount-signer.public ``` where: `` : The value of the `name` parameter is used to create an Azure resource group. To use an existing Azure resource group instead of creating a new one, specify the `--oidc-resource-group-name` argument with the existing group name as its value. `` : Specify the region of the existing cluster. `` : Specify the subscription ID of the existing cluster. `--public-key-file` : Specify the file that contains the service account public signing key for the cluster. 4. Verify that the configuration file for the Azure pod identity webhook was created by running the following command: ```terminal $ ll ./output_dir/manifests ``` ```text {title="Example output"} total 8 -rw-------. 1 cloud-user cloud-user 193 May 22 02:29 azure-ad-pod-identity-webhook-config.yaml -rw-------. 1 cloud-user cloud-user 165 May 22 02:29 cluster-authentication-02-config.yaml ``` The file `azure-ad-pod-identity-webhook-config.yaml` contains the Azure pod identity webhook configuration. 5. Set an `OIDC_ISSUER_URL` variable with the OIDC issuer URL from the generated manifests in the output directory by running the following command: ```terminal $ OIDC_ISSUER_URL=`awk '/serviceAccountIssuer/ { print $2 }' ./output_dir/manifests/cluster-authentication-02-config.yaml` ``` 6. Update the `spec.serviceAccountIssuer` parameter of the cluster `authentication` configuration by running the following command: ```terminal $ oc patch authentication cluster \ --type=merge \ -p "{\"spec\":{\"serviceAccountIssuer\":\"${OIDC_ISSUER_URL}\"}}" ``` 7. Monitor the configuration update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 8. Restart all of the pods in the cluster by running the following command: ```terminal $ oc adm reboot-machine-config-pool mcp/worker mcp/master ``` Restarting a pod updates the `serviceAccountIssuer` field and refreshes the service account public signing key. 9. Monitor the restart and update process by running the following command: ```terminal $ oc adm wait-for-node-reboot nodes --all ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All nodes rebooted ``` 10. Update the Cloud Credential Operator `spec.credentialsMode` parameter to `Manual` by running the following command: ```terminal $ oc patch cloudcredential cluster \ --type=merge \ --patch '{"spec":{"credentialsMode":"Manual"}}' ``` 11. Extract the list of `CredentialsRequest` objects from the OpenShift Container Platform release image by running the following command: ```terminal $ oc adm release extract \ --credentials-requests \ --included \ --to \ --registry-config ~/.pull-secret ``` > [!NOTE] > This command might take a few moments to run. 12. Set an `AZURE_INSTALL_RG` variable with the Azure resource group name by running the following command: ```terminal $ AZURE_INSTALL_RG=`oc get infrastructure cluster -o jsonpath --template '{ .status.platformStatus.azure.resourceGroupName }'` ``` 13. Use the `ccoctl` utility to create managed identities for all `CredentialsRequest` objects by running the following command: > [!NOTE] > The following command does not show all available options. For a complete list of options, including those that might be necessary for your specific use case, run `$ ccoctl azure create-managed-identities --help`. ```terminal $ ccoctl azure create-managed-identities \ --name \ --output-dir ./output_dir \ --region \ --subscription-id \ --credentials-requests-dir \ --issuer-url "${OIDC_ISSUER_URL}" \ --dnszone-resource-group-name \ --installation-resource-group-name "${AZURE_INSTALL_RG}" \ --network-resource-group-name \ --preserve-existing-roles ``` where: `` : Specify the name of the resource group that contains the DNS zone. `` : Optional: Specify the virtual network resource group if it is different from the cluster resource group. `--preserve-existing-roles` : Optional: Specify this flag to ensure that any custom role assignments you define on managed identities are not removed during OpenShift Container Platform updates. 14. Apply the Azure pod identity webhook configuration for Workload ID by running the following command: ```terminal $ oc apply -f ./output_dir/manifests/azure-ad-pod-identity-webhook-config.yaml ``` 15. Apply the secrets generated by the `ccoctl` utility by running the following command: ```terminal $ find ./output_dir/manifests -iname "openshift*yaml" -print0 | xargs -I {} -0 -t oc replace -f {} ``` This process might take several minutes. 16. Restart all of the pods in the cluster by running the following command: ```terminal $ oc adm reboot-machine-config-pool mcp/worker mcp/master ``` Restarting a pod updates the `serviceAccountIssuer` field and refreshes the service account public signing key. 17. Monitor the restart and update process by running the following command: ```terminal $ oc adm wait-for-node-reboot nodes --all ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All nodes rebooted ``` 18. Monitor the configuration update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 19. Optional: Remove the Azure root credentials secret by running the following command: ```terminal $ oc delete secret -n kube-system azure-credentials ``` ### Enabling AWS Security Token Service (STS) on an existing cluster {#enabling-aws-sts-existing-cluster_changing-cloud-credentials-configuration} Enable AWS Security Token Service (STS) on an existing OpenShift Container Platform cluster if you did not configure this authentication method during installation. > [!IMPORTANT] > The process to enable STS on an existing cluster is disruptive and takes a significant amount of time. Before proceeding, observe the following considerations: > > - Read the following steps and ensure that you understand and accept the time requirement. The exact time requirement varies depending on the individual cluster, but it is likely to require at least one hour. > - During this process, you must refresh all service accounts and restart all pods on the cluster. These actions are disruptive to workloads. To mitigate this impact, you can temporarily halt these services and then redeploy them when the cluster is ready. > - Do not update the cluster until this process is complete. **Prerequisites** - You have installed an OpenShift Container Platform cluster on AWS. - You have access to the cluster using an account with `cluster-admin` permissions. - You have installed the OpenShift CLI (`oc`). - You have extracted and prepared the Cloud Credential Operator utility (`ccoctl`) binary. - You have access to your AWS account by using the AWS CLI (aws). **Procedure** 1. Create an output directory for `ccoctl` generated manifests. ```terminal $ mkdir ./output_dir ``` 2. Create the AWS Identity and Access Management (IAM) OpenID Connect (OIDC) provider. 1. Extract the service account public signing key for the cluster by running the following command: ```terminal $ oc get secret/next-bound-service-account-signing-key \ -n openshift-kube-apiserver-operator \ -ojsonpath='{ .data.service-account\.pub }' | base64 -d \ > output_dir/serviceaccount-signer.public ``` This procedure uses a file named `serviceaccount-signer.public` as an example. 2. Create the AWS IAM identity provider and S3 bucket by running the following command: ```terminal $ ./ccoctl aws create-identity-provider \ --output-dir output_dir \ --name \ --region us-east-2 \ --public-key-file output_dir/serviceaccount-signer.public ``` where: `--output-dir` : Specify the output directory you created earlier. `--name` : Specify a globally unique name. This name functions as a prefix for AWS resources created by this command. `--region` : Specify the AWS region of the cluster. `--public-key-file` : Specify the relative path to the `serviceaccount-signer.public` file you created earlier. 3. Save or note the Amazon Resource Name (ARN) for the IAM identity provider. You can find this information in the final line of the output of the previous command. 3. Update the cluster authentication configuration. 1. Extract the OIDC issuer URL and update the authentication configuration of the cluster by running the following commands: ```terminal $ OIDC_ISSUER_URL=`awk '/serviceAccountIssuer/ { print $2 }' output_dir/manifests/cluster-authentication-02-config.yaml` $ oc patch authentication cluster --type=merge -p "{\"spec\":{\"serviceAccountIssuer\":\"${OIDC_ISSUER_URL}\"}}" ``` 2. Monitor the configuration update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 4. Restart pods to apply the issuer update. 1. Restart all of the pods in the cluster by running the following command: ```terminal $ oc adm reboot-machine-config-pool mcp/worker mcp/master ``` Restarting a pod updates the `serviceAccountIssuer` field and refreshes the service account public signing key. 2. Monitor the restart and update process by running the following command: ```terminal $ oc adm wait-for-node-reboot nodes --all ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All nodes rebooted ``` 5. Update the Cloud Credential Operator `spec.credentialsMode` parameter to `Manual` by running the following command: ```terminal $ oc patch cloudcredential cluster \ --type=merge \ --patch '{"spec":{"credentialsMode":"Manual"}}' ``` 6. Extract `CredentialsRequests` objects. 1. Create a `CLUSTER_VERSION` environment variable by running the following command: ```terminal $ CLUSTER_VERSION=$(oc get clusterversion version -o json | jq -r '.status.desired.version') ``` 2. Create a `CLUSTER_IMAGE` environment variable by running the following command: ```terminal $ CLUSTER_IMAGE=$(oc get clusterversion version -o json | jq -r ".status.history[] | select(.version == \"${CLUSTER_VERSION}\") | .image") ``` 3. Extract `CredentialsRequests` objects from the release image by running the following command: ```terminal $ oc adm release extract \ --credentials-requests \ --cloud=aws \ --from ${CLUSTER_IMAGE} \ --to output_dir/cred-reqs ``` 7. Create AWS IAM roles and apply secrets. 1. Create an IAM role for each `CredentialsRequests` object by running the following command: ```terminal $ ./ccoctl aws create-iam-roles \ --output-dir ./output_dir/ \ --name \ --identity-provider-arn \ --region us-east-2 \ --credentials-requests-dir ./output_dir/cred-reqs/ \ --permissions-boundary-arn= ``` where: `--output-dir` : Specify the output directory you created earlier. `--name` : Specify a globally unique name. This name functions as a prefix for AWS resources created by this command. `--identity-provider-arn` : Specify the ARN for the IAM identity provider. `--region` : Specify the AWS region of the cluster. `--credentials-requests-dir` : Specify the relative path to the folder where you extracted the `CredentialsRequest` files with the `oc adm release extract` command. `--permissions-boundary-arn` : Optional: Specify the Amazon Resource Name (ARN) of the AWS IAM policy to use as the permissions boundary for the IAM roles created by the `ccoctl` utility. 2. Apply the generated secrets by running the following command: ```terminal $ find ./output_dir/manifests -iname "openshift*yaml" -print0 | xargs -I {} -0 -t oc replace -f {} ``` 8. Finish the configuration process by restarting the cluster. 1. Restart all of the pods in the cluster by running the following command: ```terminal $ oc adm reboot-machine-config-pool mcp/worker mcp/master ``` 2. Monitor the restart and update process by running the following command: ```terminal $ oc adm wait-for-node-reboot nodes --all ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All nodes rebooted ``` 3. Monitor the configuration update progress by running the following command: ```terminal $ oc adm wait-for-stable-cluster ``` This process might take 15 minutes or longer. The following output indicates that the process is complete: ```text All clusteroperators are stable ``` 9. Optional: Remove the AWS root credentials secret by running the following command: ```terminal $ oc delete secret -n kube-system aws-creds ``` **Additional resources** {._additional-resources} - [Microsoft Entra Workload ID](/openshift-docs-markdown/authentication/managing_cloud_provider_credentials/cco-short-term-creds#cco-short-term-creds-azure_cco-short-term-creds) - [Configuring an Azure cluster to use short-term credentials](/openshift-docs-markdown/installing/installing_azure/ipi/installing-azure-customizations#installing-azure-with-short-term-creds_installing-azure-customizations) - [AWS Security Token Service](/openshift-docs-markdown/authentication/managing_cloud_provider_credentials/cco-short-term-creds#cco-short-term-creds-aws_cco-short-term-creds) - [Configuring an AWS cluster to use short-term credentials](/openshift-docs-markdown/installing/installing_aws/ipi/installing-aws-customizations#installing-aws-with-short-term-creds_installing-aws-customizations) ### Verifying that a cluster uses short-term credentials {#cco-ccoctl-install-verifying_changing-cloud-credentials-configuration} You can verify that a cluster uses short-term security credentials for individual components by checking the Cloud Credential Operator (CCO) configuration and other values in the cluster. **Prerequisites** - You deployed an OpenShift Container Platform cluster using the Cloud Credential Operator utility (`ccoctl`) to implement short-term credentials. - You installed the OpenShift CLI (`oc`). - You are logged in as a user with `cluster-admin` privileges. **Procedure** - Verify that the CCO is configured to operate in manual mode by running the following command: ```terminal $ oc get cloudcredentials cluster \ -o=jsonpath={.spec.credentialsMode} ``` The following output confirms that the CCO is operating in manual mode: ```text {title="Example output"} Manual ``` - Verify that the cluster does not have `root` credentials by running the following command: ```terminal $ oc get secrets \ -n kube-system ``` where `` is the name of the root secret for your cloud provider. | Platform | Secret name | | --- | --- | | Amazon Web Services (AWS) | `aws-creds` | | Microsoft Azure | `azure-credentials` | | Google Cloud | `gcp-credentials` | An error confirms that the root secret is not present on the cluster. ```text {title="Example output for an AWS cluster"} Error from server (NotFound): secrets "aws-creds" not found ``` - Verify that the components are using short-term security credentials for individual components by running the following command: ```terminal $ oc get authentication cluster \ -o jsonpath \ --template='{ .spec.serviceAccountIssuer }' ``` This command displays the value of the `.spec.serviceAccountIssuer` parameter in the cluster `Authentication` object. An output of a URL that is associated with your cloud provider indicates that the cluster is using manual mode with short-term credentials that are created and managed from outside of the cluster. - Azure clusters: Verify that the components are assuming the Azure client ID that is specified in the secret manifests by running the following command: ```terminal $ oc get secrets \ -n openshift-image-registry installer-cloud-credentials \ -o jsonpath='{.data}' ``` An output that contains the `azure_client_id` and `azure_federated_token_file` fields confirms that the components are assuming the Azure client ID. - Azure clusters: Verify that the pod identity webhook is running by running the following command: ```terminal $ oc get pods \ -n openshift-cloud-credential-operator ``` ```text {title="Example output"} NAME READY STATUS RESTARTS AGE cloud-credential-operator-59cf744f78-r8pbq 2/2 Running 2 71m pod-identity-webhook-548f977b4c-859lz 1/1 Running 1 70m ``` **Additional resources** {._additional-resources} - [About the Cloud Credential Operator](/openshift-docs-markdown/authentication/managing_cloud_provider_credentials/about-cloud-credential-operator#about-cloud-credential-operator) - [Determining the Cloud Credential Operator mode](/openshift-docs-markdown/authentication/managing_cloud_provider_credentials/about-cloud-credential-operator#cco-determine-mode_about-cloud-credential-operator)