---
title: Troubleshooting Windows container workload issues
---

# Troubleshooting Windows container workload issues {#troubleshooting-windows-container-workload-issues}

Use the following sections to troubleshoot Windows container workload issues.

## Windows Machine Config Operator does not install {#wmco-does-not-install_troubleshooting-windows-container-workload-issues}

If you have completed the process of installing the Windows Machine Config Operator (WMCO), but the Operator is stuck in the `InstallWaiting` phase, your issue is likely caused by a networking issue.

The WMCO requires your OpenShift Container Platform cluster to be configured with hybrid networking using OVN-Kubernetes; the WMCO cannot complete the installation process without hybrid networking available. This is necessary to manage nodes on multiple operating systems (OS) and OS variants. This must be completed during the installation of your cluster.

## Investigating why Windows Machine does not become compute node {#investigating-why-windows-machine-compute-node_troubleshooting-windows-container-workload-issues}

There are various reasons why a Windows Machine does not become a compute node. The best way to investigate this problem is to collect the Windows Machine Config Operator (WMCO) logs.

**Prerequisites**

- You installed the Windows Machine Config Operator (WMCO) using Operator Lifecycle Manager (OLM).
- You have created a Windows compute machine set.

**Procedure**

- Run the following command to collect the WMCO logs:

  ```terminal
  $ oc logs -f deployment/windows-machine-config-operator -n openshift-windows-machine-config-operator
  ```

## Accessing a Windows node {#accessing-windows-node_troubleshooting-windows-container-workload-issues}

Windows nodes cannot be accessed using the `oc debug node` command; the command requires running a privileged pod on the node, which is not yet supported for Windows. Instead, a Windows node can be accessed using a secure shell (SSH) or Remote Desktop Protocol (RDP). An SSH bastion is required for both methods.

### Accessing a Windows node using SSH {#accessing-windows-node-using-ssh_troubleshooting-windows-container-workload-issues}

You can access a Windows node by using a secure shell (SSH).

**Prerequisites**

- You have installed the Windows Machine Config Operator (WMCO) using Operator Lifecycle Manager (OLM).
- You have created a Windows compute machine set.
- You have added the key used in the `cloud-private-key` secret and the key used when creating the cluster to the ssh-agent. For security reasons, remember to remove the keys from the ssh-agent after use.
- You have connected to the Windows node [using an `ssh-bastion` pod](https://access.redhat.com/solutions/4073041).

**Procedure**

- Access the Windows node by running the following command:

  ```terminal
  $ ssh -t -o StrictHostKeyChecking=no -o ProxyCommand='ssh -A -o StrictHostKeyChecking=no \
      -o ServerAliveInterval=30 -W %h:%p core@$(oc get service --all-namespaces -l run=ssh-bastion \
      -o go-template="{{ with (index (index .items 0).status.loadBalancer.ingress 0) }}{{ or .hostname .ip }}{{end}}")' <username>@<windows_node_internal_ip>
  ```

  where:

  `<username>`
  :   Specifies the cloud provider username, such as `Administrator` for Amazon Web Services (AWS) or `capi` for Microsoft Azure.

  `<windows_node_internal_ip>`
  :   Specifies the internal IP address of the node, which can be discovered by running the following command:

  ```terminal
  $ oc get nodes <node_name> -o jsonpath={.status.addresses[?\(@.type==\"InternalIP\"\)].address}
  ```

### Accessing a Windows node using RDP {#accessing-windows-node-using-rdp_troubleshooting-windows-container-workload-issues}

You can access a Windows node by using a Remote Desktop Protocol (RDP).

**Prerequisites**

- You installed the Windows Machine Config Operator (WMCO) using Operator Lifecycle Manager (OLM).
- You have created a Windows compute machine set.
- You have added the key used in the `cloud-private-key` secret and the key used when creating the cluster to the ssh-agent. For security reasons, remember to remove the keys from the ssh-agent after use.
- You have connected to the Windows node [using an `ssh-bastion` pod](https://access.redhat.com/solutions/4073041).

**Procedure**

1. Run the following command to set up an SSH tunnel:

   ```terminal
   $ ssh -L 2020:<windows_node_internal_ip>:3389 \
       core@$(oc get service --all-namespaces -l run=ssh-bastion -o go-template="{{ with (index (index .items 0).status.loadBalancer.ingress 0) }}{{ or .hostname .ip }}{{end}}")
   ```

   where:

   `<windows_node_internal_ip>`
   :   Specifies the internal IP address of the node, which can be discovered by running the following command:

   ```terminal
   $ oc get nodes <node_name> -o jsonpath={.status.addresses[?\(@.type==\"InternalIP\"\)].address}
   ```
2. From within the resulting shell, SSH into the Windows node and run the following command to create a password for the user:

   ```terminal
   C:\> net user <username> *
   ```

   Specify the cloud provider user name, such as `Administrator` for AWS or `capi` for Azure. You can now remotely access the Windows node at `localhost:2020` using an RDP client.

## Collecting Kubernetes node logs for Windows containers {#collecting-kube-node-logs-windows_troubleshooting-windows-container-workload-issues}

Windows container logging works differently from Linux container logging; the Kubernetes node logs for Windows workloads are streamed to the `C:\var\logs` directory by default. Therefore, you must gather the Windows node logs from that directory.

**Prerequisites**

- You installed the Windows Machine Config Operator (WMCO) using Operator Lifecycle Manager (OLM).
- You have created a Windows compute machine set.

**Procedure**

1. To view the logs under all directories in `C:\var\logs`, run the following command:

   ```terminal
   $ oc adm node-logs -l kubernetes.io/os=windows --path= \
       /ip-10-0-138-252.us-east-2.compute.internal containers \
       /ip-10-0-138-252.us-east-2.compute.internal hybrid-overlay \
       /ip-10-0-138-252.us-east-2.compute.internal kube-proxy \
       /ip-10-0-138-252.us-east-2.compute.internal kubelet \
       /ip-10-0-138-252.us-east-2.compute.internal pods
   ```
2. You can now list files in the directories using the same command and view the individual log files. For example, to view the kubelet logs, run the following command:

   ```terminal
   $ oc adm node-logs -l kubernetes.io/os=windows --path=/kubelet/kubelet.log
   ```

## Collecting Windows application event logs {#collecting-windows-application-event-logs_troubleshooting-windows-container-workload-issues}

The `Get-WinEvent` shim on the kubelet `logs` endpoint can be used to collect application event logs from Windows machines.

**Prerequisites**

- You installed the Windows Machine Config Operator (WMCO) using Operator Lifecycle Manager (OLM).
- You have created a Windows compute machine set.

**Procedure**

- To view logs from all applications logging to the event logs on the Windows machine, run:

  ```terminal
  $ oc adm node-logs -l kubernetes.io/os=windows --path=journal
  ```

  The same command is executed when collecting logs with `oc adm must-gather`.

  Other Windows application logs from the event log can also be collected by specifying the respective service with a `-u` flag. For example, you can run the following command to collect logs for the containerd container runtime service:

  ```terminal
  $ oc adm node-logs -l kubernetes.io/os=windows --path=journal -u containerd
  ```

## Collecting containerd logs for Windows containers {#collecting-docker-logs-windows_troubleshooting-windows-container-workload-issues}

The Windows containerd container service does not stream log data to stdout, but instead, it stream log data to the Windows event log. You can view the containerd event logs to investigate issues you think might be caused by the Windows containerd container service.

**Prerequisites**

- You installed the Windows Machine Config Operator (WMCO) using Operator Lifecycle Manager (OLM).
- You have created a Windows compute machine set.

**Procedure**

- View the containerd logs by running the following command:

  ```terminal
  $ oc adm node-logs -l kubernetes.io/os=windows --path=containerd
  ```

**Additional resources**
{._additional-resources}

- [Configuring hybrid networking](/openshift-docs-markdown/networking/ovn_kubernetes_network_provider/configuring-hybrid-networking#configuring-hybrid-ovnkubernetes)
- [Containers on Windows troubleshooting](https://docs.microsoft.com/en-us/virtualization/windowscontainers/troubleshooting)
- [Troubleshoot host and container image mismatches](https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/update-containers#troubleshoot-host-and-container-image-mismatches)
- [Common Kubernetes problems with Windows](https://docs.microsoft.com/en-us/virtualization/windowscontainers/kubernetes/common-problems)