Skip to main content

Configuring a shared container partition for the image-based upgrade

Your single-node OpenShift clusters need to have a shared /var/lib/containers partition for the image-based upgrade. You can do this at install time.

Configuring a shared container partition between ostree stateroots​

Apply a MachineConfig to both the seed and the target clusters during installation time to create a separate partition and share the /var/lib/containers partition between the two ostree stateroots that will be used during the upgrade process.

warning

You must complete this procedure at installation time.

Procedure

  • Apply a MachineConfig to create a separate partition:

    apiVersion: machineconfiguration.openshift.io/v1
    kind: MachineConfig
    metadata:
      labels:
        machineconfiguration.openshift.io/role: master
      name: 98-var-lib-containers-partitioned
    spec:
      config:
        ignition:
          version: 3.2.0
        storage:
          disks:
            - device: /dev/disk/by-path/<root_disk>
              partitions:
                - label: var-lib-containers
                  startMiB: <start_of_partition>
                  sizeMiB: <partition_size>
          filesystems:
            - device: /dev/disk/by-partlabel/var-lib-containers
              format: xfs
              mountOptions:
                - defaults
                - prjquota
              path: /var/lib/containers
              wipeFilesystem: true
        systemd:
          units:
            - contents: |-
                # Generated by Butane
                [Unit]
                Before=local-fs.target
                Requires=systemd-fsck@dev-disk-by\x2dpartlabel-var\x2dlib\x2dcontainers.service
                After=systemd-fsck@dev-disk-by\x2dpartlabel-var\x2dlib\x2dcontainers.service

                [Mount]
                Where=/var/lib/containers
                What=/dev/disk/by-partlabel/var-lib-containers
                Type=xfs
                Options=defaults,prjquota

                [Install]
                RequiredBy=local-fs.target
              enabled: true
              name: var-lib-containers.mount

    where:

    <root_disk>
    Specifies the root disk, for example pci-0000:01:00.0-scsi-0:2:0:0.
    <start_of_partition>
    Specifies the start of the partition in MiB. If the value is too small, the installation will fail.
    <partition_size>
    Specifies a minimum size for the partition of 500 GB (512000 MiB) to ensure adequate disk space for precached images. If the value is too small, the deployments after installation will fail.

Configuring a shared container directory between ostree stateroots when using GitOps ZTP​

When you are using the GitOps Zero Touch Provisioning (ZTP) workflow, you do the following procedure to create a separate disk partition on both the seed and target cluster and to share the /var/lib/containers partition.

warning

You must complete this procedure at installation time.

Prerequisites

  • You have installed Butane. For more information, see "Installing Butane".

Procedure

  1. Create the storage.bu file:

    variant: fcos
    version: 1.3.0
    storage:
      disks:
      - device: /dev/disk/by-path/pci-<root_disk>
        wipe_table: false
        partitions:
        - label: var-lib-containers
          start_mib: <start_of_partition>
          size_mib: <partition_size>
      filesystems:
        - path: /var/lib/containers
          device: /dev/disk/by-partlabel/var-lib-containers
          format: xfs
          wipe_filesystem: true
          with_mount_unit: true
          mount_options:
            - defaults
            - prjquota

    where:

    <root_disk>
    Specifies the root disk.
    <start_of_partition>
    Specifies the start of the partition in MiB. If the value is too small, the installation will fail.
    <partition_size>
    Specifies a minimum size for the partition of 500 GB to ensure adequate disk space for precached images. If the value is too small, the deployments after installation will fail.

  2. Convert the storage.bu to an Ignition file by running the following command:

    $ butane storage.bu

    Example output:

    {"ignition":{"version":"3.2.0"},"storage":{"disks":[{"device":"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0","partitions":[{"label":"var-lib-containers","sizeMiB":0,"startMiB":250000}],"wipeTable":false}],"filesystems":[{"device":"/dev/disk/by-partlabel/var-lib-containers","format":"xfs","mountOptions":["defaults","prjquota"],"path":"/var/lib/containers","wipeFilesystem":true}]},"systemd":{"units":[{"contents":"# Generated by Butane\n[Unit]\nRequires=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\nAfter=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\n\n[Mount]\nWhere=/var/lib/containers\nWhat=/dev/disk/by-partlabel/var-lib-containers\nType=xfs\nOptions=defaults,prjquota\n\n[Install]\nRequiredBy=local-fs.target","enabled":true,"name":"var-lib-containers.mount"}]}}

  3. Copy the output into the spec.nodes[].ignitionConfigOverride field in the ClusterInstance CR:

    apiVersion: siteconfig.open-cluster-management.io/v1alpha1
    kind: ClusterInstance
    metadata:
      name: "example-sno"
      namespace: "example-sno"
    spec:
      # ...
      nodes:
        - hostName: "node1.example.com"
          role: "master"
              ignitionConfigOverride: '{"ignition":{"version":"3.2.0"},"storage":{"disks":[{"device":"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0","partitions":[{"label":"var-lib-containers","sizeMiB":0,"startMiB":250000}],"wipeTable":false}],"filesystems":[{"device":"/dev/disk/by-partlabel/var-lib-containers","format":"xfs","mountOptions":["defaults","prjquota"],"path":"/var/lib/containers","wipeFilesystem":true}]},"systemd":{"units":[{"contents":"# Generated by Butane\n[Unit]\nRequires=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\nAfter=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\n\n[Mount]\nWhere=/var/lib/containers\nWhat=/dev/disk/by-partlabel/var-lib-containers\nType=xfs\nOptions=defaults,prjquota\n\n[Install]\nRequiredBy=local-fs.target","enabled":true,"name":"var-lib-containers.mount"}]}}'

Verification

  1. During or after installation, verify on the hub cluster that the BareMetalHost object shows the annotation by running the following command:

    $ oc get bmh -n my-sno-ns my-sno -ojson | jq '.metadata.annotations["bmac.agent-install.openshift.io/ignition-config-overrides"]'

    Example output:

    "{\"ignition\":{\"version\":\"3.2.0\"},\"storage\":{\"disks\":[{\"device\":\"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0\",\"partitions\":[{\"label\":\"var-lib-containers\",\"sizeMiB\":0,\"startMiB\":250000}],\"wipeTable\":false}],\"filesystems\":[{\"device\":\"/dev/disk/by-partlabel/var-lib-containers\",\"format\":\"xfs\",\"mountOptions\":[\"defaults\",\"prjquota\"],\"path\":\"/var/lib/containers\",\"wipeFilesystem\":true}]},\"systemd\":{\"units\":[{\"contents\":\"# Generated by Butane\\n[Unit]\\nRequires=systemd-fsck@dev-disk-by\\\\x2dpartlabel-var\\\\x2dlib\\\\x2dcontainers.service\\nAfter=systemd-fsck@dev-disk-by\\\\x2dpartlabel-var\\\\x2dlib\\\\x2dcontainers.service\\n\\n[Mount]\\nWhere=/var/lib/containers\\nWhat=/dev/disk/by-partlabel/var-lib-containers\\nType=xfs\\nOptions=defaults,prjquota\\n\\n[Install]\\nRequiredBy=local-fs.target\",\"enabled\":true,\"name\":\"var-lib-containers.mount\"}]}}"

  2. After installation, check the single-node OpenShift disk status by running the following commands:

    # lsblk

    Example output:

    NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
    sda      8:0    0 446.6G  0 disk
    ├─sda1   8:1    0     1M  0 part
    ├─sda2   8:2    0   127M  0 part
    ├─sda3   8:3    0   384M  0 part /boot
    ├─sda4   8:4    0 243.6G  0 part /var
    │                                /sysroot/ostree/deploy/rhcos/var
    │                                /usr
    │                                /etc
    │                                /
    │                                /sysroot
    └─sda5   8:5    0 202.5G  0 part /var/lib/containers
    # df -h

    Example output:

    Filesystem      Size  Used Avail Use% Mounted on
    devtmpfs        4.0M     0  4.0M   0% /dev
    tmpfs           126G   84K  126G   1% /dev/shm
    tmpfs            51G   93M   51G   1% /run
    /dev/sda4       244G  5.2G  239G   3% /sysroot
    tmpfs           126G  4.0K  126G   1% /tmp
    /dev/sda5       203G  119G   85G  59% /var/lib/containers
    /dev/sda3       350M  110M  218M  34% /boot
    tmpfs            26G     0   26G   0% /run/user/1000

Additional resources