Skip to main content

Performing advanced Compliance Operator tasks

As an advanced user, you can use options in the Compliance Operator for the purpose of debugging or integrating with existing tooling.

Using the ComplianceSuite and ComplianceScan objects directly​

You can define a ComplianceSuite object directly rather than using the ScanSetting and ScanSettingBinding objects to define the suites and scans.

There following use cases are valid reasons to define a ComplianceSuite object:

  • Specifying only a single rule to scan. This can be useful for debugging together with the debug: true attribute which increases the OpenSCAP scanner verbosity, as the debug mode tends to get quite verbose otherwise. Limiting the test to one rule helps to lower the amount of debug information.
  • Providing a custom nodeSelector. In order for a remediation to be applicable, the nodeSelector must match a pool.
  • Pointing the Scan to a bespoke config map with a tailoring file.
  • For testing or development when the overhead of parsing profiles from bundles is not required.

The following example shows a ComplianceSuite that scans the worker machines with only a single rule:

apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceSuite
metadata:
  name: workers-compliancesuite
spec:
  scans:
    - name: workers-scan
      profile: xccdf_org.ssgproject.content_profile_moderate
      content: ssg-rhcos4-ds.xml
      contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:45dc...
      debug: true
      rule: xccdf_org.ssgproject.content_rule_no_direct_root_logins
      nodeSelector:
      node-role.kubernetes.io/worker: ""

The ComplianceSuite object and the ComplianceScan objects referred to above specify several attributes in a format that OpenSCAP expects.

To discover the profile, content, or rule values, you can start by creating a similar Suite from ScanSetting and ScanSettingBinding or inspect the objects parsed from the ProfileBundle objects such as rules or profiles. Those objects contain the xccdf_org identifiers you can use to refer to them from a ComplianceSuite.

Setting PriorityClass for ScanSetting scans​

In some clusters, the default PriorityClass object can be too low to guarantee pods execute scans on time. To maintain compliance or guarantee automated scanning, you can set the PriorityClass variable to ensure the Compliance Operator is always given priority in resource constrained situations.

Prerequisites

  • Optional: You have created a PriorityClass object. For more information, see "Configuring priority and preemption" in the Additional resources.

Procedure

  • Set the PriorityClass variable:

    apiVersion: compliance.openshift.io/v1alpha1
    strictNodeScan: true
    metadata:
      name: default
      namespace: openshift-compliance
    priorityClass: compliance-high-priority
    kind: ScanSetting
    showNotApplicable: false
    rawResultStorage:
      nodeSelector:
        node-role.kubernetes.io/master: ''
      pvAccessModes:
        - ReadWriteOnce
      rotation: 3
      size: 1Gi
      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
          operator: Exists
        - effect: NoExecute
          key: node.kubernetes.io/not-ready
          operator: Exists
          tolerationSeconds: 300
        - effect: NoExecute
          key: node.kubernetes.io/unreachable
          operator: Exists
          tolerationSeconds: 300
        - effect: NoSchedule
          key: node.kubernetes.io/memory-pressure
          operator: Exists
    schedule: 0 1 * * *
    roles:
      - master
      - worker
    scanTolerations:
      - operator: Exists

    where:

    PriorityClass
    If the PriorityClass referenced in the ScanSetting cannot be found, the Operator will leave the PriorityClass empty, issue a warning, and continue scheduling scans without a PriorityClass.

Using raw tailored profiles​

Although the TailoredProfile CR enables the most common tailoring operations, you can use the XCCDF standard for more flexibility in tailoring OpenSCAP profiles.

In addition, if your organization has been using OpenScap previously, you might have an existing XCCDF tailoring file and can reuse it.

The ComplianceSuite object contains an optional TailoringConfigMap attribute that you can point to a custom tailoring file. The value of the TailoringConfigMap attribute is a name of a config map which must contain a key called tailoring.xml and the value of this key is the tailoring contents.

Procedure

  1. Create the ConfigMap object from a file: 
    $ oc -n openshift-compliance \
    create configmap nist-moderate-modified \
    --from-file=tailoring.xml=/path/to/the/tailoringFile.xml
  2. Reference the tailoring file in a scan that belongs to a suite: 
    apiVersion: compliance.openshift.io/v1alpha1
    kind: ComplianceSuite
    metadata:
      name: workers-compliancesuite
    spec:
      debug: true
      scans:
        - name: workers-scan
          profile: xccdf_org.ssgproject.content_profile_moderate
          content: ssg-rhcos4-ds.xml
          contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:45dc...
          debug: true
      tailoringConfigMap:
          name: nist-moderate-modified
      nodeSelector:
        node-role.kubernetes.io/worker: ""

Performing a rescan​

You can re-run a scan on a defined schedule, such as every Monday or daily. It can also be useful to re-run a scan once after fixing a problem on a node.

To perform a single scan, annotate the scan with the compliance.openshift.io/rescan= option:

Procedure

  1. Annotate the scan to trigger a rescan:

    $ oc -n openshift-compliance \
    annotate compliancescans/rhcos4-e8-worker compliance.openshift.io/rescan=

    A rescan generates four additional mc for rhcos-moderate profile:

  2. Verify the rescan generated machine configs:

    $ oc get mc
    Example output
    75-worker-scan-chronyd-or-ntpd-specify-remote-server
    75-worker-scan-configure-usbguard-auditbackend
    75-worker-scan-service-usbguard-enabled
    75-worker-scan-usbguard-allow-hid-and-hub
    warning

    When the scan setting default-auto-apply label is applied, remediations are applied automatically and outdated remediations automatically update. If there are remediations that were not applied due to dependencies, or remediations that had been outdated, rescanning applies the remediations and might trigger a reboot. Only remediations that use MachineConfig objects trigger reboots. If there are no updates or dependencies to be applied, no reboot occurs.

Set custom storage size for results​

Although ComplianceCheckResult custom resources summarize one check across all scanned nodes, raw scanner results in ARF format are too large to store in etcd-backed Kubernetes resources.

You can store them on a per-scan persistent volume and increase the default 1 GiB size by setting the rawResultStorage.size value in a ScanSetting or ComplianceScan resource.

A related parameter is rawResultStorage.rotation which controls how many scans are retained in the PV before the older scans are rotated. The default value is 3, setting the rotation policy to 0 disables the rotation. Given the default rotation policy and an estimate of 100MB per a raw ARF scan report, you can calculate the right PV size for your environment.

Because OpenShift Container Platform can be deployed in a variety of public clouds or bare metal, the Compliance Operator cannot determine available storage configurations. By default, the Compliance Operator will try to create the PV for storing results by using the default storage class of the cluster, but a custom storage class can be configured using the rawResultStorage.StorageClassName attribute.

warning

If your cluster does not specify a default storage class, this attribute must be set.

  • Configure the ScanSetting custom resource to use a standard storage class and create persistent volumes that are 10GB in size and keep the last 10 results:
    Example ScanSetting CR
    apiVersion: compliance.openshift.io/v1alpha1
    kind: ScanSetting
    metadata:
      name: default
      namespace: openshift-compliance
    rawResultStorage:
      storageClassName: standard
      rotation: 10
      size: 10Gi
    roles:
    - worker
    - master
    scanTolerations:
    - effect: NoSchedule
      key: node-role.kubernetes.io/master
      operator: Exists
    schedule: '0 1 * * *'

Applying remediations generated by suite scans​

Although you can use the autoApplyRemediations boolean parameter in a ComplianceSuite object, you can alternatively annotate the object with compliance.openshift.io/apply-remediations. This allows the Operator to apply all of the created remediations.

Procedure

  • Apply the compliance.openshift.io/apply-remediations annotation by running the following command: 
    $ oc -n openshift-compliance \
    annotate compliancesuites/workers-compliancesuite compliance.openshift.io/apply-remediations=

Automatically update remediations​

In some cases, a scan with newer content might mark remediations as OUTDATED. As an administrator, you can apply the compliance.openshift.io/remove-outdated annotation to apply new remediations and remove the outdated ones.

Alternatively, set the autoUpdateRemediations flag in a ScanSetting or ComplianceSuite object to update the remediations automatically.

Procedure

  • Apply the compliance.openshift.io/remove-outdated annotation: 
    $ oc -n openshift-compliance \
    annotate compliancesuites/workers-compliancesuite compliance.openshift.io/remove-outdated=

Creating a custom SCC for the Compliance Operator​

In some environments, you must create a custom Security Context Constraints (SCC) file to ensure the correct permissions are available to the Compliance Operator api-resource-collector.

Prerequisites

  • You must have admin privileges.

Procedure

  1. Define the SCC in a YAML file named restricted-adjusted-compliance.yaml:

    SecurityContextConstraints object definition
      allowHostDirVolumePlugin: false
      allowHostIPC: false
      allowHostNetwork: false
      allowHostPID: false
      allowHostPorts: false
      allowPrivilegeEscalation: true
      allowPrivilegedContainer: false
      allowedCapabilities: null
      apiVersion: security.openshift.io/v1
      defaultAddCapabilities: null
      fsGroup:
        type: MustRunAs
      kind: SecurityContextConstraints
      metadata:
        name: restricted-adjusted-compliance
      priority: 30
      readOnlyRootFilesystem: false
      requiredDropCapabilities:
      - KILL
      - SETUID
      - SETGID
      - MKNOD
      runAsUser:
        type: MustRunAsRange
      seLinuxContext:
        type: MustRunAs
      supplementalGroups:
        type: RunAsAny
      users:
      - system:serviceaccount:openshift-compliance:api-resource-collector
      volumes:
      - configMap
      - downwardAPI
      - emptyDir
      - persistentVolumeClaim
      - projected
      - secret

    where:

    priority
    Specifies the priority of this SCC. This value must be higher than any other SCC that applies to the system:authenticated group.
    system:serviceaccount:openshift-compliance:api-resource-collector
    Specifies the Service Account used by Compliance Operator Scanner pod.

  2. Create the SCC:

    $ oc create -n openshift-compliance  -f restricted-adjusted-compliance.yaml
    Example output
    securitycontextconstraints.security.openshift.io/restricted-adjusted-compliance created

Verification

  1. Verify the SCC was created:

    $ oc get -n openshift-compliance scc restricted-adjusted-compliance
    Example output
    NAME                             PRIV    CAPS         SELINUX     RUNASUSER        FSGROUP     SUPGROUP   PRIORITY   READONLYROOTFS   VOLUMES
    restricted-adjusted-compliance   false   <no value>   MustRunAs   MustRunAsRange   MustRunAs   RunAsAny   30         false            ["configMap","downwardAPI","emptyDir","persistentVolumeClaim","projected","secret"]

Additional resources