[Bug] [CLI] Kyverno test does not work correctly for mutation on Namespaces

Kyverno CLI Version

1.15.0

Description

I am trying to run a test for one of my ClusterPolicies. The outcome should yield a pass; however I get a fail with the reasoning "Not Found".

  Loading values/variables ...
  Loading policies ...
  Loading resources ...
  Loading exceptions ...
  Applying 1 policy to 1 resource with 0 exceptions ...
  Checking results ...

│────│──────────────────────────────│──────────────────────│────────────────────────│────────│───────────│───────────│
│ ID │ POLICY                       │ RULE                 │ RESOURCE               │ RESULT │ REASON    │ MESSAGE   │
│────│──────────────────────────────│──────────────────────│────────────────────────│────────│───────────│───────────│
│ 1  │ add-or-warn-context-ns-label │ add-context-ns-label │ v1/Namespace/Namespace │ Fail   │ Not found │ Not found │
│────│──────────────────────────────│──────────────────────│────────────────────────│────────│───────────│───────────│


Test Summary: 0 tests passed and 1 tests failed

Steps to reproduce

1. Create the ClusterPolicy:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-or-warn-context-ns-label
spec:
  rules:
    - name: add-context-ns-label
      skipBackgroundRequests: false
      exclude:
        any:
          - resources:
              selector:
                matchExpressions:
                  - key: context
                    operator: In
                    values:
                      - "app"
                      - "infra"
                      - "lab"

      match:
        all:
          - resources:
              kinds:
                - Namespace
          - resources:
              name: "*" # needed for testing because the kyverno cli doesn't match just kind: Namespace resources
      mutate:
        patchStrategicMerge:
          metadata:
            labels:
              context: "error" # this value has to be changed in the overlays
        mutateExistingOnPolicyUpdate: true
        targets:
          - apiVersion: v1
            kind: Namespace
            preconditions:
              any:
                - key: "{{ target.metadata.labels.context }}"
                  operator: NotEquals
                  values:
                    - "app"
                    - "infra"
                    - "lab"

2. Create the kyverno-test.yaml:

apiVersion: cli.kyverno.io/v1alpha1
kind: Test
metadata:
  name: test-add-or-warn-context-label-clusterpolicy
policies:
  - ../../policies/label-enforcement/add-or-warn-context-ns-label.yaml
resources: # other Namespace resource as "trigger"
  - resources/test-sample-namespace.yaml
targetResources: #raw, unmutated resources
  - resources/default-sample-namespace.yaml
results:
  # checks whether policy adds `context: "error"` if not present
  # the raw ClusterPolicy is patched per overlay so the correct output would be "error"
  - policy: add-or-warn-context-ns-label
    rule: add-context-ns-label
    isMutatingPolicy: true
    resources:
      - default
    patchedResources: resources/patched-default-namespace.yaml # mutated resource
    kind: Namespace
    result: pass

3. Create the test resources:

# test-sample-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: test-namespace-for-test-purposes
  labels:
    sample-label: test

# default-sample-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: default
  labels:
    sample-label: test

# patched-default-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: default
  labels:
    sample-label: test
    context: "error"

4. Run the cli test via kyverno test --detailed-results <path_to_kyverno-test.yaml>

Expected behavior

I expected the test to show a pass - and I also fail to understand what exactly was not found. If I remove the wildcard from the selector of the Policy, the test also gets skipped outright.

Screenshots

No response

Kyverno logs

Slack discussion

No response

Troubleshooting

• I have read and followed the troubleshooting guide.
• I have searched other issues in this repository and mine is not recorded.