containers-storage policy key uses the unresolved graph root, so rebase fails when the store is a symlink

[qxtaiba]

Description

A node degrades during an OS update with is rejected by policy. This needs two things to be true: the container storage graph root is a symlink, and policy.json has a restrictive default.

patchPoliciesForContainerStorage is meant to cover this. It does run. It logs Temporal allow policy added for URL on every attempt and never logs Error writing temporal policy files. The rule it writes just can't be found, because the writer and the evaluator build the scope from different strings:

written   (rpm-ostree.go, from `podman system info`):  [overlay@/var/lib/containers/storage]<repo-digest>@<image-id>
looked up (PolicyConfigurationIdentity):               [overlay@/var/lib/kubelet/containers/storage]<repo-digest>@<image-id>

podman system info gives the graphroot as written in storage.conf. containers/storage runs it through filepath.EvalSymlinks when the store opens, so storageReference.PolicyConfigurationIdentity() and everything in PolicyConfigurationNamespaces() carry the resolved path instead. Nothing matches, so evaluation falls through to default.

Anyone who puts the container store on a separate disk behind a symlink will hit this.

Steps to reproduce the issue:

1. Relocate the container store: ln -sfT /var/lib/kubelet/containers/storage /var/lib/containers/storage
2. Set registrySources.allowedRegistries on image.config.openshift.io/cluster, which makes the container runtime config controller render a policy.json with default: reject and no containers-storage transport
3. With the PinnedImages feature gate on and a PinnedImageSet referenced by the pool, trigger an OS update so MCD takes the local container storage rebase path

Describe the results you received:

rpm-ostree.go] Executing local container storage rebase to <registry>/<repo>@sha256:<digest>
update.go] Running: rpm-ostree rebase --experimental ostree-unverified-image:containers-storage:<registry>/<repo>@sha256:<digest>
update.go] Rolling back applied changes to OS due to error: failed to update OS from local storage: ... error: Creating importer: failed to invoke method OpenImage: failed to invoke method OpenImage: Running image containers-storage:[overlay@/var/lib/kubelet/containers/storage+/run/containers/storage:overlay.skip_mount_home=true]<registry>/<repo>@sha256:<digest>@<image-id> is rejected by policy.
writer.go] Marking Degraded due to: "failed to update OS from local storage: ... is rejected by policy"

The pool sits at 0 updated and 1 degraded, retrying roughly every 80s, and never gets anywhere.

Describe the results you expected:

The rebase works, because the allow rule written for that image is the one the evaluator looks up.

Additional information you deem important:

• Happens every time, not intermittent.
• Hard to spot from the logs, since the workaround claims success. grep 'Temporal allow policy added for URL' and look at the graph root in the key.
• The error return from patchPoliciesForContainerStorage is swallowed on purpose, so a genuine write failure would be silent too.
• Seen on release-4.20. Same code on main.
• Fix and regression tests in the linked PR.

Output of oc adm release info --commits | grep machine-config-operator:

4.20.3

Additional environment details (platform, options, etc.):

Bare metal / platform: none, multi-node, RHCOS 9.6, container store relocated to a separate disk via symlink.