ci: investigate Debian action preparation 404 and propose runner mitigation #4085

User Story

Matt (MG) needs reliable OpenShell Debian packaging and downstream conformance checks while qualifying changes through the merge queue. After the Python PTY fix #4076 merged, Matt observed the Debian action-download failure and requested this focused investigation and a reviewable mitigation proposal.

Description

Investigate the action preparation failure in Branch E2E run 36931039020, and prepare a concrete runner configuration proposal if the failure cannot be corrected inside an OpenShell workflow.

On October 1, 2026 at 21:58:38 UTC, runner 2.337.0 failed to download the existing actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c archive from codeload. Only Set up job ran. Its ActionNotFoundException corresponds to HTTP 404 in the exact runner source and bypasses the existing three-attempt retry loop. Neither workflow steps nor action-body retries can intercept it. Same-pin Debian packaging succeeded in run 36914303849.

Impact / Why This Matters

The failure canceled the arm64 sibling and skipped package-dependent integration lanes, including K3s conformance. Manual reruns spend qualification time without addressing the failing boundary. The underlying reason for the 404 remains unproven.

Context

OpenShell SHA: 76cfd0e31d5e1633db7ccd86ad9023ef7a2461b2; NVIDIA managed Linux amd64 CPU runner, runtime v1.9.0, image 24b0d8a. There is no demonstrated OpenShell deployment reproduction because packaging never started. The original CI job is the observed reproduction; repeatability is unproven. This internal investigation task does not claim a runtime bug.

Related tracker: #3954. This task covers diagnosis and a reviewable proposal only; fleet rollout and proof of prevention remain explicit infrastructure dependencies. Do not close the broader testing tracker.

Definition of Done

• Preserve the first-failure evidence and compare successful same-pin runs.
• Verify actual runner retry/cache controls in primary sources; do not invent a workflow retry setting.
• Prepare a concrete configuration proposal with bounded population retries, failure behavior, integrity/ownership requirements, and fleet validation criteria.
• Validate the proposal's commands with controlled transient/permanent failures where practical.
• Run and report Debian packaging and downstream E2E on the draft PR SHA with matching artifacts, distinguishing baseline results from mitigation activation.
• Keep the PR draft and explicitly report runner-admin rollout limitations.