Skip to content

Idea: a final sweep step to catch invisible workflow_call startup failures #2

Description

@non7top

Not urgent — capturing the idea while it's fresh, not proposing a design yet.

What prompted this

non7top/sort4print#38 hit a real case: a caller passed package-name to
release-please-preview.yml before this repo's workflow_call.inputs
actually declared it (#1 was still open). GitHub
rejected the run with conclusion: startup_failure — and that failure
was completely invisible on the PR. gh pr checks 38 showed nothing at
all for it; only "Core tests", "Windows exe", etc.

Root cause, confirmed directly against the run
(gh api .../actions/runs/36244385379/attempts/1/jobs → total_count: 0):
GitHub's checks are posted per-job, not per-run. Any failure that
happens during workflow setup — before a job is ever scheduled — has
nothing for a check-run to attach to. That's not unique to undeclared
inputs; the same zero-job dead end applies to a bad uses: ref/tag on a
reusable workflow, a missing required input, a required secret not
passed through, or exceeding the reusable-workflow nesting depth. All of
it is only visible by directly querying run history
(gh run list --branch <b> --workflow <name> --json conclusion), never
via the PR itself.

Confirmed this isn't caught by linting either: actionlint (same version
release-please-forecast's own .pre-commit-config.yaml pins) does have
a check for this — the workflow-call rule — but only for a local,
same-repo reusable workflow reference. It doesn't fetch a remote
org/repo/...@ref workflow to validate a caller's with: block against
it, so a cross-repo case like this one sails through clean.

The rough idea

Feels like the right eventual shape is some kind of final sweeping
action — something that runs after (or alongside) the callers of this
repo's reusable workflows and explicitly verifies each call actually
produced a job/outcome, converting a silent zero-job dead end into a
real, PR-visible signal — rather than every caller repo having to notice
on its own that a check never showed up.

Not spec'd out. Candidate directions, none committed to:

  • A workflow_run-triggered watchdog in each caller repo that checks
    conclusion and comments/fails explicitly when it's not success.
    Downside: workflow_run only watches workflows in its own repo, so
    this can't live centrally in gh-workflows — it'd have to be copied into
    every caller.
  • Making the called job (e.g. preview / preview) a required status
    check on the caller's branch protection, so a missing report blocks
    merge instead of vanishing. Downside: turns a best-effort preview bot
    into a hard merge gate for every future PR, not just this failure mode.
  • Some periodic/scheduled audit across known caller repos that lists
    recent runs and flags startup_failure (or zero-job runs generally)
    directly via the API, sidestepping the PR-checks blind spot entirely.

Context

Activity

  1. non7top commented on Sep 26, 2026

    @non7top
    OwnerAuthor

    Also worth folding in: a lint action gating PRs that touch workflow/action
    files — actionlint + check-jsonschema (mirroring release-please-forecast's
    own .pre-commit-config.yaml) as a required check whenever a PR changes
    .github/workflows/** or action.yml, in this repo and in caller repos
    like sort4print.

    Complementary to the sweep idea above, not a replacement: lint catches
    same-repo authoring mistakes (bad expressions, malformed YAML, a typo'd
    local reusable-workflow input) before merge, on the PR that introduces
    them. It doesn't close the cross-repo gap actionlint already can't see —
    sort4print's PR would still have passed lint clean, since the undeclared
    package-name input was only invalid against a workflow in a different
    repo. That half still needs whatever the sweep ends up being.

    So: lint on the authoring side, sweep on the actually-ran side. Neither
    alone would have caught the sort4print case; both together would have —
    lint doesn't apply cross-repo, but the sweep would've flagged the
    zero-job startup_failure directly.

  2. non7top commented on Sep 26, 2026

    @non7top
    OwnerAuthor

    Done: #3 (shared `actions-lint.yml` reusable
    workflow) and non7top/sort4print#39 (wires sort4print into it). Both
    scoped to what lint can actually catch — same-repo authoring mistakes,
    not the cross-repo `with:`-vs-remote-schema gap that's the rest of
    this issue. Leaving this open for the sweep half.

  3. non7top commented on Sep 26, 2026

    @non7top
    OwnerAuthor

    So making this action a final action + making it required, would achieve that. Apparently the only way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions