[repository-quality] 🎯 Repository Quality Improvement Report - CLI Sibling-Command Flag & Safety-Guard Parity Gaps (2026-09-25)

[github-actions[bot]]

🎯 Repository Quality Improvement Report - CLI Sibling-Command Flag & Safety-Guard Parity Gaps

Analysis Date: 2026-09-25
Focus Area: CLI Sibling-Command Flag & Safety-Guard Parity Gaps
Strategy Type: Custom
Custom Area: Yes — ADR-45573 (2026-07-15) documented and fixed four flag-parity gaps between compile/validate/update/upgrade (adding --engine/--repo to upgrade and --approve to update), and explicitly deferred one gap (--no-security-scanner on compile/validate) as "intentionally skipped... because those commands operate exclusively on local files where the scanner is a no-op." This run re-audited the four commands post-fix and found the ADR's own scope was incomplete: upgrade internally invokes the exact same security-scan-enforcing compile/update pipeline as update, but never registered the escape hatch update has (--no-security-scanner), creating a real (not hypothetical) parity gap the ADR missed. Combined with 3 verified stale file-path references in .github/skills/, this is a two-pronged consistency-debt area: CLI surface parity and skill-documentation path accuracy.

Executive Summary

ADR-45573 fixed three of four identified flag-consistency gaps between the compile, validate, update, and upgrade commands, but its "F3" deferral reasoning — that --no-security-scanner is a no-op on compile/validate because they operate on local files only — does not extend to upgrade. Tracing upgrade's Step 4 compile call (compileWorkflowsForUpdate → shared CompileWorkflows pipeline, pkg/cli/upgrade_command.go:347-368) confirms upgrade shares the same enforcement code path as update's manifest-managed-workflow rewrite (pkg/cli/update_manifest.go:582-585,629-632), where ScanMarkdownSecurity findings hard-fail with the exact message "...or pass --no-security-scanner to skip this check" — a flag that does not exist on upgrade. Any CI pipeline running gh aw upgrade on a workflow whose upstream manifest source content trips a security-scan finding has no way to bypass it short of falling back to update or manual editing, reproducing the exact UX problem ADR-45573 was written to solve, just for a flag the ADR didn't audit.

Separately, a targeted repo-relative path audit of .github/skills/*/SKILL.md files found 3 confirmed-broken references: awf-release-integrator points at nonexistent docs/awf-config-spec.md / docs/awf-config.schema.json (the real schema lives at pkg/workflow/schemas/awf-config.schema.json; the spec lives at specs/awf-config-sources-spec.md), otel-queries points at a docs/src/content/docs/guides/custom-otlp-attributes.md that was never created, and messages still tells contributors to edit pkg/workflow/safe_outputs.go, a file that was split into 40+ safe_outputs_*.go files (confirmed via find) with no single-file successor of that name.

Recommended actions: (1) add --no-security-scanner to upgrade and thread it through to compileWorkflowsForUpdate/the manifest-update call sites it triggers, matching update's existing wiring; (2) add a lightweight CI-gated consistency test enumerating flags across the four sibling commands so gaps like this are caught automatically rather than requiring manual re-audits (ADR-45573 already added some tests for the 3 flags it fixed, but nothing generalized); (3) fix the 3 stale skill path references identified above.

Full Analysis Report

Focus Area: CLI Sibling-Command Flag & Safety-Guard Parity Gaps

Current State Assessment

Metrics Collected:

Metric	Value	Status
upgrade has --no-security-scanner flag	No (0 occurrences in pkg/cli/upgrade_command.go)	❌
update has --no-security-scanner flag	Yes (pkg/cli/update_command.go:177)	✅
upgrade shares compile/security-scan pipeline with update	Confirmed — both call CompileWorkflows/compileWorkflowsForUpdate	⚠️
Sibling commands with generalized flag-parity test	0 found (only targeted tests for ADR-45573's 3 flags)	⚠️
Broken skill doc path references (verified)	3 of 3 checked	❌
Prior ADR (45573) scope	Audited only 4 gaps, missed upgrade/--no-security-scanner	⚠️

Findings

Strengths

• ADR-45573 process (audit → fix → document → test) is a good model; the 3 flags it added (--engine/--repo on upgrade, --approve on update) are correctly wired and covered by unit tests.
• The security-scanner enforcement itself (ScanMarkdownSecurity) is consistently applied across add, add-wizard, deploy, trial, and update — upgrade is the sole omission found.
• Existing DisableSecurityScanner plumbing (pkg/cli/flags.go:42-47, resolveDeprecatedBoolFlag) is a reusable, already-tested pattern — wiring upgrade is a small, low-risk change.

Areas for Improvement

• 🟠 High: gh aw upgrade cannot bypass a security-scan hard-failure that gh aw update can, despite running through the identical enforcement code (update_manifest.go:582-585,629-632) via the shared compile pipeline. This silently blocks CI upgrade automation with no documented workaround other than switching commands.
• 🟡 Medium: No automated test asserts flag-set parity across compile/validate/update/upgrade; ADR-45573's fix required manual discovery and will silently regress as new flags are added to one sibling but not others.
• 🟡 Medium: 3 skill files (awf-release-integrator, otel-queries, messages) reference nonexistent files, actively misdirecting future agent runs that follow those skills.

Detailed Analysis

1. The upgrade / --no-security-scanner gap

pkg/cli/upgrade_command.go Step 4 (lines 347-368) calls compileWorkflowsForUpdate(opts.ctx, nil, opts.workflowDir, opts.engineOverride, opts.verbose, opts.approve), which (per pkg/cli/update_compile.go:28-44) is explicitly documented as using "the same configuration and orchestration as the compile command." This is the identical function update_command.go calls after update_manifest.go's manifest-workflow-rewrite path enforces ScanMarkdownSecurity (lines 582, 629). upgrade_command.go registers --engine, --repo, --approve, --org, --repos, --create-issue, --yes, --audit, --pre-releases, --no-fix, --no-actions, --no-compile, --disable-codemod, --create-pull-request/--pr — but never --no-security-scanner/--disable-security-scanner, which is present on add, add-wizard, deploy, trial, and update.

Because opts.approve only bypasses strict-mode manifest enforcement (new secrets/actions), not the separate ScanMarkdownSecurity findings check, --approve is not a substitute — a workflow tripping a security-scan finding during upgrade has no flag-based escape hatch at all today.

2. Missing flag-parity regression guard

ADR-45573's "Consequences" section notes: "Flag coverage is verified by new unit tests for registration, description consistency, and mutual exclusion" — but these tests target only the 3 flags fixed in that ADR, not a generalized cross-command flag inventory. A single parametrized test comparing the flag sets of compile, validate, update, upgrade (with an explicit allowlist for intentional command-specific flags) would have caught this gap immediately and will prevent future ones from requiring a full manual re-audit like this one.

3. Stale skill path references

Skill File	Broken Reference	Correct Path
.github/skills/awf-release-integrator/SKILL.md:32-33	docs/awf-config-spec.md, docs/awf-config.schema.json	specs/awf-config-sources-spec.md, pkg/workflow/schemas/awf-config.schema.json
.github/skills/otel-queries/SKILL.md:273	docs/src/content/docs/guides/custom-otlp-attributes.md	File does not exist anywhere in docs/; reference should be removed or the guide created
.github/skills/messages/SKILL.md:68	pkg/workflow/safe_outputs.go	File was decomposed into 40+ pkg/workflow/safe_outputs_*.go files; closest match for "parser" is pkg/workflow/safe_outputs_parser.go

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Add --no-security-scanner flag to gh aw upgrade

Priority: High
Estimated Effort: Small
Focus Area: CLI Sibling-Command Flag Parity

Description: gh aw upgrade runs through the same compile/security-scan-enforcing pipeline as gh aw update (via compileWorkflowsForUpdate), but lacks the --no-security-scanner flag that update (and add, add-wizard, deploy, trial) already expose via addSecurityScannerFlag/resolveDeprecatedBoolFlag. Add the flag to upgrade_command.go, thread the resolved bool value through to the relevant compile/manifest-update call sites triggered by upgrade, and update docs/src/content/docs/setup/cli.md accordingly (matching how ADR-45573 documented its 3 flag additions).

Acceptance Criteria:

• cmd.Flags() in pkg/cli/upgrade_command.go registers --no-security-scanner (and deprecated --disable-security-scanner alias) using the existing addSecurityScannerFlag helper from pkg/cli/flags.go
• The resolved flag value is threaded to whatever manifest-managed-workflow update path upgrade triggers, so ScanMarkdownSecurity findings can be bypassed identically to update
• docs/src/content/docs/setup/cli.md documents the new flag on upgrade
• New/updated unit test asserts the flag is registered and its value reaches the security-scan-skip decision (mirroring the test pattern ADR-45573 used for --engine/--repo/--approve)

Code Region: pkg/cli/upgrade_command.go (flag registration + runUpgradeCommand), pkg/cli/flags.go (addSecurityScannerFlag)

In pkg/cli/upgrade_command.go, add the --no-security-scanner flag (with deprecated
--disable-security-scanner alias) to gh aw upgrade using the existing addSecurityScannerFlag
helper in pkg/cli/flags.go, matching how it's already wired on gh aw update
(pkg/cli/update_command.go:87,177). Thread the resolved DisableSecurityScanner value through
runUpgradeCommand to whatever manifest-managed-workflow update call path is triggered during
upgrade's compile step, so that a ScanMarkdownSecurity finding can be bypassed on upgrade the
same way it already can be on update (see pkg/cli/update_manifest.go:582-585,629-632 for the
enforcement pattern to match). Update docs/src/content/docs/setup/cli.md to document the new
flag on upgrade. Add a unit test verifying the flag is registered and its value is correctly
propagated to the security-scan-skip decision, following the existing test pattern used for
ADR-45573's --engine/--repo/--approve additions.

---

Task 2: Add a cross-command flag-parity regression test

Priority: Medium
Estimated Effort: Medium
Focus Area: CLI Sibling-Command Flag Parity

Description: ADR-45573 fixed a set of flag gaps found by manual audit, but no automated test guards against the same class of drift recurring (as this run's finding demonstrates it already has, for --no-security-scanner on upgrade). Add a test that enumerates the registered flags on compile, validate, update, and upgrade, and asserts a documented, explicit allowlist for any command-specific flags — flagging any new asymmetric flag addition as a failure requiring either symmetric rollout or an allowlist update.

Acceptance Criteria:

• New test (e.g. pkg/cli/flag_parity_test.go) walks the flag sets of compile, validate, update, upgrade cobra commands
• Test fails if a flag exists on one command but not its siblings, unless present in an explicit, commented allowlist explaining the intentional asymmetry (e.g. --no-security-scanner deliberately absent from compile/validate per ADR-45573's F3 rationale)
• Test currently passes once Task 1 is merged (i.e., does not itself require Task 1, but should be added after or alongside it)

Code Region: pkg/cli/flags_test.go, new pkg/cli/flag_parity_test.go

Add a new Go test file pkg/cli/flag_parity_test.go that programmatically compares the flag
sets registered on the compile, validate, update, and upgrade cobra commands (constructed via
their New*Command functions). The test should assert that any flag present on one command is
present on all others UNLESS it appears in an explicit, code-documented allowlist of
intentional asymmetries (reference ADR-45573's F3 deferral of --no-security-scanner on
compile/validate as the first allowlist entry, with its documented rationale as the comment).
Fail the test with a clear message identifying which command(s) are missing which flag(s) when
an unlisted asymmetry is detected. This closes the gap that let --no-security-scanner remain
missing from `upgrade` despite ADR-45573's flag-consistency audit.

---

Task 3: Fix stale file-path references in awf-release-integrator and otel-queries skills

Priority: Medium
Estimated Effort: Small
Focus Area: Skill Documentation Accuracy

Description: .github/skills/awf-release-integrator/SKILL.md references docs/awf-config-spec.md and docs/awf-config.schema.json, neither of which exists; the real files are specs/awf-config-sources-spec.md and pkg/workflow/schemas/awf-config.schema.json. .github/skills/otel-queries/SKILL.md references docs/src/content/docs/guides/custom-otlp-attributes.md, which was never created anywhere under docs/. Update both skills to reference the correct existing paths (or remove the otel-queries line if no replacement guide is planned).

Acceptance Criteria:

• awf-release-integrator/SKILL.md lines 32-33 updated to specs/awf-config-sources-spec.md and pkg/workflow/schemas/awf-config.schema.json
• otel-queries/SKILL.md line 273 either removed or updated to point at an actual existing doc (verify no in-repo replacement exists before removing)
• grep re-verification confirms no remaining reference to the removed/renamed paths in either skill file

Code Region: .github/skills/awf-release-integrator/SKILL.md, .github/skills/otel-queries/SKILL.md

In .github/skills/awf-release-integrator/SKILL.md, replace the broken references to
docs/awf-config-spec.md and docs/awf-config.schema.json (lines ~32-33) with the correct
existing paths: specs/awf-config-sources-spec.md and pkg/workflow/schemas/awf-config.schema.json
respectively. In .github/skills/otel-queries/SKILL.md, remove or fix the reference (around
line 273) to docs/src/content/docs/guides/custom-otlp-attributes.md, which does not exist
anywhere in the docs/ tree — verify first whether an equivalent guide exists under a different
name before deciding whether to delete the line or repoint it. After editing, grep the repo to
confirm no other skill or doc still references the old broken paths.

---

Task 4: Fix stale pkg/workflow/safe_outputs.go reference in messages skill

Priority: Low
Estimated Effort: Small
Focus Area: Skill Documentation Accuracy

Description: .github/skills/messages/SKILL.md (line 68) instructs contributors to "update the workflow parser in pkg/workflow/safe_outputs.go" for custom message-parsing logic, but this file no longer exists — it was decomposed into 40+ files (pkg/workflow/safe_outputs_parser.go, safe_outputs_config_base.go, safe_outputs_handler_registry*.go, etc.) as part of the repo's ongoing large-file decomposition effort (see 2026-08-24 focus-area history). Update the reference to point at the correct current file(s) for message parsing logic.

Acceptance Criteria:

• Line 68 of .github/skills/messages/SKILL.md no longer references the nonexistent pkg/workflow/safe_outputs.go
• The updated reference correctly identifies which of the safe_outputs_*.go files (most likely safe_outputs_parser.go and/or safe_outputs_config_base.go) actually contains the reflection-based parser and custom-parsing-logic hook described in that sentence
• Verified by reading the target file(s) to confirm the described "reflection-based parser" logic actually lives there before finalizing the reference

Code Region: .github/skills/messages/SKILL.md, pkg/workflow/safe_outputs_parser.go

In .github/skills/messages/SKILL.md line 68, the sentence "If the message needs custom
parsing logic, update the workflow parser in pkg/workflow/safe_outputs.go or the relevant
config block" references a file that no longer exists (it was split into 40+
pkg/workflow/safe_outputs_*.go files). Read pkg/workflow/safe_outputs_parser.go and
pkg/workflow/safe_outputs_config_base.go to confirm which one(s) actually contain the
reflection-based parsing logic described, then update the skill's reference to name the
correct current file(s).

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-09-24	JS Action-Script Unit Test Coverage Gaps	Custom	Y	6 production scripts with zero test reachability identified
2026-09-23	Performance	Standard	N	Compiler confirmed single-goroutine; benchmark cold-cache bias found
2026-09-22	Integration Test Binary-Path Duplication	Custom	Y	32 duplicated hardcoded binary-discovery blocks found
2026-09-21	Custom Go Linter CI Enablement Gap (2nd follow-up)	Custom	Y	13/72 linters still unwired; prior "fixed" linters regressed
2026-09-18	Unjustified nolint Suppression Debt	Custom	Y	96/260 nolint directives lack justification
2026-08-24	Large File Decomposition Debt	Custom	Y	32 files over 800 lines found — precursor to today's safe_outputs.go split confirmed complete

---

🎯 Recommendations

Immediate Actions (This Week)

1. Add --no-security-scanner to gh aw upgrade (Task 1) — Priority: High
2. Fix the 3 stale skill path references (Tasks 3-4) — Priority: Medium/Low

Short-term Actions (This Month)

1. Add the cross-command flag-parity regression test (Task 2) — Priority: Medium
2. Consider a general "skill path linter" that greps .github/skills/*/SKILL.md for repo-relative path references and verifies they resolve, run in CI, to prevent this class of drift recurring (as it has 3 times now across messages, javascript-refactoring, error-pattern-safety, temporary-id-safe-output, awf-release-integrator, and otel-queries per this and prior runs)

Long-term Actions (This Quarter)

1. Extend the ADR-45573 flag-consistency review to cover add, add-wizard, deploy, and trial as a fifth sibling-group, since they also share DisableSecurityScanner/compile-adjacent logic and may harbor similar undetected asymmetries — Priority: Low

---

📈 Success Metrics

• upgrade/update flag parity: 1 known gap (--no-security-scanner) → 0 gaps
• Automated flag-parity coverage: 0 generalized tests → 1 regression-guarding test covering 4 sibling commands
• Broken skill path references (this batch): 3 confirmed broken → 0

---

Next Steps

1. Review and prioritise the tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate this focus area in 4-6 weeks to confirm no new flag-parity gaps have emerged in add/add-wizard/deploy/trial

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-09-26 — Focus area selected by diversity algorithm

Generated by ⚡ Repository Quality Improvement Agent · copilot · auto · 172.1 AIC · ⌖ 9.32 AIC · ⊞ 9.8K · ◷

• expires on Sep 26, 2026, 5:12 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-09-26T13:12:38.294Z.

Closed by Workflow