Honor workflow-level COPILOT_MODEL in Copilot BYOK smoke workflows

[Copilot]

The Copilot BYOK smoke workflow could exceed the AI credits budget because the compiled lockfiles still allowed repo-level Copilot model defaults to override the workflow’s intended low-cost BYOK model. This change makes the BYOK smoke workflows consistently use their workflow-defined model selection.

• Root cause

  • BYOK smoke lockfiles were compiling agent env like:

    COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || env.COPILOT_MODEL }}

  • That let repository defaults win over the workflow’s explicit COPILOT_MODEL, which changed runtime model selection and increased token/credit usage.

• Postprocess fix

  • Updated scripts/ci/postprocess-smoke-workflows.ts to normalize BYOK smoke lockfiles to:

    COPILOT_MODEL: ${{ env.COPILOT_MODEL }}

  • This applies to all smoke-copilot-byok* lockfiles so workflow-level model pins remain authoritative even when repo-level defaults are configured.

• Generated workflow updates

  • Rewrote the compiled BYOK smoke lockfiles to remove the repo-level override path:
    • smoke-copilot-byok.lock.yml
    • smoke-copilot-byok-aoai-apikey.lock.yml
    • smoke-copilot-byok-aoai-entra.lock.yml

• Regression coverage

  • Expanded postprocess regex coverage to handle:
    • empty fallback
    • hardcoded fallback
    • fallback chains through GH_AW_DEFAULT_MODEL_COPILOT
    • existing vars.GH_AW_MODEL_AGENT_COPILOT || env.COPILOT_MODEL overrides
  • Added a focused BYOK workflow test to assert the compiled lockfiles keep using workflow-level COPILOT_MODEL.

Example of the effective compiled change:

# before
COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || env.COPILOT_MODEL }}

# after
COPILOT_MODEL: ${{ env.COPILOT_MODEL }}

[github-actions]

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	96.60%	96.64%	📈 +0.04%
Statements	96.47%	96.51%	📈 +0.04%
Functions	98.78%	98.78%	➡️ +0.00%
Branches	91.22%	91.25%	📈 +0.03%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
src/config-writer.ts	89.9% → 91.1% (+1.19%)	89.9% → 91.1% (+1.19%)

---

Coverage comparison generated by scripts/ci/compare-coverage.ts

[Copilot]

Pull request overview

This PR ensures the Copilot BYOK smoke workflows always honor the workflow-level COPILOT_MODEL pin (to keep token/credit usage predictable), by post-processing compiled lockfiles to remove repo-level model override expressions.

Changes:

• Update the smoke-workflow postprocessor to rewrite any compiled step-level COPILOT_MODEL: ${{ ... }} expression in BYOK smoke lockfiles to COPILOT_MODEL: ${{ env.COPILOT_MODEL }}.
• Regenerate the affected BYOK smoke lockfiles so they no longer allow repo vars to override the workflow’s intended model.
• Add regression tests to assert the BYOK source workflow pin and the lockfile normalization behavior.

Show a summary per file

File	Description
scripts/ci/postprocess-smoke-workflows.ts	Rewrites compiled BYOK lockfile COPILOT_MODEL expressions to always reference env.COPILOT_MODEL.
scripts/ci/postprocess-smoke-workflows.test.ts	Updates/expands regex unit coverage for the BYOK COPILOT_MODEL rewrite behavior.
scripts/ci/smoke-copilot-byok-workflow.test.ts	Adds a focused regression test asserting the source workflow pin and lockfile normalization.
.github/workflows/smoke-copilot-byok.lock.yml	Updates compiled lockfile to use COPILOT_MODEL: ${{ env.COPILOT_MODEL }} (no repo override).
.github/workflows/smoke-copilot-byok-aoai-apikey.lock.yml	Same compiled lockfile normalization for AOAI API key variant.
.github/workflows/smoke-copilot-byok-aoai-entra.lock.yml	Same compiled lockfile normalization for AOAI Entra variant.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 6/6 changed files
• Comments generated: 1

[lpcox]

@copilot address review feedback

[Copilot]

@copilot address review feedback

Addressed in 33cf18c.

[github-actions]

Smoke Test: Copilot PAT Auth Validation

Test	Result
GitHub MCP connectivity	✅
GitHub.com HTTP	✅ (200)
File write/read	✅

Overall: PASS

PR: "Honor workflow-level COPILOT_MODEL in Copilot BYOK smoke workflows" — author @Copilot, assignees @lpcox @Copilot

Auth mode: PAT (COPILOT_GITHUB_TOKEN)

🔑 PAT report filed by Smoke Copilot PAT

[github-actions]

Smoke Test: Copilot BYOK (Direct) Mode — PASS ✅

Test	Result
GitHub MCP connectivity	✅
GitHub.com connectivity	✅
File write/read	✅
BYOK inference (api-proxy → api.githubcopilot.com)	✅

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy sidecar to api.githubcopilot.com.

Overall: PASS | Authors: @Copilot, @lpcox

🔑 BYOK report filed by Smoke Copilot BYOK

[github-actions]

🧪 Smoke Test Results

Test	Status
GitHub MCP connectivity	✅
GitHub.com HTTP connectivity	✅
File write/read	✅

PR: Honor workflow-level COPILOT_MODEL in Copilot BYOK smoke workflows
Author: @Copilot | Assignees: @lpcox, @Copilot
Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot

[github-actions]

Smoke test results

• Merged PR: fix: add noop fallback to smoke-copilot workflows on non-PR triggers
• Merged PR: Reduce test-coverage-reporter prompt surface and coverage payload size
• GitHub PR list: ✅
• GitHub title check: ✅
• File write/read: ✅
• Discussion lookup/comment: ✅
• Build: ✅
  Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

[github-actions]

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	1/1 passed	✅ PASS
Node.js	execa	✅	1/1 passed	✅ PASS
Node.js	p-limit	✅	1/1 passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #4876 · ◷

[github-actions]

Smoke Test Results

• ❌ Redis (host.docker.internal:6379): timeout — no response
• ❌ PostgreSQL pg_isready (host.docker.internal:5432): no response
• ❌ PostgreSQL SELECT 1: unreachable

Overall: FAIL

host.docker.internal resolves to 172.17.0.1 but both ports are unreachable (connection timeout). Service containers may not be running or ports are blocked.

🔌 Service connectivity validated by Smoke Services

[github-actions]

PR #4876 by @Copilot, @lpcox

• GitHub MCP connectivity: ✅
• GitHub.com connectivity: ✅
• File write/read: ✅
• BYOK inference: ✅
  Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
  Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

[github-actions]

Honor workflow-level COPILOT_MODEL in Copilot BYOK smoke workflows: ✅
GitHub.com connectivity: ✅
File I/O smoke test: ✅
Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw): ✅
Overall: PASS
@Copilot @lpcox

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)