Report a scan's CI blocking-rule verdict from corgea list, and write the report before the gate exits - #154
Conversation
--out-format/--out-file ran after the --fail and --block-on gates, so a scan that violated a blocking rule exited 1 without ever writing the report. A pipeline that gates on policy is exactly the one that needs the report file, to ingest the findings it just failed on. The report and the SBOM now run before the gates. Both bodies moved into write_scan_report and write_sbom rather than being reordered in place, which also collapses the four near-identical out-format branches into one server-rendered path plus the JSON case. Co-authored-by: ibrahim <ibrahim@corgea.com>
A pipeline that skips a duplicate scan for a commit it has already scanned has no verdict to gate on: --block-on lives on the scan, so the skip path fell back to counting vulnerabilities from the previous scan, which is a different question from the one the CI rules answer. corgea list --block-on <slugs> attaches a blocking_verdict to every listed scan: the rules asked for, whether they blocked, how many issues did, and which rules tripped. Only 'complete' is a final answer; a scan still running, one that failed, or a server still resolving license dependencies reports 'unavailable'/'pending' with block null rather than a field a consumer would read as a pass. An evaluation error exits 1 for the same reason. --sha narrows the listing to one commit, which is how the duplicate-skip path reaches its scan in a single request. It takes the full SHA because the server matches exactly, and the returned scans are re-checked against it so a backend that ignores the filter cannot answer with another commit's verdict. A verdict costs one request per scan and the endpoint re-evaluates every finding, so the pass is capped and the default page shrinks to the ten scans it evaluates. Co-authored-by: ibrahim <ibrahim@corgea.com>
corgea ls --block-on/--sha are new backward-compatible flags and the report ordering only adds a guarantee, so SemVer puts this at a minor bump. Cargo.toml is the single source of truth: PyPI reads it via maturin and npm takes its version from the release tag. Co-authored-by: ibrahim <ibrahim@corgea.com>
The scan listing exits 1 with "No Corgea project found" when an unconfirmed project returns no scans, which is the right answer for a listing of every scan. With --sha an empty page is the expected answer for a commit that has not been scanned yet — the case a duplicate-skip path handles by scanning — so it now reports the empty result and says which commit had no scan. Co-authored-by: ibrahim <ibrahim@corgea.com>
| .filter(|scan| { | ||
| scan.git_sha | ||
| .as_deref() | ||
| .is_some_and(|value| value.trim().eq_ignore_ascii_case(sha)) |
There was a problem hiding this comment.
P1 — Do not treat a dirty snapshot as the commit's reusable scan. This filter only compares git_sha. However, BLAST uploads retain the HEAD SHA when the archived worktree differs and explicitly send dirty=true (src/scanners/blast.rs:241-246, src/utils/api.rs:353-357); targeted/excluded scans are marked dirty for the same reason. A dirty or partial scan at H can therefore become .results[0] for a clean CI checkout at H, and the documented jq gate can accept that scan's block: false and skip scanning the code that will actually be deployed. Deserialize the scan-list worktree_dirty field and only reuse an exact clean snapshot (Some(false)) for --sha blocking verdicts; fail closed/force a fresh scan for true or legacy null. Add an end-to-end case where a dirty scan with the requested SHA is returned first.
There was a problem hiding this comment.
I agree with this finding and think it should be addressed.
high: Dirty snapshots can be reused as clean commit scans
SHA filtering checks only git_sha, so a dirty or partial scan associated with the same HEAD is retained and can supply a passing verdict for different source content. For duplicate-scan gating, require an explicitly clean snapshot; treat dirty or unknown cleanliness as unavailable.
Proof or reproduction:
Given {"git_sha":"<HEAD>","worktree_dirty":true,"blocking_verdict":{"status":"complete","block":false}}, retain_scans_at_sha keeps the scan because it compares only git_sha, allowing the documented results[0] gate to pass.
| }, | ||
| "block": response.block, | ||
| "blocked_issues": response.blocked_count(), | ||
| "triggered_rules": triggered_slugs(&response.blocking_issues), |
There was a problem hiding this comment.
P2 — triggered_rules is incomplete once blocking issues paginate. check_blocking_rules(..., None, ...) sends page=1, while BlockingRuleResponse exposes total_pages and explicitly documents that blocking_issues contains only the requested page (src/utils/api.rs:1226-1239, 1581-1607). Deriving the field from this one page omits any rule whose violations appear only on a later page, even though the new JSON contract and example present triggered_rules as the scan's rule set. Fetch pages 2..=total_pages and aggregate/deduplicate their slugs before emitting the verdict (or remove/rename the field so it does not claim a complete set), with a two-page test where page 2 contains a different rule.
There was a problem hiding this comment.
I agree with this finding and think it should be addressed.
nitpick: triggered_rules omits rules from later pages
The blocking-rule request fetches one page, but triggered_rules is derived solely from that page. For paginated violations, the reported rule set is incomplete. Aggregate all response pages or rename/document the field as partial.
Proof or reproduction:
Page 1: total_pages=2, triggered_by_slugs=["criticals"]; page 2: triggered_by_slugs=["malicious-deps"]. The emitted triggered_rules is only ["criticals"].
| } else { | ||
| VERDICT_STATUS_PENDING | ||
| }, | ||
| "block": response.block, |
There was a problem hiding this comment.
P1 — A pending evaluation currently exposes a false-pass boolean. Unlike scan --block-on, this path returns pending immediately instead of polling to completion, yet it copies the backend's interim block: false. That contradicts this file's fail-closed contract and README.md:75-78 (which says non-final verdicts leave block null), and lets a consumer that checks .blocking_verdict.block == false accept an evaluation whose license/dependency work is unfinished. Emit null until response.is_complete() and extend verdict_from_response_marks_an_unfinished_evaluation_pending to assert it.
| "block": response.block, | |
| "block": if response.is_complete() { | |
| Value::Bool(response.block) | |
| } else { | |
| Value::Null | |
| }, |
There was a problem hiding this comment.
I agree with this finding and think it should be addressed.
high: Pending verdict exposes a false-pass boolean
verdict_from_response copies response.block even when the evaluation is pending. This contradicts the fail-closed contract and README, which state that non-final verdicts have block:null. Emit null unless response.is_complete(); also correct the SKILL.md table that currently documents pending as false.
Proof or reproduction:
let verdict = verdict_from_response("criticals", &pending_response_with_block_false); assert!(verdict["block"].is_null()); // currently fails because block is false
| } | ||
| }; | ||
| let scans = match sha.as_deref() { | ||
| Some(sha) => retain_scans_at_sha(scans, sha), |
There was a problem hiding this comment.
P2 — Make the scan selected by the documented results[0] gate deterministic. An exact SHA can have multiple scans (pipeline retry/re-run), but this retains the server's unspecified order while skills/corgea/SKILL.md:436-440 gates only on .results[0]. If an older scan appears first, CI can consume its stale findings and pass even though a newer scan of the same commit blocks. After filtering out non-exact/dirty snapshots, sort matching scans by created_at descending (or reject ambiguity) and add a test with two scans at the same SHA in reverse chronological response order.
There was a problem hiding this comment.
I agree with this finding and think it should be addressed.
high: Documented results[0] gate can select a stale scan
retain_scans_at_sha preserves server order when multiple scans share a SHA, while the documented CI command consumes only results[0]. An older passing scan can therefore be selected ahead of a newer blocking scan. Sort matching scans by created_at descending or reject ambiguous results.
Proof or reproduction:
Input order: [{"id":"old","created_at":"2026-01-01","block":false},{"id":"new","created_at":"2026-02-01","block":true}]. Filtering preserves this order, so `.results[0].blocking_verdict.block == false` passes.
| /// | ||
| /// Falls back to rule ids against backends that do not send slugs yet. | ||
| pub fn triggered_slug_summary(issues: &[utils::api::BlockingIssue]) -> String { | ||
| pub fn triggered_slugs(issues: &[utils::api::BlockingIssue]) -> Vec<String> { |
There was a problem hiding this comment.
🧹 Quality - The code uses "Vec::contains" inside nested loops in "triggered_slugs", causing O(n²) complexity when de-duplicating identifiers. This approach slows down as input grows. View in Corgea ↗
More Details
🎟️Issue Explanation: The code uses "Vec::contains" inside nested loops in "triggered_slugs", causing O(n²) complexity when de-duplicating identifiers. This approach slows down as input grows.
- Using "Vec::contains" repeatedly for de-duplication adds costly linear scans for every identifier in "triggered_slugs".
- This O(n²) behavior limits scalability and increases latency when processing larger "issues" slices.
- The current method complicates maintenance by mixing de-duplication logic and order preservation across nested loops in "triggered_slugs".
🪄Fix Explanation: The code now uses a "HashSet" to track identifiers already emitted. This replaces repeated linear searches with average constant-time lookups, improving performance while preserving insertion order in the output vector.
- "seen.insert(identifier.clone())" performs efficient duplicate detection during iteration.
- The vector "names" remains responsible for preserving the original identifier order.
- Replacing "names.contains(&identifier)" avoids repeatedly scanning all previously collected values.
- Deduplication improves from quadratic time to average linear time as the issue and identifier count grows.
diff --git a/src/scanners/blast.rs b/src/scanners/blast.rs
index bf8a623..c18ee0e 100644
--- a/src/scanners/blast.rs
+++ b/src/scanners/blast.rs
@@ -659,13 +659,14 @@ pub fn normalize_block_on(block_on: Option<&str>) -> Result<Option<String>, Stri
/// Falls back to rule ids against backends that do not send slugs yet.
pub fn triggered_slugs(issues: &[utils::api::BlockingIssue]) -> Vec<String> {
let mut names: Vec<String> = Vec::new();
+ let mut seen = std::collections::HashSet::new();
for issue in issues {
let identifiers = match &issue.triggered_by_slugs {
Some(slugs) if !slugs.is_empty() => slugs.clone(),
_ => issue.triggered_by_rules.clone(),
};
for identifier in identifiers {
- if !names.contains(&identifier) {
+ if seen.insert(identifier.clone()) {
names.push(identifier);
}
}
To apply the fix, Download .patch.
There was a problem hiding this comment.
Automated review risk: 4/5.
CI gating can incorrectly reuse unsafe or stale scans and expose a pending verdict as a pass. These must be fixed before merge.
Critical or high-priority changes must be addressed.
Automatic approval was not submitted: automated review found critical or high-priority findings.
1. A verdict for a scan that already ran
Their pipeline skips a scan when the commit SHA has already been scanned, and today derives the pass/fail decision from the previous scan's vulnerability counts. With
--block-onthe decision moved to the CI rules, and a rule verdict lives only on a scan that is running — so the duplicate-skip path had nothing to gate on. They asked for ablocking_verdict/policy_resultfield on the scan object incorgea list --json.corgea list --block-on <slugs>now attaches one, using the same slugs and the same endpoint (check_blocking_rules) thatcorgea scan --block-ongates on. No backend change is needed: that endpoint already takes any scan id plusblock_on, and/scansalready accepts ashafilter.corgea ls --sha "$(git rev-parse HEAD)" --block-on criticals,malicious-deps --jsonstatusis what says whetherblockcan be trusted, and onlycompleteis final:statusblockcompletetrue/falsependingfalse, not yet finalunavailablereasonnullThe failure modes are deliberately loud rather than silently passing:
blockisnull(notfalse) when there is no verdict, a scan that has not completed is never evaluated (its findings are still arriving, so a "pass" would be meaningless), and an evaluation error — including an unknown, inactive, or PR-scoped slug — exits 1.--shanarrows the listing to one commit so the lookup is a single request. It requires the full SHA, because the server matches exactly and a prefix would answer "no scans", which a duplicate-skip check would read as "never scanned". The returned scans are re-checked against the SHA client-side, so a backend that ignores the filter cannot attribute another commit's verdict to this one. An empty page under--shanow reports "no scan of this commit" instead of the pre-existing "No Corgea project found" miss.A verdict costs one request per scan and the server re-evaluates every finding in that scan, so the pass covers at most 10 scans per page and
--block-onshrinks the default page to 10.Non-JSON output gets a Blocking column:
2.
--out-format/--out-filehonored under--block-onThey also asked whether the report file can survive a policy failure, so it can still be ingested into Harness STO. It could not: report generation ran after the
--fail/--block-ongates, whichexit(1)— so precisely the runs a pipeline cares about produced no report.The report and the SBOM now run before the gates. Rather than reordering in place, both bodies moved into
write_scan_report/write_sbom, which also collapses the four near-identical--out-formatbranches into one server-rendered path plus the JSON case. The e2e test asserts the ordering through the stub's ordered request plan, so a future reorder fails the test rather than silently dropping the file again.Testing
--block-ongate;list --block-onattaches verdicts and does not evaluate an unfinished scan;--shais sent and re-checked; a short SHA is rejected before dialing the API; the flags are rejected on issue listings; an evaluation error exits 1../harness cipasses locally in full (strict clippy, format, dep audit, 698 tests, coverage gate).Version bumped to 1.11.0: new backward-compatible flags, plus a guarantee added to existing ones.