Skip to content

Report a scan's CI blocking-rule verdict from corgea list, and write the report before the gate exits - #154

Closed
Ibrahimrahhal wants to merge 4 commits into
mainfrom
cursor/list-blocking-verdict-and-report-with-block-on-82d1
Closed

Report a scan's CI blocking-rule verdict from corgea list, and write the report before the gate exits#154
Ibrahimrahhal wants to merge 4 commits into
mainfrom
cursor/list-blocking-verdict-and-report-with-block-on-82d1

Conversation

@Ibrahimrahhal

@Ibrahimrahhal Ibrahimrahhal commented Aug 13, 2026

Copy link
Copy Markdown
Member

1. A verdict for a scan that already ran

Their pipeline skips a scan when the commit SHA has already been scanned, and today derives the pass/fail decision from the previous scan's vulnerability counts. With --block-on the decision moved to the CI rules, and a rule verdict lives only on a scan that is running — so the duplicate-skip path had nothing to gate on. They asked for a blocking_verdict/policy_result field on the scan object in corgea list --json.

corgea list --block-on <slugs> now attaches one, using the same slugs and the same endpoint (check_blocking_rules) that corgea scan --block-on gates on. No backend change is needed: that endpoint already takes any scan id plus block_on, and /scans already accepts a sha filter.

corgea ls --sha "$(git rev-parse HEAD)" --block-on criticals,malicious-deps --json
"blocking_verdict": {
  "block_on": ["criticals", "malicious-deps"],
  "status": "complete",
  "block": true,
  "blocked_issues": 3,
  "triggered_rules": ["criticals"]
}

status is what says whether block can be trusted, and only complete is final:

status Meaning block
complete Final verdict true/false
pending Server still resolving the scan's license dependencies; retry false, not yet final
unavailable No verdict — scan never completed, or past the per-page cap; see reason null

The failure modes are deliberately loud rather than silently passing: block is null (not false) when there is no verdict, a scan that has not completed is never evaluated (its findings are still arriving, so a "pass" would be meaningless), and an evaluation error — including an unknown, inactive, or PR-scoped slug — exits 1.

--sha narrows the listing to one commit so the lookup is a single request. It requires the full SHA, because the server matches exactly and a prefix would answer "no scans", which a duplicate-skip check would read as "never scanned". The returned scans are re-checked against the SHA client-side, so a backend that ignores the filter cannot attribute another commit's verdict to this one. An empty page under --sha now reports "no scan of this commit" instead of the pre-existing "No Corgea project found" miss.

A verdict costs one request per scan and the server re-evaluates every finding in that scan, so the pass covers at most 10 scans per page and --block-on shrinks the default page to 10.

Non-JSON output gets a Blocking column:

Scan ID        Project   Status     Repo          Branch   SHA        Blocking
scan-blocked   demo      complete   corgea/demo   main     01234567   BLOCKED: criticals
scan-clean     demo      complete   corgea/demo   main     aaaaaaaa   pass
scan-running   demo      scanning   corgea/demo   main     bbbbbbbb   N/A

2. --out-format/--out-file honored under --block-on

They also asked whether the report file can survive a policy failure, so it can still be ingested into Harness STO. It could not: report generation ran after the --fail/--block-on gates, which exit(1) — so precisely the runs a pipeline cares about produced no report.

The report and the SBOM now run before the gates. Rather than reordering in place, both bodies moved into write_scan_report/write_sbom, which also collapses the four near-identical --out-format branches into one server-rendered path plus the JSON case. The e2e test asserts the ordering through the stub's ordered request plan, so a future reorder fails the test rather than silently dropping the file again.

Testing

  • 11 unit tests for the verdict shape, SHA normalization, the SHA re-check, and the Blocking column.
  • e2e: the report is written before a tripped --block-on gate; list --block-on attaches verdicts and does not evaluate an unfinished scan; --sha is sent and re-checked; a short SHA is rejected before dialing the API; the flags are rejected on issue listings; an evaluation error exits 1.
  • ./harness ci passes locally in full (strict clippy, format, dep audit, 698 tests, coverage gate).

Version bumped to 1.11.0: new backward-compatible flags, plus a guarantee added to existing ones.

Open in Web Open in Cursor 

cursoragent and others added 4 commits August 13, 2026 09:06
--out-format/--out-file ran after the --fail and --block-on gates, so a
scan that violated a blocking rule exited 1 without ever writing the
report. A pipeline that gates on policy is exactly the one that needs the
report file, to ingest the findings it just failed on.

The report and the SBOM now run before the gates. Both bodies moved into
write_scan_report and write_sbom rather than being reordered in place,
which also collapses the four near-identical out-format branches into one
server-rendered path plus the JSON case.

Co-authored-by: ibrahim <ibrahim@corgea.com>
A pipeline that skips a duplicate scan for a commit it has already
scanned has no verdict to gate on: --block-on lives on the scan, so the
skip path fell back to counting vulnerabilities from the previous scan,
which is a different question from the one the CI rules answer.

corgea list --block-on <slugs> attaches a blocking_verdict to every
listed scan: the rules asked for, whether they blocked, how many issues
did, and which rules tripped. Only 'complete' is a final answer; a scan
still running, one that failed, or a server still resolving license
dependencies reports 'unavailable'/'pending' with block null rather than
a field a consumer would read as a pass. An evaluation error exits 1 for
the same reason.

--sha narrows the listing to one commit, which is how the duplicate-skip
path reaches its scan in a single request. It takes the full SHA because
the server matches exactly, and the returned scans are re-checked against
it so a backend that ignores the filter cannot answer with another
commit's verdict. A verdict costs one request per scan and the endpoint
re-evaluates every finding, so the pass is capped and the default page
shrinks to the ten scans it evaluates.

Co-authored-by: ibrahim <ibrahim@corgea.com>
corgea ls --block-on/--sha are new backward-compatible flags and the
report ordering only adds a guarantee, so SemVer puts this at a minor
bump. Cargo.toml is the single source of truth: PyPI reads it via maturin
and npm takes its version from the release tag.

Co-authored-by: ibrahim <ibrahim@corgea.com>
The scan listing exits 1 with "No Corgea project found" when an
unconfirmed project returns no scans, which is the right answer for a
listing of every scan. With --sha an empty page is the expected answer for
a commit that has not been scanned yet — the case a duplicate-skip path
handles by scanning — so it now reports the empty result and says which
commit had no scan.

Co-authored-by: ibrahim <ibrahim@corgea.com>
@Ibrahimrahhal
Ibrahimrahhal marked this pull request as ready for review August 13, 2026 09:17
Comment thread src/list.rs
.filter(|scan| {
scan.git_sha
.as_deref()
.is_some_and(|value| value.trim().eq_ignore_ascii_case(sha))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — Do not treat a dirty snapshot as the commit's reusable scan. This filter only compares git_sha. However, BLAST uploads retain the HEAD SHA when the archived worktree differs and explicitly send dirty=true (src/scanners/blast.rs:241-246, src/utils/api.rs:353-357); targeted/excluded scans are marked dirty for the same reason. A dirty or partial scan at H can therefore become .results[0] for a clean CI checkout at H, and the documented jq gate can accept that scan's block: false and skip scanning the code that will actually be deployed. Deserialize the scan-list worktree_dirty field and only reuse an exact clean snapshot (Some(false)) for --sha blocking verdicts; fail closed/force a fresh scan for true or legacy null. Add an end-to-end case where a dirty scan with the requested SHA is returned first.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this finding and think it should be addressed.

high: Dirty snapshots can be reused as clean commit scans

SHA filtering checks only git_sha, so a dirty or partial scan associated with the same HEAD is retained and can supply a passing verdict for different source content. For duplicate-scan gating, require an explicitly clean snapshot; treat dirty or unknown cleanliness as unavailable.

Proof or reproduction:

Given {"git_sha":"<HEAD>","worktree_dirty":true,"blocking_verdict":{"status":"complete","block":false}}, retain_scans_at_sha keeps the scan because it compares only git_sha, allowing the documented results[0] gate to pass.

Comment thread src/list.rs
},
"block": response.block,
"blocked_issues": response.blocked_count(),
"triggered_rules": triggered_slugs(&response.blocking_issues),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — triggered_rules is incomplete once blocking issues paginate. check_blocking_rules(..., None, ...) sends page=1, while BlockingRuleResponse exposes total_pages and explicitly documents that blocking_issues contains only the requested page (src/utils/api.rs:1226-1239, 1581-1607). Deriving the field from this one page omits any rule whose violations appear only on a later page, even though the new JSON contract and example present triggered_rules as the scan's rule set. Fetch pages 2..=total_pages and aggregate/deduplicate their slugs before emitting the verdict (or remove/rename the field so it does not claim a complete set), with a two-page test where page 2 contains a different rule.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this finding and think it should be addressed.

nitpick: triggered_rules omits rules from later pages

The blocking-rule request fetches one page, but triggered_rules is derived solely from that page. For paginated violations, the reported rule set is incomplete. Aggregate all response pages or rename/document the field as partial.

Proof or reproduction:

Page 1: total_pages=2, triggered_by_slugs=["criticals"]; page 2: triggered_by_slugs=["malicious-deps"]. The emitted triggered_rules is only ["criticals"].

Comment thread src/list.rs
} else {
VERDICT_STATUS_PENDING
},
"block": response.block,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — A pending evaluation currently exposes a false-pass boolean. Unlike scan --block-on, this path returns pending immediately instead of polling to completion, yet it copies the backend's interim block: false. That contradicts this file's fail-closed contract and README.md:75-78 (which says non-final verdicts leave block null), and lets a consumer that checks .blocking_verdict.block == false accept an evaluation whose license/dependency work is unfinished. Emit null until response.is_complete() and extend verdict_from_response_marks_an_unfinished_evaluation_pending to assert it.

Suggested change
"block": response.block,
"block": if response.is_complete() {
Value::Bool(response.block)
} else {
Value::Null
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this finding and think it should be addressed.

high: Pending verdict exposes a false-pass boolean

verdict_from_response copies response.block even when the evaluation is pending. This contradicts the fail-closed contract and README, which state that non-final verdicts have block:null. Emit null unless response.is_complete(); also correct the SKILL.md table that currently documents pending as false.

Proof or reproduction:

let verdict = verdict_from_response("criticals", &pending_response_with_block_false); assert!(verdict["block"].is_null()); // currently fails because block is false

Comment thread src/list.rs
}
};
let scans = match sha.as_deref() {
Some(sha) => retain_scans_at_sha(scans, sha),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — Make the scan selected by the documented results[0] gate deterministic. An exact SHA can have multiple scans (pipeline retry/re-run), but this retains the server's unspecified order while skills/corgea/SKILL.md:436-440 gates only on .results[0]. If an older scan appears first, CI can consume its stale findings and pass even though a newer scan of the same commit blocks. After filtering out non-exact/dirty snapshots, sort matching scans by created_at descending (or reject ambiguity) and add a test with two scans at the same SHA in reverse chronological response order.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this finding and think it should be addressed.

high: Documented results[0] gate can select a stale scan

retain_scans_at_sha preserves server order when multiple scans share a SHA, while the documented CI command consumes only results[0]. An older passing scan can therefore be selected ahead of a newer blocking scan. Sort matching scans by created_at descending or reject ambiguous results.

Proof or reproduction:

Input order: [{"id":"old","created_at":"2026-01-01","block":false},{"id":"new","created_at":"2026-02-01","block":true}]. Filtering preserves this order, so `.results[0].blocking_verdict.block == false` passes.

Comment thread src/scanners/blast.rs
///
/// Falls back to rule ids against backends that do not send slugs yet.
pub fn triggered_slug_summary(issues: &[utils::api::BlockingIssue]) -> String {
pub fn triggered_slugs(issues: &[utils::api::BlockingIssue]) -> Vec<String> {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Quality - The code uses "Vec::contains" inside nested loops in "triggered_slugs", causing O(n²) complexity when de-duplicating identifiers. This approach slows down as input grows. View in Corgea ↗

More Details
🎟️Issue Explanation: The code uses "Vec::contains" inside nested loops in "triggered_slugs", causing O(n²) complexity when de-duplicating identifiers. This approach slows down as input grows.

- Using "Vec::contains" repeatedly for de-duplication adds costly linear scans for every identifier in "triggered_slugs".
- This O(n²) behavior limits scalability and increases latency when processing larger "issues" slices.
- The current method complicates maintenance by mixing de-duplication logic and order preservation across nested loops in "triggered_slugs".

🪄Fix Explanation: The code now uses a "HashSet" to track identifiers already emitted. This replaces repeated linear searches with average constant-time lookups, improving performance while preserving insertion order in the output vector.
- "seen.insert(identifier.clone())" performs efficient duplicate detection during iteration.
- The vector "names" remains responsible for preserving the original identifier order.
- Replacing "names.contains(&identifier)" avoids repeatedly scanning all previously collected values.
- Deduplication improves from quadratic time to average linear time as the issue and identifier count grows.
diff --git a/src/scanners/blast.rs b/src/scanners/blast.rs
index bf8a623..c18ee0e 100644
--- a/src/scanners/blast.rs
+++ b/src/scanners/blast.rs
@@ -659,13 +659,14 @@ pub fn normalize_block_on(block_on: Option<&str>) -> Result<Option<String>, Stri
 /// Falls back to rule ids against backends that do not send slugs yet.
 pub fn triggered_slugs(issues: &[utils::api::BlockingIssue]) -> Vec<String> {
     let mut names: Vec<String> = Vec::new();
+    let mut seen = std::collections::HashSet::new();
     for issue in issues {
         let identifiers = match &issue.triggered_by_slugs {
             Some(slugs) if !slugs.is_empty() => slugs.clone(),
             _ => issue.triggered_by_rules.clone(),
         };
         for identifier in identifiers {
-            if !names.contains(&identifier) {
+            if seen.insert(identifier.clone()) {
                 names.push(identifier);
             }
         }

To apply the fix, Download .patch.

@corgea-security corgea-security left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review risk: 4/5.

CI gating can incorrectly reuse unsafe or stale scans and expose a pending verdict as a pass. These must be fixed before merge.

Critical or high-priority changes must be addressed.

Automatic approval was not submitted: automated review found critical or high-priority findings.

@corgea-security corgea-security added the dennis-reviewed Dennis completed an automated review label Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dennis-reviewed Dennis completed an automated review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants