You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Integrates @sverka/checks with the Definition Graph so checks run as host shell steps from the repo root with safely quoted commands and preserved outputs. Previously checks resolved to OperationSpec; now they resolve to a StepDefinition with a single shell operation string.
Built-in resolver produces steps with id checks/<checkId>, runtime { mode: "host", workingDir: ctx.root }, and one shell operation whose command joins individually quoted args.
Resolution table covers Node/Python/Rust/Go, honors proposal reason in polyglot repos, and Node entries validate root scripts.
synthesizeCheckSteps(checks, ctx, resolver) returns ResolvedCheck[], skips unresolved checks, deduplicates by resolved.checkId, and preserves outputs for findings.
Public API in @sverka/checks exports synthesizeCheckSteps; Spec 14 updated; extractFindings unchanged (SARIF).
Migration
Replace operation.command/args with step.operations[0].command (already quoted and joined).
Update any step ID assumptions to checks/<checkId>.
Expect checks to run from the repo root via runtime.workingDir.
Written for commit 110ccbb. Summary will update on new commits.
CodeAnt-AI Description
Integrate project checks as executable Definition Graph steps
What Changed
Built-in checks now resolve into host-run steps with commands for Node, Python, Rust, and Go projects
Added a public way to turn proposed checks into graph steps
Unavailable checks are skipped, and duplicate check IDs run only once
Preserved SARIF findings extraction while updating the checks specification and coverage
Impact
✅ Checks can run in the Definition Graph ✅ Fewer duplicate check runs ✅ Unsupported checks no longer create invalid steps
💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
@codeant-ai ask: Your question here
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
@codeant-ai ask: Can you suggest a safer alternative to storing this secret?
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
@codeant-ai: Your feedback here
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
@codeant-ai: Do not flag unused imports.
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
@codeant-ai: review
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.
Learnt from: CR
Repo: sverka-dev/sverka PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-08-12T07:24:02.495Z
Learning: Applies to **/src/index.ts : - **Public API:** Everything public is exported from `src/index.ts`.
Applied to files:
packages/checks/src/index.ts
packages/checks/src/__tests__/public-api.test.ts
🪛 LanguageToolspecs/14-checks/spec.md
[style] ~119-~119: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ... lint for Node/npm → StepDefinition. 4. Resolver returns null for unknown checkId. 5. Re...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~120-~120: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...er returns null for unknown checkId. 5. Resolver validates package.json scripts (Node en...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
🔇 Additional comments (7)
packages/checks/src/synthesize.ts (1)
23-27: Use one check ID for deduplication.
The lookup uses check.checkId, but insertion uses resolved.checkId. A custom resolver can return a different ID and allow duplicate proposed checks through synthesis.
No change needed. Non-SARIF formats are explicitly deferred in v0, and extractFindings skips them by design. The specification and package use the same format union.
> Likely an incorrect or invalid review comment.
📝 Walkthrough
Summary by CodeRabbit
New Features
Added a public API for converting proposed checks into executable steps.
Checks now support host-runtime shell execution with consistent step details.
Unresolved checks are skipped and duplicate checks are consolidated.
Expanded the checks specification to document resolution, step synthesis, SARIF findings, outputs, and error handling.
Tests
Added comprehensive coverage for check synthesis, resolution, deduplication, runtime behavior, and public exports.
Walkthrough
The checks package now resolves checks into StepDefinition values, synthesizes deduplicated steps, exports synthesizeCheckSteps, updates resolver tests, and defines the Checks Integration specification.
Changes
Checks step synthesis
Layer / File(s)
Summary
Resolver StepDefinition model specs/14-checks/spec.md, packages/checks/src/resolver.ts
The resolver now returns ResolvedCheck.step with host-runtime shell operations and empty input, output, and dependency lists. The specification documents the related interfaces and behavior.
Step synthesis and export packages/checks/src/synthesize.ts, packages/checks/src/index.ts
synthesizeCheckSteps resolves proposed checks, skips unresolved checks, retains the first result for each check ID, and is exported publicly.
Resolver and synthesis validation packages/checks/src/__tests__/*
Tests now validate resolved steps, package-manager precedence, custom resolvers, synthesis behavior, and the public runtime export.
Check synthesis can create duplicate check steps when a resolver changes a check’s ID, which may cause the same check to run more than once. The PR is otherwise mergeable with owner awareness and follow-up to use one consistent deduplication key.
Running ultrareview automatically — This refactor changes the checks package's public contract, swapping OperationSpec for StepDefinition and switching commands to single strings, which could break planner and engine integrations; a signature change of this scope warrants deeper inspection.. I'll post findings when complete.
I can't run this ultrareview because you've reached your trial's review limit. Trial plans have lower review limits than paid plans. Upgrade now to resume reviews.
To help optimise your usage, you can tune cubic to get the most out of your usage limits:
All review discussions are resolved and the diff addresses their concerns, but CI did not run for this non-trivial host-shell/Definition Graph integration. Human review is needed before merging without automated verification.
The reason will be displayed to describe this comment to others. Learn more.
Review Summary
This PR successfully adapts @sverka/checks to work with the new Definition Graph and Run Plan architecture. The changes are well-implemented with strong test coverage (42 checks tests, 178 total across affected packages).
Key Changes:
Resolver now produces StepDefinition (new graph model) instead of OperationSpec
New synthesizeCheckSteps converts ProposedCheck[] → StepDefinition[] with proper deduplication
Check steps use checks/<checkId> ID pattern and run in host mode
extractFindings reused unchanged for SARIF normalization
Test Coverage:
✅ All 7 new synthesize tests passing
✅ 35 existing resolver tests passing
✅ 178 tests across 4 affected packages green
✅ TypeScript/lint/build clean, no any types
The implementation correctly handles resolution failures, deduplicates by checkId, and maintains backward compatibility. No defects found that would block merge.
You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.
• Emit check executions as StepDefinition shell steps for the new Definition Graph model.
• Add synthesizeCheckSteps to convert ProposedCheck lists into deduped, resolved check steps.
• Expand Spec 14 docs and update/add unit tests for the new public API.
The following are alternative approaches to this PR:
1. Keep structured command/args in step operations
➕ Avoids shell-string quoting/escaping ambiguities across platforms
➕ Preserves intent for engines that can exec without a shell
➕ Easier to extend with env/cwd later without parsing
➖ Requires StepDefinition operation schema support (or a new op kind)
➖ More plumbing work across engine/executor if only shell-string is supported today
2. Return resolution diagnostics instead of silently skipping unresolved checks
➕ Makes missing toolchains/mappings visible to callers and users
➕ Enables better UX (warnings/errors) while still supporting partial graphs
➖ API surface grows (needs error channel and policy decisions)
➖ May be noisy in polyglot repos where some checks are intentionally inapplicable
Recommendation: The PR’s approach is appropriate for Wave J: represent checks as host-mode shell StepDefinitions and keep extraction unchanged. If the runtime/engine later needs safer process execution, consider evolving the operation model to carry argv separately (or adding an exec-style operation) and optionally returning resolution diagnostics to avoid silent omission of intended checks.
Files changed (7) +275 / -96
Enhancement (2) +33 / -1
index.tsExport synthesizeCheckSteps from @sverka/checks public API+2/-1
Export synthesizeCheckSteps from @sverka/checks public API
• Adds synthesizeCheckSteps to the package exports and annotates the public API header comment with the Spec 14 reference.
synthesize.tsAdd synthesizeCheckSteps to build deduped check steps for the graph+31/-0
Add synthesizeCheckSteps to build deduped check steps for the graph
• Introduces a helper that resolves proposed checks into StepDefinitions, skipping unresolved checks and deduplicating by checkId to produce a stable StepDefinition[] for Definition Graph inclusion.
resolver.tsEmit StepDefinition shell steps instead of OperationSpec+21/-41
Emit StepDefinition shell steps instead of OperationSpec
• Updates the resolver interfaces and implementation to return ResolvedCheck.step: StepDefinition. Built-in mappings now produce host-mode steps with id checks/<checkId> and a single shell operation command assembled from command+args.
resolver.test.tsUpdate resolver tests for StepDefinition-based resolution+27/-41
Update resolver tests for StepDefinition-based resolution
• Rewrites assertions to validate StepDefinition fields (step id, host runtime, shell operation command string) rather than OperationSpec command/args. Updates the custom resolver test fixture to return a StepDefinition with a shell operation.
spec.mdActivate and document Spec 14 checks integration details+109/-12
Activate and document Spec 14 checks integration details
• Replaces the stub with an active spec describing goals, non-goals, interfaces/exports, data model conventions (step id pattern, host runtime, shell op), error handling, and an explicit test plan aligned with the implementation.
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
The PR successfully migrates the checks package to the new Definition Graph but introduces a regression in the data lifecycle. Specifically, synthesizeCheckSteps discards the outputs metadata required for the extractFindings API. This oversight prevents the engine from locating and parsing SARIF results after execution, effectively breaking the integration's end-to-end functionality.
While Codacy analysis indicates the PR is 'up to standards', the increased complexity in synthesize.ts and the use of fragile shell command construction in the resolver are significant concerns. The deduplication logic also contains an inconsistency that could lead to redundant processing. These issues should be addressed to ensure the reliability and performance of the check execution pipeline.
About this PR
The integration between synthesis and findings extraction is currently broken. While the PR fulfills the requirement to export extractFindings, the synthesis process discards the data (outputs) that extractFindings needs to operate. A broader architectural adjustment is needed to preserve this mapping throughout the graph execution.
1 comment outside of the diffpackages/checks/src/resolver.ts
line 83 🟡 MEDIUM RISK
Suggestion: Reading package.json on every resolve call is inefficient. Try running the following prompt in your coding agent: > Refactor createBuiltinResolver in packages/checks/src/resolver.ts to cache the result of readRootPackageJson based on the ctx.root path to avoid redundant disk I/O.
Test suggestions
Found recommended test scenario: synthesizeCheckSteps converts multiple valid proposed checks into StepDefinitions
Found recommended test scenario: synthesizeCheckSteps skips checks that the resolver cannot map to a command
Found recommended test scenario: synthesizeCheckSteps deduplicates multiple checks that share the same checkId
Found recommended test scenario: CheckResolver produces correct ID with 'checks/' prefix
Found recommended test scenario: CheckResolver sets the runtime mode to 'host' for generated steps
Found recommended test scenario: CheckResolver correctly flattens command and args into a single shell command string
Found recommended test scenario: Public API exports all required functions and types according to Spec 14
synthesizeCheckSteps() checks duplicates using the input check.checkId but records
resolved.checkId, so a resolver that normalizes/aliases IDs can still emit duplicate steps (two
different input ids resolve to the same canonical id) or skip valid checks. Dedup should be applied
consistently using the resolved identity (ideally resolved.step.id) after resolution succeeds.
ⓘ Recommendations generated based on similar findings in past PRs
Evidence
The implementation checks seen using the input checkId but then stores the resolved checkId. Since
ResolvedCheck exposes checkId as an arbitrary string, a resolver can legally return a different id
than the input, making the dedup logic inconsistent.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`synthesizeCheckSteps()` uses two different keys for deduplication:
- pre-resolution: `seen.has(check.checkId)`
- post-resolution: `seen.add(resolved.checkId)`
If a custom resolver aliases/normalizes ids (e.g., multiple checkIds mapping to one canonical checkId), this can yield duplicate steps or incorrect skipping.
## Issue Context
The public API supports custom resolvers, and the resolver contract allows `ResolvedCheck.checkId` to be any string.
## Fix Focus Areas
- Apply dedup *after* resolution, using a single key consistently.
- Prefer dedup by `resolved.step.id` (graph-visible identity) or document/enforce that `resolved.checkId` must equal `check.checkId`.
- Add a regression test with a custom resolver where two different input `checkId`s resolve to the same canonical id, and assert only one step is returned.
## Fix Focus Areas
- packages/checks/src/synthesize.ts[22-28]
- packages/checks/src/__tests__/synthesize.test.ts[1-80]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
2. Duplicate check step IDs 🐞 Bug≡ Correctness
Description
createBuiltinResolver() sets StepDefinition.id to checks/<checkId>, so resolving multiple
ProposedChecks with the same checkId (but different reasons/ecosystems) yields steps with identical
ids but potentially different commands. Any integration that uses resolver.resolve directly (without
synthesizeCheckSteps or equivalent dedup) can construct a graph/run plan with duplicate step ids,
making step addressing ambiguous and risking the wrong command being executed or a step being
dropped.
ⓘ Recommendations generated based on similar findings in past PRs
Evidence
The built-in resolver hardcodes step ids to checks/<checkId>, while the planner can emit multiple
ProposedChecks with the same checkId across different ecosystem reasons (e.g., multiple test
drivers), meaning multiple resolved steps can share the same id unless deduplicated externally.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`createBuiltinResolver()` currently produces `StepDefinition.id` as `checks/${check.checkId}`. The planner can emit multiple ProposedChecks sharing the same `checkId` (e.g., `test`) across different ecosystems/reasons, so resolving those individually can create multiple StepDefinitions with the same id but different `shell.command`.
## Issue Context
- The planner’s drivers include multiple entries with the same `checkId` (notably `test`) across Node/Rust/Go, and it deduplicates by `(checkId, reason)`.
- `synthesizeCheckSteps()` happens to deduplicate by `checkId`, but `createBuiltinResolver()` is public and can be used directly by consumers.
## Fix Focus Areas
- Ensure `StepDefinition.id` is unique per ProposedCheck (e.g., incorporate `check.id` while keeping the `checks/` prefix).
- Update/extend tests to reflect the new id format and/or add a regression test that resolving Node-test and Rust-test yields different step ids.
### Suggested direction
Change:
- `id: \`checks/${check.checkId}\``
To something like:
- `id: \`checks/${check.checkId}/${check.id}\`` (or another stable suffix based on `check.id`).
## Fix Focus Areas
- packages/checks/src/resolver.ts[101-110]
- packages/checks/src/__tests__/resolver.test.ts[11-132]
- packages/checks/src/__tests__/synthesize.test.ts[1-80]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
The modified specification document is stored at specs/14-checks/spec.md, but the file name
spec.md does not begin with a numeric identifier + separator as required. This breaks the
repository convention for discoverable, consistently numbered specs.
ⓘ Recommendations generated based on similar findings in past PRs
Evidence
PR Compliance ID 2663931 requires spec documents introduced or modified to be placed under specs/
and have filenames starting with a numeric identifier. This PR modifies specs/14-checks/spec.md,
whose filename spec.md does not include the required numeric prefix.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
A specification document was modified, but its filename (`spec.md`) does not follow the required numbering convention (numeric identifier + separator + name).
## Issue Context
Compliance requires all spec documents introduced or modified to live under `specs/` and have filenames beginning with a numeric identifier (e.g., `014-checks-integration.md`). The current path `specs/14-checks/spec.md` violates the filename requirement.
## Fix Focus Areas
- specs/14-checks/spec.md[1-1]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Review mode: ⚖️ Balanced: This is a behavior-changing integration across resolver, synthesis, public API, and graph-facing StepDefinitions; it has meaningful contract and execution-semantics risk, but not enough independent logic density to justify redundant extended review.
Tip of the day
💡 Did you know, you can type 'qodo, fix this' on a finding and the fix lands right on your PR
Adapted @sverka/checks to work with the new Definition Graph + Run Plan:
- Resolver now produces StepDefinition (with shell operations) instead of
the old OperationSpec
- New synthesizeCheckSteps: converts ProposedChecks → StepDefinition[]
for inclusion in a Definition Graph
- extractFindings reused unchanged (SARIF normalization)
- Check steps use ID pattern checks/<checkId>, runtime mode host
- Deduplicates by checkId, skips unresolved checks
42 checks tests pass (7 new synthesize + 35 existing). No any types.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- Set runtime.workingDir to ctx.root in resolved check steps.
- Shell-quote each argument before joining the command string.
- Return ResolvedCheck[] from synthesizeCheckSteps and preserve resolver outputs.
- Use resolved.checkId for both deduplication and generated step id.
- Update synthesize/resolver tests and spec 14.
Co-Authored-By: Petr Plenkov <petr.plenkov@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
codeant-ai
Bot
added
the
size:L
![amazon-q-developer[bot]](https://avatars.githubusercontent.com/in/1220912?s=60&v=4){kind=small}
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
baz-reviewer
Bot
added
baz: ready to merge
baz: needs review
and removed
baz: needs review
baz: ready to merge
labels
User description
Summary
@sverka/checksto work with the new Definition Graph + Run PlanStepDefinition(with shell operations) instead of oldOperationSpecsynthesizeCheckSteps: converts ProposedChecks → StepDefinition[] for inclusion in a Definition GraphextractFindingsreused unchanged (SARIF normalization)checks/<checkId>, runtime mode host, dedup by checkIdTest plan
anytypesGenerated with Devin
Summary by cubic
Integrates
@sverka/checkswith the Definition Graph so checks run as host shell steps from the repo root with safely quoted commands and preserved outputs. Previously checks resolved toOperationSpec; now they resolve to aStepDefinitionwith a single shell operation string.checks/<checkId>, runtime{ mode: "host", workingDir: ctx.root }, and one shell operation whose command joins individually quoted args.synthesizeCheckSteps(checks, ctx, resolver)returnsResolvedCheck[], skips unresolved checks, deduplicates byresolved.checkId, and preservesoutputsfor findings.@sverka/checksexportssynthesizeCheckSteps; Spec 14 updated;extractFindingsunchanged (SARIF).Migration
operation.command/argswithstep.operations[0].command(already quoted and joined).checks/<checkId>.runtime.workingDir.Written for commit 110ccbb. Summary will update on new commits.
CodeAnt-AI Description
Integrate project checks as executable Definition Graph steps
What Changed
Impact
✅ Checks can run in the Definition Graph✅ Fewer duplicate check runs✅ Unsupported checks no longer create invalid steps💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.