Repository navigation
feat(ci): AI PR review via You.com Inference Router (PR-Agent) - #99
EdwardIrby wants to merge 9 commits into
Conversation
|
Droid encountered an error —— View job Droid is reviewing code and running a security check… |
|
Droid encountered an error —— View job Droid is reviewing code and running a security check… |
PR Reviewer Guide 🔍(Review updated until commit b30dc0c)Here are some key observations to aid the review process:
⚙️ Agent run details
✅ Resolved findings.github/workflows/ai-pr-review.yml:28Security Concern The production API key .github/workflows/ai-pr-review.yml:29Potential Bug The environment variable |
|
Droid encountered an error —— View job Droid is reviewing code and running a security check… |
|
Droid finished @EdwardIrby's task —— View job Droid review complete. The PR adds a single GitHub Actions workflow for AI-powered PR review via the You.com Inference Router. The most critical issue is an invalid JSON format in SKILLS.PATHS (single quotes instead of double quotes) that will prevent the review-guidelines skill from loading — a core feature of this PR. Additionally, the workflow lacks a fork-PR guard present in the sibling droid-review.yml, causing auth failures on external contributor PRs, and pins a secret-handling third-party action to a mutable tag rather than a commit SHA. |
|
Persistent review updated to latest commit c4b089e |
![factory-droid[bot]](https://avatars.githubusercontent.com/in/358017?s=60&v=4){kind=small}
| GITHUB_ACTION_CONFIG.AUTO_IMPROVE: "false" | ||
| GITHUB_ACTION_CONFIG.PR_ACTIONS: '["opened", "reopened", "ready_for_review", "synchronize"]' | ||
| SKILLS.ENABLED: "true" | ||
| SKILLS.PATHS: "['.agents/skills/review-guidelines/SKILL.md']" No newline at end of file |
There was a problem hiding this comment.
[P1] SKILLS.PATHS uses single-quoted string instead of valid JSON
The value "['.agents/skills/review-guidelines/SKILL.md']" uses Python-style single quotes inside the list literal. Dynaconf (used by PR-Agent) parses env vars whose value starts with [ via json.loads, which rejects single-quoted strings — json.loads("['x']") raises JSONDecodeError. When parsing fails, Dynaconf falls back to a raw string, so PR-Agent receives the literal text ['.agents/skills/review-guidelines/SKILL.md'] instead of a one-element list. The review-guidelines skill — a core feature of this PR — will never be loaded. Every other list env var in this file (CONFIG.FALLBACK_MODELS, GITHUB_ACTION_CONFIG.PR_ACTIONS) correctly uses double-quoted JSON; this one is inconsistent.
| SKILLS.PATHS: "['.agents/skills/review-guidelines/SKILL.md']" | |
| SKILLS.PATHS: '[".agents/skills/review-guidelines/SKILL.md"]' |
| group: ${{ github.workflow }}-${{ github.event.pull_request.number }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: |
There was a problem hiding this comment.
[P2] No fork-PR guard causes auth failure on external contributor PRs
The sibling droid-review.yml checks whether the PR author is an internal contributor before running review. This workflow has no equivalent guard. For pull_request events from forks, GitHub withholds repository secrets, so secrets.YDC_API_KEY resolves to an empty string. PR-Agent will then call https://api.you.com/v1 without a valid API key and fail with an auth error on every external contributor PR, wasting CI minutes and producing no review. Adding a permission-check step (as in droid-review.yml) or an if: guard checking github.event.pull_request.head.repo.full_name == github.repository would skip execution cleanly for external contributors.
| fetch-depth: 1 | ||
|
|
||
| - name: Run PR-Agent review via You.com Inference Router | ||
| uses: qodo-ai/pr-agent@v0.46.0 |
There was a problem hiding this comment.
[P2] [security] Third-party action pinned to mutable tag with secret access
The qodo-ai/pr-agent@v0.46.0 action is pinned to a mutable Git tag rather than an immutable commit SHA. This action receives both secrets.YDC_API_KEY (via the OPENAI_KEY env var) and secrets.GITHUB_TOKEN as environment variables, giving it full access to these credentials at runtime. If the v0.46.0 tag is moved to a malicious or compromised commit in the upstream qodo-ai/pr-agent repository — through account compromise, supply-chain attack, or a rogue maintainer — the action could exfiltrate the You.com API key and the GitHub token on every PR review run. Pinning to a commit SHA (e.g., qodo-ai/pr-agent@<full-sha>) prevents tag-swap attacks and is the recommended GitHub security hardening practice for actions that handle secrets.
|
Droid finished @EdwardIrby's task —— View job Droid review complete. The PR adds a single GitHub Actions workflow for AI-powered PR review via the You.com Inference Router. Three issues were already reported in prior review comments (invalid JSON in SKILLS.PATHS, missing fork-PR guard, and mutable tag pinning for a secret-handling third-party action). This validation confirms one additional security finding: because the workflow checks out the PR branch before loading the SKILL.md review guidelines, a PR author can modify those guidelines to inject malicious review instructions, enabling review evasion. |
|
Persistent review updated to latest commit 91aaef6 |
| SKILLS.ENABLED: "true" | ||
| SKILLS.PATHS: "['.agents/skills/review-guidelines/SKILL.md']" No newline at end of file |
There was a problem hiding this comment.
[P2] [security] Prompt injection via attacker-controlled SKILL.md loaded from PR branch
The workflow checks out the PR head ref (line 18) and then passes SKILLS.PATHS to PR-Agent (line 40), causing the AI reviewer to load its review guidelines from a file whose content is controlled by the PR author. Since the checkout includes the PR's own changes, a contributor can modify .agents/skills/review-guidelines/SKILL.md in their PR to inject malicious instructions into the AI review agent — for example, instructing it to always post a positive review, suppress security findings, or auto-describe the PR with a benign summary that conceals malicious changes. With AUTO_REVIEW: "true" and AUTO_DESCRIBE: "true", these injected instructions directly affect automated comments and PR description modifications that human reviewers rely on. The fix is to load review guidelines from the base branch (e.g., a separate checkout of github.base_ref for the guidelines file, or hardcode the guidelines inline as env config) rather than from the PR under review.
|
Droid finished @EdwardIrby's task —— View job Droid review complete. The PR adds a single GitHub Actions workflow for AI-powered PR review via the You.com Inference Router. Three issues were already reported in prior review comments (invalid JSON in SKILLS.PATHS, missing fork-PR guard, and mutable tag pinning for the secret-handling third-party action). This review identifies one additional security finding: the pull_request trigger uses the PR's workflow YAML for same-repo PRs, allowing internal contributors to modify the workflow and exfiltrate secrets before merge review. |
|
Persistent review updated to latest commit b032236 |
PR Code Suggestions ✨Latest suggestions up to b628069 Explore these optional code suggestions:
⚙️ Agent run details
Previous suggestionsSuggestions up to commit b032236
|
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, ready_for_review, reopened, synchronize] |
There was a problem hiding this comment.
[P2] [security] pull_request trigger allows workflow self-modification with secret access
For same-repository PRs, GitHub executes the workflow file from the PR's merge ref, not from the base branch. An internal contributor — or a compromised account with branch-creation rights — can modify this workflow YAML in their PR to exfiltrate secrets.YDC_API_KEY and secrets.GITHUB_TOKEN, and the modified workflow will run with full secret access before any maintainer reviews or merges the change. The sibling droid-review.yml mitigates this class of risk with an explicit collaborator-permission check (gh api repos/.../collaborators/.../permission) that gates secret access before checkout; this workflow has no equivalent guard. Pinning the third-party action to a commit SHA (already recommended) does not address this, because the attacker controls the workflow YAML itself, not the action version. Consider adding an if guard that checks the PR author's collaborator permission before the step that handles secrets, or restricting workflow file modifications to protected branches only.
…tch external review - workflow_dispatch path uses pr-agent CLI (action runner has no dispatch handler) - GITHUB__USER_TOKEN double-underscore for dynaconf SECTION__KEY mapping - fix concurrency group to use inputs.pr_number on dispatch - gate internal step on is_internal, dedupe config, update skip message - drop unneeded id-token/actions permissions
|
Droid encountered an error —— View job Code review subagent complete. Waiting for security-reviewer subagent to finish before merging findings and writing |
github-actions
Bot
added
Review effort 2/5
and removed
Review effort 1/5
Possible security concern
labels
|
Persistent review updated to latest commit 99d1529 |
PR Code Suggestions ✨No code suggestions found for the PR. ⚙️ Agent run details
|
|
Droid finished @EdwardIrby's task —— View job Droid review complete — PR #99 validation Droid review complete. The PR adds a well-structured AI PR review workflow that mirrors the sibling droid-review.yml pattern with appropriate permission scoping, concurrency control, and immutable SHA pinning for the third-party action. Most issues identified in prior review rounds (SKILLS.PATHS quoting, fork-PR guard, mutable tag pinning, SKILL.md injection, secret exfiltration via pull_request trigger) have been addressed or are already documented in existing PR comments. One remaining concern: the external review path runs commands sequentially with bash errexit, so a failure in the secondary |
|
Persistent review updated to latest commit b628069 |
| - name: Auto Review (external) | ||
| if: github.event_name == 'workflow_dispatch' | ||
| run: | | ||
| PR_URL="https://github.com/${{ github.repository }}/pull/$PR_NUMBER" |
There was a problem hiding this comment.
[P2] External review: describe failure blocks review due to bash errexit
The three pr-agent CLI commands (describe, review, improve) run sequentially in a single run block. GitHub Actions invokes Linux shells with bash -eo pipefail by default, so if pr-agent describe exits non-zero (e.g., a transient You.com API timeout or model error), the script terminates immediately and pr-agent review — the core functionality of this workflow — never executes. The internal path avoids this because the PR-Agent Action handles all three tools internally and can continue past individual failures. Consider running review first, separating the commands into distinct steps, or tolerating non-critical failures with || true on describe and improve.
…n Auto Review - synchronize events are handled before GITHUB_ACTION_CONFIG.PR_ACTIONS in pr-agent's runner and skipped without push trigger config; enable handle_push_trigger + push_commands so pushes to open PRs are re-reviewed (matches the Droid Auto Review behavior this workflow mirrors) - list router model IDs in additional_reasoning_effort_models, otherwise pr-agent drops reasoning_effort for models LiteLLM metadata doesn't know - drop issues: write (PR-Agent only posts to PRs)
Verification report (against qodo-ai/pr-agent @ v0.46.0 source)Checked every config claim in this PR against the actual pr-agent source, not just the docs. ✅ Confirmed correct
🐛 Two bugs found and fixed (cbb Pushed commit)
Also dropped
|
|
Droid encountered an error —— View job Droid is reviewing code and running a security check… |
|
Droid encountered an error —— View job Droid is reviewing code and running a security check… |
|
Persistent review updated to latest commit ca31ed4 |
ca31ed4 to
b628069
Compare
|
Droid finished @EdwardIrby's task —— View job Droid review complete. The workflow is well-structured with a proper internal/external contributor split and immutable SHA pinning. One real P1 bug remains: |
|
Persistent review updated to latest commit b30dc0c |
1 similar comment
|
Persistent review updated to latest commit b30dc0c |
| CONFIG.ADDITIONAL_REASONING_EFFORT_MODELS: '["openai/deepseek-v4-flash", "openai/qwen3.8-flash"]' | ||
| PR_REVIEWER.NUM_MAX_FINDINGS: "5" | ||
| SKILLS.ENABLED: "true" | ||
| SKILLS.PATHS: "['.agents/skills/review-guidelines/SKILL.md']" |
There was a problem hiding this comment.
[P1] SKILLS.PATHS uses single-quoted string instead of valid JSON
The value "['.agents/skills/review-guidelines/SKILL.md']" uses Python-style single quotes inside the list literal. Dynaconf (PR-Agent's config loader) parses env vars whose value starts with [ via json.loads, which rejects single-quoted strings, so Dynaconf falls back to treating the value as a raw string. PR-Agent then receives the literal text ['.agents/skills/review-guidelines/SKILL.md'] instead of a one-element list, and the review-guidelines skill — a core feature of this PR — will never be loaded. Every other list env var in this file (CONFIG.FALLBACK_MODELS, GITHUB_ACTION_CONFIG.PR_ACTIONS, CONFIG.ADDITIONAL_REASONING_EFFORT_MODELS, GITHUB_ACTION_CONFIG.PUSH_COMMANDS) correctly uses double-quoted JSON; this one is inconsistent. The same bug exists at line 116 (SKILLS__PATHS) on the external CLI path. This was previously reported on the old file ai-pr-review.yml before it was renamed to review.yml, but the single-quote value was never fixed.
| SKILLS.PATHS: "['.agents/skills/review-guidelines/SKILL.md']" | |
| SKILLS.PATHS: '[".agents/skills/review-guidelines/SKILL.md"]' |
1 participant
User description
🤖 AI PR reviews via the You.com Inference Router
Automated AI reviews on pull requests, running on our own You.com Inference Router instead of a third-party API (this mirrors
.github/workflows/droid-review.yml, but reviews are grounded in our repo's own review guidelines at.agents/skills/review-guidelines/SKILL.mdrather than Factory's directory).The Inference Router speaks the OpenAI Chat Completions format, so Qodo Merge (PR-Agent) works with it directly — we only change the base URL, API key, and model name.
What this PR adds
One workflow file:
.github/workflows/ai-pr-review.yml. It runs on every new PR and every push to an open PR, posting a review and a summary as PR comments:/reviewvia PR-Agent'sauto_review— security, tests, effort estimate/describeviaauto_describe— PR summary comment[skills]support (skills.enabled+skills.pathspointing directly at.agents/skills/review-guidelines/SKILL.md)Configuration
https://api.you.com/v1secrets.YDC_API_KEY(production key from the API Keys page)openai/deepseek-v4-flash.agents/skills/review-guidelines/SKILL.mdWhy this is simple
YDC_API_KEYrepo secret is needed.GET https://api.you.com/v1/modelslists the model IDs our account can use.SECTION.KEYenv names; lists/booleans parse correctly) and the PR-Agent[skills]loader accepts a directSKILL.mdfile path.Things to watch
openai/deepseek-v4-flashto force the OpenAI provider onto our base URL. If the first run errors, check whether the rawdeepseek-v4-flashID is needed instead.pull_request_targetpattern later, similar to Droid'sworkflow_dispatchfallback).Open questions
deepseek-v4-flashfor speed/cost;deepseek-v4-proorqwen3.8-maxfor deeper reviews?Refs: PR-Agent GitHub Action docs · Configuration reference
PR Type
Enhancement
Description
Add Auto Review workflow using You.com Inference Router
Support internal and external contributor review paths
Configure PR-Agent with high reasoning, fallback models, and review guidelines
Enable push-trigger re-reviews and run details reporting
Diagram Walkthrough
flowchart LR A["PR Event"] --> B{"Internal Contributor?"} B -- "Yes" --> C["PR-Agent Action<br/>(auto_review, describe, improve)"] B -- "No" --> D["workflow_dispatch<br/>(manual trigger)"] D --> E["PR-Agent CLI<br/>(describe, review, improve)"] C --> F["Review Comments"] E --> FFile Walkthrough
review.yml
Add Auto Review workflow with internal/external paths.github/workflows/review.yml
dispatch
⚙️ Agent run details