Skip to content

Add Devin CLI vendor adapter and verify local Desktop integration #90

Open

Description

@letya999

Request

Add a first-class devin vendor adapter for Probity so Devin CLI can enforce command/file policies and the existing TDD rule through its local lifecycle hooks.

Target environment

  • Windows 11, PowerShell
  • Devin CLI 3000.11.1 (cc4e349ca55e)
  • Devin Desktop installed product version 3.10.48 (bundled runtime 3000.10.48 / fcf7ba39 verified headlessly; GUI not exercised)
  • Probity 1.10.1

Devin exposes PreToolUse and PostToolUse, but Probity's vendor registry has no Devin entry. Devin tools include exec, write, edit, and apply_patch; selecting the Claude/Codex vendor is not a reliable translation. TDD additionally needs the session's actual tool outputs, including failed assertions and background-command results.

Proposed approach

  1. Capture representative local hook payloads and verify blocking on the installed CLI.
  2. Add a vendor adapter, raw-event classifier and session-history reader using the existing engine and rules.
  3. Prefer a native transcript when supported; otherwise discuss a session-isolated lifecycle event journal rather than assuming Claude-compatible history.
  4. Support an explicitly configured AI validator first (Config.ai, including Codex); do not claim Devin-native SDK/auth reuse before it is verified. Avoid recursive validator hook execution.
  5. Add test-driven coverage for real payloads, malformed known actions, multiple-file writes, cross-session isolation, and red/green/setup-error TDD outcomes; document CLI/Desktop verification separately.

I'm preparing a focused implementation in a fork and will link the PR and actual validation results here. @nizos, feedback on the history source and initial validator approach would be appreciated.

Implementation and verification

Adapter PR: #91. Separate dependency advisory fixes: #92. @nizos is mentioned in both PRs.

Native CLI and Desktop-bundled runtime hooks block forbidden commands/writes, allow clean controls, block production edits before a behavioral red, and allow the same edit after a real failing assertion. The test then passes. Fresh global CLI installation also rejects an import/setup failure as TDD evidence. AI rules use an explicitly configured Codex validator; no Devin-native SDK/auth support is claimed.

Fork Linux CI passes: 565 tests passed, 26 skipped, with lint, format and typecheck green: https://github.com/letya999/probity/actions/runs/38053606602 . The 15 new adapter tests pass on Windows; 31 existing full-suite Windows failures were reproduced with identical labels against the upstream base. Desktop GUI and cloud handoff remain unverified.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions