Skip to content

test: assert what a provider refuses instead of skipping - #138

Closed
HarshMN2345 wants to merge 1 commit into
mainfrom
test/harden-adapter-contract
Closed

test: assert what a provider refuses instead of skipping#138
HarshMN2345 wants to merge 1 commit into
mainfrom
test/harden-adapter-contract

Conversation

@HarshMN2345

Copy link
Copy Markdown
Member

Follow-up to #126. That PR moved every non provider specific test into Base and declared what differs. This one closes the gap it left: a declared capability dropped the test that needed it, so $supportsTags = false meant nobody ever checked what the adapter does when asked for a tag.

Now a capability narrows a test instead of removing it. Where the adapter documents a refusal, the shared test asserts it; where the provider answers differently, the test asserts that answer.

Refusals are asserted, not skipped

assertRefused() requires the message the Git defaults already use, X() is not supported by <name>, so a 404 from a repository that never existed cannot pass for a refusal. Eight capabilities assert their refusal: repository languages, commit statuses and their lookup, pull request creation and lookup, tags, check runs, namespaces, and installation repositories.

GitHub's three stubs said Not implemented; Gogs's seven said Pull request API is not supported by Gogs. Both now name the method, which is what makes the assertion meaningful.

Skips that were hiding an answer

test was now
testListBranchesEmptyRepository skipped on Gogs asserts [main] — the adapter asks for an initial commit
testListTagsCommitlessRepository skipped on Gogs asserts [], which holds either way
testGetCommitAuthorAvatar skipped on GitLab asserts ''
assertCommitAuthorLinks asserted nothing when a flag was false asserts '', so a provider that starts reporting one flips the flag
testGetOwnerNameWith{out,Zero,Null}RepositoryId three tests, skipped on GitHub and Bitbucket one test, runs everywhere
testGenerateCloneCommandWithTag skipped on GitHub runs everywhere; the command is built, never run, so the tag need not exist
testGetUserWithInvalidUsername skipped on Bitbucket runs everywhere
testGetRepositoryAfterDeleteFails skipped on GitLab runs everywhere
languages on Bitbucket assertIsArray on a method typed : array asserts []

Adapter changes

Sharing a test meant running it against a provider for the first time, which is where these came from.

  • Gitea reports synchronized for a pushed head. Every consumer acts on synchronize — Appwrite's four VCS event handlers each test for it — so a pull request update on Gitea, Forgejo and Gogs was read as an action nobody handles. The adapter normalizes it; every other action passes through as sent.
  • GitLab raised a bare Exception for a repository that does not exist, which is why GitLabTest had to declare $repositoryNotFoundException = \Exception::class. getRepositoryName() and getOwnerName() now raise RepositoryNotFound on a 404, and the flag is gone.
  • GitLab answered the old path of a deleted project through the redirect it issues for a moved-aside project, so getRepository() reported a repository that had been deleted. It no longer follows that redirect, which also drops $deletesRepositoriesSynchronously.
  • GitHub::getUser() sent no credential, the only call in the adapter that did not, and returned the whole response rather than the body — a 404 came back looking like a user.
  • getInstallationRepository() was stubbed in three adapters with the wording each chose. Only GitHub models installations, so the refusal moved to Git beside the other four.

Tests hoisted out of the adapter classes

Two tests were provider specific only because the payload going in was.

  • Pull request action mapping, previously written out in both GitLabTest and BitbucketTest, is now a declared $pullRequestActions map of native to shared name. Base asserts every provider covers the three actions consumers act on and that nothing falls outside the vocabulary. Bitbucket names the action in the event rather than the payload, which one hook expresses.
  • The event describes the head commit, not the first one listed, previously a GitLab test. pushPayload() takes the commits to list before the head, and Base asserts the head's hash, message, author and url.

GitLabTest now holds no test of its own. BitbucketTest keeps the three that are genuinely Bitbucket: batched pushes, a linked commit author, and its repository shape.

Also

Following the monorepo's test conventions: final on the leaf test classes, #[\Override] on real overrides, and thirteen cleanup blocks moved onto discardRepositories(), which retries and reports a repository it could not remove instead of leaving it to contaminate later runs. testGetDeletedRepositoryFails looked up a repository that was never deleted; it is now testGetNonExistingRepositoryFails, beside the one that does delete.

The adapter guide still told contributors to add tests to tests/VCS/Adapter/VCSTest.php, which has never existed. It now describes the hooks, the flags, and what belongs in an adapter's own class.

Verification

composer lint and composer check (PHPStan level 8) pass. Base holds the same 102 tests; what changed is how many of them assert rather than skip. The suites are the real check, so CI is the verification.

🤖 Generated with Claude Code

A declared capability dropped the test that needed it. Where the adapter
documents a refusal, the shared test now asserts it, and where the provider
answers differently the test asserts that answer.

Four behaviours the sharing surfaced:

- Gitea says 'synchronized' for a pushed head. Consumers only act on
  'synchronize', so a pull request update was read as an unknown action.
- GitLab raised a bare Exception for a repository that does not exist, and
  answered the old path of a deleted one through a redirect.
- GitHub asked for a user with no credential, the only call in the adapter
  that did.
- getInstallationRepository() was stubbed three times over; the refusal
  belongs on Git, next to the other four.

GitLab's test class keeps no test of its own, and Bitbucket only what only
Bitbucket does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

Thanks for contributing! This repository is a read-only mirror; development for this library happens in packages/vcs in the utopia-php monorepo. Please open this pull request there instead.

@github-actions github-actions Bot closed this Sep 7, 2026
@HarshMN2345
HarshMN2345 deleted the test/harden-adapter-contract branch September 7, 2026 07:05
@greptile-apps

greptile-apps Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR strengthens the shared adapter suite by asserting unsupported capabilities and provider-specific responses instead of broadly skipping tests. It also normalizes Gitea-family pull-request update actions, corrects GitHub user lookup response handling, standardizes unsupported-operation messages, and makes GitLab repository lookups reject redirects.

  • Centralizes capability refusal assertions and pull-request action mapping tests in Base.
  • Adds provider declarations and richer webhook fixtures for shared behavior.
  • Introduces retrying repository cleanup and updates adapter contributor documentation.
  • One test-quality issue remains: refusal assertions do not verify the exact method or provider named by the exception.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking weakness in how the new refusal tests validate exception messages.

The runtime adapter changes are consistent with existing contracts and consumers; the only accepted concern is that the new test helper can produce false confidence by accepting refusal messages for the wrong method or provider.

Files Needing Attention: tests/VCS/Base.php

Important Files Changed

Filename Overview
tests/VCS/Base.php Expands shared coverage and cleanup behavior, but assertRefused() does not enforce the exact refusal message it documents.
src/VCS/Adapter/Git/GitHub.php Authenticates user lookup, returns the decoded user body, handles HTTP failures, and standardizes unsupported-operation messages.
src/VCS/Adapter/Git/GitLab.php Prevents redirected repository lookups and consistently distinguishes repository-not-found responses.
src/VCS/Adapter/Git/Gitea.php Normalizes the provider-native synchronized pull-request action to the shared synchronize vocabulary.
src/VCS/Adapter/Git.php Supplies the shared unsupported implementation for installation repository lookup.
tests/VCS/Adapter/BitbucketTest.php Declares Bitbucket-specific capabilities and moves its pull-request action behavior into shared test hooks.

Fix all with Greploop Fix All in Claude Code Fix All in Codex

Prompt To Fix All With AI
### Issue 1
tests/VCS/Base.php:376
**Refusal Check Is Too Broad**

`assertRefused()` is intended to verify the exact refusal message, but this pattern accepts any method name, any provider name, and arbitrary trailing text. For example, `getPullRequest() is not supported by github` could satisfy a `createPullRequest()` check. This can let the shared capability tests pass without enforcing the contract introduced here. Pass the expected method into the helper and compare the complete message, including `$this->vcsAdapter->getName()`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "test: assert what a provider refuses ins..." | Re-trigger Greptile

Comment thread tests/VCS/Base.php
protected function assertRefused(callable $call): void
{
$this->expectException(Exception::class);
$this->expectExceptionMessageMatches('/^\w+\(\) is not supported by /');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Refusal Check Is Too Broad

assertRefused() is intended to verify the exact refusal message, but this pattern accepts any method name, any provider name, and arbitrary trailing text. For example, getPullRequest() is not supported by github could satisfy a createPullRequest() check. This can let the shared capability tests pass without enforcing the contract introduced here. Pass the expected method into the helper and compare the complete message, including $this->vcsAdapter->getName().

Prompt To Fix With AI
This is a comment left during a code review.
Path: tests/VCS/Base.php
Line: 376

Comment:
**Refusal Check Is Too Broad**

`assertRefused()` is intended to verify the exact refusal message, but this pattern accepts any method name, any provider name, and arbitrary trailing text. For example, `getPullRequest() is not supported by github` could satisfy a `createPullRequest()` check. This can let the shared capability tests pass without enforcing the contract introduced here. Pass the expected method into the helper and compare the complete message, including `$this->vcsAdapter->getName()`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant