Skip to content

feat(release): automate Python SDK releases through GitHub Actions and Git tags - #2688

Merged
nan-yu merged 19 commits into
mainfrom
release-automation
Sep 22, 2026
Merged

nan-yu merged 19 commits into
mainfrom
release-automation

Conversation

@nan-yu

@nan-yu nan-yu commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Replaces the manual release.sh workflow with an automated, tag-driven release pipeline for a2ui-core and a2ui-agent-sdk via GitHub Actions and the Google OSS Exit Gate.

Resolves #1818 (credential management blocker) and #1928 (tag-driven versioning).


Key Changes

  1. Tag-driven dynamic versioning (hatch-vcs):

    • Package versions are derived directly from Git tags formatted as python/<pypi-name>/v<version> (e.g. python/a2ui-core/v0.1.1, python/a2ui-agent-sdk/v0.6.0).
    • Replaced hardcoded versions in version.py files with lightweight shims that query importlib.metadata.version().
    • Updated python_ci.yml with a tag reachability check so local and CI builds exercise the dynamic versioning path.
  2. GitHub Actions release workflows:

    • .github/workflows/release-pypi.yml: Maintainer-dispatched workflow supporting package selection (a2ui-core, a2ui-agent-sdk, or both), bump levels (patch, minor, major), and a dry_run mode. Handles changelog cutting, packaging, artifact verification, WIF authentication to Google Cloud, and staging in the Exit Gate Artifact Registry.
    • .github/workflows/release-verify-pypi.yml: Automated hourly backstop that checks PyPI for asynchronous publish completion and updates GitHub Release notes with live package links.
  3. Release automation scripts (.github/scripts/):

    • release_version.py: Semantic version arithmetic, preflight validation against PyPI, changelog cutting, and cross-package dependency constraint checks.
    • release_artifacts.py: Inspects built .whl and .tar.gz metadata against planned release versions and generates selective Exit Gate manifests.
    • confirm_pypi.py: GitHub Release creation, PyPI polling, and backstop reconciliation.
    • Comprehensive test suite in .github/scripts/tests/ (76 unit tests).
  4. Documentation:

    • Updated docs/contributing/release.md with maintainer instructions, parameter descriptions, and troubleshooting steps.
    • Removed deprecated agent_sdks/python/release.sh.

Verification

  • Script unit tests: All 76 tests pass (python3 -m unittest discover -s .github/scripts/tests).
  • Python CI suite: All 810 SDK tests pass (uv run pytest), format checks pass (pyink), and mypy type checks pass.
  • Workflow dry-run: Verified end-to-end execution on fork (nan-yu/A2UI, run 35268792791), confirming version calculation, changelog generation, wheel build, artifact verification, twine check, and WIF authentication.
  • Exit Gate approval: Internal builder configuration approved and submitted in cl/983322172.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request migrates the Python SDK release process to a VCS-based versioning workflow using hatch-vcs and GitHub Actions, replacing manual versioning and local scripts with automated preflight checks, artifact verification, and PyPI publication confirmation. The review feedback highlights three key areas for improvement: optimizing discover_pending in confirm_pypi.py to avoid sequential network requests in a loop, filtering out non-conforming tags in release_version.py to prevent potential ValueError crashes, and catching tarfile.TarError in release_artifacts.py to robustly handle corrupted sdist files.

Comment thread .github/scripts/confirm_pypi.py
Comment thread .github/scripts/release_version.py
Comment thread .github/scripts/release_artifacts.py
Replace the hand-edited `__version__` strings in a2ui_core and
a2ui_agent with hatch-vcs, which derives each package's version from its
own tag series (`python-a2ui-core-v*` and `python-a2ui-agent-sdk-v*`).
This removes the manual version bump step from the release process and
makes the tag the single source of truth.

`version.py` stays as a public shim in both packages, now reading the
version back from the installed distribution metadata, so
`a2ui.core.version.__version__` and `a2ui.version.__version__` are
unchanged for callers.

`fallback-version` is set to the last released version rather than 0.0.0
so that a checkout without tags (CI at the default fetch-depth of 1)
still satisfies the `a2ui-core>=0.1.1,<0.2.0` constraint that
a2ui-agent-sdk declares.
… SDKs

Adds .github/scripts/release_version.py, which reads each package's
version from its own tag series, computes the next version for a bump
level, cuts the changelog and runs the preflight checks that the release
workflow depends on.

The checks stop three mistakes that are painful to undo once a version
reaches PyPI: releasing with an empty Unreleased changelog section,
reusing a version that already has a tag, and bumping a2ui-core outside
the range that a2ui-agent-sdk pins it to.

Standard library only, so it runs before any dependency sync. The
existing `python3 -m unittest discover -s .github/scripts/tests` step in
python_ci.yml picks up the tests with no workflow change.
…fest

Adds .github/scripts/release_artifacts.py. `verify` reads the name and
version back out of every wheel and sdist and compares them to the
release plan, which catches a build that missed the release tag and
silently used the fallback version. `manifest` writes a selective
publishing manifest naming only the packages built in this run, rather
than `publish_all: true`, which would also publish stale artifacts left
in the Artifact Registry by an earlier failed run.
Replaces the manual release.sh with a dispatched workflow that derives
the version from git tags, builds, stages in the OSS Exit Gate Artifact
Registry and triggers publishing.

The Exit Gate authorizes this repository through Workload Identity
Federation, and the binding is an exact string match on the OIDC
`workflow_ref` claim, which embeds the ref the run was triggered from.
A tag-triggered workflow therefore cannot authenticate: it presents
`@refs/tags/...` against an allowlist entry that can only name
`@refs/heads/main`, and future tags cannot be enumerated in advance. So
the release is dispatched from main and creates its own tag rather than
being started by one. The tag is pushed only after the upload succeeds,
so a failed run does not burn a version.

Also adds a read-only workflow for checking Exit Gate access without
publishing, and an hourly backstop that links releases to PyPI when
publishing takes longer than the release run waits.

Publishing uses a selective manifest naming only the packages built in
this run, rather than publish_all, which would also publish stale
artifacts left in the registry by an earlier failed run.

Requires a matching builder entry in the internal Exit Gate project
config before it can run.
A '*/CHANGELOG.md' pathspec could also pick up an unrelated changelog
from the Dart or web packages if one happened to be dirty.
The `Restrict Default Branch` ruleset requires a pull request and lists no
bypass actors, so `git push --follow-tags origin HEAD:main` would have been
refused. It would have been refused after the artifacts were already staged in
the Artifact Registry, leaving orphans behind to clean up by hand.

Only tags are pushed now, which no ruleset covers, and the changelog edit is
raised as an ordinary pull request by a new `changelog` job. That job is
separate from `release` so that a failure in it cannot stop `confirm` from
linking the GitHub releases once PyPI has the artifacts.

Tags point at the commit that was the tip of `main` at checkout rather than at
the changelog commit. The repository requires linear history, so a tag created
on a branch commit would be left unreachable once the pull request is squashed,
and the `git describe --match` check in python_ci.yml would start failing.

Also from the review:

  * The dry-run cleanup runs on `always()` and skips versions that were never
    staged. A partial upload is exactly when the cleanup matters, and the
    default `success()` condition would have skipped it.
  * A new step fails the run if `main` advanced while the tests and the build
    were running, before anything leaves the runner.
  * `update_release` no longer passes `--latest`. With two packages the badge
    went to whichever one PyPI happened to publish last.
  * `discover_pending` excludes drafts and looks further back, so a pending
    Python release cannot fall off the end behind releases for other languages.
  * The PyPI duplicate check retries, and the plan step reads its inputs from
    the environment.
A `both` release pushed one tag at a time. If the second push failed, the
repository was left with a2ui-core tagged and a2ui-agent-sdk not, at a point
where the artifacts are already staged and recovery means working out by hand
which half landed. `git push --atomic` makes it all or nothing.

Also records why the concurrency group is not keyed by the `package` input:
doing so would put `both` and `a2ui-core` in different groups and let them run
at the same time, which is the collision the group exists to prevent.
`notes` raised straight out of `read_unreleased` when a changelog had no
`## Unreleased` heading, printing a traceback and exiting 1.

Callers read empty stdout as "nothing to release". A malformed changelog is a
different answer and has to be distinguishable from it, so this reports on
stderr in the same `error: ...` shape the `check` subcommand uses, and leaves
stdout empty.

Tests cover all three outcomes: entries present, section empty, heading
missing.
Turns a one-line request such as "release a2ui-core patch" into the release
sequence, so a maintainer does not have to carry the checklist in their head.

The skill does not release anything itself. Every mutating action is a
`gh workflow run release-pypi.yml` dispatch, and the work happens on the
runner. Local calls are limited to the read-only `notes`, `plan` and `check`
subcommands, which answer two questions before a CI run is spent: is there
anything to release, and will preflight reject it.

What it adds over dispatching the workflow by hand:

  * Refuses to start while the previous release's changelog pull request is
    still open. Until that merges the entries remain under `## Unreleased` and
    the next release would repeat them in its notes. The workflow cannot see
    this.
  * Derives the package set and bump level from the changelogs rather than the
    wording of the request, and puts the resulting versions up for approval.
  * Flags that an a2ui-core minor or major bump needs the a2ui-agent-sdk pin
    widened first, which is a separate reviewed change and cannot be done
    during a release.
  * Explains that the run stays active for several minutes after the release
    job goes green, because the confirm job is polling PyPI, and that a
    confirm timeout is not a failed release.

Publishing is irreversible, so the dry run is mandatory and there is an
explicit confirmation gate, quoting full version numbers, before any run is
dispatched with dry_run disabled.
A pull request opened by the workflow could never be merged. GitHub does not
start workflow runs for events caused by the built-in GITHUB_TOKEN, so none of
the nine contexts the `Restrict Default Branch` ruleset requires would ever
report, and the ruleset allows no bypass actors.

That would have deadlocked releases rather than merely annoying someone: until
the changelog lands the entries stay under `## Unreleased`, so every later
release would repeat them in its notes, and the release skill refuses to start
while a changelog change is outstanding.

The repository solved this once before with a personal access token (#2396),
but that secret no longer exists. Sources also disagree on whether such a pull
request gets no runs at all or runs in an approval-required state, so the fix
deliberately does not depend on which it is.

The job now pushes the branch and puts a compare link in the run summary. A
maintainer opens the pull request from there, and because a person opened it,
CI runs normally. Nothing else changes: cutting the changelog and committing
are still automated, and the job still cannot push to `main`.

Also drops `pull-requests: write`, which the job no longer needs, and switches
the preflight guard in the release skill from looking for an open pull request
to looking for the branch. The pull request may not exist yet, whereas the
branch always does, and the repository deletes branches on merge.
Comment thread .github/workflows/release-pypi.yml Outdated
Comment thread .github/workflows/release-pypi.yml Outdated
The release is internally consistent whether or not main advances while it
builds: the artifacts come from the checked-out commit, the tags point at
that commit, and the notes come from that commit's changelog. A later merge
does not invalidate any of it.

The guarded window covered dependency installation, the full test suite, the
build, and the metadata checks, so the check aborted runs that had nothing
wrong with them. Main moves often enough that this would have been a routine
annoyance rather than a rare safety net.
When a live release run fails during staging (for example, if package: both
uploads a2ui-core but fails on a2ui-agent-sdk), the previous condition
only ran on inputs.dry_run, leaving stranded artifacts in the Artifact Registry.
A subsequent retry would then fail with duplicate version errors.

Update the cleanup step condition to also trigger on failure(), rename the
step to drop the (dry run) suffix, pass DRY_RUN via env, and produce an
appropriate step summary message distinguishing dry runs from failed live runs.
In GitHub Actions, id-token: write is scoped at the job level, exposing the
OIDC token minting endpoint to all steps within the job. Previously, the
single release job ran uv sync --all-packages, uv run pytest, and uv build
with id-token: write and contents: write.

Split the workflow into an unprivileged build job (contents: read) that builds
and tests distributions, and a privileged release job (id-token: write,
contents: write) that downloads pre-built artifacts, stages them to Artifact
Registry, pushes git tags, and triggers the Exit Gate.
@nan-yu
nan-yu requested a review from gspencergoog September 21, 2026 23:34
@nan-yu
nan-yu merged commit c08702a into main Sep 22, 2026
36 checks passed
@nan-yu
nan-yu deleted the release-automation branch September 22, 2026 18:20
@github-project-automation github-project-automation Bot moved this from Todo to Done in A2UI Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automate release for python and npm more.

2 participants