Protobuf definitions and generated gRPC bindings for the Message-Service interface. This repository is the single source of truth for the wire contract; both the service (Python) and consuming pipelines (Python today, potentially Rust and other languages later) pull their bindings from here.
message-service-proto/
|- proto/
| `- message_service/
| `- v1/
| `- message_service.proto # canonical contract
|- python/
| `- message_service_proto/ # generated Python + gRPC stubs
| `- v1/
|- scripts/
| `- generate-python.sh # regeneration script
|- pyproject.toml # Python distribution
`- README.md
- The gRPC package name carries the major version (
message_service.v1). Breaking changes bump the package suffix (v1->v2);v1andv2can coexist in the repo and in generated bindings during migration. - Within a major version, only additive changes are permitted: new optional fields, new enum values (with
_UNSPECIFIED = 0preserved), new RPC methods. Field numbers are permanent; deprecated fields are markedreserved. - The package version in
pyproject.tomlfollows semver: minor bumps for additive contract changes, patch bumps for generated-code regeneration with no source change.
In the consumer's pyproject.toml:
[tool.poetry.dependencies]
message-service-proto = { git = "ssh://git@<host>/message-service-proto.git", rev = "v0.1.0" }Pin to a tag or commit SHA for reproducible builds. Air-gapped sites should mirror the repo to a local Git server and reference the mirror.
Then in consumer code:
import grpc
from message_service_proto.v1 import message_service_pb2, message_service_pb2_grpc
async with grpc.aio.insecure_channel("message-service:50051") as channel:
stub = message_service_pb2_grpc.MessageServiceStub(channel)
response = await stub.BeginRun(
message_service_pb2.BeginRunRequest(
pipeline_type="etl-nightly",
run_tags=["production"],
# ...
)
)If you're using the synchronous client API, create a grpc.Channel instead and call stub.BeginRun(...) without await.
For developers working on both the service and the contract simultaneously, a local path dependency avoids the push-pull cycle:
[tool.poetry.dependencies]
message-service-proto = { path = "../message-service-proto", develop = true }develop = true installs in editable mode - changes to the generated stubs show up immediately without reinstall. Do not commit local-path references; switch back to the git form before merging.
A typical workflow:
# Start dev iteration
cd message-service-proto
vim proto/message_service/v1/message_service.proto
./scripts/generate-python.sh
cd ../Message-Service
# Temporarily swap to local path for iteration
poetry remove message-service-proto
poetry add --editable ../message-service-proto
# ... iterate ...
# Ready to merge: regenerate, tag proto repo, swap back
cd ../message-service-proto
git tag v0.2.0 && git push --tags
cd ../Message-Service
poetry remove message-service-proto
poetry add "message-service-proto @ git+ssh://git@<host>/message-service-proto.git@v0.2.0"- Edit
proto/message_service/v1/message_service.proto. - Verify the change is additive (or accept that you need to cut a
v2package). - Regenerate Python bindings:
./scripts/generate-python.sh. - Commit generated files alongside the proto change - generated stubs are committed, not gitignored. This lets consumers install directly from git without running code generation.
- Bump the version in
pyproject.toml. - Tag the commit:
git tag v0.X.0 && git push --tags.
Consumers installing via git dependency do not run the codegen script. Committing generated files makes the repo directly installable. The alternative - a build hook that regenerates on install - adds complexity and a hard dependency on grpcio-tools at install time.
Use the repo Makefile for Bash/Make-based local validation, or the PowerShell scripts in scripts/ for Windows-native development. The full contributor workflow, prerequisites, and command mapping live in docs/development.md.
Generated protobuf files are intentionally excluded from Ruff; they are validated by regeneration, import checks, and the proto-focused Buf workflow instead. CI also regenerates the Python stubs and fails if the checked-in generated files drift from the .proto source.
Use docs/releasing.md for the tagged release checklist, including pre-tag verification, artifact checks, changelog updates, and GitHub release steps. The checked-in release history lives in CHANGELOG.md.
- Rust: planned. The proto files are language-agnostic; a
rust/directory with aCargo.tomlandbuild.rsinvokingtonic-buildwould produce tonic bindings from the same sources. Tracked on the service-side ROADMAP. - Other languages: add a sibling top-level directory (
go/,typescript/, etc.) with its own build config. Theproto/directory is the shared source of truth.
- Requirements:
Message-Service/docs/L1-REQ.md, specifically theL1-API-*andL1-RUN-*categories. - Error-code enumeration: proto file comments cross-reference each
ErrorCodevalue to its originating L2 requirement.