Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Message Service Proto

Protobuf definitions and generated gRPC bindings for the Message-Service interface. This repository is the single source of truth for the wire contract; both the service (Python) and consuming pipelines (Python today, potentially Rust and other languages later) pull their bindings from here.

Repository layout

message-service-proto/
|- proto/
|  `- message_service/
|     `- v1/
|        `- message_service.proto    # canonical contract
|- python/
|  `- message_service_proto/         # generated Python + gRPC stubs
|     `- v1/
|- scripts/
|  `- generate-python.sh             # regeneration script
|- pyproject.toml                    # Python distribution
`- README.md

Versioning policy

  • The gRPC package name carries the major version (message_service.v1). Breaking changes bump the package suffix (v1 -> v2); v1 and v2 can coexist in the repo and in generated bindings during migration.
  • Within a major version, only additive changes are permitted: new optional fields, new enum values (with _UNSPECIFIED = 0 preserved), new RPC methods. Field numbers are permanent; deprecated fields are marked reserved.
  • The package version in pyproject.toml follows semver: minor bumps for additive contract changes, patch bumps for generated-code regeneration with no source change.

Consumer usage - Python

Option A: git dependency (production pattern)

In the consumer's pyproject.toml:

[tool.poetry.dependencies]
message-service-proto = { git = "ssh://git@<host>/message-service-proto.git", rev = "v0.1.0" }

Pin to a tag or commit SHA for reproducible builds. Air-gapped sites should mirror the repo to a local Git server and reference the mirror.

Then in consumer code:

import grpc

from message_service_proto.v1 import message_service_pb2, message_service_pb2_grpc

async with grpc.aio.insecure_channel("message-service:50051") as channel:
    stub = message_service_pb2_grpc.MessageServiceStub(channel)
    response = await stub.BeginRun(
        message_service_pb2.BeginRunRequest(
            pipeline_type="etl-nightly",
            run_tags=["production"],
            # ...
        )
    )

If you're using the synchronous client API, create a grpc.Channel instead and call stub.BeginRun(...) without await.

Option B: local path dependency (development pattern)

For developers working on both the service and the contract simultaneously, a local path dependency avoids the push-pull cycle:

[tool.poetry.dependencies]
message-service-proto = { path = "../message-service-proto", develop = true }

develop = true installs in editable mode - changes to the generated stubs show up immediately without reinstall. Do not commit local-path references; switch back to the git form before merging.

Switching between A and B

A typical workflow:

# Start dev iteration
cd message-service-proto
vim proto/message_service/v1/message_service.proto
./scripts/generate-python.sh
cd ../Message-Service
# Temporarily swap to local path for iteration
poetry remove message-service-proto
poetry add --editable ../message-service-proto
# ... iterate ...

# Ready to merge: regenerate, tag proto repo, swap back
cd ../message-service-proto
git tag v0.2.0 && git push --tags
cd ../Message-Service
poetry remove message-service-proto
poetry add "message-service-proto @ git+ssh://git@<host>/message-service-proto.git@v0.2.0"

Contributor instructions

Making a contract change

  1. Edit proto/message_service/v1/message_service.proto.
  2. Verify the change is additive (or accept that you need to cut a v2 package).
  3. Regenerate Python bindings: ./scripts/generate-python.sh.
  4. Commit generated files alongside the proto change - generated stubs are committed, not gitignored. This lets consumers install directly from git without running code generation.
  5. Bump the version in pyproject.toml.
  6. Tag the commit: git tag v0.X.0 && git push --tags.

Why commit generated files?

Consumers installing via git dependency do not run the codegen script. Committing generated files makes the repo directly installable. The alternative - a build hook that regenerates on install - adds complexity and a hard dependency on grpcio-tools at install time.

Development workflow

Use the repo Makefile for Bash/Make-based local validation, or the PowerShell scripts in scripts/ for Windows-native development. The full contributor workflow, prerequisites, and command mapping live in docs/development.md.

Generated protobuf files are intentionally excluded from Ruff; they are validated by regeneration, import checks, and the proto-focused Buf workflow instead. CI also regenerates the Python stubs and fails if the checked-in generated files drift from the .proto source.

Releasing

Use docs/releasing.md for the tagged release checklist, including pre-tag verification, artifact checks, changelog updates, and GitHub release steps. The checked-in release history lives in CHANGELOG.md.

Future languages

  • Rust: planned. The proto files are language-agnostic; a rust/ directory with a Cargo.toml and build.rs invoking tonic-build would produce tonic bindings from the same sources. Tracked on the service-side ROADMAP.
  • Other languages: add a sibling top-level directory (go/, typescript/, etc.) with its own build config. The proto/ directory is the shared source of truth.

See also

  • Requirements: Message-Service/docs/L1-REQ.md, specifically the L1-API-* and L1-RUN-* categories.
  • Error-code enumeration: proto file comments cross-reference each ErrorCode value to its originating L2 requirement.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages