Skip to content

Make launch stage the single preview source of truth in schema codegen#5611

Merged
janniklasrose merged 4 commits into
mainfrom
janniklasrose/5443-follow-up
Jun 16, 2026
Merged

Make launch stage the single preview source of truth in schema codegen#5611
janniklasrose merged 4 commits into
mainfrom
janniklasrose/5443-follow-up

Conversation

@janniklasrose

@janniklasrose janniklasrose commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Changes

Two follow-ups requested in the review of #5443:

  1. LaunchStage type + enforcement (comment). Add a LaunchStage type in internal/clijson (the cli.json contract package) with the known stages (GA, PUBLIC_PREVIEW, PUBLIC_BETA, PRIVATE_PREVIEW) as constants and a closed-set ParseLaunchStage. Schema generation validates every launch_stage read from .codegen/cli.json and fails with an actionable error on any unrecognized stage, so a stage introduced upstream can't silently render as GA. bundle/internal/annotation keeps only the presentation mapping (PreviewTag), keyed by clijson.LaunchStage.

  2. Drop the x-databricks-preview remnant (comment). x-databricks-preview existed only as a remnant of previous codegen. The published schema now carries x-databricks-launch-stage instead, and the Python (pydabs) codegen reads the launch stage exclusively (its LaunchStage class mirrors clijson.LaunchStage).

Behavior

Behavior-preserving refactor — the set of hidden/experimental fields is unchanged:

  • jsonschema.json / jsonschema_for_docs.json: each x-databricks-preview: PRIVATE entry becomes x-databricks-launch-stage: PRIVATE_PREVIEW; doNotSuggest is unchanged.
  • annotations_openapi.yml: the derived x-databricks-preview entries are dropped (x-databricks-launch-stage was already present).
  • Generated pydabs models under python/databricks/bundles/** are byte-identical.

Design notes

  • x-databricks-launch-stage is emitted into the published schema only for the private-preview stage — the only stage any consumer acts on (doNotSuggest and pydabs experimental). Other stages still surface as the description prefix and the per-value enumDescriptions labels, matching the original rationale in Expose stage/release status in schema #5443 for keeping the published schema lean.
  • The contract still types launch_stage as a plain string (the clijson wire types are generated by genkit). Fully guaranteeing that no value outside the closed set can be produced requires upstreaming the producer to emit the LaunchStage enum directly; until then clijson.ParseLaunchStage enforces the set where the contract is consumed.

Testing

  • go test ./internal/clijson/... ./bundle/internal/... ./bundle/schema/... ./libs/jsonschema/...
  • go test ./acceptance -run TestAccept/bundle/refschema
  • Python codegen unit tests (python/codegen/codegen_tests/)
  • Re-ran ./task generate-schema, generate-schema-docs, and pydabs-codegen; verified the only schema change is the x-databricks-previewx-databricks-launch-stage swap and that the generated pydabs models are unchanged.

Follow-up to #5443.

This pull request and its description were written by Isaac, an AI coding agent.

Two follow-ups from #5443:

Introduce an annotation.LaunchStage type with the known stages as named constants and a single source-of-truth tag map. The annotation parser now validates every launch_stage read from cli.json and fails codegen on an unrecognized stage, so a stage added upstream can't slip through unmapped.

Drop the x-databricks-preview remnant. The published schema now carries x-databricks-launch-stage, emitted only for private-preview fields (where it sets doNotSuggest and the Python codegen reads it to mark fields experimental). Generated pydabs models are byte-identical; only the schema key and value change.

Co-authored-by: Isaac
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: d4e7e90

Run: 27642929366

Env 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 7 15 264 994 6:11
💚​ aws windows 7 15 266 992 9:03
💚​ aws-ucws linux 7 15 360 908 6:51
💚​ aws-ucws windows 7 15 362 906 10:11
💚​ azure linux 1 17 267 992 6:02
💚​ azure windows 1 17 269 990 7:17
💚​ azure-ucws linux 1 17 365 904 7:46
💚​ azure-ucws windows 1 17 367 902 8:18
💚​ gcp linux 1 17 263 995 6:08
💚​ gcp windows 1 17 265 993 9:40
22 interesting tests: 15 SKIP, 7 RECOVERED
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/grants/select 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
Top 25 slowest tests (at least 2 minutes):
duration env testname
6:02 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:25 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:07 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:00 aws-ucws windows TestAccept
3:42 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:39 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:29 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:20 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:18 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:13 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:10 aws windows TestAccept
3:10 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:04 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:01 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:59 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:59 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:53 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:49 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:45 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:39 azure windows TestAccept
2:39 gcp windows TestAccept
2:34 azure-ucws windows TestAccept
2:34 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:30 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:25 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

Comment thread python/codegen/codegen/jsonschema.py
Comment thread bundle/internal/annotation/preview.go Outdated
…llow-up

# Conflicts:
#	bundle/schema/jsonschema.json
#	bundle/schema/jsonschema_for_docs.json
Per review of #5611: the launch stage is a contract concept, so its type, constants, and closed-set validation now live in internal/clijson (the cli.json contract package) instead of bundle/internal/annotation. The annotation package keeps only the presentation mapping (PreviewTag), now keyed by clijson.LaunchStage.

Fully guaranteeing that no other value can be produced by the contract requires the upstream producer to emit this enum for launch_stage; until then clijson.ParseLaunchStage enforces the closed set where the contract is consumed.

Co-authored-by: Isaac
Per review of #5611: the launch-stage holder in the Python (pydabs) codegen is renamed from Stage to LaunchStage to match clijson.LaunchStage in the Go code, and all four stages are mirrored as constants for completeness even though jsonschema.json only carries the private-preview stage today.

No change to the generated bundle models.

Co-authored-by: Isaac
@janniklasrose janniklasrose enabled auto-merge June 16, 2026 19:36
@janniklasrose janniklasrose added this pull request to the merge queue Jun 16, 2026
Merged via the queue into main with commit 30daa1c Jun 16, 2026
31 checks passed
@janniklasrose janniklasrose deleted the janniklasrose/5443-follow-up branch June 16, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants