fix(serve): restore JumpStart default instance type from model spec#6045
Open
e-davidson wants to merge 2 commits into
Open
fix(serve): restore JumpStart default instance type from model spec#6045e-davidson wants to merge 2 commits into
e-davidson wants to merge 2 commits into
Conversation
In v2, ModelBuilder selected the model spec's default instance type when the user did not specify compute. In v3 this silently regressed: __post_init__ calls _initialize_compute_config() before self.region is assigned, raising AttributeError inside the spec-resolution chain that a bare 'except Exception: pass' swallows, so every no-compute build falls back to ml.m5.large instead of the spec default. Affects public hub deploys too. Fix: - Extract _initialize_region() and call it first in __post_init__ so region is available before compute config initialization. - Replace the bare except with a warning log so future failures in the spec-resolution chain are visible instead of silent. Tests (mirrors master-v2's approach, unit-only with mocking, no E2E): - Fixture is a real JumpStartModelSpecs built from master-v2 constants.py BASE_SPEC; JumpStartModelsAccessor.get_model_specs is patched to return it. The spec default is set to ml.g5.12xlarge and ml.m5.large excluded from supported types so the tests discriminate the buggy fallback from the correct spec default. - 10 unit tests, red-green verified on current master: 8 fail on baseline (ml.m5.large != ml.g5.12xlarge), 10/10 pass with the fix. Private hub unit suite (17 tests) unaffected.
e-davidson
had a problem deploying
to
manual-approval
July 17, 2026 21:13 — with 
GitHub Actions
Error
GitHub Actions
Error
e-davidson
had a problem deploying
to
manual-approval
July 17, 2026 21:13 — with
GitHub Actions
Error
GitHub Actions
Error
Add TestUserOverrideFlowsDownstream (4 tests) closing the coverage gap between the override-wins assertions and the downstream-forwarding assertions, which previously only used the spec default: - deploy() hands the user's instance type (not the spec default) to _deploy - _deploy_core_endpoint places the user's instance type into the ProductionVariant for CreateEndpointConfig - _build_for_jumpstart forwards the user's instance type to get_init_kwargs, which drives instance-type-specific image URI and env var selection - instance_type via the Compute object behaves identically to a direct instance_type: honored verbatim, spec lookup skipped These are behavior pins, not regression discriminators: they pass on both baseline and fix (override was never broken; the bug only affected the no-compute default path). Suite is now 14 tests: baseline 8 failed 6 passed, with fix 14/14.
e-davidson
requested a deployment
to
manual-approval
July 17, 2026 21:31 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
e-davidson
requested a deployment
to
manual-approval
July 17, 2026 21:31 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
In the v2 SDK,
ModelBuilderselected the model spec's default instance type when the user did not specify compute. In v3 this silently regressed: every no-compute JumpStart build falls back toml.m5.largeregardless of what the model spec recommends. This affects public hub deploys as well.Root cause
__post_init__calls_initialize_compute_config()beforeself.regionis assigned. The spec-resolution chain inside it raisesAttributeError, which a bareexcept Exception: passswallows, so the builder silently takes the hardcodedml.m5.largefallback instead of the spec default.Fix
_initialize_region()and call it first in__post_init__, so region is available before compute config initialization.except Exception: passwith a warning log, so future failures in the spec-resolution chain are visible instead of silently changing behavior.Tests
Mirrors master-v2's testing approach for this behavior: unit tests with mocking, no E2E.
JumpStartModelSpecsbuilt from master-v2constants.pyBASE_SPEC;JumpStartModelsAccessor.get_model_specsis patched to return it (the real spec-resolution chain runs, only the network fetch is mocked).ml.g5.12xlargewithml.m5.largeexcluded from supported types, so the tests discriminate the buggy fallback from the correct spec default.ml.m5.large != ml.g5.12xlarge), 10/10 pass with the fix. Existing private hub unit suite (17 tests) unaffected.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.