Skip to content

refactor(server): seal backend construction behind a plan - #691

Open
Graffioh wants to merge 2 commits into
Luce-Org:mainfrom
Graffioh:codex/backend-plan-boundary
Open

refactor(server): seal backend construction behind a plan#691
Graffioh wants to merge 2 commits into
Luce-Org:mainfrom
Graffioh:codex/backend-plan-boundary

Conversation

@Graffioh

@Graffioh Graffioh commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

First PR in the server-refactor stack. It makes backend construction consume one model-aware, normalized plan and gives the resulting values stable, read-only groupings.

  • make BackendArgs own operator input, including model and draft paths
  • reduce server admission input to a narrow BackendAdmissionContext
  • resolve GGUF metadata, placement compatibility, warnings, and SpecLA fallback once in prepare_backend()
  • make create_backend(const BackendPlan&) the only backend-construction API
  • make retained architecture and layer-split configs own their path storage
  • group effective values into model, placement, cache, speculation, execution, and DeepSeek4 views
  • use the same normalized values for construction, startup logging, tokenizer setup, cache policy, and /props
  • keep ServerConfig and mutable BackendArgs out of backend construction
operator input + server admission facts
                 |
            prepare_backend
                 |
        read-only BackendPlan
      / model     placement \
 cache  speculation  execution  architecture options
                 |
     construction + server reporting

Ownership and boundary

Persistent backend configs use std::string for required paths and std::optional<std::string> for optional draft paths. Configs are moved into backend members, and .c_str() is used only at immediate C API boundaries. Constructed backends therefore do not borrow path storage from BackendArgs or BackendPlan.

BackendPlan is still the effective configuration source shared by construction and the server. This keeps normalization consistent without adding a second reporting snapshot. Such a snapshot should be introduced only if it can replace this dependency rather than duplicate the same state.

SpecLA environment mutation is deliberately unchanged: its graph helpers still consume process environment today, so removing that mechanism is a separate behavioral change.

Stack

This is PR 1 of 2:

  1. refactor(server): seal backend construction behind a plan #691 — backend planning, owned configs, and grouped effective values (this PR, head 25fc625fd)
  2. refactor(server): establish owned generation lifecycle #692 — owned generation lifecycle and runtime owner (head 2e6309477)

The history is intentionally two commits: c1a451fc0 establishes the sealed planning boundary and owned persistent configs; 25fc625fd groups the resulting effective values by concern.

#693 has been incorporated into this PR. The branches live in a fork, so both PRs target main; #692 is based on the previous head a567baef9 and needs a restack after this PR's review amendment.

Review only #692's follow-up delta: Graffioh/lucebox-hub@codex/backend-plan-boundary...codex/luce-engine-implementation

Verification

  • compile-time ownership contracts cover every persistent backend and layer-split config
  • backend-plan tests cover owned-path lifetime, construction API sealing, normalization, KVFlash fallback, unsupported-model fallback, and missing-draft failure
  • feature-gate tests cover the server admission projection
  • dflash_server, test_feature_gate, test_server_unit, test_deepseek4_unit, test_generate, and test_dflash build on HIP/gfx1151
  • 449/449 model-free server and plan/feature tests pass
  • git diff --check

Supersedes the docs-only proposal in #688 with executable boundaries and tests.

Review amendment

Consolidated SpecLA fallback cleanup in the plan builder, preserving warnings, DDTree selection, explicit thresholds, and environment actions. Added coverage for explicit DDTree settings on unsupported SpecLA models.

The feature-gate and backend-plan suites pass with both CUDA and HIP placement defines. The added case also passes against the pre-amendment implementation. git diff --check passes. These checks are model-free; no new GPU inference run was performed for this cleanup.

@Graffioh
Graffioh force-pushed the codex/backend-plan-boundary branch 6 times, most recently from 1208435 to ab10aee Compare September 2, 2026 08:27
@Graffioh
Graffioh marked this pull request as ready for review September 2, 2026 12:40
@Graffioh
Graffioh marked this pull request as draft September 2, 2026 12:41

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 12 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread server/src/common/backend_plan.cpp
Comment thread server/test/test_backend_plan.cpp
@Graffioh
Graffioh marked this pull request as ready for review September 2, 2026 13:02

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 12 files

Requires human review: Auto-approval blocked by 2 unresolved issues from a previous review of this commit.

Re-trigger cubic

@Graffioh
Graffioh marked this pull request as draft September 2, 2026 13:26
@Graffioh
Graffioh marked this pull request as ready for review September 2, 2026 19:20

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 12 files

Requires human review: Auto-approval blocked by 2 unresolved issues from a previous review of this commit.

Re-trigger cubic

@Graffioh
Graffioh force-pushed the codex/backend-plan-boundary branch 2 times, most recently from 7733bc2 to 4313f21 Compare September 2, 2026 19:57
@Graffioh
Graffioh force-pushed the codex/backend-plan-boundary branch from 4313f21 to a15ad98 Compare September 3, 2026 08:14
@Graffioh
Graffioh force-pushed the codex/backend-plan-boundary branch 2 times, most recently from 6c9abb0 to 1c7111c Compare September 3, 2026 11:55
@Graffioh

Graffioh commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@Graffioh I have started the AI code review. It will take a few minutes to complete.

@Graffioh
Graffioh force-pushed the codex/backend-plan-boundary branch from 1c7111c to a567bae Compare September 3, 2026 12:04

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread server/test/test_backend_plan.cpp
@Graffioh
Graffioh force-pushed the codex/backend-plan-boundary branch from a567bae to 25fc625 Compare September 5, 2026 05:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant