You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(deploy): the control-owned shared artifact store — record contract, read-only mount, consumer binding - #6999
Maintainer, 2026-10-19: prebuilt bundles, modules, the assembly cache and the NuGet cache are content-addressed, yet every portal keeps its own copy on its own /data. On 2026-10-19 the public instance's 128 GiB share filled (≈121 GiB of it prebuilt-bundles/) and every core CD and module publication failed with "The specified share is full". The decision: one store per environment, owned by the CONTROL instance, produced only by the build server, mounted read-only by every other instance. Design: Doc/Architecture/SharedArtifactStore (in this PR and in the companion storage-prune PR; identical file).
What (increment 1, platform half)
Contract — DeploymentContent.ArtifactStore (role: Owner|Consumer, volume, consumer: sourceNamespace + sourceClaim) and VolumeClaim.ReadOnly. Either role renders PreWarm__PrebuiltBundleRoot=<mount>/prebuilt-bundles; a consumer also renders PreWarm__PrebuiltBundleRetention__Delete=false. DeploymentPortalConfig.ArtifactStoreProblems names every shape that would give the store a second writer or no store. WithArtifactStore, WithVolume(readOnly); parity table row.
Chart — persistence.<name>.readOnly → readOnly: true on the mount and the claim.
Operator — hosting-artifact-store-bind: the static read-only PV (ReadOnlyMany, csi.readOnly, Retain, distinct handle) + claim binding the owner's share into a consumer namespace, the share read off the owner's bound PV (never typed into a record); an existing binding is verified, never overwritten. ClusterRole gains persistentvolumes: create. Declared in the not-yet-wired list until the Plugins plan step emits it.
Evidence
MeshWeaver.Deployment.Contract.Test: 95/95 green, incl. the new ArtifactStoreRecordTest. Reverting the consumer read-only guard and the sweep disarm turns two of them red (checked).
Operator behaviour suite, bind block: 17/17 green; making the PV writable and skipping the share check turns 2 red (checked).
helm template with persistence.artifacts.readOnly=true renders readOnly: true on both the mount and the claim.
Live effect
None on merge: no record declares artifactStore yet. The ClusterRole change reaches the cluster only through Memex's helm-release lane. The record changes (control as Owner, then each portal as Consumer) and BAKE_PUBLISH_TARGETS are maintainer steps, listed in the design doc.
…ct, read-only chart mount, consumer binding
Maintainer, 2026-10-19: the content-addressed build output every portal copies onto its own /data
(prebuilt bundles first) moves to ONE store per environment, OWNED by the control instance, produced
only by the build server, and mounted READ-ONLY by every other instance. This is the platform half of
increment 1 (Doc/Architecture/SharedArtifactStore):
- contract: DeploymentContent.ArtifactStore (role Owner|Consumer, the volume that carries it, and for
a consumer the owner's namespace + claim) and VolumeClaim.ReadOnly; either role renders
PreWarm__PrebuiltBundleRoot as <mount>/prebuilt-bundles, a consumer also disarms its per-portal
bundle sweep; DeploymentPortalConfig.ArtifactStoreProblems names every shape that would give the
store a second writer or no store (writable consumer, consumer creating a claim, store on /data,
bundles.registry beside it, missing source); WithArtifactStore + WithVolume(readOnly).
- chart: persistence.<name>.readOnly renders readOnly on the mount and the claim.
- operator: hosting-artifact-store-bind creates the STATIC read-only PV (ReadOnlyMany, csi.readOnly,
Retain, distinct handle) + claim that bind the owner's share into a consumer namespace — the share is
read off the owner's bound PV, never typed into a record; an existing binding is verified, never
overwritten. ClusterRole: persistentvolumes create.
Nothing here changes a live instance: no record declares the block yet, and the Plugins plan step that
emits the binding lands after this image is on hosting-operator:main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Introduces the platform foundation for a control-owned shared artifact store, reducing duplicate build artifacts across portals. No deployment record enables it yet.
Changes:
Adds Owner/Consumer contracts, validation and bundle-root configuration.
Adds read-only Helm mounts and an operator command for consumer bindings.
Adds regression coverage and architecture documentation.
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 6fab96e. SyncVolumes now passes isReadOnly to WithVolume, and a change of the flag counts as stale in the replacement comparison, just like a change of source.
… sweep not scheduled (Enabled=false), Aspire honours readOnly, doc links and supported procedure
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The red brew install + test (macOS) is infrastructure and has nothing to do with this diff. Homebrew moved azure-cli from homebrew/core to a cask ("No available formula with the name azure-cli … brew install --cask azure-cli"), so the job fails before it touches anything this PR changes. The check is not required.
[P2] Remove the duplicated closing brackets from the links in both new manuals. [Storage Prune]](...) does not form an inline Markdown link. The same typo occurs in SharedArtifactStore.md:121,138,178–181 and StoragePrune.md:110–112, leaving the measurement, retention and publication references without working navigation. Replace ]]( with ]( at those locations and retain the corrected ../ targets.
…t>/{rid}/prebuilt-bundles, expanded by the portal at boot
A compatibility key names no architecture and the publisher refuses a second architecture under one
key, so an amd64 and an arm64 build would collide in one shared store. The store keeps one bundle
store per RID in the layout every reader already reads; the record renders the root with a {rid}
placeholder and ConfigureMemexMesh expands it to the process's own <os>-<arch> before anything reads
the key (BundleRootRuntimeIdentifier). The producer needs no new code: one BAKE_PUBLISH_TARGETS
entry per architecture, <account>/<share>/<rid>. Docs: fixed the doubled ]] in the sibling links.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[P2] Honor ReadOnly for fixed-path volumes too, or reject those uses explicitly. A normal users volume with ReadOnly=true still mounts writable in Helm: its fixed mount and claim reference never read this flag (deployment.yaml:438–439, 706–707), and both new loops exclude it. Aspire honors the same setting, so the record now has different write permissions across renderers. Rejecting reserved names specifically for artifact stores does not cover ordinary volume declarations.
Allow explicit false to replace an existing read-only setting
[P2] An explicit readOnly: false cannot undo an earlier true: this OR permanently preserves the read-only setting through WithVolume. Reconfiguring the volume as writable therefore still produces a read-only mount. Use a nullable optional argument so omission preserves the existing setting and explicit false replaces it.
[P2] An Aspire Consumer still gets its own Docker volume rather than the owner's store. SyncVolumes always derives the source as <resource-name>-<volume-name> and does not resolve SourceNamespace or SourceClaim. Owner and Consumer resources therefore receive different volumes, while the Consumer's bundle root points into its separate read-only volume and finds none of the owner's publications. Resolve the owner volume for Consumers, or explicitly reject this role on Aspire until shared binding is supported.
…tore; registry template kept; Aspire refuses a Consumer
- hosting-pv-purge keeps every share a binding labels meshweaver.io/purpose=artifact-store or any other
namespace's PV references, deleting only the stale PV object (first behaviour tests for pv-purge;
disabling the guard turns 3 red).
- BundleRootRuntimeIdentifier keeps the unexpanded root at PreWarm:PrebuiltBundleRootTemplate for readers
that serve other runtimes.
- Aspire's SyncVolumes refuses artifactStore.role Consumer by name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Answers to the review's summary items that have no thread:
"Consumer teardown … can delete the shared backing store": correct, and serious. hosting-pv-purge deletes the share behind every Released PV a torn-down namespace claimed, and a consumer's binding PV is claimed by the consumer's namespace. Fixed in this PR: pv-purge keeps every share that a binding labels meshweaver.io/purpose=artifact-storeor that any other namespace's PV references, and deletes only the stale PV object. Both conditions are read from one kubectl get pv -o json listing, and a listing that cannot be read deletes nothing. These are the first behaviour tests pv-purge has had (stubs/pv-purge): the own share is deleted; the store share and a cross-referenced unlabelled share are kept; shares_kept is reported. Disabling the guard turns 3 of them red (checked).
"Resolve the owner volume for Aspire Consumers": Aspire now refuses artifactStore.role: Consumer by name in SyncVolumes. A consumer's binding is a Kubernetes static PV, and a Docker volume of its own would hold none of the owner's publications.
Existing-binding validation: hosting-artifact-store-bind already refuses an existing PV pointing at another share, a writable one, or one claimed by another claim, and refuses a consumer claim bound to another volume. All of these are covered in the bind block.
Make runtime-specific publication a rollout prerequisite rather than saying the producer needs no new code. main-cd.yml:3554–3555 gives every architecture lane the same target list, and publish-bake-bundles.sh:1708–1710 publishes to every entry. Adding both runtime paths sends each architecture to both stores, populating the wrong subtree and triggering the other architecture's collision refusal. Also, the current ARM64 lane produces linux-arm64, not osx-arm64; the publisher accepts only Linux runtimes. Correct this paragraph and rollout step 3: require per-lane target selection, use linux-arm64 for the existing ARM64 build, and mark macOS publication as pending a macOS producer.
On "Make runtime-specific publication a rollout prerequisite" (previously missed, no thread): correct on every point. The doc no longer says the producer needs no new code. Per-lane target selection is now increment 2 and a rollout prerequisite for maintainer step 3: today every lane gets the same list and publishes to every entry, and the lease path is derived from the same targets by bake-publication-lock.sh and rebind-bake-scope.sh. The existing arm64 lane is named linux-arm64, and osx-arm64 is marked as waiting for a macOS producer. The identical copy in #7000 carries the same text.
On the latest summary, both items are valid and fixed: (1) the PV and the claim are now created (kubectl create -f -), never applied, so a resource that appears between the read and the write fails the run instead of being patched into shape unverified; (2) an existing binding must carry its own volumeHandle (5th segment = the PV name), and one sharing the owner's handle is refused. The bind block is now 23/23.
The reason will be displayed to describe this comment to others. Learn more.
Added PrebuiltBundleRetentionStartupTest. It starts the real hosted service with a non-blank root and fires ApplicationStarted. Enabled is the positive control: the first pass runs and records its fault at once on a mesh-less host. Disabled records nothing. Making the guard if (false && …) turns it red (checked).
Wrong share: the owner PV's explicit storageAccount/shareName/resourceGroup now win over the handle, the same way pv-purge reads them. Tested with an owner whose attributes differ from its handle.
Stale staging credentials: an existing binding whose nodeStageSecretRef differs from the owner's current one is refused. Tested.
Conflicting reservation blocks recovery: when the PV is created and the claim creation then fails, a re-run verifies that PV (same share, read-only, Retain, labelled, own handle, owner's Secret) and creates the claim. A PV or claim that fails verification is a named refusal by design: a binding pointing elsewhere is a person's decision, and this command never deletes. The refusal says exactly which object to remove.
Bind block: 25/25.
…ssify against the retention contract (+ StoragePrune, SharedArtifactStore docs) (#7000)
* feat(operator): hosting-storage-prune — measure what fills an artifact volume, classify it against the retention contract
On 2026-10-19 the public instance's 128 GiB /data share was 100% full and every core CD and module
publication failed with "The specified share is full". The portal's own bundle sweep had collected
nothing for weeks — correctly: the fleet consumer inventory was incomplete — but its verdict was a
log line in one pod.
hosting-storage-prune walks a data volume (or, with --claim, the control-owned shared artifact
store) in a short-lived Job in the instance's namespace, read-only by default, and reports ONE
gzip+base64 JSON fact the control plane records: the share's size and use, the top-level
directories, and every prebuilt-bundle identity and generation classified as current, young,
unsealed, superseded, legacy or other. It never deletes under prebuilt-bundles/ (release artifacts:
deletion needs the complete consumer inventory, the protected-label set and the publication lease).
--execute deletes only abandoned assembly-cache temp files (>1 day) and dumps shadowed by a dump
volume (>7 days), each re-checked in the pod at the moment of deletion.
Behaviour tests run the walker FOR REAL on a fixture tree: the execute run deletes exactly the two
permitted files and leaves every other byte — current/sealed/unsealed/young generations, legacy
identities, release markers, caches, the young temp file, the young dump, a temp file behind a
symlink — untouched.
Docs: Doc/Architecture/StoragePrune (the measurement and the classes) and
Doc/Architecture/SharedArtifactStore (the control-owned store, architecture-aware layout, one-way
sync to a client environment's control, local-module builds, maintainer steps); linked from the
retention contract.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: sibling links resolve (../), supported surfaces in the maintainer steps — same text as #6999
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: StoragePrune states the schedule is prepared, not running; consumers switch only after the rid-aware reader/publisher
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(operator): storage-prune classifies a legacy identity once, never again through its generations
Measured on the first report-only run against the public instance's data share (2026-10-20):
181 superseded and 197 unsealed generations at 0 KiB were the generations inside legacy identities,
counted a second time. The identity is the item; its generations are state only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(operator): storage-prune walks every per-architecture store of the shared store (<rid>/prebuilt-bundles); docs links fixed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: SharedArtifactStore — template key, Aspire refusal, teardown keeps the store (same text as #6999)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: no store role on an instance serving other runtimes (same text as #6999)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(operator): a failed read is never a zero — storage-prune reports unreadable paths, marks the report incomplete, and an incomplete walk deletes nothing
Review: an incomplete filesystem scan could read as a successful one (du/stat errors went to
/dev/null and counted as 0). The walker now emits @@mw-store unreadable for every failed read (on
fd 3, so a read inside $(…) cannot swallow it), the report carries complete/unreadableCount and
storage_complete=false, and --execute keeps every candidate when any read failed. Tested with a du
that fails for one identity.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(operator): storage-prune — a value flag without a value is refused (no endless shift loop); a failing listing marks the walk incomplete
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(operator): hosting-storage-prune is wired — MeshWeaver.Plugins#3993 emits it; move it into the plan list
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: the prune command ships with #7000 — status says so until its image is live
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: per-lane target selection is a rollout prerequisite (increment 2); the arm64 lane is linux-arm64
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Maintainer, 2026-10-19: prebuilt bundles, modules, the assembly cache and the NuGet cache are content-addressed, yet every portal keeps its own copy on its own
/data. On 2026-10-19 the public instance's 128 GiB share filled (≈121 GiB of itprebuilt-bundles/) and every core CD and module publication failed with "The specified share is full". The decision: one store per environment, owned by the CONTROL instance, produced only by the build server, mounted read-only by every other instance. Design:Doc/Architecture/SharedArtifactStore(in this PR and in the companion storage-prune PR; identical file).What (increment 1, platform half)
DeploymentContent.ArtifactStore(role: Owner|Consumer,volume, consumer:sourceNamespace+sourceClaim) andVolumeClaim.ReadOnly. Either role rendersPreWarm__PrebuiltBundleRoot=<mount>/prebuilt-bundles; a consumer also rendersPreWarm__PrebuiltBundleRetention__Delete=false.DeploymentPortalConfig.ArtifactStoreProblemsnames every shape that would give the store a second writer or no store.WithArtifactStore,WithVolume(readOnly); parity table row.persistence.<name>.readOnly→readOnly: trueon the mount and the claim.hosting-artifact-store-bind: the static read-only PV (ReadOnlyMany,csi.readOnly, Retain, distinct handle) + claim binding the owner's share into a consumer namespace, the share read off the owner's bound PV (never typed into a record); an existing binding is verified, never overwritten. ClusterRole gainspersistentvolumes: create. Declared in the not-yet-wired list until the Plugins plan step emits it.Evidence
MeshWeaver.Deployment.Contract.Test: 95/95 green, incl. the newArtifactStoreRecordTest. Reverting the consumer read-only guard and the sweep disarm turns two of them red (checked).helm templatewithpersistence.artifacts.readOnly=truerendersreadOnly: trueon both the mount and the claim.Live effect
None on merge: no record declares
artifactStoreyet. The ClusterRole change reaches the cluster only through Memex's helm-release lane. The record changes (control as Owner, then each portal as Consumer) andBAKE_PUBLISH_TARGETSare maintainer steps, listed in the design doc.🤖 Generated with Claude Code