Skip to content

Add opt-in planning for materialized forecast updates - #1176

Draft
aldenks wants to merge 1 commit into
mainfrom
materialized-update-planning
Draft

aldenks wants to merge 1 commit into
mainfrom
materialized-update-planning

Conversation

@aldenks

@aldenks aldenks commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Add opt-in planning for materialized forecast updates before enabling earlier triggers. This first layer adds the shared planner and jobs:list permission but opts in no datasets, so deployed data-processing behavior remains unchanged. #1177 enables each target's planning together with its virtual trigger.

Worker 0 can freeze a shared work/skip plan before writing: yield to an older unfinished Job of the same CronJob, remove init times already in the primary store, and trim the end to the newest source-ready candidate in wxopticon. Interior inits remain included. Replacement pods reuse the same template end and labels. Validation retries retain the frontier and ignore their finished-writing parent. A reprocessing option supports GEFS35's delayed extension.

The planned follow-up pairs are AIFS Single (about 46 minutes of launch headroom, 29 recent matched runs), GFS forecast (11–12 minutes, one recent complete sample plus retained Job timing), GEFS35 extension (2h35–2h45 before the daily cron), and HRRR48 (small, variable surface-file gains). These estimates precede source-summary/polling overhead; this infrastructure-only PR claims no latency improvement. Analysis cadence remains unchanged. HRRR analysis would require extra updates for its larger gain; GFS/GEFS analysis have small gains with dependency-window complications. ICON-EU and HRDPS archivers run too late to help; existing IFS46 triggers and unpaired products are unchanged.

When enabled, the Job guard is deliberately non-atomic: Icechunk rejects conflicting primary publication, while deprecated Zarr replicas retain rare exposure. A retry can still yield to an unrelated older writer, and a stuck Pending trigger can block later updates until resolved. Frontier deduplication removes routine re-download repair, so sub-threshold holes require a repair backfill. Source-summary errors fail target planning. Skip plans remain as small per-Job files for late pods.

Rollout: grant the deployer jobs:list using the updated bootstrap Role, deploy this infrastructure, verify the Role and unchanged cron behavior, then enable the follow-up PR. No live rollout was performed.

Validation: repository-wide format, lint and type checks pass. The shared-path suite passes 343 tests; all affected dataset suites pass 327 tests, including slow tests. The infrastructure-only layer independently passes 247 tests from an exported index, plus its own format/lint/type checks. Three independent review passes are complete; no high- or medium-severity findings remain. Both commit signatures are verified by GitHub. Remote CI passes on amd64 and arm64, including Kubernetes admission and CodeQL checks.

@aldenks
aldenks added this pull request to stack #1178 October 7, 2026 20:54
@aldenks aldenks changed the title materialized update planning Add opt-in planning for materialized forecast updates Oct 7, 2026
@aldenks aldenks self-assigned this Oct 7, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant