From 80627993d72312bd7906a71606e31675b26beb6c Mon Sep 17 00:00:00 2001 From: bdchatham Date: Thu, 13 Aug 2026 12:52:55 -0700 Subject: [PATCH 01/13] feat(sidecar): build the node sidecar from this repo as its own module MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 3: the sidecar implementation moves out of the seictl repo and becomes a second binary built here. seictl keeps its CLI; what it loses is the server the controller actually talks to. sidecar/ carries the sei-chain graph and nothing else does. It restates all 11 of sei-chain's replaces, not the one the plan for this originally claimed — two of them matter beyond resolution. 99designs/keyring is substituted for cosmos/keyring, and that is the library which decrypts the validator operator keyring, so a partial port would silently change key handling. go-ethereum redirects to the Sei fork sei-chain/app expects; since a replace only applies from the main module, any module linking this graph without it resolves upstream geth instead. Both verified resolving through their forks. The module is wired by filesystem replace, not go.work. A workspace promotes a used module's replaces to main-module status, and this one pins x/crypto and grpc *down* from what the controller needs, so a workspace build would diverge from every GOWORK=off build — Docker, release CI, and any external consumer. internal/patch could not come with it: seictl's CLI needs it too and Go will not export an internal package across repos. Duplicating 245 lines of merge logic across a module boundary is the same drift-with-no-guard shape this migration exists to remove, so it lands in the contract module as sidecarapi/tomlpatch and both consumers import it. That widens sidecarapi past "the wire contract" — a deliberate trade, recorded rather than silent. Two startup refusals, because the failures they replace were silent: - SEI_HOME is required with no fallback. serve.go read it from a package-level var that seictl's root main.go owned and which does not move; the fallback was "/sei" while the controller mounts the data PVC at $HOME/.sei. Dropping the wiring would not crash — it would produce a running, probe-passing sidecar writing genesis and config into an empty directory, fleet-wide. - A configured SEI_KEYRING_BACKEND with SEI_SIDECAR_AUTHN_MODE unauthenticated is refused, before the keyring is opened. Unauthenticated is the empty string, binds every interface and installs no middleware, and the two settings were previously decided independently and never compared. One dropped variable exposed POST /v0/tasks — gov-vote included — to any pod in the cluster with an open keyring, while probes passed and tasks succeeded. Extracted as checkKeyringNeedsAuthn and covered for all five combinations, plus a test pinning that unauthenticated stays the zero value, since the guard's correctness depends on an unset variable matching it. The image installs a seictl symlink beside sei-sidecar. The controller renders Command: []string{"seictl", "serve"} into every pod spec, and controller and sidecar images roll independently, so a rename has no safe ordering in either direction — and it fails quietly, not loudly: seid blocks on a shell loop polling /v0/healthz behind a StartupProbe with FailureThreshold 86400 at 5s, so a mis-ordered rollout hangs for days rather than alerting. With the shim, adopting the image is a pure images.sidecar reference change. Removal condition is in the Dockerfile. CI and the Makefile loop MODULES throughout. Go package patterns stop at a nested module boundary, so a root-only run would have left 23,700 lines — including every test touching validator keys — uncompiled, unlinted and untested with a green check. Verified: gofmt, goimports, build, vet, tidy-check and verify-generated clean across all three modules; 14 root + 3 sidecarapi + 8 sidecar test packages pass; the controller's build closure still holds zero chain-graph packages; SEI_HOME unset and keyring-without-authn both refuse to start when exercised against the built binary. Not verified here: the image build (no Docker daemon available) and the ECR publish, which needs a sei/sei-sidecar repository provisioned in the platform repo first. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 2 +- CLAUDE.md | 25 +- Makefile | 18 +- sidecar/Dockerfile | 49 + sidecar/actions/process.go | 146 + sidecar/actions/process_test.go | 174 + sidecar/docs/authn.md | 85 + sidecar/docs/keyring.md | 76 + sidecar/engine/checkpoint.go | 24 + sidecar/engine/engine.go | 485 +++ sidecar/engine/engine_e2e_test.go | 313 ++ sidecar/engine/engine_test.go | 1203 +++++++ sidecar/engine/mark_not_ready_test.go | 433 +++ sidecar/engine/metrics.go | 54 + sidecar/engine/sqlite_migrations.go | 151 + sidecar/engine/sqlite_store.go | 273 ++ sidecar/engine/sqlite_store_test.go | 389 +++ sidecar/engine/store.go | 41 + sidecar/engine/typed_handler.go | 50 + sidecar/engine/typed_handler_test.go | 154 + sidecar/engine/types.go | 153 + sidecar/go.mod | 251 ++ sidecar/go.sum | 2947 +++++++++++++++++ sidecar/main.go | 58 + sidecar/rpc/client.go | 131 + sidecar/rpc/client_test.go | 160 + sidecar/rpc/status.go | 72 + sidecar/rpc/status_test.go | 125 + sidecar/rpc/types.go | 61 + sidecar/s3/client.go | 82 + sidecar/s3/client_test.go | 146 + sidecar/s3/emit.go | 150 + sidecar/s3/emit_test.go | 167 + sidecar/s3/errors.go | 55 + sidecar/s3/errors_test.go | 120 + sidecar/serve.go | 255 ++ sidecar/serve_test.go | 148 + sidecar/server/auth.go | 135 + sidecar/server/auth_test.go | 145 + sidecar/server/keyring.go | 108 + sidecar/server/keyring_test.go | 124 + sidecar/server/server.go | 249 ++ sidecar/server/server_test.go | 519 +++ sidecar/shadow/close_test.go | 50 + sidecar/shadow/comparator.go | 197 ++ sidecar/shadow/comparator_migration_test.go | 142 + sidecar/shadow/comparator_test.go | 441 +++ sidecar/shadow/fetch.go | 43 + sidecar/shadow/fetch_test.go | 243 ++ sidecar/shadow/keysource.go | 129 + sidecar/shadow/keysource_test.go | 115 + sidecar/shadow/layer0.go | 66 + sidecar/shadow/layer1.go | 119 + sidecar/shadow/layer2.go | 123 + sidecar/shadow/layer2_test.go | 244 ++ sidecar/shadow/metrics.go | 35 + sidecar/shadow/render.go | 152 + sidecar/shadow/render_layer2_test.go | 41 + sidecar/shadow/report.go | 57 + sidecar/shadow/types.go | 164 + sidecar/startup_guard_test.go | 88 + sidecar/tasks/assemble_genesis.go | 702 ++++ sidecar/tasks/assemble_genesis_dedup_test.go | 149 + .../tasks/assemble_genesis_external_test.go | 467 +++ sidecar/tasks/assemble_genesis_test.go | 379 +++ sidecar/tasks/assemble_genesis_validators.go | 162 + .../tasks/assemble_genesis_validators_test.go | 210 ++ sidecar/tasks/await_condition.go | 164 + sidecar/tasks/await_condition_test.go | 326 ++ sidecar/tasks/config.go | 118 + sidecar/tasks/config_apply.go | 138 + sidecar/tasks/config_apply_test.go | 257 ++ sidecar/tasks/config_reload.go | 82 + sidecar/tasks/config_reload_test.go | 88 + sidecar/tasks/config_test.go | 327 ++ sidecar/tasks/config_validate.go | 77 + sidecar/tasks/config_validate_test.go | 75 + sidecar/tasks/defaults/config.toml | 18 + sidecar/tasks/defaults/defaults.go | 6 + sidecar/tasks/evm_logical_digest.go | 337 ++ sidecar/tasks/evm_logical_digest_test.go | 329 ++ sidecar/tasks/generate_gentx.go | 459 +++ sidecar/tasks/generate_gentx_test.go | 44 + sidecar/tasks/generate_identity.go | 96 + sidecar/tasks/generate_identity_test.go | 73 + sidecar/tasks/genesis.go | 220 ++ sidecar/tasks/genesis_overrides.go | 106 + sidecar/tasks/genesis_overrides_test.go | 329 ++ sidecar/tasks/genesis_peers.go | 135 + sidecar/tasks/genesis_test.go | 186 ++ sidecar/tasks/genesis_writeback.go | 55 + sidecar/tasks/gov_param_change.go | 181 + sidecar/tasks/gov_param_change_test.go | 166 + sidecar/tasks/gov_result.go | 122 + sidecar/tasks/gov_result_test.go | 231 ++ sidecar/tasks/gov_software_upgrade.go | 154 + sidecar/tasks/gov_software_upgrade_test.go | 145 + sidecar/tasks/gov_vote.go | 109 + sidecar/tasks/gov_vote_test.go | 100 + sidecar/tasks/mark_not_ready.go | 52 + sidecar/tasks/mark_not_ready_test.go | 39 + sidecar/tasks/peers.go | 16 + sidecar/tasks/ready.go | 15 + sidecar/tasks/reset_data.go | 176 + sidecar/tasks/reset_data_test.go | 177 + sidecar/tasks/restart_seid.go | 201 ++ sidecar/tasks/restart_seid_test.go | 192 ++ sidecar/tasks/result_compare.go | 346 ++ sidecar/tasks/result_compare_test.go | 208 ++ sidecar/tasks/result_export.go | 311 ++ sidecar/tasks/result_export_test.go | 572 ++++ sidecar/tasks/sign_and_broadcast.go | 517 +++ sidecar/tasks/sign_and_broadcast_test.go | 890 +++++ sidecar/tasks/snapshot_restore.go | 330 ++ sidecar/tasks/snapshot_restore_test.go | 438 +++ sidecar/tasks/snapshot_upload.go | 494 +++ sidecar/tasks/snapshot_upload_metrics.go | 50 + sidecar/tasks/snapshot_upload_test.go | 710 ++++ sidecar/tasks/statesync.go | 362 ++ sidecar/tasks/statesync_test.go | 697 ++++ sidecar/tasks/stop_seid.go | 110 + sidecar/tasks/stop_seid_test.go | 76 + sidecar/tasks/transactions.go | 211 ++ sidecar/tasks/transactions_test.go | 50 + .../tasks/typed_handler_integration_test.go | 339 ++ sidecar/tasks/upload_genesis_artifacts.go | 155 + .../tasks/upload_genesis_artifacts_test.go | 124 + sidecarapi/go.mod | 1 + sidecarapi/go.sum | 9 + sidecarapi/tomlpatch/file.go | 51 + sidecarapi/tomlpatch/json.go | 55 + sidecarapi/tomlpatch/merge.go | 36 + sidecarapi/tomlpatch/merge_test.go | 55 + sidecarapi/tomlpatch/toml.go | 54 + 134 files changed, 28392 insertions(+), 4 deletions(-) create mode 100644 sidecar/Dockerfile create mode 100644 sidecar/actions/process.go create mode 100644 sidecar/actions/process_test.go create mode 100644 sidecar/docs/authn.md create mode 100644 sidecar/docs/keyring.md create mode 100644 sidecar/engine/checkpoint.go create mode 100644 sidecar/engine/engine.go create mode 100644 sidecar/engine/engine_e2e_test.go create mode 100644 sidecar/engine/engine_test.go create mode 100644 sidecar/engine/mark_not_ready_test.go create mode 100644 sidecar/engine/metrics.go create mode 100644 sidecar/engine/sqlite_migrations.go create mode 100644 sidecar/engine/sqlite_store.go create mode 100644 sidecar/engine/sqlite_store_test.go create mode 100644 sidecar/engine/store.go create mode 100644 sidecar/engine/typed_handler.go create mode 100644 sidecar/engine/typed_handler_test.go create mode 100644 sidecar/engine/types.go create mode 100644 sidecar/go.mod create mode 100644 sidecar/go.sum create mode 100644 sidecar/main.go create mode 100644 sidecar/rpc/client.go create mode 100644 sidecar/rpc/client_test.go create mode 100644 sidecar/rpc/status.go create mode 100644 sidecar/rpc/status_test.go create mode 100644 sidecar/rpc/types.go create mode 100644 sidecar/s3/client.go create mode 100644 sidecar/s3/client_test.go create mode 100644 sidecar/s3/emit.go create mode 100644 sidecar/s3/emit_test.go create mode 100644 sidecar/s3/errors.go create mode 100644 sidecar/s3/errors_test.go create mode 100644 sidecar/serve.go create mode 100644 sidecar/serve_test.go create mode 100644 sidecar/server/auth.go create mode 100644 sidecar/server/auth_test.go create mode 100644 sidecar/server/keyring.go create mode 100644 sidecar/server/keyring_test.go create mode 100644 sidecar/server/server.go create mode 100644 sidecar/server/server_test.go create mode 100644 sidecar/shadow/close_test.go create mode 100644 sidecar/shadow/comparator.go create mode 100644 sidecar/shadow/comparator_migration_test.go create mode 100644 sidecar/shadow/comparator_test.go create mode 100644 sidecar/shadow/fetch.go create mode 100644 sidecar/shadow/fetch_test.go create mode 100644 sidecar/shadow/keysource.go create mode 100644 sidecar/shadow/keysource_test.go create mode 100644 sidecar/shadow/layer0.go create mode 100644 sidecar/shadow/layer1.go create mode 100644 sidecar/shadow/layer2.go create mode 100644 sidecar/shadow/layer2_test.go create mode 100644 sidecar/shadow/metrics.go create mode 100644 sidecar/shadow/render.go create mode 100644 sidecar/shadow/render_layer2_test.go create mode 100644 sidecar/shadow/report.go create mode 100644 sidecar/shadow/types.go create mode 100644 sidecar/startup_guard_test.go create mode 100644 sidecar/tasks/assemble_genesis.go create mode 100644 sidecar/tasks/assemble_genesis_dedup_test.go create mode 100644 sidecar/tasks/assemble_genesis_external_test.go create mode 100644 sidecar/tasks/assemble_genesis_test.go create mode 100644 sidecar/tasks/assemble_genesis_validators.go create mode 100644 sidecar/tasks/assemble_genesis_validators_test.go create mode 100644 sidecar/tasks/await_condition.go create mode 100644 sidecar/tasks/await_condition_test.go create mode 100644 sidecar/tasks/config.go create mode 100644 sidecar/tasks/config_apply.go create mode 100644 sidecar/tasks/config_apply_test.go create mode 100644 sidecar/tasks/config_reload.go create mode 100644 sidecar/tasks/config_reload_test.go create mode 100644 sidecar/tasks/config_test.go create mode 100644 sidecar/tasks/config_validate.go create mode 100644 sidecar/tasks/config_validate_test.go create mode 100644 sidecar/tasks/defaults/config.toml create mode 100644 sidecar/tasks/defaults/defaults.go create mode 100644 sidecar/tasks/evm_logical_digest.go create mode 100644 sidecar/tasks/evm_logical_digest_test.go create mode 100644 sidecar/tasks/generate_gentx.go create mode 100644 sidecar/tasks/generate_gentx_test.go create mode 100644 sidecar/tasks/generate_identity.go create mode 100644 sidecar/tasks/generate_identity_test.go create mode 100644 sidecar/tasks/genesis.go create mode 100644 sidecar/tasks/genesis_overrides.go create mode 100644 sidecar/tasks/genesis_overrides_test.go create mode 100644 sidecar/tasks/genesis_peers.go create mode 100644 sidecar/tasks/genesis_test.go create mode 100644 sidecar/tasks/genesis_writeback.go create mode 100644 sidecar/tasks/gov_param_change.go create mode 100644 sidecar/tasks/gov_param_change_test.go create mode 100644 sidecar/tasks/gov_result.go create mode 100644 sidecar/tasks/gov_result_test.go create mode 100644 sidecar/tasks/gov_software_upgrade.go create mode 100644 sidecar/tasks/gov_software_upgrade_test.go create mode 100644 sidecar/tasks/gov_vote.go create mode 100644 sidecar/tasks/gov_vote_test.go create mode 100644 sidecar/tasks/mark_not_ready.go create mode 100644 sidecar/tasks/mark_not_ready_test.go create mode 100644 sidecar/tasks/peers.go create mode 100644 sidecar/tasks/ready.go create mode 100644 sidecar/tasks/reset_data.go create mode 100644 sidecar/tasks/reset_data_test.go create mode 100644 sidecar/tasks/restart_seid.go create mode 100644 sidecar/tasks/restart_seid_test.go create mode 100644 sidecar/tasks/result_compare.go create mode 100644 sidecar/tasks/result_compare_test.go create mode 100644 sidecar/tasks/result_export.go create mode 100644 sidecar/tasks/result_export_test.go create mode 100644 sidecar/tasks/sign_and_broadcast.go create mode 100644 sidecar/tasks/sign_and_broadcast_test.go create mode 100644 sidecar/tasks/snapshot_restore.go create mode 100644 sidecar/tasks/snapshot_restore_test.go create mode 100644 sidecar/tasks/snapshot_upload.go create mode 100644 sidecar/tasks/snapshot_upload_metrics.go create mode 100644 sidecar/tasks/snapshot_upload_test.go create mode 100644 sidecar/tasks/statesync.go create mode 100644 sidecar/tasks/statesync_test.go create mode 100644 sidecar/tasks/stop_seid.go create mode 100644 sidecar/tasks/stop_seid_test.go create mode 100644 sidecar/tasks/transactions.go create mode 100644 sidecar/tasks/transactions_test.go create mode 100644 sidecar/tasks/typed_handler_integration_test.go create mode 100644 sidecar/tasks/upload_genesis_artifacts.go create mode 100644 sidecar/tasks/upload_genesis_artifacts_test.go create mode 100644 sidecarapi/tomlpatch/file.go create mode 100644 sidecarapi/tomlpatch/json.go create mode 100644 sidecarapi/tomlpatch/merge.go create mode 100644 sidecarapi/tomlpatch/merge_test.go create mode 100644 sidecarapi/tomlpatch/toml.go diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1882f140..b4612a71 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -15,7 +15,7 @@ jobs: # One job per Go module. golangci-lint resolves packages with Go # patterns, which stop at a nested module boundary — a root-only run # would never lint sidecarapi/ and would pass while it was broken. - module: [".", "sidecarapi"] + module: [".", "sidecarapi", "sidecar"] steps: - uses: actions/checkout@v4 - uses: actions/setup-go@v5 diff --git a/CLAUDE.md b/CLAUDE.md index 76bf6c5a..7fd8718c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,6 @@ # sei-k8s-controller -Kubernetes operator for managing Sei blockchain nodes. Single binary, three controllers: `SeiNetwork` (genesis-ceremony orchestration: bootstraps a chain's genesis.json and founding validator set, owns the child SeiNodes), `SeiNode` (individual node lifecycle), and `SeiNodeTask` (sidecar-driven task execution). +Kubernetes operator for managing Sei blockchain nodes, plus the per-node sidecar it drives. Two binaries across three Go modules. Three controllers: `SeiNetwork` (genesis-ceremony orchestration: bootstraps a chain's genesis.json and founding validator set, owns the child SeiNodes), `SeiNode` (individual node lifecycle), and `SeiNodeTask` (sidecar-driven task execution). ## Architecture @@ -10,6 +10,29 @@ Kubernetes operator for managing Sei blockchain nodes. Single binary, three cont - **Entry point**: `cmd/main.go` — thin binary that creates a `manager.Manager` and registers both controllers - **Framework**: controller-runtime v0.23.1 / kubebuilder v4.12.0 +### Modules + +Three Go modules, wired by filesystem `replace` — **not** `go.work`. A workspace promotes a used module's `replace` directives to main-module status, and `sidecar/` pins `golang.org/x/crypto` and `google.golang.org/grpc` *down* from what the controller needs, so a workspace build would diverge from every `GOWORK=off` build (Docker, release CI, external consumers). + +| Module | Contains | Dependency profile | +|---|---|---| +| `.` (root) | the controller, `api/v1alpha1`, `sdk/sei` | controller-runtime + k8s. **No chain graph** | +| `sidecarapi/` | `api/` (OpenAPI spec), `client/` (generated), `wire/` (contract types), `tomlpatch/` | light; ~10 modules. **No replace directives** — a dependency module's replaces are ignored, so any here would be a silent no-op | +| `sidecar/` | the sidecar binary: `tasks/`, `engine/`, `server/`, `s3/`, `shadow/`, `rpc/`, `actions/` | the sei-chain graph; restates all 11 of sei-chain's replaces | + +Anything that walks packages must loop `MODULES` in the Makefile. Go package patterns stop at a nested module boundary, so `go list ./...` in the root does **not** see `sidecarapi/` or `sidecar/` — a root-only lint or test passes while a whole module goes uncompiled. + +Two checks keep the controller tidyable, both in `make ci`. The `depguard` rule `contract-stays-light` in `.golangci.yml` denies the chain graph to anything under `sidecarapi/`, `_test.go` files included — that is the import a test added once before, and it stopped every consumer's `go mod tidy` from working. `make tidy-check` runs `go mod tidy -diff` per module, which catches the unresolvable graph that import produces. Neither covers a third-party dependency that transitively reaches the chain graph while still resolving; that is a dependency-review question, not a lint one. + +### The sidecar binary + +`sidecar/main.go` → `sei-sidecar`, published to ECR as `sei/sei-sidecar`. The image also installs a `seictl` symlink because the controller renders `Command: []string{"seictl", "serve"}` into every pod spec (`internal/noderesource/`, `internal/task/bootstrap_resources.go`). Controller and sidecar images roll independently, so a rename without the shim has no safe ordering — and the failure is silent, not loud: seid blocks on a shell loop polling `/v0/healthz` behind a StartupProbe with `FailureThreshold: 86400` at 5s. Remove the shim only after the controller stops rendering `Command` and every cell has rolled past that controller. + +Two startup refusals in `sidecar/` are load-bearing; do not soften them into defaults: + +- `SEI_HOME` is **required**, with no fallback. It previously defaulted to `/sei` while the controller mounts the data PVC at `$HOME/.sei`, so a dropped value produced a running, probe-passing sidecar writing genesis and config into an empty directory. +- A configured `SEI_KEYRING_BACKEND` with `SEI_SIDECAR_AUTHN_MODE` unauthenticated is refused (`checkKeyringNeedsAuthn`). Unauthenticated binds all interfaces and installs no middleware, so that combination exposes the sign-tx API — gov-vote included — to any pod in the cluster while the keyring is open. + ## Subagents Always use the available subagents for relevant work: diff --git a/Makefile b/Makefile index 1cf82b6b..0a9da6e3 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,7 @@ IMG ?= sei-k8s-controller:latest +# The sidecar publishes to ECR as sei/sei-sidecar; the tag is supplied by CI. +SIDECAR_IMG ?= sei-sidecar:latest +CONTAINER_TOOL ?= docker GOLANGCI_LINT ?= $(shell which golangci-lint 2>/dev/null || echo $(HOME)/go/bin/golangci-lint) # Pinned tool versions. Bump together: setup-envtest's release branch tracks @@ -21,13 +24,24 @@ SETUP_ENVTEST ?= $(LOCALBIN)/setup-envtest # module boundary — `go list ./...` in the root does NOT descend into # sidecarapi/ — so anything that walks packages must loop this list or the # nested module goes unbuilt, unlinted and untested while CI stays green. -MODULES ?= . sidecarapi +MODULES ?= . sidecarapi sidecar -.PHONY: build test test-modules test-integration test-all lint lint-modules tidy-check manifests generate verify-generated setup-envtest ci docker-build docker-push +.PHONY: build build-sidecar docker-build-sidecar test test-modules test-integration test-all lint lint-modules tidy-check manifests generate verify-generated setup-envtest ci docker-build docker-push build: ## Build manager binary. go build -o bin/manager ./cmd/ +build-sidecar: ## Build the sidecar binary. + @# GOWORK=off keeps this identical to the Docker and release builds: the + @# sidecar module pins x/crypto and grpc *down*, and a workspace would + @# promote those replaces into the root module's resolution. + cd sidecar && GOWORK=off go build -o ../bin/sei-sidecar . + +docker-build-sidecar: ## Build the sidecar container image. + @# Build context is the repo root — sidecar/ resolves sidecarapi/ through a + @# filesystem replace, so both module trees must be in the context. + $(CONTAINER_TOOL) build -f sidecar/Dockerfile -t $(SIDECAR_IMG) . + test: test-modules ## Run tests (root module with coverage, then every other module). go test $$(go list ./... | grep -v /e2e) -coverprofile cover.out diff --git a/sidecar/Dockerfile b/sidecar/Dockerfile new file mode 100644 index 00000000..1d8952db --- /dev/null +++ b/sidecar/Dockerfile @@ -0,0 +1,49 @@ +# The per-node sidecar image. Built from the repo root as the build context, so +# the sidecarapi module this one resolves through a filesystem `replace` is +# present: +# +# docker build -f sidecar/Dockerfile -t . +# +# GOWORK=off is set for the build because this module pins golang.org/x/crypto +# and google.golang.org/grpc *down* relative to the controller module. Under a +# workspace those replaces would be promoted to main-module status and leak into +# the root module's resolution; off keeps this build identical to a consumer's. +FROM docker.io/golang:1.26 AS build + +ENV GOWORK=off +WORKDIR /workspace + +# Manifests first so the dependency layer caches independently of source edits. +# Both modules are needed: sidecar/go.mod's `replace` target must exist before +# `go mod download` can resolve it. +COPY sidecar/go.mod sidecar/go.sum ./sidecar/ +COPY sidecarapi/go.mod sidecarapi/go.sum ./sidecarapi/ +RUN cd sidecar && go mod download + +COPY sidecarapi/ ./sidecarapi/ +COPY sidecar/ ./sidecar/ + +ARG TARGETOS +ARG TARGETARCH +RUN cd sidecar && CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} \ + go build -o /go/bin/sei-sidecar . + +FROM gcr.io/distroless/static-debian12 +COPY --from=build /go/bin/sei-sidecar /usr/bin/sei-sidecar + +# Compatibility shim, deliberately temporary. The controller renders +# `Command: []string{"seictl", "serve"}` into every SeiNode pod spec +# (internal/noderesource/noderesource.go, internal/task/bootstrap_resources.go). +# Controller rollouts and sidecar-image rollouts are independent, so a binary +# rename with no shim has no safe ordering: an old controller against a new +# image fails to find `seictl`, and a new controller against an old image fails +# to find the new name. Either way the pod does not crash loudly — the seid +# container blocks on a shell loop polling /v0/healthz behind a StartupProbe +# with FailureThreshold 86400 at 5s, so it hangs for days instead of alerting. +# +# With the shim, adopting this image is a pure `images.sidecar` reference +# change. Remove it only after the controller stops rendering `Command` (letting +# this ENTRYPOINT decide) and every cell has rolled past that controller. +COPY --from=build /go/bin/sei-sidecar /usr/bin/seictl + +ENTRYPOINT ["/usr/bin/sei-sidecar"] diff --git a/sidecar/actions/process.go b/sidecar/actions/process.go new file mode 100644 index 00000000..43dce18d --- /dev/null +++ b/sidecar/actions/process.go @@ -0,0 +1,146 @@ +package actions + +import ( + "context" + "errors" + "fmt" + "os" + "strconv" + "strings" + "syscall" + "time" + + "github.com/sei-protocol/seilog" +) + +var logger = seilog.NewLogger("seictl", "actions") + +const ( + DefaultGracePeriod = 30 * time.Second + exitPollInterval = 100 * time.Millisecond +) + +// ProcessSignaler abstracts process discovery and signaling for testability. +type ProcessSignaler interface { + FindPID(processName string) (int, error) + Signal(pid int, sig syscall.Signal) error + Alive(pid int) bool +} + +// FindPID scans /proc for a process whose argv[0] matches processName. +// Requires shareProcessNamespace: true in the Kubernetes pod spec. +func FindPID(processName string) (int, error) { + entries, err := os.ReadDir("/proc") + if err != nil { + return 0, fmt.Errorf("reading /proc: %w", err) + } + for _, e := range entries { + if !e.IsDir() { + continue + } + pid, err := strconv.Atoi(e.Name()) + if err != nil { + continue + } + cmdline, err := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", pid)) + if err != nil || len(cmdline) == 0 { + continue + } + exe := firstArg(cmdline) + if exe == processName || strings.HasSuffix(exe, "/"+processName) { + return pid, nil + } + } + return 0, fmt.Errorf("process %q not found in /proc", processName) +} + +// SignalPID sends a signal to the given process. +func SignalPID(pid int, sig syscall.Signal) error { + proc, err := os.FindProcess(pid) + if err != nil { + return err + } + return proc.Signal(sig) +} + +// PIDAlive returns true if the process is still running. +func PIDAlive(pid int) bool { + proc, err := os.FindProcess(pid) + if err != nil { + return false + } + return proc.Signal(syscall.Signal(0)) == nil +} + +// isProcessGone returns true if the error indicates the process no longer exists. +func isProcessGone(err error) bool { + return errors.Is(err, os.ErrProcessDone) || errors.Is(err, syscall.ESRCH) +} + +// GracefulStop sends SIGTERM to the named process, waits up to +// gracePeriod for it to exit, then escalates to SIGKILL. +// When signaler is nil the package-level functions are used directly. +func GracefulStop(ctx context.Context, signaler ProcessSignaler, processName string, gracePeriod time.Duration) error { + findPID := FindPID + signal := SignalPID + alive := PIDAlive + if signaler != nil { + findPID = signaler.FindPID + signal = signaler.Signal + alive = signaler.Alive + } + + pid, err := findPID(processName) + if err != nil { + if isProcessGone(err) { + logger.Info("process already exited", "process", processName) + return nil + } + return fmt.Errorf("finding %s process: %w", processName, err) + } + logger.Info("sending SIGTERM", "process", processName, "pid", pid) + + if err := signal(pid, syscall.SIGTERM); err != nil { + if isProcessGone(err) { + logger.Info("process exited before SIGTERM delivered", "process", processName, "pid", pid) + return nil + } + return fmt.Errorf("sending SIGTERM to pid %d: %w", pid, err) + } + + ticker := time.NewTicker(exitPollInterval) + defer ticker.Stop() + deadline := time.After(gracePeriod) + + for { + select { + case <-ctx.Done(): + return ctx.Err() + case <-deadline: + logger.Warn("process did not exit after SIGTERM, sending SIGKILL", "process", processName, "pid", pid) + if err := signal(pid, syscall.SIGKILL); err != nil { + if isProcessGone(err) { + logger.Info("process exited before SIGKILL delivered", "process", processName, "pid", pid) + return nil + } + return fmt.Errorf("sending SIGKILL to pid %d: %w", pid, err) + } + logger.Info("SIGKILL sent", "process", processName, "pid", pid) + return nil + case <-ticker.C: + if !alive(pid) { + logger.Info("process exited after SIGTERM", "process", processName, "pid", pid) + return nil + } + } + } +} + +func firstArg(cmdline []byte) string { + for i, c := range cmdline { + if c == 0 { + return string(cmdline[:i]) + } + } + return string(cmdline) +} diff --git a/sidecar/actions/process_test.go b/sidecar/actions/process_test.go new file mode 100644 index 00000000..c19366a0 --- /dev/null +++ b/sidecar/actions/process_test.go @@ -0,0 +1,174 @@ +package actions + +import ( + "context" + "fmt" + "os" + "sync/atomic" + "syscall" + "testing" + "time" +) + +type mockSignaler struct { + findPID int + findErr error + signalFn func(pid int, sig syscall.Signal) error + signalErr error + alive atomic.Bool + signals []syscall.Signal +} + +func (m *mockSignaler) FindPID(string) (int, error) { return m.findPID, m.findErr } + +func (m *mockSignaler) Signal(pid int, sig syscall.Signal) error { + m.signals = append(m.signals, sig) + if m.signalFn != nil { + return m.signalFn(pid, sig) + } + return m.signalErr +} + +func (m *mockSignaler) Alive(int) bool { return m.alive.Load() } + +func TestGracefulStop_ImmediateExit(t *testing.T) { + sig := &mockSignaler{findPID: 42} + sig.alive.Store(false) + + if err := GracefulStop(context.Background(), sig, "seid", time.Second); err != nil { + t.Fatalf("expected success, got %v", err) + } + if len(sig.signals) != 1 || sig.signals[0] != syscall.SIGTERM { + t.Errorf("expected single SIGTERM, got %v", sig.signals) + } +} + +func TestGracefulStop_EscalatesToSIGKILL(t *testing.T) { + sig := &mockSignaler{findPID: 42} + sig.alive.Store(true) + + if err := GracefulStop(context.Background(), sig, "seid", 200*time.Millisecond); err != nil { + t.Fatalf("expected success after SIGKILL, got %v", err) + } + if len(sig.signals) < 2 { + t.Fatalf("expected >= 2 signals, got %d", len(sig.signals)) + } + if sig.signals[0] != syscall.SIGTERM { + t.Errorf("first signal: expected SIGTERM, got %v", sig.signals[0]) + } + if sig.signals[len(sig.signals)-1] != syscall.SIGKILL { + t.Errorf("last signal: expected SIGKILL, got %v", sig.signals[len(sig.signals)-1]) + } +} + +func TestGracefulStop_FindPIDError(t *testing.T) { + sig := &mockSignaler{findErr: fmt.Errorf("not found")} + if err := GracefulStop(context.Background(), sig, "seid", time.Second); err == nil { + t.Fatal("expected error") + } +} + +func TestGracefulStop_FindPIDProcessGone(t *testing.T) { + sig := &mockSignaler{findErr: syscall.ESRCH} + err := GracefulStop(context.Background(), sig, "seid", time.Second) + if err != nil { + t.Fatalf("expected success when process already gone, got %v", err) + } + if len(sig.signals) != 0 { + t.Errorf("expected no signals sent, got %v", sig.signals) + } +} + +func TestGracefulStop_SIGTERMProcessGone(t *testing.T) { + sig := &mockSignaler{ + findPID: 42, + signalFn: func(_ int, s syscall.Signal) error { + if s == syscall.SIGTERM { + return os.ErrProcessDone + } + return nil + }, + } + err := GracefulStop(context.Background(), sig, "seid", time.Second) + if err != nil { + t.Fatalf("expected success when process exited before SIGTERM, got %v", err) + } +} + +func TestGracefulStop_SIGKILLProcessGone(t *testing.T) { + sig := &mockSignaler{ + findPID: 42, + signalFn: func(_ int, s syscall.Signal) error { + if s == syscall.SIGKILL { + return syscall.ESRCH + } + return nil + }, + } + sig.alive.Store(true) + + err := GracefulStop(context.Background(), sig, "seid", 200*time.Millisecond) + if err != nil { + t.Fatalf("expected success when process exited before SIGKILL, got %v", err) + } +} + +func TestGracefulStop_SignalError(t *testing.T) { + sig := &mockSignaler{findPID: 42, signalErr: fmt.Errorf("permission denied")} + if err := GracefulStop(context.Background(), sig, "seid", time.Second); err == nil { + t.Fatal("expected error") + } +} + +func TestGracefulStop_ContextCancellation(t *testing.T) { + sig := &mockSignaler{findPID: 42} + sig.alive.Store(true) + + ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond) + defer cancel() + + err := GracefulStop(ctx, sig, "seid", 10*time.Second) + if err == nil { + t.Fatal("expected context error") + } + if err != context.DeadlineExceeded { + t.Fatalf("expected DeadlineExceeded, got %v", err) + } +} + +func TestFirstArg(t *testing.T) { + tests := []struct { + name string + cmdline []byte + expected string + }{ + {"null delimited", []byte("seid\x00start\x00--home\x00/sei"), "seid"}, + {"no null byte", []byte("seid"), "seid"}, + {"full path", []byte("/usr/bin/seid\x00start"), "/usr/bin/seid"}, + {"null at start", []byte("\x00rest"), ""}, + {"empty", []byte{}, ""}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := firstArg(tt.cmdline) + if got != tt.expected { + t.Errorf("firstArg(%q) = %q, want %q", tt.cmdline, got, tt.expected) + } + }) + } +} + +func TestIsProcessGone(t *testing.T) { + if !isProcessGone(os.ErrProcessDone) { + t.Error("expected ErrProcessDone to be recognized") + } + if !isProcessGone(syscall.ESRCH) { + t.Error("expected ESRCH to be recognized") + } + if isProcessGone(fmt.Errorf("something else")) { + t.Error("expected generic error to not be recognized") + } + if isProcessGone(nil) { + t.Error("expected nil to not be recognized") + } +} diff --git a/sidecar/docs/authn.md b/sidecar/docs/authn.md new file mode 100644 index 00000000..ec3a65bd --- /dev/null +++ b/sidecar/docs/authn.md @@ -0,0 +1,85 @@ +# Sidecar HTTP API authentication + +The sidecar exposes its task API over plain HTTP. Authentication is +controlled by a single environment variable; the runtime trust +boundary is the pod, not the loopback interface. + +## Environment contract + +| Env | Values | Default | Behavior | +|---|---|---|---| +| `SEI_SIDECAR_AUTHN_MODE` | unset \| `unauthenticated` \| `trusted-header` | unset | unset/`unauthenticated`: bind all interfaces, no auth. `trusted-header`: bind loopback, require `X-Remote-User` on non-probe paths. | + +Parsing is strict: any non-empty value other than `unauthenticated` +or `trusted-header` is a startup error so a typo cannot silently +degrade a hardened deployment. + +## `trusted-header` mode + +The sidecar is paired with an in-pod `kube-rbac-proxy` container on +TLS `:8443`. The proxy performs TokenReview + a single coarse +`create seinodetasks.sei.io` SubjectAccessReview against the K8s API, +then forwards passed requests to `127.0.0.1:7777` with `X-Remote-User` +naming the authenticated identity. + +The sidecar requires exactly one non-empty `X-Remote-User` value per +request. Two values fail closed — that would mean the proxy is +appending rather than overwriting, which would let an attacker- +supplied header arrive first. + +### Bypass paths + +Four paths skip the `X-Remote-User` check because their callers do +not carry auth headers: + +| Path | Caller | +|---|---| +| `/v0/healthz` | kubelet readiness probe | +| `/v0/startupz` | kubelet startup probe | +| `/v0/livez` | kubelet liveness probe | +| `/v0/metrics` | Prometheus scrape | + +The `kube-rbac-proxy` `--allow-paths` flag must include all four so +probes and scrapes traverse the proxy without TokenReview. The +authoritative list is logged at sidecar startup in trusted-header +mode and is exposed programmatically via `server.BypassPaths()`. + +Rejected requests are counted in the +`seictl_sidecar_authn_rejections_total{reason}` metric — labels +`missing_header`, `duplicate_header`, `empty_header` distinguish the +common misconfigurations. + +### Pod-level isolation requirements + +The trust boundary is the pod. Pods running in `trusted-header` mode +MUST NOT enable any of: + +- `hostNetwork: true` — collapses loopback into the host's network + namespace, exposing `127.0.0.1:7777` to every other `hostNetwork` + pod on the node. +- `hostPID: true` — lets off-pod processes attach. +- `hostIPC: true` — exposes SysV IPC across the pod boundary. + +A colocated container in the pod can still reach `127.0.0.1:7777` +directly and forge `X-Remote-User`. With `shareProcessNamespace: +true` (the current `SeiNode` default) it can also read +`/proc//mem` and exfiltrate the unlocked keyring — memory +read is the load-bearing threat, not header forgery. The middleware +does not defend against either; the controller-side configuration of +the pod is what establishes the boundary. + +## Controller-side contract + +The contract this sidecar exposes to `sei-k8s-controller`: + +- Env var: `SEI_SIDECAR_AUTHN_MODE=trusted-header` +- Internal port: `7777`, bound to `127.0.0.1` +- Probe paths: `/v0/healthz` (readiness), `/v0/startupz` (startup), `/v0/livez` (liveness) +- Prometheus path: `/v0/metrics` +- `kube-rbac-proxy --allow-paths`: `/v0/healthz,/v0/startupz,/v0/livez,/v0/metrics` +- Forwarded header: `X-Remote-User` — proxy MUST overwrite, not + append +- Pod isolation: `hostNetwork`/`hostPID`/`hostIPC` all false + +Kubelet probes hit the pod IP, not loopback. With the sidecar bound +loopback-only, probes must be routed through the proxy port. diff --git a/sidecar/docs/keyring.md b/sidecar/docs/keyring.md new file mode 100644 index 00000000..3365707e --- /dev/null +++ b/sidecar/docs/keyring.md @@ -0,0 +1,76 @@ +# Sidecar keyring backend + +The sidecar opens a Cosmos SDK keyring at startup when configured to do so. +The keyring is the entry point for all transaction-signing tasks (governance +votes, software-upgrade proposals, deposits — Component C of the in-pod +governance signing design). + +This document covers the operator-facing contract. The full design — including +the controller-side Secret projection that materializes the keyring directory +in the pod — lives in the [in-pod-governance-signing design](https://github.com/sei-protocol/bdchatham-designs/blob/main/designs/governance-signing/in-pod-governance-signing.md) (relocated to the bdchatham-designs repo per Design 05 / PLT-497). + +## Environment contract + +| Env | Values | Default | Required when | +|---|---|---|---| +| `SEI_KEYRING_BACKEND` | `test` \| `file` \| `os` | unset (governance signing disabled) | sign-tx tasks in use | +| `SEI_KEYRING_DIR` | absolute path | `/keyring-file` where `` is the `--home` flag (defaults to `$SEI_HOME`) | required for `file` | +| `SEI_KEYRING_PASSPHRASE` | string | unset | `backend == file` | + +For the `file` backend, a trailing `/keyring-file` path segment is stripped before handoff to the SDK — the Cosmos SDK keyring re-appends `keyring-file/` internally, so callers passing `/sei/keyring-file` and `/sei` both end up at `/sei/keyring-file/*`. This matches both operator mental models. + +Unset `SEI_KEYRING_BACKEND` is the default: the sidecar starts normally +and rejects sign-tx submissions with `keyring not configured`. The node's +genesis-ceremony tasks (e.g. `generate-gentx`) continue to function — those +use a separate, in-process test backend that is intentionally isolated from +production keys (see "Genesis isolation" below). + +Unknown values for `SEI_KEYRING_BACKEND` cause the sidecar to refuse to start. +KMS / HSM / Vault / remote-signer backends are not supported yet. + +## Fail-fast semantics + +When `SEI_KEYRING_BACKEND` is set, the sidecar: + +1. Reads `SEI_KEYRING_BACKEND`, `SEI_KEYRING_DIR`, `SEI_KEYRING_PASSPHRASE`. +2. Validates the backend value and (for `file`) the presence of a passphrase. + A missing passphrase on the file backend is a startup error — operators + see a clear `SEI_KEYRING_PASSPHRASE required when SEI_KEYRING_BACKEND=file` + message and the pod CrashLoopBackOffs. +3. Opens the keyring through the Cosmos SDK. +4. Runs a structural liveness check (`kr.List()`) with a bounded retry of 3 + attempts, 2 seconds between attempts. The retry absorbs the rare kubelet + Secret-mount race where the projected file is briefly absent. +5. Wipes `SEI_KEYRING_PASSPHRASE` from the process environment so the secret + no longer appears in `/proc//environ` for the lifetime of the + container. + +An empty keyring is a permitted outcome of the smoke test — the sidecar +trusts that callers will supply key names that exist when they submit +sign-tx tasks, and surfaces missing-key errors at that point. + +## Trust model + +- The passphrase lives in the process env between `os.Getenv` and + `os.Unsetenv` — a window of a few milliseconds at startup. +- The passphrase is never logged. The sidecar logs `keyring opened` with the + backend and directory only. +- Errors returned from the keyring-open path are scrubbed of any verbatim + occurrence of the passphrase before they leave the function. +- The keyring directory is mounted read-only; the sidecar never writes to it. +- Operators must rotate the passphrase by re-projecting the Secret and + restarting the sidecar pod (see the operator runbook in the design doc). + +## Genesis isolation + +`sidecar/tasks/generate_gentx.go` continues to use `keyring.BackendTest` +unconditionally. The gentx validator key is throwaway by design and must not +share state with the operator's production keyring. The two code paths share +no keyring object and live in different packages. + +## Operator runbook + +See the "Operator runbook: creating the Secrets" section of the +[in-pod-governance-signing design](https://github.com/sei-protocol/bdchatham-designs/blob/main/designs/governance-signing/in-pod-governance-signing.md) for the end-to-end flow: +generating the keyring locally, building the projected Kubernetes Secret, +and wiring it to the validator CRD. diff --git a/sidecar/engine/checkpoint.go b/sidecar/engine/checkpoint.go new file mode 100644 index 00000000..d186d0ec --- /dev/null +++ b/sidecar/engine/checkpoint.go @@ -0,0 +1,24 @@ +package engine + +// TxMarker is the pre-broadcast idempotency record for a sign-tx task — +// engine-owned metadata read during crash recovery, distinct from a handler's +// TaskResult.Result. It carries the signed tx bytes so a re-run re-broadcasts +// the identical tx rather than re-signing (which risks a double submit). +type TxMarker struct { + TaskID string + TxHash string + TxBytes []byte + AccountNumber uint64 + Sequence uint64 + ChainID string +} + +// Checkpointer persists a TxMarker durably before broadcast and retrieves it on +// re-execution, so a crash between broadcast and result-persist re-adopts the +// in-flight tx instead of signing a second one. Nil when no durable store is +// configured (handlers then broadcast without the guard, and log it). +type Checkpointer interface { + SaveTxMarker(m *TxMarker) error + // GetTxMarker returns (nil, nil) when no marker exists for taskID. + GetTxMarker(taskID string) (*TxMarker, error) +} diff --git a/sidecar/engine/engine.go b/sidecar/engine/engine.go new file mode 100644 index 00000000..8dda24bc --- /dev/null +++ b/sidecar/engine/engine.go @@ -0,0 +1,485 @@ +package engine + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "runtime/debug" + "sync" + "sync/atomic" + "time" + + "github.com/google/uuid" + "github.com/sei-protocol/seilog" +) + +var log = seilog.NewLogger("seictl", "engine") + +// ErrInvalidTaskID is returned when a caller-provided task ID is not a valid UUID. +var ErrInvalidTaskID = fmt.Errorf("task ID must be a valid UUID") + +// validateTaskID checks that a non-empty ID string is a valid UUID. +func validateTaskID(id string) error { + if id == "" { + return nil + } + if _, err := uuid.Parse(id); err != nil { + return fmt.Errorf("%w: %q", ErrInvalidTaskID, id) + } + return nil +} + +// Engine is the task executor. Every submitted task runs in its own +// goroutine. The store is the single source of truth for all task state. +// The engine context propagates to all handlers — on SIGTERM the +// context is cancelled and handlers observe ctx.Done() to stop gracefully. +type Engine struct { + handlers map[TaskType]TaskHandler + ctx context.Context + ready atomic.Bool + store ResultStore + mu sync.Mutex + + // cancels holds the cancel func of every currently running task, keyed by + // task ID, so RemoveResult can stop a task's goroutine. Each entry carries + // the generation that registered it so cleanup is a compare-and-delete: a + // resubmit under the same ID registers a fresh entry with a newer generation + // and overwrites the prior one, and a superseded registration's late cleanup + // must not touch the newer entry. Guarded by mu. + cancels map[string]cancelEntry + + // gen mints a strictly-increasing, process-lifetime-unique generation for + // every newTaskContext registration. It discriminates a superseded + // registration's late cleanup from a newer registration's entry even when + // both share a row's run number — run is scoped to a row's lifetime and + // resets to 1 after a delete, so it collides across a DELETE-then-resubmit; + // gen never repeats. + gen atomic.Int64 + + // Config is set once during single-threaded startup before Submit + // is reachable; read-only thereafter. No synchronization. + Config ExecutionConfig +} + +// cancelEntry is a registered task's cancel func tagged with the generation that +// registered it, so clearCancel and RemoveResult can distinguish "my +// registration's entry" from "a newer registration that overwrote mine". +type cancelEntry struct { + cancel context.CancelFunc + gen int64 +} + +// NewEngine creates a new Engine. The engine runs until ctx is cancelled. +// Callers MUST install handler dependencies on e.Config before +// RehydrateStaleTasks, else a rehydrated handler races with the write. +func NewEngine(ctx context.Context, handlers map[TaskType]TaskHandler, store ResultStore) *Engine { + return &Engine{ + handlers: handlers, + ctx: ctx, + store: store, + cancels: make(map[string]cancelEntry), + } +} + +// RehydrateStaleTasks re-executes tasks left in "running" state by a +// previous process that exited before completing them. Run count is +// NOT incremented — rehydration is crash recovery of an incomplete +// run, not a new run. Must be called only after Config is installed. +func (e *Engine) RehydrateStaleTasks() { + stale, err := e.store.ListStaleTasks() + if err != nil { + log.Error("failed to list stale tasks", "err", err) + return + } + + // A node hold must win crash recovery deterministically. Run any stale + // mark-not-ready SYNCHRONOUSLY FIRST: its handler purges stranded mark-ready + // rows from the store and its completion hook flips readiness false. This + // removes the Store(true)/Store(false) race that goroutine dispatch would + // otherwise create between a stranded mark-ready and a stranded + // mark-not-ready — a race that could transiently release a held seid. + // + // CONTRACT: a hold handler run here must stay bounded (a single store op). + // This executes before the HTTP server's ListenAndServe, so livez does not + // serve yet — a slow handler here silently delays startup with no signal. + holdSeen := false + for _, tr := range stale { + if TaskType(tr.Type) != TaskMarkNotReady { + continue + } + holdSeen = true + if handler, ok := e.resolveStaleHandler(tr); ok { + log.Info("rehydrating hold synchronously before other tasks", + "type", tr.Type, "id", tr.ID, "run", tr.Run) + e.mu.Lock() + ctx, gen := e.newTaskContext(tr.ID) + e.mu.Unlock() + e.runTaskSync(ctx, tr.ID, TaskType(tr.Type), handler, tr.Params, tr.SubmittedAt, tr.Run, gen) + } + } + + // Re-read the source of truth after the purge so the mark-ready rows it + // deleted (and the now-terminal mark-not-ready) are not dispatched. + if holdSeen { + stale, err = e.store.ListStaleTasks() + if err != nil { + log.Error("failed to re-list stale tasks after hold", "err", err) + return + } + } + + for _, tr := range stale { + switch TaskType(tr.Type) { + case TaskMarkNotReady: + continue // handled synchronously above + case TaskMarkReady: + // Durable hold supersession: never rehydrate a stranded mark-ready + // while a hold at least as new as it exists in the store. Keyed on + // the persisted mark-not-ready record (any status), this survives + // restarts — a failed purge persists the hold Failed, which the + // in-run holdSeen flag (used only for the re-list above) cannot see + // on a later boot. + if e.markReadySuperseded(tr) { + continue + } + } + if handler, ok := e.resolveStaleHandler(tr); ok { + log.Info("rehydrating stale task", "type", tr.Type, "id", tr.ID, "run", tr.Run) + e.mu.Lock() + ctx, gen := e.newTaskContext(tr.ID) + e.mu.Unlock() + e.runTask(ctx, tr.ID, TaskType(tr.Type), handler, tr.Params, tr.SubmittedAt, tr.Run, gen) + } + } +} + +// markReadySuperseded reports whether a stranded mark-ready must not be +// rehydrated because a hold supersedes it, and if so records that outcome. +// +// The store is the durable source of truth: a mark-not-ready record of ANY +// status (a failed purge persists the hold Failed, so this must not be limited +// to running rows) submitted no earlier than the mark-ready means the hold +// attempt supersedes the stale release. The superseded mark-ready is marked +// Failed ("superseded by hold") so a controller polling it terminates, rather +// than being left running across restarts. A lone stranded mark-ready with no +// newer hold is not superseded — that init-time crash recovery path is +// load-bearing and must still rehydrate. On a store read error it fails closed +// (treats the mark-ready as superseded) rather than risk releasing a hold. +func (e *Engine) markReadySuperseded(mr TaskResult) bool { + hold, err := e.store.LatestByType(string(TaskMarkNotReady)) + if err != nil { + log.Error("checking for a superseding hold; refusing to rehydrate mark-ready", "id", mr.ID, "err", err) + return true + } + if hold == nil || hold.SubmittedAt.Before(mr.SubmittedAt) { + return false + } + log.Warn("stranded mark-ready superseded by a hold; marking failed instead of rehydrating", + "markReadyID", mr.ID, "holdID", hold.ID, "holdStatus", hold.Status) + t := time.Now().UTC() + mr.Status = TaskStatusFailed + mr.Error = "superseded by hold" + mr.CompletedAt = &t + if err := e.store.Save(&mr); err != nil { + log.Error("failed to persist superseded mark-ready", "id", mr.ID, "err", err) + } + return true +} + +// Submit starts a task in its own goroutine and returns its ID. +// +// The engine follows a cloud-API model for task lifecycle: +// - If no task with this ID exists, create and execute it (run 1). +// - If the task is running or completed, return its ID (idempotent no-op). +// - If the task failed, re-execute it with an incremented run counter. +// +// The caller submits a stable key and the engine owns the execution lifecycle. +func (e *Engine) Submit(task Task) (string, error) { + handler, ok := e.handlers[task.Type] + if !ok { + return "", fmt.Errorf("unknown task type: %s", task.Type) + } + + if err := validateTaskID(task.ID); err != nil { + return "", err + } + + id := task.ID + if id == "" { + id = uuid.New().String() + } + + e.mu.Lock() + defer e.mu.Unlock() + + run := 1 + if existing, _ := e.store.Get(id); existing != nil { + switch existing.Status { + case TaskStatusRunning, TaskStatusCompleted: + return id, nil + case TaskStatusFailed: + run = existing.Run + 1 + } + } + + now := time.Now().UTC() + tr := &TaskResult{ + ID: id, + Type: string(task.Type), + Status: TaskStatusRunning, + Run: run, + Params: task.Params, + SubmittedAt: now, + } + + if err := e.store.Save(tr); err != nil { + return "", fmt.Errorf("persist task: %w", err) + } + + log.Info("task submitted", "type", task.Type, "id", id, "run", run) + taskSubmissions.WithLabelValues(string(task.Type)).Inc() + ctx, gen := e.newTaskContext(id) + e.runTask(ctx, id, task.Type, handler, task.Params, now, run, gen) + + return id, nil +} + +// newTaskContext derives a cancellable, task-tagged context from the engine +// root and registers its cancel func under id, tagged with a freshly-minted +// generation, so RemoveResult can stop the task and clearCancel can tell this +// registration's entry from a newer one's. It returns the context and the +// generation it minted; the caller threads that generation to clearCancel so +// cleanup removes only its own entry. The task id is threaded through ctx for +// handlers that need it (e.g. sign-tx memo tagging). Callers MUST hold e.mu; +// clearCancel removes the entry when the task terminates. +func (e *Engine) newTaskContext(id string) (context.Context, int64) { + gen := e.gen.Add(1) + ctx, cancel := context.WithCancel(e.ctx) + e.cancels[id] = cancelEntry{cancel: cancel, gen: gen} + return WithTaskID(ctx, id), gen +} + +// clearCancel cancels and unregisters a task's context when its run reaches a +// terminal state, so completed/failed tasks do not leak cancel funcs. It is a +// compare-and-delete on (id, gen): if a newer registration has overwritten the +// entry, this superseded registration leaves the newer entry untouched and does +// NOT cancel it — otherwise a stale run's late cleanup would kill a live +// context. Idempotent: a no-op when RemoveResult or a newer registration already +// removed this entry. +func (e *Engine) clearCancel(id string, gen int64) { + e.mu.Lock() + defer e.mu.Unlock() + if entry, ok := e.cancels[id]; ok && entry.gen == gen { + entry.cancel() + delete(e.cancels, id) + } +} + +// runTask spawns a goroutine to run the handler and persist the result. +func (e *Engine) runTask(ctx context.Context, id string, taskType TaskType, handler TaskHandler, params map[string]any, submittedAt time.Time, run int, gen int64) { + go e.runTaskSync(ctx, id, taskType, handler, params, submittedAt, run, gen) +} + +// runTaskSync runs the handler and persists the result, blocking until done. +// runTask wraps it in a goroutine; RehydrateStaleTasks calls it directly for a +// stale mark-not-ready that must complete (purge + readiness flip) before any +// other stale task is dispatched. ctx is the per-task cancellable context from +// newTaskContext. +func (e *Engine) runTaskSync(ctx context.Context, id string, taskType TaskType, handler TaskHandler, params map[string]any, submittedAt time.Time, run int, gen int64) { + defer e.clearCancel(id, gen) + result, err := e.executeRecovered(ctx, taskType, handler, params) + + // The task's context was cancelled — either engine shutdown (e.ctx) or an + // explicit DELETE (RemoveResult cancelled this task's context). In both + // cases, leave the store untouched: on shutdown the row stays 'running' so + // RehydrateStaleTasks resumes it on restart (persisting a spurious Failed + // would strand an in-flight sign-tx); on DELETE the handler already removed + // the row, so writing nothing is the true-deletion outcome. Keying on a + // cancelled context (not only a context.Canceled-wrapping error) also + // suppresses the case where cancellation surfaces as an unrelated handler + // error, avoiding a confusing Failed row after the row is already gone. The + // guard matches only context.Canceled, never any ctx error: a + // context.DeadlineExceeded (e.g. a per-task timeout added to newTaskContext) + // falls through to the normal Failed-persistence path so a timed-out one-shot + // task reaches Failed rather than being stranded in 'running'. A run that + // SUCCEEDED (err == nil) is always persisted, even under cancellation, so a + // completed non-idempotent task is never re-run on restart. + if err != nil && (errors.Is(err, context.Canceled) || errors.Is(ctx.Err(), context.Canceled)) { + log.Info("task cancelled; leaving store untouched", + "type", taskType, "id", id, "run", run) + return + } + + t := time.Now().UTC() + tr := &TaskResult{ + ID: id, + Type: string(taskType), + Run: run, + Params: params, + SubmittedAt: submittedAt, + CompletedAt: &t, + } + // Stamp on both paths — a failed run may still carry a result (e.g. a + // tx hash); a panic yields nil, so nothing partial is stamped. + tr.Result = result + if err != nil { + tr.Error = err.Error() + tr.Status = TaskStatusFailed + } else { + tr.Status = TaskStatusCompleted + } + + if storeErr := e.store.Save(tr); storeErr != nil { + log.Error("failed to persist task result", "id", id, "err", storeErr) + } +} + +// resolveStaleHandler returns the handler for a stale task. When no handler is +// registered it marks the task failed, persists that, and returns ok=false. +func (e *Engine) resolveStaleHandler(tr TaskResult) (TaskHandler, bool) { + handler, ok := e.handlers[TaskType(tr.Type)] + if ok { + return handler, true + } + log.Warn("stale task has no handler, marking failed", "type", tr.Type, "id", tr.ID) + t := time.Now().UTC() + tr.Status = TaskStatusFailed + tr.Error = "no handler registered for task type" + tr.CompletedAt = &t + if err := e.store.Save(&tr); err != nil { + log.Error("failed to persist stale task failure", "id", tr.ID, "err", err) + } + return nil, false +} + +// executeRecovered runs execute under a recover so a handler panic becomes a +// failed TaskResult instead of taking down the shared sidecar process. It +// guards only the handler goroutine; a task that spawns its own goroutines must +// recover within them (e.g. s3.streamGzip's writer). +func (e *Engine) executeRecovered(ctx context.Context, taskType TaskType, handler TaskHandler, params map[string]any) (result json.RawMessage, err error) { + defer func() { + if r := recover(); r != nil { + log.Error("task handler panicked", "type", taskType, "panic", r, "stack", string(debug.Stack())) + taskPanics.WithLabelValues(string(taskType)).Inc() + taskFailures.WithLabelValues(string(taskType)).Inc() + err = fmt.Errorf("task handler panicked: %v", r) + } + }() + return e.execute(ctx, taskType, handler, params) +} + +// execute runs a handler synchronously and logs the outcome. +func (e *Engine) execute(ctx context.Context, taskType TaskType, handler TaskHandler, params map[string]any) (json.RawMessage, error) { + start := time.Now() + result, err := handler(ctx, params) + if err != nil { + elapsed := time.Since(start) + log.Error("task failed", "type", taskType, "elapsed", elapsed.Round(time.Millisecond), "err", err) + taskDuration.WithLabelValues(string(taskType), "failed").Observe(elapsed.Seconds()) + taskFailures.WithLabelValues(string(taskType)).Inc() + return result, err + } + elapsed := time.Since(start) + log.Info("task completed", "type", taskType, "elapsed", elapsed.Round(time.Millisecond)) + taskDuration.WithLabelValues(string(taskType), "completed").Observe(elapsed.Seconds()) + // These two hooks are the only writers of e.ready: mark-ready flips it true, + // mark-not-ready flips it false to re-arm the start gate for a node hold. + // The flip happens only on handler success — a failed mark-not-ready purge + // leaves readiness untouched (fail-safe: nothing half-held). + // + // Crash-recovery ordering is enforced in RehydrateStaleTasks, which runs a + // stranded mark-not-ready synchronously (purging stranded mark-ready rows) + // before dispatching anything else, so no Store(true)/Store(false) race can + // transiently release a hold. The one interlock this engine does NOT own: on + // the live path, a controller-submitted mark-ready concurrent with an active + // hold — that mutual exclusion is the controller's (reapproval suppression + // keyed on adoptedWorkflow), not enforced here. + switch taskType { + case TaskMarkReady: + e.ready.Store(true) + case TaskMarkNotReady: + e.ready.Store(false) + } + return result, nil +} + +// Healthz returns true after the engine has been marked ready. +// Use as a readiness check. +func (e *Engine) Healthz() bool { + return e.ready.Load() +} + +// Livez returns nil when the engine's backing store is responsive. +// Use as a liveness check — a non-nil error means the process is wedged +// (e.g., SQLite WAL corruption, PVC read-only). +func (e *Engine) Livez() error { + return e.store.Ping() +} + +// Status returns the engine's current state. +func (e *Engine) Status() StatusResponse { + status := "Initializing" + if e.ready.Load() { + status = "Ready" + } + return StatusResponse{Status: status} +} + +// RecentResults returns the most recent task results across all states. +func (e *Engine) RecentResults() []TaskResult { + results, err := e.store.List(100) + if err != nil { + log.Error("store.List failed", "err", err) + return nil + } + return results +} + +// GetResult returns a task by ID, or nil if not found. +func (e *Engine) GetResult(id string) *TaskResult { + r, err := e.store.Get(id) + if err != nil { + log.Error("store.Get failed", "id", id, "err", err) + return nil + } + return r +} + +// RemoveResult removes a task by ID. It returns (found, err): found reports +// whether a row existed, and a non-nil err means the store delete failed and +// the caller should retry the DELETE. +// +// If the task is currently running, its context is cancelled first so the +// goroutine stops rather than being orphaned by the row deletion. Cancelling +// and clearing the registry entry are separate steps: the task is always +// cancelled to stop the work, but its entry is only removed once store.Delete +// confirms the row is gone. If store.Delete fails the row stays 'running' and +// the failure is surfaced — the caller can retry the DELETE, and failing that a +// process restart's RehydrateStaleTasks resumes the row. Either path recovers +// it, so a failed delete never strands a task in 'running' with nothing able to +// act on it. An already-terminal task has no registered cancel func, so this is +// a plain delete. +func (e *Engine) RemoveResult(id string) (bool, error) { + e.mu.Lock() + entry, hadEntry := e.cancels[id] + if hadEntry { + entry.cancel() + } + e.mu.Unlock() + + deleted, err := e.store.Delete(id) + if err != nil { + log.Error("store.Delete failed; task left recoverable", "id", id, "err", err) + return false, err + } + + if hadEntry { + e.mu.Lock() + if cur, ok := e.cancels[id]; ok && cur.gen == entry.gen { + delete(e.cancels, id) + } + e.mu.Unlock() + } + return deleted, nil +} diff --git a/sidecar/engine/engine_e2e_test.go b/sidecar/engine/engine_e2e_test.go new file mode 100644 index 00000000..099b1d61 --- /dev/null +++ b/sidecar/engine/engine_e2e_test.go @@ -0,0 +1,313 @@ +package engine + +import ( + "context" + "encoding/json" + "errors" + "path/filepath" + "sync" + "sync/atomic" + "testing" + "time" +) + +// newFileStore opens a file-backed SQLite store in a temp directory. +// The DB file and WAL/SHM files are cleaned up automatically by t.TempDir. +func newFileStore(t *testing.T) (*SQLiteStore, string) { + t.Helper() + dbPath := filepath.Join(t.TempDir(), "sidecar.db") + store, err := openStore(dbPath) + if err != nil { + t.Fatalf("open file store: %v", err) + } + t.Cleanup(func() { store.Close() }) + return store, dbPath +} + +// reopenStore closes the existing store and opens a new one against the +// same DB file, simulating a process restart. +func reopenStore(t *testing.T, old *SQLiteStore, dbPath string) *SQLiteStore { + t.Helper() + if err := old.Close(); err != nil { + t.Fatalf("close old store: %v", err) + } + store, err := openStore(dbPath) + if err != nil { + t.Fatalf("reopen store: %v", err) + } + t.Cleanup(func() { store.Close() }) + return store +} + +func TestE2E_TaskLifecycle(t *testing.T) { + store, dbPath := newFileStore(t) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + var handlerCalls atomic.Int32 + handlers := map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, params map[string]any) (json.RawMessage, error) { + handlerCalls.Add(1) + if params["fail"] == true { + return nil, errors.New("intentional failure") + } + return nil, nil + }, + TaskMarkReady: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + return nil, nil + }, + } + + eng := NewEngine(ctx, handlers, store) + + // --- Phase 1: Submit tasks and verify completion --- + + t.Run("submit_and_complete", func(t *testing.T) { + // Submit a successful task with a deterministic ID. + const successID = "11111111-1111-1111-1111-111111111111" + id, err := eng.Submit(Task{ + ID: successID, + Type: TaskConfigPatch, + Params: map[string]any{"file": "config.toml"}, + }) + if err != nil { + t.Fatalf("submit: %v", err) + } + if id != successID { + t.Fatalf("expected ID %q, got %q", successID, id) + } + + result := waitForResult(t, eng, id) + if result.Status != TaskStatusCompleted { + t.Fatalf("expected completed, got %q", result.Status) + } + if result.Error != "" { + t.Fatalf("expected no error, got %q", result.Error) + } + if result.CompletedAt == nil { + t.Fatal("expected CompletedAt to be set") + } + }) + + t.Run("submit_and_fail", func(t *testing.T) { + // Submit a task that will fail. + const failID = "22222222-2222-2222-2222-222222222222" + id, err := eng.Submit(Task{ + ID: failID, + Type: TaskConfigPatch, + Params: map[string]any{"fail": true}, + }) + if err != nil { + t.Fatalf("submit: %v", err) + } + + result := waitForResult(t, eng, id) + if result.Status != TaskStatusFailed { + t.Fatalf("expected failed, got %q", result.Status) + } + if result.Error != "intentional failure" { + t.Fatalf("expected error %q, got %q", "intentional failure", result.Error) + } + }) + + t.Run("dedup_completed", func(t *testing.T) { + // Re-submitting with the same ID should return existing without executing. + before := handlerCalls.Load() + const successID = "11111111-1111-1111-1111-111111111111" + id, err := eng.Submit(Task{ID: successID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("dedup submit: %v", err) + } + if id != successID { + t.Fatalf("dedup should return same ID") + } + // Handler should not have been called again. + time.Sleep(50 * time.Millisecond) + if handlerCalls.Load() != before { + t.Fatal("handler should not have been called for deduped task") + } + }) + + t.Run("mark_ready", func(t *testing.T) { + if eng.Healthz() { + t.Fatal("should not be ready yet") + } + id, _ := eng.Submit(Task{Type: TaskMarkReady}) + waitForResult(t, eng, id) + if !eng.Healthz() { + t.Fatal("should be ready after mark-ready") + } + }) + + // --- Phase 2: Verify RecentResults lists everything --- + + t.Run("recent_results", func(t *testing.T) { + results := eng.RecentResults() + if len(results) < 3 { + t.Fatalf("expected at least 3 results, got %d", len(results)) + } + + // Verify newest-first ordering. + for i := 1; i < len(results); i++ { + if results[i].SubmittedAt.After(results[i-1].SubmittedAt) { + t.Fatalf("results not ordered newest-first at index %d", i) + } + } + }) + + // --- Phase 3: Remove a task --- + + t.Run("remove_task", func(t *testing.T) { + const failID = "22222222-2222-2222-2222-222222222222" + if deleted, err := eng.RemoveResult(failID); err != nil || !deleted { + t.Fatalf("expected remove to return (true, nil), got (%v, %v)", deleted, err) + } + if eng.GetResult(failID) != nil { + t.Fatal("expected nil after removal") + } + if deleted, err := eng.RemoveResult(failID); err != nil || deleted { + t.Fatalf("second remove should return (false, nil), got (%v, %v)", deleted, err) + } + }) + + // --- Phase 5: Simulate restart — close store, reopen, verify persistence --- + + t.Run("survives_restart", func(t *testing.T) { + // Cancel the engine context and close the store. + cancel() + store2 := reopenStore(t, store, dbPath) + + ctx2, cancel2 := context.WithCancel(context.Background()) + defer cancel2() + eng2 := NewEngine(ctx2, handlers, store2) + + // The successful task should still be there. + const successID = "11111111-1111-1111-1111-111111111111" + result := eng2.GetResult(successID) + if result == nil { + t.Fatal("completed task should survive restart") + } + if result.Status != TaskStatusCompleted { + t.Fatalf("expected completed, got %q", result.Status) + } + if result.Params["file"] != "config.toml" { + t.Fatalf("params not preserved: %v", result.Params) + } + + // The removed task should stay removed. + const failID = "22222222-2222-2222-2222-222222222222" + if eng2.GetResult(failID) != nil { + t.Fatal("removed task should not reappear after restart") + } + + // RecentResults should return persisted results. + results := eng2.RecentResults() + if len(results) < 2 { + t.Fatalf("expected at least 2 results after restart, got %d", len(results)) + } + + // Dedup should work against persisted state. + id, err := eng2.Submit(Task{ID: successID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("dedup after restart: %v", err) + } + if id != successID { + t.Fatalf("dedup should return existing ID") + } + + // New tasks should work on the reopened store. + const newID = "44444444-4444-4444-4444-444444444444" + newTaskID, err := eng2.Submit(Task{ID: newID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("new task after restart: %v", err) + } + waitForResult(t, eng2, newTaskID) + newResult := eng2.GetResult(newTaskID) + if newResult.Status != TaskStatusCompleted { + t.Fatalf("new task should complete, got %q", newResult.Status) + } + }) +} + +func TestE2E_StaleTaskRehydration(t *testing.T) { + store, dbPath := newFileStore(t) + + // Simulate a previous crash: insert a task left as "running". + stale := &TaskResult{ + ID: "66666666-6666-6666-6666-666666666666", + Type: "config-patch", + Status: TaskStatusRunning, + SubmittedAt: time.Now().UTC(), + } + store.Save(stale) + + // Reopen store and create a new engine (simulates restart). + store2 := reopenStore(t, store, dbPath) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + handlers := map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + } + eng := NewEngine(ctx, handlers, store2) + eng.RehydrateStaleTasks() + + // Stale task should be re-executed and complete successfully. + result := waitForResult(t, eng, stale.ID) + if result.Status != TaskStatusCompleted { + t.Fatalf("expected rehydrated task to complete, got %q", result.Status) + } +} + +func TestE2E_ConcurrentSubmit(t *testing.T) { + store, _ := newFileStore(t) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + var execCount atomic.Int32 + handlers := map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + execCount.Add(1) + time.Sleep(10 * time.Millisecond) + return nil, nil + }, + } + + eng := NewEngine(ctx, handlers, store) + + // Submit 10 tasks concurrently. + const n = 10 + ids := make([]string, n) + errs := make([]error, n) + var wg sync.WaitGroup + wg.Add(n) + + for i := 0; i < n; i++ { + go func(i int) { + defer wg.Done() + ids[i], errs[i] = eng.Submit(Task{Type: TaskConfigPatch}) + }(i) + } + + wg.Wait() + + for i, err := range errs { + if err != nil { + t.Fatalf("submit %d: %v", i, err) + } + } + + // Wait for all to complete. + for _, id := range ids { + waitForResult(t, eng, id) + } + + if c := execCount.Load(); c != int32(n) { + t.Fatalf("expected %d executions, got %d", n, c) + } + + // All should be in the store. + results := eng.RecentResults() + if len(results) < n { + t.Fatalf("expected at least %d results, got %d", n, len(results)) + } +} diff --git a/sidecar/engine/engine_test.go b/sidecar/engine/engine_test.go new file mode 100644 index 00000000..51b55cba --- /dev/null +++ b/sidecar/engine/engine_test.go @@ -0,0 +1,1203 @@ +package engine + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/prometheus/client_golang/prometheus/testutil" +) + +func newTestEngine(t *testing.T, handlers map[TaskType]TaskHandler) *Engine { + t.Helper() + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + store, err := NewMemoryStore() + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { store.Close() }) + return NewEngine(ctx, handlers, store) +} + +func waitForHealthz(t *testing.T, eng *Engine) { + t.Helper() + deadline := time.Now().Add(2 * time.Second) + for time.Now().Before(deadline) { + if eng.Healthz() { + return + } + time.Sleep(5 * time.Millisecond) + } + t.Fatal("timed out waiting for healthz") +} + +func waitForStatus(t *testing.T, eng *Engine, want string) { + t.Helper() + deadline := time.Now().Add(2 * time.Second) + for time.Now().Before(deadline) { + if eng.Status().Status == want { + return + } + time.Sleep(5 * time.Millisecond) + } + t.Fatalf("timed out waiting for status %q, got %q", want, eng.Status().Status) +} + +func waitForResult(t *testing.T, eng *Engine, id string) *TaskResult { + t.Helper() + deadline := time.Now().Add(2 * time.Second) + for time.Now().Before(deadline) { + if r := eng.GetResult(id); r != nil && r.CompletedAt != nil { + return r + } + time.Sleep(5 * time.Millisecond) + } + t.Fatalf("timed out waiting for result %s", id) + return nil +} + +func cancelRegistrySize(eng *Engine) int { + eng.mu.Lock() + defer eng.mu.Unlock() + return len(eng.cancels) +} + +// --- Submit tests --- + +func TestSubmitAccepts(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskMarkReady: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + id, err := eng.Submit(Task{Type: TaskMarkReady}) + if err != nil { + t.Fatalf("expected nil, got %v", err) + } + if id == "" { + t.Fatal("expected non-empty ID") + } +} + +func TestSubmitCapturesInBandResult(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + return json.RawMessage(`{"genesisHash":"deadbeef"}`), nil + }, + }) + + id, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("submit: %v", err) + } + + result := waitForResult(t, eng, id) + if result.Status != TaskStatusCompleted { + t.Fatalf("status = %q, want completed", result.Status) + } + if string(result.Result) != `{"genesisHash":"deadbeef"}` { + t.Fatalf("result payload = %q, want in-band genesisHash", string(result.Result)) + } +} + +// A ctx-cancelled run (engine shutdown mid-task) must be left 'running' for +// RehydrateStaleTasks, not persisted as Failed — else an in-flight sign-tx is +// stranded (its result/marker never revisited). +func TestRunTaskCtxCancel_LeavesRunningForRehydrate(t *testing.T) { + ran := make(chan struct{}) + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + close(ran) + return nil, context.Canceled // simulates a poll truncated by shutdown + }, + }) + + id, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("submit: %v", err) + } + <-ran + time.Sleep(50 * time.Millisecond) // let runTask's post-handler guard run + + r := eng.GetResult(id) + if r == nil { + t.Fatal("task row missing") + } + if r.Status != TaskStatusRunning { + t.Fatalf("ctx-cancelled task must stay running for rehydration, got %q", r.Status) + } + if r.CompletedAt != nil { + t.Fatal("truncated task must not be marked terminal") + } +} + +func TestSubmitNoResultPayloadIsNil(t *testing.T) { + // Backward-compat: a handler that emits nothing leaves Result nil and + // the field is omitted from the wire, so the deployed controller is + // unaffected. + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + id, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("submit: %v", err) + } + + result := waitForResult(t, eng, id) + if result.Status != TaskStatusCompleted { + t.Fatalf("status = %q, want completed", result.Status) + } + if result.Result != nil { + t.Fatalf("result payload = %q, want nil for a handler that emits nothing", string(result.Result)) + } + out, err := json.Marshal(result) + if err != nil { + t.Fatalf("marshal: %v", err) + } + if strings.Contains(string(out), `"result"`) { + t.Fatalf("nil result must be omitted from the wire; got %s", out) + } +} + +func TestSubmitFailedTaskStampsResult(t *testing.T) { + // The engine stamps the handler's returned result on both the success + // and error paths, so a failed run still carries its (partial) result. + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + return json.RawMessage(`{"genesisHash":"partial"}`), errors.New("boom") + }, + }) + + id, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("submit: %v", err) + } + + result := waitForResult(t, eng, id) + if result.Status != TaskStatusFailed { + t.Fatalf("status = %q, want failed", result.Status) + } + if result.Error != "boom" { + t.Fatalf("error = %q, want boom", result.Error) + } + if string(result.Result) != `{"genesisHash":"partial"}` { + t.Fatalf("failed task should carry its result; got %q", string(result.Result)) + } +} + +func TestSubmitRejectsUnknownType(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{}) + + _, err := eng.Submit(Task{Type: "nonexistent"}) + if err == nil { + t.Fatal("expected error for unknown task type") + } +} + +func TestSubmitCallerProvidedID(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + const customID = "aaaaaaaa-1111-2222-3333-444444444444" + id, err := eng.Submit(Task{ID: customID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("submit: %v", err) + } + if id != customID { + t.Fatalf("expected ID %q, got %q", customID, id) + } + + result := waitForResult(t, eng, id) + if result.ID != customID { + t.Fatalf("result ID = %q, want %q", result.ID, customID) + } +} + +func TestSubmitInvalidIDReturnsTypedError(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + _, err := eng.Submit(Task{ID: "not-a-uuid", Type: TaskConfigPatch}) + if err == nil { + t.Fatal("expected error for non-UUID ID") + } + if !errors.Is(err, ErrInvalidTaskID) { + t.Fatalf("expected ErrInvalidTaskID, got: %v", err) + } +} + +func TestSubmitDedupExistingActive(t *testing.T) { + started := make(chan struct{}) + blocked := make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + close(started) + <-blocked + return nil, nil + }, + }, newTestStore(t)) + + const dedupID = "bbbbbbbb-1111-2222-3333-444444444444" + id1, err := eng.Submit(Task{ID: dedupID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("first submit: %v", err) + } + + select { + case <-started: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for task to start") + } + + id2, err := eng.Submit(Task{ID: dedupID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("second submit: %v", err) + } + if id2 != id1 { + t.Fatalf("dedup should return same ID: got %q and %q", id1, id2) + } + + close(blocked) +} + +func TestSubmitDedupExistingCompleted(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + const dedupID = "cccccccc-1111-2222-3333-444444444444" + id1, _ := eng.Submit(Task{ID: dedupID, Type: TaskConfigPatch}) + waitForResult(t, eng, id1) + + id2, err := eng.Submit(Task{ID: dedupID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("dedup submit: %v", err) + } + if id2 != id1 { + t.Fatalf("dedup should return same ID: got %q and %q", id1, id2) + } +} + +func TestSubmitNoIDGeneratesUUID(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + id1, _ := eng.Submit(Task{Type: TaskConfigPatch}) + id2, _ := eng.Submit(Task{Type: TaskConfigPatch}) + + if id1 == "" || id2 == "" { + t.Fatal("expected non-empty generated IDs") + } + if id1 == id2 { + t.Fatal("generated IDs should be unique") + } +} + +func TestSubmitConcurrent(t *testing.T) { + var callCount atomic.Int32 + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + callCount.Add(1) + time.Sleep(20 * time.Millisecond) + return nil, nil + }, + }) + + id1, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("first submit: %v", err) + } + id2, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("second submit: %v", err) + } + + waitForResult(t, eng, id1) + waitForResult(t, eng, id2) + + if c := callCount.Load(); c != 2 { + t.Fatalf("expected 2 concurrent executions, got %d", c) + } +} + +func TestMarkReadySetsHealthz(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskMarkReady: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + if eng.Healthz() { + t.Fatal("healthz should be false before mark-ready") + } + + _, _ = eng.Submit(Task{Type: TaskMarkReady}) + waitForHealthz(t, eng) + + if !eng.Healthz() { + t.Fatal("healthz should be true after mark-ready") + } +} + +func TestHealthzMonotonicity(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskMarkReady: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + return nil, context.DeadlineExceeded + }, + }) + + id, _ := eng.Submit(Task{Type: TaskMarkReady}) + waitForHealthz(t, eng) + waitForResult(t, eng, id) + + id2, _ := eng.Submit(Task{Type: TaskConfigPatch}) + waitForResult(t, eng, id2) + + if !eng.Healthz() { + t.Fatal("healthz should remain true after runtime failure") + } +} + +func TestStatusReflectsReady(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskMarkReady: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + if eng.Status().Status != "Initializing" { + t.Fatalf("expected Initializing initially, got %q", eng.Status().Status) + } + + _, _ = eng.Submit(Task{Type: TaskMarkReady}) + waitForStatus(t, eng, "Ready") + + if eng.Status().Status != "Ready" { + t.Fatalf("expected Ready after mark-ready, got %q", eng.Status().Status) + } +} + +// --- Result tests --- + +func TestGetResultReturnsCompleted(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + result := waitForResult(t, eng, id) + + if result.ID != id { + t.Fatalf("expected ID %q, got %q", id, result.ID) + } + if result.Type != string(TaskConfigPatch) { + t.Fatalf("expected type %q, got %q", TaskConfigPatch, result.Type) + } + if result.Status != TaskStatusCompleted { + t.Fatalf("expected status %q, got %q", TaskStatusCompleted, result.Status) + } + if result.Error != "" { + t.Fatalf("expected no error, got %q", result.Error) + } +} + +func TestGetResultReturnsFailure(t *testing.T) { + handlerErr := errors.New("handler failed") + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, handlerErr }, + }) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + result := waitForResult(t, eng, id) + + if result.Status != TaskStatusFailed { + t.Fatalf("expected status %q, got %q", TaskStatusFailed, result.Status) + } + if result.Error != handlerErr.Error() { + t.Fatalf("expected error %q, got %q", handlerErr.Error(), result.Error) + } +} + +func TestGetResultReturnsRunning(t *testing.T) { + started := make(chan struct{}) + blocked := make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + close(started) + <-blocked + return nil, nil + }, + }, newTestStore(t)) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + + select { + case <-started: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for task to start") + } + + result := eng.GetResult(id) + if result == nil { + t.Fatal("expected non-nil result for active task") + } + if result.Status != TaskStatusRunning { + t.Fatalf("expected status %q, got %q", TaskStatusRunning, result.Status) + } + if result.CompletedAt != nil { + t.Fatal("expected CompletedAt to be nil for running task") + } + + close(blocked) + completed := waitForResult(t, eng, id) + if completed.Status != TaskStatusCompleted { + t.Fatalf("expected status %q after completion, got %q", TaskStatusCompleted, completed.Status) + } +} + +func TestRecentResultsReturnsAll(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + var ids []string + for i := 0; i < 7; i++ { + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + ids = append(ids, id) + } + for _, id := range ids { + waitForResult(t, eng, id) + } + + results := eng.RecentResults() + if len(results) != 7 { + t.Fatalf("expected 7 results, got %d", len(results)) + } +} + +func TestRecentResultsIncludesActive(t *testing.T) { + started := make(chan struct{}) + blocked := make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + close(started) + <-blocked + return nil, nil + }, + }, newTestStore(t)) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + + select { + case <-started: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for task to start") + } + + results := eng.RecentResults() + if len(results) != 1 { + t.Fatalf("expected 1 result (active), got %d", len(results)) + } + if results[0].ID != id { + t.Fatalf("expected ID %q, got %q", id, results[0].ID) + } + if results[0].Status != TaskStatusRunning { + t.Fatalf("expected status %q, got %q", TaskStatusRunning, results[0].Status) + } + + close(blocked) +} + +func TestRemoveResult(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + waitForResult(t, eng, id) + + if deleted, err := eng.RemoveResult(id); err != nil || !deleted { + t.Fatalf("expected remove to return (true, nil), got (%v, %v)", deleted, err) + } + if deleted, err := eng.RemoveResult(id); err != nil || deleted { + t.Fatalf("second remove should return (false, nil), got (%v, %v)", deleted, err) + } + if eng.GetResult(id) != nil { + t.Fatal("expected nil after removal") + } +} + +func TestRemoveActiveTaskCancels(t *testing.T) { + started := make(chan struct{}) + stopped := make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(ctx context.Context, _ map[string]any) (json.RawMessage, error) { + close(started) + <-ctx.Done() + close(stopped) + return nil, ctx.Err() + }, + }, newTestStore(t)) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + + select { + case <-started: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for task to start") + } + + if deleted, err := eng.RemoveResult(id); err != nil || !deleted { + t.Fatalf("expected remove to return (true, nil), got (%v, %v)", deleted, err) + } + select { + case <-stopped: + case <-time.After(2 * time.Second): + t.Fatal("task goroutine did not observe cancellation after RemoveResult") + } + if eng.GetResult(id) != nil { + t.Fatal("expected nil after removal") + } +} + +// A task that completes on its own must not leak its cancel func in the +// registry — otherwise every finished task pins a context until shutdown. +func TestCancelRegistryClearedOnCompletion(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + waitForResult(t, eng, id) + + // clearCancel runs in the goroutine's defer, just after the result is + // persisted, so poll rather than reading the registry once. + deadline := time.Now().Add(2 * time.Second) + for time.Now().Before(deadline) { + if cancelRegistrySize(eng) == 0 { + return + } + time.Sleep(5 * time.Millisecond) + } + t.Fatalf("cancel func leaked after completion: registry size %d", cancelRegistrySize(eng)) +} + +// Rehydrated tasks must get a per-task cancellable context too, not the engine +// root — else a stale task resumed after a crash is un-cancellable and DELETE +// would orphan its goroutine. +func TestRemoveRehydratedTaskCancels(t *testing.T) { + started := make(chan struct{}) + stopped := make(chan struct{}) + store := newTestStore(t) + const id = "cccccccc-1111-2222-3333-444444444444" + if err := store.Save(&TaskResult{ + ID: id, Type: string(TaskConfigPatch), Status: TaskStatusRunning, Run: 1, + SubmittedAt: time.Now().UTC(), + }); err != nil { + t.Fatalf("seed stale task: %v", err) + } + + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(ctx context.Context, _ map[string]any) (json.RawMessage, error) { + close(started) + <-ctx.Done() + close(stopped) + return nil, ctx.Err() + }, + }, store) + eng.RehydrateStaleTasks() + + select { + case <-started: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for rehydrated task to start") + } + + if deleted, err := eng.RemoveResult(id); err != nil || !deleted { + t.Fatalf("expected remove to return (true, nil), got (%v, %v)", deleted, err) + } + select { + case <-stopped: + case <-time.After(2 * time.Second): + t.Fatal("rehydrated task goroutine did not observe cancellation") + } + if eng.GetResult(id) != nil { + t.Fatal("expected nil after removal") + } +} + +// A superseded registration's late cleanup must not cancel or drop a newer +// registration's entry under the same ID: cleanup is a compare-and-delete on the +// generation the registration minted, so the newer run's live context and entry +// survive. Driving newTaskContext/clearCancel directly makes the interleaving +// deterministic under -race. +func TestClearCancelIgnoresSupersededRun(t *testing.T) { + eng := newTestEngine(t, nil) + const id = "aaaaaaaa-1111-2222-3333-444444444444" + + eng.mu.Lock() + _, gen1 := eng.newTaskContext(id) // first registration + retryCtx, gen2 := eng.newTaskContext(id) // second overwrites the entry + eng.mu.Unlock() + + eng.clearCancel(id, gen1) // the superseded registration's late defer must be a no-op + + if retryCtx.Err() != nil { + t.Fatal("the superseded run's stale clearCancel cancelled the live context") + } + if cancelRegistrySize(eng) != 1 { + t.Fatalf("the live entry must survive the superseded run's cleanup; size %d", cancelRegistrySize(eng)) + } + + eng.clearCancel(id, gen2) // the live registration's own cleanup still cancels and removes it + if retryCtx.Err() == nil { + t.Fatal("the live run's own clearCancel must cancel its context") + } + if cancelRegistrySize(eng) != 0 { + t.Fatalf("the live run's entry must be removed by its own clearCancel; size %d", cancelRegistrySize(eng)) + } +} + +// A resubmit after a delete reuses the row-scoped run number (run resets to 1 +// once the row is gone), so cleanup must key on the never-reused generation, not +// run. When a registration's context is cancelled and its entry removed (as a +// successful RemoveResult does) and a fresh registration then lands under the +// same ID with the same run number before the first's goroutine wakes, the +// first's late cleanup must leave the fresh registration's context and entry +// intact. Driven through the real registry mutation path so the run-number +// collision is genuine, not simulated. +func TestClearCancelIgnoresSupersededRunAfterResubmit(t *testing.T) { + eng := newTestEngine(t, nil) + const id = "aaaaaaaa-1111-2222-3333-444444444444" + + // Registration A. In a live run this is Submit's newTaskContext with run 1. + eng.mu.Lock() + ctxA, genA := eng.newTaskContext(id) + eng.mu.Unlock() + + // RemoveResult cancels A's context and removes its entry on a successful + // delete. A's goroutine has not yet woken to run its deferred clearCancel. + eng.mu.Lock() + if entry, ok := eng.cancels[id]; ok && entry.gen == genA { + entry.cancel() + delete(eng.cancels, id) + } + eng.mu.Unlock() + if ctxA.Err() == nil { + t.Fatal("the delete should have cancelled registration A's context") + } + + // A fresh resubmit lands under the same ID. run would again be 1 here (the + // row is gone), but the generation is strictly newer. + eng.mu.Lock() + ctxB, genB := eng.newTaskContext(id) + eng.mu.Unlock() + if genB <= genA { + t.Fatalf("generation must be strictly increasing: genA=%d genB=%d", genA, genB) + } + + // A finally wakes and runs its deferred cleanup. Under a run-keyed registry + // both A and B carry run 1, so this would match and cancel B; keyed on the + // generation it is a no-op. + eng.clearCancel(id, genA) + + if ctxB.Err() != nil { + t.Fatal("registration A's stale cleanup cancelled the resubmit's live context") + } + if cancelRegistrySize(eng) != 1 { + t.Fatalf("the resubmit's entry must survive A's stale cleanup; size %d", cancelRegistrySize(eng)) + } +} + +// toggleDeleteFailStore makes store.Delete fail on demand so a DELETE can be +// driven through the failure path and then a retry through the success path. +type toggleDeleteFailStore struct { + *SQLiteStore + failDelete atomic.Bool +} + +func (s *toggleDeleteFailStore) Delete(id string) (bool, error) { + if s.failDelete.Load() { + return false, fmt.Errorf("simulated delete failure") + } + return s.SQLiteStore.Delete(id) +} + +// When store.Delete fails, RemoveResult still cancels the goroutine (work stops) +// but surfaces the error and leaves the row 'running' so it stays recoverable — +// a DELETE retry (or rehydration on restart) can still act on it. It must not +// report the row removed, and it must not drop the registry entry, so the row is +// never stranded 'running' with no goroutine that Submit's dedup would refuse to +// re-run. +func TestRemoveResultFailedDeleteDoesNotStrand(t *testing.T) { + inner, err := NewMemoryStore() + if err != nil { + t.Fatalf("memory store: %v", err) + } + t.Cleanup(func() { _ = inner.Close() }) + store := &toggleDeleteFailStore{SQLiteStore: inner} + store.failDelete.Store(true) + + started := make(chan struct{}) + stopped := make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(ctx context.Context, _ map[string]any) (json.RawMessage, error) { + close(started) + <-ctx.Done() + close(stopped) + return nil, ctx.Err() + }, + }, store) + + id, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("submit: %v", err) + } + select { + case <-started: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for task to start") + } + + deleted, err := eng.RemoveResult(id) + if err == nil { + t.Fatal("RemoveResult must surface the store.Delete failure so the caller can retry") + } + if deleted { + t.Fatal("RemoveResult must not report the row removed when store.Delete failed") + } + + // The work is still cancelled — cancellation is unconditional. + select { + case <-stopped: + case <-time.After(2 * time.Second): + t.Fatal("task goroutine did not observe cancellation despite the delete failure") + } + + // The row survives and stays 'running', so it remains actionable. + r := eng.GetResult(id) + if r == nil { + t.Fatal("row must survive a failed delete so it stays recoverable") + } + if r.Status != TaskStatusRunning { + t.Fatalf("row status = %q, want running (recoverable)", r.Status) + } + + // A DELETE retry against a now-healthy store recovers it. + store.failDelete.Store(false) + deleted, err = eng.RemoveResult(id) + if err != nil { + t.Fatalf("DELETE retry: %v", err) + } + if !deleted { + t.Fatal("DELETE retry should report the row removed") + } + if eng.GetResult(id) != nil { + t.Fatal("row should be gone after a successful DELETE retry") + } +} + +// A handler that surfaces cancellation as a non-context.Canceled error must not +// resurrect a Failed row after DELETE removed it: the ctx.Err() guard keys the +// no-op on the cancelled context, not only on a context.Canceled-wrapping error. +func TestRunTaskSyncSuppressesErrorUnderCancellation(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + return nil, errors.New("boom") + }, + }) + + const id = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" + ctx, cancel := context.WithCancel(context.Background()) + cancel() // the task's context is already cancelled, as after RemoveResult + eng.runTaskSync(WithTaskID(ctx, id), id, TaskConfigPatch, eng.handlers[TaskConfigPatch], nil, time.Now().UTC(), 1, 1) + + if r := eng.GetResult(id); r != nil { + t.Fatalf("non-context error under a cancelled ctx must not persist; got %q row", r.Status) + } +} + +// A per-task deadline surfaces as context.DeadlineExceeded, NOT context.Canceled. +// The suppression guard in runTaskSync keys only on Canceled, so a timed-out task +// must fall through to Failed-persistence rather than being stranded 'running'. +func TestRunTaskSyncPersistsFailedOnDeadlineExceeded(t *testing.T) { + cases := []struct { + name string + id string + ctx func(t *testing.T) context.Context + }{ + { + name: "handler-internal deadline with a live task context", + id: "11111111-1111-1111-1111-111111111111", + ctx: func(*testing.T) context.Context { return context.Background() }, + }, + { + name: "task context itself deadline-exceeded", + id: "22222222-2222-2222-2222-222222222222", + ctx: func(t *testing.T) context.Context { + ctx, cancel := context.WithTimeout(context.Background(), time.Nanosecond) + t.Cleanup(cancel) + <-ctx.Done() + return ctx + }, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + return nil, context.DeadlineExceeded + }, + }) + + ctx := WithTaskID(tc.ctx(t), tc.id) + eng.runTaskSync(ctx, tc.id, TaskConfigPatch, eng.handlers[TaskConfigPatch], nil, time.Now().UTC(), 1, 1) + + r := eng.GetResult(tc.id) + if r == nil { + t.Fatal("DeadlineExceeded must persist a row; got nil (wrongly suppressed)") + } + if r.Status != TaskStatusFailed { + t.Fatalf("status = %q, want %q", r.Status, TaskStatusFailed) + } + if !strings.Contains(r.Error, context.DeadlineExceeded.Error()) { + t.Fatalf("error = %q, want it to carry %q", r.Error, context.DeadlineExceeded.Error()) + } + }) + } +} + +// --- Context cancellation --- + +func TestContextCancellationStopsEngine(t *testing.T) { + var mu sync.Mutex + executed := 0 + ctx, cancel := context.WithCancel(context.Background()) + store, _ := NewMemoryStore() + t.Cleanup(func() { store.Close() }) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + mu.Lock() + executed++ + mu.Unlock() + return nil, nil + }, + }, store) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + waitForResult(t, eng, id) + cancel() + + mu.Lock() + if executed != 1 { + t.Fatalf("expected 1 execution, got %d", executed) + } + mu.Unlock() +} + +// --- Long-running task tests --- + +func TestLongRunningTaskCompletion(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + result := waitForResult(t, eng, id) + + if result.Status != TaskStatusCompleted { + t.Fatalf("expected status %q, got %q", TaskStatusCompleted, result.Status) + } +} + +func TestLongRunningTaskFailure(t *testing.T) { + handlerErr := errors.New("fatal crash") + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, handlerErr }, + }) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + result := waitForResult(t, eng, id) + + if result.Status != TaskStatusFailed { + t.Fatalf("expected status %q, got %q", TaskStatusFailed, result.Status) + } + if result.Error != handlerErr.Error() { + t.Fatalf("expected error %q, got %q", handlerErr.Error(), result.Error) + } +} + +func TestLongRunningTaskDoesNotBlockOthers(t *testing.T) { + bgStarted := make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(ctx context.Context, _ map[string]any) (json.RawMessage, error) { + close(bgStarted) + <-ctx.Done() + return nil, ctx.Err() + }, + TaskMarkReady: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }, newTestStore(t)) + + _, _ = eng.Submit(Task{Type: TaskConfigPatch}) + + select { + case <-bgStarted: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for long-running task to start") + } + + id, err := eng.Submit(Task{Type: TaskMarkReady}) + if err != nil { + t.Fatalf("second submit should not be blocked: %v", err) + } + waitForResult(t, eng, id) +} + +// --- Status-aware Submit tests --- + +func TestSubmitReExecutesFailedTask(t *testing.T) { + calls := 0 + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + calls++ + if calls == 1 { + return nil, errors.New("transient failure") + } + return nil, nil + }, + }) + + const taskID = "dddddddd-1111-2222-3333-444444444444" + + // First submit: task fails. + id1, err := eng.Submit(Task{ID: taskID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("first submit: %v", err) + } + r1 := waitForResult(t, eng, id1) + if r1.Status != TaskStatusFailed { + t.Fatalf("expected failed, got %s", r1.Status) + } + if r1.Run != 1 { + t.Fatalf("expected run=1, got %d", r1.Run) + } + + // Second submit with same ID: should re-execute. + id2, err := eng.Submit(Task{ID: taskID, Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("second submit: %v", err) + } + if id2 != id1 { + t.Fatalf("expected same ID, got %q and %q", id1, id2) + } + + r2 := waitForResult(t, eng, id2) + if r2.Status != TaskStatusCompleted { + t.Fatalf("expected completed on retry, got %s", r2.Status) + } + if r2.Run != 2 { + t.Fatalf("expected run=2, got %d", r2.Run) + } + if calls != 2 { + t.Fatalf("expected handler called twice, got %d", calls) + } +} + +func TestSubmitReExecutesFailedTaskThatFailsAgain(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + return nil, errors.New("persistent failure") + }, + }) + + const taskID = "eeeeeeee-1111-2222-3333-444444444444" + + id, _ := eng.Submit(Task{ID: taskID, Type: TaskConfigPatch}) + waitForResult(t, eng, id) + + // Re-submit: still fails. + eng.Submit(Task{ID: taskID, Type: TaskConfigPatch}) + r := waitForResult(t, eng, id) + + if r.Status != TaskStatusFailed { + t.Fatalf("expected failed, got %s", r.Status) + } + if r.Run != 2 { + t.Fatalf("expected run=2, got %d", r.Run) + } +} + +func TestSubmitDoesNotIncrementRunOnRehydration(t *testing.T) { + // Create a store with a stale running task (simulates pod crash). + store := newTestStore(t) + now := time.Now().UTC() + _ = store.Save(&TaskResult{ + ID: "ffffffff-1111-2222-3333-444444444444", + Type: string(TaskConfigPatch), + Status: TaskStatusRunning, + Run: 1, + SubmittedAt: now, + }) + + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }, store) + eng.RehydrateStaleTasks() + + r := waitForResult(t, eng, "ffffffff-1111-2222-3333-444444444444") + if r.Run != 1 { + t.Fatalf("expected run=1 after rehydration (not incremented), got %d", r.Run) + } + if r.Status != TaskStatusCompleted { + t.Fatalf("expected completed after rehydration, got %s", r.Status) + } +} + +func TestSubmitConcurrentSameFailedID(t *testing.T) { + started := make(chan struct{}, 2) + blocked := make(chan struct{}) + var callCount atomic.Int32 + + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + store := newTestStore(t) + eng := NewEngine(ctx, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + callCount.Add(1) + started <- struct{}{} + <-blocked + return nil, nil + }, + }, store) + + const taskID = "11111111-2222-3333-4444-555555555555" + + // Seed a failed task directly in the store. + now := time.Now().UTC() + _ = store.Save(&TaskResult{ + ID: taskID, + Type: string(TaskConfigPatch), + Status: TaskStatusFailed, + Run: 1, + Error: "failed", + SubmittedAt: now, + CompletedAt: &now, + }) + + // Two concurrent submits of the same failed ID. + var wg sync.WaitGroup + wg.Add(2) + for i := 0; i < 2; i++ { + go func() { + defer wg.Done() + eng.Submit(Task{ID: taskID, Type: TaskConfigPatch}) + }() + } + wg.Wait() + + // Unblock the handler and wait for completion. + close(blocked) + waitForResult(t, eng, taskID) + + // The mutex serializes Submit: the first sees "failed" and re-executes, + // the second sees "running" (from the first's Save) and no-ops. + if c := callCount.Load(); c != 1 { + t.Fatalf("expected exactly 1 re-execution, got %d", c) + } +} + +func TestSubmitRunFieldOnFirstSubmit(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + + id, _ := eng.Submit(Task{Type: TaskConfigPatch}) + r := waitForResult(t, eng, id) + + if r.Run != 1 { + t.Fatalf("expected run=1 on first submit, got %d", r.Run) + } +} + +func TestTaskErrorProducesRichErrorString(t *testing.T) { + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + return nil, &TaskError{ + Task: "config-patch", + Operation: "S3", + Message: "bucket not found", + Hint: "check SEI_SNAPSHOT_BUCKET", + } + }, + }) + + id, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("submit: %v", err) + } + + r := waitForResult(t, eng, id) + if r.Status != TaskStatusFailed { + t.Fatalf("expected Failed, got %s", r.Status) + } + if !strings.Contains(r.Error, "config-patch") { + t.Errorf("error should contain task name, got: %s", r.Error) + } + if !strings.Contains(r.Error, "bucket not found") { + t.Errorf("error should contain message, got: %s", r.Error) + } + if !strings.Contains(r.Error, "hint:") { + t.Errorf("error should contain hint, got: %s", r.Error) + } +} + +func TestSubmitHandlerPanicBecomesFailedTask(t *testing.T) { + before := testutil.ToFloat64(taskPanics.WithLabelValues(string(TaskConfigPatch))) + + eng := newTestEngine(t, map[TaskType]TaskHandler{ + TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + panic("kaboom") + }, + }) + + id, err := eng.Submit(Task{Type: TaskConfigPatch}) + if err != nil { + t.Fatalf("submit: %v", err) + } + + r := waitForResult(t, eng, id) + if r.Status != TaskStatusFailed { + t.Fatalf("panicking handler should produce Failed, got %s", r.Status) + } + if !strings.Contains(r.Error, "panicked") || !strings.Contains(r.Error, "kaboom") { + t.Errorf("error should describe the panic, got: %s", r.Error) + } + + after := testutil.ToFloat64(taskPanics.WithLabelValues(string(TaskConfigPatch))) + if after != before+1 { + t.Errorf("taskPanics delta = %v, want 1", after-before) + } +} diff --git a/sidecar/engine/mark_not_ready_test.go b/sidecar/engine/mark_not_ready_test.go new file mode 100644 index 00000000..881dec4d --- /dev/null +++ b/sidecar/engine/mark_not_ready_test.go @@ -0,0 +1,433 @@ +package engine + +import ( + "context" + "encoding/json" + "fmt" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/google/uuid" +) + +// noopHandler is a result-less handler; the engine's completion hook is what +// flips the readiness flag for mark-ready / mark-not-ready. +func noopHandler(context.Context, map[string]any) (json.RawMessage, error) { return nil, nil } + +func engineOver(t *testing.T, store ResultStore, handlers map[TaskType]TaskHandler) *Engine { + t.Helper() + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + return NewEngine(ctx, handlers, store) +} + +// seedStrandedAt writes a task record stuck in "running" at the given submit +// time — the state an ungraceful kill leaves between Submit's Save(running) and +// the completing Save. RehydrateStaleTasks re-runs exactly these. +func seedStrandedAt(t *testing.T, store ResultStore, taskType TaskType, submittedAt time.Time) string { + t.Helper() + id := uuid.New().String() + if err := store.Save(&TaskResult{ + ID: id, + Type: string(taskType), + Status: TaskStatusRunning, + Run: 1, + SubmittedAt: submittedAt, + }); err != nil { + t.Fatalf("seeding stranded %s: %v", taskType, err) + } + return id +} + +func seedStrandedMarkReady(t *testing.T, store ResultStore) string { + return seedStrandedAt(t, store, TaskMarkReady, time.Now().UTC()) +} + +func seedStrandedMarkNotReady(t *testing.T, store ResultStore) string { + return seedStrandedAt(t, store, TaskMarkNotReady, time.Now().UTC()) +} + +func countMarkReady(t *testing.T, store ResultStore) int { + t.Helper() + results, err := store.List(100) + if err != nil { + t.Fatalf("list: %v", err) + } + n := 0 + for _, r := range results { + if r.Type == string(TaskMarkReady) { + n++ + } + } + return n +} + +func ensureStaysNotReady(t *testing.T, eng *Engine) { + t.Helper() + deadline := time.Now().Add(200 * time.Millisecond) + for time.Now().Before(deadline) { + if eng.Healthz() { + t.Fatal("engine became ready after rehydration: a stranded mark-ready was not purged") + } + time.Sleep(5 * time.Millisecond) + } +} + +// Control: demonstrates the release path the purge exists to close. A stranded +// running mark-ready rehydrated on restart re-marks the engine ready — which, +// mid-hold, would release seid onto a wiped data directory. +func TestRehydrate_StrandedMarkReadyReleasesWithoutPurge(t *testing.T) { + store, dbPath := newFileStore(t) + seedStrandedMarkReady(t, store) + store = reopenStore(t, store, dbPath) // simulate a process restart + + eng := engineOver(t, store, map[TaskType]TaskHandler{TaskMarkReady: noopHandler}) + eng.RehydrateStaleTasks() + + waitForHealthz(t, eng) +} + +// Guard: with the mark-ready records purged (what mark-not-ready's handler +// does) first, the restart's rehydration finds nothing to run and the engine +// stays not-ready across the restart. +func TestMarkNotReadyPurge_PreventsRehydrateRelease(t *testing.T) { + store, dbPath := newFileStore(t) + seedStrandedMarkReady(t, store) + + n, err := store.DeleteByType(string(TaskMarkReady)) + if err != nil || n != 1 { + t.Fatalf("purge: n=%d err=%v (want 1, nil)", n, err) + } + + store = reopenStore(t, store, dbPath) // simulate a process restart + eng := engineOver(t, store, map[TaskType]TaskHandler{TaskMarkReady: noopHandler}) + eng.RehydrateStaleTasks() + + ensureStaysNotReady(t, eng) +} + +// End-to-end through Submit: mark-ready makes the engine ready; mark-not-ready +// purges every mark-ready record and then the engine flips ready false. The +// purge (handler) completes before the flip (completion hook), so observing +// both post-conditions after the result lands proves purge-then-flip ordering. +func TestMarkNotReady_PurgesThenFlipsReadyFalse(t *testing.T) { + store, err := NewMemoryStore() + if err != nil { + t.Fatalf("memory store: %v", err) + } + t.Cleanup(func() { _ = store.Close() }) + + purge := func(context.Context, map[string]any) (json.RawMessage, error) { + if _, err := store.DeleteByType(string(TaskMarkReady)); err != nil { + return nil, err + } + return nil, nil + } + eng := engineOver(t, store, map[TaskType]TaskHandler{ + TaskMarkReady: noopHandler, + TaskMarkNotReady: purge, + }) + + id, err := eng.Submit(Task{Type: TaskMarkReady}) + if err != nil { + t.Fatalf("submit mark-ready: %v", err) + } + waitForResult(t, eng, id) + waitForHealthz(t, eng) + + // A second stranded mark-ready alongside the completed one, so the purge + // has to clear more than a single record. + seedStrandedMarkReady(t, store) + if countMarkReady(t, store) == 0 { + t.Fatal("expected mark-ready records present before purge") + } + + id2, err := eng.Submit(Task{Type: TaskMarkNotReady}) + if err != nil { + t.Fatalf("submit mark-not-ready: %v", err) + } + waitForResult(t, eng, id2) + + if eng.Healthz() { + t.Fatal("expected readiness flipped false after mark-not-ready") + } + if n := countMarkReady(t, store); n != 0 { + t.Fatalf("expected all mark-ready records purged, got %d", n) + } +} + +// Both a mark-ready and a mark-not-ready stranded running by a crash in the +// [Submit-saved-running … purge-commit] window. RehydrateStaleTasks must run +// the hold synchronously first, so the flag ends deterministically false (held) +// with no dependence on goroutine scheduling. Looped to stress the ordering +// under -race, where the pre-fix concurrent dispatch would flake. +func TestRehydrate_StrandedHoldWinsDeterministically(t *testing.T) { + for i := 0; i < 50; i++ { + store, dbPath := newFileStore(t) + seedStrandedMarkReady(t, store) + seedStrandedMarkNotReady(t, store) + store = reopenStore(t, store, dbPath) // simulate a process restart + + purge := func(context.Context, map[string]any) (json.RawMessage, error) { + _, err := store.DeleteByType(string(TaskMarkReady)) + return nil, err + } + eng := engineOver(t, store, map[TaskType]TaskHandler{ + TaskMarkReady: noopHandler, + TaskMarkNotReady: purge, + }) + eng.RehydrateStaleTasks() + + // Synchronous hold-first: the mark-not-ready purged the stranded + // mark-ready before the re-list, so nothing dispatched can flip ready. + if eng.Healthz() { + t.Fatalf("iteration %d: engine ready after rehydration — hold released", i) + } + } +} + +// On the live path the engine does NOT serialize a controller-submitted +// mark-ready against an active hold; that mutual exclusion is the controller's +// (reapproval suppression via adoptedWorkflow). This test asserts only that the +// engine survives the concurrency with no torn store state — the final readiness +// value is deliberately unasserted, being the controller's contract. +func TestLivePath_ConcurrentMarkReadyDuringHold_NoTornState(t *testing.T) { + store, err := NewMemoryStore() + if err != nil { + t.Fatalf("memory store: %v", err) + } + t.Cleanup(func() { _ = store.Close() }) + + purge := func(context.Context, map[string]any) (json.RawMessage, error) { + _, e := store.DeleteByType(string(TaskMarkReady)) + return nil, e + } + eng := engineOver(t, store, map[TaskType]TaskHandler{ + TaskMarkReady: noopHandler, + TaskMarkNotReady: purge, + }) + + // Become ready first. + id0, err := eng.Submit(Task{Type: TaskMarkReady}) + if err != nil { + t.Fatalf("submit initial mark-ready: %v", err) + } + waitForResult(t, eng, id0) + waitForHealthz(t, eng) + + // Race a hold against a fresh mark-ready. + holdID := uuid.New().String() + readyID := uuid.New().String() + var wg sync.WaitGroup + wg.Add(2) + go func() { defer wg.Done(); _, _ = eng.Submit(Task{ID: holdID, Type: TaskMarkNotReady}) }() + go func() { defer wg.Done(); _, _ = eng.Submit(Task{ID: readyID, Type: TaskMarkReady}) }() + wg.Wait() + + // The hold's own record is never self-purged, so it reaches terminal. + waitForResult(t, eng, holdID) + + // Drain: wait until no row is left running (the concurrent mark-ready either + // completed or was purged mid-flight — both terminal for our purposes), then + // a brief settle for any in-flight completed-save to land. + deadline := time.Now().Add(2 * time.Second) + for time.Now().Before(deadline) { + results, err := store.List(100) + if err != nil { + t.Fatalf("list during drain: %v", err) + } + if countRunning(results) == 0 { + break + } + time.Sleep(5 * time.Millisecond) + } + time.Sleep(20 * time.Millisecond) + + // No torn state: the store is queryable and no row is stuck running. + results, err := store.List(100) + if err != nil { + t.Fatalf("list after race: %v", err) + } + if n := countRunning(results); n != 0 { + t.Errorf("expected no running rows after concurrent hold/ready, got %d", n) + } + + // Final readiness is the controller's contract, not the engine's — do not + // assert it. Reading it just confirms the atomic is well-defined either way. + _ = eng.Healthz() +} + +func countRunning(results []TaskResult) int { + n := 0 + for _, r := range results { + if r.Status == TaskStatusRunning { + n++ + } + } + return n +} + +// failingPurgeStore is a ResultStore whose DeleteByType always fails, modelling +// a broken store during a hold's purge. Every other operation delegates. +type failingPurgeStore struct { + *SQLiteStore +} + +func (failingPurgeStore) DeleteByType(string) (int, error) { + return 0, fmt.Errorf("simulated purge failure") +} + +// When the synchronous hold's purge fails, the stranded mark-ready survives the +// purge — but the durable supersession rule must still refuse to rehydrate it +// (a newer mark-not-ready record exists) and mark it Failed ("superseded by +// hold") so a controller poll terminates. Readiness stays false. +func TestRehydrate_FailedPurgeSupersedesStrandedMarkReady(t *testing.T) { + inner, err := NewMemoryStore() + if err != nil { + t.Fatalf("memory store: %v", err) + } + t.Cleanup(func() { _ = inner.Close() }) + store := failingPurgeStore{inner} + + base := time.Now().UTC() + readyID := seedStrandedAt(t, store, TaskMarkReady, base) + seedStrandedAt(t, store, TaskMarkNotReady, base.Add(time.Second)) // hold is newer + + purge := func(context.Context, map[string]any) (json.RawMessage, error) { + _, err := store.DeleteByType(string(TaskMarkReady)) + return nil, err // fails + } + eng := engineOver(t, store, map[TaskType]TaskHandler{ + TaskMarkReady: noopHandler, + TaskMarkNotReady: purge, + }) + eng.RehydrateStaleTasks() + + if eng.Healthz() { + t.Fatal("engine ready after rehydration: stranded mark-ready was dispatched despite a newer hold") + } + r, err := store.Get(readyID) + if err != nil { + t.Fatalf("get stranded mark-ready: %v", err) + } + if r == nil { + t.Fatal("stranded mark-ready was removed; a failed purge should leave it recorded as superseded") + } + if r.Status != TaskStatusFailed { + t.Errorf("stranded mark-ready status = %q, want failed (superseded)", r.Status) + } + if r.Error != "superseded by hold" { + t.Errorf("stranded mark-ready error = %q, want \"superseded by hold\"", r.Error) + } + ensureStaysNotReady(t, eng) +} + +// The Bugbot residual: a transient purge failure persists the mark-not-ready +// Failed (non-stale), so the NEXT boot sees only the still-running mark-ready +// and no stale hold. The durable supersession rule (keyed on the persisted hold +// record, any status) must keep the node held across BOTH restarts. +func TestRehydrate_HoldSurvivesFailedPurgeAcrossTwoRestarts(t *testing.T) { + for i := 0; i < 30; i++ { + store, dbPath := newFileStore(t) + base := time.Now().UTC() + readyID := seedStrandedAt(t, store, TaskMarkReady, base) // earlier lifecycle's release + seedStrandedAt(t, store, TaskMarkNotReady, base.Add(time.Second)) // the newer hold + + // Restart N: the purge fails transiently; runTaskSync's failure Save + // persists the mark-not-ready Failed, making it non-stale next boot. + store = reopenStore(t, store, dbPath) + failPurge := func(context.Context, map[string]any) (json.RawMessage, error) { + return nil, fmt.Errorf("transient purge failure") + } + engN := engineOver(t, store, map[TaskType]TaskHandler{ + TaskMarkReady: noopHandler, + TaskMarkNotReady: failPurge, + }) + engN.RehydrateStaleTasks() + if engN.Healthz() { + t.Fatalf("iteration %d restart N: hold released", i) + } + + // Restart N+1: no stale mark-not-ready remains (it is Failed); only the + // still-running mark-ready. The durable rule must keep the node held. + store = reopenStore(t, store, dbPath) + okPurge := func(context.Context, map[string]any) (json.RawMessage, error) { + _, err := store.DeleteByType(string(TaskMarkReady)) + return nil, err + } + engN1 := engineOver(t, store, map[TaskType]TaskHandler{ + TaskMarkReady: noopHandler, + TaskMarkNotReady: okPurge, + }) + engN1.RehydrateStaleTasks() + + if engN1.Healthz() { + t.Fatalf("iteration %d restart N+1: hold escaped across two restarts", i) + } + r, err := store.Get(readyID) + if err != nil { + t.Fatalf("iteration %d: get mark-ready: %v", i, err) + } + if r == nil || r.Status != TaskStatusFailed { + t.Fatalf("iteration %d: mark-ready not terminal-superseded: %+v", i, r) + } + } +} + +// relistFailStore fails the second (post-purge) ListStaleTasks call, modelling a +// store that breaks between the hold's purge and the dispatch loop. +type relistFailStore struct { + *SQLiteStore + listCalls int +} + +func (s *relistFailStore) ListStaleTasks() ([]TaskResult, error) { + s.listCalls++ + if s.listCalls >= 2 { + return nil, fmt.Errorf("simulated re-list failure") + } + return s.SQLiteStore.ListStaleTasks() +} + +// A re-list error after the synchronous hold must abort rehydration before the +// dispatch loop: no stale task is dispatched and readiness stays false. +func TestRehydrate_RelistErrorAbortsDispatch(t *testing.T) { + inner, err := NewMemoryStore() + if err != nil { + t.Fatalf("memory store: %v", err) + } + t.Cleanup(func() { _ = inner.Close() }) + store := &relistFailStore{SQLiteStore: inner} + + seedStrandedMarkNotReady(t, store) + // A third stale task that loop 2 would dispatch if it were reached. + otherID := uuid.New().String() + if err := store.Save(&TaskResult{ + ID: otherID, + Type: string(TaskConfigPatch), + Status: TaskStatusRunning, + Run: 1, + SubmittedAt: time.Now().UTC(), + }); err != nil { + t.Fatalf("seeding other stale task: %v", err) + } + + var otherRan atomic.Bool + handlers := map[TaskType]TaskHandler{ + TaskMarkNotReady: func(context.Context, map[string]any) (json.RawMessage, error) { return nil, nil }, + TaskConfigPatch: func(context.Context, map[string]any) (json.RawMessage, error) { + otherRan.Store(true) + return nil, nil + }, + } + eng := engineOver(t, store, handlers) + eng.RehydrateStaleTasks() + + ensureStaysNotReady(t, eng) + time.Sleep(50 * time.Millisecond) // give any erroneous dispatch a chance to run + if otherRan.Load() { + t.Fatal("a stale task was dispatched despite the re-list error") + } +} diff --git a/sidecar/engine/metrics.go b/sidecar/engine/metrics.go new file mode 100644 index 00000000..ff63ab9c --- /dev/null +++ b/sidecar/engine/metrics.go @@ -0,0 +1,54 @@ +package engine + +import ( + "github.com/prometheus/client_golang/prometheus" +) + +var ( + // taskDuration records the wall-clock execution time of the task handler + // (measured inside the goroutine, not submission latency). + taskDuration = prometheus.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "seictl_task_execution_duration_seconds", + Help: "Wall-clock execution time of task handlers in seconds (measured inside the goroutine, not submission latency).", + Buckets: prometheus.DefBuckets, + }, + []string{"type", "status"}, + ) + + // taskSubmissions counts the total number of task submissions. + taskSubmissions = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "seictl_task_submissions_total", + Help: "Total number of tasks submitted.", + }, + []string{"type"}, + ) + + // taskFailures counts the total number of task failures. + taskFailures = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "seictl_task_failures_total", + Help: "Total number of tasks that failed.", + }, + []string{"type"}, + ) + + // taskPanics counts handler panics recovered by the engine. A non-zero + // value means a handler crashed and was converted to a failed task rather + // than taking the sidecar down; it also increments taskFailures. + taskPanics = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "seictl_task_panics_total", + Help: "Total number of task handler panics recovered by the engine.", + }, + []string{"type"}, + ) +) + +func init() { + prometheus.MustRegister(taskDuration) + prometheus.MustRegister(taskSubmissions) + prometheus.MustRegister(taskFailures) + prometheus.MustRegister(taskPanics) +} diff --git a/sidecar/engine/sqlite_migrations.go b/sidecar/engine/sqlite_migrations.go new file mode 100644 index 00000000..2db1f2e0 --- /dev/null +++ b/sidecar/engine/sqlite_migrations.go @@ -0,0 +1,151 @@ +package engine + +import "database/sql" + +// migrate runs pending schema migrations. Each version is wrapped in an +// explicit transaction so that DDL and the user_version bump are atomic. +func migrate(db *sql.DB) error { + var version int + _ = db.QueryRow("PRAGMA user_version").Scan(&version) + + if version < 1 { + tx, err := db.Begin() + if err != nil { + return err + } + defer tx.Rollback() + + if _, err := tx.Exec(` + CREATE TABLE IF NOT EXISTS task_results ( + id TEXT PRIMARY KEY, + type TEXT NOT NULL, + status TEXT NOT NULL, + params TEXT, + schedule TEXT, + error TEXT NOT NULL DEFAULT '', + submitted_at TEXT NOT NULL, + completed_at TEXT, + next_run_at TEXT + ); + CREATE INDEX IF NOT EXISTS idx_task_results_submitted_at + ON task_results (submitted_at DESC); + CREATE INDEX IF NOT EXISTS idx_task_results_schedule + ON task_results (next_run_at) WHERE schedule IS NOT NULL; + `); err != nil { + return err + } + + if _, err := tx.Exec("PRAGMA user_version = 1"); err != nil { + return err + } + + if err := tx.Commit(); err != nil { + return err + } + } + + if version < 2 { + tx, err := db.Begin() + if err != nil { + return err + } + defer tx.Rollback() + + if _, err := tx.Exec(` + DROP INDEX IF EXISTS idx_task_results_schedule; + ALTER TABLE task_results DROP COLUMN schedule; + ALTER TABLE task_results DROP COLUMN next_run_at; + `); err != nil { + return err + } + + if _, err := tx.Exec("PRAGMA user_version = 2"); err != nil { + return err + } + + if err := tx.Commit(); err != nil { + return err + } + } + + if version < 3 { + tx, err := db.Begin() + if err != nil { + return err + } + defer tx.Rollback() + + if _, err := tx.Exec(` + ALTER TABLE task_results ADD COLUMN run INTEGER NOT NULL DEFAULT 1; + `); err != nil { + return err + } + + if _, err := tx.Exec("PRAGMA user_version = 3"); err != nil { + return err + } + + if err := tx.Commit(); err != nil { + return err + } + } + + if version < 4 { + tx, err := db.Begin() + if err != nil { + return err + } + defer tx.Rollback() + + // result holds a handler's structured output as raw JSON; NULL + // for the common case of a handler that emits no result. + if _, err := tx.Exec(` + ALTER TABLE task_results ADD COLUMN result TEXT; + `); err != nil { + return err + } + + if _, err := tx.Exec("PRAGMA user_version = 4"); err != nil { + return err + } + + if err := tx.Commit(); err != nil { + return err + } + } + + if version < 5 { + tx, err := db.Begin() + if err != nil { + return err + } + defer tx.Rollback() + + // tx_markers: pre-broadcast signed-tx bytes so a crashed sign-tx task + // re-adopts the identical tx instead of re-signing. Engine-owned. A + // pre-v5 downgrade leaves markers unread (the guard degrades). + if _, err := tx.Exec(` + CREATE TABLE IF NOT EXISTS tx_markers ( + task_id TEXT PRIMARY KEY, + tx_hash TEXT NOT NULL, + tx_bytes BLOB NOT NULL, + account_number INTEGER NOT NULL, + sequence INTEGER NOT NULL, + chain_id TEXT NOT NULL, + created_at TEXT NOT NULL + ); + `); err != nil { + return err + } + + if _, err := tx.Exec("PRAGMA user_version = 5"); err != nil { + return err + } + + if err := tx.Commit(); err != nil { + return err + } + } + + return nil +} diff --git a/sidecar/engine/sqlite_store.go b/sidecar/engine/sqlite_store.go new file mode 100644 index 00000000..bfb84b0d --- /dev/null +++ b/sidecar/engine/sqlite_store.go @@ -0,0 +1,273 @@ +package engine + +import ( + "database/sql" + "encoding/json" + "errors" + "fmt" + "time" + + _ "modernc.org/sqlite" +) + +// SQLiteStore persists task results in a SQLite database. +type SQLiteStore struct { + db *sql.DB +} + +// NewSQLiteStore opens (or creates) a SQLite database at dbPath and runs +// any pending schema migrations. The file is opened in WAL mode with +// pragmas tuned for a single-writer sidecar workload. +// +// The database file must reside on a local or block-device-backed +// filesystem (e.g. EBS, GCE PD, local SSD). WAL mode is unsafe on +// NFS-backed volumes (EFS, Azure Files, CephFS over NFS) because they +// do not support the POSIX byte-range locks that SQLite requires for +// the shared-memory (-shm) file. +func NewSQLiteStore(dbPath string) (*SQLiteStore, error) { + return openStore(dbPath) +} + +// NewMemoryStore returns a SQLiteStore backed by an in-memory SQLite +// database (the ":memory:" DSN). The database exists only for the +// lifetime of the returned store — nothing is written to disk. Useful +// for tests and non-sidecar CLI commands. +func NewMemoryStore() (*SQLiteStore, error) { + return openStore(":memory:") +} + +func openStore(dsn string) (*SQLiteStore, error) { + db, err := sql.Open("sqlite", dsn) + if err != nil { + return nil, fmt.Errorf("open sqlite: %w", err) + } + db.SetMaxOpenConns(1) + + for _, pragma := range []string{ + "PRAGMA journal_mode=WAL", + "PRAGMA busy_timeout=5000", + "PRAGMA synchronous=NORMAL", + } { + if _, err := db.Exec(pragma); err != nil { + db.Close() + return nil, fmt.Errorf("%s: %w", pragma, err) + } + } + + if err := migrate(db); err != nil { + db.Close() + return nil, fmt.Errorf("migrate: %w", err) + } + + return &SQLiteStore{db: db}, nil +} + +func (s *SQLiteStore) Save(r *TaskResult) error { + params, err := json.Marshal(r.Params) + if err != nil { + return fmt.Errorf("marshal params: %w", err) + } + + _, err = s.db.Exec(` + INSERT OR REPLACE INTO task_results + (id, type, status, run, params, result, error, submitted_at, completed_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`, + r.ID, + r.Type, + string(r.Status), + r.Run, + string(params), + nullableRawJSON(r.Result), + r.Error, + r.SubmittedAt.UTC().Format(time.RFC3339Nano), + formatNullableTime(r.CompletedAt), + ) + return err +} + +func (s *SQLiteStore) Get(id string) (*TaskResult, error) { + row := s.db.QueryRow(selectColumns+` WHERE id = ?`, id) + r, err := scanTaskResult(row) + if errors.Is(err, sql.ErrNoRows) { + return nil, nil + } + if err != nil { + return nil, err + } + return r, nil +} + +func (s *SQLiteStore) List(limit int) ([]TaskResult, error) { + return s.queryMany(selectColumns+` ORDER BY submitted_at DESC LIMIT ?`, limit) +} + +func (s *SQLiteStore) ListStaleTasks() ([]TaskResult, error) { + return s.queryMany(selectColumns+` WHERE status = ?`, string(TaskStatusRunning)) +} + +func (s *SQLiteStore) Delete(id string) (bool, error) { + res, err := s.db.Exec("DELETE FROM task_results WHERE id = ?", id) + if err != nil { + return false, err + } + n, _ := res.RowsAffected() + return n > 0, nil +} + +func (s *SQLiteStore) DeleteByType(taskType string) (int, error) { + res, err := s.db.Exec("DELETE FROM task_results WHERE type = ?", taskType) + if err != nil { + return 0, err + } + n, _ := res.RowsAffected() + return int(n), nil +} + +func (s *SQLiteStore) LatestByType(taskType string) (*TaskResult, error) { + row := s.db.QueryRow(selectColumns+` WHERE type = ? ORDER BY submitted_at DESC LIMIT 1`, taskType) + r, err := scanTaskResult(row) + if errors.Is(err, sql.ErrNoRows) { + return nil, nil + } + if err != nil { + return nil, err + } + return r, nil +} + +// SaveTxMarker persists a pre-broadcast marker and fsyncs it (via checkpoint, +// since the store runs synchronous=NORMAL) before returning, so it survives a +// crash. Callers MUST let it return before broadcasting. +func (s *SQLiteStore) SaveTxMarker(m *TxMarker) error { + if _, err := s.db.Exec(` + INSERT OR REPLACE INTO tx_markers + (task_id, tx_hash, tx_bytes, account_number, sequence, chain_id, created_at) + VALUES (?, ?, ?, ?, ?, ?, ?)`, + m.TaskID, m.TxHash, m.TxBytes, m.AccountNumber, m.Sequence, m.ChainID, + time.Now().UTC().Format(time.RFC3339Nano), + ); err != nil { + return err + } + // Ignoring FULL's busy row is safe only under SetMaxOpenConns(1) (no + // concurrent readers); if that's raised, use TRUNCATE or assert busy==0. + if _, err := s.db.Exec("PRAGMA wal_checkpoint(FULL)"); err != nil { + return fmt.Errorf("checkpoint tx marker: %w", err) + } + return nil +} + +func (s *SQLiteStore) GetTxMarker(taskID string) (*TxMarker, error) { + row := s.db.QueryRow( + `SELECT task_id, tx_hash, tx_bytes, account_number, sequence, chain_id + FROM tx_markers WHERE task_id = ?`, taskID) + var m TxMarker + err := row.Scan(&m.TaskID, &m.TxHash, &m.TxBytes, &m.AccountNumber, &m.Sequence, &m.ChainID) + if errors.Is(err, sql.ErrNoRows) { + return nil, nil + } + if err != nil { + return nil, err + } + return &m, nil +} + +func (s *SQLiteStore) Ping() error { + var n int + return s.db.QueryRow("SELECT 1").Scan(&n) +} + +func (s *SQLiteStore) Close() error { + return s.db.Close() +} + +// --- query helpers --- + +const selectColumns = ` + SELECT id, type, status, run, params, result, error, submitted_at, completed_at + FROM task_results` + +// queryMany executes a query and scans all rows into TaskResults. +func (s *SQLiteStore) queryMany(query string, args ...any) ([]TaskResult, error) { + rows, err := s.db.Query(query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + + var results []TaskResult + for rows.Next() { + r, err := scanTaskResult(rows) + if err != nil { + return nil, err + } + results = append(results, *r) + } + return results, rows.Err() +} + +// rowScanner abstracts *sql.Row and *sql.Rows for shared scan logic. +type rowScanner interface { + Scan(dest ...any) error +} + +func scanTaskResult(s rowScanner) (*TaskResult, error) { + var ( + r TaskResult + status string + paramsJSON string + resultJSON sql.NullString + submittedAt string + completedAt sql.NullString + ) + + if err := s.Scan( + &r.ID, &r.Type, &status, &r.Run, ¶msJSON, &resultJSON, + &r.Error, &submittedAt, &completedAt, + ); err != nil { + return nil, err + } + + r.Status = TaskStatus(status) + + if paramsJSON != "" { + if err := json.Unmarshal([]byte(paramsJSON), &r.Params); err != nil { + return nil, fmt.Errorf("unmarshal params: %w", err) + } + } + + if resultJSON.Valid && resultJSON.String != "" { + r.Result = json.RawMessage(resultJSON.String) + } + + t, err := time.Parse(time.RFC3339Nano, submittedAt) + if err != nil { + return nil, fmt.Errorf("parse submitted_at: %w", err) + } + r.SubmittedAt = t + + if completedAt.Valid { + t, err := time.Parse(time.RFC3339Nano, completedAt.String) + if err != nil { + return nil, fmt.Errorf("parse completed_at: %w", err) + } + r.CompletedAt = &t + } + + return &r, nil +} + +func formatNullableTime(t *time.Time) any { + if t == nil { + return nil + } + return t.UTC().Format(time.RFC3339Nano) +} + +// nullableRawJSON binds a result payload as SQL NULL when empty so the +// common no-result case stores NULL rather than an empty string. +func nullableRawJSON(r json.RawMessage) any { + if len(r) == 0 { + return nil + } + return string(r) +} diff --git a/sidecar/engine/sqlite_store_test.go b/sidecar/engine/sqlite_store_test.go new file mode 100644 index 00000000..fd232298 --- /dev/null +++ b/sidecar/engine/sqlite_store_test.go @@ -0,0 +1,389 @@ +package engine + +import ( + "encoding/json" + "testing" + "time" +) + +func newTestStore(t *testing.T) *SQLiteStore { + t.Helper() + s, err := NewMemoryStore() + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { s.Close() }) + return s +} + +func TestStoreSaveAndGet(t *testing.T) { + s := newTestStore(t) + now := time.Now().Truncate(time.Nanosecond) + completed := now.Add(time.Second) + + r := &TaskResult{ + ID: "aaaaaaaa-1111-2222-3333-444444444444", + Type: "config-patch", + Status: TaskStatusCompleted, + Params: map[string]any{"file": "config.toml", "nested": map[string]any{"key": "val"}}, + Error: "", + SubmittedAt: now, + CompletedAt: &completed, + } + + if err := s.Save(r); err != nil { + t.Fatalf("save: %v", err) + } + + got, err := s.Get(r.ID) + if err != nil { + t.Fatalf("get: %v", err) + } + if got == nil { + t.Fatal("expected non-nil result") + } + if got.ID != r.ID { + t.Fatalf("ID = %q, want %q", got.ID, r.ID) + } + if got.Type != r.Type { + t.Fatalf("Type = %q, want %q", got.Type, r.Type) + } + if got.Status != r.Status { + t.Fatalf("Status = %q, want %q", got.Status, r.Status) + } + if got.Error != r.Error { + t.Fatalf("Error = %q, want %q", got.Error, r.Error) + } + if got.CompletedAt == nil { + t.Fatal("expected non-nil CompletedAt") + } + + // Verify nested params survived JSON round-trip. + nested, ok := got.Params["nested"].(map[string]any) + if !ok { + t.Fatalf("expected nested map, got %T", got.Params["nested"]) + } + if nested["key"] != "val" { + t.Fatalf("nested.key = %q, want %q", nested["key"], "val") + } +} + +func TestStoreGetNotFound(t *testing.T) { + s := newTestStore(t) + + got, err := s.Get("nonexistent-id") + if err != nil { + t.Fatalf("get: %v", err) + } + if got != nil { + t.Fatalf("expected nil, got %+v", got) + } +} + +func TestStoreSaveUpsert(t *testing.T) { + s := newTestStore(t) + now := time.Now().Truncate(time.Nanosecond) + + r := &TaskResult{ + ID: "bbbbbbbb-1111-2222-3333-444444444444", + Type: "config-patch", + Status: TaskStatusRunning, + SubmittedAt: now, + } + if err := s.Save(r); err != nil { + t.Fatalf("first save: %v", err) + } + + // Update status. + completed := now.Add(time.Second) + r.Status = TaskStatusCompleted + r.CompletedAt = &completed + if err := s.Save(r); err != nil { + t.Fatalf("second save: %v", err) + } + + got, _ := s.Get(r.ID) + if got.Status != TaskStatusCompleted { + t.Fatalf("Status = %q after upsert, want %q", got.Status, TaskStatusCompleted) + } + if got.CompletedAt == nil { + t.Fatal("expected CompletedAt after upsert") + } +} + +func TestStoreListOrdering(t *testing.T) { + s := newTestStore(t) + base := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC) + + for i := 0; i < 5; i++ { + r := &TaskResult{ + ID: "list-" + string(rune('a'+i)) + "0000000-0000-0000-0000-000000000000", + Type: "config-patch", + Status: TaskStatusCompleted, + SubmittedAt: base.Add(time.Duration(i) * time.Minute), + } + if err := s.Save(r); err != nil { + t.Fatalf("save %d: %v", i, err) + } + } + + results, err := s.List(10) + if err != nil { + t.Fatalf("list: %v", err) + } + if len(results) != 5 { + t.Fatalf("expected 5 results, got %d", len(results)) + } + + // Newest first. + for i := 1; i < len(results); i++ { + if results[i].SubmittedAt.After(results[i-1].SubmittedAt) { + t.Fatalf("results not ordered newest-first at index %d", i) + } + } +} + +func TestStoreListLimit(t *testing.T) { + s := newTestStore(t) + base := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC) + + for i := 0; i < 20; i++ { + r := &TaskResult{ + ID: "limit-" + string(rune('a'+i)) + "000000-0000-0000-0000-000000000000", + Type: "config-patch", + Status: TaskStatusCompleted, + SubmittedAt: base.Add(time.Duration(i) * time.Minute), + } + if err := s.Save(r); err != nil { + t.Fatalf("save %d: %v", i, err) + } + } + + results, err := s.List(5) + if err != nil { + t.Fatalf("list: %v", err) + } + if len(results) != 5 { + t.Fatalf("expected 5 results, got %d", len(results)) + } +} + +func TestStoreDelete(t *testing.T) { + s := newTestStore(t) + + r := &TaskResult{ + ID: "cccccccc-1111-2222-3333-444444444444", + Type: "config-patch", + Status: TaskStatusCompleted, + SubmittedAt: time.Now(), + } + if err := s.Save(r); err != nil { + t.Fatal(err) + } + + deleted, err := s.Delete(r.ID) + if err != nil { + t.Fatalf("delete: %v", err) + } + if !deleted { + t.Fatal("expected delete to return true") + } + + got, _ := s.Get(r.ID) + if got != nil { + t.Fatal("expected nil after delete") + } +} + +func TestStoreDeleteNotFound(t *testing.T) { + s := newTestStore(t) + + deleted, err := s.Delete("nonexistent-id") + if err != nil { + t.Fatalf("delete: %v", err) + } + if deleted { + t.Fatal("expected delete to return false for nonexistent ID") + } +} + +func TestStoreMigrateIdempotent(t *testing.T) { + s := newTestStore(t) + + // Running migrate again on the same DB should be a no-op. + if err := migrate(s.db); err != nil { + t.Fatalf("second migrate: %v", err) + } +} + +func TestStorePing(t *testing.T) { + s := newTestStore(t) + if err := s.Ping(); err != nil { + t.Fatalf("Ping on healthy store: %v", err) + } + + s.Close() + if err := s.Ping(); err == nil { + t.Fatal("expected error from Ping on closed store") + } +} + +func TestStoreRunFieldRoundTrip(t *testing.T) { + s := newTestStore(t) + now := time.Now().Truncate(time.Nanosecond) + + r := &TaskResult{ + ID: "run-rt-0000-0000-0000-000000000000", + Type: "config-patch", + Status: TaskStatusFailed, + Run: 3, + Error: "transient", + SubmittedAt: now, + } + if err := s.Save(r); err != nil { + t.Fatalf("save: %v", err) + } + + got, err := s.Get(r.ID) + if err != nil { + t.Fatalf("get: %v", err) + } + if got.Run != 3 { + t.Fatalf("Run = %d, want 3", got.Run) + } +} + +func TestStoreResultFieldRoundTrip(t *testing.T) { + s := newTestStore(t) + now := time.Now().Truncate(time.Nanosecond) + + r := &TaskResult{ + ID: "res-rt00-0000-0000-0000-000000000000", + Type: "assemble-and-upload-genesis", + Status: TaskStatusCompleted, + Run: 1, + Result: json.RawMessage(`{"genesisHash":"abc123"}`), + SubmittedAt: now, + } + if err := s.Save(r); err != nil { + t.Fatalf("save: %v", err) + } + + got, err := s.Get(r.ID) + if err != nil { + t.Fatalf("get: %v", err) + } + if string(got.Result) != `{"genesisHash":"abc123"}` { + t.Fatalf("Result = %q, want round-tripped payload", string(got.Result)) + } +} + +func TestStoreNilResultIsNil(t *testing.T) { + // A handler that emits no result stores NULL and reads back as nil. + s := newTestStore(t) + + r := &TaskResult{ + ID: "res-nil0-0000-0000-0000-000000000000", + Type: "config-patch", + Status: TaskStatusCompleted, + SubmittedAt: time.Now(), + } + if err := s.Save(r); err != nil { + t.Fatalf("save: %v", err) + } + + got, err := s.Get(r.ID) + if err != nil { + t.Fatalf("get: %v", err) + } + if got.Result != nil { + t.Fatalf("Result = %q, want nil", string(got.Result)) + } +} + +func TestStoreNullableFields(t *testing.T) { + s := newTestStore(t) + + r := &TaskResult{ + ID: "dddddddd-1111-2222-3333-444444444444", + Type: "snapshot-restore", + Status: TaskStatusRunning, + SubmittedAt: time.Now(), + // CompletedAt nil; Params nil. + } + if err := s.Save(r); err != nil { + t.Fatalf("save: %v", err) + } + + got, err := s.Get(r.ID) + if err != nil { + t.Fatalf("get: %v", err) + } + if got.CompletedAt != nil { + t.Fatal("expected nil CompletedAt") + } +} + +// TestTxMarkerRoundTrip covers the pre-broadcast idempotency marker store: +// save+get is byte-identical, a missing key returns (nil, nil), and a second +// save for the same TaskID (INSERT OR REPLACE) overwrites cleanly. +func TestTxMarkerRoundTrip(t *testing.T) { + s := newTestStore(t) + + m := &TxMarker{ + TaskID: "task-round-trip", + TxHash: "ABCDEF0123456789", + TxBytes: []byte{0x00, 0xDE, 0xAD, 0xBE, 0xEF, 0x00, 0xFF}, + AccountNumber: 17, + Sequence: 42, + ChainID: "pacific-1", + } + if err := s.SaveTxMarker(m); err != nil { + t.Fatalf("SaveTxMarker: %v", err) + } + + got, err := s.GetTxMarker(m.TaskID) + if err != nil { + t.Fatalf("GetTxMarker: %v", err) + } + if got == nil { + t.Fatal("expected marker, got nil") + } + if string(got.TxBytes) != string(m.TxBytes) { + t.Fatalf("TxBytes not byte-identical: got %x want %x", got.TxBytes, m.TxBytes) + } + if got.TxHash != m.TxHash || got.AccountNumber != m.AccountNumber || + got.Sequence != m.Sequence || got.ChainID != m.ChainID { + t.Fatalf("marker fields differ: got %+v want %+v", got, m) + } + + // Missing key → (nil, nil). + miss, err := s.GetTxMarker("missing") + if err != nil { + t.Fatalf("GetTxMarker(missing): %v", err) + } + if miss != nil { + t.Fatalf("expected nil for missing marker, got %+v", miss) + } + + // INSERT OR REPLACE: second save with same TaskID overwrites. + m2 := &TxMarker{ + TaskID: m.TaskID, + TxHash: "1111111111111111", + TxBytes: []byte{0x11, 0x22}, + AccountNumber: 100, + Sequence: 101, + ChainID: "atlantic-2", + } + if err := s.SaveTxMarker(m2); err != nil { + t.Fatalf("second SaveTxMarker: %v", err) + } + got2, err := s.GetTxMarker(m.TaskID) + if err != nil { + t.Fatalf("GetTxMarker after replace: %v", err) + } + if got2.TxHash != m2.TxHash || string(got2.TxBytes) != string(m2.TxBytes) || + got2.Sequence != m2.Sequence || got2.ChainID != m2.ChainID { + t.Fatalf("replace did not overwrite cleanly: got %+v want %+v", got2, m2) + } +} diff --git a/sidecar/engine/store.go b/sidecar/engine/store.go new file mode 100644 index 00000000..2e941ab6 --- /dev/null +++ b/sidecar/engine/store.go @@ -0,0 +1,41 @@ +package engine + +// ResultStore persists task results across all lifecycle states. +// Implementations must be safe for concurrent use. +type ResultStore interface { + // Save persists a TaskResult. If a result with the same ID already + // exists, it is overwritten (upsert). + Save(r *TaskResult) error + + // Get returns a result by ID, or (nil, nil) when not found. + Get(id string) (*TaskResult, error) + + // List returns the most recent results, newest first, up to limit. + List(limit int) ([]TaskResult, error) + + // ListStaleTasks returns tasks left in "running" state from a + // previous process that exited without completing them. + ListStaleTasks() ([]TaskResult, error) + + // Delete removes a result by ID. Returns true if it existed. + Delete(id string) (bool, error) + + // DeleteByType removes all results of the given task type and returns + // how many rows were removed. Used by mark-not-ready to purge recorded + // mark-ready results so a stranded running one cannot rehydrate and + // release a node hold after a data wipe. + DeleteByType(taskType string) (int, error) + + // LatestByType returns the most recently submitted result of the given + // task type (any status), or (nil, nil) when none exists. Rehydration + // uses it to detect a hold that supersedes a stranded mark-ready even + // after the hold's own record has gone terminal (e.g. a failed purge + // persisted it Failed), which a stale-only scan would miss. + LatestByType(taskType string) (*TaskResult, error) + + // Ping verifies the store is responsive. Used by liveness checks. + Ping() error + + // Close releases underlying resources. + Close() error +} diff --git a/sidecar/engine/typed_handler.go b/sidecar/engine/typed_handler.go new file mode 100644 index 00000000..dc0788e5 --- /dev/null +++ b/sidecar/engine/typed_handler.go @@ -0,0 +1,50 @@ +package engine + +import ( + "context" + "encoding/json" + "fmt" +) + +// TypedHandler wraps a result-less typed handler into a TaskHandler. The +// map[string]any params are marshaled to JSON and unmarshaled into the typed +// struct T, giving handlers compile-time type safety without changing the +// engine's dispatch mechanism. Handlers that produce a structured result use +// TypedHandlerWithResult instead. +func TypedHandler[T any](fn func(ctx context.Context, params T) error) TaskHandler { + return TypedHandlerWithResult(func(ctx context.Context, params T) (json.RawMessage, error) { + return nil, fn(ctx, params) + }) +} + +// TypedHandlerWithResult wraps a typed handler that returns a structured +// result into a TaskHandler. R is marshaled to json.RawMessage and returned +// alongside the error, so the engine persists it on both the success and +// error paths (an error return may still carry a meaningful R). A nil/zero R +// that marshals to "null" is treated as no result. +func TypedHandlerWithResult[T, R any](fn func(ctx context.Context, params T) (R, error)) TaskHandler { + return func(ctx context.Context, params map[string]any) (json.RawMessage, error) { + data, err := json.Marshal(params) + if err != nil { + return nil, fmt.Errorf("marshaling params: %w", err) + } + var typed T + if err := json.Unmarshal(data, &typed); err != nil { + return nil, fmt.Errorf("parsing params: %w", err) + } + result, ferr := fn(ctx, typed) + raw, merr := json.Marshal(result) + if merr != nil { + // A result that won't marshal shouldn't mask the handler's own + // outcome; surface it only when the handler otherwise succeeded. + if ferr == nil { + return nil, fmt.Errorf("marshaling result: %w", merr) + } + return nil, ferr + } + if string(raw) == "null" { + raw = nil + } + return raw, ferr + } +} diff --git a/sidecar/engine/typed_handler_test.go b/sidecar/engine/typed_handler_test.go new file mode 100644 index 00000000..6271917c --- /dev/null +++ b/sidecar/engine/typed_handler_test.go @@ -0,0 +1,154 @@ +package engine + +import ( + "context" + "testing" +) + +func TestTypedHandler_HappyPath(t *testing.T) { + type req struct { + Name string `json:"name"` + Age int `json:"age"` + } + + var captured req + handler := TypedHandler(func(_ context.Context, r req) error { + captured = r + return nil + }) + + _, err := handler(context.Background(), map[string]any{ + "name": "alice", + "age": float64(30), + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if captured.Name != "alice" { + t.Errorf("Name = %q, want %q", captured.Name, "alice") + } + if captured.Age != 30 { + t.Errorf("Age = %d, want 30", captured.Age) + } +} + +func TestTypedHandler_MalformedJSON(t *testing.T) { + type req struct { + Value int `json:"value"` + } + + handler := TypedHandler(func(_ context.Context, r req) error { + return nil + }) + + // A channel cannot be marshaled to JSON, causing a marshal error. + _, err := handler(context.Background(), map[string]any{ + "value": make(chan int), + }) + if err == nil { + t.Fatal("expected error for un-marshalable params") + } +} + +func TestTypedHandler_NilParams(t *testing.T) { + type req struct { + Name string `json:"name"` + Age int `json:"age"` + } + + var captured req + handler := TypedHandler(func(_ context.Context, r req) error { + captured = r + return nil + }) + + _, err := handler(context.Background(), nil) + if err != nil { + t.Fatalf("unexpected error for nil params: %v", err) + } + if captured.Name != "" { + t.Errorf("Name = %q, want empty string", captured.Name) + } + if captured.Age != 0 { + t.Errorf("Age = %d, want 0", captured.Age) + } +} + +func TestTypedHandler_EmptyParams(t *testing.T) { + type req struct { + Name string `json:"name"` + Age int `json:"age"` + } + + var captured req + handler := TypedHandler(func(_ context.Context, r req) error { + captured = r + return nil + }) + + _, err := handler(context.Background(), map[string]any{}) + if err != nil { + t.Fatalf("unexpected error for empty params: %v", err) + } + if captured.Name != "" { + t.Errorf("Name = %q, want empty string", captured.Name) + } + if captured.Age != 0 { + t.Errorf("Age = %d, want 0", captured.Age) + } +} + +func TestTypedHandler_NestedStruct(t *testing.T) { + type inner struct { + Key string `json:"key"` + Value string `json:"value"` + } + type req struct { + Nested inner `json:"nested"` + } + + var captured req + handler := TypedHandler(func(_ context.Context, r req) error { + captured = r + return nil + }) + + _, err := handler(context.Background(), map[string]any{ + "nested": map[string]any{ + "key": "foo", + "value": "bar", + }, + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if captured.Nested.Key != "foo" { + t.Errorf("Nested.Key = %q, want %q", captured.Nested.Key, "foo") + } + if captured.Nested.Value != "bar" { + t.Errorf("Nested.Value = %q, want %q", captured.Nested.Value, "bar") + } +} + +func TestTypedHandler_Float64ToInt64Coercion(t *testing.T) { + type req struct { + Height int64 `json:"height"` + } + + var captured req + handler := TypedHandler(func(_ context.Context, r req) error { + captured = r + return nil + }) + + // JSON numbers arrive as float64 when unmarshaled into map[string]any. + _, err := handler(context.Background(), map[string]any{ + "height": float64(198030000), + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if captured.Height != 198030000 { + t.Errorf("Height = %d, want 198030000", captured.Height) + } +} diff --git a/sidecar/engine/types.go b/sidecar/engine/types.go new file mode 100644 index 00000000..2b7aff1f --- /dev/null +++ b/sidecar/engine/types.go @@ -0,0 +1,153 @@ +package engine + +import ( + "context" + "encoding/json" + "fmt" + "time" + + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/keyring" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/wire" +) + +// TaskType identifies the kind of task to execute. +// TaskType aliases wire.TaskType so engine call sites and the dispatch-map key +// type are unchanged; the consts below re-export wire.Task* likewise. +type TaskType = wire.TaskType + +const ( + TaskSnapshotRestore = wire.TaskSnapshotRestore + TaskConfigPatch = wire.TaskConfigPatch + TaskConfigApply = wire.TaskConfigApply + TaskConfigValidate = wire.TaskConfigValidate + TaskConfigReload = wire.TaskConfigReload + TaskMarkReady = wire.TaskMarkReady + TaskRestartSeid = wire.TaskRestartSeid + TaskConfigureGenesis = wire.TaskConfigureGenesis + TaskConfigureStateSync = wire.TaskConfigureStateSync + TaskSnapshotUpload = wire.TaskSnapshotUpload + TaskSnapshotUploadOnce = wire.TaskSnapshotUploadOnce + TaskResultExport = wire.TaskResultExport + TaskAwaitCondition = wire.TaskAwaitCondition + TaskGenerateIdentity = wire.TaskGenerateIdentity + TaskGenerateGentx = wire.TaskGenerateGentx + TaskUploadGenesisArtifacts = wire.TaskUploadGenesisArtifacts + TaskAssembleAndUploadGenesis = wire.TaskAssembleAndUploadGenesis + TaskSetGenesisPeers = wire.TaskSetGenesisPeers + TaskGovVote = wire.TaskGovVote + TaskGovSoftwareUpgrade = wire.TaskGovSoftwareUpgrade + TaskGovParamChange = wire.TaskGovParamChange + TaskEvmLogicalDigest = wire.TaskEvmLogicalDigest + TaskMarkNotReady = wire.TaskMarkNotReady + TaskStopSeid = wire.TaskStopSeid + TaskResetData = wire.TaskResetData +) + +// Task is a unit of work submitted by the controller. When ID is set, the +// engine uses it as the canonical task identifier (enabling deterministic +// IDs from the controller). When empty, the engine generates a random UUID. +type Task struct { + ID string `json:"id,omitempty"` + Type TaskType `json:"type"` + Params map[string]any `json:"params,omitempty"` +} + +// TaskHandler executes a specific task type. Handlers MUST be idempotent: +// the engine may re-execute a handler after a crash recovery. The returned +// json.RawMessage is the handler's optional structured result, persisted on +// TaskResult.Result and surfaced over GET /v0/tasks/{id}; handlers with no +// result return nil. The engine stamps the result on both the success and +// error paths (a handler returning an error may still carry a result, e.g. a +// tx hash for an inclusion-undetermined gov submit). +type TaskHandler func(ctx context.Context, params map[string]any) (json.RawMessage, error) + +type taskIDKey struct{} + +// TaskIDFromContext returns the engine-assigned task ID for the current +// handler, or "" when ctx is not engine-produced. Sign-tx handlers use +// this to derive the per-task memo tag. +func TaskIDFromContext(ctx context.Context) string { + if v, ok := ctx.Value(taskIDKey{}).(string); ok { + return v + } + return "" +} + +// WithTaskID attaches a task ID to ctx for handler consumption. The engine +// calls this in newTaskContext; tests use it to bypass Submit. +func WithTaskID(ctx context.Context, id string) context.Context { + return context.WithValue(ctx, taskIDKey{}, id) +} + +// TaskStatus represents the lifecycle state of a task. +type TaskStatus string + +const ( + TaskStatusRunning TaskStatus = "running" + TaskStatusCompleted TaskStatus = "completed" + TaskStatusFailed TaskStatus = "failed" +) + +// TaskError is a structured error that includes operator-actionable context. +// Task handlers return this to provide rich error detail beyond a plain string. +type TaskError struct { + Task string `json:"task"` + Operation string `json:"operation"` + Message string `json:"message"` + Hint string `json:"hint,omitempty"` + Retryable bool `json:"retryable"` + Cause string `json:"cause,omitempty"` +} + +func (e *TaskError) Error() string { + s := fmt.Sprintf("%s: %s: %s", e.Task, e.Operation, e.Message) + if e.Hint != "" { + s += fmt.Sprintf(" [hint: %s]", e.Hint) + } + return s +} + +// TaskResult records a task and its outcome. +// +// Result carries a handler's structured output (e.g. assemble-genesis emits +// {"genesisHash":""}). It is optional and additive: handlers that +// emit nothing leave it nil and it is omitted from the wire, so the +// currently-deployed controller is unaffected. This in-band channel — read +// by the controller over the trusted GET /v0/tasks/{id} path — is the +// authenticated alternative to publishing results through attacker-writable +// shared storage. +type TaskResult struct { + ID string `json:"id"` + Type string `json:"type"` + Status TaskStatus `json:"status"` + Run int `json:"run"` + Params map[string]any `json:"params,omitempty"` + Result json.RawMessage `json:"result,omitempty"` + Error string `json:"error,omitempty"` + SubmittedAt time.Time `json:"submittedAt"` + CompletedAt *time.Time `json:"completedAt,omitempty"` +} + +// StatusResponse is the shape returned by the status endpoint. +type StatusResponse struct { + Status string `json:"status"` +} + +// ExecutionConfig carries process-wide deps the engine exposes to handlers. +// Fields are nil when the corresponding subsystem is not configured. +type ExecutionConfig struct { + // Keyring is opened from SEI_KEYRING_BACKEND. Nil when unset; + // sign-tx handlers report a clear error rather than panic. + Keyring keyring.Keyring + + // RPC talks to the co-located seid CometBFT RPC. Sign-tx handlers + // use it for the chain-confusion guard and inclusion polling. + RPC *rpc.Client + + // Checkpointer persists a pre-broadcast TxMarker so a crashed sign-tx + // task re-adopts its in-flight tx on re-run rather than re-signing. + // Nil when no durable store is configured. + Checkpointer Checkpointer +} diff --git a/sidecar/go.mod b/sidecar/go.mod new file mode 100644 index 00000000..c393e4f2 --- /dev/null +++ b/sidecar/go.mod @@ -0,0 +1,251 @@ +module github.com/sei-protocol/sei-k8s-controller/sidecar + +go 1.26.0 + +require ( + github.com/aws/aws-sdk-go-v2 v1.43.5 + github.com/aws/aws-sdk-go-v2/config v1.32.36 + github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.12 + github.com/aws/aws-sdk-go-v2/service/s3 v1.107.1 + github.com/aws/smithy-go v1.27.7 + github.com/ethereum/go-ethereum v1.16.8 + github.com/google/uuid v1.6.0 + github.com/prometheus/client_golang v1.23.2 + github.com/sei-protocol/sei-chain v0.0.29-fix.0.20260326202429-c9b42951fef7 + github.com/sei-protocol/sei-config v0.0.25 + github.com/sei-protocol/sei-k8s-controller/sidecarapi v0.0.0 + github.com/sei-protocol/seilog v0.0.3 + github.com/urfave/cli/v3 v3.6.1 + modernc.org/sqlite v1.18.1 +) + +require ( + filippo.io/edwards25519 v1.1.0 // indirect + github.com/99designs/keyring v1.2.1 // indirect + github.com/BurntSushi/toml v1.5.0 // indirect + github.com/DataDog/zstd v1.5.7 // indirect + github.com/Microsoft/go-winio v0.6.2 // indirect + github.com/RaduBerinde/axisds v0.0.0-20250419182453-5135a0650657 // indirect + github.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 // indirect + github.com/alitto/pond v1.8.3 // indirect + github.com/armon/go-metrics v0.4.1 // indirect + github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.17 // indirect + github.com/aws/aws-sdk-go-v2/credentials v1.19.35 // indirect + github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36 // indirect + github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36 // indirect + github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36 // indirect + github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.16 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.29 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.36 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.37 // indirect + github.com/aws/aws-sdk-go-v2/service/signin v1.5.5 // indirect + github.com/aws/aws-sdk-go-v2/service/sso v1.33.5 // indirect + github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.5 // indirect + github.com/aws/aws-sdk-go-v2/service/sts v1.45.5 // indirect + github.com/benbjohnson/immutable v0.4.3 // indirect + github.com/beorn7/perks v1.0.1 // indirect + github.com/bgentry/speakeasy v0.2.0 // indirect + github.com/bits-and-blooms/bitset v1.24.3 // indirect + github.com/btcsuite/btcd/btcec/v2 v2.3.5 // indirect + github.com/cenkalti/backoff/v4 v4.3.0 // indirect + github.com/cespare/xxhash v1.1.0 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b // indirect + github.com/cockroachdb/errors v1.12.0 // indirect + github.com/cockroachdb/logtags v0.0.0-20241215232642-bb51bb14a506 // indirect + github.com/cockroachdb/pebble/v2 v2.1.3 // indirect + github.com/cockroachdb/redact v1.1.6 // indirect + github.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b // indirect + github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect + github.com/coinbase/rosetta-sdk-go v0.7.0 // indirect + github.com/confio/ics23/go v0.9.0 // indirect + github.com/consensys/gnark-crypto v0.18.0 // indirect + github.com/cosmos/btcutil v1.0.5 // indirect + github.com/cosmos/go-bip39 v1.0.0 // indirect + github.com/cosmos/gorocksdb v1.2.0 // indirect + github.com/cosmos/ledger-cosmos-go v1.0.0 // indirect + github.com/crate-crypto/go-ipa v0.0.0-20240724233137-53bbb0ceb27a // indirect + github.com/crate-crypto/go-kzg-4844 v1.1.0 // indirect + github.com/creachadair/taskgroup v0.3.2 // indirect + github.com/danieljoos/wincred v1.1.2 // indirect + github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect + github.com/deckarep/golang-set/v2 v2.6.0 // indirect + github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect + github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f // indirect + github.com/dgraph-io/badger/v3 v3.2103.2 // indirect + github.com/dgraph-io/ristretto v0.2.0 // indirect + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/dvsekhvalnov/jose2go v1.7.0 // indirect + github.com/ethereum/c-kzg-4844 v1.0.0 // indirect + github.com/ethereum/go-verkle v0.2.2 // indirect + github.com/felixge/httpsnoop v1.0.4 // indirect + github.com/fsnotify/fsnotify v1.9.0 // indirect + github.com/getsentry/sentry-go v0.35.0 // indirect + github.com/go-kit/kit v0.13.0 // indirect + github.com/go-logr/logr v1.4.3 // indirect + github.com/go-logr/stdr v1.2.2 // indirect + github.com/go-ole/go-ole v1.3.0 // indirect + github.com/go-viper/mapstructure/v2 v2.5.0 // indirect + github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect + github.com/gogo/gateway v1.1.0 // indirect + github.com/gogo/protobuf v1.3.3 // indirect + github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect + github.com/golang/protobuf v1.5.4 // indirect + github.com/golang/snappy v1.0.0 // indirect + github.com/google/btree v1.1.3 // indirect + github.com/google/flatbuffers v25.2.10+incompatible // indirect + github.com/google/go-cmp v0.7.0 // indirect + github.com/google/orderedcode v0.0.1 // indirect + github.com/gorilla/handlers v1.5.2 // indirect + github.com/gorilla/mux v1.8.1 // indirect + github.com/gorilla/websocket v1.5.3 // indirect + github.com/grafana/pyroscope-go/godeltaprof v0.1.8 // indirect + github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect + github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect + github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect + github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c // indirect + github.com/hashicorp/go-immutable-radix v1.3.1 // indirect + github.com/hashicorp/golang-lru v1.0.2 // indirect + github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect + github.com/hdevalence/ed25519consensus v0.2.0 // indirect + github.com/holiman/uint256 v1.3.2 // indirect + github.com/improbable-eng/grpc-web v0.15.0 // indirect + github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/jmhodges/levigo v1.0.0 // indirect + github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect + github.com/keybase/go-keychain v0.0.0-20190712205309-48d3d31d256d // indirect + github.com/klauspost/compress v1.18.3 // indirect + github.com/klauspost/cpuid/v2 v2.3.0 // indirect + github.com/kr/pretty v0.3.1 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/kylelemons/godebug v1.1.0 // indirect + github.com/ledgerwatch/erigon-lib v0.0.0-20230210071639-db0e7ed11263 // indirect + github.com/lib/pq v1.10.9 // indirect + github.com/libp2p/go-buffer-pool v0.1.0 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882 // indirect + github.com/mitchellh/mapstructure v1.5.0 // indirect + github.com/mtibben/percent v0.2.1 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/oasisprotocol/curve25519-voi v0.0.0-20230904125328-1f23a7beb09a // indirect + github.com/patrickmn/go-cache v2.1.0+incompatible // indirect + github.com/pelletier/go-toml/v2 v2.2.4 // indirect + github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741 // indirect + github.com/pkg/errors v0.9.1 // indirect + github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.17.0 // indirect + github.com/rakyll/statik v0.1.7 // indirect + github.com/regen-network/cosmos-proto v0.3.1 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect + github.com/rogpeppe/go-internal v1.14.1 // indirect + github.com/rs/cors v1.11.1 // indirect + github.com/sagikazarmark/locafero v0.11.0 // indirect + github.com/sasha-s/go-deadlock v0.3.5 // indirect + github.com/sei-protocol/sei-tm-db v0.0.5 // indirect + github.com/shirou/gopsutil v3.21.11+incompatible // indirect + github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect + github.com/spf13/afero v1.15.0 // indirect + github.com/spf13/cast v1.10.0 // indirect + github.com/spf13/cobra v1.10.2 // indirect + github.com/spf13/pflag v1.0.10 // indirect + github.com/spf13/viper v1.21.0 // indirect + github.com/stretchr/testify v1.11.1 // indirect + github.com/subosito/gotenv v1.6.0 // indirect + github.com/supranational/blst v0.3.16-0.20250831170142-f48500c1fdbe // indirect + github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d // indirect + github.com/tendermint/crypto v0.0.0-20191022145703-50d29ede1e15 // indirect + github.com/tendermint/go-amino v0.16.0 // indirect + github.com/tendermint/tm-db v0.6.8-0.20220519162814-e24b96538a12 // indirect + github.com/tidwall/gjson v1.14.2 // indirect + github.com/tidwall/match v1.1.1 // indirect + github.com/tidwall/pretty v1.2.0 // indirect + github.com/tidwall/tinylru v1.1.0 // indirect + github.com/tidwall/wal v1.2.1 // indirect + github.com/tklauser/go-sysconf v0.3.15 // indirect + github.com/tklauser/numcpus v0.10.0 // indirect + github.com/yusufpapurcu/wmi v1.2.4 // indirect + github.com/zbiljic/go-filelock v0.0.0-20170914061330-1dbf7103ab7d // indirect + github.com/zeebo/blake3 v0.2.4 // indirect + github.com/zondax/golem v0.27.0 // indirect + github.com/zondax/hid v0.9.2 // indirect + github.com/zondax/ledger-go v1.0.1 // indirect + go.etcd.io/bbolt v1.4.0-alpha.0.0.20240404170359-43604f3112c5 // indirect + go.opencensus.io v0.24.0 // indirect + go.opentelemetry.io/auto/sdk v1.2.1 // indirect + go.opentelemetry.io/otel v1.39.0 // indirect + go.opentelemetry.io/otel/exporters/jaeger v1.9.0 // indirect + go.opentelemetry.io/otel/metric v1.39.0 // indirect + go.opentelemetry.io/otel/sdk v1.39.0 // indirect + go.opentelemetry.io/otel/trace v1.39.0 // indirect + go.uber.org/multierr v1.11.0 // indirect + go.uber.org/zap v1.27.0 // indirect + go.yaml.in/yaml/v2 v2.4.3 // indirect + go.yaml.in/yaml/v3 v3.0.4 // indirect + golang.org/x/crypto v0.47.0 // indirect + golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect + golang.org/x/mod v0.32.0 // indirect + golang.org/x/net v0.49.0 // indirect + golang.org/x/sync v0.19.0 // indirect + golang.org/x/sys v0.40.0 // indirect + golang.org/x/term v0.39.0 // indirect + golang.org/x/text v0.34.0 // indirect + golang.org/x/time v0.13.0 // indirect + golang.org/x/tools v0.41.0 // indirect + google.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect + google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c // indirect + google.golang.org/grpc v1.75.0 // indirect + google.golang.org/protobuf v1.36.11 // indirect + gopkg.in/yaml.v2 v2.4.0 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect + k8s.io/component-base v0.35.0 // indirect + lukechampine.com/uint128 v1.2.0 // indirect + modernc.org/cc/v3 v3.36.3 // indirect + modernc.org/ccgo/v3 v3.16.9 // indirect + modernc.org/libc v1.17.1 // indirect + modernc.org/mathutil v1.5.0 // indirect + modernc.org/memory v1.2.1 // indirect + modernc.org/opt v0.1.3 // indirect + modernc.org/strutil v1.1.3 // indirect + modernc.org/token v1.0.0 // indirect + nhooyr.io/websocket v1.8.6 // indirect +) + +replace github.com/sei-protocol/sei-k8s-controller/sidecarapi => ../sidecarapi + +// sei-chain's go.mod uses replace directives for forked dependencies, and Go +// ignores replace directives from transitive dependencies — so this module, +// which carries the chain graph, must restate all of them. These must stay in +// sync with sei-chain's go.mod. +// +// Two are load-bearing beyond dependency resolution: +// +// - 99designs/keyring is the library that decrypts the validator operator +// keyring. The substitution changes that code path, so a partial port here +// is a silent change to key handling. +// - go-ethereum redirects to the Sei fork, which sei-chain/app expects. Since +// a replace only applies from the main module, anything that links this +// graph without the redirect silently resolves upstream geth — a latent +// encoding and signing divergence. +// +// golang.org/x/crypto and google.golang.org/grpc are pinned *down* relative to +// what the controller module wants. That is why this module cannot share a +// go.work with the root: a workspace promotes a used module's replaces to +// main-module status, so a local build would resolve these downgrades for the +// controller too and diverge from every GOWORK=off build. +replace ( + github.com/99designs/keyring => github.com/cosmos/keyring v1.1.7-0.20210622111912-ef00f8ac3d76 + github.com/btcsuite/btcd => github.com/btcsuite/btcd v0.23.2 + github.com/confio/ics23/go => github.com/cosmos/cosmos-sdk/ics23/go v0.8.0 + github.com/ethereum/go-ethereum => github.com/sei-protocol/go-ethereum v1.15.7-sei-16 + github.com/gin-gonic/gin => github.com/gin-gonic/gin v1.7.0 + github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.3-alpha.regen.1 + github.com/keybase/go-keychain => github.com/keybase/go-keychain v0.0.0-20190712205309-48d3d31d256d + github.com/syndtr/goleveldb => github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 + github.com/tendermint/tm-db => github.com/sei-protocol/tm-db v0.0.4 + golang.org/x/crypto => golang.org/x/crypto v0.31.0 + google.golang.org/grpc => google.golang.org/grpc v1.57.1 +) diff --git a/sidecar/go.sum b/sidecar/go.sum new file mode 100644 index 00000000..5959cfe4 --- /dev/null +++ b/sidecar/go.sum @@ -0,0 +1,2947 @@ +cloud.google.com/go v0.0.0-20170206221025-ce650573d812/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= +cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= +cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= +cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= +cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= +cloud.google.com/go v0.44.3/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= +cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= +cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= +cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= +cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= +cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= +cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= +cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk= +cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs= +cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc= +cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY= +cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI= +cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk= +cloud.google.com/go v0.75.0/go.mod h1:VGuuCn7PG0dwsd5XPVm2Mm3wlh3EL55/79EKB6hlPTY= +cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg= +cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8= +cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0= +cloud.google.com/go v0.83.0/go.mod h1:Z7MJUsANfY0pYPdw0lbnivPx4/vhy/e2FEkSkF7vAVY= +cloud.google.com/go v0.84.0/go.mod h1:RazrYuxIK6Kb7YrzzhPoLmCVzl7Sup4NrbKPg8KHSUM= +cloud.google.com/go v0.87.0/go.mod h1:TpDYlFy7vuLzZMMZ+B6iRiELaY7z/gJPaqbMx6mlWcY= +cloud.google.com/go v0.90.0/go.mod h1:kRX0mNRHe0e2rC6oNakvwQqzyDmg57xJ+SZU1eT2aDQ= +cloud.google.com/go v0.93.3/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+YI= +cloud.google.com/go v0.94.1/go.mod h1:qAlAugsXlC+JWO+Bke5vCtc9ONxjQT3drlTTnAplMW4= +cloud.google.com/go v0.97.0/go.mod h1:GF7l59pYBVlXQIBLx3a761cZ41F9bBH3JUlihCt2Udc= +cloud.google.com/go v0.99.0/go.mod h1:w0Xx2nLzqWJPuozYQX+hFfCSI8WioryfRDzkoI/Y2ZA= +cloud.google.com/go v0.100.1/go.mod h1:fs4QogzfH5n2pBXBP9vRiU+eCny7lD2vmFZy79Iuw1U= +cloud.google.com/go v0.100.2/go.mod h1:4Xra9TjzAeYHrl5+oeLlzbM2k3mjVhZh4UqTZ//w99A= +cloud.google.com/go v0.102.0/go.mod h1:oWcCzKlqJ5zgHQt9YsaeTY9KzIvjyy0ArmiBUgpQ+nc= +cloud.google.com/go v0.102.1/go.mod h1:XZ77E9qnTEnrgEOvr4xzfdX5TRo7fB4T2F4O6+34hIU= +cloud.google.com/go v0.104.0/go.mod h1:OO6xxXdJyvuJPcEPBLN9BJPD+jep5G1+2U5B5gkRYtA= +cloud.google.com/go v0.105.0/go.mod h1:PrLgOJNe5nfE9UMxKxgXj4mD3voiP+YQ6gdt6KMFOKM= +cloud.google.com/go v0.107.0/go.mod h1:wpc2eNrD7hXUTy8EKS10jkxpZBjASrORK7goS+3YX2I= +cloud.google.com/go v0.110.0/go.mod h1:SJnCLqQ0FCFGSZMUNUf84MV3Aia54kn7pi8st7tMzaY= +cloud.google.com/go/accessapproval v1.4.0/go.mod h1:zybIuC3KpDOvotz59lFe5qxRZx6C75OtwbisN56xYB4= +cloud.google.com/go/accessapproval v1.5.0/go.mod h1:HFy3tuiGvMdcd/u+Cu5b9NkO1pEICJ46IR82PoUdplw= +cloud.google.com/go/accessapproval v1.6.0/go.mod h1:R0EiYnwV5fsRFiKZkPHr6mwyk2wxUJ30nL4j2pcFY2E= +cloud.google.com/go/accesscontextmanager v1.3.0/go.mod h1:TgCBehyr5gNMz7ZaH9xubp+CE8dkrszb4oK9CWyvD4o= +cloud.google.com/go/accesscontextmanager v1.4.0/go.mod h1:/Kjh7BBu/Gh83sv+K60vN9QE5NJcd80sU33vIe2IFPE= +cloud.google.com/go/accesscontextmanager v1.6.0/go.mod h1:8XCvZWfYw3K/ji0iVnp+6pu7huxoQTLmxAbVjbloTtM= +cloud.google.com/go/accesscontextmanager v1.7.0/go.mod h1:CEGLewx8dwa33aDAZQujl7Dx+uYhS0eay198wB/VumQ= +cloud.google.com/go/aiplatform v1.22.0/go.mod h1:ig5Nct50bZlzV6NvKaTwmplLLddFx0YReh9WfTO5jKw= +cloud.google.com/go/aiplatform v1.24.0/go.mod h1:67UUvRBKG6GTayHKV8DBv2RtR1t93YRu5B1P3x99mYY= +cloud.google.com/go/aiplatform v1.27.0/go.mod h1:Bvxqtl40l0WImSb04d0hXFU7gDOiq9jQmorivIiWcKg= +cloud.google.com/go/aiplatform v1.35.0/go.mod h1:7MFT/vCaOyZT/4IIFfxH4ErVg/4ku6lKv3w0+tFTgXQ= +cloud.google.com/go/aiplatform v1.36.1/go.mod h1:WTm12vJRPARNvJ+v6P52RDHCNe4AhvjcIZ/9/RRHy/k= +cloud.google.com/go/aiplatform v1.37.0/go.mod h1:IU2Cv29Lv9oCn/9LkFiiuKfwrRTq+QQMbW+hPCxJGZw= +cloud.google.com/go/analytics v0.11.0/go.mod h1:DjEWCu41bVbYcKyvlws9Er60YE4a//bK6mnhWvQeFNI= +cloud.google.com/go/analytics v0.12.0/go.mod h1:gkfj9h6XRf9+TS4bmuhPEShsh3hH8PAZzm/41OOhQd4= +cloud.google.com/go/analytics v0.17.0/go.mod h1:WXFa3WSym4IZ+JiKmavYdJwGG/CvpqiqczmL59bTD9M= +cloud.google.com/go/analytics v0.18.0/go.mod h1:ZkeHGQlcIPkw0R/GW+boWHhCOR43xz9RN/jn7WcqfIE= +cloud.google.com/go/analytics v0.19.0/go.mod h1:k8liqf5/HCnOUkbawNtrWWc+UAzyDlW89doe8TtoDsE= +cloud.google.com/go/apigateway v1.3.0/go.mod h1:89Z8Bhpmxu6AmUxuVRg/ECRGReEdiP3vQtk4Z1J9rJk= +cloud.google.com/go/apigateway v1.4.0/go.mod h1:pHVY9MKGaH9PQ3pJ4YLzoj6U5FUDeDFBllIz7WmzJoc= +cloud.google.com/go/apigateway v1.5.0/go.mod h1:GpnZR3Q4rR7LVu5951qfXPJCHquZt02jf7xQx7kpqN8= +cloud.google.com/go/apigeeconnect v1.3.0/go.mod h1:G/AwXFAKo0gIXkPTVfZDd2qA1TxBXJ3MgMRBQkIi9jc= +cloud.google.com/go/apigeeconnect v1.4.0/go.mod h1:kV4NwOKqjvt2JYR0AoIWo2QGfoRtn/pkS3QlHp0Ni04= +cloud.google.com/go/apigeeconnect v1.5.0/go.mod h1:KFaCqvBRU6idyhSNyn3vlHXc8VMDJdRmwDF6JyFRqZ8= +cloud.google.com/go/apigeeregistry v0.4.0/go.mod h1:EUG4PGcsZvxOXAdyEghIdXwAEi/4MEaoqLMLDMIwKXY= +cloud.google.com/go/apigeeregistry v0.5.0/go.mod h1:YR5+s0BVNZfVOUkMa5pAR2xGd0A473vA5M7j247o1wM= +cloud.google.com/go/apigeeregistry v0.6.0/go.mod h1:BFNzW7yQVLZ3yj0TKcwzb8n25CFBri51GVGOEUcgQsc= +cloud.google.com/go/apikeys v0.4.0/go.mod h1:XATS/yqZbaBK0HOssf+ALHp8jAlNHUgyfprvNcBIszU= +cloud.google.com/go/apikeys v0.5.0/go.mod h1:5aQfwY4D+ewMMWScd3hm2en3hCj+BROlyrt3ytS7KLI= +cloud.google.com/go/apikeys v0.6.0/go.mod h1:kbpXu5upyiAlGkKrJgQl8A0rKNNJ7dQ377pdroRSSi8= +cloud.google.com/go/appengine v1.4.0/go.mod h1:CS2NhuBuDXM9f+qscZ6V86m1MIIqPj3WC/UoEuR1Sno= +cloud.google.com/go/appengine v1.5.0/go.mod h1:TfasSozdkFI0zeoxW3PTBLiNqRmzraodCWatWI9Dmak= +cloud.google.com/go/appengine v1.6.0/go.mod h1:hg6i0J/BD2cKmDJbaFSYHFyZkgBEfQrDg/X0V5fJn84= +cloud.google.com/go/appengine v1.7.0/go.mod h1:eZqpbHFCqRGa2aCdope7eC0SWLV1j0neb/QnMJVWx6A= +cloud.google.com/go/appengine v1.7.1/go.mod h1:IHLToyb/3fKutRysUlFO0BPt5j7RiQ45nrzEJmKTo6E= +cloud.google.com/go/area120 v0.5.0/go.mod h1:DE/n4mp+iqVyvxHN41Vf1CR602GiHQjFPusMFW6bGR4= +cloud.google.com/go/area120 v0.6.0/go.mod h1:39yFJqWVgm0UZqWTOdqkLhjoC7uFfgXRC8g/ZegeAh0= +cloud.google.com/go/area120 v0.7.0/go.mod h1:a3+8EUD1SX5RUcCs3MY5YasiO1z6yLiNLRiFrykbynY= +cloud.google.com/go/area120 v0.7.1/go.mod h1:j84i4E1RboTWjKtZVWXPqvK5VHQFJRF2c1Nm69pWm9k= +cloud.google.com/go/artifactregistry v1.6.0/go.mod h1:IYt0oBPSAGYj/kprzsBjZ/4LnG/zOcHyFHjWPCi6SAQ= +cloud.google.com/go/artifactregistry v1.7.0/go.mod h1:mqTOFOnGZx8EtSqK/ZWcsm/4U8B77rbcLP6ruDU2Ixk= +cloud.google.com/go/artifactregistry v1.8.0/go.mod h1:w3GQXkJX8hiKN0v+at4b0qotwijQbYUqF2GWkZzAhC0= +cloud.google.com/go/artifactregistry v1.9.0/go.mod h1:2K2RqvA2CYvAeARHRkLDhMDJ3OXy26h3XW+3/Jh2uYc= +cloud.google.com/go/artifactregistry v1.11.1/go.mod h1:lLYghw+Itq9SONbCa1YWBoWs1nOucMH0pwXN1rOBZFI= +cloud.google.com/go/artifactregistry v1.11.2/go.mod h1:nLZns771ZGAwVLzTX/7Al6R9ehma4WUEhZGWV6CeQNQ= +cloud.google.com/go/artifactregistry v1.12.0/go.mod h1:o6P3MIvtzTOnmvGagO9v/rOjjA0HmhJ+/6KAXrmYDCI= +cloud.google.com/go/artifactregistry v1.13.0/go.mod h1:uy/LNfoOIivepGhooAUpL1i30Hgee3Cu0l4VTWHUC08= +cloud.google.com/go/asset v1.5.0/go.mod h1:5mfs8UvcM5wHhqtSv8J1CtxxaQq3AdBxxQi2jGW/K4o= +cloud.google.com/go/asset v1.7.0/go.mod h1:YbENsRK4+xTiL+Ofoj5Ckf+O17kJtgp3Y3nn4uzZz5s= +cloud.google.com/go/asset v1.8.0/go.mod h1:mUNGKhiqIdbr8X7KNayoYvyc4HbbFO9URsjbytpUaW0= +cloud.google.com/go/asset v1.9.0/go.mod h1:83MOE6jEJBMqFKadM9NLRcs80Gdw76qGuHn8m3h8oHQ= +cloud.google.com/go/asset v1.10.0/go.mod h1:pLz7uokL80qKhzKr4xXGvBQXnzHn5evJAEAtZiIb0wY= +cloud.google.com/go/asset v1.11.1/go.mod h1:fSwLhbRvC9p9CXQHJ3BgFeQNM4c9x10lqlrdEUYXlJo= +cloud.google.com/go/asset v1.12.0/go.mod h1:h9/sFOa4eDIyKmH6QMpm4eUK3pDojWnUhTgJlk762Hg= +cloud.google.com/go/asset v1.13.0/go.mod h1:WQAMyYek/b7NBpYq/K4KJWcRqzoalEsxz/t/dTk4THw= +cloud.google.com/go/assuredworkloads v1.5.0/go.mod h1:n8HOZ6pff6re5KYfBXcFvSViQjDwxFkAkmUFffJRbbY= +cloud.google.com/go/assuredworkloads v1.6.0/go.mod h1:yo2YOk37Yc89Rsd5QMVECvjaMKymF9OP+QXWlKXUkXw= +cloud.google.com/go/assuredworkloads v1.7.0/go.mod h1:z/736/oNmtGAyU47reJgGN+KVoYoxeLBoj4XkKYscNI= +cloud.google.com/go/assuredworkloads v1.8.0/go.mod h1:AsX2cqyNCOvEQC8RMPnoc0yEarXQk6WEKkxYfL6kGIo= +cloud.google.com/go/assuredworkloads v1.9.0/go.mod h1:kFuI1P78bplYtT77Tb1hi0FMxM0vVpRC7VVoJC3ZoT0= +cloud.google.com/go/assuredworkloads v1.10.0/go.mod h1:kwdUQuXcedVdsIaKgKTp9t0UJkE5+PAVNhdQm4ZVq2E= +cloud.google.com/go/automl v1.5.0/go.mod h1:34EjfoFGMZ5sgJ9EoLsRtdPSNZLcfflJR39VbVNS2M0= +cloud.google.com/go/automl v1.6.0/go.mod h1:ugf8a6Fx+zP0D59WLhqgTDsQI9w07o64uf/Is3Nh5p8= +cloud.google.com/go/automl v1.7.0/go.mod h1:RL9MYCCsJEOmt0Wf3z9uzG0a7adTT1fe+aObgSpkCt8= +cloud.google.com/go/automl v1.8.0/go.mod h1:xWx7G/aPEe/NP+qzYXktoBSDfjO+vnKMGgsApGJJquM= +cloud.google.com/go/automl v1.12.0/go.mod h1:tWDcHDp86aMIuHmyvjuKeeHEGq76lD7ZqfGLN6B0NuU= +cloud.google.com/go/baremetalsolution v0.3.0/go.mod h1:XOrocE+pvK1xFfleEnShBlNAXf+j5blPPxrhjKgnIFc= +cloud.google.com/go/baremetalsolution v0.4.0/go.mod h1:BymplhAadOO/eBa7KewQ0Ppg4A4Wplbn+PsFKRLo0uI= +cloud.google.com/go/baremetalsolution v0.5.0/go.mod h1:dXGxEkmR9BMwxhzBhV0AioD0ULBmuLZI8CdwalUxuss= +cloud.google.com/go/batch v0.3.0/go.mod h1:TR18ZoAekj1GuirsUsR1ZTKN3FC/4UDnScjT8NXImFE= +cloud.google.com/go/batch v0.4.0/go.mod h1:WZkHnP43R/QCGQsZ+0JyG4i79ranE2u8xvjq/9+STPE= +cloud.google.com/go/batch v0.7.0/go.mod h1:vLZN95s6teRUqRQ4s3RLDsH8PvboqBK+rn1oevL159g= +cloud.google.com/go/beyondcorp v0.2.0/go.mod h1:TB7Bd+EEtcw9PCPQhCJtJGjk/7TC6ckmnSFS+xwTfm4= +cloud.google.com/go/beyondcorp v0.3.0/go.mod h1:E5U5lcrcXMsCuoDNyGrpyTm/hn7ne941Jz2vmksAxW8= +cloud.google.com/go/beyondcorp v0.4.0/go.mod h1:3ApA0mbhHx6YImmuubf5pyW8srKnCEPON32/5hj+RmM= +cloud.google.com/go/beyondcorp v0.5.0/go.mod h1:uFqj9X+dSfrheVp7ssLTaRHd2EHqSL4QZmH4e8WXGGU= +cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= +cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= +cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= +cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= +cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= +cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= +cloud.google.com/go/bigquery v1.42.0/go.mod h1:8dRTJxhtG+vwBKzE5OseQn/hiydoQN3EedCaOdYmxRA= +cloud.google.com/go/bigquery v1.43.0/go.mod h1:ZMQcXHsl+xmU1z36G2jNGZmKp9zNY5BUua5wDgmNCfw= +cloud.google.com/go/bigquery v1.44.0/go.mod h1:0Y33VqXTEsbamHJvJHdFmtqHvMIY28aK1+dFsvaChGc= +cloud.google.com/go/bigquery v1.47.0/go.mod h1:sA9XOgy0A8vQK9+MWhEQTY6Tix87M/ZurWFIxmF9I/E= +cloud.google.com/go/bigquery v1.48.0/go.mod h1:QAwSz+ipNgfL5jxiaK7weyOhzdoAy1zFm0Nf1fysJac= +cloud.google.com/go/bigquery v1.49.0/go.mod h1:Sv8hMmTFFYBlt/ftw2uN6dFdQPzBlREY9yBh7Oy7/4Q= +cloud.google.com/go/bigquery v1.50.0/go.mod h1:YrleYEh2pSEbgTBZYMJ5SuSr0ML3ypjRB1zgf7pvQLU= +cloud.google.com/go/billing v1.4.0/go.mod h1:g9IdKBEFlItS8bTtlrZdVLWSSdSyFUZKXNS02zKMOZY= +cloud.google.com/go/billing v1.5.0/go.mod h1:mztb1tBc3QekhjSgmpf/CV4LzWXLzCArwpLmP2Gm88s= +cloud.google.com/go/billing v1.6.0/go.mod h1:WoXzguj+BeHXPbKfNWkqVtDdzORazmCjraY+vrxcyvI= +cloud.google.com/go/billing v1.7.0/go.mod h1:q457N3Hbj9lYwwRbnlD7vUpyjq6u5U1RAOArInEiD5Y= +cloud.google.com/go/billing v1.12.0/go.mod h1:yKrZio/eu+okO/2McZEbch17O5CB5NpZhhXG6Z766ss= +cloud.google.com/go/billing v1.13.0/go.mod h1:7kB2W9Xf98hP9Sr12KfECgfGclsH3CQR0R08tnRlRbc= +cloud.google.com/go/binaryauthorization v1.1.0/go.mod h1:xwnoWu3Y84jbuHa0zd526MJYmtnVXn0syOjaJgy4+dM= +cloud.google.com/go/binaryauthorization v1.2.0/go.mod h1:86WKkJHtRcv5ViNABtYMhhNWRrD1Vpi//uKEy7aYEfI= +cloud.google.com/go/binaryauthorization v1.3.0/go.mod h1:lRZbKgjDIIQvzYQS1p99A7/U1JqvqeZg0wiI5tp6tg0= +cloud.google.com/go/binaryauthorization v1.4.0/go.mod h1:tsSPQrBd77VLplV70GUhBf/Zm3FsKmgSqgm4UmiDItk= +cloud.google.com/go/binaryauthorization v1.5.0/go.mod h1:OSe4OU1nN/VswXKRBmciKpo9LulY41gch5c68htf3/Q= +cloud.google.com/go/certificatemanager v1.3.0/go.mod h1:n6twGDvcUBFu9uBgt4eYvvf3sQ6My8jADcOVwHmzadg= +cloud.google.com/go/certificatemanager v1.4.0/go.mod h1:vowpercVFyqs8ABSmrdV+GiFf2H/ch3KyudYQEMM590= +cloud.google.com/go/certificatemanager v1.6.0/go.mod h1:3Hh64rCKjRAX8dXgRAyOcY5vQ/fE1sh8o+Mdd6KPgY8= +cloud.google.com/go/channel v1.8.0/go.mod h1:W5SwCXDJsq/rg3tn3oG0LOxpAo6IMxNa09ngphpSlnk= +cloud.google.com/go/channel v1.9.0/go.mod h1:jcu05W0my9Vx4mt3/rEHpfxc9eKi9XwsdDL8yBMbKUk= +cloud.google.com/go/channel v1.11.0/go.mod h1:IdtI0uWGqhEeatSB62VOoJ8FSUhJ9/+iGkJVqp74CGE= +cloud.google.com/go/channel v1.12.0/go.mod h1:VkxCGKASi4Cq7TbXxlaBezonAYpp1GCnKMY6tnMQnLU= +cloud.google.com/go/cloudbuild v1.3.0/go.mod h1:WequR4ULxlqvMsjDEEEFnOG5ZSRSgWOywXYDb1vPE6U= +cloud.google.com/go/cloudbuild v1.4.0/go.mod h1:5Qwa40LHiOXmz3386FrjrYM93rM/hdRr7b53sySrTqA= +cloud.google.com/go/cloudbuild v1.6.0/go.mod h1:UIbc/w9QCbH12xX+ezUsgblrWv+Cv4Tw83GiSMHOn9M= +cloud.google.com/go/cloudbuild v1.7.0/go.mod h1:zb5tWh2XI6lR9zQmsm1VRA+7OCuve5d8S+zJUul8KTg= +cloud.google.com/go/cloudbuild v1.9.0/go.mod h1:qK1d7s4QlO0VwfYn5YuClDGg2hfmLZEb4wQGAbIgL1s= +cloud.google.com/go/clouddms v1.3.0/go.mod h1:oK6XsCDdW4Ib3jCCBugx+gVjevp2TMXFtgxvPSee3OM= +cloud.google.com/go/clouddms v1.4.0/go.mod h1:Eh7sUGCC+aKry14O1NRljhjyrr0NFC0G2cjwX0cByRk= +cloud.google.com/go/clouddms v1.5.0/go.mod h1:QSxQnhikCLUw13iAbffF2CZxAER3xDGNHjsTAkQJcQA= +cloud.google.com/go/cloudtasks v1.5.0/go.mod h1:fD92REy1x5woxkKEkLdvavGnPJGEn8Uic9nWuLzqCpY= +cloud.google.com/go/cloudtasks v1.6.0/go.mod h1:C6Io+sxuke9/KNRkbQpihnW93SWDU3uXt92nu85HkYI= +cloud.google.com/go/cloudtasks v1.7.0/go.mod h1:ImsfdYWwlWNJbdgPIIGJWC+gemEGTBK/SunNQQNCAb4= +cloud.google.com/go/cloudtasks v1.8.0/go.mod h1:gQXUIwCSOI4yPVK7DgTVFiiP0ZW/eQkydWzwVMdHxrI= +cloud.google.com/go/cloudtasks v1.9.0/go.mod h1:w+EyLsVkLWHcOaqNEyvcKAsWp9p29dL6uL9Nst1cI7Y= +cloud.google.com/go/cloudtasks v1.10.0/go.mod h1:NDSoTLkZ3+vExFEWu2UJV1arUyzVDAiZtdWcsUyNwBs= +cloud.google.com/go/compute v0.1.0/go.mod h1:GAesmwr110a34z04OlxYkATPBEfVhkymfTBXtfbBFow= +cloud.google.com/go/compute v1.3.0/go.mod h1:cCZiE1NHEtai4wiufUhW8I8S1JKkAnhnQJWM7YD99wM= +cloud.google.com/go/compute v1.5.0/go.mod h1:9SMHyhJlzhlkJqrPAc839t2BZFTSk6Jdj6mkzQJeu0M= +cloud.google.com/go/compute v1.6.0/go.mod h1:T29tfhtVbq1wvAPo0E3+7vhgmkOYeXjhFvz/FMzPu0s= +cloud.google.com/go/compute v1.6.1/go.mod h1:g85FgpzFvNULZ+S8AYq87axRKuf2Kh7deLqV/jJ3thU= +cloud.google.com/go/compute v1.7.0/go.mod h1:435lt8av5oL9P3fv1OEzSbSUe+ybHXGMPQHHZWZxy9U= +cloud.google.com/go/compute v1.10.0/go.mod h1:ER5CLbMxl90o2jtNbGSbtfOpQKR0t15FOtRsugnLrlU= +cloud.google.com/go/compute v1.12.0/go.mod h1:e8yNOBcBONZU1vJKCvCoDw/4JQsA0dpM4x/6PIIOocU= +cloud.google.com/go/compute v1.12.1/go.mod h1:e8yNOBcBONZU1vJKCvCoDw/4JQsA0dpM4x/6PIIOocU= +cloud.google.com/go/compute v1.13.0/go.mod h1:5aPTS0cUNMIc1CE546K+Th6weJUNQErARyZtRXDJ8GE= +cloud.google.com/go/compute v1.14.0/go.mod h1:YfLtxrj9sU4Yxv+sXzZkyPjEyPBZfXHUvjxega5vAdo= +cloud.google.com/go/compute v1.18.0/go.mod h1:1X7yHxec2Ga+Ss6jPyjxRxpu2uu7PLgsOVXvgU0yacs= +cloud.google.com/go/compute v1.19.0/go.mod h1:rikpw2y+UMidAe9tISo04EHNOIf42RLYF/q8Bs93scU= +cloud.google.com/go/compute v1.19.1/go.mod h1:6ylj3a05WF8leseCdIf77NK0g1ey+nj5IKd5/kvShxE= +cloud.google.com/go/compute/metadata v0.1.0/go.mod h1:Z1VN+bulIf6bt4P/C37K4DyZYZEXYonfTBHHFPO/4UU= +cloud.google.com/go/compute/metadata v0.2.0/go.mod h1:zFmK7XCadkQkj6TtorcaGlCW1hT1fIilQDwofLpJ20k= +cloud.google.com/go/compute/metadata v0.2.1/go.mod h1:jgHgmJd2RKBGzXqF5LR2EZMGxBkeanZ9wwa75XHJgOM= +cloud.google.com/go/compute/metadata v0.2.3/go.mod h1:VAV5nSsACxMJvgaAuX6Pk2AawlZn8kiOGuCv6gTkwuA= +cloud.google.com/go/contactcenterinsights v1.3.0/go.mod h1:Eu2oemoePuEFc/xKFPjbTuPSj0fYJcPls9TFlPNnHHY= +cloud.google.com/go/contactcenterinsights v1.4.0/go.mod h1:L2YzkGbPsv+vMQMCADxJoT9YiTTnSEd6fEvCeHTYVck= +cloud.google.com/go/contactcenterinsights v1.6.0/go.mod h1:IIDlT6CLcDoyv79kDv8iWxMSTZhLxSCofVV5W6YFM/w= +cloud.google.com/go/container v1.6.0/go.mod h1:Xazp7GjJSeUYo688S+6J5V+n/t+G5sKBTFkKNudGRxg= +cloud.google.com/go/container v1.7.0/go.mod h1:Dp5AHtmothHGX3DwwIHPgq45Y8KmNsgN3amoYfxVkLo= +cloud.google.com/go/container v1.13.1/go.mod h1:6wgbMPeQRw9rSnKBCAJXnds3Pzj03C4JHamr8asWKy4= +cloud.google.com/go/container v1.14.0/go.mod h1:3AoJMPhHfLDxLvrlVWaK57IXzaPnLaZq63WX59aQBfM= +cloud.google.com/go/container v1.15.0/go.mod h1:ft+9S0WGjAyjDggg5S06DXj+fHJICWg8L7isCQe9pQA= +cloud.google.com/go/containeranalysis v0.5.1/go.mod h1:1D92jd8gRR/c0fGMlymRgxWD3Qw9C1ff6/T7mLgVL8I= +cloud.google.com/go/containeranalysis v0.6.0/go.mod h1:HEJoiEIu+lEXM+k7+qLCci0h33lX3ZqoYFdmPcoO7s4= +cloud.google.com/go/containeranalysis v0.7.0/go.mod h1:9aUL+/vZ55P2CXfuZjS4UjQ9AgXoSw8Ts6lemfmxBxI= +cloud.google.com/go/containeranalysis v0.9.0/go.mod h1:orbOANbwk5Ejoom+s+DUCTTJ7IBdBQJDcSylAx/on9s= +cloud.google.com/go/datacatalog v1.3.0/go.mod h1:g9svFY6tuR+j+hrTw3J2dNcmI0dzmSiyOzm8kpLq0a0= +cloud.google.com/go/datacatalog v1.5.0/go.mod h1:M7GPLNQeLfWqeIm3iuiruhPzkt65+Bx8dAKvScX8jvs= +cloud.google.com/go/datacatalog v1.6.0/go.mod h1:+aEyF8JKg+uXcIdAmmaMUmZ3q1b/lKLtXCmXdnc0lbc= +cloud.google.com/go/datacatalog v1.7.0/go.mod h1:9mEl4AuDYWw81UGc41HonIHH7/sn52H0/tc8f8ZbZIE= +cloud.google.com/go/datacatalog v1.8.0/go.mod h1:KYuoVOv9BM8EYz/4eMFxrr4DUKhGIOXxZoKYF5wdISM= +cloud.google.com/go/datacatalog v1.8.1/go.mod h1:RJ58z4rMp3gvETA465Vg+ag8BGgBdnRPEMMSTr5Uv+M= +cloud.google.com/go/datacatalog v1.12.0/go.mod h1:CWae8rFkfp6LzLumKOnmVh4+Zle4A3NXLzVJ1d1mRm0= +cloud.google.com/go/datacatalog v1.13.0/go.mod h1:E4Rj9a5ZtAxcQJlEBTLgMTphfP11/lNaAshpoBgemX8= +cloud.google.com/go/dataflow v0.6.0/go.mod h1:9QwV89cGoxjjSR9/r7eFDqqjtvbKxAK2BaYU6PVk9UM= +cloud.google.com/go/dataflow v0.7.0/go.mod h1:PX526vb4ijFMesO1o202EaUmouZKBpjHsTlCtB4parQ= +cloud.google.com/go/dataflow v0.8.0/go.mod h1:Rcf5YgTKPtQyYz8bLYhFoIV/vP39eL7fWNcSOyFfLJE= +cloud.google.com/go/dataform v0.3.0/go.mod h1:cj8uNliRlHpa6L3yVhDOBrUXH+BPAO1+KFMQQNSThKo= +cloud.google.com/go/dataform v0.4.0/go.mod h1:fwV6Y4Ty2yIFL89huYlEkwUPtS7YZinZbzzj5S9FzCE= +cloud.google.com/go/dataform v0.5.0/go.mod h1:GFUYRe8IBa2hcomWplodVmUx/iTL0FrsauObOM3Ipr0= +cloud.google.com/go/dataform v0.6.0/go.mod h1:QPflImQy33e29VuapFdf19oPbE4aYTJxr31OAPV+ulA= +cloud.google.com/go/dataform v0.7.0/go.mod h1:7NulqnVozfHvWUBpMDfKMUESr+85aJsC/2O0o3jWPDE= +cloud.google.com/go/datafusion v1.4.0/go.mod h1:1Zb6VN+W6ALo85cXnM1IKiPw+yQMKMhB9TsTSRDo/38= +cloud.google.com/go/datafusion v1.5.0/go.mod h1:Kz+l1FGHB0J+4XF2fud96WMmRiq/wj8N9u007vyXZ2w= +cloud.google.com/go/datafusion v1.6.0/go.mod h1:WBsMF8F1RhSXvVM8rCV3AeyWVxcC2xY6vith3iw3S+8= +cloud.google.com/go/datalabeling v0.5.0/go.mod h1:TGcJ0G2NzcsXSE/97yWjIZO0bXj0KbVlINXMG9ud42I= +cloud.google.com/go/datalabeling v0.6.0/go.mod h1:WqdISuk/+WIGeMkpw/1q7bK/tFEZxsrFJOJdY2bXvTQ= +cloud.google.com/go/datalabeling v0.7.0/go.mod h1:WPQb1y08RJbmpM3ww0CSUAGweL0SxByuW2E+FU+wXcM= +cloud.google.com/go/dataplex v1.3.0/go.mod h1:hQuRtDg+fCiFgC8j0zV222HvzFQdRd+SVX8gdmFcZzA= +cloud.google.com/go/dataplex v1.4.0/go.mod h1:X51GfLXEMVJ6UN47ESVqvlsRplbLhcsAt0kZCCKsU0A= +cloud.google.com/go/dataplex v1.5.2/go.mod h1:cVMgQHsmfRoI5KFYq4JtIBEUbYwc3c7tXmIDhRmNNVQ= +cloud.google.com/go/dataplex v1.6.0/go.mod h1:bMsomC/aEJOSpHXdFKFGQ1b0TDPIeL28nJObeO1ppRs= +cloud.google.com/go/dataproc v1.7.0/go.mod h1:CKAlMjII9H90RXaMpSxQ8EU6dQx6iAYNPcYPOkSbi8s= +cloud.google.com/go/dataproc v1.8.0/go.mod h1:5OW+zNAH0pMpw14JVrPONsxMQYMBqJuzORhIBfBn9uI= +cloud.google.com/go/dataproc v1.12.0/go.mod h1:zrF3aX0uV3ikkMz6z4uBbIKyhRITnxvr4i3IjKsKrw4= +cloud.google.com/go/dataqna v0.5.0/go.mod h1:90Hyk596ft3zUQ8NkFfvICSIfHFh1Bc7C4cK3vbhkeo= +cloud.google.com/go/dataqna v0.6.0/go.mod h1:1lqNpM7rqNLVgWBJyk5NF6Uen2PHym0jtVJonplVsDA= +cloud.google.com/go/dataqna v0.7.0/go.mod h1:Lx9OcIIeqCrw1a6KdO3/5KMP1wAmTc0slZWwP12Qq3c= +cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= +cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= +cloud.google.com/go/datastore v1.10.0/go.mod h1:PC5UzAmDEkAmkfaknstTYbNpgE49HAgW2J1gcgUfmdM= +cloud.google.com/go/datastore v1.11.0/go.mod h1:TvGxBIHCS50u8jzG+AW/ppf87v1of8nwzFNgEZU1D3c= +cloud.google.com/go/datastream v1.2.0/go.mod h1:i/uTP8/fZwgATHS/XFu0TcNUhuA0twZxxQ3EyCUQMwo= +cloud.google.com/go/datastream v1.3.0/go.mod h1:cqlOX8xlyYF/uxhiKn6Hbv6WjwPPuI9W2M9SAXwaLLQ= +cloud.google.com/go/datastream v1.4.0/go.mod h1:h9dpzScPhDTs5noEMQVWP8Wx8AFBRyS0s8KWPx/9r0g= +cloud.google.com/go/datastream v1.5.0/go.mod h1:6TZMMNPwjUqZHBKPQ1wwXpb0d5VDVPl2/XoS5yi88q4= +cloud.google.com/go/datastream v1.6.0/go.mod h1:6LQSuswqLa7S4rPAOZFVjHIG3wJIjZcZrw8JDEDJuIs= +cloud.google.com/go/datastream v1.7.0/go.mod h1:uxVRMm2elUSPuh65IbZpzJNMbuzkcvu5CjMqVIUHrww= +cloud.google.com/go/deploy v1.4.0/go.mod h1:5Xghikd4VrmMLNaF6FiRFDlHb59VM59YoDQnOUdsH/c= +cloud.google.com/go/deploy v1.5.0/go.mod h1:ffgdD0B89tToyW/U/D2eL0jN2+IEV/3EMuXHA0l4r+s= +cloud.google.com/go/deploy v1.6.0/go.mod h1:f9PTHehG/DjCom3QH0cntOVRm93uGBDt2vKzAPwpXQI= +cloud.google.com/go/deploy v1.8.0/go.mod h1:z3myEJnA/2wnB4sgjqdMfgxCA0EqC3RBTNcVPs93mtQ= +cloud.google.com/go/dialogflow v1.15.0/go.mod h1:HbHDWs33WOGJgn6rfzBW1Kv807BE3O1+xGbn59zZWI4= +cloud.google.com/go/dialogflow v1.16.1/go.mod h1:po6LlzGfK+smoSmTBnbkIZY2w8ffjz/RcGSS+sh1el0= +cloud.google.com/go/dialogflow v1.17.0/go.mod h1:YNP09C/kXA1aZdBgC/VtXX74G/TKn7XVCcVumTflA+8= +cloud.google.com/go/dialogflow v1.18.0/go.mod h1:trO7Zu5YdyEuR+BhSNOqJezyFQ3aUzz0njv7sMx/iek= +cloud.google.com/go/dialogflow v1.19.0/go.mod h1:JVmlG1TwykZDtxtTXujec4tQ+D8SBFMoosgy+6Gn0s0= +cloud.google.com/go/dialogflow v1.29.0/go.mod h1:b+2bzMe+k1s9V+F2jbJwpHPzrnIyHihAdRFMtn2WXuM= +cloud.google.com/go/dialogflow v1.31.0/go.mod h1:cuoUccuL1Z+HADhyIA7dci3N5zUssgpBJmCzI6fNRB4= +cloud.google.com/go/dialogflow v1.32.0/go.mod h1:jG9TRJl8CKrDhMEcvfcfFkkpp8ZhgPz3sBGmAUYJ2qE= +cloud.google.com/go/dlp v1.6.0/go.mod h1:9eyB2xIhpU0sVwUixfBubDoRwP+GjeUoxxeueZmqvmM= +cloud.google.com/go/dlp v1.7.0/go.mod h1:68ak9vCiMBjbasxeVD17hVPxDEck+ExiHavX8kiHG+Q= +cloud.google.com/go/dlp v1.9.0/go.mod h1:qdgmqgTyReTz5/YNSSuueR8pl7hO0o9bQ39ZhtgkWp4= +cloud.google.com/go/documentai v1.7.0/go.mod h1:lJvftZB5NRiFSX4moiye1SMxHx0Bc3x1+p9e/RfXYiU= +cloud.google.com/go/documentai v1.8.0/go.mod h1:xGHNEB7CtsnySCNrCFdCyyMz44RhFEEX2Q7UD0c5IhU= +cloud.google.com/go/documentai v1.9.0/go.mod h1:FS5485S8R00U10GhgBC0aNGrJxBP8ZVpEeJ7PQDZd6k= +cloud.google.com/go/documentai v1.10.0/go.mod h1:vod47hKQIPeCfN2QS/jULIvQTugbmdc0ZvxxfQY1bg4= +cloud.google.com/go/documentai v1.16.0/go.mod h1:o0o0DLTEZ+YnJZ+J4wNfTxmDVyrkzFvttBXXtYRMHkM= +cloud.google.com/go/documentai v1.18.0/go.mod h1:F6CK6iUH8J81FehpskRmhLq/3VlwQvb7TvwOceQ2tbs= +cloud.google.com/go/domains v0.6.0/go.mod h1:T9Rz3GasrpYk6mEGHh4rymIhjlnIuB4ofT1wTxDeT4Y= +cloud.google.com/go/domains v0.7.0/go.mod h1:PtZeqS1xjnXuRPKE/88Iru/LdfoRyEHYA9nFQf4UKpg= +cloud.google.com/go/domains v0.8.0/go.mod h1:M9i3MMDzGFXsydri9/vW+EWz9sWb4I6WyHqdlAk0idE= +cloud.google.com/go/edgecontainer v0.1.0/go.mod h1:WgkZ9tp10bFxqO8BLPqv2LlfmQF1X8lZqwW4r1BTajk= +cloud.google.com/go/edgecontainer v0.2.0/go.mod h1:RTmLijy+lGpQ7BXuTDa4C4ssxyXT34NIuHIgKuP4s5w= +cloud.google.com/go/edgecontainer v0.3.0/go.mod h1:FLDpP4nykgwwIfcLt6zInhprzw0lEi2P1fjO6Ie0qbc= +cloud.google.com/go/edgecontainer v1.0.0/go.mod h1:cttArqZpBB2q58W/upSG++ooo6EsblxDIolxa3jSjbY= +cloud.google.com/go/errorreporting v0.3.0/go.mod h1:xsP2yaAp+OAW4OIm60An2bbLpqIhKXdWR/tawvl7QzU= +cloud.google.com/go/essentialcontacts v1.3.0/go.mod h1:r+OnHa5jfj90qIfZDO/VztSFqbQan7HV75p8sA+mdGI= +cloud.google.com/go/essentialcontacts v1.4.0/go.mod h1:8tRldvHYsmnBCHdFpvU+GL75oWiBKl80BiqlFh9tp+8= +cloud.google.com/go/essentialcontacts v1.5.0/go.mod h1:ay29Z4zODTuwliK7SnX8E86aUF2CTzdNtvv42niCX0M= +cloud.google.com/go/eventarc v1.7.0/go.mod h1:6ctpF3zTnaQCxUjHUdcfgcA1A2T309+omHZth7gDfmc= +cloud.google.com/go/eventarc v1.8.0/go.mod h1:imbzxkyAU4ubfsaKYdQg04WS1NvncblHEup4kvF+4gw= +cloud.google.com/go/eventarc v1.10.0/go.mod h1:u3R35tmZ9HvswGRBnF48IlYgYeBcPUCjkr4BTdem2Kw= +cloud.google.com/go/eventarc v1.11.0/go.mod h1:PyUjsUKPWoRBCHeOxZd/lbOOjahV41icXyUY5kSTvVY= +cloud.google.com/go/filestore v1.3.0/go.mod h1:+qbvHGvXU1HaKX2nD0WEPo92TP/8AQuCVEBXNY9z0+w= +cloud.google.com/go/filestore v1.4.0/go.mod h1:PaG5oDfo9r224f8OYXURtAsY+Fbyq/bLYoINEK8XQAI= +cloud.google.com/go/filestore v1.5.0/go.mod h1:FqBXDWBp4YLHqRnVGveOkHDf8svj9r5+mUDLupOWEDs= +cloud.google.com/go/filestore v1.6.0/go.mod h1:di5unNuss/qfZTw2U9nhFqo8/ZDSc466dre85Kydllg= +cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk= +cloud.google.com/go/firestore v1.9.0/go.mod h1:HMkjKHNTtRyZNiMzu7YAsLr9K3X2udY2AMwDaMEQiiE= +cloud.google.com/go/functions v1.6.0/go.mod h1:3H1UA3qiIPRWD7PeZKLvHZ9SaQhR26XIJcC0A5GbvAk= +cloud.google.com/go/functions v1.7.0/go.mod h1:+d+QBcWM+RsrgZfV9xo6KfA1GlzJfxcfZcRPEhDDfzg= +cloud.google.com/go/functions v1.8.0/go.mod h1:RTZ4/HsQjIqIYP9a9YPbU+QFoQsAlYgrwOXJWHn1POY= +cloud.google.com/go/functions v1.9.0/go.mod h1:Y+Dz8yGguzO3PpIjhLTbnqV1CWmgQ5UwtlpzoyquQ08= +cloud.google.com/go/functions v1.10.0/go.mod h1:0D3hEOe3DbEvCXtYOZHQZmD+SzYsi1YbI7dGvHfldXw= +cloud.google.com/go/functions v1.12.0/go.mod h1:AXWGrF3e2C/5ehvwYo/GH6O5s09tOPksiKhz+hH8WkA= +cloud.google.com/go/functions v1.13.0/go.mod h1:EU4O007sQm6Ef/PwRsI8N2umygGqPBS/IZQKBQBcJ3c= +cloud.google.com/go/gaming v1.5.0/go.mod h1:ol7rGcxP/qHTRQE/RO4bxkXq+Fix0j6D4LFPzYTIrDM= +cloud.google.com/go/gaming v1.6.0/go.mod h1:YMU1GEvA39Qt3zWGyAVA9bpYz/yAhTvaQ1t2sK4KPUA= +cloud.google.com/go/gaming v1.7.0/go.mod h1:LrB8U7MHdGgFG851iHAfqUdLcKBdQ55hzXy9xBJz0+w= +cloud.google.com/go/gaming v1.8.0/go.mod h1:xAqjS8b7jAVW0KFYeRUxngo9My3f33kFmua++Pi+ggM= +cloud.google.com/go/gaming v1.9.0/go.mod h1:Fc7kEmCObylSWLO334NcO+O9QMDyz+TKC4v1D7X+Bc0= +cloud.google.com/go/gkebackup v0.2.0/go.mod h1:XKvv/4LfG829/B8B7xRkk8zRrOEbKtEam6yNfuQNH60= +cloud.google.com/go/gkebackup v0.3.0/go.mod h1:n/E671i1aOQvUxT541aTkCwExO/bTer2HDlj4TsBRAo= +cloud.google.com/go/gkebackup v0.4.0/go.mod h1:byAyBGUwYGEEww7xsbnUTBHIYcOPy/PgUWUtOeRm9Vg= +cloud.google.com/go/gkeconnect v0.5.0/go.mod h1:c5lsNAg5EwAy7fkqX/+goqFsU1Da/jQFqArp+wGNr/o= +cloud.google.com/go/gkeconnect v0.6.0/go.mod h1:Mln67KyU/sHJEBY8kFZ0xTeyPtzbq9StAVvEULYK16A= +cloud.google.com/go/gkeconnect v0.7.0/go.mod h1:SNfmVqPkaEi3bF/B3CNZOAYPYdg7sU+obZ+QTky2Myw= +cloud.google.com/go/gkehub v0.9.0/go.mod h1:WYHN6WG8w9bXU0hqNxt8rm5uxnk8IH+lPY9J2TV7BK0= +cloud.google.com/go/gkehub v0.10.0/go.mod h1:UIPwxI0DsrpsVoWpLB0stwKCP+WFVG9+y977wO+hBH0= +cloud.google.com/go/gkehub v0.11.0/go.mod h1:JOWHlmN+GHyIbuWQPl47/C2RFhnFKH38jH9Ascu3n0E= +cloud.google.com/go/gkehub v0.12.0/go.mod h1:djiIwwzTTBrF5NaXCGv3mf7klpEMcST17VBTVVDcuaw= +cloud.google.com/go/gkemulticloud v0.3.0/go.mod h1:7orzy7O0S+5kq95e4Hpn7RysVA7dPs8W/GgfUtsPbrA= +cloud.google.com/go/gkemulticloud v0.4.0/go.mod h1:E9gxVBnseLWCk24ch+P9+B2CoDFJZTyIgLKSalC7tuI= +cloud.google.com/go/gkemulticloud v0.5.0/go.mod h1:W0JDkiyi3Tqh0TJr//y19wyb1yf8llHVto2Htf2Ja3Y= +cloud.google.com/go/grafeas v0.2.0/go.mod h1:KhxgtF2hb0P191HlY5besjYm6MqTSTj3LSI+M+ByZHc= +cloud.google.com/go/gsuiteaddons v1.3.0/go.mod h1:EUNK/J1lZEZO8yPtykKxLXI6JSVN2rg9bN8SXOa0bgM= +cloud.google.com/go/gsuiteaddons v1.4.0/go.mod h1:rZK5I8hht7u7HxFQcFei0+AtfS9uSushomRlg+3ua1o= +cloud.google.com/go/gsuiteaddons v1.5.0/go.mod h1:TFCClYLd64Eaa12sFVmUyG62tk4mdIsI7pAnSXRkcFo= +cloud.google.com/go/iam v0.1.0/go.mod h1:vcUNEa0pEm0qRVpmWepWaFMIAI8/hjB9mO8rNCJtF6c= +cloud.google.com/go/iam v0.3.0/go.mod h1:XzJPvDayI+9zsASAFO68Hk07u3z+f+JrT2xXNdp4bnY= +cloud.google.com/go/iam v0.5.0/go.mod h1:wPU9Vt0P4UmCux7mqtRu6jcpPAb74cP1fh50J3QpkUc= +cloud.google.com/go/iam v0.6.0/go.mod h1:+1AH33ueBne5MzYccyMHtEKqLE4/kJOibtffMHDMFMc= +cloud.google.com/go/iam v0.7.0/go.mod h1:H5Br8wRaDGNc8XP3keLc4unfUUZeyH3Sfl9XpQEYOeg= +cloud.google.com/go/iam v0.8.0/go.mod h1:lga0/y3iH6CX7sYqypWJ33hf7kkfXJag67naqGESjkE= +cloud.google.com/go/iam v0.11.0/go.mod h1:9PiLDanza5D+oWFZiH1uG+RnRCfEGKoyl6yo4cgWZGY= +cloud.google.com/go/iam v0.12.0/go.mod h1:knyHGviacl11zrtZUoDuYpDgLjvr28sLQaG0YB2GYAY= +cloud.google.com/go/iam v0.13.0/go.mod h1:ljOg+rcNfzZ5d6f1nAUJ8ZIxOaZUVoS14bKCtaLZ/D0= +cloud.google.com/go/iap v1.4.0/go.mod h1:RGFwRJdihTINIe4wZ2iCP0zF/qu18ZwyKxrhMhygBEc= +cloud.google.com/go/iap v1.5.0/go.mod h1:UH/CGgKd4KyohZL5Pt0jSKE4m3FR51qg6FKQ/z/Ix9A= +cloud.google.com/go/iap v1.6.0/go.mod h1:NSuvI9C/j7UdjGjIde7t7HBz+QTwBcapPE07+sSRcLk= +cloud.google.com/go/iap v1.7.0/go.mod h1:beqQx56T9O1G1yNPph+spKpNibDlYIiIixiqsQXxLIo= +cloud.google.com/go/iap v1.7.1/go.mod h1:WapEwPc7ZxGt2jFGB/C/bm+hP0Y6NXzOYGjpPnmMS74= +cloud.google.com/go/ids v1.1.0/go.mod h1:WIuwCaYVOzHIj2OhN9HAwvW+DBdmUAdcWlFxRl+KubM= +cloud.google.com/go/ids v1.2.0/go.mod h1:5WXvp4n25S0rA/mQWAg1YEEBBq6/s+7ml1RDCW1IrcY= +cloud.google.com/go/ids v1.3.0/go.mod h1:JBdTYwANikFKaDP6LtW5JAi4gubs57SVNQjemdt6xV4= +cloud.google.com/go/iot v1.3.0/go.mod h1:r7RGh2B61+B8oz0AGE+J72AhA0G7tdXItODWsaA2oLs= +cloud.google.com/go/iot v1.4.0/go.mod h1:dIDxPOn0UvNDUMD8Ger7FIaTuvMkj+aGk94RPP0iV+g= +cloud.google.com/go/iot v1.5.0/go.mod h1:mpz5259PDl3XJthEmh9+ap0affn/MqNSP4My77Qql9o= +cloud.google.com/go/iot v1.6.0/go.mod h1:IqdAsmE2cTYYNO1Fvjfzo9po179rAtJeVGUvkLN3rLE= +cloud.google.com/go/kms v1.4.0/go.mod h1:fajBHndQ+6ubNw6Ss2sSd+SWvjL26RNo/dr7uxsnnOA= +cloud.google.com/go/kms v1.5.0/go.mod h1:QJS2YY0eJGBg3mnDfuaCyLauWwBJiHRboYxJ++1xJNg= +cloud.google.com/go/kms v1.6.0/go.mod h1:Jjy850yySiasBUDi6KFUwUv2n1+o7QZFyuUJg6OgjA0= +cloud.google.com/go/kms v1.8.0/go.mod h1:4xFEhYFqvW+4VMELtZyxomGSYtSQKzM178ylFW4jMAg= +cloud.google.com/go/kms v1.9.0/go.mod h1:qb1tPTgfF9RQP8e1wq4cLFErVuTJv7UsSC915J8dh3w= +cloud.google.com/go/kms v1.10.0/go.mod h1:ng3KTUtQQU9bPX3+QGLsflZIHlkbn8amFAMY63m8d24= +cloud.google.com/go/kms v1.10.1/go.mod h1:rIWk/TryCkR59GMC3YtHtXeLzd634lBbKenvyySAyYI= +cloud.google.com/go/language v1.4.0/go.mod h1:F9dRpNFQmJbkaop6g0JhSBXCNlO90e1KWx5iDdxbWic= +cloud.google.com/go/language v1.6.0/go.mod h1:6dJ8t3B+lUYfStgls25GusK04NLh3eDLQnWM3mdEbhI= +cloud.google.com/go/language v1.7.0/go.mod h1:DJ6dYN/W+SQOjF8e1hLQXMF21AkH2w9wiPzPCJa2MIE= +cloud.google.com/go/language v1.8.0/go.mod h1:qYPVHf7SPoNNiCL2Dr0FfEFNil1qi3pQEyygwpgVKB8= +cloud.google.com/go/language v1.9.0/go.mod h1:Ns15WooPM5Ad/5no/0n81yUetis74g3zrbeJBE+ptUY= +cloud.google.com/go/lifesciences v0.5.0/go.mod h1:3oIKy8ycWGPUyZDR/8RNnTOYevhaMLqh5vLUXs9zvT8= +cloud.google.com/go/lifesciences v0.6.0/go.mod h1:ddj6tSX/7BOnhxCSd3ZcETvtNr8NZ6t/iPhY2Tyfu08= +cloud.google.com/go/lifesciences v0.8.0/go.mod h1:lFxiEOMqII6XggGbOnKiyZ7IBwoIqA84ClvoezaA/bo= +cloud.google.com/go/logging v1.6.1/go.mod h1:5ZO0mHHbvm8gEmeEUHrmDlTDSu5imF6MUP9OfilNXBw= +cloud.google.com/go/logging v1.7.0/go.mod h1:3xjP2CjkM3ZkO73aj4ASA5wRPGGCRrPIAeNqVNkzY8M= +cloud.google.com/go/longrunning v0.1.1/go.mod h1:UUFxuDWkv22EuY93jjmDMFT5GPQKeFVJBIF6QlTqdsE= +cloud.google.com/go/longrunning v0.3.0/go.mod h1:qth9Y41RRSUE69rDcOn6DdK3HfQfsUI0YSmW3iIlLJc= +cloud.google.com/go/longrunning v0.4.1/go.mod h1:4iWDqhBZ70CvZ6BfETbvam3T8FMvLK+eFj0E6AaRQTo= +cloud.google.com/go/managedidentities v1.3.0/go.mod h1:UzlW3cBOiPrzucO5qWkNkh0w33KFtBJU281hacNvsdE= +cloud.google.com/go/managedidentities v1.4.0/go.mod h1:NWSBYbEMgqmbZsLIyKvxrYbtqOsxY1ZrGM+9RgDqInM= +cloud.google.com/go/managedidentities v1.5.0/go.mod h1:+dWcZ0JlUmpuxpIDfyP5pP5y0bLdRwOS4Lp7gMni/LA= +cloud.google.com/go/maps v0.1.0/go.mod h1:BQM97WGyfw9FWEmQMpZ5T6cpovXXSd1cGmFma94eubI= +cloud.google.com/go/maps v0.6.0/go.mod h1:o6DAMMfb+aINHz/p/jbcY+mYeXBoZoxTfdSQ8VAJaCw= +cloud.google.com/go/maps v0.7.0/go.mod h1:3GnvVl3cqeSvgMcpRlQidXsPYuDGQ8naBis7MVzpXsY= +cloud.google.com/go/mediatranslation v0.5.0/go.mod h1:jGPUhGTybqsPQn91pNXw0xVHfuJ3leR1wj37oU3y1f4= +cloud.google.com/go/mediatranslation v0.6.0/go.mod h1:hHdBCTYNigsBxshbznuIMFNe5QXEowAuNmmC7h8pu5w= +cloud.google.com/go/mediatranslation v0.7.0/go.mod h1:LCnB/gZr90ONOIQLgSXagp8XUW1ODs2UmUMvcgMfI2I= +cloud.google.com/go/memcache v1.4.0/go.mod h1:rTOfiGZtJX1AaFUrOgsMHX5kAzaTQ8azHiuDoTPzNsE= +cloud.google.com/go/memcache v1.5.0/go.mod h1:dk3fCK7dVo0cUU2c36jKb4VqKPS22BTkf81Xq617aWM= +cloud.google.com/go/memcache v1.6.0/go.mod h1:XS5xB0eQZdHtTuTF9Hf8eJkKtR3pVRCcvJwtm68T3rA= +cloud.google.com/go/memcache v1.7.0/go.mod h1:ywMKfjWhNtkQTxrWxCkCFkoPjLHPW6A7WOTVI8xy3LY= +cloud.google.com/go/memcache v1.9.0/go.mod h1:8oEyzXCu+zo9RzlEaEjHl4KkgjlNDaXbCQeQWlzNFJM= +cloud.google.com/go/metastore v1.5.0/go.mod h1:2ZNrDcQwghfdtCwJ33nM0+GrBGlVuh8rakL3vdPY3XY= +cloud.google.com/go/metastore v1.6.0/go.mod h1:6cyQTls8CWXzk45G55x57DVQ9gWg7RiH65+YgPsNh9s= +cloud.google.com/go/metastore v1.7.0/go.mod h1:s45D0B4IlsINu87/AsWiEVYbLaIMeUSoxlKKDqBGFS8= +cloud.google.com/go/metastore v1.8.0/go.mod h1:zHiMc4ZUpBiM7twCIFQmJ9JMEkDSyZS9U12uf7wHqSI= +cloud.google.com/go/metastore v1.10.0/go.mod h1:fPEnH3g4JJAk+gMRnrAnoqyv2lpUCqJPWOodSaf45Eo= +cloud.google.com/go/monitoring v1.7.0/go.mod h1:HpYse6kkGo//7p6sT0wsIC6IBDET0RhIsnmlA53dvEk= +cloud.google.com/go/monitoring v1.8.0/go.mod h1:E7PtoMJ1kQXWxPjB6mv2fhC5/15jInuulFdYYtlcvT4= +cloud.google.com/go/monitoring v1.12.0/go.mod h1:yx8Jj2fZNEkL/GYZyTLS4ZtZEZN8WtDEiEqG4kLK50w= +cloud.google.com/go/monitoring v1.13.0/go.mod h1:k2yMBAB1H9JT/QETjNkgdCGD9bPF712XiLTVr+cBrpw= +cloud.google.com/go/networkconnectivity v1.4.0/go.mod h1:nOl7YL8odKyAOtzNX73/M5/mGZgqqMeryi6UPZTk/rA= +cloud.google.com/go/networkconnectivity v1.5.0/go.mod h1:3GzqJx7uhtlM3kln0+x5wyFvuVH1pIBJjhCpjzSt75o= +cloud.google.com/go/networkconnectivity v1.6.0/go.mod h1:OJOoEXW+0LAxHh89nXd64uGG+FbQoeH8DtxCHVOMlaM= +cloud.google.com/go/networkconnectivity v1.7.0/go.mod h1:RMuSbkdbPwNMQjB5HBWD5MpTBnNm39iAVpC3TmsExt8= +cloud.google.com/go/networkconnectivity v1.10.0/go.mod h1:UP4O4sWXJG13AqrTdQCD9TnLGEbtNRqjuaaA7bNjF5E= +cloud.google.com/go/networkconnectivity v1.11.0/go.mod h1:iWmDD4QF16VCDLXUqvyspJjIEtBR/4zq5hwnY2X3scM= +cloud.google.com/go/networkmanagement v1.4.0/go.mod h1:Q9mdLLRn60AsOrPc8rs8iNV6OHXaGcDdsIQe1ohekq8= +cloud.google.com/go/networkmanagement v1.5.0/go.mod h1:ZnOeZ/evzUdUsnvRt792H0uYEnHQEMaz+REhhzJRcf4= +cloud.google.com/go/networkmanagement v1.6.0/go.mod h1:5pKPqyXjB/sgtvB5xqOemumoQNB7y95Q7S+4rjSOPYY= +cloud.google.com/go/networksecurity v0.5.0/go.mod h1:xS6fOCoqpVC5zx15Z/MqkfDwH4+m/61A3ODiDV1xmiQ= +cloud.google.com/go/networksecurity v0.6.0/go.mod h1:Q5fjhTr9WMI5mbpRYEbiexTzROf7ZbDzvzCrNl14nyU= +cloud.google.com/go/networksecurity v0.7.0/go.mod h1:mAnzoxx/8TBSyXEeESMy9OOYwo1v+gZ5eMRnsT5bC8k= +cloud.google.com/go/networksecurity v0.8.0/go.mod h1:B78DkqsxFG5zRSVuwYFRZ9Xz8IcQ5iECsNrPn74hKHU= +cloud.google.com/go/notebooks v1.2.0/go.mod h1:9+wtppMfVPUeJ8fIWPOq1UnATHISkGXGqTkxeieQ6UY= +cloud.google.com/go/notebooks v1.3.0/go.mod h1:bFR5lj07DtCPC7YAAJ//vHskFBxA5JzYlH68kXVdk34= +cloud.google.com/go/notebooks v1.4.0/go.mod h1:4QPMngcwmgb6uw7Po99B2xv5ufVoIQ7nOGDyL4P8AgA= +cloud.google.com/go/notebooks v1.5.0/go.mod h1:q8mwhnP9aR8Hpfnrc5iN5IBhrXUy8S2vuYs+kBJ/gu0= +cloud.google.com/go/notebooks v1.7.0/go.mod h1:PVlaDGfJgj1fl1S3dUwhFMXFgfYGhYQt2164xOMONmE= +cloud.google.com/go/notebooks v1.8.0/go.mod h1:Lq6dYKOYOWUCTvw5t2q1gp1lAp0zxAxRycayS0iJcqQ= +cloud.google.com/go/optimization v1.1.0/go.mod h1:5po+wfvX5AQlPznyVEZjGJTMr4+CAkJf2XSTQOOl9l4= +cloud.google.com/go/optimization v1.2.0/go.mod h1:Lr7SOHdRDENsh+WXVmQhQTrzdu9ybg0NecjHidBq6xs= +cloud.google.com/go/optimization v1.3.1/go.mod h1:IvUSefKiwd1a5p0RgHDbWCIbDFgKuEdB+fPPuP0IDLI= +cloud.google.com/go/orchestration v1.3.0/go.mod h1:Sj5tq/JpWiB//X/q3Ngwdl5K7B7Y0KZ7bfv0wL6fqVA= +cloud.google.com/go/orchestration v1.4.0/go.mod h1:6W5NLFWs2TlniBphAViZEVhrXRSMgUGDfW7vrWKvsBk= +cloud.google.com/go/orchestration v1.6.0/go.mod h1:M62Bevp7pkxStDfFfTuCOaXgaaqRAga1yKyoMtEoWPQ= +cloud.google.com/go/orgpolicy v1.4.0/go.mod h1:xrSLIV4RePWmP9P3tBl8S93lTmlAxjm06NSm2UTmKvE= +cloud.google.com/go/orgpolicy v1.5.0/go.mod h1:hZEc5q3wzwXJaKrsx5+Ewg0u1LxJ51nNFlext7Tanwc= +cloud.google.com/go/orgpolicy v1.10.0/go.mod h1:w1fo8b7rRqlXlIJbVhOMPrwVljyuW5mqssvBtU18ONc= +cloud.google.com/go/osconfig v1.7.0/go.mod h1:oVHeCeZELfJP7XLxcBGTMBvRO+1nQ5tFG9VQTmYS2Fs= +cloud.google.com/go/osconfig v1.8.0/go.mod h1:EQqZLu5w5XA7eKizepumcvWx+m8mJUhEwiPqWiZeEdg= +cloud.google.com/go/osconfig v1.9.0/go.mod h1:Yx+IeIZJ3bdWmzbQU4fxNl8xsZ4amB+dygAwFPlvnNo= +cloud.google.com/go/osconfig v1.10.0/go.mod h1:uMhCzqC5I8zfD9zDEAfvgVhDS8oIjySWh+l4WK6GnWw= +cloud.google.com/go/osconfig v1.11.0/go.mod h1:aDICxrur2ogRd9zY5ytBLV89KEgT2MKB2L/n6x1ooPw= +cloud.google.com/go/oslogin v1.4.0/go.mod h1:YdgMXWRaElXz/lDk1Na6Fh5orF7gvmJ0FGLIs9LId4E= +cloud.google.com/go/oslogin v1.5.0/go.mod h1:D260Qj11W2qx/HVF29zBg+0fd6YCSjSqLUkY/qEenQU= +cloud.google.com/go/oslogin v1.6.0/go.mod h1:zOJ1O3+dTU8WPlGEkFSh7qeHPPSoxrcMbbK1Nm2iX70= +cloud.google.com/go/oslogin v1.7.0/go.mod h1:e04SN0xO1UNJ1M5GP0vzVBFicIe4O53FOfcixIqTyXo= +cloud.google.com/go/oslogin v1.9.0/go.mod h1:HNavntnH8nzrn8JCTT5fj18FuJLFJc4NaZJtBnQtKFs= +cloud.google.com/go/phishingprotection v0.5.0/go.mod h1:Y3HZknsK9bc9dMi+oE8Bim0lczMU6hrX0UpADuMefr0= +cloud.google.com/go/phishingprotection v0.6.0/go.mod h1:9Y3LBLgy0kDTcYET8ZH3bq/7qni15yVUoAxiFxnlSUA= +cloud.google.com/go/phishingprotection v0.7.0/go.mod h1:8qJI4QKHoda/sb/7/YmMQ2omRLSLYSu9bU0EKCNI+Lk= +cloud.google.com/go/policytroubleshooter v1.3.0/go.mod h1:qy0+VwANja+kKrjlQuOzmlvscn4RNsAc0e15GGqfMxg= +cloud.google.com/go/policytroubleshooter v1.4.0/go.mod h1:DZT4BcRw3QoO8ota9xw/LKtPa8lKeCByYeKTIf/vxdE= +cloud.google.com/go/policytroubleshooter v1.5.0/go.mod h1:Rz1WfV+1oIpPdN2VvvuboLVRsB1Hclg3CKQ53j9l8vw= +cloud.google.com/go/policytroubleshooter v1.6.0/go.mod h1:zYqaPTsmfvpjm5ULxAyD/lINQxJ0DDsnWOP/GZ7xzBc= +cloud.google.com/go/privatecatalog v0.5.0/go.mod h1:XgosMUvvPyxDjAVNDYxJ7wBW8//hLDDYmnsNcMGq1K0= +cloud.google.com/go/privatecatalog v0.6.0/go.mod h1:i/fbkZR0hLN29eEWiiwue8Pb+GforiEIBnV9yrRUOKI= +cloud.google.com/go/privatecatalog v0.7.0/go.mod h1:2s5ssIFO69F5csTXcwBP7NPFTZvps26xGzvQ2PQaBYg= +cloud.google.com/go/privatecatalog v0.8.0/go.mod h1:nQ6pfaegeDAq/Q5lrfCQzQLhubPiZhSaNhIgfJlnIXs= +cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= +cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= +cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= +cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU= +cloud.google.com/go/pubsub v1.26.0/go.mod h1:QgBH3U/jdJy/ftjPhTkyXNj543Tin1pRYcdcPRnFIRI= +cloud.google.com/go/pubsub v1.27.1/go.mod h1:hQN39ymbV9geqBnfQq6Xf63yNhUAhv9CZhzp5O6qsW0= +cloud.google.com/go/pubsub v1.28.0/go.mod h1:vuXFpwaVoIPQMGXqRyUQigu/AX1S3IWugR9xznmcXX8= +cloud.google.com/go/pubsub v1.30.0/go.mod h1:qWi1OPS0B+b5L+Sg6Gmc9zD1Y+HaM0MdUr7LsupY1P4= +cloud.google.com/go/pubsublite v1.5.0/go.mod h1:xapqNQ1CuLfGi23Yda/9l4bBCKz/wC3KIJ5gKcxveZg= +cloud.google.com/go/pubsublite v1.6.0/go.mod h1:1eFCS0U11xlOuMFV/0iBqw3zP12kddMeCbj/F3FSj9k= +cloud.google.com/go/pubsublite v1.7.0/go.mod h1:8hVMwRXfDfvGm3fahVbtDbiLePT3gpoiJYJY+vxWxVM= +cloud.google.com/go/recaptchaenterprise v1.3.1/go.mod h1:OdD+q+y4XGeAlxRaMn1Y7/GveP6zmq76byL6tjPE7d4= +cloud.google.com/go/recaptchaenterprise/v2 v2.1.0/go.mod h1:w9yVqajwroDNTfGuhmOjPDN//rZGySaf6PtFVcSCa7o= +cloud.google.com/go/recaptchaenterprise/v2 v2.2.0/go.mod h1:/Zu5jisWGeERrd5HnlS3EUGb/D335f9k51B/FVil0jk= +cloud.google.com/go/recaptchaenterprise/v2 v2.3.0/go.mod h1:O9LwGCjrhGHBQET5CA7dd5NwwNQUErSgEDit1DLNTdo= +cloud.google.com/go/recaptchaenterprise/v2 v2.4.0/go.mod h1:Am3LHfOuBstrLrNCBrlI5sbwx9LBg3te2N6hGvHn2mE= +cloud.google.com/go/recaptchaenterprise/v2 v2.5.0/go.mod h1:O8LzcHXN3rz0j+LBC91jrwI3R+1ZSZEWrfL7XHgNo9U= +cloud.google.com/go/recaptchaenterprise/v2 v2.6.0/go.mod h1:RPauz9jeLtB3JVzg6nCbe12qNoaa8pXc4d/YukAmcnA= +cloud.google.com/go/recaptchaenterprise/v2 v2.7.0/go.mod h1:19wVj/fs5RtYtynAPJdDTb69oW0vNHYDBTbB4NvMD9c= +cloud.google.com/go/recommendationengine v0.5.0/go.mod h1:E5756pJcVFeVgaQv3WNpImkFP8a+RptV6dDLGPILjvg= +cloud.google.com/go/recommendationengine v0.6.0/go.mod h1:08mq2umu9oIqc7tDy8sx+MNJdLG0fUi3vaSVbztHgJ4= +cloud.google.com/go/recommendationengine v0.7.0/go.mod h1:1reUcE3GIu6MeBz/h5xZJqNLuuVjNg1lmWMPyjatzac= +cloud.google.com/go/recommender v1.5.0/go.mod h1:jdoeiBIVrJe9gQjwd759ecLJbxCDED4A6p+mqoqDvTg= +cloud.google.com/go/recommender v1.6.0/go.mod h1:+yETpm25mcoiECKh9DEScGzIRyDKpZ0cEhWGo+8bo+c= +cloud.google.com/go/recommender v1.7.0/go.mod h1:XLHs/W+T8olwlGOgfQenXBTbIseGclClff6lhFVe9Bs= +cloud.google.com/go/recommender v1.8.0/go.mod h1:PkjXrTT05BFKwxaUxQmtIlrtj0kph108r02ZZQ5FE70= +cloud.google.com/go/recommender v1.9.0/go.mod h1:PnSsnZY7q+VL1uax2JWkt/UegHssxjUVVCrX52CuEmQ= +cloud.google.com/go/redis v1.7.0/go.mod h1:V3x5Jq1jzUcg+UNsRvdmsfuFnit1cfe3Z/PGyq/lm4Y= +cloud.google.com/go/redis v1.8.0/go.mod h1:Fm2szCDavWzBk2cDKxrkmWBqoCiL1+Ctwq7EyqBCA/A= +cloud.google.com/go/redis v1.9.0/go.mod h1:HMYQuajvb2D0LvMgZmLDZW8V5aOC/WxstZHiy4g8OiA= +cloud.google.com/go/redis v1.10.0/go.mod h1:ThJf3mMBQtW18JzGgh41/Wld6vnDDc/F/F35UolRZPM= +cloud.google.com/go/redis v1.11.0/go.mod h1:/X6eicana+BWcUda5PpwZC48o37SiFVTFSs0fWAJ7uQ= +cloud.google.com/go/resourcemanager v1.3.0/go.mod h1:bAtrTjZQFJkiWTPDb1WBjzvc6/kifjj4QBYuKCCoqKA= +cloud.google.com/go/resourcemanager v1.4.0/go.mod h1:MwxuzkumyTX7/a3n37gmsT3py7LIXwrShilPh3P1tR0= +cloud.google.com/go/resourcemanager v1.5.0/go.mod h1:eQoXNAiAvCf5PXxWxXjhKQoTMaUSNrEfg+6qdf/wots= +cloud.google.com/go/resourcemanager v1.6.0/go.mod h1:YcpXGRs8fDzcUl1Xw8uOVmI8JEadvhRIkoXXUNVYcVo= +cloud.google.com/go/resourcemanager v1.7.0/go.mod h1:HlD3m6+bwhzj9XCouqmeiGuni95NTrExfhoSrkC/3EI= +cloud.google.com/go/resourcesettings v1.3.0/go.mod h1:lzew8VfESA5DQ8gdlHwMrqZs1S9V87v3oCnKCWoOuQU= +cloud.google.com/go/resourcesettings v1.4.0/go.mod h1:ldiH9IJpcrlC3VSuCGvjR5of/ezRrOxFtpJoJo5SmXg= +cloud.google.com/go/resourcesettings v1.5.0/go.mod h1:+xJF7QSG6undsQDfsCJyqWXyBwUoJLhetkRMDRnIoXA= +cloud.google.com/go/retail v1.8.0/go.mod h1:QblKS8waDmNUhghY2TI9O3JLlFk8jybHeV4BF19FrE4= +cloud.google.com/go/retail v1.9.0/go.mod h1:g6jb6mKuCS1QKnH/dpu7isX253absFl6iE92nHwlBUY= +cloud.google.com/go/retail v1.10.0/go.mod h1:2gDk9HsL4HMS4oZwz6daui2/jmKvqShXKQuB2RZ+cCc= +cloud.google.com/go/retail v1.11.0/go.mod h1:MBLk1NaWPmh6iVFSz9MeKG/Psyd7TAgm6y/9L2B4x9Y= +cloud.google.com/go/retail v1.12.0/go.mod h1:UMkelN/0Z8XvKymXFbD4EhFJlYKRx1FGhQkVPU5kF14= +cloud.google.com/go/run v0.2.0/go.mod h1:CNtKsTA1sDcnqqIFR3Pb5Tq0usWxJJvsWOCPldRU3Do= +cloud.google.com/go/run v0.3.0/go.mod h1:TuyY1+taHxTjrD0ZFk2iAR+xyOXEA0ztb7U3UNA0zBo= +cloud.google.com/go/run v0.8.0/go.mod h1:VniEnuBwqjigv0A7ONfQUaEItaiCRVujlMqerPPiktM= +cloud.google.com/go/run v0.9.0/go.mod h1:Wwu+/vvg8Y+JUApMwEDfVfhetv30hCG4ZwDR/IXl2Qg= +cloud.google.com/go/scheduler v1.4.0/go.mod h1:drcJBmxF3aqZJRhmkHQ9b3uSSpQoltBPGPxGAWROx6s= +cloud.google.com/go/scheduler v1.5.0/go.mod h1:ri073ym49NW3AfT6DZi21vLZrG07GXr5p3H1KxN5QlI= +cloud.google.com/go/scheduler v1.6.0/go.mod h1:SgeKVM7MIwPn3BqtcBntpLyrIJftQISRrYB5ZtT+KOk= +cloud.google.com/go/scheduler v1.7.0/go.mod h1:jyCiBqWW956uBjjPMMuX09n3x37mtyPJegEWKxRsn44= +cloud.google.com/go/scheduler v1.8.0/go.mod h1:TCET+Y5Gp1YgHT8py4nlg2Sew8nUHMqcpousDgXJVQc= +cloud.google.com/go/scheduler v1.9.0/go.mod h1:yexg5t+KSmqu+njTIh3b7oYPheFtBWGcbVUYF1GGMIc= +cloud.google.com/go/secretmanager v1.6.0/go.mod h1:awVa/OXF6IiyaU1wQ34inzQNc4ISIDIrId8qE5QGgKA= +cloud.google.com/go/secretmanager v1.8.0/go.mod h1:hnVgi/bN5MYHd3Gt0SPuTPPp5ENina1/LxM+2W9U9J4= +cloud.google.com/go/secretmanager v1.9.0/go.mod h1:b71qH2l1yHmWQHt9LC80akm86mX8AL6X1MA01dW8ht4= +cloud.google.com/go/secretmanager v1.10.0/go.mod h1:MfnrdvKMPNra9aZtQFvBcvRU54hbPD8/HayQdlUgJpU= +cloud.google.com/go/security v1.5.0/go.mod h1:lgxGdyOKKjHL4YG3/YwIL2zLqMFCKs0UbQwgyZmfJl4= +cloud.google.com/go/security v1.7.0/go.mod h1:mZklORHl6Bg7CNnnjLH//0UlAlaXqiG7Lb9PsPXLfD0= +cloud.google.com/go/security v1.8.0/go.mod h1:hAQOwgmaHhztFhiQ41CjDODdWP0+AE1B3sX4OFlq+GU= +cloud.google.com/go/security v1.9.0/go.mod h1:6Ta1bO8LXI89nZnmnsZGp9lVoVWXqsVbIq/t9dzI+2Q= +cloud.google.com/go/security v1.10.0/go.mod h1:QtOMZByJVlibUT2h9afNDWRZ1G96gVywH8T5GUSb9IA= +cloud.google.com/go/security v1.12.0/go.mod h1:rV6EhrpbNHrrxqlvW0BWAIawFWq3X90SduMJdFwtLB8= +cloud.google.com/go/security v1.13.0/go.mod h1:Q1Nvxl1PAgmeW0y3HTt54JYIvUdtcpYKVfIB8AOMZ+0= +cloud.google.com/go/securitycenter v1.13.0/go.mod h1:cv5qNAqjY84FCN6Y9z28WlkKXyWsgLO832YiWwkCWcU= +cloud.google.com/go/securitycenter v1.14.0/go.mod h1:gZLAhtyKv85n52XYWt6RmeBdydyxfPeTrpToDPw4Auc= +cloud.google.com/go/securitycenter v1.15.0/go.mod h1:PeKJ0t8MoFmmXLXWm41JidyzI3PJjd8sXWaVqg43WWk= +cloud.google.com/go/securitycenter v1.16.0/go.mod h1:Q9GMaLQFUD+5ZTabrbujNWLtSLZIZF7SAR0wWECrjdk= +cloud.google.com/go/securitycenter v1.18.1/go.mod h1:0/25gAzCM/9OL9vVx4ChPeM/+DlfGQJDwBy/UC8AKK0= +cloud.google.com/go/securitycenter v1.19.0/go.mod h1:LVLmSg8ZkkyaNy4u7HCIshAngSQ8EcIRREP3xBnyfag= +cloud.google.com/go/servicecontrol v1.4.0/go.mod h1:o0hUSJ1TXJAmi/7fLJAedOovnujSEvjKCAFNXPQ1RaU= +cloud.google.com/go/servicecontrol v1.5.0/go.mod h1:qM0CnXHhyqKVuiZnGKrIurvVImCs8gmqWsDoqe9sU1s= +cloud.google.com/go/servicecontrol v1.10.0/go.mod h1:pQvyvSRh7YzUF2efw7H87V92mxU8FnFDawMClGCNuAA= +cloud.google.com/go/servicecontrol v1.11.0/go.mod h1:kFmTzYzTUIuZs0ycVqRHNaNhgR+UMUpw9n02l/pY+mc= +cloud.google.com/go/servicecontrol v1.11.1/go.mod h1:aSnNNlwEFBY+PWGQ2DoM0JJ/QUXqV5/ZD9DOLB7SnUk= +cloud.google.com/go/servicedirectory v1.4.0/go.mod h1:gH1MUaZCgtP7qQiI+F+A+OpeKF/HQWgtAddhTbhL2bs= +cloud.google.com/go/servicedirectory v1.5.0/go.mod h1:QMKFL0NUySbpZJ1UZs3oFAmdvVxhhxB6eJ/Vlp73dfg= +cloud.google.com/go/servicedirectory v1.6.0/go.mod h1:pUlbnWsLH9c13yGkxCmfumWEPjsRs1RlmJ4pqiNjVL4= +cloud.google.com/go/servicedirectory v1.7.0/go.mod h1:5p/U5oyvgYGYejufvxhgwjL8UVXjkuw7q5XcG10wx1U= +cloud.google.com/go/servicedirectory v1.8.0/go.mod h1:srXodfhY1GFIPvltunswqXpVxFPpZjf8nkKQT7XcXaY= +cloud.google.com/go/servicedirectory v1.9.0/go.mod h1:29je5JjiygNYlmsGz8k6o+OZ8vd4f//bQLtvzkPPT/s= +cloud.google.com/go/servicemanagement v1.4.0/go.mod h1:d8t8MDbezI7Z2R1O/wu8oTggo3BI2GKYbdG4y/SJTco= +cloud.google.com/go/servicemanagement v1.5.0/go.mod h1:XGaCRe57kfqu4+lRxaFEAuqmjzF0r+gWHjWqKqBvKFo= +cloud.google.com/go/servicemanagement v1.6.0/go.mod h1:aWns7EeeCOtGEX4OvZUWCCJONRZeFKiptqKf1D0l/Jc= +cloud.google.com/go/servicemanagement v1.8.0/go.mod h1:MSS2TDlIEQD/fzsSGfCdJItQveu9NXnUniTrq/L8LK4= +cloud.google.com/go/serviceusage v1.3.0/go.mod h1:Hya1cozXM4SeSKTAgGXgj97GlqUvF5JaoXacR1JTP/E= +cloud.google.com/go/serviceusage v1.4.0/go.mod h1:SB4yxXSaYVuUBYUml6qklyONXNLt83U0Rb+CXyhjEeU= +cloud.google.com/go/serviceusage v1.5.0/go.mod h1:w8U1JvqUqwJNPEOTQjrMHkw3IaIFLoLsPLvsE3xueec= +cloud.google.com/go/serviceusage v1.6.0/go.mod h1:R5wwQcbOWsyuOfbP9tGdAnCAc6B9DRwPG1xtWMDeuPA= +cloud.google.com/go/shell v1.3.0/go.mod h1:VZ9HmRjZBsjLGXusm7K5Q5lzzByZmJHf1d0IWHEN5X4= +cloud.google.com/go/shell v1.4.0/go.mod h1:HDxPzZf3GkDdhExzD/gs8Grqk+dmYcEjGShZgYa9URw= +cloud.google.com/go/shell v1.6.0/go.mod h1:oHO8QACS90luWgxP3N9iZVuEiSF84zNyLytb+qE2f9A= +cloud.google.com/go/spanner v1.41.0/go.mod h1:MLYDBJR/dY4Wt7ZaMIQ7rXOTLjYrmxLE/5ve9vFfWos= +cloud.google.com/go/spanner v1.44.0/go.mod h1:G8XIgYdOK+Fbcpbs7p2fiprDw4CaZX63whnSMLVBxjk= +cloud.google.com/go/spanner v1.45.0/go.mod h1:FIws5LowYz8YAE1J8fOS7DJup8ff7xJeetWEo5REA2M= +cloud.google.com/go/speech v1.6.0/go.mod h1:79tcr4FHCimOp56lwC01xnt/WPJZc4v3gzyT7FoBkCM= +cloud.google.com/go/speech v1.7.0/go.mod h1:KptqL+BAQIhMsj1kOP2la5DSEEerPDuOP/2mmkhHhZQ= +cloud.google.com/go/speech v1.8.0/go.mod h1:9bYIl1/tjsAnMgKGHKmBZzXKEkGgtU+MpdDPTE9f7y0= +cloud.google.com/go/speech v1.9.0/go.mod h1:xQ0jTcmnRFFM2RfX/U+rk6FQNUF6DQlydUSyoooSpco= +cloud.google.com/go/speech v1.14.1/go.mod h1:gEosVRPJ9waG7zqqnsHpYTOoAS4KouMRLDFMekpJ0J0= +cloud.google.com/go/speech v1.15.0/go.mod h1:y6oH7GhqCaZANH7+Oe0BhgIogsNInLlz542tg3VqeYI= +cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= +cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= +cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= +cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= +cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= +cloud.google.com/go/storage v1.14.0/go.mod h1:GrKmX003DSIwi9o29oFT7YDnHYwZoctc3fOKtUw0Xmo= +cloud.google.com/go/storage v1.22.1/go.mod h1:S8N1cAStu7BOeFfE8KAQzmyyLkK8p/vmRq6kuBTW58Y= +cloud.google.com/go/storage v1.23.0/go.mod h1:vOEEDNFnciUMhBeT6hsJIn3ieU5cFRmzeLgDvXzfIXc= +cloud.google.com/go/storage v1.27.0/go.mod h1:x9DOL8TK/ygDUMieqwfhdpQryTeEkhGKMi80i/iqR2s= +cloud.google.com/go/storage v1.28.1/go.mod h1:Qnisd4CqDdo6BGs2AD5LLnEsmSQ80wQ5ogcBBKhU86Y= +cloud.google.com/go/storage v1.29.0/go.mod h1:4puEjyTKnku6gfKoTfNOU/W+a9JyuVNxjpS5GBrB8h4= +cloud.google.com/go/storagetransfer v1.5.0/go.mod h1:dxNzUopWy7RQevYFHewchb29POFv3/AaBgnhqzqiK0w= +cloud.google.com/go/storagetransfer v1.6.0/go.mod h1:y77xm4CQV/ZhFZH75PLEXY0ROiS7Gh6pSKrM8dJyg6I= +cloud.google.com/go/storagetransfer v1.7.0/go.mod h1:8Giuj1QNb1kfLAiWM1bN6dHzfdlDAVC9rv9abHot2W4= +cloud.google.com/go/storagetransfer v1.8.0/go.mod h1:JpegsHHU1eXg7lMHkvf+KE5XDJ7EQu0GwNJbbVGanEw= +cloud.google.com/go/talent v1.1.0/go.mod h1:Vl4pt9jiHKvOgF9KoZo6Kob9oV4lwd/ZD5Cto54zDRw= +cloud.google.com/go/talent v1.2.0/go.mod h1:MoNF9bhFQbiJ6eFD3uSsg0uBALw4n4gaCaEjBw9zo8g= +cloud.google.com/go/talent v1.3.0/go.mod h1:CmcxwJ/PKfRgd1pBjQgU6W3YBwiewmUzQYH5HHmSCmM= +cloud.google.com/go/talent v1.4.0/go.mod h1:ezFtAgVuRf8jRsvyE6EwmbTK5LKciD4KVnHuDEFmOOA= +cloud.google.com/go/talent v1.5.0/go.mod h1:G+ODMj9bsasAEJkQSzO2uHQWXHHXUomArjWQQYkqK6c= +cloud.google.com/go/texttospeech v1.4.0/go.mod h1:FX8HQHA6sEpJ7rCMSfXuzBcysDAuWusNNNvN9FELDd8= +cloud.google.com/go/texttospeech v1.5.0/go.mod h1:oKPLhR4n4ZdQqWKURdwxMy0uiTS1xU161C8W57Wkea4= +cloud.google.com/go/texttospeech v1.6.0/go.mod h1:YmwmFT8pj1aBblQOI3TfKmwibnsfvhIBzPXcW4EBovc= +cloud.google.com/go/tpu v1.3.0/go.mod h1:aJIManG0o20tfDQlRIej44FcwGGl/cD0oiRyMKG19IQ= +cloud.google.com/go/tpu v1.4.0/go.mod h1:mjZaX8p0VBgllCzF6wcU2ovUXN9TONFLd7iz227X2Xg= +cloud.google.com/go/tpu v1.5.0/go.mod h1:8zVo1rYDFuW2l4yZVY0R0fb/v44xLh3llq7RuV61fPM= +cloud.google.com/go/trace v1.3.0/go.mod h1:FFUE83d9Ca57C+K8rDl/Ih8LwOzWIV1krKgxg6N0G28= +cloud.google.com/go/trace v1.4.0/go.mod h1:UG0v8UBqzusp+z63o7FK74SdFE+AXpCLdFb1rshXG+Y= +cloud.google.com/go/trace v1.8.0/go.mod h1:zH7vcsbAhklH8hWFig58HvxcxyQbaIqMarMg9hn5ECA= +cloud.google.com/go/trace v1.9.0/go.mod h1:lOQqpE5IaWY0Ixg7/r2SjixMuc6lfTFeO4QGM4dQWOk= +cloud.google.com/go/translate v1.3.0/go.mod h1:gzMUwRjvOqj5i69y/LYLd8RrNQk+hOmIXTi9+nb3Djs= +cloud.google.com/go/translate v1.4.0/go.mod h1:06Dn/ppvLD6WvA5Rhdp029IX2Mi3Mn7fpMRLPvXT5Wg= +cloud.google.com/go/translate v1.5.0/go.mod h1:29YDSYveqqpA1CQFD7NQuP49xymq17RXNaUDdc0mNu0= +cloud.google.com/go/translate v1.6.0/go.mod h1:lMGRudH1pu7I3n3PETiOB2507gf3HnfLV8qlkHZEyos= +cloud.google.com/go/translate v1.7.0/go.mod h1:lMGRudH1pu7I3n3PETiOB2507gf3HnfLV8qlkHZEyos= +cloud.google.com/go/video v1.8.0/go.mod h1:sTzKFc0bUSByE8Yoh8X0mn8bMymItVGPfTuUBUyRgxk= +cloud.google.com/go/video v1.9.0/go.mod h1:0RhNKFRF5v92f8dQt0yhaHrEuH95m068JYOvLZYnJSw= +cloud.google.com/go/video v1.12.0/go.mod h1:MLQew95eTuaNDEGriQdcYn0dTwf9oWiA4uYebxM5kdg= +cloud.google.com/go/video v1.13.0/go.mod h1:ulzkYlYgCp15N2AokzKjy7MQ9ejuynOJdf1tR5lGthk= +cloud.google.com/go/video v1.14.0/go.mod h1:SkgaXwT+lIIAKqWAJfktHT/RbgjSuY6DobxEp0C5yTQ= +cloud.google.com/go/video v1.15.0/go.mod h1:SkgaXwT+lIIAKqWAJfktHT/RbgjSuY6DobxEp0C5yTQ= +cloud.google.com/go/videointelligence v1.6.0/go.mod h1:w0DIDlVRKtwPCn/C4iwZIJdvC69yInhW0cfi+p546uU= +cloud.google.com/go/videointelligence v1.7.0/go.mod h1:k8pI/1wAhjznARtVT9U1llUaFNPh7muw8QyOUpavru4= +cloud.google.com/go/videointelligence v1.8.0/go.mod h1:dIcCn4gVDdS7yte/w+koiXn5dWVplOZkE+xwG9FgK+M= +cloud.google.com/go/videointelligence v1.9.0/go.mod h1:29lVRMPDYHikk3v8EdPSaL8Ku+eMzDljjuvRs105XoU= +cloud.google.com/go/videointelligence v1.10.0/go.mod h1:LHZngX1liVtUhZvi2uNS0VQuOzNi2TkY1OakiuoUOjU= +cloud.google.com/go/vision v1.2.0/go.mod h1:SmNwgObm5DpFBme2xpyOyasvBc1aPdjvMk2bBk0tKD0= +cloud.google.com/go/vision/v2 v2.2.0/go.mod h1:uCdV4PpN1S0jyCyq8sIM42v2Y6zOLkZs+4R9LrGYwFo= +cloud.google.com/go/vision/v2 v2.3.0/go.mod h1:UO61abBx9QRMFkNBbf1D8B1LXdS2cGiiCRx0vSpZoUo= +cloud.google.com/go/vision/v2 v2.4.0/go.mod h1:VtI579ll9RpVTrdKdkMzckdnwMyX2JILb+MhPqRbPsY= +cloud.google.com/go/vision/v2 v2.5.0/go.mod h1:MmaezXOOE+IWa+cS7OhRRLK2cNv1ZL98zhqFFZaaH2E= +cloud.google.com/go/vision/v2 v2.6.0/go.mod h1:158Hes0MvOS9Z/bDMSFpjwsUrZ5fPrdwuyyvKSGAGMY= +cloud.google.com/go/vision/v2 v2.7.0/go.mod h1:H89VysHy21avemp6xcf9b9JvZHVehWbET0uT/bcuY/0= +cloud.google.com/go/vmmigration v1.2.0/go.mod h1:IRf0o7myyWFSmVR1ItrBSFLFD/rJkfDCUTO4vLlJvsE= +cloud.google.com/go/vmmigration v1.3.0/go.mod h1:oGJ6ZgGPQOFdjHuocGcLqX4lc98YQ7Ygq8YQwHh9A7g= +cloud.google.com/go/vmmigration v1.5.0/go.mod h1:E4YQ8q7/4W9gobHjQg4JJSgXXSgY21nA5r8swQV+Xxc= +cloud.google.com/go/vmmigration v1.6.0/go.mod h1:bopQ/g4z+8qXzichC7GW1w2MjbErL54rk3/C843CjfY= +cloud.google.com/go/vmwareengine v0.1.0/go.mod h1:RsdNEf/8UDvKllXhMz5J40XxDrNJNN4sagiox+OI208= +cloud.google.com/go/vmwareengine v0.2.2/go.mod h1:sKdctNJxb3KLZkE/6Oui94iw/xs9PRNC2wnNLXsHvH8= +cloud.google.com/go/vmwareengine v0.3.0/go.mod h1:wvoyMvNWdIzxMYSpH/R7y2h5h3WFkx6d+1TIsP39WGY= +cloud.google.com/go/vpcaccess v1.4.0/go.mod h1:aQHVbTWDYUR1EbTApSVvMq1EnT57ppDmQzZ3imqIk4w= +cloud.google.com/go/vpcaccess v1.5.0/go.mod h1:drmg4HLk9NkZpGfCmZ3Tz0Bwnm2+DKqViEpeEpOq0m8= +cloud.google.com/go/vpcaccess v1.6.0/go.mod h1:wX2ILaNhe7TlVa4vC5xce1bCnqE3AeH27RV31lnmZes= +cloud.google.com/go/webrisk v1.4.0/go.mod h1:Hn8X6Zr+ziE2aNd8SliSDWpEnSS1u4R9+xXZmFiHmGE= +cloud.google.com/go/webrisk v1.5.0/go.mod h1:iPG6fr52Tv7sGk0H6qUFzmL3HHZev1htXuWDEEsqMTg= +cloud.google.com/go/webrisk v1.6.0/go.mod h1:65sW9V9rOosnc9ZY7A7jsy1zoHS5W9IAXv6dGqhMQMc= +cloud.google.com/go/webrisk v1.7.0/go.mod h1:mVMHgEYH0r337nmt1JyLthzMr6YxwN1aAIEc2fTcq7A= +cloud.google.com/go/webrisk v1.8.0/go.mod h1:oJPDuamzHXgUc+b8SiHRcVInZQuybnvEW72PqTc7sSg= +cloud.google.com/go/websecurityscanner v1.3.0/go.mod h1:uImdKm2wyeXQevQJXeh8Uun/Ym1VqworNDlBXQevGMo= +cloud.google.com/go/websecurityscanner v1.4.0/go.mod h1:ebit/Fp0a+FWu5j4JOmJEV8S8CzdTkAS77oDsiSqYWQ= +cloud.google.com/go/websecurityscanner v1.5.0/go.mod h1:Y6xdCPy81yi0SQnDY1xdNTNpfY1oAgXUlcfN3B3eSng= +cloud.google.com/go/workflows v1.6.0/go.mod h1:6t9F5h/unJz41YqfBmqSASJSXccBLtD1Vwf+KmJENM0= +cloud.google.com/go/workflows v1.7.0/go.mod h1:JhSrZuVZWuiDfKEFxU0/F1PQjmpnpcoISEXH2bcHC3M= +cloud.google.com/go/workflows v1.8.0/go.mod h1:ysGhmEajwZxGn1OhGOGKsTXc5PyxOc0vfKf5Af+to4M= +cloud.google.com/go/workflows v1.9.0/go.mod h1:ZGkj1aFIOd9c8Gerkjjq7OW7I5+l6cSvT3ujaO/WwSA= +cloud.google.com/go/workflows v1.10.0/go.mod h1:fZ8LmRmZQWacon9UCX1r/g/DfAXx5VcPALq2CxzdePw= +cosmossdk.io/errors v1.0.2 h1:wcYiJz08HThbWxd/L4jObeLaLySopyyuUFB5w4AGpCo= +cosmossdk.io/errors v1.0.2/go.mod h1:0rjgiHkftRYPj//3DrD6y8hcm40HcPv/dR4R/4efr0k= +dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= +filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA= +filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4= +gioui.org v0.0.0-20210308172011-57750fc8a0a6/go.mod h1:RSH6KIUZ0p2xy5zHDxgAM4zumjgTw83q2ge/PI+yyw8= +git.sr.ht/~sbinet/gg v0.3.1/go.mod h1:KGYtlADtqsqANL9ueOFkWymvzUvLMQllU5Ixo+8v3pc= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.0.0/go.mod h1:uGG2W01BaETf0Ozp+QxxKJdMBNRWPdstHG0Fmdwn1/U= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.6.0/go.mod h1:bjGvMhVMb+EEm3VRNQawDMUyMMjo+S5ewNjflkep/0Q= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.0/go.mod h1:bjGvMhVMb+EEm3VRNQawDMUyMMjo+S5ewNjflkep/0Q= +github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.0.0/go.mod h1:+6sju8gk8FRmSajX3Oz4G5Gm7P+mbqE9FVaXXFYTkCM= +github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.0/go.mod h1:OQeznEEkTZ9OrhHJoDD8ZDq51FHgXjqtP9z6bEwBq9U= +github.com/Azure/azure-sdk-for-go/sdk/internal v1.0.0/go.mod h1:eWRD7oawr1Mu1sLCawqVc0CUiF43ia3qQMxLscsKQ9w= +github.com/Azure/azure-sdk-for-go/sdk/internal v1.3.0/go.mod h1:okt5dMMTOFjX/aovMlrjvvXoPMBVSPzk9185BT0+eZM= +github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal v1.0.0/go.mod h1:ceIuwmxDWptoW3eCqSXlnPsZFKh4X+R38dWPv7GS9Vs= +github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources v1.0.0/go.mod h1:s1tW/At+xHqjNFvWU4G0c0Qv33KOhvbGNj0RCTQDV8s= +github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.2.0/go.mod h1:c+Lifp3EDEamAkPVzMooRNOK6CZjNSdEnf1A7jsI9u4= +github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.2.0/go.mod h1:+6KLcKIVgxoBDMqMO/Nvy7bZ9a0nbU3I1DtFQK3YvB4= +github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0= +github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= +github.com/AzureAD/microsoft-authentication-library-for-go v0.4.0/go.mod h1:Vt9sXTKwMyGcOxSmLDMnGPgqsUg7m8pe215qMLrDXw4= +github.com/AzureAD/microsoft-authentication-library-for-go v1.0.0/go.mod h1:kgDmCTgBzIEPFElEF+FK0SdjAor06dRq2Go927dnQ6o= +github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= +github.com/BurntSushi/toml v1.2.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ= +github.com/BurntSushi/toml v1.4.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho= +github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg= +github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho= +github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= +github.com/CloudyKit/fastprinter v0.0.0-20200109182630-33d98a066a53/go.mod h1:+3IMCy2vIlbG1XG/0ggNQv0SvxCAIpPM5b1nCz56Xno= +github.com/CloudyKit/jet/v6 v6.2.0/go.mod h1:d3ypHeIRNo2+XyqnGA8s+aphtcVpjP5hPwP/Lzo7Ro4= +github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ= +github.com/DataDog/zstd v1.4.1/go.mod h1:1jcaCB/ufaK+sKp1NBhlGmpz41jOoPQ35bpF36t7BBo= +github.com/DataDog/zstd v1.4.5/go.mod h1:1jcaCB/ufaK+sKp1NBhlGmpz41jOoPQ35bpF36t7BBo= +github.com/DataDog/zstd v1.5.7 h1:ybO8RBeh29qrxIhCA9E8gKY6xfONU9T6G6aP9DTKfLE= +github.com/DataDog/zstd v1.5.7/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw= +github.com/GoogleCloudPlatform/cloudsql-proxy v0.0.0-20190129172621-c8b1d7a94ddf/go.mod h1:aJ4qN3TfrelA6NZ6AXsXRfmEVaYin3EDbSPJrKS8OXo= +github.com/HdrHistogram/hdrhistogram-go v1.1.2/go.mod h1:yDgFjdqOqDEKOvasDdhWNXYg9BVp4O+o5f6V/ehm6Oo= +github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c/go.mod h1:X0CRv0ky0k6m906ixxpzmDRLvX58TFUKS2eePweuyxk= +github.com/Joker/hpp v1.0.0/go.mod h1:8x5n+M1Hp5hC0g8okX3sR3vFQwynaX/UgSOM9MeBKzY= +github.com/Joker/jade v1.1.3/go.mod h1:T+2WLyt7VH6Lp0TRxQrUYEs64nRc83wkMQrfeIQKduM= +github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible/go.mod h1:r7JcOSlj0wfOMncg0iLm8Leh48TZaKVeNIfJntJ2wa0= +github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= +github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= +github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 h1:TngWCqHvy9oXAN6lEVMRuU21PR1EtLVZJmdB18Gu3Rw= +github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5/go.mod h1:lmUJ/7eu/Q8D7ML55dXQrVaamCz2vxCfdQBasLZfHKk= +github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE= +github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= +github.com/RaduBerinde/axisds v0.0.0-20250419182453-5135a0650657 h1:8XBWWQD+vFF+JqOsm16t0Kab1a7YWV8+GISVEP8AuZ8= +github.com/RaduBerinde/axisds v0.0.0-20250419182453-5135a0650657/go.mod h1:UHGJonU9z4YYGKJxSaC6/TNcLOBptpmM5m2Cksbnw0Y= +github.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 h1:bsU8Tzxr/PNz75ayvCnxKZWEYdLMPDkUgticP4a4Bvk= +github.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54/go.mod h1:0tr7FllbE9gJkHq7CVeeDDFAFKQVy5RnCSSNBOvdqbc= +github.com/Shopify/goreferrer v0.0.0-20220729165902-8cddb4f5de06/go.mod h1:7erjKLwalezA0k99cWs5L11HWOAPNjdUZ6RxH1BXbbM= +github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo= +github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI= +github.com/StackExchange/wmi v1.2.1/go.mod h1:rcmrprowKIVzvc+NUiLncP2uuArMWLCbu9SBzvHz7e8= +github.com/VictoriaMetrics/fastcache v1.12.2 h1:N0y9ASrJ0F6h0QaC3o6uJb3NIZ9VKLjCM7NQbSmF7WI= +github.com/VictoriaMetrics/fastcache v1.12.2/go.mod h1:AmC+Nzz1+3G2eCPapF6UcsnkThDcMsQicp4xDukwJYI= +github.com/VividCortex/gohistogram v1.0.0 h1:6+hBz+qvs0JOrrNhhmR7lFxo5sINxBCGXrdtl/UvroE= +github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g= +github.com/Zilliqa/gozilliqa-sdk v1.2.1-0.20201201074141-dd0ecada1be6/go.mod h1:eSYp2T6f0apnuW8TzhV3f6Aff2SE8Dwio++U4ha4yEM= +github.com/aclements/go-gg v0.0.0-20170118225347-6dbb4e4fefb0/go.mod h1:55qNq4vcpkIuHowELi5C8e+1yUHtoLoOUR9QU5j7Tes= +github.com/aclements/go-moremath v0.0.0-20210112150236-f10218a38794/go.mod h1:7e+I0LQFUI9AXWxOfsQROs9xPhoJtbsyWcjJqDd4KPY= +github.com/aclements/go-perfevent v0.0.0-20240301234650-f7843625020f h1:JjxwchlOepwsUWcQwD2mLUAGE9aCp0/ehy6yCHFBOvo= +github.com/aclements/go-perfevent v0.0.0-20240301234650-f7843625020f/go.mod h1:tMDTce/yLLN/SK8gMOxQfnyeMeCg8KGzp0D1cbECEeo= +github.com/adlio/schema v1.3.9 h1:MLYk1VX1dn7xHW7Kdm1ywKKLjh19DRnrc65axS5xQA8= +github.com/adlio/schema v1.3.9/go.mod h1:GnxXztHzNh6pIc7qm3sw+jsmHrXgBy/x2RBSkKZ3L4w= +github.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII= +github.com/afex/hystrix-go v0.0.0-20180502004556-fa1af6a1f4f5/go.mod h1:SkGFH1ia65gfNATL8TAiHDNxPzPdmEL5uirI2Uyuz6c= +github.com/ajg/form v1.5.1/go.mod h1:uL1WgH+h2mgNtvBq0339dVnzXdBETtL2LeUXaIv25UY= +github.com/ajstarks/deck v0.0.0-20200831202436-30c9fc6549a9/go.mod h1:JynElWSGnm/4RlzPXRlREEwqTHAN3T56Bv2ITsFT3gY= +github.com/ajstarks/deck/generate v0.0.0-20210309230005-c3f852c02e19/go.mod h1:T13YZdzov6OU0A1+RfKZiZN9ca6VeKdBdyDV+BY97Tk= +github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af/go.mod h1:K08gAheRH3/J6wwsYMMT4xOr94bZjxIelGM0+d/wbFw= +github.com/ajstarks/svgo v0.0.0-20210923152817-c3b6e2f0c527/go.mod h1:K08gAheRH3/J6wwsYMMT4xOr94bZjxIelGM0+d/wbFw= +github.com/ajstarks/svgo v0.0.0-20211024235047-1546f124cd8b/go.mod h1:1KcenG0jGWcpt8ov532z81sp/kMMUG485J2InIOyADM= +github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= +github.com/alitto/pond v1.8.3 h1:ydIqygCLVPqIX/USe5EaV/aSRXTRXDEI9JwuDdu+/xs= +github.com/alitto/pond v1.8.3/go.mod h1:CmvIIGd5jKLasGI3D87qDkQxjzChdKMmnXMg3fG6M6Q= +github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156/go.mod h1:Cb/ax3seSYIx7SuZdm2G2xzfwmv3TPSk2ucNfQESPXM= +github.com/andybalholm/brotli v1.0.4/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig= +github.com/andybalholm/brotli v1.0.5/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig= +github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ= +github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY= +github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= +github.com/apache/arrow-go/v18 v18.4.1 h1:q/jVkBWCJOB9reDgaIZIdruLQUb1kbkvOnOFezVH1C4= +github.com/apache/arrow-go/v18 v18.4.1/go.mod h1:tLyFubsAl17bvFdUAy24bsSvA/6ww95Iqi67fTpGu3E= +github.com/apache/arrow/go/v10 v10.0.1/go.mod h1:YvhnlEePVnBS4+0z3fhPfUy7W1Ikj0Ih0vcRo/gZ1M0= +github.com/apache/arrow/go/v11 v11.0.0/go.mod h1:Eg5OsL5H+e299f7u5ssuXsuHQVEGC4xei5aX110hRiI= +github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ= +github.com/apache/thrift v0.13.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ= +github.com/apache/thrift v0.16.0/go.mod h1:PHK3hniurgQaNMZYaCLEqXKsYK8upmhPbmdP2FXSqgU= +github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= +github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8= +github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= +github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA= +github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4= +github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= +github.com/aryann/difflib v0.0.0-20170710044230-e206f873d14a/go.mod h1:DAHtR1m6lCRdSC2Tm3DSWRPvIPr6xNKyeHdqDQSQT+A= +github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQwij/eHl5CU= +github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= +github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g= +github.com/aws/aws-sdk-go-v2 v1.21.2/go.mod h1:ErQhvNuEMhJjweavOYhxVkn2RUx7kQXVATHrjKtxIpM= +github.com/aws/aws-sdk-go-v2 v1.43.5 h1:yKT5GYnFWhuDo+DqKvE5ZPwVn3RjC4MAeBtZGlh6AVM= +github.com/aws/aws-sdk-go-v2 v1.43.5/go.mod h1:wZjAJppCntyOGgVSmgVTfDyRJK5PHOasO6Wsy8U7Axk= +github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.17 h1:mn+Vxb9zgz/FE/yDTcFim3DZ1qpcrxR+qBQkBrl6bzA= +github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.17/go.mod h1:eDfmEFxu+BSVsUGLbzJhWjpOurv1mqczClS97yI8wdk= +github.com/aws/aws-sdk-go-v2/config v1.18.45/go.mod h1:ZwDUgFnQgsazQTnWfeLWk5GjeqTQTL8lMkoE1UXzxdE= +github.com/aws/aws-sdk-go-v2/config v1.32.36 h1:mX6ietU7UlB4w/2IUaexJdsyUDvhTd+jYPjVePiyi6s= +github.com/aws/aws-sdk-go-v2/config v1.32.36/go.mod h1:rMpV4xk7ZK59edraSaHP0jsWrztWTT5tbCwWY495hug= +github.com/aws/aws-sdk-go-v2/credentials v1.13.43/go.mod h1:zWJBz1Yf1ZtX5NGax9ZdNjhhI4rgjfgsyk6vTY1yfVg= +github.com/aws/aws-sdk-go-v2/credentials v1.19.35 h1:Cxua2RVdRwL0sfjHM/SnQoOnQ7xKng9m5EQBO8BnZlg= +github.com/aws/aws-sdk-go-v2/credentials v1.19.35/go.mod h1:9XQ+RSIGPkycr+oCJYnB1uTv5kMVVR+rd2vYK0Hxj2w= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.13/go.mod h1:f/Ib/qYjhV2/qdsf79H3QP/eRE4AkVyEf6sk7XfZ1tg= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36 h1:gucL1KH/PAYbpTpBg09CiVpBdTu4qkCl8C7xOTBixUg= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36/go.mod h1:usTB+PHhNMhrx2dxUeHcM7OrT5pySvmjYI++IsefPN0= +github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.12 h1:yIZV4/Eg+W8AN0MaI+PshJQ0sfsv6Hgsos6WmaojzFk= +github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.12/go.mod h1:lBJJRVakFaZwpIFvzSBKcLkLCki8jg086seIleOq8Ic= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.43/go.mod h1:auo+PiyLl0n1l8A0e8RIeR8tOzYPfZZH/JNlrJ8igTQ= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36 h1:5CrzwxDqf4w3x1Vs3/NiZ0nsC34Hbm3pIDMWbsLebOE= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36/go.mod h1:A3gHdKZIvG/QXERzZwcxNS3RNDFcRCuhhTFBYp+V/nw= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.37/go.mod h1:Qe+2KtKml+FEsQF/DHmDV+xjtche/hwoF75EG4UlHW8= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36 h1:A4N2f4YPcST0v+dWtX+xrpPPCL9VTBhoIFFUWYqbacE= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36/go.mod h1:B/Qr859uxWUEfZeGotK5KAEoof4Q9YWgNtPSwV6jcyk= +github.com/aws/aws-sdk-go-v2/internal/ini v1.3.45/go.mod h1:lD5M20o09/LCuQ2mE62Mb/iSdSlCNuj6H5ci7tW7OsE= +github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37 h1:oyd3ke4V9AhKcRR7rRgxk1VyI+DjK2CBQtbxh3OkdaA= +github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37/go.mod h1:aA9D7SqfG9IC1b7FLD7Iyc8Q4JN0a8gHhNjN4zPlIaI= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.16 h1:iE4NGbvqUZnHDqddQAauZzCILYtFjOHwRM5MOOKLB5A= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.16/go.mod h1:VsjEgrP+ibcou8TlWA4tYaB+0OojuhirsmCe+U60hTA= +github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.29 h1:E65Hj648dOV6FuUfI0mYXXhQRHbsi7n+B9h6fZPJO/E= +github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.29/go.mod h1:xLrF9yNTCs92VZSpdEd68EJbgcdw3SMR74RO6QDzWHE= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.37/go.mod h1:vBmDnwWXWxNPFRMmG2m/3MKOe+xEcMDo1tanpaWCcck= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.36 h1:fx2ujmozWn+C/GtfXfz5k6Ckzza40ElOpIW7d92fLWQ= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.36/go.mod h1:QT2ufGVJ+xTRxtXPHTQ1kHkAdWIKPCmD+BqYAXWv8/4= +github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.37 h1:KGHa9iZCrgtkOsFfXb0S4ywsjostA/hau7WE9aSb43E= +github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.37/go.mod h1:FV79f0DSnZIEGsQjWenENGtUycrasyAaJZO+zRanLHA= +github.com/aws/aws-sdk-go-v2/service/route53 v1.30.2/go.mod h1:TQZBt/WaQy+zTHoW++rnl8JBrmZ0VO6EUbVua1+foCA= +github.com/aws/aws-sdk-go-v2/service/s3 v1.107.1 h1:VUTtUJMuRNMkb/7NIKmd8NQaeQLPGCMoTJxkYKre4qM= +github.com/aws/aws-sdk-go-v2/service/s3 v1.107.1/go.mod h1:WvUaO0lP5GNMs1R6cs6qvB3mqo16GLta8yfOuf55Rpc= +github.com/aws/aws-sdk-go-v2/service/signin v1.5.5 h1:0VTFBfOgPJrUSpGMgzoi8qLcXF5dbmiBuxpo14eBWUw= +github.com/aws/aws-sdk-go-v2/service/signin v1.5.5/go.mod h1:sNZYlBxoohYMBYl47BO/bFtAM6I8HSsPa1qwwPPRGoQ= +github.com/aws/aws-sdk-go-v2/service/sso v1.15.2/go.mod h1:gsL4keucRCgW+xA85ALBpRFfdSLH4kHOVSnLMSuBECo= +github.com/aws/aws-sdk-go-v2/service/sso v1.33.5 h1:jDQARFp1mJ2PEnllQf01nfFXGfWMJ59e0/HCHUTTZCk= +github.com/aws/aws-sdk-go-v2/service/sso v1.33.5/go.mod h1:OcT2AhgTuxGAwZk5hgxaNLGpS33W8s8dUQadGVDVY9I= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.17.3/go.mod h1:a7bHA82fyUXOm+ZSWKU6PIoBxrjSprdLoM8xPYvzYVg= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.5 h1:8xo1q9ttkYqMJ6vOXX67FPSpVEI7BWKVTKh77g82w+8= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.5/go.mod h1:hbBeEUrZg6VddXYZpbKPyF0tl4XEnM+Dbx92RW3vmZI= +github.com/aws/aws-sdk-go-v2/service/sts v1.23.2/go.mod h1:Eows6e1uQEsc4ZaHANmsPRzAKcVDrcmjjWiih2+HUUQ= +github.com/aws/aws-sdk-go-v2/service/sts v1.45.5 h1:eQ5BtXDrPg2wK0AjtVPzeBhUpYPeqHE/ptiH7xJRGek= +github.com/aws/aws-sdk-go-v2/service/sts v1.45.5/go.mod h1:f9ImhnOISY7BuTZLM8qHepCYnglHBVLk5wVzatmP++w= +github.com/aws/smithy-go v1.15.0/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA= +github.com/aws/smithy-go v1.27.7 h1:Zgj5z4LfcDYoQIVk+n/yGdTkP/2y6ZT5vYxe0fp7bqE= +github.com/aws/smithy-go v1.27.7/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc= +github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4= +github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= +github.com/benbjohnson/immutable v0.4.3 h1:GYHcksoJ9K6HyAUpGxwZURrbTkXA0Dh4otXGqbhdrjA= +github.com/benbjohnson/immutable v0.4.3/go.mod h1:qJIKKSmdqz1tVzNtst1DZzvaqOU1onk1rc03IeM3Owk= +github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= +github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= +github.com/bgentry/speakeasy v0.2.0 h1:tgObeVOf8WAvtuAX6DhJ4xks4CFNwPDZiqzGqIHE51E= +github.com/bgentry/speakeasy v0.2.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= +github.com/bits-and-blooms/bitset v1.7.0/go.mod h1:gIdJ4wp64HaoK2YrL1Q5/N7Y16edYb8uY+O0FJTyyDA= +github.com/bits-and-blooms/bitset v1.14.2/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8= +github.com/bits-and-blooms/bitset v1.17.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8= +github.com/bits-and-blooms/bitset v1.24.3 h1:Bte86SlO3lwPQqww+7BE9ZuUCKIjfqnG5jtEyqA9y9Y= +github.com/bits-and-blooms/bitset v1.24.3/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8= +github.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM= +github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= +github.com/boombuler/barcode v1.0.0/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8= +github.com/boombuler/barcode v1.0.1/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8= +github.com/btcsuite/btcd v0.23.2 h1:/YOgUp25sdCnP5ho6Hl3s0E438zlX+Kak7E6TgBgoT0= +github.com/btcsuite/btcd v0.23.2/go.mod h1:0QJIIN1wwIXF/3G/m87gIwGniDMDQqjVn4SZgnFpsYY= +github.com/btcsuite/btcd/btcec/v2 v2.1.0/go.mod h1:2VzYrv4Gm4apmbVVsSq5bqf1Ec8v56E48Vt0Y/umPgA= +github.com/btcsuite/btcd/btcec/v2 v2.1.3/go.mod h1:ctjw4H1kknNJmRN4iP1R7bTQ+v3GJkZBd6mui8ZsAZE= +github.com/btcsuite/btcd/btcec/v2 v2.2.0/go.mod h1:U7MHm051Al6XmscBQ0BoNydpOTsFAn707034b5nY8zU= +github.com/btcsuite/btcd/btcec/v2 v2.3.5 h1:dpAlnAwmT1yIBm3exhT1/8iUSD98RDJM5vqJVQDQLiU= +github.com/btcsuite/btcd/btcec/v2 v2.3.5/go.mod h1:m22FrOAiuxl/tht9wIqAoGHcbnCCaPWyauO8y2LGGtQ= +github.com/btcsuite/btcd/btcutil v1.1.0/go.mod h1:5OapHB7A2hBBWLm48mmw4MOHNJCcUBTwmWH/0Jn8VHE= +github.com/btcsuite/btcd/chaincfg/chainhash v1.0.0/go.mod h1:7SFka0XMvUgj3hfZtydOrQY2mwhPclbT2snogU7SQQc= +github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1 h1:q0rUy8C/TYNBQS1+CGKw68tLOFYSNEs0TFnxxnS9+4U= +github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1/go.mod h1:7SFka0XMvUgj3hfZtydOrQY2mwhPclbT2snogU7SQQc= +github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f/go.mod h1:TdznJufoqS23FtqVCzL0ZqgP5MqXbb4fg/WgDys70nA= +github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d/go.mod h1:+5NJ2+qvTyV9exUAL/rxXi3DcLg2Ts+ymUAY5y4NvMg= +github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd/go.mod h1:HHNXQzUsZCxOoE+CPiyCTO6x34Zs86zZUiwtpXoGdtg= +github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtEyQwv5/p4Mg4C0fgbePVuGr935/5ddU9Z3TmDRY= +github.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs= +github.com/casbin/casbin/v2 v2.1.2/go.mod h1:YcPU1XXisHhLzuxH9coDNf2FbKpjGlbCg3n9yuLkIJQ= +github.com/cenkalti/backoff v2.2.1+incompatible h1:tNowT99t7UNflLxfYYSlKYsBpXdEet03Pg2g16Swow4= +github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM= +github.com/cenkalti/backoff/v4 v4.1.1/go.mod h1:scbssz8iZGpm3xbr14ovlUdkxfGXNInqkPWOWmG2CLw= +github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8= +github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE= +github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= +github.com/census-instrumentation/opencensus-proto v0.3.0/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= +github.com/census-instrumentation/opencensus-proto v0.4.1/go.mod h1:4T9NM4+4Vw91VeyqjLS6ao50K5bOcLKN6Q42XnYaRYw= +github.com/cespare/cp v0.1.0/go.mod h1:SOGHArjBr4JWaSDEVpWpo/hNg6RoKrls6Oh40hiwW+s= +github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= +github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= +github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cheekybits/is v0.0.0-20150225183255-68e9c0620927/go.mod h1:h/aW8ynjgkuj+NQRlZcDbAbM1ORAbXjXX77sX7T289U= +github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= +github.com/chzyer/logex v1.2.0/go.mod h1:9+9sk7u7pGNWYMkh0hdiL++6OeibzJccyQU4p4MedaY= +github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= +github.com/chzyer/readline v1.5.0/go.mod h1:x22KAscuvRqlLoK9CsoYsmxoXZMMFVyOl86cAH8qUic= +github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= +github.com/chzyer/test v0.0.0-20210722231415-061457976a23/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= +github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag= +github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I= +github.com/clbanning/x2j v0.0.0-20191024224557-825249438eec/go.mod h1:jMjuTZXRI4dUb/I5gc9Hdhagfvm9+RyrPryS/auMzxE= +github.com/cloudflare/cloudflare-go v0.114.0/go.mod h1:O7fYfFfA6wKqKFn2QIR9lhj7FDw6VQCGOY6hd2TBtd0= +github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI= +github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI= +github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cncf/xds/go v0.0.0-20211001041855-01bcc9b48dfe/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cncf/xds/go v0.0.0-20220314180256-7f1daf1720fc/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cncf/xds/go v0.0.0-20230105202645-06c439db220b/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b h1:SHlYZ/bMx7frnmeqCu+xm0TCxXLzX3jQIVuFbnFGtFU= +github.com/cockroachdb/crlib v0.0.0-20241112164430-1264a2edc35b/go.mod h1:Gq51ZeKaFCXk6QwuGM0w1dnaOqc/F5zKT2zA9D6Xeac= +github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8= +github.com/cockroachdb/datadriven v1.0.2/go.mod h1:a9RdTaap04u637JoCzcUoIcDmvwSUtcUFtT/C3kJlTU= +github.com/cockroachdb/datadriven v1.0.3-0.20230413201302-be42291fc80f/go.mod h1:a9RdTaap04u637JoCzcUoIcDmvwSUtcUFtT/C3kJlTU= +github.com/cockroachdb/datadriven v1.0.3-0.20250407164829-2945557346d5 h1:UycK/E0TkisVrQbSoxvU827FwgBBcZ95nRRmpj/12QI= +github.com/cockroachdb/datadriven v1.0.3-0.20250407164829-2945557346d5/go.mod h1:jsaKMvD3RBCATk1/jbUZM8C9idWBJME9+VRZ5+Liq1g= +github.com/cockroachdb/errors v1.11.3/go.mod h1:m4UIW4CDjx+R5cybPsNrRbreomiFqt8o1h1wUVazSd8= +github.com/cockroachdb/errors v1.12.0 h1:d7oCs6vuIMUQRVbi6jWWWEJZahLCfJpnJSVobd1/sUo= +github.com/cockroachdb/errors v1.12.0/go.mod h1:SvzfYNNBshAVbZ8wzNc/UPK3w1vf0dKDUP41ucAIf7g= +github.com/cockroachdb/fifo v0.0.0-20240606204812-0bbfbd93a7ce h1:giXvy4KSc/6g/esnpM7Geqxka4WSqI1SZc7sMJFd3y4= +github.com/cockroachdb/fifo v0.0.0-20240606204812-0bbfbd93a7ce/go.mod h1:9/y3cnZ5GKakj/H4y9r9GTjCvAFta7KLgSHPJJYc52M= +github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs= +github.com/cockroachdb/logtags v0.0.0-20241215232642-bb51bb14a506 h1:ASDL+UJcILMqgNeV5jiqR4j+sTuvQNHdf2chuKj1M5k= +github.com/cockroachdb/logtags v0.0.0-20241215232642-bb51bb14a506/go.mod h1:Mw7HqKr2kdtu6aYGn3tPmAftiP3QPX63LdK/zcariIo= +github.com/cockroachdb/metamorphic v0.0.0-20231108215700-4ba948b56895 h1:XANOgPYtvELQ/h4IrmPAohXqe2pWA8Bwhejr3VQoZsA= +github.com/cockroachdb/metamorphic v0.0.0-20231108215700-4ba948b56895/go.mod h1:aPd7gM9ov9M8v32Yy5NJrDyOcD8z642dqs+F0CeNXfA= +github.com/cockroachdb/pebble v1.1.2/go.mod h1:4exszw1r40423ZsmkG/09AFEG83I0uDgfujJdbL6kYU= +github.com/cockroachdb/pebble v1.1.5 h1:5AAWCBWbat0uE0blr8qzufZP5tBjkRyy/jWe1QWLnvw= +github.com/cockroachdb/pebble v1.1.5/go.mod h1:17wO9el1YEigxkP/YtV8NtCivQDgoCyBg5c4VR/eOWo= +github.com/cockroachdb/pebble/v2 v2.1.3 h1:irU503OnjRoJBrkZQIJvwv9c4WvpUeOJxhRApojB8D8= +github.com/cockroachdb/pebble/v2 v2.1.3/go.mod h1:B1UgWsyR+L+UvZXNgpxw+WqsUKA8VQ/bb//FXOHghB8= +github.com/cockroachdb/redact v1.1.5/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg= +github.com/cockroachdb/redact v1.1.6 h1:zXJBwDZ84xJNlHl1rMyCojqyIxv+7YUpQiJLQ7n4314= +github.com/cockroachdb/redact v1.1.6/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg= +github.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b h1:VXvSNzmr8hMj8XTuY0PT9Ane9qZGul/p67vGYwl9BFI= +github.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b/go.mod h1:yBRu/cnL4ks9bgy4vAASdjIW+/xMlFwuHKqtmh3GZQg= +github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 h1:zuQyyAKVxetITBuuhv3BI9cMrmStnpT18zmgmTxunpo= +github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06/go.mod h1:7nc4anLGjupUW/PeY5qiNYsdNXj7zopG+eqsS7To5IQ= +github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI= +github.com/codegangsta/inject v0.0.0-20150114235600-33e0aa1cb7c0/go.mod h1:4Zcjuz89kmFXt9morQgcfYZAYZ5n8WHjt81YYWIwtTM= +github.com/coinbase/rosetta-sdk-go v0.7.0 h1:lmTO/JEpCvZgpbkOITL95rA80CPKb5CtMzLaqF2mCNg= +github.com/coinbase/rosetta-sdk-go v0.7.0/go.mod h1:7nD3oBPIiHqhRprqvMgPoGxe/nyq3yftRmpsy29coWE= +github.com/consensys/bavard v0.1.13/go.mod h1:9ItSMtA/dXMAiL7BG6bqW2m3NdSEObYWoH223nGHukI= +github.com/consensys/bavard v0.1.22/go.mod h1:k/zVjHHC4B+PQy1Pg7fgvG3ALicQw540Crag8qx+dZs= +github.com/consensys/gnark-crypto v0.12.1/go.mod h1:v2Gy7L/4ZRosZ7Ivs+9SfUDr0f5UlG+EM5t7MPHiLuY= +github.com/consensys/gnark-crypto v0.13.0/go.mod h1:wKqwsieaKPThcFkHe0d0zMsbHEUWFmZcG7KBCse210o= +github.com/consensys/gnark-crypto v0.14.0/go.mod h1:CU4UijNPsHawiVGNxe9co07FkzCeWHHrb1li/n1XoU0= +github.com/consensys/gnark-crypto v0.18.0 h1:vIye/FqI50VeAr0B3dx+YjeIvmc3LWz4yEfbWBpTUf0= +github.com/consensys/gnark-crypto v0.18.0/go.mod h1:L3mXGFTe1ZN+RSJ+CLjUt9x7PNdx8ubaYfDROyp2Z8c= +github.com/containerd/continuity v0.3.0 h1:nisirsYROK15TAMVukJOUyGJjz4BNQJBVsNvAXZJ/eg= +github.com/containerd/continuity v0.3.0/go.mod h1:wJEAIwKOm/pBZuBd0JmeTvnLquTB1Ag8espWhkykbPM= +github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk= +github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= +github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk= +github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= +github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= +github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= +github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= +github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc= +github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= +github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= +github.com/cosmos/btcutil v1.0.5 h1:t+ZFcX77LpKtDBhjucvnOH8C2l2ioGsBNEQ3jef8xFk= +github.com/cosmos/btcutil v1.0.5/go.mod h1:IyB7iuqZMJlthe2tkIFL33xPyzbFYP0XVdS8P5lUPis= +github.com/cosmos/cosmos-sdk/ics23/go v0.8.0 h1:iKclrn3YEOwk4jQHT2ulgzuXyxmzmPczUalMwW4XH9k= +github.com/cosmos/cosmos-sdk/ics23/go v0.8.0/go.mod h1:2a4dBq88TUoqoWAU5eu0lGvpFP3wWDPgdHPargtyw30= +github.com/cosmos/go-bip39 v1.0.0 h1:pcomnQdrdH22njcAatO0yWojsUnCO3y2tNoV1cb6hHY= +github.com/cosmos/go-bip39 v1.0.0/go.mod h1:RNJv0H/pOIVgxw6KS7QeX2a0Uo0aKUlfhZ4xuwvCdJw= +github.com/cosmos/gorocksdb v1.2.0 h1:d0l3jJG8M4hBouIZq0mDUHZ+zjOx044J3nGRskwTb4Y= +github.com/cosmos/gorocksdb v1.2.0/go.mod h1:aaKvKItm514hKfNJpUJXnnOWeBnk2GL4+Qw9NHizILw= +github.com/cosmos/keyring v1.1.7-0.20210622111912-ef00f8ac3d76 h1:DdzS1m6o/pCqeZ8VOAit/gyATedRgjvkVI+UCrLpyuU= +github.com/cosmos/keyring v1.1.7-0.20210622111912-ef00f8ac3d76/go.mod h1:0mkLWIoZuQ7uBoospo5Q9zIpqq6rYCPJDSUdeCJvPM8= +github.com/cosmos/ledger-cosmos-go v1.0.0 h1:jNKW89nPf0vR0EkjHG8Zz16h6p3zqwYEOxlHArwgYtw= +github.com/cosmos/ledger-cosmos-go v1.0.0/go.mod h1:mGaw2wDOf+Z6SfRJsMGxU9DIrBa4du0MAiPlpPhLAOE= +github.com/cpuguy83/go-md2man v1.0.10 h1:BSKMNlYxDvnunlTymqtgONjNnaRV1sTpcovwwjF22jk= +github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE= +github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= +github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= +github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= +github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= +github.com/cpuguy83/go-md2man/v2 v2.0.5/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= +github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= +github.com/cpuguy83/go-md2man/v2 v2.0.7 h1:zbFlGlXEAKlwXpmvle3d8Oe3YnkKIK4xSRTd3sHPnBo= +github.com/cpuguy83/go-md2man/v2 v2.0.7/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= +github.com/crate-crypto/go-ipa v0.0.0-20240223125850-b1e8a79f509c/go.mod h1:geZJZH3SzKCqnz5VT0q/DyIG/tvu/dZk+VIfXicupJs= +github.com/crate-crypto/go-ipa v0.0.0-20240724233137-53bbb0ceb27a h1:W8mUrRp6NOVl3J+MYp5kPMoUZPp7aOYHtaua31lwRHg= +github.com/crate-crypto/go-ipa v0.0.0-20240724233137-53bbb0ceb27a/go.mod h1:sTwzHBvIzm2RfVCGNEBZgRyjwK40bVoun3ZnGOCafNM= +github.com/crate-crypto/go-kzg-4844 v1.1.0 h1:EN/u9k2TF6OWSHrCCDBBU6GLNMq88OspHHlMnHfoyU4= +github.com/crate-crypto/go-kzg-4844 v1.1.0/go.mod h1:JolLjpSff1tCCJKaJx4psrlEdlXuJEC996PL3tTAFks= +github.com/creachadair/taskgroup v0.3.2 h1:zlfutDS+5XG40AOxcHDSThxKzns8Tnr9jnr6VqkYlkM= +github.com/creachadair/taskgroup v0.3.2/go.mod h1:wieWwecHVzsidg2CsUnFinW1faVN4+kq+TDlRJQ0Wbk= +github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c/go.mod h1:GyV+0YP4qX0UQ7r2MoYZ+AvYDp12OF5yg4q8rGnyNh4= +github.com/danieljoos/wincred v1.0.2/go.mod h1:SnuYRW9lp1oJrZX/dXJqr0cPK5gYXqx3EJbmjhLdK9U= +github.com/danieljoos/wincred v1.1.2 h1:QLdCxFs1/Yl4zduvBdcHB8goaYk9RARS2SgLLRuAyr0= +github.com/danieljoos/wincred v1.1.2/go.mod h1:GijpziifJoIBfYh+S7BbkdUTU4LfM+QnGqR5Vl2tAx0= +github.com/davecgh/go-spew v0.0.0-20161028175848-04cdfd42973b/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/deckarep/golang-set/v2 v2.6.0 h1:XfcQbWM1LlMB8BsJ8N9vW5ehnnPVIw0je80NsVHagjM= +github.com/deckarep/golang-set/v2 v2.6.0/go.mod h1:VAky9rY/yGXJOLEDv3OMci+7wtDpOF4IN+y82NBOac4= +github.com/decred/dcrd/crypto/blake256 v1.0.0/go.mod h1:sQl2p6Y26YV+ZOcSTP6thNdn47hh8kt6rqSlvmrXFAc= +github.com/decred/dcrd/crypto/blake256 v1.1.0 h1:zPMNGQCm0g4QTY27fOCorQW7EryeQ/U0x++OzVrdms8= +github.com/decred/dcrd/crypto/blake256 v1.1.0/go.mod h1:2OfgNZ5wDpcsFmHmCK5gZTPcCXqlm2ArzUIkw9czNJo= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1/go.mod h1:hyedUtir6IdtD/7lIxGeCxkaw7y45JueMRL4DIyJDKs= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 h1:NMZiJj8QnKe1LgsbDayM4UoHwbvwDRwnI3hwNaAHRnc= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40= +github.com/decred/dcrd/lru v1.0.0/go.mod h1:mxKOwFd7lFjN2GZYsiz/ecgqR6kkYAl+0pz0tEMk218= +github.com/deepmap/oapi-codegen v1.6.0/go.mod h1:ryDa9AgbELGeB+YEXE1dR53yAjHwFvE9iAUlWl9Al3M= +github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f h1:U5y3Y5UE0w7amNe7Z5G/twsBW0KEalRQXZzf8ufSh9I= +github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f/go.mod h1:xH/i4TFMt8koVQZ6WFms69WAsDWr2XsYL3Hkl7jkoLE= +github.com/dgraph-io/badger/v2 v2.2007.2/go.mod h1:26P/7fbL4kUZVEVKLAKXkBXKOydDmM2p1e+NhhnBCAE= +github.com/dgraph-io/badger/v2 v2.2007.4/go.mod h1:vSw/ax2qojzbN6eXHIx6KPKtCSHJN/Uz0X0VPruTIhk= +github.com/dgraph-io/badger/v3 v3.2103.2 h1:dpyM5eCJAtQCBcMCZcT4UBZchuTJgCywerHHgmxfxM8= +github.com/dgraph-io/badger/v3 v3.2103.2/go.mod h1:RHo4/GmYcKKh5Lxu63wLEMHJ70Pac2JqZRYGhlyAo2M= +github.com/dgraph-io/ristretto v0.0.3-0.20200630154024-f66de99634de/go.mod h1:KPxhHT9ZxKefz+PCeOGsrHpl1qZ7i70dGTu2u+Ahh6E= +github.com/dgraph-io/ristretto v0.0.3/go.mod h1:KPxhHT9ZxKefz+PCeOGsrHpl1qZ7i70dGTu2u+Ahh6E= +github.com/dgraph-io/ristretto v0.1.0/go.mod h1:fux0lOrBhrVCJd3lcTHsIJhq1T2rokOu6v9Vcb3Q9ug= +github.com/dgraph-io/ristretto v0.2.0 h1:XAfl+7cmoUDWW/2Lx8TGZQjjxIQ2Ley9DSf52dru4WE= +github.com/dgraph-io/ristretto v0.2.0/go.mod h1:8uBHCU/PBV4Ag0CJrP47b9Ofby5dqWNh4FicAdoqFNU= +github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= +github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw= +github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 h1:fAjc9m62+UWV/WAFKLNi6ZS0675eEUC9y3AlwSbQu1Y= +github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw= +github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc= +github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= +github.com/djherbis/atime v1.1.0/go.mod h1:28OF6Y8s3NQWwacXc5eZTsEsiMzp7LF8MbXE+XJPdBE= +github.com/dlclark/regexp2 v1.4.1-0.20201116162257-a2a8dda75c91/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc= +github.com/dlclark/regexp2 v1.7.0 h1:7lJfhqlPssTb1WQx4yvTHN0uElPEv52sbaECrAQxjAo= +github.com/dlclark/regexp2 v1.7.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= +github.com/dnaeon/go-vcr v1.1.0/go.mod h1:M7tiix8f0r6mKKJ3Yq/kqU1OYf3MnfmBWVbPx/yU9ko= +github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ= +github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ= +github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec= +github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= +github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= +github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= +github.com/donovanhide/eventsource v0.0.0-20210830082556-c59027999da0/go.mod h1:56wL82FO0bfMU5RvfXoIwSOP2ggqqxT+tAfNEIyxuHw= +github.com/dop251/goja v0.0.0-20211022113120-dc8c55024d06/go.mod h1:R9ET47fwRVRPZnOGvHxxhuZcbrMCuiqOz3Rlrh4KSnk= +github.com/dop251/goja v0.0.0-20230605162241-28ee0ee714f3 h1:+3HCtB74++ClLy8GgjUQYeC8R4ILzVcIe8+5edAJJnE= +github.com/dop251/goja v0.0.0-20230605162241-28ee0ee714f3/go.mod h1:QMWlm50DNe14hD7t24KEqZuUdC9sOTy8W6XbCU1mlw4= +github.com/dop251/goja_nodejs v0.0.0-20210225215109-d91c329300e7/go.mod h1:hn7BA7c8pLvoGndExHudxTDKZ84Pyvv+90pbBjbTz0Y= +github.com/dop251/goja_nodejs v0.0.0-20211022123610-8dd9abb0616d/go.mod h1:DngW8aVqWbuLRMHItjPUyqdj+HWPvnQe8V8y1nDpIbM= +github.com/duckdb/duckdb-go-bindings v0.1.23 h1:sJRXraxfC/gdHI2T7oHqrdp1VdKemrgqWGQ8986mH1c= +github.com/duckdb/duckdb-go-bindings v0.1.23/go.mod h1:WA7U/o+b37MK2kiOPPueVZ+FIxt5AZFCjszi8hHeH18= +github.com/duckdb/duckdb-go-bindings/darwin-amd64 v0.1.23 h1:Xyw1fWu4jzOtv2Hqkaehr7f+qbIWNRfBMbZyD+g8dyU= +github.com/duckdb/duckdb-go-bindings/darwin-amd64 v0.1.23/go.mod h1:jfbOHwGZqNCpMAxV4g4g5jmWr0gKdMvh2fGusPubxC4= +github.com/duckdb/duckdb-go-bindings/darwin-arm64 v0.1.23 h1:85Xomx5NxZ+Nt+VepUJzuMYbBTH+nB6JlBXIyJuTovA= +github.com/duckdb/duckdb-go-bindings/darwin-arm64 v0.1.23/go.mod h1:zLVtv1a7TBuTPvuAi32AIbnuw7jjaX5JElZ+urv1ydc= +github.com/duckdb/duckdb-go-bindings/linux-amd64 v0.1.23 h1:RGw8mDqQl9JdlCYV0PAfGBuVAgOguiL5Vz5W8pH8fGw= +github.com/duckdb/duckdb-go-bindings/linux-amd64 v0.1.23/go.mod h1:GCaBoYnuLZEva7BXzdXehTbqh9VSvpLB80xcmxGBGs8= +github.com/duckdb/duckdb-go-bindings/linux-arm64 v0.1.23 h1:f8NHa8DGes7vg55BxeMVm0ycddEJTRHEt813USdL0/I= +github.com/duckdb/duckdb-go-bindings/linux-arm64 v0.1.23/go.mod h1:kpQSpJmDSSZQ3ikbZR1/8UqecqMeUkWFjFX2xZxlCuI= +github.com/duckdb/duckdb-go-bindings/windows-amd64 v0.1.23 h1:HJqVo+09gT6LQWW6PlN/c7K8s0eQhv5giE7kJcMGMSU= +github.com/duckdb/duckdb-go-bindings/windows-amd64 v0.1.23/go.mod h1:wa+egSGXTPS16NPADFCK1yFyt3VSXxUS6Pt2fLnvRPM= +github.com/duckdb/duckdb-go/arrowmapping v0.0.26 h1:XKhWpNkLtIbcBE2vnKm7FaAju3daplxo8MJIXOAY/Zg= +github.com/duckdb/duckdb-go/arrowmapping v0.0.26/go.mod h1:R7egXxZcy0hxKY/MsoM2xjkMvRo4H07TffDhYCnhKfQ= +github.com/duckdb/duckdb-go/mapping v0.0.25 h1:z4RhivKCIRv0MWQwtYekqH+ikoA29/n8L+rzgreKvsc= +github.com/duckdb/duckdb-go/mapping v0.0.25/go.mod h1:CIo3WbNx3Txl+VO9+P5eNCN9ZifUA/KIp9NY1rTG/uo= +github.com/duckdb/duckdb-go/v2 v2.5.3 h1:GlT+bXW+/gCYo0Q8P9L6IvvKRzMM0/tDXj5fKkoAfCM= +github.com/duckdb/duckdb-go/v2 v2.5.3/go.mod h1:+mGhZCF5tHYIdBWrp7+KGj6JnTXdm+sBTh3ZSLhXorE= +github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= +github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/dvsekhvalnov/jose2go v0.0.0-20200901110807-248326c1351b/go.mod h1:7BvyPhdbLxMXIYTFPLsyJRFMsKmOZnQmzh6Gb+uquuM= +github.com/dvsekhvalnov/jose2go v1.7.0 h1:bnQc8+GMnidJZA8zc6lLEAb4xNrIqHwO+9TzqvtQZPo= +github.com/dvsekhvalnov/jose2go v1.7.0/go.mod h1:QsHjhyTlD/lAVqn/NSbVZmSCGeDehTB/mPZadG+mhXU= +github.com/eapache/go-resiliency v1.1.0/go.mod h1:kFI+JgMyC7bLPUVY133qvEBtVayf5mFgVsvEsIPBvNs= +github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21/go.mod h1:+020luEh2TKB4/GOp8oxxtq0Daoen/Cii55CzbTV6DU= +github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I= +github.com/edsrzf/mmap-go v1.0.0/go.mod h1:YO35OhQPt3KJa3ryjFM5Bs14WD66h8eGKpfaBNrHW5M= +github.com/eknkc/amber v0.0.0-20171010120322-cdade1c07385/go.mod h1:0vRUJqYpeSZifjYj7uP3BG/gKcuzL9xWVV/Y+cK33KM= +github.com/envoyproxy/go-control-plane v0.10.2-0.20220325020618-49ff273808a1/go.mod h1:KJwIaB5Mv44NWtYuAOFCVOjcI94vtpEz2JU/D2v6IjE= +github.com/envoyproxy/go-control-plane v0.10.3/go.mod h1:fJJn/j26vwOu972OllsvAgJJM//w9BV6Fxbg2LuVd34= +github.com/envoyproxy/go-control-plane v0.11.1-0.20230524094728-9239064ad72f/go.mod h1:sfYdkwUW4BA3PbKjySwjJy+O4Pu0h62rlqCMHNk+K+Q= +github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= +github.com/envoyproxy/protoc-gen-validate v0.6.7/go.mod h1:dyJXwwfPK2VSqiB9Klm1J6romD608Ba7Hij42vrOBCo= +github.com/envoyproxy/protoc-gen-validate v0.9.1/go.mod h1:OKNgG7TCp5pF4d6XftA0++PMirau2/yoOwVac3AbF2w= +github.com/envoyproxy/protoc-gen-validate v0.10.1/go.mod h1:DRjgyB0I43LtJapqN6NiRwroiAU2PaFuvk/vjgh61ss= +github.com/ethereum/c-kzg-4844 v1.0.0 h1:0X1LBXxaEtYD9xsyj9B9ctQEZIpnvVDeoBx8aHEwTNA= +github.com/ethereum/c-kzg-4844 v1.0.0/go.mod h1:VewdlzQmpT5QSrVhbBuGoCdFJkpaJlO1aQputP83wc0= +github.com/ethereum/evmc/v12 v12.1.0 h1:fUIzJNnXa9VPYx253lDS7L9iBZtP+tlpTdZst5e6Pks= +github.com/ethereum/evmc/v12 v12.1.0/go.mod h1:80jmft01io35nSmrX70bKFR/lncwFuqE90iLLSMyMAE= +github.com/ethereum/go-verkle v0.2.2 h1:I2W0WjnrFUIzzVPwm8ykY+7pL2d4VhlsePn4j7cnFk8= +github.com/ethereum/go-verkle v0.2.2/go.mod h1:M3b90YRnzqKyyzBEWJGqj8Qff4IDeXnzFw0P9bFw3uk= +github.com/facebookgo/ensure v0.0.0-20200202191622-63f1cf65ac4c h1:8ISkoahWXwZR41ois5lSJBSVw4D0OV19Ht/JSTzvSv0= +github.com/facebookgo/ensure v0.0.0-20200202191622-63f1cf65ac4c/go.mod h1:Yg+htXGokKKdzcwhuNDwVvN+uBxDGXJ7G/VN1d8fa64= +github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 h1:JWuenKqqX8nojtoVVWjGfOF9635RETekkoH6Cc9SX0A= +github.com/facebookgo/stack v0.0.0-20160209184415-751773369052/go.mod h1:UbMTZqLaRiH3MsBH8va0n7s1pQYcu3uTb8G4tygF4Zg= +github.com/facebookgo/subset v0.0.0-20200203212716-c811ad88dec4 h1:7HZCaLC5+BZpmbhCOZJ293Lz68O7PYrF2EzeiFMwCLk= +github.com/facebookgo/subset v0.0.0-20200203212716-c811ad88dec4/go.mod h1:5tD+neXqOorC30/tWg0LCSkrqj/AR6gu8yY8/fpw1q0= +github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= +github.com/fatih/color v1.10.0/go.mod h1:ELkj/draVOlAH/xkhN6mQ50Qd0MPOk5AAr3maGEBuJM= +github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE= +github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M= +github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= +github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= +github.com/ferranbt/fastssz v0.1.2/go.mod h1:X5UPrE2u1UJjxHA8X54u04SBwdAQjG2sFtWs39YxyWs= +github.com/fjl/gencodec v0.1.0/go.mod h1:Um1dFHPONZGTHog1qD1NaWjXJW/SPB38wPv0O8uZ2fI= +github.com/flosch/pongo2/v4 v4.0.2/go.mod h1:B5ObFANs/36VwxxlgKpdchIJHMvHB562PW+BWPhwZD8= +github.com/fogleman/gg v1.2.1-0.20190220221249-0403632d5b90/go.mod h1:R/bRT+9gY/C5z7JzPU0zXsXHKM4/ayA+zqcVNZzPa1k= +github.com/fogleman/gg v1.3.0/go.mod h1:R/bRT+9gY/C5z7JzPU0zXsXHKM4/ayA+zqcVNZzPa1k= +github.com/fortytw2/leaktest v1.3.0 h1:u8491cBMTQ8ft8aeV+adlcytMZylmA5nnwwkRZjI8vw= +github.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHquHwclZch5g= +github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVBjqR7JHJk0brhHOZYGmfBYOrK0ZhYMEtBr4= +github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20= +github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8= +github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0= +github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= +github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= +github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU= +github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw= +github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k= +github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0= +github.com/garslo/gogen v0.0.0-20170306192744-1d203ffc1f61/go.mod h1:Q0X6pkwTILDlzrGEckF6HKjXe48EgsY/l7K7vhY4MW8= +github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff/go.mod h1:x7DCsMOv1taUwEWCzT4cmDeAkigA5/QCwUodaVOe8Ww= +github.com/getkin/kin-openapi v0.53.0/go.mod h1:7Yn5whZr5kJi6t+kShccXS8ae1APpYTW6yheSwk8Yi4= +github.com/getsentry/sentry-go v0.27.0/go.mod h1:lc76E2QywIyW8WuBnwl8Lc4bkmQH4+w1gwTf25trprY= +github.com/getsentry/sentry-go v0.35.0 h1:+FJNlnjJsZMG3g0/rmmP7GiKjQoUF5EXfEtBwtPtkzY= +github.com/getsentry/sentry-go v0.35.0/go.mod h1:C55omcY9ChRQIUcVcGcs+Zdy4ZpQGvNJ7JYHIoSWOtE= +github.com/ghemawat/stream v0.0.0-20171120220530-696b145b53b9 h1:r5GgOLGbza2wVHRzK7aAj6lWZjfbAwiu/RDCVOKjRyM= +github.com/ghemawat/stream v0.0.0-20171120220530-696b145b53b9/go.mod h1:106OIgooyS7OzLDOpUGgm9fA3bQENb/cFSyyBmMoJDs= +github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= +github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE= +github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI= +github.com/gin-gonic/gin v1.7.0 h1:jGB9xAJQ12AIGNB4HguylppmDK1Am9ppF7XnGXXJuoU= +github.com/gin-gonic/gin v1.7.0/go.mod h1:jD2toBW3GZUr5UMcdrwQA10I7RuaFOl/SGeDjXkfUtY= +github.com/go-chi/chi/v5 v5.0.0/go.mod h1:BBug9lr0cqtdAhsu6R4AAdvufI0/XBzAQSsUqJpoZOs= +github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA= +github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og= +github.com/go-fonts/dejavu v0.1.0/go.mod h1:4Wt4I4OU2Nq9asgDCteaAaWZOV24E+0/Pwo0gppep4g= +github.com/go-fonts/latin-modern v0.2.0/go.mod h1:rQVLdDMK+mK1xscDwsqM5J8U2jrRa3T0ecnM9pNujks= +github.com/go-fonts/liberation v0.1.1/go.mod h1:K6qoJYypsmfVjWg8KOVDQhLc8UDgIK2HYqyqAO9z7GY= +github.com/go-fonts/liberation v0.2.0/go.mod h1:K6qoJYypsmfVjWg8KOVDQhLc8UDgIK2HYqyqAO9z7GY= +github.com/go-fonts/stix v0.1.0/go.mod h1:w/c1f0ldAUlJmLBvlbkvVXLAD+tAMqobIIQpmnUIzUY= +github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= +github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= +github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= +github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgOZ7o= +github.com/go-kit/kit v0.13.0 h1:OoneCcHKHQ03LfBpoQCUfCluwd2Vt3ohz+kvbJneZAU= +github.com/go-kit/kit v0.13.0/go.mod h1:phqEHMMUbyrCFCTgH48JueqrM3md2HcAZ8N3XE4FKDg= +github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY= +github.com/go-latex/latex v0.0.0-20210118124228-b3d85cf34e07/go.mod h1:CO1AlKB2CSIqUrmQPqA0gdRIlnLEY0gK5JGjh37zN5U= +github.com/go-latex/latex v0.0.0-20210823091927-c0d11ff05a81/go.mod h1:SX0U8uGpxhq9o2S/CELCSUxEWWAuoCUcVCQWv7G2OCk= +github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= +github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= +github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= +github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= +github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/go-martini/martini v0.0.0-20170121215854-22fa46961aab/go.mod h1:/P9AEU963A2AYjv4d1V5eVL1CQbEJq6aCNHDDjibzu8= +github.com/go-ole/go-ole v1.2.5/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0= +github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0= +github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE= +github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78= +github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= +github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= +github.com/go-pdf/fpdf v0.5.0/go.mod h1:HzcnA+A23uwogo0tp9yU+l3V+KXhiESpt1PMayhOh5M= +github.com/go-pdf/fpdf v0.6.0/go.mod h1:HzcnA+A23uwogo0tp9yU+l3V+KXhiESpt1PMayhOh5M= +github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4= +github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8= +github.com/go-playground/locales v0.14.0 h1:u50s323jtVGugKlcYeyzC0etD1HifMjqmJqb8WugfUU= +github.com/go-playground/locales v0.14.0/go.mod h1:sawfccIbzZTqEDETgFXqTho0QybSa7l++s0DH+LDiLs= +github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA= +github.com/go-playground/universal-translator v0.18.0 h1:82dyy6p4OuJq4/CByFNOn/jYrnRPArHwAcmLoJZxyho= +github.com/go-playground/universal-translator v0.18.0/go.mod h1:UvRDBj+xPUEGrFYl+lu/H90nyDXpg0fqeB/AQUGNTVA= +github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4= +github.com/go-playground/validator/v10 v10.11.1 h1:prmOlTVv+YjZjmRmNSF3VmspqJIxJWXmqUsHwfTRRkQ= +github.com/go-playground/validator/v10 v10.11.1/go.mod h1:i+3WkQ1FvaUjjxh1kSvIA4dMGDBiPU55YFDl0WbKdWU= +github.com/go-redis/redis/v8 v8.11.5/go.mod h1:gREzHqY1hg6oD9ngVRbLStwAWKhA0FEgq8Jd4h5lpwo= +github.com/go-sourcemap/sourcemap v2.1.3+incompatible h1:W1iEw64niKVGogNgBN3ePyLFfuisuzeidWPMPWmECqU= +github.com/go-sourcemap/sourcemap v2.1.3+incompatible/go.mod h1:F8jJfvm2KbVjc5NqelyYJmf/v5J0dwNLS2mL4sNA1Jg= +github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= +github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= +github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= +github.com/go-stack/stack v1.8.1/go.mod h1:dcoOX6HbPZSZptuspn9bctJ+N/CnF5gGygcUP3XYfe4= +github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= +github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro= +github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= +github.com/gobwas/httphead v0.0.0-20180130184737-2c6c146eadee/go.mod h1:L0fX3K22YWvt/FAX9NnzrNzcI4wNYi9Yku4O0LKYflo= +github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU= +github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM= +github.com/gobwas/pool v0.2.0/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw= +github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og= +github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw= +github.com/gobwas/ws v1.0.2/go.mod h1:szmBTxLgaFppYjEmNtny/v3w89xOydFnnZMcgRRu/EM= +github.com/gobwas/ws v1.1.0 h1:7RFti/xnNkMJnrK7D1yQ/iCIB5OrrY/54/H930kIbHA= +github.com/gobwas/ws v1.1.0/go.mod h1:nzvNcVha5eUziGrbxFCo6qFIojQHjJV5cLYIbezhfL0= +github.com/goccy/go-json v0.9.11/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I= +github.com/goccy/go-json v0.10.4/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M= +github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4= +github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M= +github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 h1:ZpnhV/YsD2/4cESfV5+Hoeu/iUR3ruzNvZ+yQfO03a0= +github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2/go.mod h1:bBOAhwG1umN6/6ZUMtDFBMQR8jRg9O75tm9K00oMsK4= +github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= +github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU= +github.com/gofrs/flock v0.13.0 h1:95JolYOvGMqeH31+FC7D2+uULf6mG61mEZ/A8dRYMzw= +github.com/gofrs/flock v0.13.0/go.mod h1:jxeyy9R1auM5S6JYDBhDt+E2TCo7DkratH4Pgi8P+Z0= +github.com/gogo/gateway v1.1.0 h1:u0SuhL9+Il+UbjM9VIE3ntfRujKbvVpFvNB4HbjeVQ0= +github.com/gogo/gateway v1.1.0/go.mod h1:S7rR8FRQyG3QFESeSv4l2WnsyzlCLG0CzBbUUo/mbic= +github.com/gogo/googleapis v0.0.0-20180223154316-0cd9801be74a/go.mod h1:gf4bu3Q80BeJ6H1S1vYPm8/ELATdvryBaNFGgqEef3s= +github.com/gogo/googleapis v1.4.1/go.mod h1:2lpHqI5OcWCtVElxXnPt+s8oJvMpySlOyM6xDCrzib4= +github.com/gogo/status v1.1.0/go.mod h1:BFv9nrluPLmrS0EmGVvLaPNmRosr9KapBYd5/hpY1WM= +github.com/golang-jwt/jwt v3.2.1+incompatible/go.mod h1:8pz2t5EyA70fFQQSrl6XZXzqecmYZeUEB8OUGHkxJ+I= +github.com/golang-jwt/jwt v3.2.2+incompatible h1:IfV12K8xAKAnZqdXVzCZ+TOjboZ2keLg81eXfW3O+oY= +github.com/golang-jwt/jwt v3.2.2+incompatible/go.mod h1:8pz2t5EyA70fFQQSrl6XZXzqecmYZeUEB8OUGHkxJ+I= +github.com/golang-jwt/jwt/v4 v4.2.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg= +github.com/golang-jwt/jwt/v4 v4.4.3/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0= +github.com/golang-jwt/jwt/v4 v4.5.0/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0= +github.com/golang-jwt/jwt/v4 v4.5.1 h1:JdqV9zKUdtaa9gdPlywC3aeoEsR681PlKC+4F5gQgeo= +github.com/golang-jwt/jwt/v4 v4.5.1/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0= +github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0/go.mod h1:E/TSTwGwJL78qG/PmXZO1EjYhfJinVAhrmmHX6Z8B9k= +github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= +github.com/golang/glog v1.0.0/go.mod h1:EWib/APOK0SL3dFbYqvxE3UYd8E6s1ouQ7iEp/0LWV4= +github.com/golang/glog v1.1.0/go.mod h1:pfYeQZ3JWZoXTV5sFc986z3HTpwQs9At6P4ImfuP3NQ= +github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 h1:f+oWsMOmNPc8JmEHVZIycC7hBoQxHH9pNKQORJNozsQ= +github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8/go.mod h1:wcDNUvekVysuuOpQKo3191zZyTpiI6se1N1ULghS0sw= +github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= +github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= +github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= +github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= +github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= +github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= +github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8= +github.com/golang/mock v1.6.0/go.mod h1:p6yTPP+5HYm5mzsMV8JkE6ZKdX+/wYM6Hr+LicevLPs= +github.com/golang/mock v1.7.0-rc.1 h1:YojYx61/OLFsiv6Rw1Z96LpldJIy31o+UHmwAUMJ6/U= +github.com/golang/mock v1.7.0-rc.1/go.mod h1:s42URUywIqd+OcERslBJvOjepvNymP31m3q8d/GkuRs= +github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.0/go.mod h1:Qd/q+1AKNOZr9uGQzbzCmRO6sUih6GTPZv6a1/R87v0= +github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= +github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= +github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk= +github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= +github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= +github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= +github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= +github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= +github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= +github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= +github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= +github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= +github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM= +github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= +github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= +github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/golang/snappy v0.0.5-0.20220116011046-fa5810519dcb/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/golang/snappy v1.0.0 h1:Oy607GVXHs7RtbggtPBnr2RmDArIsAefDwvrdWvRhGs= +github.com/golang/snappy v1.0.0/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/golangci/lint-1 v0.0.0-20181222135242-d2cdd8c08219/go.mod h1:/X8TswGSh1pIozq4ZwCfxS0WA5JGXguxk94ar/4c87Y= +github.com/gonum/blas v0.0.0-20181208220705-f22b278b28ac/go.mod h1:P32wAyui1PQ58Oce/KYkOqQv8cVw1zAapXOl+dRFGbc= +github.com/gonum/floats v0.0.0-20181209220543-c233463c7e82/go.mod h1:PxC8OnwL11+aosOB5+iEPoV3picfs8tUpkVd0pDo+Kg= +github.com/gonum/internal v0.0.0-20181124074243-f884aa714029/go.mod h1:Pu4dmpkhSyOzRwuXkOgAvijx4o+4YMUJJo9OvPYMkks= +github.com/gonum/lapack v0.0.0-20181123203213-e4cdc5a0bff9/go.mod h1:XA3DeT6rxh2EAE789SSiSJNqxPaC0aE9J8NTOI0Jo/A= +github.com/gonum/matrix v0.0.0-20181209220409-c518dec07be9/go.mod h1:0EXg4mc1CNP0HCqCz+K4ts155PXIlUywf0wqN+GfPZw= +github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= +github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= +github.com/google/btree v1.1.3 h1:CVpQJjYgC4VbzxeGVHfvZrv1ctoYCAI8vbl07Fcxlyg= +github.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4= +github.com/google/flatbuffers v1.12.1/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8= +github.com/google/flatbuffers v2.0.8+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8= +github.com/google/flatbuffers v25.2.10+incompatible h1:F3vclr7C3HpB1k9mxCGRMXq6FdUalZ6H/pNX4FP1v0Q= +github.com/google/flatbuffers v25.2.10+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8= +github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= +github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE= +github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU= +github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= +github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= +github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= +github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= +github.com/google/martian/v3 v3.2.1/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk= +github.com/google/martian/v3 v3.3.2/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk= +github.com/google/orderedcode v0.0.1 h1:UzfcAexk9Vhv8+9pNOgRu41f16lHq725vPwnSeiG/Us= +github.com/google/orderedcode v0.0.1/go.mod h1:iVyU4/qPKHY5h/wSd6rZZCDcLJNxiWO6dvsYES2Sb20= +github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= +github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= +github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210601050228-01bbb1931b22/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210609004039-a478d1d731e9/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20230207041349-798e818bf904 h1:4/hN5RUoecvl+RmJRE2YxKWtnnQls6rQjjW5oV7qg2U= +github.com/google/pprof v0.0.0-20230207041349-798e818bf904/go.mod h1:uglQLonpP8qtYCYyzA+8c/9qtqgA3qsXGYqCPKARAFg= +github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= +github.com/google/safehtml v0.0.2/go.mod h1:L4KWwDsUJdECRAEpZoBn3O64bQaywRscowZjJAzjHnU= +github.com/google/subcommands v1.2.0/go.mod h1:ZjhPrFU+Olkh9WazFPsl27BQ4UPiG37m3yTrtFlrHVk= +github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/googleapis/enterprise-certificate-proxy v0.0.0-20220520183353-fd19c99a87aa/go.mod h1:17drOmN3MwGY7t0e+Ei9b45FFGA3fBs3x36SsCg1hq8= +github.com/googleapis/enterprise-certificate-proxy v0.1.0/go.mod h1:17drOmN3MwGY7t0e+Ei9b45FFGA3fBs3x36SsCg1hq8= +github.com/googleapis/enterprise-certificate-proxy v0.2.0/go.mod h1:8C0jb7/mgJe/9KK8Lm7X9ctZC2t60YyIpYEI16jx0Qg= +github.com/googleapis/enterprise-certificate-proxy v0.2.1/go.mod h1:AwSRAtLfXpU5Nm3pW+v7rGDHp09LsPtGY9MduiEsR9k= +github.com/googleapis/enterprise-certificate-proxy v0.2.3/go.mod h1:AwSRAtLfXpU5Nm3pW+v7rGDHp09LsPtGY9MduiEsR9k= +github.com/googleapis/gax-go v0.0.0-20161107002406-da06d194a00e/go.mod h1:SFVmujtThgffbyetf+mdk2eWhX2bMyUtNHzFKcPA9HY= +github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= +github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= +github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0= +github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM= +github.com/googleapis/gax-go/v2 v2.2.0/go.mod h1:as02EH8zWkzwUoLbBaFeQ+arQaj/OthfcblKl4IGNaM= +github.com/googleapis/gax-go/v2 v2.3.0/go.mod h1:b8LNqSzNabLiUpXKkY7HAR5jr6bIT99EXz9pXxye9YM= +github.com/googleapis/gax-go/v2 v2.4.0/go.mod h1:XOTVJ59hdnfJLIP/dh8n5CGryZR2LxK9wbMD5+iXC6c= +github.com/googleapis/gax-go/v2 v2.5.1/go.mod h1:h6B0KMMFNtI2ddbGJn3T3ZbwkeT6yqEF02fYlzkUCyo= +github.com/googleapis/gax-go/v2 v2.6.0/go.mod h1:1mjbznJAPHFpesgE5ucqfYEscaz5kMdcIDwU/6+DDoY= +github.com/googleapis/gax-go/v2 v2.7.0/go.mod h1:TEop28CZZQ2y+c0VxMUmu1lV+fQx57QpBWsYpwqHJx8= +github.com/googleapis/gax-go/v2 v2.7.1/go.mod h1:4orTrqY6hXxxaUL4LHIPl6lGo8vAE38/qKbhSAKP6QI= +github.com/googleapis/go-type-adapters v1.0.0/go.mod h1:zHW75FOG2aur7gAO2B+MLby+cLsWGBF62rFAi7WjWO4= +github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g= +github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= +github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k= +github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg= +github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c= +github.com/gorilla/handlers v1.5.2 h1:cLTUSsNkgcwhgRqvCNmdbRWG0A3N4F+M2nWKdScwyEE= +github.com/gorilla/handlers v1.5.2/go.mod h1:dX+xVpaxdSw+q0Qek8SSsl3dfMk3jNddUkMzo0GtH0w= +github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs= +github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs= +github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So= +github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY= +github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ= +github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4= +github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ= +github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ= +github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= +github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/grafana/pyroscope-go/godeltaprof v0.1.8 h1:iwOtYXeeVSAeYefJNaxDytgjKtUuKQbJqgAIjlnicKg= +github.com/grafana/pyroscope-go/godeltaprof v0.1.8/go.mod h1:2+l7K7twW49Ct4wFluZD3tZ6e0SjanjcUUBPVD/UuGU= +github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc h1:GN2Lv3MGO7AS6PrRoT6yV5+wkrOpcszoIsO4+4ds248= +github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc/go.mod h1:+JKpmjMGhpgPL+rXZ5nsZieVzvarn86asRlBg4uNGnk= +github.com/graph-gophers/graphql-go v1.3.0/go.mod h1:9CQHMSxwO4MprSdzoIEobiHpoLtHm77vfxsvsIN5Vuc= +github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= +github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= +github.com/grpc-ecosystem/go-grpc-middleware v1.2.2/go.mod h1:EaizFBKfUKtMIF5iaDEhniwNedqGo9FuLFzppDr3uwI= +github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 h1:UH//fgunKIs4JdUbpDl1VZCDaL56wXCB/5+wF6uHfaI= +github.com/grpc-ecosystem/go-grpc-middleware v1.4.0/go.mod h1:g5qyo/la0ALbONm6Vbp88Yd8NsDy6rZz+RcrMPxvld8= +github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 h1:Ovs26xHkKqVztRpIrF/92BcuyuQ/YW4NSIpoGtfXNho= +github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= +github.com/grpc-ecosystem/grpc-gateway v1.8.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= +github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= +github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= +github.com/grpc-ecosystem/grpc-gateway v1.16.0 h1:gmcG1KaJ57LophUzW0Hy8NmPhnMZb4M0+kPpLofRdBo= +github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0/go.mod h1:hgWBS7lorOAVIJEQMi4ZsPv9hVvWI6+ch50m39Pf2Ks= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3/go.mod h1:o//XUCC/F+yRGJoPO/VU0GSB0f8Nhgmxx0VIRUvaC0w= +github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c h1:6rhixN/i8ZofjG1Y75iExal34USq5p+wiN1tpie8IrU= +github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c/go.mod h1:NMPJylDgVpX0MLRlPy15sqSwOFv/U1GZ2m21JhFfek0= +github.com/guptarohit/asciigraph v0.5.5/go.mod h1:dYl5wwK4gNsnFf9Zp+l06rFiDZ5YtXM6x7SRWZ3KGag= +github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q= +github.com/hashicorp/consul/api v1.3.0/go.mod h1:MmDNSzIMUjNpY/mQ398R4bk2FnqQLoPndWW5VkKPlCE= +github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8= +github.com/hashicorp/consul/sdk v0.3.0/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8= +github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= +github.com/hashicorp/go-bexpr v0.1.10 h1:9kuI5PFotCboP3dkDYFr/wi0gg0QVbSNz5oFRpxn4uE= +github.com/hashicorp/go-bexpr v0.1.10/go.mod h1:oxlubA2vC/gFVfX1A6JGp7ls7uCDlfJn732ehYYg+g0= +github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= +github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= +github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= +github.com/hashicorp/go-immutable-radix v1.3.1 h1:DKHmCUm2hRBK510BaiZlwvpD40f8bJFeZnpfm2KLowc= +github.com/hashicorp/go-immutable-radix v1.3.1/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= +github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= +github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= +github.com/hashicorp/go-retryablehttp v0.5.3/go.mod h1:9B5zBasrRhHXnJnui7y6sL7es7NDiJgTc6Er0maI1Xs= +github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU= +github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU= +github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4= +github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= +github.com/hashicorp/go-uuid v1.0.1 h1:fv1ep09latC32wFoVwnqcnKJGnMSdBanPczbHAYm1BE= +github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= +github.com/hashicorp/go-version v1.2.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= +github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90= +github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= +github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= +github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c= +github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= +github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= +github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= +github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ= +github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I= +github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc= +github.com/hdevalence/ed25519consensus v0.2.0 h1:37ICyZqdyj0lAZ8P4D1d1id3HqbbG1N3iBb1Tb4rdcU= +github.com/hdevalence/ed25519consensus v0.2.0/go.mod h1:w3BHWjwJbFU29IRHL1Iqkw3sus+7FctEyM4RqDxYNzo= +github.com/holiman/billy v0.0.0-20240216141850-2abb0c79d3c4 h1:X4egAf/gcS1zATw6wn4Ej8vjuVGxeHdan+bRb2ebyv4= +github.com/holiman/billy v0.0.0-20240216141850-2abb0c79d3c4/go.mod h1:5GuXa7vkL8u9FkFuWdVvfR5ix8hRB7DbOAaYULamFpc= +github.com/holiman/bloomfilter/v2 v2.0.3 h1:73e0e/V0tCydx14a0SCYS/EWCxgwLZ18CZcZKVu0fao= +github.com/holiman/bloomfilter/v2 v2.0.3/go.mod h1:zpoh+gs7qcpqrHr3dB55AMiJwo0iURXE7ZOP9L9hSkA= +github.com/holiman/uint256 v1.2.0/go.mod h1:y4ga/t+u+Xwd7CpDgZESaRcWy0I7XMlTMA25ApIH5Jw= +github.com/holiman/uint256 v1.3.2 h1:a9EgMPSC1AAaj1SZL5zIQD3WbwTuHrMGOerLjGmM/TA= +github.com/holiman/uint256 v1.3.2/go.mod h1:EOMSn4q6Nyt9P6efbI3bueV4e1b3dGlUCXeiRV4ng7E= +github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= +github.com/hudl/fargo v1.3.0/go.mod h1:y3CKSmjA+wD2gak7sUSXTAoopbhU08POFhmITJgmKTg= +github.com/huin/goupnp v1.3.0 h1:UvLUlWDNpoUdYzb2TCn+MuTWtcjXKSza2n6CBdQ0xXc= +github.com/huin/goupnp v1.3.0/go.mod h1:gnGPsThkYa7bFi/KWmEysQRf48l2dvR5bxr2OFckNX8= +github.com/hydrogen18/memlistener v1.0.0/go.mod h1:qEIFzExnS6016fRpRfxrExeVn2gbClQA99gQhnIcdhE= +github.com/iancoleman/strcase v0.2.0/go.mod h1:iwCmte+B7n89clKwxIoIXy/HfoL7AsD47ZCWhYzw7ho= +github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= +github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= +github.com/ianlancetaylor/demangle v0.0.0-20220319035150-800ac71e25c2/go.mod h1:aYm2/VgdVmcIU8iMfdMvDMsRAQjcfZSKFby6HOFvi/w= +github.com/imkira/go-interpol v1.1.0/go.mod h1:z0h2/2T3XF8kyEPpRgJ3kmNv+C43p+I/CoI+jC3w2iA= +github.com/improbable-eng/grpc-web v0.15.0 h1:BN+7z6uNXZ1tQGcNAuaU1YjsLTApzkjt2tzCixLaUPQ= +github.com/improbable-eng/grpc-web v0.15.0/go.mod h1:1sy9HKV4Jt9aEs9JSnkWlRJPuPtwNr0l57L4f878wP8= +github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= +github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= +github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/influxdata/influxdb-client-go/v2 v2.4.0/go.mod h1:vLNHdxTJkIf2mSLvGrpj8TCcISApPoXkaxP8g9uRlW8= +github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo= +github.com/influxdata/influxdb1-client v0.0.0-20220302092344-a9ab5670611c/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo= +github.com/influxdata/line-protocol v0.0.0-20200327222509-2487e7298839/go.mod h1:xaLFMmpvUxqXtVkUJfg9QmT88cDaCJ3ZKgdZ78oO8Qo= +github.com/iris-contrib/go.uuid v2.0.0+incompatible/go.mod h1:iz2lgM/1UnEf1kP0L/+fafWORmlnuysV2EMP8MW+qe0= +github.com/iris-contrib/httpexpect/v2 v2.12.1/go.mod h1:7+RB6W5oNClX7PTwJgJnsQP3ZuUUYB3u61KCqeSgZ88= +github.com/iris-contrib/schema v0.0.6/go.mod h1:iYszG0IOsuIsfzjymw1kMzTL8YQcCWlm65f3wX8J5iA= +github.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus= +github.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc= +github.com/jedisct1/go-minisign v0.0.0-20230811132847-661be99b8267/go.mod h1:h1nSAbGFqGVzn6Jyl1R/iCcBUHN4g+gW1u9CoBTrb9E= +github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= +github.com/jhump/protoreflect v1.18.0 h1:TOz0MSR/0JOZ5kECB/0ufGnC2jdsgZ123Rd/k4Z5/2w= +github.com/jhump/protoreflect v1.18.0/go.mod h1:ezWcltJIVF4zYdIFM+D/sHV4Oh5LNU08ORzCGfwvTz8= +github.com/jhump/protoreflect/v2 v2.0.0-beta.1 h1:Dw1rslK/VotaUGYsv53XVWITr+5RCPXfvvlGrM/+B6w= +github.com/jhump/protoreflect/v2 v2.0.0-beta.1/go.mod h1:D9LBEowZyv8/iSu97FU2zmXG3JxVTmNw21mu63niFzU= +github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= +github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo= +github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U= +github.com/jmhodges/levigo v1.0.0 h1:q5EC36kV79HWeTBWsod3mG11EgStG3qArTKcvlksN1U= +github.com/jmhodges/levigo v1.0.0/go.mod h1:Q6Qx+uH3RAqyK4rFQroq9RL7mdkABMcfhEI+nNuzMJQ= +github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= +github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= +github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= +github.com/jrick/logrotate v1.0.0/go.mod h1:LNinyqDIJnpAur+b8yyulnQw/wDuN1+BYKlTRt3OuAQ= +github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= +github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= +github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= +github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= +github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= +github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= +github.com/jung-kurt/gofpdf v1.0.0/go.mod h1:7Id9E/uU8ce6rXgefFLlgrJj/GYY22cpxn+r32jIOes= +github.com/jung-kurt/gofpdf v1.0.3-0.20190309125859-24315acbbda5/go.mod h1:7Id9E/uU8ce6rXgefFLlgrJj/GYY22cpxn+r32jIOes= +github.com/karalabe/hid v1.0.1-0.20240306101548-573246063e52/go.mod h1:qk1sX/IBgppQNcGCRoj90u6EGC056EBoIc1oEjCWla8= +github.com/kataras/blocks v0.0.7/go.mod h1:UJIU97CluDo0f+zEjbnbkeMRlvYORtmc1304EeyXf4I= +github.com/kataras/golog v0.1.8/go.mod h1:rGPAin4hYROfk1qT9wZP6VY2rsb4zzc37QpdPjdkqVw= +github.com/kataras/iris/v12 v12.2.0/go.mod h1:BLzBpEunc41GbE68OUaQlqX4jzi791mx5HU04uPb90Y= +github.com/kataras/jwt v0.1.8/go.mod h1:Q5j2IkcIHnfwy+oNY3TVWuEBJNw0ADgCcXK9CaZwV4o= +github.com/kataras/neffos v0.0.21/go.mod h1:FeGka8lu8cjD2H+0OpBvW8c6xXawy3fj5VX6xcIJ1Fg= +github.com/kataras/pio v0.0.11/go.mod h1:38hH6SWH6m4DKSYmRhlrCJ5WItwWgCVrTNU62XZyUvI= +github.com/kataras/sitemap v0.0.6/go.mod h1:dW4dOCNs896OR1HmG+dMLdT7JjDk7mYBzoIRwuj5jA4= +github.com/kataras/tunnel v0.0.4/go.mod h1:9FkU4LaeifdMWqZu7o20ojmW4B7hdhv2CMLwfnHGpYw= +github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs= +github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8= +github.com/keybase/go-keychain v0.0.0-20190712205309-48d3d31d256d h1:Z+RDyXzjKE0i2sTjZ/b1uxiGtPhFy34Ou/Tk0qwN0kM= +github.com/keybase/go-keychain v0.0.0-20190712205309-48d3d31d256d/go.mod h1:JJNrCn9otv/2QP4D7SMJBgaleKpOf66PnW6F5WGNRIc= +github.com/kilic/bls12-381 v0.1.0/go.mod h1:vDTTHJONJ6G+P2R74EhnyotQDTliQDnFEwhdmfzw1ig= +github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= +github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= +github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4= +github.com/klauspost/asmfmt v1.3.2/go.mod h1:AG8TuvYojzulgDAMCnYn50l/5QV3Bs/tp6j0HLHbNSE= +github.com/klauspost/compress v1.10.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= +github.com/klauspost/compress v1.11.7/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= +github.com/klauspost/compress v1.12.3/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg= +github.com/klauspost/compress v1.15.0/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk= +github.com/klauspost/compress v1.15.9/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU= +github.com/klauspost/compress v1.15.11/go.mod h1:QPwzmACJjUTFsnSHH934V6woptycfrDDJnH7hvFVbGM= +github.com/klauspost/compress v1.16.0/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE= +github.com/klauspost/compress v1.18.3 h1:9PJRvfbmTabkOX8moIpXPbMMbYN60bWImDDU7L+/6zw= +github.com/klauspost/compress v1.18.3/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= +github.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= +github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= +github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y= +github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= +github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg= +github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= +github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= +github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v0.0.0-20170224010052-a616ab194758/go.mod h1:B69LEHPfb2qLo0BaaOLcbitczOKLWTsrBG9LczfCD4k= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/labstack/echo/v4 v4.2.1/go.mod h1:AA49e0DZ8kk5jTOOCKNuPR6oTnBS0dYiM4FW1e6jwpg= +github.com/labstack/echo/v4 v4.10.0/go.mod h1:S/T/5fy/GigaXnHTkh0ZGe4LpkkQysvRjFMSUTkDRNQ= +github.com/labstack/gommon v0.3.0/go.mod h1:MULnywXg0yavhxWKc+lOruYdAhDwPK9wf0OL7NoOu+k= +github.com/labstack/gommon v0.4.0/go.mod h1:uW6kP17uPlLJsD3ijUYn3/M5bAxtlZhMI6m3MFxTMTM= +github.com/leanovate/gopter v0.2.9/go.mod h1:U2L/78B+KVFIx2VmW6onHJQzXtFb+p5y3y2Sh+Jxxv8= +github.com/leanovate/gopter v0.2.11 h1:vRjThO1EKPb/1NsDXuDrzldR28RLkBflWYcU9CvzWu4= +github.com/leanovate/gopter v0.2.11/go.mod h1:aK3tzZP/C+p1m3SPRE4SYZFGP7jjkuSI4f7Xvpt0S9c= +github.com/ledgerwatch/erigon-lib v0.0.0-20230210071639-db0e7ed11263 h1:LGEzZvf33Y1NhuP5+jI/ni9l1TFS6oYPDilgy74NusM= +github.com/ledgerwatch/erigon-lib v0.0.0-20230210071639-db0e7ed11263/go.mod h1:OXgMDuUo2lZ3NpH29ZvMYbk+LxFd5ffDl2Z2mGMuY/I= +github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII= +github.com/leodido/go-urn v1.2.1 h1:BqpAaACuzVSgi/VLzGZIobT2z4v53pjosyNd9Yv6n/w= +github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY= +github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw= +github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o= +github.com/libp2p/go-buffer-pool v0.1.0 h1:oK4mSFcQz7cTQIfqbe4MIj9gLW+mnanjyFtc6cdF0Y8= +github.com/libp2p/go-buffer-pool v0.1.0/go.mod h1:N+vh8gMqimBzdKkSMVuydVDq+UV5QTWy5HSiZacSbPg= +github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743/go.mod h1:qklhhLq1aX+mtWk9cPHPzaBjWImj5ULL6C7HFJtXQMM= +github.com/lightstep/lightstep-tracer-go v0.18.1/go.mod h1:jlF1pusYV4pidLvZ+XD0UBX0ZE6WURAspgAczcDHrL4= +github.com/linxGnu/grocksdb v1.8.11 h1:BGol9e5gB1BrsTvOxloC88pe70TCqgrfLNwkyWW0kD8= +github.com/linxGnu/grocksdb v1.8.11/go.mod h1:xZCIb5Muw+nhbDK4Y5UJuOrin5MceOuiXkVUR7vp4WY= +github.com/lucasjones/reggen v0.0.0-20180717132126-cdb49ff09d77/go.mod h1:5ELEyG+X8f+meRWHuqUOewBOhvHkl7M76pdGEansxW4= +github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0/go.mod h1:zJYVVT2jmtg6P3p1VtQj7WsuWi/y4VnjVBn7F8KPB3I= +github.com/lyft/protoc-gen-star v0.6.0/go.mod h1:TGAoBVkt8w7MPG72TrKIu85MIdXwDuzJYeZuUPFPNwA= +github.com/lyft/protoc-gen-star v0.6.1/go.mod h1:TGAoBVkt8w7MPG72TrKIu85MIdXwDuzJYeZuUPFPNwA= +github.com/lyft/protoc-gen-star/v2 v2.0.1/go.mod h1:RcCdONR2ScXaYnQC5tUzxzlpA3WVYF7/opLeUgcQs/o= +github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= +github.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60= +github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE= +github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0= +github.com/mailgun/raymond/v2 v2.0.48/go.mod h1:lsgvL50kgt1ylcFJYZiULi5fjPBkkhNfj4KA0W54Z18= +github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= +github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= +github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/matryer/moq v0.0.0-20190312154309-6cfb0558e1bd/go.mod h1:9ELz6aaclSIGnZBoaSLZ3NAl1VTufbOrXBPvtcy6WiQ= +github.com/matryer/try v0.0.0-20161228173917-9ac251b645a2/go.mod h1:0KeJpeMD6o+O4hW7qJOT7vyQPKrWmj26uf5wMc/IiIs= +github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= +github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= +github.com/mattn/go-colorable v0.1.7/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= +github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= +github.com/mattn/go-colorable v0.1.11/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4= +github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg= +github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE= +github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8= +github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= +github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= +github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= +github.com/mattn/go-isatty v0.0.9/go.mod h1:YNRxwqDuOph6SZLI9vUUz6OYw3QyUt7WiY2yME+cCiQ= +github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= +github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94= +github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= +github.com/mattn/go-isatty v0.0.17/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= +github.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= +github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= +github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= +github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc= +github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= +github.com/mattn/go-sqlite3 v1.14.5/go.mod h1:WVKg1VTActs4Qso6iwGbiFih2UIHo0ENGwNd0Lj+XmI= +github.com/mattn/go-sqlite3 v1.14.14 h1:qZgc/Rwetq+MtyE18WhzjokPD93dNqLGNT3QJuLvBGw= +github.com/mattn/go-sqlite3 v1.14.14/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU= +github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= +github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= +github.com/mediocregopher/radix/v3 v3.8.1/go.mod h1:8FL3F6UQRXHXIBSPUs5h0RybMF8i4n7wVopoX3x7Bv8= +github.com/microcosm-cc/bluemonday v1.0.23/go.mod h1:mN70sk7UkkF8TUr2IGBpNN0jAgStuPzlK76QuruE/z4= +github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= +github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8/go.mod h1:mC1jAcsrzbxHt8iiaC+zU4b1ylILSosueou12R++wfY= +github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3/go.mod h1:RagcQ7I8IeTMnF8JTXieKnO4Z6JCsikNEzj0DwauVzE= +github.com/minio/highwayhash v1.0.2/go.mod h1:BQskDq+xkJ12lmlUUi7U0M5Swg3EWR+dLTk+kldvVxY= +github.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882 h1:0lgqHvJWHLGW5TuObJrfyEi6+ASTKDBWikGvPqy9Yiw= +github.com/minio/minlz v1.0.1-0.20250507153514-87eb42fe8882/go.mod h1:qT0aEB35q79LLornSzeDH75LBf3aH1MV+jB5w9Wasec= +github.com/minio/sha256-simd v0.1.0/go.mod h1:2FMWW+8GMoPweT6+pI63m9YE3Lmw4J71hV56Chs1E/U= +github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM= +github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc= +github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= +github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= +github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= +github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0= +github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg= +github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY= +github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= +github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= +github.com/mitchellh/mapstructure v1.3.2/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= +github.com/mitchellh/mapstructure v1.3.3/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= +github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= +github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY= +github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= +github.com/mitchellh/pointerstructure v1.2.0 h1:O+i9nHnXS3l/9Wu7r4NrEdwA2VFTicjUEN1uBnDo34A= +github.com/mitchellh/pointerstructure v1.2.0/go.mod h1:BRAsLI5zgXmw97Lf6s25bs8ohIXc3tViBH44KcwB2g4= +github.com/mmcloughlin/addchain v0.4.0/go.mod h1:A86O+tHqZLMNO4w6ZZ4FlVQEadcoqkyU72HC5wJ4RlU= +github.com/mmcloughlin/profile v0.1.1/go.mod h1:IhHD7q1ooxgwTgjxQYkACGA77oFTDdFVejUS1/tS/qU= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee h1:W5t00kpgFdJifH4BDsTlE89Zl93FEloxaWZfGcifgq8= +github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/modocache/gover v0.0.0-20171022184752-b58185e213c5/go.mod h1:caMODM3PzxT8aQXRPkAt8xlV/e7d7w8GM5g0fa5F0D8= +github.com/montanaflynn/stats v0.6.6/go.mod h1:etXPPgVO6n31NxCd9KQUMvCM+ve0ruNzt6R8Bnaayow= +github.com/montanaflynn/stats v0.7.0/go.mod h1:etXPPgVO6n31NxCd9KQUMvCM+ve0ruNzt6R8Bnaayow= +github.com/mtibben/percent v0.2.1 h1:5gssi8Nqo8QU/r2pynCm+hBQHpkB/uNK7BJCFogWdzs= +github.com/mtibben/percent v0.2.1/go.mod h1:KG9uO+SZkUp+VkRHsCdYQV3XSZrrSpR3O9ibNBTZrns= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= +github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f h1:KUppIJq7/+SVif2QVs3tOP0zanoHgBEVAwHxUSIzRqU= +github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= +github.com/mwitkow/grpc-proxy v0.0.0-20181017164139-0f1106ef9c76/go.mod h1:x5OoJHDHqxHS801UIuhqGl6QdSAEJvtausosHSdazIo= +github.com/naoina/go-stringutil v0.1.0/go.mod h1:XJ2SJL9jCtBh+P9q5btrd/Ylo8XwT/h1USek5+NqSA0= +github.com/naoina/toml v0.1.2-0.20170918210437-9fafd6967416/go.mod h1:NBIhNtsFMo3G2szEBne+bO4gS192HuIYRqfvOWb4i1E= +github.com/nats-io/jwt v0.3.0/go.mod h1:fRYCDE99xlTsqUzISS1Bi75UBJ6ljOJQOAAu5VglpSg= +github.com/nats-io/jwt v0.3.2/go.mod h1:/euKqTS1ZD+zzjYrY7pseZrTtWQSjujC7xjPc8wL6eU= +github.com/nats-io/jwt/v2 v2.3.0/go.mod h1:0tqz9Hlu6bCBFLWAASKhE5vUA4c24L9KPUUgvwumE/k= +github.com/nats-io/nats-server/v2 v2.1.2/go.mod h1:Afk+wRZqkMQs/p45uXdrVLuab3gwv3Z8C4HTBu8GD/k= +github.com/nats-io/nats-server/v2 v2.9.11/go.mod h1:b0oVuxSlkvS3ZjMkncFeACGyZohbO4XhSqW1Lt7iRRY= +github.com/nats-io/nats.go v1.9.1/go.mod h1:ZjDU1L/7fJ09jvUSRVBR2e7+RnLiiIQyqyzEE/Zbp4w= +github.com/nats-io/nats.go v1.19.0/go.mod h1:tLqubohF7t4z3du1QDPYJIQQyhb4wl6DhjxEajSI7UA= +github.com/nats-io/nats.go v1.23.0/go.mod h1:ki/Scsa23edbh8IRZbCuNXR9TDcbvfaSijKtaqQgw+Q= +github.com/nats-io/nkeys v0.1.0/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w= +github.com/nats-io/nkeys v0.1.3/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w= +github.com/nats-io/nkeys v0.3.0/go.mod h1:gvUNGjVcM2IPr5rCsRsC6Wb3Hr2CQAm08dsxtV6A5y4= +github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c= +github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo= +github.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM= +github.com/neilotoole/errgroup v0.1.5/go.mod h1:Q2nLGf+594h0CLBs/Mbg6qOr7GtqDK7C2S41udRnToE= +github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= +github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= +github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= +github.com/nxadm/tail v1.4.11 h1:8feyoE3OzPrcshW5/MJ4sGESc5cqmGkGCWlco4l0bqY= +github.com/nxadm/tail v1.4.11/go.mod h1:OTaG3NK980DZzxbRq6lEuzgU+mug70nY11sMd4JXXHc= +github.com/oasisprotocol/curve25519-voi v0.0.0-20230904125328-1f23a7beb09a h1:dlRvE5fWabOchtH7znfiFCcOvmIYgOeAS5ifBXBlh9Q= +github.com/oasisprotocol/curve25519-voi v0.0.0-20230904125328-1f23a7beb09a/go.mod h1:hVoHR2EVESiICEMbg137etN/Lx+lSrHPTD39Z/uE+2s= +github.com/oklog/oklog v0.3.2/go.mod h1:FCV+B7mhrz4o+ueLpx+KqkyXRGMWOYEvfiXtdGtbWGs= +github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA= +github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= +github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo= +github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec= +github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY= +github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/ginkgo v1.10.3/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= +github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY= +github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= +github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= +github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= +github.com/onsi/ginkgo/v2 v2.0.0/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c= +github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c= +github.com/onsi/ginkgo/v2 v2.1.4/go.mod h1:um6tUpWM/cxCK3/FK8BXqEiUMUwRgSM4JXG47RKZmLU= +github.com/onsi/ginkgo/v2 v2.1.6/go.mod h1:MEH45j8TBi6u9BMogfbp0stKC5cdGjumZj5Y7AG4VIk= +github.com/onsi/ginkgo/v2 v2.3.0/go.mod h1:Eew0uilEqZmIEZr8JrvYlvOM7Rr6xzTmMV8AyFNU9d0= +github.com/onsi/ginkgo/v2 v2.4.0/go.mod h1:iHkDK1fKGcBoEHT5W7YBq4RFWaQulw+caOMkAt4OrFo= +github.com/onsi/ginkgo/v2 v2.5.0/go.mod h1:Luc4sArBICYCS8THh8v3i3i5CuSZO+RaQRaJoeNwomw= +github.com/onsi/ginkgo/v2 v2.7.0/go.mod h1:yjiuMwPokqY1XauOgju45q3sJt6VzQ/Fict1LFVcsAo= +github.com/onsi/ginkgo/v2 v2.8.1/go.mod h1:N1/NbDngAFcSLdyZ+/aYTYGSlq9qMCS/cNKGJjy+csc= +github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= +github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= +github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= +github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY= +github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs= +github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro= +github.com/onsi/gomega v1.20.1/go.mod h1:DtrZpjmvpn2mPm4YWQa0/ALMDj9v4YxLgojwPeREyVo= +github.com/onsi/gomega v1.21.1/go.mod h1:iYAIXgPSaDHak0LCMA+AWBpIKBr8WZicMxnE8luStNc= +github.com/onsi/gomega v1.22.1/go.mod h1:x6n7VNe4hw0vkyYUM4mjIXx3JbLiPaBPNgB7PRQ1tuM= +github.com/onsi/gomega v1.24.0/go.mod h1:Z/NWtiqwBrwUt4/2loMmHL63EDLnYHmVbuBpDr2vQAg= +github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM= +github.com/onsi/gomega v1.26.0/go.mod h1:r+zV744Re+DiYCIPRlYOTxn0YkOLcAnW8k1xXdMPGhM= +github.com/onsi/gomega v1.27.1 h1:rfztXRbg6nv/5f+Raen9RcGoSecHIFgBBLQK3Wdj754= +github.com/onsi/gomega v1.27.1/go.mod h1:aHX5xOykVYzWOV4WqQy0sy8BQptgukenXpCXfadcIAw= +github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk= +github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= +github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= +github.com/opencontainers/image-spec v1.1.0-rc2 h1:2zx/Stx4Wc5pIPDvIxHXvXtQFW/7XWJGmnM7r3wg034= +github.com/opencontainers/image-spec v1.1.0-rc2/go.mod h1:3OVijpioIKYWTqjiG0zfF6wvoJ4fAXGbjdZuI2NgsRQ= +github.com/opencontainers/runc v1.1.14 h1:rgSuzbmgz5DUJjeSnw337TxDbRuqjs6iqQck/2weR6w= +github.com/opencontainers/runc v1.1.14/go.mod h1:E4C2z+7BxR7GHXp0hAY53mek+x49X1LjPNeMTfRGvOA= +github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492/go.mod h1:Ngi6UdF0k5OKD5t5wlmGhe/EDKPoUM3BXZSSfIuJbis= +github.com/opentracing/basictracer-go v1.0.0/go.mod h1:QfBfYuafItcjQuMwinw9GhYKwFXS9KnPs5lxoYwgW74= +github.com/opentracing/opentracing-go v1.0.2/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= +github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= +github.com/openzipkin-contrib/zipkin-go-opentracing v0.4.5/go.mod h1:/wsWhb9smxSfWAKL3wpBW7V8scJMt8N8gnaMCS9E/cA= +github.com/openzipkin/zipkin-go v0.1.6/go.mod h1:QgAqvLzwWbR/WpD4A3cGpPtJrZXNIiJc5AZX7/PBEpw= +github.com/openzipkin/zipkin-go v0.2.1/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4= +github.com/openzipkin/zipkin-go v0.2.2/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4= +github.com/ory/dockertest v3.3.5+incompatible h1:iLLK6SQwIhcbrG783Dghaaa3WPzGc+4Emza6EbVUUGA= +github.com/ory/dockertest v3.3.5+incompatible/go.mod h1:1vX4m9wsvi00u5bseYwXaSnhNrne+V0E6LAcBILJdPs= +github.com/pact-foundation/pact-go v1.0.4/go.mod h1:uExwJY4kCzNPcHRj+hCR/HBbOOIwwtUjcrb0b5/5kLM= +github.com/parquet-go/parquet-go v0.25.1 h1:l7jJwNM0xrk0cnIIptWMtnSnuxRkwq53S+Po3KG8Xgo= +github.com/parquet-go/parquet-go v0.25.1/go.mod h1:AXBuotO1XiBtcqJb/FKFyjBG4aqa3aQAAWF3ZPzCanY= +github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= +github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY= +github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= +github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc= +github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ= +github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k= +github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= +github.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= +github.com/pelletier/go-toml/v2 v2.0.5/go.mod h1:OMHamSCAODeSsVrwwvcJOaoN0LIUIaFVNZzmWyNfXas= +github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4= +github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= +github.com/performancecopilot/speed v3.0.0+incompatible/go.mod h1:/CLtqpZ5gBg1M9iaPbIdPPGyKcA8hKdoy6hAWba7Yac= +github.com/peterh/liner v1.1.1-0.20190123174540-a2c9a5303de7/go.mod h1:CRroGNssyjTd/qIG2FyxByd2S8JEAZXBl4qUrZf8GS0= +github.com/petermattis/goid v0.0.0-20240813172612-4fcff4a6cae7/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4= +github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741 h1:KPpdlQLZcHfTMQRi6bFQ7ogNO0ltFT4PmtwTLW4W+14= +github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4= +github.com/phpdave11/gofpdf v1.4.2/go.mod h1:zpO6xFn9yxo3YLyMvW8HcKWVdbNqgIfOOp2dXMnm1mY= +github.com/phpdave11/gofpdi v1.0.12/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI= +github.com/phpdave11/gofpdi v1.0.13/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI= +github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1/go.mod h1:3/3N9NVKO0jef7pBehbT1qWhCMrIgbYNnFAZCqQ5LRc= +github.com/pierrec/lz4 v2.0.5+incompatible h1:2xWsjqPFWcplujydGg4WmhC/6fZqK42wMM8aXeqhl0I= +github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY= +github.com/pierrec/lz4/v4 v4.1.15/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4= +github.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU= +github.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4= +github.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4= +github.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8= +github.com/pion/dtls/v2 v2.2.7 h1:cSUBsETxepsCSFSxC3mc/aDo14qQLMSL+O6IjG28yV8= +github.com/pion/dtls/v2 v2.2.7/go.mod h1:8WiMkebSHFD0T+dIU+UeBaoV7kDhOW5oDCzZ7WZ/F9s= +github.com/pion/logging v0.2.2 h1:M9+AIj/+pxNsDfAT64+MAVgJO0rsyLnoJKCqf//DoeY= +github.com/pion/logging v0.2.2/go.mod h1:k0/tDVsRCX2Mb2ZEmTqNa7CWsQPc+YYCB7Q+5pahoms= +github.com/pion/stun v0.3.5 h1:uLUCBCkQby4S1cf6CGuR9QrVOKcvUwFeemaC865QHDg= +github.com/pion/stun/v2 v2.0.0 h1:A5+wXKLAypxQri59+tmQKVs7+l6mMM+3d+eER9ifRU0= +github.com/pion/stun/v2 v2.0.0/go.mod h1:22qRSh08fSEttYUmJZGlriq9+03jtVmXNODgLccj8GQ= +github.com/pion/transport v0.13.1 h1:/UH5yLeQtwm2VZIPjxwnNFxjS4DFhyLfS4GlfuKUzfA= +github.com/pion/transport/v2 v2.2.1 h1:7qYnCBlpgSJNYMbLCKuSY9KbQdBFoETvPNETv0y4N7c= +github.com/pion/transport/v2 v2.2.1/go.mod h1:cXXWavvCnFF6McHTft3DWS9iic2Mftcz1Aq29pGcU5g= +github.com/pion/transport/v3 v3.0.1 h1:gDTlPJwROfSfz6QfSi0ZmeCSkFcnWWiiR9ES0ouANiM= +github.com/pion/transport/v3 v3.0.1/go.mod h1:UY7kiITrlMv7/IKgd5eTUcaahZx5oUN3l9SzK5f5xE0= +github.com/pkg/browser v0.0.0-20210115035449-ce105d075bb4/go.mod h1:N6UoU20jOqggOuDwUaBQpluzLNDqif3kq9z2wpdYEfQ= +github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8/go.mod h1:HKlIX3XHQyzLZPlr7++PzdhaXEj94dEiJgZDTsxEqUI= +github.com/pkg/diff v0.0.0-20200914180035-5b29258ca4f7/go.mod h1:zO8QMzTeZd5cpnIkz/Gn6iK0jDfGicM1nynOkkPIl28= +github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= +github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/profile v1.2.1/go.mod h1:hJw3o1OdXxsrSjjVksARp5W95eeEaEfptyVZyv6JUPA= +github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI= +github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg= +github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= +github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE= +github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U= +github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= +github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs= +github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= +github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= +github.com/prometheus/client_golang v1.3.0/go.mod h1:hJaj2vgQTGQmVCsAACORcieXFeDPbaTKGT+JTgUa3og= +github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU= +github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= +github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= +github.com/prometheus/client_golang v1.12.0/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= +github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= +github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/client_model v0.1.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/client_model v0.2.1-0.20210607210712-147c58e9608a/go.mod h1:LDGWKZIo7rky3hgvBe+caln+Dr3dPggB5dvjtD7w9+w= +github.com/prometheus/client_model v0.3.0/go.mod h1:LDGWKZIo7rky3hgvBe+caln+Dr3dPggB5dvjtD7w9+w= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= +github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA= +github.com/prometheus/common v0.9.1/go.mod h1:yhUN8i9wzaXS3w1O07YhxHEBxD+W35wd8bs7vj7HSQ4= +github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= +github.com/prometheus/common v0.15.0/go.mod h1:U+gB1OBLb1lF3O42bTCL+FK18tX9Oar16Clt/msog/s= +github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= +github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/otlptranslator v0.0.2 h1:+1CdeLVrRQ6Psmhnobldo0kTp96Rj80DRXRd5OSnMEQ= +github.com/prometheus/otlptranslator v0.0.2/go.mod h1:P8AwMgdD7XEr6QRUJ2QWLpiAZTgTE2UYgjlu3svompI= +github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= +github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= +github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A= +github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= +github.com/prometheus/procfs v0.3.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= +github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= +github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= +github.com/prometheus/procfs v0.17.0 h1:FuLQ+05u4ZI+SS/w9+BWEM2TXiHKsUQ9TADiRH7DuK0= +github.com/prometheus/procfs v0.17.0/go.mod h1:oPQLaDAMRbA+u8H5Pbfq+dl3VDAvHxMUOVhe0wYB2zw= +github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= +github.com/protolambda/bls12-381-util v0.1.0/go.mod h1:cdkysJTRpeFeuUVx/TXGDQNMTiRAalk1vQw3TYTHcE4= +github.com/protolambda/messagediff v1.4.0/go.mod h1:LboJp0EwIbJsePYpzh5Op/9G1/4mIztMRYzzwR0dR2M= +github.com/protolambda/zrnt v0.34.1/go.mod h1:A0fezkp9Tt3GBLATSPIbuY4ywYESyAuc/FFmPKg8Lqs= +github.com/protolambda/ztyp v0.2.2/go.mod h1:9bYgKGqg3wJqT9ac1gI2hnVb0STQq7p/1lapqrqY1dU= +github.com/prysmaticlabs/gohashtree v0.0.1-alpha.0.20220714111606-acbb2962fb48/go.mod h1:4pWaT30XoEx1j8KNJf3TV+E3mQkaufn7mf+jRNb/Fuk= +github.com/rakyll/statik v0.1.7 h1:OF3QCZUuyPxuGEP7B4ypUa7sB/iHtqOTDYZXGM8KOdQ= +github.com/rakyll/statik v0.1.7/go.mod h1:AlZONWzMtEnMs7W4e/1LURLiI49pIMmp6V9Unghqrcc= +github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= +github.com/regen-network/cosmos-proto v0.3.1 h1:rV7iM4SSFAagvy8RiyhiACbWEGotmqzywPxOvwMdxcg= +github.com/regen-network/cosmos-proto v0.3.1/go.mod h1:jO0sVX6a1B36nmE8C9xBFXpNwWejXC7QqCOnH3O0+YM= +github.com/regen-network/protobuf v1.3.3-alpha.regen.1 h1:OHEc+q5iIAXpqiqFKeLpu5NwTIkVXUs48vFMwzqpqY4= +github.com/regen-network/protobuf v1.3.3-alpha.regen.1/go.mod h1:2DjTFR1HhMQhiWC5sZ4OhQ3+NtdbZ6oBDKQwq5Ou+FI= +github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 h1:OdAsTTz6OkFY5QxjkYwrChwuRruF69c169dPK26NUlk= +github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= +github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ= +github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= +github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= +github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= +github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= +github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc= +github.com/rogpeppe/go-internal v1.8.0/go.mod h1:WmiCO8CzOY8rg0OYDC4/i/2WRWAB6poM+XZ2dLUbcbE= +github.com/rogpeppe/go-internal v1.8.1/go.mod h1:JeRgkft04UBgHMgCIwADu4Pn6Mtm5d4nPKWu0nJ5d+o= +github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= +github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4= +github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= +github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= +github.com/rs/cors v1.7.0/go.mod h1:gFx+x8UowdsKA9AchylcLynDq+nNFfI8FkUZdN/jGCU= +github.com/rs/cors v1.11.1 h1:eU3gRzXLRK57F5rKMGMZURNdIG4EoAmX8k94r9wXWHA= +github.com/rs/cors v1.11.1/go.mod h1:XyqrcTp5zjWr1wsJ8PIRZssZ8b/WMcMf71DJnit4EMU= +github.com/russross/blackfriday v1.5.2 h1:HyvC0ARfnZBqnXwABFeSZHpKvJHJJfPz81GNueLj0oo= +github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g= +github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= +github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/ruudk/golang-pdf417 v0.0.0-20181029194003-1af4ab5afa58/go.mod h1:6lfFZQK844Gfx8o5WFuvpxWRwnSoipWe/p622j1v06w= +github.com/ruudk/golang-pdf417 v0.0.0-20201230142125-a7e3863a1245/go.mod h1:pQAZKsJ8yyVxGRWYNEm9oFB8ieLgKFnamEyDmSA0BRk= +github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= +github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc= +github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik= +github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E= +github.com/sanity-io/litter v1.5.5/go.mod h1:9gzJgR2i4ZpjZHsKvUXIRQVk7P+yM3e+jAF7bU2UI5U= +github.com/sasha-s/go-deadlock v0.3.5 h1:tNCOEEDG6tBqrNDOX35j/7hL5FcFViG6awUGROb2NsU= +github.com/sasha-s/go-deadlock v0.3.5/go.mod h1:bugP6EGbdGYObIlx7pUZtWqlvo8k9H6vCBBsiChJQ5U= +github.com/schollz/closestmatch v2.1.0+incompatible/go.mod h1:RtP1ddjLong6gTkbtmuhtR2uUrrJOpYzYRvbcPAid+g= +github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= +github.com/segmentio/fasthash v1.0.3/go.mod h1:waKX8l2N8yckOgmSsXJi7x1ZfdKZ4x7KRMzBtS3oedY= +github.com/sei-protocol/go-ethereum v1.15.7-sei-16 h1:MUvhmn5acNwS/9smQ19gdl6Qh3cNjukTQzz4u8GiUHs= +github.com/sei-protocol/go-ethereum v1.15.7-sei-16/go.mod h1:+S9k+jFzlyVTNcYGvqFhzN/SFhI6vA+aOY4T5tLSPL0= +github.com/sei-protocol/goutils v0.0.2 h1:Bfa7Sv+4CVLNM20QcpvGb81B8C5HkQC/kW1CQpIbXDA= +github.com/sei-protocol/goutils v0.0.2/go.mod h1:iYE2DuJfEnM+APPehr2gOUXfuLuPsVxorcDO+Tzq9q8= +github.com/sei-protocol/sei-chain v0.0.29-fix.0.20260326202429-c9b42951fef7 h1:kW+yxMoSm4RvpP5VT5lFfSa+dqnOE9ZpapE2qNIJwvc= +github.com/sei-protocol/sei-chain v0.0.29-fix.0.20260326202429-c9b42951fef7/go.mod h1:R1/EoOY+MKvwH0k5ivTtG9mLYOQvm0Ni/Trjxrep3t4= +github.com/sei-protocol/sei-config v0.0.25 h1:YHW6YOD3DWSF5QRo+Om4TLeQ9o8E8qnG9jcR8E8fjGo= +github.com/sei-protocol/sei-config v0.0.25/go.mod h1:zcEdLzyIH2AyP0/QRBE3s4Y9eGn0C/qAUx1c4o4EROU= +github.com/sei-protocol/sei-load v0.0.0-20251007135253-78fbdc141082 h1:f2sY8OcN60UL1/6POx+HDMZ4w04FTZtSScnrFSnGZHg= +github.com/sei-protocol/sei-load v0.0.0-20251007135253-78fbdc141082/go.mod h1:V0fNURAjS6A8+sA1VllegjNeSobay3oRUW5VFZd04bA= +github.com/sei-protocol/sei-tm-db v0.0.5 h1:3WONKdSXEqdZZeLuWYfK5hP37TJpfaUa13vAyAlvaQY= +github.com/sei-protocol/sei-tm-db v0.0.5/go.mod h1:Cpa6rGyczgthq7/0pI31jys2Fw0Nfrc+/jKdP1prVqY= +github.com/sei-protocol/seilog v0.0.3 h1:Zi7oWXdX5jv92dY8n482xH032LtNebC89Y+qYZlBn0Y= +github.com/sei-protocol/seilog v0.0.3/go.mod h1:CKg58wraWnB3gRxWQ0v1rIVr0gmDHjkfP1bM2giKFFU= +github.com/sei-protocol/tm-db v0.0.4 h1:7Y4EU62Xzzg6wKAHEotm7SXQR0aPLcGhKHkh3qd0tnk= +github.com/sei-protocol/tm-db v0.0.4/go.mod h1:PWsIWOTwdwC7Ow/GUvx8HgUJTO691pBuorIQD8JvwAs= +github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo= +github.com/shirou/gopsutil v3.21.4-0.20210419000835-c7a38de76ee5+incompatible/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA= +github.com/shirou/gopsutil v3.21.11+incompatible h1:+1+c1VGhc88SSonWP6foOcLhvnKlUeu/erjjvaPEYiI= +github.com/shirou/gopsutil v3.21.11+incompatible/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA= +github.com/shirou/gopsutil/v3 v3.23.2/go.mod h1:gv0aQw33GLo3pG8SiWKiQrbDzbRY1K80RyZJ7V4Th1M= +github.com/shurcooL/go v0.0.0-20200502201357-93f07166e636/go.mod h1:TDJrrUr11Vxrven61rcy3hJMUqaf/CLWYhHNPmT14Lk= +github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749/go.mod h1:ZY1cvUeJuFPAdZ/B6v7RHavJWZn2YPVFQ1OSXhCGOkg= +github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= +github.com/shurcooL/vfsgen v0.0.0-20200824052919-0d455de96546/go.mod h1:TrYk7fJVaAttu97ZZKrO9UbRa8izdowaMIZcxYMbVaw= +github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= +github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= +github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= +github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= +github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= +github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= +github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ= +github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= +github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec= +github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= +github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= +github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60= +github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= +github.com/sony/gobreaker v0.4.1/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= +github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw= +github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U= +github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= +github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI= +github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= +github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= +github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4= +github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I= +github.com/spf13/afero v1.9.2/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y= +github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I= +github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg= +github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= +github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= +github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY= +github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo= +github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= +github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU= +github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE= +github.com/spf13/cobra v1.2.1/go.mod h1:ExllRjgxM/piMAM+3tAZvg8fsklGAf3tPfi+i8t68Nk= +github.com/spf13/cobra v1.5.0/go.mod h1:dWXEIy2H428czQCjInthrTRUg7yKbok+2Qi/yBIJoUM= +github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y= +github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= +github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= +github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= +github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo= +github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= +github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= +github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk= +github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s= +github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE= +github.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns= +github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU= +github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY= +github.com/status-im/keycard-go v0.2.0/go.mod h1:wlp8ZLbsmrF6g6WjugPAx+IzoLrkdf9+mHxBEeo3Hbg= +github.com/streadway/amqp v0.0.0-20190404075320-75d898a42a94/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw= +github.com/streadway/amqp v0.0.0-20190827072141-edfb9018d271/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw= +github.com/streadway/handy v0.0.0-20190108123426-d5acb3125c2a/go.mod h1:qNTQ5P5JnDBl6z3cMAg/SywNDC5ABu5ApDIw6lUbRmI= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= +github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= +github.com/stretchr/testify v0.0.0-20161117074351-18a02ba4a312/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= +github.com/stretchr/testify v1.2.1/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= +github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= +github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= +github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= +github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8= +github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU= +github.com/supranational/blst v0.3.11/go.mod h1:jZJtfjgudtNl4en1tzwPIV3KjUnQUvG3/j+w+fVonLw= +github.com/supranational/blst v0.3.14/go.mod h1:jZJtfjgudtNl4en1tzwPIV3KjUnQUvG3/j+w+fVonLw= +github.com/supranational/blst v0.3.16-0.20250831170142-f48500c1fdbe h1:nbdqkIGOGfUAD54q1s2YBcBz/WcsxCO9HUQ4aGV5hUw= +github.com/supranational/blst v0.3.16-0.20250831170142-f48500c1fdbe/go.mod h1:jZJtfjgudtNl4en1tzwPIV3KjUnQUvG3/j+w+fVonLw= +github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 h1:epCh84lMvA70Z7CTTCmYQn2CKbY8j86K7/FAIr141uY= +github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7/go.mod h1:q4W45IWZaF22tdD+VEXcAWRA037jwmWEB5VWYORlTpc= +github.com/tailscale/depaware v0.0.0-20210622194025-720c4b409502/go.mod h1:p9lPsd+cx33L3H9nNoecRRxPssFKUwwI50I3pZ0yT+8= +github.com/tdewolff/minify/v2 v2.12.4/go.mod h1:h+SRvSIX3kwgwTFOpSckvSxgax3uy8kZTSF1Ojrr3bk= +github.com/tdewolff/parse/v2 v2.6.4/go.mod h1:woz0cgbLwFdtbjJu8PIKxhW05KplTFQkOdX78o+Jgrs= +github.com/tdewolff/test v1.0.7/go.mod h1:6DAvZliBAAnD7rhVgwaM7DE5/d9NMOAJ09SqYqeK4QE= +github.com/tendermint/crypto v0.0.0-20191022145703-50d29ede1e15 h1:hqAk8riJvK4RMWx1aInLzndwxKalgi5rTqgfXxOxbEI= +github.com/tendermint/crypto v0.0.0-20191022145703-50d29ede1e15/go.mod h1:z4YtwM70uOnk8h0pjJYlj3zdYwi9l03By6iAIF5j/Pk= +github.com/tendermint/go-amino v0.16.0 h1:GyhmgQKvqF82e2oZeuMSp9JTN0N09emoSZlb2lyGa2E= +github.com/tendermint/go-amino v0.16.0/go.mod h1:TQU0M1i/ImAo+tYpZi73AU3V/dKeCoMC9Sphe2ZwGME= +github.com/tidwall/btree v1.7.0 h1:L1fkJH/AuEh5zBnnBbmTwQ5Lt+bRJ5A8EWecslvo9iI= +github.com/tidwall/btree v1.7.0/go.mod h1:twD9XRA5jj9VUQGELzDO4HPQTNJsoWWfYEL+EUQ2cKY= +github.com/tidwall/gjson v1.6.7/go.mod h1:zeFuBCIqD4sN/gmqBzZ4j7Jd6UcA2Fc56x7QFsv+8fI= +github.com/tidwall/gjson v1.10.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= +github.com/tidwall/gjson v1.14.2 h1:6BBkirS0rAHjumnjHF6qgy5d2YAJ1TLIaFE2lzfOLqo= +github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= +github.com/tidwall/match v1.0.3/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM= +github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA= +github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM= +github.com/tidwall/pretty v1.0.2/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk= +github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs= +github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU= +github.com/tidwall/sjson v1.1.4/go.mod h1:wXpKXu8CtDjKAZ+3DrKY5ROCorDFahq8l0tey/Lx1fg= +github.com/tidwall/tinylru v1.1.0 h1:XY6IUfzVTU9rpwdhKUF6nQdChgCdGjkMfLzbWyiau6I= +github.com/tidwall/tinylru v1.1.0/go.mod h1:3+bX+TJ2baOLMWTnlyNWHh4QMnFyARg2TLTQ6OFbzw8= +github.com/tidwall/wal v1.2.1 h1:xQvwnRF3e+xBC4NvFvl1mPGJHU0aH5zNzlUKnKGIImA= +github.com/tidwall/wal v1.2.1/go.mod h1:r6lR1j27W9EPalgHiB7zLJDYu3mzW5BQP5KrzBpYY/E= +github.com/tklauser/go-sysconf v0.3.11/go.mod h1:GqXfhXY3kiPa0nAXPDIQIWzJbMCB7AmcWpGR8lSZfqI= +github.com/tklauser/go-sysconf v0.3.12/go.mod h1:Ho14jnntGE1fpdOqQEEaiKRpvIavV0hSfmBq8nJbHYI= +github.com/tklauser/go-sysconf v0.3.15 h1:VE89k0criAymJ/Os65CSn1IXaol+1wrsFHEB8Ol49K4= +github.com/tklauser/go-sysconf v0.3.15/go.mod h1:Dmjwr6tYFIseJw7a3dRLJfsHAMXZ3nEnL/aZY+0IuI4= +github.com/tklauser/numcpus v0.6.0/go.mod h1:FEZLMke0lhOUG6w2JadTzp0a+Nl8PF/GFkQ5UVIcaL4= +github.com/tklauser/numcpus v0.6.1/go.mod h1:1XfjsgE2zo8GVw7POkMbHENHzVg3GzmoZ9fESEdAacY= +github.com/tklauser/numcpus v0.10.0 h1:18njr6LDBk1zuna922MgdjQuJFjrdppsZG60sHGfjso= +github.com/tklauser/numcpus v0.10.0/go.mod h1:BiTKazU708GQTYF4mB+cmlpT2Is1gLk7XVuEeem8LsQ= +github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= +github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= +github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM= +github.com/tyler-smith/go-bip39 v1.0.2/go.mod h1:sJ5fKU0s6JVwZjjcUEX2zFOnvq0ASQ2K9Zr6cf67kNs= +github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc= +github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw= +github.com/ugorji/go v1.2.7 h1:qYhyWUUd6WbiM+C6JZAUkIJt/1WrjzNHY9+KCIjVqTo= +github.com/ugorji/go v1.2.7/go.mod h1:nF9osbDWLy6bDVv/Rtoh6QgnvNDpmCalQV5urGCCS6M= +github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0= +github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY= +github.com/ugorji/go/codec v1.2.7 h1:YPXUKf7fYbp/y8xloBqZOw2qaVggbfwMlI8WM3wZUJ0= +github.com/ugorji/go/codec v1.2.7/go.mod h1:WGN1fab3R1fzQlVQTkfxVtIBhWDRqOviHU95kRgeqEY= +github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA= +github.com/urfave/cli v1.22.1 h1:+mkCCcOFKPnCmVYVcURKps1Xe+3zP90gSYGNfRkjoIY= +github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= +github.com/urfave/cli/v2 v2.24.1/go.mod h1:GHupkWPMM0M/sj1a2b4wUrWBPzazNrIjouW6fmdJLxc= +github.com/urfave/cli/v2 v2.27.5 h1:WoHEJLdsXr6dDWoJgMq/CboDmyY/8HMMH1fTECbih+w= +github.com/urfave/cli/v2 v2.27.5/go.mod h1:3Sevf16NykTbInEnD0yKkjDAeZDS0A6bzhBH5hrMvTQ= +github.com/urfave/cli/v3 v3.6.1 h1:j8Qq8NyUawj/7rTYdBGrxcH7A/j7/G8Q5LhWEW4G3Mo= +github.com/urfave/cli/v3 v3.6.1/go.mod h1:ysVLtOEmg2tOy6PknnYVhDoouyC/6N42TMeoMzskhso= +github.com/urfave/negroni v1.0.0/go.mod h1:Meg73S6kFm/4PpbYdq35yYWoCZ9mS/YSx+lKnmiohz4= +github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc= +github.com/valyala/fasthttp v1.40.0/go.mod h1:t/G+3rLek+CyY9bnIE+YlMRddxVAAGjhxndDB4i4C0I= +github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8= +github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ= +github.com/valyala/fasttemplate v1.2.2/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ= +github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc= +github.com/vmihailenco/msgpack/v5 v5.1.4/go.mod h1:C5gboKD0TJPqWDTVTtrQNfRbiBwHZGo8UTqP/9/XvLI= +github.com/vmihailenco/msgpack/v5 v5.3.5/go.mod h1:7xyJ9e+0+9SaZT0Wt1RGleJXzli6Q/V5KbhBonMG9jc= +github.com/vmihailenco/tagparser v0.1.2/go.mod h1:OeAg3pn3UbLjkWt+rN9oFYB6u/cQgqMEUPoW2WPyhdI= +github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds= +github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU= +github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ= +github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y= +github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= +github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q= +github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673/go.mod h1:N3UwUGtsrSj3ccvlPHLoLsHnpR27oXr4ZE984MbSER8= +github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 h1:gEOO8jv9F4OT7lGCjxCBTO/36wtF6j2nSip77qHd4x4= +github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1/go.mod h1:Ohn+xnUBiLI6FVj/9LpzZWtj1/D6lUovWYBkxHVV3aM= +github.com/yalp/jsonpath v0.0.0-20180802001716-5cc68e5049a0/go.mod h1:/LWChgwKmvncFJFHJ7Gvn9wZArjbV5/FppcK2fKk/tI= +github.com/ybbus/jsonrpc v2.1.2+incompatible/go.mod h1:XJrh1eMSzdIYFbM08flv0wp5G35eRniyeGut1z+LSiE= +github.com/yosssi/ace v0.0.5/go.mod h1:ALfIzm2vT7t5ZE7uoIZqF3TQ7SAOyupFZnkrF5id+K0= +github.com/yudai/gojsondiff v1.0.0/go.mod h1:AY32+k2cwILAkW1fbgxQ5mUmMiZFgLIV+FBNExI05xg= +github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82/go.mod h1:lgjkn3NuSvDfVJdfcVVdX+jpBxNmX4rDAzaS45IcYoM= +github.com/yudai/pp v2.0.1+incompatible/go.mod h1:PuxR/8QJ7cyCkFp/aUDS+JY727OFEZkTdatxwunjIkc= +github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= +github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= +github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= +github.com/yusufpapurcu/wmi v1.2.2/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0= +github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0= +github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0= +github.com/zbiljic/go-filelock v0.0.0-20170914061330-1dbf7103ab7d h1:XQyeLr7N9iY9mi+TGgsBFkj54+j3fdoo8e2u6zrGP5A= +github.com/zbiljic/go-filelock v0.0.0-20170914061330-1dbf7103ab7d/go.mod h1:hoMeDjlNXTNqVwrCk8YDyaBS2g5vFfEX2ezMi4vb6CY= +github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ= +github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0= +github.com/zeebo/blake3 v0.2.4 h1:KYQPkhpRtcqh0ssGYcKLG1JYvddkEA8QwCM/yBqhaZI= +github.com/zeebo/blake3 v0.2.4/go.mod h1:7eeQ6d2iXWRGF6npfaxl2CU+xy2Fjo2gxeyZGCRUjcE= +github.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo= +github.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4= +github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0= +github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA= +github.com/zondax/golem v0.27.0 h1:IbBjGIXF3SoGOZHsILJvIM/F/ylwJzMcHAcggiqniPw= +github.com/zondax/golem v0.27.0/go.mod h1:AmorCgJPt00L8xN1VrMBe13PSifoZksnQ1Ge906bu4A= +github.com/zondax/hid v0.9.2 h1:WCJFnEDMiqGF64nlZz28E9qLVZ0KSJ7xpc5DLEyma2U= +github.com/zondax/hid v0.9.2/go.mod h1:l5wttcP0jwtdLjqjMMWFVEE7d1zO0jvSPA9OPZxWpEM= +github.com/zondax/ledger-go v1.0.1 h1:Ks/2tz/dOF+dbRynfZ0dEhcdL1lqw43Sa0zMXHpQ3aQ= +github.com/zondax/ledger-go v1.0.1/go.mod h1:j7IgMY39f30apthJYMd1YsHZRqdyu4KbVmUp0nU78X0= +go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= +go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= +go.etcd.io/bbolt v1.3.7/go.mod h1:N9Mkw9X8x5fupy0IKsmuqVtoGDyxsaDlbk4Rd05IAQw= +go.etcd.io/bbolt v1.4.0-alpha.0.0.20240404170359-43604f3112c5 h1:qxen9oVGzDdIRP6ejyAJc760RwW4SnVDiTYTzwnXuxo= +go.etcd.io/bbolt v1.4.0-alpha.0.0.20240404170359-43604f3112c5/go.mod h1:eW0HG9/oHQhvRCvb1/pIXW4cOvtDqeQK+XSi3TnwaXY= +go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738/go.mod h1:dnLIgRNXwCJa5e+c6mIZCrds/GIG4ncV9HhK5PX7jPg= +go.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs= +go.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g= +go.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ= +go.etcd.io/gofail v0.1.0/go.mod h1:VZBCXYGZhHAinaBiiqYvuDynvahNsAyLFwB3kEHKz1M= +go.opencensus.io v0.20.1/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk= +go.opencensus.io v0.20.2/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk= +go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= +go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= +go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= +go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= +go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= +go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= +go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E= +go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= +go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= +go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= +go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= +go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48= +go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8= +go.opentelemetry.io/otel/exporters/jaeger v1.9.0 h1:gAEgEVGDWwFjcis9jJTOJqZNxDzoZfR12WNIxr7g9Ww= +go.opentelemetry.io/otel/exporters/jaeger v1.9.0/go.mod h1:hquezOLVAybNW6vanIxkdLXTXvzlj2Vn3wevSP15RYs= +go.opentelemetry.io/otel/exporters/prometheus v0.60.0 h1:cGtQxGvZbnrWdC2GyjZi0PDKVSLWP/Jocix3QWfXtbo= +go.opentelemetry.io/otel/exporters/prometheus v0.60.0/go.mod h1:hkd1EekxNo69PTV4OWFGZcKQiIqg0RfuWExcPKFvepk= +go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0= +go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs= +go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18= +go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE= +go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8= +go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew= +go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI= +go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA= +go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= +go.opentelemetry.io/proto/otlp v0.15.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U= +go.opentelemetry.io/proto/otlp v0.19.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U= +go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= +go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= +go.uber.org/atomic v1.5.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= +go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= +go.uber.org/automaxprocs v1.5.1/go.mod h1:BF4eumQw0P9GtnuxxovUd06vwm1o18oMzFtK66vU6XU= +go.uber.org/automaxprocs v1.5.2/go.mod h1:eRbA25aqJrxAbsLO0xy5jVwPt7FQnRgjW+efnwa1WM0= +go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= +go.uber.org/multierr v1.3.0/go.mod h1:VgVr7evmIr6uPjLBxg28wmKNXyqE9akIJ5XnfpiKl+4= +go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= +go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= +go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= +go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA= +go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= +go.uber.org/zap v1.13.0/go.mod h1:zwrFLgMcdUuIBviXEYEH1YKNaOBnKXsx2IPda5bBwHM= +go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= +go.uber.org/zap v1.18.1/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= +go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8= +go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= +go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= +go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= +go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= +go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= +golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= +golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= +golang.org/x/exp v0.0.0-20180321215751-8460e604b9de/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= +golang.org/x/exp v0.0.0-20180807140117-3d87b88a115f/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= +golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= +golang.org/x/exp v0.0.0-20190125153040-c74c464bbbf2/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= +golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= +golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= +golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= +golang.org/x/exp v0.0.0-20191002040644-a1355ae1e2c3/go.mod h1:NOZ3BPKG0ec/BKJQgnvsSFpcKLM5xXVWnvZS97DWHgE= +golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= +golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= +golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= +golang.org/x/exp v0.0.0-20200331195152-e8c3332aa8e5/go.mod h1:4M0jN8W1tt0AVLNr8HDosyJCDCDuyL9N9+3m7wDWgKw= +golang.org/x/exp v0.0.0-20220426173459-3bcf042a4bf5/go.mod h1:lgLbSvA5ygNOMpwM/9anMpWVlVJ7Z+cHWq/eFuinpGE= +golang.org/x/exp v0.0.0-20220827204233-334a2380cb91/go.mod h1:cyybsKvd6eL0RnXn6p/Grxp8F5bW7iYuBgsNCOHpMYE= +golang.org/x/exp v0.0.0-20230626212559-97b1e661b5df/go.mod h1:FXUEEKJgO7OQYeo8N01OfiKP8RXMtf6e8aTskBGqWdc= +golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU= +golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU= +golang.org/x/image v0.0.0-20180708004352-c73c2afc3b81/go.mod h1:ux5Hcp/YLpHSI86hEcLt0YII63i6oz57MZXIpbrjZUs= +golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= +golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/image v0.0.0-20190910094157-69e4b8554b2a/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/image v0.0.0-20200119044424-58c23975cae1/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/image v0.0.0-20200430140353-33d19683fad8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/image v0.0.0-20200618115811-c13761719519/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/image v0.0.0-20201208152932-35266b937fa6/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/image v0.0.0-20210216034530-4410531fe030/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/image v0.0.0-20210607152325-775e3b0c77b9/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM= +golang.org/x/image v0.0.0-20210628002857-a66eb6448b8d/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM= +golang.org/x/image v0.0.0-20211028202545-6944b10bf410/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM= +golang.org/x/image v0.0.0-20220302094943-723b81ca9867/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM= +golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= +golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= +golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= +golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= +golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= +golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= +golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= +golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= +golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= +golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.5.0/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro= +golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro= +golang.org/x/mod v0.6.0-dev.0.20211013180041-c96bc1413d57/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY= +golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY= +golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= +golang.org/x/mod v0.6.0/go.mod h1:4mET923SAdbXp2ki8ey+zGs1SLqsuM2Y0uvdZR/fUNI= +golang.org/x/mod v0.7.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/mod v0.9.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/mod v0.11.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/mod v0.22.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY= +golang.org/x/mod v0.32.0 h1:9F4d3PHLljb6x//jOyokMv3eX+YDeepZSEo3mFJy93c= +golang.org/x/mod v0.32.0/go.mod h1:SgipZ/3h2Ci89DlEtEXWUk/HteuRin+HHhN+WbNhguU= +golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190125091013-d26f9f9a57f3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190327091125-710a502c58a2/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= +golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= +golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200421231249-e086a090c8fd/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20200813134508-3edf25e44fcc/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20200904194848-62affa334b73/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc= +golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= +golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= +golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220325170049-de3da57026de/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220412020605-290c469a71a5/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220607020251-c690dde0001d/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/net v0.0.0-20220617184016-355a448f1bc9/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/net v0.0.0-20220909164309-bea034e7d591/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk= +golang.org/x/net v0.0.0-20221012135044-0b7e1fb9d458/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk= +golang.org/x/net v0.0.0-20221014081412-f15817d10f9b/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk= +golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco= +golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY= +golang.org/x/net v0.3.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE= +golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE= +golang.org/x/net v0.5.0/go.mod h1:DivGGAXEgPSlEBzxGzZI+ZLohi+xUj054jfeKui00ws= +golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= +golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= +golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc= +golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns= +golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg= +golang.org/x/net v0.12.0/go.mod h1:zEVYFnQC7m/vmpQFELhcD1EWkZlX69l4oqgmer6hfKA= +golang.org/x/net v0.14.0/go.mod h1:PpSgVXXLK0OxS0F31C1/tv6XNguvCrnXIDrFMspZIUI= +golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk= +golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44= +golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg= +golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM= +golang.org/x/net v0.34.0/go.mod h1:di0qlW3YNM5oh6GqDGQr92MyTozJPmybPK4Ev/Gm31k= +golang.org/x/net v0.36.0/go.mod h1:bFmbeoIPfrw4sMHNhb4J9f6+tPziuGjq7Jk/38fxi1I= +golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o= +golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8= +golang.org/x/oauth2 v0.0.0-20170207211851-4464e7848382/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= +golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= +golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc= +golang.org/x/oauth2 v0.0.0-20220309155454-6242fa91716a/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc= +golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc= +golang.org/x/oauth2 v0.0.0-20220608161450-d0670ef3b1eb/go.mod h1:jaDAt6Dkxork7LmZnYtzbRWj0W47D86a3TGe0YHBvmE= +golang.org/x/oauth2 v0.0.0-20220622183110-fd043fe589d2/go.mod h1:jaDAt6Dkxork7LmZnYtzbRWj0W47D86a3TGe0YHBvmE= +golang.org/x/oauth2 v0.0.0-20220822191816-0ebed06d0094/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg= +golang.org/x/oauth2 v0.0.0-20220909003341-f21342109be1/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg= +golang.org/x/oauth2 v0.0.0-20221006150949-b44042a4b9c1/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg= +golang.org/x/oauth2 v0.0.0-20221014153046-6fdb5e3db783/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg= +golang.org/x/oauth2 v0.5.0/go.mod h1:9/XBHVqLaWO3/BRHs5jbpYCnOZVjj5V0ndyaAM7KB4I= +golang.org/x/oauth2 v0.6.0/go.mod h1:ycmewcwgD4Rpr3eZJLSB4Kyyljb3qDh40vJ8STE5HKw= +golang.org/x/oauth2 v0.7.0/go.mod h1:hPLQkd9LyjfXTiRohC/41GhcFqxisoUQ99sCUOHO9x4= +golang.org/x/perf v0.0.0-20230113213139-801c7ef9e5c5/go.mod h1:UBKtEnL8aqnd+0JHqZ+2qoMDwtuy6cYhhKNoHLBiTQc= +golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220929204114-8fcdb60fdcc0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= +golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= +golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190130150945-aca44879d564/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190626221950-04f50cda93cb/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190712062909-fae7ac547cb7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191220142924-d4481acd189f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200420163511-1957bb5e6d1f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200814200057-3d37ad5750ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200826173525-f9321e4c35a6/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200909081042-eff7692f9009/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201101102859-da207088b7d1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201207223542-d4d67f95c62d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210225134936-a50acf3fe073/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210304124612-50617c2ba197/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210603125802-9665404d3644/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210616045830-e2b7044e8c71/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210816183151-1e6c022a8912/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210819135213-f52c844e1c1c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211103235746-7861aae1554b/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220128215802-99c3d69c2c27/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220328115105-d36c6a25d886/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220422013727-9388b58f7150/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220502124256-b6088ccd6cba/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220610221304-9f5ed59c137d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220615213510-4f61da869c0c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220624220833-87e55d714810/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220829200755-d48e67d00261/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220919091848-fb04ddd9f9c8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.7.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.9.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.24.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.40.0 h1:DBZZqJ2Rkml6QMQsZywtnjnnGvHza6BTfYFWY9kjEWQ= +golang.org/x/sys v0.40.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE= +golang.org/x/telemetry v0.0.0-20240521205824-bda55230c457/go.mod h1:pRgIJT+bRLFKnoM1ldnzKoxTIn14Yxz928LQRYYgIN0= +golang.org/x/telemetry v0.0.0-20260109210033-bd525da824e2 h1:O1cMQHRfwNpDfDJerqRoE2oD+AFlyid87D40L/OkkJo= +golang.org/x/telemetry v0.0.0-20260109210033-bd525da824e2/go.mod h1:b7fPSJ0pKZ3ccUh8gnTONJxhn3c/PS6tyzQvyqw4iA8= +golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc= +golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA= +golang.org/x/term v0.4.0/go.mod h1:9P2UbLfCdcvo3p/nzKvsmas4TnlujnuoV9hGgYzW1lQ= +golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= +golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U= +golang.org/x/term v0.7.0/go.mod h1:P32HKFT3hSsZrRxla30E9HqToFYAQPCMs/zFMBUFqPY= +golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo= +golang.org/x/term v0.10.0/go.mod h1:lpqdcUyK/oCiQxvxVrppt5ggO2KCZ5QblwqPnfZ6d5o= +golang.org/x/term v0.11.0/go.mod h1:zC9APTIj3jG3FdV/Ons+XE1riIZXG4aZ4GTHiPZJPIU= +golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU= +golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk= +golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58= +golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY= +golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM= +golang.org/x/term v0.28.0/go.mod h1:Sw/lC2IAUZ92udQNf3WodGtn4k/XoLyZoh8v/8uiwek= +golang.org/x/term v0.29.0/go.mod h1:6bl4lRlvVuDgSf3179VpIxBF0o10JUpXWOnI7nErv7s= +golang.org/x/term v0.39.0 h1:RclSuaJf32jOqZz74CkPA9qFuVTX7vhLlpfj/IGWlqY= +golang.org/x/term v0.39.0/go.mod h1:yxzUCTP/U+FzoxfdKmLaA0RV1WgE0VY7hXBwKtY/4ww= +golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= +golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ= +golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= +golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= +golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= +golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= +golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= +golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= +golang.org/x/text v0.11.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE= +golang.org/x/text v0.12.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE= +golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE= +golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= +golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY= +golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk= +golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA= +golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20201208040808-7e3f01d25324/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20220922220347-f3bd1da661af/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.1.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.2.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.9.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= +golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= +golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= +golang.org/x/tools v0.0.0-20180525024113-a5b4c53f6e8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20190206041539-40960b6deb8e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= +golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= +golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= +golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= +golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20190927191325-030b2cf1153e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191108193012-7d206e10da11/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200103221440-774c71fcf114/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= +golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= +golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8= +golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= +golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= +golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= +golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE= +golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201211185031-d93e913c1a58/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= +golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.8-0.20211029000441-d6a9af8af023/go.mod h1:nABZi5QlRsZVlzPpHl034qft6wpY4eDcsTt5AaioBiU= +golang.org/x/tools v0.1.9/go.mod h1:nABZi5QlRsZVlzPpHl034qft6wpY4eDcsTt5AaioBiU= +golang.org/x/tools v0.1.10/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E= +golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= +golang.org/x/tools v0.2.0/go.mod h1:y4OqIKeOV/fWJetJ8bXPU1sEVniLMIyDAZWeHdV+NTA= +golang.org/x/tools v0.3.0/go.mod h1:/rWhSS2+zyEVwoJf8YAX6L2f0ntZ7Kn/mGgAWcipA5k= +golang.org/x/tools v0.4.0/go.mod h1:UE5sM2OK9E/d67R0ANs2xJizIymRP5gJU295PvKXxjQ= +golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= +golang.org/x/tools v0.7.0/go.mod h1:4pg6aUX35JBAogB10C9AtvVL+qowtN4pT3CGSQex14s= +golang.org/x/tools v0.11.0/go.mod h1:anzJrxPjNtfgiYQYirP2CPGzGLxrH2u2QBhn6Bf3qY8= +golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= +golang.org/x/tools v0.29.0/go.mod h1:KMQVMRsVxU6nHCFXrBPhDB8XncLNLM0lIy/F14RP588= +golang.org/x/tools v0.41.0 h1:a9b8iMweWG+S0OBnlU36rzLp20z1Rp10w+IY2czHTQc= +golang.org/x/tools v0.41.0/go.mod h1:XSY6eDqxVNiYgezAVqqCeihT4j1U2CCsqvH3WhQpnlg= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20220411194840-2f41105eb62f/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20220517211312-f3a8303e98df/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8= +golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8= +golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8= +golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da h1:noIWHXmPHxILtqtCOPIhSt0ABwskkZKjD3bXGnZGpNY= +golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90= +gonum.org/v1/gonum v0.0.0-20180816165407-929014505bf4/go.mod h1:Y+Yx5eoAFn32cQvJDxZx5Dpnq+c3wtXuadVZAcxbbBo= +gonum.org/v1/gonum v0.8.2/go.mod h1:oe/vMfY3deqTw+1EZJhuvEW2iwGF1bW9wwu7XCu0+v0= +gonum.org/v1/gonum v0.9.3/go.mod h1:TZumC3NeyVQskjXqmyWt4S3bINhy7B4eYwW69EbyX+0= +gonum.org/v1/gonum v0.11.0/go.mod h1:fSG4YDCxxUZQJ7rKsQrj0gMOg00Il0Z96/qMA4bVQhA= +gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0/go.mod h1:wa6Ws7BG/ESfp6dHfk7C6KdzKA7wR7u/rKwOGE66zvw= +gonum.org/v1/plot v0.0.0-20190515093506-e2840ee46a6b/go.mod h1:Wt8AAjI+ypCyYX3nZBvf6cAIx93T+c/OS2HFAYskSZc= +gonum.org/v1/plot v0.9.0/go.mod h1:3Pcqqmp6RHvJI72kgb8fThyUnav364FOsdDo2aGW5lY= +gonum.org/v1/plot v0.10.0/go.mod h1:JWIHJ7U20drSQb/aDpTetJzfC1KlAPldJLpkSy88dvQ= +gonum.org/v1/plot v0.10.1/go.mod h1:VZW5OlhkL1mysU9vaqNHnsy86inf6Ot+jB3r+BczCEo= +google.golang.org/api v0.0.0-20170206182103-3d017632ea10/go.mod h1:4mhQ8q/RsB7i+udVvVy5NUi08OU8ZlA0gRVgrF7VFY0= +google.golang.org/api v0.3.1/go.mod h1:6wY9I6uQWHQ8EM57III9mq/AjF+i8G65rmVagqKMtkk= +google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= +google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= +google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= +google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= +google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= +google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= +google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM= +google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc= +google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg= +google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE= +google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8= +google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU= +google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94= +google.golang.org/api v0.44.0/go.mod h1:EBOGZqzyhtvMDoxwS97ctnh0zUmYY6CxqXsc1AvkYD8= +google.golang.org/api v0.47.0/go.mod h1:Wbvgpq1HddcWVtzsVLyfLp8lDg6AA241LmgIL59tHXo= +google.golang.org/api v0.48.0/go.mod h1:71Pr1vy+TAZRPkPs/xlCf5SsU8WjuAWv1Pfjbtukyy4= +google.golang.org/api v0.50.0/go.mod h1:4bNT5pAuq5ji4SRZm+5QIkjny9JAyVD/3gaSihNefaw= +google.golang.org/api v0.51.0/go.mod h1:t4HdrdoNgyN5cbEfm7Lum0lcLDLiise1F8qDKX00sOU= +google.golang.org/api v0.54.0/go.mod h1:7C4bFFOvVDGXjfDTAsgGwDgAxRDeQ4X8NvUedIt6z3k= +google.golang.org/api v0.55.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE= +google.golang.org/api v0.56.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE= +google.golang.org/api v0.57.0/go.mod h1:dVPlbZyBo2/OjBpmvNdpn2GRm6rPy75jyU7bmhdrMgI= +google.golang.org/api v0.61.0/go.mod h1:xQRti5UdCmoCEqFxcz93fTl338AVqDgyaDRuOZ3hg9I= +google.golang.org/api v0.63.0/go.mod h1:gs4ij2ffTRXwuzzgJl/56BdwJaA194ijkfn++9tDuPo= +google.golang.org/api v0.67.0/go.mod h1:ShHKP8E60yPsKNw/w8w+VYaj9H6buA5UqDp8dhbQZ6g= +google.golang.org/api v0.70.0/go.mod h1:Bs4ZM2HGifEvXwd50TtW70ovgJffJYw2oRCOFU/SkfA= +google.golang.org/api v0.71.0/go.mod h1:4PyU6e6JogV1f9eA4voyrTY2batOLdgZ5qZ5HOCc4j8= +google.golang.org/api v0.74.0/go.mod h1:ZpfMZOVRMywNyvJFeqL9HRWBgAuRfSjJFpe9QtRRyDs= +google.golang.org/api v0.75.0/go.mod h1:pU9QmyHLnzlpar1Mjt4IbapUCy8J+6HD6GeELN69ljA= +google.golang.org/api v0.77.0/go.mod h1:pU9QmyHLnzlpar1Mjt4IbapUCy8J+6HD6GeELN69ljA= +google.golang.org/api v0.78.0/go.mod h1:1Sg78yoMLOhlQTeF+ARBoytAcH1NNyyl390YMy6rKmw= +google.golang.org/api v0.80.0/go.mod h1:xY3nI94gbvBrE0J6NHXhxOmW97HG7Khjkku6AFB3Hyg= +google.golang.org/api v0.84.0/go.mod h1:NTsGnUFJMYROtiquksZHBWtHfeMC7iYthki7Eq3pa8o= +google.golang.org/api v0.85.0/go.mod h1:AqZf8Ep9uZ2pyTvgL+x0D3Zt0eoT9b5E8fmzfu6FO2g= +google.golang.org/api v0.90.0/go.mod h1:+Sem1dnrKlrXMR/X0bPnMWyluQe4RsNoYfmNLhOIkzw= +google.golang.org/api v0.93.0/go.mod h1:+Sem1dnrKlrXMR/X0bPnMWyluQe4RsNoYfmNLhOIkzw= +google.golang.org/api v0.95.0/go.mod h1:eADj+UBuxkh5zlrSntJghuNeg8HwQ1w5lTKkuqaETEI= +google.golang.org/api v0.96.0/go.mod h1:w7wJQLTM+wvQpNf5JyEcBoxK0RH7EDrh/L4qfsuJ13s= +google.golang.org/api v0.97.0/go.mod h1:w7wJQLTM+wvQpNf5JyEcBoxK0RH7EDrh/L4qfsuJ13s= +google.golang.org/api v0.98.0/go.mod h1:w7wJQLTM+wvQpNf5JyEcBoxK0RH7EDrh/L4qfsuJ13s= +google.golang.org/api v0.99.0/go.mod h1:1YOf74vkVndF7pG6hIHuINsM7eWwpVTAfNMNiL91A08= +google.golang.org/api v0.100.0/go.mod h1:ZE3Z2+ZOr87Rx7dqFsdRQkRBk36kDtp/h+QpHbB7a70= +google.golang.org/api v0.102.0/go.mod h1:3VFl6/fzoA+qNuS1N1/VfXY4LjoXN/wzeIp7TweWwGo= +google.golang.org/api v0.103.0/go.mod h1:hGtW6nK1AC+d9si/UBhw8Xli+QMOf6xyNAyJw4qU9w0= +google.golang.org/api v0.106.0/go.mod h1:2Ts0XTHNVWxypznxWOYUeI4g3WdP9Pk2Qk58+a/O9MY= +google.golang.org/api v0.107.0/go.mod h1:2Ts0XTHNVWxypznxWOYUeI4g3WdP9Pk2Qk58+a/O9MY= +google.golang.org/api v0.108.0/go.mod h1:2Ts0XTHNVWxypznxWOYUeI4g3WdP9Pk2Qk58+a/O9MY= +google.golang.org/api v0.110.0/go.mod h1:7FC4Vvx1Mooxh8C5HWjzZHcavuS2f6pmJpZx60ca7iI= +google.golang.org/api v0.111.0/go.mod h1:qtFHvU9mhgTJegR31csQ+rwxyUTHOKFqCKWp1J0fdw0= +google.golang.org/api v0.114.0/go.mod h1:ifYI2ZsFK6/uGddGfAD5BMxlnkBqCmqHSDUVi45N5Yg= +google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= +google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= +google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= +google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= +google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= +google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= +google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= +google.golang.org/genproto v0.0.0-20180518175338-11a468237815/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= +google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= +google.golang.org/genproto v0.0.0-20180831171423-11092d34479b/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= +google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190530194941-fb225487d101/go.mod h1:z3L6/3dTEVtUr6QSP8miRzeRqwQOioJ9I66odjN4I7s= +google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= +google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= +google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= +google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= +google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200324203455-a04cca1dde73/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200423170343-7949de9c1215/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= +google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= +google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA= +google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210108203827-ffc7fda8c3d7/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210126160654-44e461bb6506/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210226172003-ab064af71705/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210329143202-679c6ae281ee/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A= +google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A= +google.golang.org/genproto v0.0.0-20210513213006-bf773b8c8384/go.mod h1:P3QM42oQyzQSnHPnZ/vqoCdDmzH28fzWByN9asMeM8A= +google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= +google.golang.org/genproto v0.0.0-20210604141403-392c879c8b08/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= +google.golang.org/genproto v0.0.0-20210608205507-b6d2f5bf0d7d/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= +google.golang.org/genproto v0.0.0-20210624195500-8bfb893ecb84/go.mod h1:SzzZ/N+nwJDaO1kznhnlzqS8ocJICar6hYhVyhi++24= +google.golang.org/genproto v0.0.0-20210713002101-d411969a0d9a/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k= +google.golang.org/genproto v0.0.0-20210716133855-ce7ef5c701ea/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k= +google.golang.org/genproto v0.0.0-20210728212813-7823e685a01f/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48= +google.golang.org/genproto v0.0.0-20210805201207-89edb61ffb67/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48= +google.golang.org/genproto v0.0.0-20210813162853-db860fec028c/go.mod h1:cFeNkxwySK631ADgubI+/XFU/xp8FD5KIVV4rj8UC5w= +google.golang.org/genproto v0.0.0-20210821163610-241b8fcbd6c8/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210903162649-d08c68adba83/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210909211513-a8c4777a87af/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210924002016-3dee208752a0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20211118181313-81c1377c94b1/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20211206160659-862468c7d6e0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20211208223120-3a66f561d7aa/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20211221195035-429b39de9b1c/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20220126215142-9970aeb2e350/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20220207164111-0872dc986b00/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20220218161850-94dd64e39d7c/go.mod h1:kGP+zUP2Ddo0ayMi4YuN7C3WZyJvGLZRh8Z5wnAqvEI= +google.golang.org/genproto v0.0.0-20220222213610-43724f9ea8cf/go.mod h1:kGP+zUP2Ddo0ayMi4YuN7C3WZyJvGLZRh8Z5wnAqvEI= +google.golang.org/genproto v0.0.0-20220304144024-325a89244dc8/go.mod h1:kGP+zUP2Ddo0ayMi4YuN7C3WZyJvGLZRh8Z5wnAqvEI= +google.golang.org/genproto v0.0.0-20220310185008-1973136f34c6/go.mod h1:kGP+zUP2Ddo0ayMi4YuN7C3WZyJvGLZRh8Z5wnAqvEI= +google.golang.org/genproto v0.0.0-20220324131243-acbaeb5b85eb/go.mod h1:hAL49I2IFola2sVEjAn7MEwsja0xp51I0tlGAf9hz4E= +google.golang.org/genproto v0.0.0-20220329172620-7be39ac1afc7/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo= +google.golang.org/genproto v0.0.0-20220407144326-9054f6ed7bac/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo= +google.golang.org/genproto v0.0.0-20220413183235-5e96e2839df9/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo= +google.golang.org/genproto v0.0.0-20220414192740-2d67ff6cf2b4/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo= +google.golang.org/genproto v0.0.0-20220421151946-72621c1f0bd3/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo= +google.golang.org/genproto v0.0.0-20220429170224-98d788798c3e/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo= +google.golang.org/genproto v0.0.0-20220502173005-c8bf987b8c21/go.mod h1:RAyBrSAP7Fh3Nc84ghnVLDPuV51xc9agzmm4Ph6i0Q4= +google.golang.org/genproto v0.0.0-20220505152158-f39f71e6c8f3/go.mod h1:RAyBrSAP7Fh3Nc84ghnVLDPuV51xc9agzmm4Ph6i0Q4= +google.golang.org/genproto v0.0.0-20220518221133-4f43b3371335/go.mod h1:RAyBrSAP7Fh3Nc84ghnVLDPuV51xc9agzmm4Ph6i0Q4= +google.golang.org/genproto v0.0.0-20220523171625-347a074981d8/go.mod h1:RAyBrSAP7Fh3Nc84ghnVLDPuV51xc9agzmm4Ph6i0Q4= +google.golang.org/genproto v0.0.0-20220608133413-ed9918b62aac/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA= +google.golang.org/genproto v0.0.0-20220616135557-88e70c0c3a90/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA= +google.golang.org/genproto v0.0.0-20220617124728-180714bec0ad/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA= +google.golang.org/genproto v0.0.0-20220624142145-8cd45d7dbd1f/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA= +google.golang.org/genproto v0.0.0-20220628213854-d9e0b6570c03/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA= +google.golang.org/genproto v0.0.0-20220722212130-b98a9ff5e252/go.mod h1:GkXuJDJ6aQ7lnJcRF+SJVgFdQhypqgl3LB1C9vabdRE= +google.golang.org/genproto v0.0.0-20220801145646-83ce21fca29f/go.mod h1:iHe1svFLAZg9VWz891+QbRMwUv9O/1Ww+/mngYeThbc= +google.golang.org/genproto v0.0.0-20220815135757-37a418bb8959/go.mod h1:dbqgFATTzChvnt+ujMdZwITVAJHFtfyN1qUhDqEiIlk= +google.golang.org/genproto v0.0.0-20220817144833-d7fd3f11b9b1/go.mod h1:dbqgFATTzChvnt+ujMdZwITVAJHFtfyN1qUhDqEiIlk= +google.golang.org/genproto v0.0.0-20220822174746-9e6da59bd2fc/go.mod h1:dbqgFATTzChvnt+ujMdZwITVAJHFtfyN1qUhDqEiIlk= +google.golang.org/genproto v0.0.0-20220829144015-23454907ede3/go.mod h1:dbqgFATTzChvnt+ujMdZwITVAJHFtfyN1qUhDqEiIlk= +google.golang.org/genproto v0.0.0-20220829175752-36a9c930ecbf/go.mod h1:dbqgFATTzChvnt+ujMdZwITVAJHFtfyN1qUhDqEiIlk= +google.golang.org/genproto v0.0.0-20220913154956-18f8339a66a5/go.mod h1:0Nb8Qy+Sk5eDzHnzlStwW3itdNaWoZA5XeSG+R3JHSo= +google.golang.org/genproto v0.0.0-20220914142337-ca0e39ece12f/go.mod h1:0Nb8Qy+Sk5eDzHnzlStwW3itdNaWoZA5XeSG+R3JHSo= +google.golang.org/genproto v0.0.0-20220915135415-7fd63a7952de/go.mod h1:0Nb8Qy+Sk5eDzHnzlStwW3itdNaWoZA5XeSG+R3JHSo= +google.golang.org/genproto v0.0.0-20220916172020-2692e8806bfa/go.mod h1:0Nb8Qy+Sk5eDzHnzlStwW3itdNaWoZA5XeSG+R3JHSo= +google.golang.org/genproto v0.0.0-20220919141832-68c03719ef51/go.mod h1:0Nb8Qy+Sk5eDzHnzlStwW3itdNaWoZA5XeSG+R3JHSo= +google.golang.org/genproto v0.0.0-20220920201722-2b89144ce006/go.mod h1:ht8XFiar2npT/g4vkk7O0WYS1sHOHbdujxbEp7CJWbw= +google.golang.org/genproto v0.0.0-20220926165614-551eb538f295/go.mod h1:woMGP53BroOrRY3xTxlbr8Y3eB/nzAvvFM83q7kG2OI= +google.golang.org/genproto v0.0.0-20220926220553-6981cbe3cfce/go.mod h1:woMGP53BroOrRY3xTxlbr8Y3eB/nzAvvFM83q7kG2OI= +google.golang.org/genproto v0.0.0-20221010155953-15ba04fc1c0e/go.mod h1:3526vdqwhZAwq4wsRUaVG555sVgsNmIjRtO7t/JH29U= +google.golang.org/genproto v0.0.0-20221014173430-6e2ab493f96b/go.mod h1:1vXfmgAz9N9Jx0QA82PqRVauvCz1SGSz739p0f183jM= +google.golang.org/genproto v0.0.0-20221014213838-99cd37c6964a/go.mod h1:1vXfmgAz9N9Jx0QA82PqRVauvCz1SGSz739p0f183jM= +google.golang.org/genproto v0.0.0-20221024153911-1573dae28c9c/go.mod h1:9qHF0xnpdSfF6knlcsnpzUu5y+rpwgbvsyGAZPBMg4s= +google.golang.org/genproto v0.0.0-20221024183307-1bc688fe9f3e/go.mod h1:9qHF0xnpdSfF6knlcsnpzUu5y+rpwgbvsyGAZPBMg4s= +google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c/go.mod h1:CGI5F/G+E5bKwmfYo09AXuVN4dD894kIKUFmVbP2/Fo= +google.golang.org/genproto v0.0.0-20221109142239-94d6d90a7d66/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= +google.golang.org/genproto v0.0.0-20221114212237-e4508ebdbee1/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= +google.golang.org/genproto v0.0.0-20221117204609-8f9c96812029/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= +google.golang.org/genproto v0.0.0-20221118155620-16455021b5e6/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= +google.golang.org/genproto v0.0.0-20221201164419-0e50fba7f41c/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= +google.golang.org/genproto v0.0.0-20221201204527-e3fa12d562f3/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= +google.golang.org/genproto v0.0.0-20221202195650-67e5cbc046fd/go.mod h1:cTsE614GARnxrLsqKREzmNYJACSWWpAWdNMwnD7c2BE= +google.golang.org/genproto v0.0.0-20221227171554-f9683d7f8bef/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230110181048-76db0878b65f/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230112194545-e10362b5ecf9/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230113154510-dbe35b8444a5/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230123190316-2c411cf9d197/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230124163310-31e0e69b6fc2/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230125152338-dcaf20b6aeaa/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230127162408-596548ed4efa/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230209215440-0dfe4f8abfcc/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM= +google.golang.org/genproto v0.0.0-20230216225411-c8e22ba71e44/go.mod h1:8B0gmkoRebU8ukX6HP+4wrVQUY1+6PkQ44BSyIlflHA= +google.golang.org/genproto v0.0.0-20230222225845-10f96fb3dbec/go.mod h1:3Dl5ZL0q0isWJt+FVcfpQyirqemEuLAK/iFvg1UP1Hw= +google.golang.org/genproto v0.0.0-20230223222841-637eb2293923/go.mod h1:3Dl5ZL0q0isWJt+FVcfpQyirqemEuLAK/iFvg1UP1Hw= +google.golang.org/genproto v0.0.0-20230303212802-e74f57abe488/go.mod h1:TvhZT5f700eVlTNwND1xoEZQeWTB2RY/65kplwl/bFA= +google.golang.org/genproto v0.0.0-20230306155012-7f2fa6fef1f4/go.mod h1:NWraEVixdDnqcqQ30jipen1STv2r/n24Wb7twVTGR4s= +google.golang.org/genproto v0.0.0-20230320184635-7606e756e683/go.mod h1:NWraEVixdDnqcqQ30jipen1STv2r/n24Wb7twVTGR4s= +google.golang.org/genproto v0.0.0-20230323212658-478b75c54725/go.mod h1:UUQDJDOlWu4KYeJZffbWgBkS1YFobzKbLVfK69pe0Ak= +google.golang.org/genproto v0.0.0-20230330154414-c0448cd141ea/go.mod h1:UUQDJDOlWu4KYeJZffbWgBkS1YFobzKbLVfK69pe0Ak= +google.golang.org/genproto v0.0.0-20230331144136-dcfb400f0633/go.mod h1:UUQDJDOlWu4KYeJZffbWgBkS1YFobzKbLVfK69pe0Ak= +google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1/go.mod h1:nKE/iIaLqn2bQwXBg8f1g2Ylh6r5MN5CmZvuzZCgsCU= +google.golang.org/genproto v0.0.0-20230525234025-438c736192d0/go.mod h1:9ExIQyXL5hZrHzQceCwuSYwZZ5QZBazOcprJ5rgs3lY= +google.golang.org/genproto v0.0.0-20230526161137-0005af68ea54/go.mod h1:zqTuNwFlFRsw5zIts5VnzLQxSRqh+CGOTVMlYbY0Eyk= +google.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4= +google.golang.org/genproto v0.0.0-20250603155806-513f23925822/go.mod h1:HubltRL7rMh0LfnQPkMH4NPDFEWp0jw3vixw7jEM53s= +google.golang.org/genproto/googleapis/api v0.0.0-20230525234020-1aefcd67740a/go.mod h1:ts19tUU+Z0ZShN1y3aPyq2+O3d5FUNNgT6FtOzmrNn8= +google.golang.org/genproto/googleapis/api v0.0.0-20230525234035-dd9d682886f9/go.mod h1:vHYtlOoi6TsQ3Uk2yxR7NI5z8uoV+3pZtR4jmHIkRig= +google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 h1:FiusG7LWj+4byqhbvmB+Q93B/mOxJLN2DTozDuZm4EU= +google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7/go.mod h1:kXqgZtrWaf6qS3jZOCnCH7WYfrvFjkC51bM8fz3RsCA= +google.golang.org/genproto/googleapis/rpc v0.0.0-20230525234015-3fc162c6f38a/go.mod h1:xURIpW9ES5+/GZhnV6beoEtxQrnkRGIfP5VQG2tCBLc= +google.golang.org/genproto/googleapis/rpc v0.0.0-20230525234030-28d5490b6b19/go.mod h1:66JfowdXAEgad5O9NnYcsNPLCPZJD++2L9X0PCMODrA= +google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c h1:qXWI/sQtv5UKboZ/zUk7h+mrf/lXORyI+n9DKDAusdg= +google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c/go.mod h1:gw1tLEfykwDz2ET4a12jcXt4couGAm7IwsVaTy0Sflo= +google.golang.org/grpc v1.57.1 h1:upNTNqv0ES+2ZOOqACwVtS3Il8M12/+Hz41RCPzAjQg= +google.golang.org/grpc v1.57.1/go.mod h1:Sd+9RMTACXwmub0zcNY2c4arhtrbBYD1AUHI/dt16Mo= +google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw= +google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= +google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= +google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= +google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= +google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= +google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= +google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= +google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= +google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= +google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= +google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +google.golang.org/protobuf v1.29.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +google.golang.org/protobuf v1.29.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= +google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw= +google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= +google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20200902074654-038fdea0a05b/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw= +gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= +gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= +gopkg.in/gcfg.v1 v1.2.3/go.mod h1:yesOnuUOFQAhST5vPY4nbZsb/huCgGGXlipJsBn0b3o= +gopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= +gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= +gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc= +gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc= +gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= +gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= +gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= +gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI= +gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74= +gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= +gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= +honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= +honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= +honnef.co/go/tools v0.1.3/go.mod h1:NgwopIslSNH47DimFoV78dnkksY2EFtX0ajyb3K/las= +k8s.io/component-base v0.35.0 h1:+yBrOhzri2S1BVqyVSvcM3PtPyx5GUxCK2tinZz1G94= +k8s.io/component-base v0.35.0/go.mod h1:85SCX4UCa6SCFt6p3IKAPej7jSnF3L8EbfSyMZayJR0= +lukechampine.com/uint128 v1.1.1/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk= +lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI= +lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk= +modernc.org/cc/v3 v3.36.0/go.mod h1:NFUHyPn4ekoC/JHeZFfZurN6ixxawE1BnVonP/oahEI= +modernc.org/cc/v3 v3.36.2/go.mod h1:NFUHyPn4ekoC/JHeZFfZurN6ixxawE1BnVonP/oahEI= +modernc.org/cc/v3 v3.36.3 h1:uISP3F66UlixxWEcKuIWERa4TwrZENHSL8tWxZz8bHg= +modernc.org/cc/v3 v3.36.3/go.mod h1:NFUHyPn4ekoC/JHeZFfZurN6ixxawE1BnVonP/oahEI= +modernc.org/ccgo/v3 v3.0.0-20220428102840-41399a37e894/go.mod h1:eI31LL8EwEBKPpNpA4bU1/i+sKOwOrQy8D87zWUcRZc= +modernc.org/ccgo/v3 v3.0.0-20220430103911-bc99d88307be/go.mod h1:bwdAnOoaIt8Ax9YdWGjxWsdkPcZyRPHqrOvJxaKAKGw= +modernc.org/ccgo/v3 v3.16.4/go.mod h1:tGtX0gE9Jn7hdZFeU88slbTh1UtCYKusWOoCJuvkWsQ= +modernc.org/ccgo/v3 v3.16.6/go.mod h1:tGtX0gE9Jn7hdZFeU88slbTh1UtCYKusWOoCJuvkWsQ= +modernc.org/ccgo/v3 v3.16.8/go.mod h1:zNjwkizS+fIFDrDjIAgBSCLkWbJuHF+ar3QRn+Z9aws= +modernc.org/ccgo/v3 v3.16.9 h1:AXquSwg7GuMk11pIdw7fmO1Y/ybgazVkMhsZWCV0mHM= +modernc.org/ccgo/v3 v3.16.9/go.mod h1:zNMzC9A9xeNUepy6KuZBbugn3c0Mc9TeiJO4lgvkJDo= +modernc.org/ccorpus v1.11.6 h1:J16RXiiqiCgua6+ZvQot4yUuUy8zxgqbqEEUuGPlISk= +modernc.org/ccorpus v1.11.6/go.mod h1:2gEUTrWqdpH2pXsmTM1ZkjeSrUWDpjMu2T6m29L/ErQ= +modernc.org/httpfs v1.0.6 h1:AAgIpFZRXuYnkjftxTAZwMIiwEqAfk8aVB2/oA6nAeM= +modernc.org/httpfs v1.0.6/go.mod h1:7dosgurJGp0sPaRanU53W4xZYKh14wfzX420oZADeHM= +modernc.org/libc v0.0.0-20220428101251-2d5f3daf273b/go.mod h1:p7Mg4+koNjc8jkqwcoFBJx7tXkpj00G77X7A72jXPXA= +modernc.org/libc v1.16.0/go.mod h1:N4LD6DBE9cf+Dzf9buBlzVJndKr/iJHG97vGLHYnb5A= +modernc.org/libc v1.16.1/go.mod h1:JjJE0eu4yeK7tab2n4S1w8tlWd9MxXLRzheaRnAKymU= +modernc.org/libc v1.16.17/go.mod h1:hYIV5VZczAmGZAnG15Vdngn5HSF5cSkbvfz2B7GRuVU= +modernc.org/libc v1.16.19/go.mod h1:p7Mg4+koNjc8jkqwcoFBJx7tXkpj00G77X7A72jXPXA= +modernc.org/libc v1.17.0/go.mod h1:XsgLldpP4aWlPlsjqKRdHPqCxCjISdHfM/yeWC5GyW0= +modernc.org/libc v1.17.1 h1:Q8/Cpi36V/QBfuQaFVeisEBs3WqoGAJprZzmf7TfEYI= +modernc.org/libc v1.17.1/go.mod h1:FZ23b+8LjxZs7XtFMbSzL/EhPxNbfZbErxEHc7cbD9s= +modernc.org/mathutil v1.2.2/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E= +modernc.org/mathutil v1.4.1/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E= +modernc.org/mathutil v1.5.0 h1:rV0Ko/6SfM+8G+yKiyI830l3Wuz1zRutdslNoQ0kfiQ= +modernc.org/mathutil v1.5.0/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E= +modernc.org/memory v1.1.1/go.mod h1:/0wo5ibyrQiaoUoH7f9D8dnglAmILJ5/cxZlRECf+Nw= +modernc.org/memory v1.2.0/go.mod h1:/0wo5ibyrQiaoUoH7f9D8dnglAmILJ5/cxZlRECf+Nw= +modernc.org/memory v1.2.1 h1:dkRh86wgmq/bJu2cAS2oqBCz/KsMZU7TUM4CibQ7eBs= +modernc.org/memory v1.2.1/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU= +modernc.org/opt v0.1.1/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0= +modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4= +modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0= +modernc.org/sqlite v1.18.1 h1:ko32eKt3jf7eqIkCgPAeHMBXw3riNSLhl2f3loEF7o8= +modernc.org/sqlite v1.18.1/go.mod h1:6ho+Gow7oX5V+OiOQ6Tr4xeqbx13UZ6t+Fw9IRUG4d4= +modernc.org/strutil v1.1.1/go.mod h1:DE+MQQ/hjKBZS2zNInV5hhcipt5rLPWkmpbGeW5mmdw= +modernc.org/strutil v1.1.3 h1:fNMm+oJklMGYfU9Ylcywl0CO5O6nTfaowNsh2wpPjzY= +modernc.org/strutil v1.1.3/go.mod h1:MEHNA7PdEnEwLvspRMtWTNnp2nnyvMfkimT1NKNAGbw= +modernc.org/tcl v1.13.1 h1:npxzTwFTZYM8ghWicVIX1cRWzj7Nd8i6AqqX2p+IYao= +modernc.org/tcl v1.13.1/go.mod h1:XOLfOwzhkljL4itZkK6T72ckMgvj0BDsnKNdZVUOecw= +modernc.org/token v1.0.0 h1:a0jaWiNMDhDUtqOj09wvjWWAqd3q7WpBulmL9H2egsk= +modernc.org/token v1.0.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= +modernc.org/z v1.5.1 h1:RTNHdsrOpeoSeOF4FbzTo8gBYByaJ5xT7NgZ9ZqRiJM= +modernc.org/z v1.5.1/go.mod h1:eWFB510QWW5Th9YGZT81s+LwvaAs3Q2yr4sP0rmLkv8= +moul.io/http2curl/v2 v2.3.0/go.mod h1:RW4hyBjTWSYDOxapodpNEtX0g5Eb16sxklBqmd2RHcE= +nhooyr.io/websocket v1.8.6 h1:s+C3xAMLwGmlI31Nyn/eAehUlZPwfYZu2JXM621Q5/k= +nhooyr.io/websocket v1.8.6/go.mod h1:B70DZP8IakI65RVQ51MsWP/8jndNma26DVA/nFSCgW0= +pgregory.net/rapid v1.2.0 h1:keKAYRcjm+e1F0oAuU5F5+YPAWcyxNNRK2wud503Gnk= +pgregory.net/rapid v1.2.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04= +rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= +rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4= +rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= +rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= +rsc.io/tmplfunc v0.0.3/go.mod h1:AG3sTPzElb1Io3Yg4voV9AGZJuleGAwaVRxL9M49PhA= +sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o= +sourcegraph.com/sourcegraph/appdash v0.0.0-20190731080439-ebfcffb1b5c0/go.mod h1:hI742Nqp5OhwiqlzhgfbWU4mW4yO10fP+LoT9WOswdU= diff --git a/sidecar/main.go b/sidecar/main.go new file mode 100644 index 00000000..84a8e640 --- /dev/null +++ b/sidecar/main.go @@ -0,0 +1,58 @@ +// Command sei-sidecar is the per-node sidecar: a task executor and HTTP API +// that the controller drives over kube-rbac-proxy. It runs as a native +// restartable sidecar container beside seid in every SeiNode pod. +// +// It ships as `serve` under a root command rather than as a bare binary because +// the controller renders `Command: []string{"seictl", "serve"}` into every pod +// spec. The image keeps a `seictl` symlink for that reason; see Dockerfile. +package main + +import ( + "context" + "fmt" + "os" + + "github.com/sei-protocol/seilog" + "github.com/urfave/cli/v3" +) + +// destinations holds flag-bound values shared with the subcommands, mirroring +// the shape the seictl CLI used so serve.go's call sites are unchanged. +// +// home is the one that matters. It binds SEI_HOME, which the controller sets to +// the node's data-PVC mount. In seictl this flag lived on the root command and +// serve.go read it from here; the wiring is reproduced deliberately, because +// losing it does not fail — it silently relocates every write off the PVC. +var destinations = struct { + home string +}{} + +func main() { + cmd := &cli.Command{ + Name: "sei-sidecar", + Usage: "Sei node sidecar: task executor and HTTP API", + Flags: []cli.Flag{ + &cli.StringFlag{ + Name: "home", + Sources: cli.EnvVars("SEI_HOME"), + // No Value: an unset home must fail, not default. The previous + // fallback was "/sei" while the controller mounts the data PVC + // at $HOME/.sei, so a dropped SEI_HOME produced a running, + // probe-passing sidecar writing genesis and config into an + // empty directory. Required turns that into a startup error. + Required: true, + Destination: &destinations.home, + TakesFile: true, + Config: cli.StringConfig{TrimSpace: true}, + Usage: "seid home directory (the node's data volume)", + }, + }, + Commands: []*cli.Command{&serveCmd}, + } + + if err := cmd.Run(context.Background(), os.Args); err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + _ = seilog.Close() + os.Exit(1) + } +} diff --git a/sidecar/rpc/client.go b/sidecar/rpc/client.go new file mode 100644 index 00000000..60a5ffae --- /dev/null +++ b/sidecar/rpc/client.go @@ -0,0 +1,131 @@ +package rpc + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "time" +) + +const defaultTimeout = 10 * time.Second + +// HTTPDoer abstracts HTTP requests for testability. +type HTTPDoer interface { + Do(req *http.Request) (*http.Response, error) +} + +// rpcError represents a JSON-RPC error returned by CometBFT. +type rpcError struct { + Code int `json:"code"` + Message string `json:"message"` + Data string `json:"data"` +} + +// envelope is the JSON-RPC response wrapper returned by standard CometBFT +// HTTP RPC endpoints. Note: seid's CometBFT fork returns flat JSON without +// this wrapper — Client.Get handles both formats. +type envelope struct { + JSONRPC string `json:"jsonrpc"` + Result json.RawMessage `json:"result"` + Error *rpcError `json:"error,omitempty"` +} + +// Client performs HTTP GET requests against a CometBFT RPC endpoint +// and handles the JSON-RPC envelope unwrapping. +type Client struct { + endpoint string + httpClient HTTPDoer + timeout time.Duration +} + +// NewClient creates a CometBFT RPC client. Pass "" for endpoint to +// use the default localhost address. Pass nil for httpClient to use +// http.DefaultClient with no custom timeout. +func NewClient(endpoint string, httpClient HTTPDoer) *Client { + if endpoint == "" { + endpoint = DefaultEndpoint + } + if httpClient == nil { + httpClient = &http.Client{} + } + return &Client{ + endpoint: endpoint, + httpClient: httpClient, + timeout: defaultTimeout, + } +} + +// SetTimeout overrides the per-request context timeout. +func (c *Client) SetTimeout(d time.Duration) { c.timeout = d } + +// Endpoint returns the configured RPC base URL. +func (c *Client) Endpoint() string { return c.endpoint } + +// Get performs an HTTP GET to endpoint+path and returns the inner result +// as raw JSON. It handles both response formats: +// - JSON-RPC envelope (standard CometBFT): {"jsonrpc":"2.0","result":{...}} +// → returns the unwrapped "result" value +// - Flat JSON (seid): {"node_info":{...},"sync_info":{...}} +// → returns the body as-is +// +// This dual-format support is necessary because seid's CometBFT fork +// returns flat responses while standard CometBFT uses JSON-RPC envelopes. +func (c *Client) Get(ctx context.Context, path string) (json.RawMessage, error) { + body, err := c.doGet(ctx, path) + if err != nil { + return nil, err + } + + var env envelope + if err := json.Unmarshal(body, &env); err != nil { + return nil, fmt.Errorf("decoding JSON response from %s: %w", path, err) + } + + // Discriminate by the presence of "jsonrpc":"2.0" — only real JSON-RPC + // envelopes carry this field. Seid's flat responses never will. + if env.JSONRPC == "2.0" { + if env.Error != nil { + return nil, fmt.Errorf("JSON-RPC error from %s: %s (code %d, data: %s)", + path, env.Error.Message, env.Error.Code, env.Error.Data) + } + if len(env.Result) == 0 { + return nil, fmt.Errorf("empty result in JSON-RPC response from %s", path) + } + return env.Result, nil + } + + // Flat JSON (seid format) — return the body as-is. + return json.RawMessage(body), nil +} + +// GetRaw performs an HTTP GET and returns the entire response body +// without envelope unwrapping. Use for archival paths that store the +// verbatim JSON-RPC response (e.g., S3 export). +func (c *Client) GetRaw(ctx context.Context, path string) ([]byte, error) { + return c.doGet(ctx, path) +} + +func (c *Client) doGet(ctx context.Context, path string) ([]byte, error) { + ctx, cancel := context.WithTimeout(ctx, c.timeout) + defer cancel() + + req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.endpoint+path, nil) + if err != nil { + return nil, err + } + + resp, err := c.httpClient.Do(req) + if err != nil { + return nil, err + } + defer func() { _ = resp.Body.Close() }() + + if resp.StatusCode != http.StatusOK { + body, _ := io.ReadAll(io.LimitReader(resp.Body, 512)) + return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, body) + } + + return io.ReadAll(resp.Body) +} diff --git a/sidecar/rpc/client_test.go b/sidecar/rpc/client_test.go new file mode 100644 index 00000000..21256a7f --- /dev/null +++ b/sidecar/rpc/client_test.go @@ -0,0 +1,160 @@ +package rpc + +import ( + "context" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" +) + +func TestClient_Get_UnwrapsEnvelope(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`{"jsonrpc":"2.0","id":-1,"result":{"sync_info":{"latest_block_height":"42"}}}`)) + })) + defer srv.Close() + + c := NewClient(srv.URL, nil) + raw, err := c.Get(context.Background(), "/status") + if err != nil { + t.Fatalf("Get: %v", err) + } + + got := string(raw) + if !strings.Contains(got, "latest_block_height") { + t.Errorf("expected unwrapped result containing latest_block_height, got %s", got) + } + if strings.Contains(got, "jsonrpc") { + t.Error("result should not contain the JSON-RPC envelope") + } +} + +func TestClient_Get_FlatJSON_SeidFormat(t *testing.T) { + // seid returns flat JSON without the JSON-RPC envelope. + flat := `{"node_info":{"id":"abc123"},"sync_info":{"latest_block_height":"42","catching_up":false}}` + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(flat)) + })) + defer srv.Close() + + c := NewClient(srv.URL, nil) + raw, err := c.Get(context.Background(), "/status") + if err != nil { + t.Fatalf("Get: %v", err) + } + + got := string(raw) + if !strings.Contains(got, "abc123") { + t.Errorf("expected flat result containing node ID, got %s", got) + } + if !strings.Contains(got, "latest_block_height") { + t.Errorf("expected flat result containing latest_block_height, got %s", got) + } +} + +func TestClient_Get_FlatJSON_WithResultKey(t *testing.T) { + // A flat response that happens to contain a "result" data key must NOT + // be mistaken for a JSON-RPC envelope. + flat := `{"result":{"code":0,"log":"ok"},"hash":"ABC123"}` + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(flat)) + })) + defer srv.Close() + + c := NewClient(srv.URL, nil) + raw, err := c.Get(context.Background(), "/tx") + if err != nil { + t.Fatalf("Get: %v", err) + } + + got := string(raw) + // Should return the full body, not just the inner "result" value. + if !strings.Contains(got, "hash") { + t.Errorf("expected full flat body with hash field, got %s", got) + } + if !strings.Contains(got, "ABC123") { + t.Errorf("expected full flat body with hash value, got %s", got) + } +} + +func TestClient_Get_HTTPError(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusBadGateway) + _, _ = w.Write([]byte("bad gateway")) + })) + defer srv.Close() + + c := NewClient(srv.URL, nil) + _, err := c.Get(context.Background(), "/status") + if err == nil { + t.Fatal("expected error for non-200 response") + } + if !strings.Contains(err.Error(), "502") { + t.Errorf("expected HTTP 502 in error, got: %v", err) + } +} + +func TestClient_Get_MalformedJSON(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`not json`)) + })) + defer srv.Close() + + c := NewClient(srv.URL, nil) + _, err := c.Get(context.Background(), "/status") + if err == nil { + t.Fatal("expected error for malformed JSON") + } +} + +func TestClient_GetRaw_ReturnsFullBody(t *testing.T) { + body := `{"jsonrpc":"2.0","id":-1,"result":{"txs_results":[]}}` + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(body)) + })) + defer srv.Close() + + c := NewClient(srv.URL, nil) + raw, err := c.GetRaw(context.Background(), "/block_results?height=1") + if err != nil { + t.Fatalf("GetRaw: %v", err) + } + if string(raw) != body { + t.Errorf("expected full body %q, got %q", body, string(raw)) + } +} + +func TestClient_Get_RPCError(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`{"jsonrpc":"2.0","id":-1,"error":{"code":-32603,"message":"Internal error","data":"height 999999 is not available"}}`)) + })) + defer srv.Close() + + c := NewClient(srv.URL, nil) + _, err := c.Get(context.Background(), "/block?height=999999") + if err == nil { + t.Fatal("expected error for JSON-RPC error response") + } + if !strings.Contains(err.Error(), "Internal error") { + t.Errorf("expected error to contain CometBFT message, got: %v", err) + } + if !strings.Contains(err.Error(), "height 999999 is not available") { + t.Errorf("expected error to contain data field, got: %v", err) + } +} + +func TestClient_SetTimeout(t *testing.T) { + c := NewClient("", nil) + c.SetTimeout(500 * time.Millisecond) + if c.timeout != 500*time.Millisecond { + t.Errorf("timeout = %v, want 500ms", c.timeout) + } +} + +func TestClient_DefaultEndpoint(t *testing.T) { + c := NewClient("", nil) + if c.Endpoint() != DefaultEndpoint { + t.Errorf("endpoint = %q, want %q", c.Endpoint(), DefaultEndpoint) + } +} diff --git a/sidecar/rpc/status.go b/sidecar/rpc/status.go new file mode 100644 index 00000000..f35273ea --- /dev/null +++ b/sidecar/rpc/status.go @@ -0,0 +1,72 @@ +package rpc + +import ( + "context" + "encoding/json" + "fmt" + "strconv" + "time" + + seiconfig "github.com/sei-protocol/sei-config" +) + +const statusTimeout = 500 * time.Millisecond + +// DefaultEndpoint is the local CometBFT RPC address. +var DefaultEndpoint = fmt.Sprintf("http://localhost:%d", seiconfig.PortRPC) + +// NodeStatus holds the fields we care about from CometBFT /status. +type NodeStatus struct { + LatestBlockHeight int64 + CatchingUp bool +} + +// StatusClient queries a CometBFT node's /status endpoint. +type StatusClient struct { + client *Client +} + +// NewStatusClient creates a client targeting the given RPC endpoint. +// Pass "" for the default localhost endpoint. Pass nil for the default +// HTTP client. +func NewStatusClient(endpoint string, httpClient HTTPDoer) *StatusClient { + c := NewClient(endpoint, httpClient) + c.SetTimeout(statusTimeout) + return &StatusClient{client: c} +} + +// Endpoint returns the configured RPC endpoint. +func (c *StatusClient) Endpoint() string { return c.client.Endpoint() } + +// Status queries the node and returns the parsed status. +func (c *StatusClient) Status(ctx context.Context) (*NodeStatus, error) { + raw, err := c.client.Get(ctx, "/status") + if err != nil { + return nil, err + } + + var result StatusResult + if err := json.Unmarshal(raw, &result); err != nil { + return nil, fmt.Errorf("decoding /status result: %w", err) + } + + h, err := strconv.ParseInt(result.SyncInfo.LatestBlockHeight, 10, 64) + if err != nil { + return nil, fmt.Errorf("parsing latest_block_height %q: %w", + result.SyncInfo.LatestBlockHeight, err) + } + + return &NodeStatus{ + LatestBlockHeight: h, + CatchingUp: result.SyncInfo.CatchingUp, + }, nil +} + +// LatestHeight is a convenience wrapper returning just the height. +func (c *StatusClient) LatestHeight(ctx context.Context) (int64, error) { + s, err := c.Status(ctx) + if err != nil { + return 0, err + } + return s.LatestBlockHeight, nil +} diff --git a/sidecar/rpc/status_test.go b/sidecar/rpc/status_test.go new file mode 100644 index 00000000..3ba61299 --- /dev/null +++ b/sidecar/rpc/status_test.go @@ -0,0 +1,125 @@ +package rpc + +import ( + "context" + "fmt" + "net/http" + "net/http/httptest" + "testing" +) + +// wrapResult wraps a JSON payload in the CometBFT JSON-RPC envelope. +func wrapResult(inner string) string { + return fmt.Sprintf(`{"jsonrpc":"2.0","id":-1,"result":%s}`, inner) +} + +func TestStatusClient_LatestHeight(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(wrapResult(`{"sync_info":{"latest_block_height":"12345","catching_up":false}}`))) + })) + defer srv.Close() + + c := NewStatusClient(srv.URL, nil) + h, err := c.LatestHeight(context.Background()) + if err != nil { + t.Fatalf("LatestHeight: %v", err) + } + if h != 12345 { + t.Errorf("height = %d, want 12345", h) + } +} + +func TestStatusClient_CatchingUp(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(wrapResult(`{"sync_info":{"latest_block_height":"100","catching_up":true}}`))) + })) + defer srv.Close() + + c := NewStatusClient(srv.URL, nil) + s, err := c.Status(context.Background()) + if err != nil { + t.Fatalf("Status: %v", err) + } + if !s.CatchingUp { + t.Error("expected CatchingUp=true") + } +} + +func TestStatusClient_HTTPError(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusInternalServerError) + _, _ = w.Write([]byte("internal error")) + })) + defer srv.Close() + + c := NewStatusClient(srv.URL, nil) + _, err := c.LatestHeight(context.Background()) + if err == nil { + t.Fatal("expected error for 500 response") + } +} + +func TestStatusClient_ConnectionRefused(t *testing.T) { + c := NewStatusClient("http://127.0.0.1:1", nil) + _, err := c.LatestHeight(context.Background()) + if err == nil { + t.Fatal("expected error for refused connection") + } +} + +func TestStatusClient_MalformedJSON(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`not json`)) + })) + defer srv.Close() + + c := NewStatusClient(srv.URL, nil) + _, err := c.LatestHeight(context.Background()) + if err == nil { + t.Fatal("expected error for malformed JSON") + } +} + +func TestStatusClient_InvalidBlockHeight(t *testing.T) { + tests := []struct { + name string + payload string + }{ + {"empty height", wrapResult(`{"sync_info":{"latest_block_height":"","catching_up":false}}`)}, + {"non-numeric height", wrapResult(`{"sync_info":{"latest_block_height":"abc","catching_up":false}}`)}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(tt.payload)) + })) + defer srv.Close() + + c := NewStatusClient(srv.URL, nil) + _, err := c.LatestHeight(context.Background()) + if err == nil { + t.Fatal("expected error for invalid block height") + } + }) + } +} + +func TestStatusClient_DefaultEndpoint(t *testing.T) { + c := NewStatusClient("", nil) + if c.Endpoint() != DefaultEndpoint { + t.Errorf("endpoint = %q, want %q", c.Endpoint(), DefaultEndpoint) + } +} + +func TestStatusClient_EmptyResult(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`{"jsonrpc":"2.0","id":-1}`)) + })) + defer srv.Close() + + c := NewStatusClient(srv.URL, nil) + _, err := c.LatestHeight(context.Background()) + if err == nil { + t.Fatal("expected error for empty result") + } +} diff --git a/sidecar/rpc/types.go b/sidecar/rpc/types.go new file mode 100644 index 00000000..f1605272 --- /dev/null +++ b/sidecar/rpc/types.go @@ -0,0 +1,61 @@ +package rpc + +import "encoding/json" + +// StatusResult is the inner "result" of the CometBFT /status response, +// after the JSON-RPC envelope has been stripped by Client.Get. +type StatusResult struct { + NodeInfo NodeInfo `json:"node_info"` + SyncInfo SyncInfo `json:"sync_info"` +} + +// NodeInfo identifies a CometBFT node. +type NodeInfo struct { + ID string `json:"id"` + // Network is the chain ID the node is configured for; CometBFT's + // protocol term. Compared against task params.chainId for the + // chain-confusion guard in sidecar/tasks/sign_and_broadcast.go. + Network string `json:"network"` +} + +// SyncInfo reports chain sync state. +type SyncInfo struct { + LatestBlockHeight string `json:"latest_block_height"` + CatchingUp bool `json:"catching_up"` +} + +// BlockResult is the inner "result" of the CometBFT /block response. +type BlockResult struct { + BlockID BlockID `json:"block_id"` + Block Block `json:"block"` +} + +// BlockID identifies a block by hash. +type BlockID struct { + Hash string `json:"hash"` +} + +// Block holds the subset of block fields we need. +type Block struct { + Header BlockHeader `json:"header"` +} + +// BlockHeader holds consensus-critical header fields for comparison. +type BlockHeader struct { + AppHash string `json:"app_hash"` + LastResultsHash string `json:"last_results_hash"` +} + +// BlockResultsResult is the inner "result" of the CometBFT /block_results response. +type BlockResultsResult struct { + TxsResults []TxResult `json:"txs_results"` +} + +// TxResult holds a single transaction execution result. +type TxResult struct { + Code int `json:"code"` + Log string `json:"log"` + GasUsed string `json:"gas_used"` + GasWanted string `json:"gas_wanted"` + Events json.RawMessage `json:"events"` +} diff --git a/sidecar/s3/client.go b/sidecar/s3/client.go new file mode 100644 index 00000000..95507d07 --- /dev/null +++ b/sidecar/s3/client.go @@ -0,0 +1,82 @@ +package s3 + +import ( + "context" + "fmt" + + "github.com/aws/aws-sdk-go-v2/config" + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + "github.com/aws/aws-sdk-go-v2/service/s3" +) + +// TransferClient abstracts S3 downloads. DownloadObject uses the transfer +// manager's io.WriterAt path for parallel byte-range downloads. +type TransferClient interface { + DownloadObject(ctx context.Context, input *transfermanager.DownloadObjectInput, opts ...func(*transfermanager.Options)) (*transfermanager.DownloadObjectOutput, error) +} + +// TransferClientFactory builds a TransferClient for a given region. +type TransferClientFactory func(ctx context.Context, region string) (TransferClient, error) + +// DefaultTransferClientFactory creates a transfer manager backed by a real +// S3 service client. +func DefaultTransferClientFactory(ctx context.Context, region string) (TransferClient, error) { + cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion(region)) + if err != nil { + return nil, fmt.Errorf("loading AWS config: %w", err) + } + return transfermanager.New(s3.NewFromConfig(cfg)), nil +} + +// Uploader abstracts the transfermanager upload call for testing. +type Uploader interface { + UploadObject(ctx context.Context, input *transfermanager.UploadObjectInput, opts ...func(*transfermanager.Options)) (*transfermanager.UploadObjectOutput, error) +} + +// UploaderFactory builds an Uploader for a given region. +type UploaderFactory func(ctx context.Context, region string) (Uploader, error) + +// DefaultUploaderFactory creates a transfermanager.Client backed by a real S3 client. +func DefaultUploaderFactory(ctx context.Context, region string) (Uploader, error) { + cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion(region)) + if err != nil { + return nil, fmt.Errorf("loading AWS config: %w", err) + } + return transfermanager.New(s3.NewFromConfig(cfg)), nil +} + +// ObjectLister abstracts S3 ListObjectsV2 for snapshot discovery. +type ObjectLister interface { + ListObjectsV2(ctx context.Context, input *s3.ListObjectsV2Input, opts ...func(*s3.Options)) (*s3.ListObjectsV2Output, error) +} + +// ObjectListerFactory builds an ObjectLister for a given region. +type ObjectListerFactory func(ctx context.Context, region string) (ObjectLister, error) + +// DefaultObjectListerFactory creates a real S3 client for listing objects. +func DefaultObjectListerFactory(ctx context.Context, region string) (ObjectLister, error) { + cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion(region)) + if err != nil { + return nil, fmt.Errorf("loading AWS config: %w", err) + } + return s3.NewFromConfig(cfg), nil +} + +// Downloader abstracts S3 GetObject for streaming reads. Unlike +// TransferClient (which writes to io.WriterAt), Downloader returns +// a streaming io.ReadCloser body suitable for gzip decompression. +type Downloader interface { + GetObject(ctx context.Context, input *s3.GetObjectInput, opts ...func(*s3.Options)) (*s3.GetObjectOutput, error) +} + +// DownloaderFactory builds a Downloader for a given region. +type DownloaderFactory func(ctx context.Context, region string) (Downloader, error) + +// DefaultDownloaderFactory creates a real S3 client for streaming downloads. +func DefaultDownloaderFactory(ctx context.Context, region string) (Downloader, error) { + cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion(region)) + if err != nil { + return nil, fmt.Errorf("loading AWS config: %w", err) + } + return s3.NewFromConfig(cfg), nil +} diff --git a/sidecar/s3/client_test.go b/sidecar/s3/client_test.go new file mode 100644 index 00000000..1a4c2baf --- /dev/null +++ b/sidecar/s3/client_test.go @@ -0,0 +1,146 @@ +package s3 + +import ( + "sync" + "testing" +) + +// WriteAtBuffer is a goroutine-safe in-memory io.WriterAt, used for +// downloading small S3 objects (e.g. latest.txt) via DownloadObject. +type WriteAtBuffer struct { + mu sync.Mutex + buf []byte +} + +func (w *WriteAtBuffer) WriteAt(p []byte, off int64) (int, error) { + w.mu.Lock() + defer w.mu.Unlock() + end := int(off) + len(p) + if end > len(w.buf) { + grown := make([]byte, end) + copy(grown, w.buf) + w.buf = grown + } + copy(w.buf[off:], p) + return len(p), nil +} + +func (w *WriteAtBuffer) Bytes() []byte { + w.mu.Lock() + defer w.mu.Unlock() + out := make([]byte, len(w.buf)) + copy(out, w.buf) + return out +} + +func TestWriteAtBuffer_BasicWriteAndRead(t *testing.T) { + var buf WriteAtBuffer + data := []byte("hello world") + n, err := buf.WriteAt(data, 0) + if err != nil { + t.Fatalf("WriteAt error: %v", err) + } + if n != len(data) { + t.Fatalf("WriteAt returned %d, want %d", n, len(data)) + } + got := string(buf.Bytes()) + if got != "hello world" { + t.Errorf("Bytes() = %q, want %q", got, "hello world") + } +} + +func TestWriteAtBuffer_NonZeroOffset(t *testing.T) { + var buf WriteAtBuffer + buf.WriteAt([]byte("aaa"), 0) + buf.WriteAt([]byte("bbb"), 3) + + got := string(buf.Bytes()) + if got != "aaabbb" { + t.Errorf("Bytes() = %q, want %q", got, "aaabbb") + } +} + +func TestWriteAtBuffer_OverlappingWrite(t *testing.T) { + var buf WriteAtBuffer + buf.WriteAt([]byte("AAAA"), 0) + buf.WriteAt([]byte("BB"), 1) + + got := string(buf.Bytes()) + if got != "ABBA" { + t.Errorf("Bytes() = %q, want %q", got, "ABBA") + } +} + +func TestWriteAtBuffer_SparseWrite(t *testing.T) { + var buf WriteAtBuffer + buf.WriteAt([]byte("X"), 5) + + got := buf.Bytes() + if len(got) != 6 { + t.Fatalf("len(Bytes()) = %d, want 6", len(got)) + } + for i := 0; i < 5; i++ { + if got[i] != 0 { + t.Errorf("Bytes()[%d] = %d, want 0", i, got[i]) + } + } + if got[5] != 'X' { + t.Errorf("Bytes()[5] = %q, want 'X'", got[5]) + } +} + +func TestWriteAtBuffer_GrowsBuffer(t *testing.T) { + var buf WriteAtBuffer + buf.WriteAt([]byte("ab"), 0) + buf.WriteAt([]byte("cdef"), 2) + + got := string(buf.Bytes()) + if got != "abcdef" { + t.Errorf("Bytes() = %q, want %q", got, "abcdef") + } +} + +func TestWriteAtBuffer_BytesReturnsDefensiveCopy(t *testing.T) { + var buf WriteAtBuffer + buf.WriteAt([]byte("original"), 0) + + snapshot := buf.Bytes() + snapshot[0] = 'X' + + got := string(buf.Bytes()) + if got != "original" { + t.Errorf("internal buffer was mutated via Bytes() return: got %q, want %q", got, "original") + } +} + +func TestWriteAtBuffer_EmptyBuffer(t *testing.T) { + var buf WriteAtBuffer + got := buf.Bytes() + if len(got) != 0 { + t.Errorf("Bytes() on empty buffer has len %d, want 0", len(got)) + } +} + +func TestWriteAtBuffer_ConcurrentWrites(t *testing.T) { + var buf WriteAtBuffer + var wg sync.WaitGroup + + for i := range 100 { + wg.Add(1) + go func(offset int) { + defer wg.Done() + buf.WriteAt([]byte{byte(offset)}, int64(offset)) + }(i) + } + wg.Wait() + + got := buf.Bytes() + if len(got) != 100 { + t.Fatalf("len(Bytes()) = %d, want 100", len(got)) + } + for i := range 100 { + if got[i] != byte(i) { + t.Errorf("Bytes()[%d] = %d, want %d", i, got[i], i) + } + } +} diff --git a/sidecar/s3/emit.go b/sidecar/s3/emit.go new file mode 100644 index 00000000..d6a4a764 --- /dev/null +++ b/sidecar/s3/emit.go @@ -0,0 +1,150 @@ +package s3 + +import ( + "compress/gzip" + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "hash" + "io" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + tmtypes "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager/types" +) + +// EmitResult reports what was published: the hex SHA-256 over the uncompressed +// payload, surfaced in the TaskResult/log so an operator can verify the +// decompressed object after gunzip. It is the canonical reader-verifiable seal; +// it is not embedded in the object itself (a payload cannot carry the hash of +// its own bytes). +type EmitResult struct { + // UncompressedSHA256 is the hex digest of the bytes fed into gzip. + UncompressedSHA256 string +} + +// StreamGzipNDJSON gzips each record as one NDJSON line and streams it to S3 +// without buffering the whole payload, computing a SHA-256 over the +// uncompressed bytes as they pass. The integrity seal is twofold: +// +// - ChecksumAlgorithm=SHA256 on the put: the SDK sends a trailing aws-chunked +// checksum, so S3 validates a SHA-256 of the compressed bytes it received +// without precomputing — the wire seal. For a multipart upload (payload +// over the uploader's threshold) S3 stores the composite-of-parts form +// (`-N`), still a valid per-part seal but not a flat SHA-256 of the +// body a reader can recompute. +// - the returned UncompressedSHA256: the logical seal over the pre-gzip +// payload, surfaced via EmitResult so a reader verifies the decompressed +// bytes independently. This, not the S3-side checksum, is the canonical +// reader-verifiable seal. +// +// io.Pipe backpressure is preserved: the marshaling goroutine blocks on the +// uploader's reads, so memory stays bounded to the uploader's part-buffer pool +// (~part_size × (concurrency+1)), independent of total payload size. +func StreamGzipNDJSON[T any](ctx context.Context, uploader Uploader, bucket, key string, records []T) (EmitResult, error) { + return streamGzip(ctx, uploader, bucket, key, func(w io.Writer) error { + return encodeNDJSON(w, records) + }) +} + +// StreamGzipJSON gzips a single indented JSON object and streams it to S3 with +// the same uncompressed-payload seal as StreamGzipNDJSON. +func StreamGzipJSON(ctx context.Context, uploader Uploader, bucket, key string, obj any) (EmitResult, error) { + return streamGzip(ctx, uploader, bucket, key, func(w io.Writer) error { + return encodeIndentedJSON(w, obj) + }) +} + +// StreamGzipFunc gzips and streams whatever write emits, under the same seal as +// StreamGzipNDJSON. It exists for producers that generate the payload lazily +// (e.g. querying one block at a time) and so must not materialize the whole +// record set in memory first. write receives the destination writer and must +// emit the complete uncompressed payload. +func StreamGzipFunc(ctx context.Context, uploader Uploader, bucket, key string, write func(io.Writer) error) (EmitResult, error) { + return streamGzip(ctx, uploader, bucket, key, write) +} + +// streamGzip wires the io.Pipe -> hash-tee -> gzip pipeline and the S3 upload. +// write emits the uncompressed payload into the supplied writer. +func streamGzip(ctx context.Context, uploader Uploader, bucket, key string, write func(io.Writer) error) (EmitResult, error) { + pr, pw := io.Pipe() + h := sha256.New() + + writeErr := make(chan error, 1) + go func() { + writeErr <- writeGzip(pw, h, write) + }() + + _, uploadErr := uploader.UploadObject(ctx, &transfermanager.UploadObjectInput{ + Bucket: aws.String(bucket), + Key: aws.String(key), + Body: pr, + ContentType: aws.String("application/gzip"), + ChecksumAlgorithm: tmtypes.ChecksumAlgorithmSha256, + }) + if uploadErr != nil { + // Unblock the writer goroutine if the upload aborted early. + pr.CloseWithError(uploadErr) + } + + wErr := <-writeErr + if uploadErr != nil { + return EmitResult{}, uploadErr + } + if wErr != nil { + return EmitResult{}, wErr + } + return EmitResult{UncompressedSHA256: hex.EncodeToString(h.Sum(nil))}, nil +} + +// writeGzip tees the uncompressed payload through h (the integrity hash) while +// gzipping it into the pipe. The pipe is closed with the terminal error so the +// uploader observes it instead of a truncated, silently-valid object. +func writeGzip(pw *io.PipeWriter, h hash.Hash, write func(io.Writer) error) (retErr error) { + defer func() { + if retErr != nil { + pw.CloseWithError(retErr) + } else { + _ = pw.Close() + } + }() + + gw := gzip.NewWriter(pw) + defer func() { + if err := gw.Close(); err != nil && retErr == nil { + retErr = fmt.Errorf("closing gzip writer: %w", err) + } + }() + + // Registered last so it runs first: a panic in write (e.g. a record's + // MarshalJSON over attacker-adjacent chain data) becomes retErr, so the + // pipe-close defer does CloseWithError — the uploader aborts instead of + // publishing a truncated object, and the panic never escapes this + // task-spawned goroutine to crash the sidecar. + defer func() { + if r := recover(); r != nil { + retErr = fmt.Errorf("panic in payload writer: %v", r) + } + }() + + return write(io.MultiWriter(gw, h)) +} + +func encodeNDJSON[T any](w io.Writer, records []T) error { + enc := json.NewEncoder(w) + for i := range records { + // Encoder.Encode appends '\n', yielding one record per line. + if err := enc.Encode(records[i]); err != nil { + return fmt.Errorf("marshaling record %d: %w", i, err) + } + } + return nil +} + +func encodeIndentedJSON(w io.Writer, obj any) error { + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + return enc.Encode(obj) +} diff --git a/sidecar/s3/emit_test.go b/sidecar/s3/emit_test.go new file mode 100644 index 00000000..05239117 --- /dev/null +++ b/sidecar/s3/emit_test.go @@ -0,0 +1,167 @@ +package s3 + +import ( + "bytes" + "compress/gzip" + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "io" + "strings" + "testing" + + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + tmtypes "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager/types" +) + +// captureUploader records the put input and drains the streamed body into a +// buffer, mimicking S3 reading the gzip stream off the pipe. +type captureUploader struct { + in *transfermanager.UploadObjectInput + body []byte + err error +} + +func (u *captureUploader) UploadObject(_ context.Context, in *transfermanager.UploadObjectInput, _ ...func(*transfermanager.Options)) (*transfermanager.UploadObjectOutput, error) { + u.in = in + if u.err != nil { + // Don't drain — exercise the early-abort path where the writer + // goroutine must be unblocked via CloseWithError. + return nil, u.err + } + var buf bytes.Buffer + if _, err := io.Copy(&buf, in.Body); err != nil { + return nil, err + } + u.body = buf.Bytes() + return &transfermanager.UploadObjectOutput{}, nil +} + +func gunzip(t *testing.T, b []byte) []byte { + t.Helper() + r, err := gzip.NewReader(bytes.NewReader(b)) + if err != nil { + t.Fatalf("gzip.NewReader: %v", err) + } + out, err := io.ReadAll(r) + if err != nil { + t.Fatalf("reading gzip: %v", err) + } + return out +} + +func TestStreamGzipNDJSON_RoundTripAndSeal(t *testing.T) { + up := &captureUploader{} + records := []map[string]any{ + {"height": 1, "ok": true}, + {"height": 2, "ok": false}, + } + + res, err := StreamGzipNDJSON(context.Background(), up, "bkt", "k.ndjson.gz", records) + if err != nil { + t.Fatalf("StreamGzipNDJSON: %v", err) + } + + payload := gunzip(t, up.body) + lines := strings.Split(strings.TrimRight(string(payload), "\n"), "\n") + if len(lines) != 2 { + t.Fatalf("got %d NDJSON lines, want 2: %q", len(lines), payload) + } + + // The returned seal must equal SHA-256 over the uncompressed payload. + want := sha256.Sum256(payload) + if res.UncompressedSHA256 != hex.EncodeToString(want[:]) { + t.Errorf("UncompressedSHA256 = %s, want %s", res.UncompressedSHA256, hex.EncodeToString(want[:])) + } + + // The put must request a SHA-256 checksum (the wire seal over stored bytes). + if up.in.ChecksumAlgorithm != tmtypes.ChecksumAlgorithmSha256 { + t.Errorf("ChecksumAlgorithm = %q, want SHA256", up.in.ChecksumAlgorithm) + } +} + +func TestStreamGzipJSON_RoundTripAndSeal(t *testing.T) { + up := &captureUploader{} + obj := map[string]any{"match": true, "height": 42} + + res, err := StreamGzipJSON(context.Background(), up, "bkt", "k.json.gz", obj) + if err != nil { + t.Fatalf("StreamGzipJSON: %v", err) + } + + payload := gunzip(t, up.body) + want := sha256.Sum256(payload) + if res.UncompressedSHA256 != hex.EncodeToString(want[:]) { + t.Errorf("UncompressedSHA256 = %s, want %s", res.UncompressedSHA256, hex.EncodeToString(want[:])) + } + if !strings.Contains(string(payload), "\"match\": true") { + t.Errorf("payload not indented JSON: %q", payload) + } +} + +func TestStreamGzipFunc_LazyWriter(t *testing.T) { + up := &captureUploader{} + res, err := StreamGzipFunc(context.Background(), up, "bkt", "k", func(w io.Writer) error { + _, err := io.WriteString(w, "line-a\nline-b\n") + return err + }) + if err != nil { + t.Fatalf("StreamGzipFunc: %v", err) + } + payload := gunzip(t, up.body) + if string(payload) != "line-a\nline-b\n" { + t.Errorf("payload = %q", payload) + } + want := sha256.Sum256(payload) + if res.UncompressedSHA256 != hex.EncodeToString(want[:]) { + t.Errorf("seal mismatch") + } +} + +func TestStreamGzip_WriterErrorPropagates(t *testing.T) { + up := &captureUploader{} + sentinel := errors.New("boom") + _, err := StreamGzipFunc(context.Background(), up, "bkt", "k", func(io.Writer) error { + return sentinel + }) + if !errors.Is(err, sentinel) { + t.Fatalf("err = %v, want wrap of sentinel", err) + } +} + +func TestStreamGzip_WriterPanicBecomesError(t *testing.T) { + // A panic in the payload writer (e.g. a record's MarshalJSON over + // attacker-adjacent chain data) runs on a task-spawned goroutine outside the + // engine's handler recover. It must convert to a returned error — not crash + // the sidecar. Reaching this assertion at all proves the process survived. + up := &captureUploader{} + _, err := StreamGzipFunc(context.Background(), up, "bkt", "k", func(io.Writer) error { + panic("marshal blew up") + }) + if err == nil { + t.Fatal("expected an error from a panicking writer, got nil") + } + if !strings.Contains(err.Error(), "panic in payload writer") { + t.Fatalf("err = %v, want it to name the recovered panic", err) + } +} + +func TestStreamGzip_UploadErrorUnblocksWriter(t *testing.T) { + // A writer large enough to fill the pipe and gzip buffers would deadlock if + // the pipe were never closed on upload failure. Reaching the return without + // hanging proves CloseWithError unblocked it (go test's timeout is the + // backstop if it does not). + up := &captureUploader{err: errors.New("s3 down")} + _, err := StreamGzipFunc(context.Background(), up, "bkt", "k", func(w io.Writer) error { + for i := 0; i < 100000; i++ { + if _, werr := io.WriteString(w, "padding-line\n"); werr != nil { + return werr + } + } + return nil + }) + if err == nil { + t.Fatal("expected upload error, got nil") + } +} diff --git a/sidecar/s3/errors.go b/sidecar/s3/errors.go new file mode 100644 index 00000000..df84030b --- /dev/null +++ b/sidecar/s3/errors.go @@ -0,0 +1,55 @@ +package s3 + +import ( + "errors" + "fmt" + "net" + + "github.com/aws/smithy-go" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +// ClassifyS3Error wraps an S3 error with operator-actionable context. +// The returned TaskError includes the task name, S3 coordinates, a +// human-readable message, and a hint for resolution. +func ClassifyS3Error(task, bucket, key, region string, err error) *engine.TaskError { + te := &engine.TaskError{ + Task: task, + Operation: "S3", + Cause: err.Error(), + } + + var apiErr smithy.APIError + if errors.As(err, &apiErr) { + switch apiErr.ErrorCode() { + case "NoSuchBucket": + te.Message = fmt.Sprintf("bucket %q does not exist in region %s", bucket, region) + te.Hint = "verify SEI_SNAPSHOT_BUCKET and SEI_SNAPSHOT_REGION environment variables" + case "NoSuchKey": + te.Message = fmt.Sprintf("object %q not found in s3://%s", key, bucket) + te.Hint = "the snapshot or genesis file may not have been uploaded yet" + case "AccessDenied": + te.Message = fmt.Sprintf("access denied to s3://%s/%s in region %s", bucket, key, region) + te.Hint = "check the pod's ServiceAccount IAM role has s3:GetObject permission on this bucket" + case "SlowDown", "ServiceUnavailable": + te.Message = fmt.Sprintf("S3 throttled request to s3://%s/%s", bucket, key) + te.Hint = "transient S3 throttling; the task will be retried" + te.Retryable = true + default: + te.Message = fmt.Sprintf("S3 error %s on s3://%s/%s", apiErr.ErrorCode(), bucket, key) + } + return te + } + + var netErr *net.OpError + if errors.As(err, &netErr) { + te.Message = fmt.Sprintf("network error accessing s3://%s/%s", bucket, key) + te.Hint = "check VPC endpoints and DNS resolution for S3" + te.Retryable = true + return te + } + + te.Message = fmt.Sprintf("unexpected error accessing s3://%s/%s: %v", bucket, key, err) + return te +} diff --git a/sidecar/s3/errors_test.go b/sidecar/s3/errors_test.go new file mode 100644 index 00000000..28471196 --- /dev/null +++ b/sidecar/s3/errors_test.go @@ -0,0 +1,120 @@ +package s3 + +import ( + "fmt" + "net" + "strings" + "testing" + + "github.com/aws/smithy-go" +) + +type mockAPIError struct { + code string + message string +} + +func (e *mockAPIError) Error() string { return e.message } +func (e *mockAPIError) ErrorCode() string { return e.code } +func (e *mockAPIError) ErrorMessage() string { return e.message } +func (e *mockAPIError) ErrorFault() smithy.ErrorFault { return smithy.FaultUnknown } + +func TestClassifyS3Error_NoSuchBucket(t *testing.T) { + err := &mockAPIError{code: "NoSuchBucket", message: "bucket not found"} + te := ClassifyS3Error("snapshot-restore", "my-bucket", "key", "us-east-1", err) + + if !strings.Contains(te.Message, "my-bucket") { + t.Errorf("expected bucket in message, got: %s", te.Message) + } + if !strings.Contains(te.Hint, "SEI_SNAPSHOT_BUCKET") { + t.Errorf("expected env var hint, got: %s", te.Hint) + } + if te.Retryable { + t.Error("NoSuchBucket should not be retryable") + } + if te.Task != "snapshot-restore" { + t.Errorf("Task = %q, want snapshot-restore", te.Task) + } +} + +func TestClassifyS3Error_NoSuchKey(t *testing.T) { + err := &mockAPIError{code: "NoSuchKey", message: "not found"} + te := ClassifyS3Error("configure-genesis", "bucket", "genesis.json", "eu-central-1", err) + + if !strings.Contains(te.Message, "genesis.json") { + t.Errorf("expected key in message, got: %s", te.Message) + } + if te.Retryable { + t.Error("NoSuchKey should not be retryable") + } +} + +func TestClassifyS3Error_AccessDenied(t *testing.T) { + err := &mockAPIError{code: "AccessDenied", message: "forbidden"} + te := ClassifyS3Error("snapshot-restore", "bucket", "key", "us-east-1", err) + + if !strings.Contains(te.Hint, "IAM") { + t.Errorf("expected IAM hint, got: %s", te.Hint) + } + if te.Retryable { + t.Error("AccessDenied should not be retryable") + } +} + +func TestClassifyS3Error_SlowDown(t *testing.T) { + err := &mockAPIError{code: "SlowDown", message: "throttled"} + te := ClassifyS3Error("snapshot-restore", "bucket", "key", "us-east-1", err) + + if !te.Retryable { + t.Error("SlowDown should be retryable") + } +} + +func TestClassifyS3Error_NetworkError(t *testing.T) { + err := &net.OpError{Op: "dial", Net: "tcp", Err: fmt.Errorf("connection refused")} + te := ClassifyS3Error("snapshot-restore", "bucket", "key", "us-east-1", err) + + if !te.Retryable { + t.Error("network error should be retryable") + } + if !strings.Contains(te.Hint, "VPC") { + t.Errorf("expected VPC hint, got: %s", te.Hint) + } +} + +func TestClassifyS3Error_UnknownError(t *testing.T) { + err := fmt.Errorf("something unexpected") + te := ClassifyS3Error("export-state", "bucket", "key", "us-east-1", err) + + if te.Retryable { + t.Error("unknown error should not be retryable") + } + if te.Cause != "something unexpected" { + t.Errorf("Cause = %q, want original error", te.Cause) + } +} + +func TestClassifyS3Error_ErrorString(t *testing.T) { + err := &mockAPIError{code: "NoSuchBucket", message: "bucket not found"} + te := ClassifyS3Error("snapshot-restore", "my-bucket", "key", "us-east-1", err) + + s := te.Error() + if !strings.Contains(s, "snapshot-restore") { + t.Errorf("Error() should contain task name, got: %s", s) + } + if !strings.Contains(s, "[hint:") { + t.Errorf("Error() should contain hint, got: %s", s) + } +} + +func TestClassifyS3Error_UnknownAPICode(t *testing.T) { + err := &mockAPIError{code: "InternalError", message: "server error"} + te := ClassifyS3Error("snapshot-restore", "bucket", "key", "us-east-1", err) + + if !strings.Contains(te.Message, "InternalError") { + t.Errorf("expected error code in message, got: %s", te.Message) + } + if te.Retryable { + t.Error("unknown API error should not be retryable by default") + } +} diff --git a/sidecar/serve.go b/sidecar/serve.go new file mode 100644 index 00000000..f907a74e --- /dev/null +++ b/sidecar/serve.go @@ -0,0 +1,255 @@ +package main + +import ( + "context" + "errors" + "fmt" + "os" + "os/signal" + "path/filepath" + "slices" + "syscall" + "time" + + "github.com/sei-protocol/seilog" + "github.com/urfave/cli/v3" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" + "github.com/sei-protocol/sei-k8s-controller/sidecar/server" + "github.com/sei-protocol/sei-k8s-controller/sidecar/tasks" +) + +var serveLog = seilog.NewLogger("seictl", "serve") + +var serveCmd = cli.Command{ + Name: "serve", + Usage: "Start the sidecar task executor and HTTP API", + Flags: []cli.Flag{ + &cli.StringFlag{ + Name: "port", + Sources: cli.EnvVars("SEI_SIDECAR_PORT"), + Value: "7777", + Usage: "Port for the sidecar HTTP API", + }, + }, + Action: func(ctx context.Context, cmd *cli.Command) error { + defer func() { _ = seilog.Close() }() + + ctx, stop := signal.NotifyContext(ctx, syscall.SIGTERM, syscall.SIGINT) + defer stop() + + // destinations.home is bound to SEI_HOME on the root command and marked + // Required there, so an unset value fails before we reach this point. + // It used to fall back to "/sei" while the controller mounts the data + // PVC elsewhere, which made a dropped SEI_HOME silent. + homeDir := destinations.home + port := cmd.String("port") + chainID := os.Getenv("SEI_CHAIN_ID") + genesisBucket := os.Getenv("SEI_GENESIS_BUCKET") + genesisRegion := os.Getenv("SEI_GENESIS_REGION") + snapshotBucket := os.Getenv("SEI_SNAPSHOT_BUCKET") + snapshotRegion := os.Getenv("SEI_SNAPSHOT_REGION") + + podName := os.Getenv("HOSTNAME") + if podName == "" { + if h, err := os.Hostname(); err == nil { + podName = h + } + } + if podName == "" { + podName = "unknown" + } + + for _, kv := range []struct{ name, val string }{ + {"SEI_CHAIN_ID", chainID}, + {"SEI_GENESIS_BUCKET", genesisBucket}, + {"SEI_GENESIS_REGION", genesisRegion}, + {"SEI_SNAPSHOT_BUCKET", snapshotBucket}, + {"SEI_SNAPSHOT_REGION", snapshotRegion}, + } { + if kv.val == "" { + return fmt.Errorf("required environment variable %s is not set", kv.name) + } + } + + var snapshotUploadInterval time.Duration + if raw := os.Getenv("SEI_SNAPSHOT_UPLOAD_INTERVAL"); raw != "" { + parsed, err := time.ParseDuration(raw) + if err != nil { + return fmt.Errorf("invalid SEI_SNAPSHOT_UPLOAD_INTERVAL %q: %w", raw, err) + } + snapshotUploadInterval = parsed + } + + var snapshotUploadTimeout time.Duration + if raw := os.Getenv("SEI_SNAPSHOT_UPLOAD_TIMEOUT"); raw != "" { + parsed, err := time.ParseDuration(raw) + if err != nil { + return fmt.Errorf("invalid SEI_SNAPSHOT_UPLOAD_TIMEOUT %q: %w", raw, err) + } + snapshotUploadTimeout = parsed + } + + authnMode, err := server.AuthnMode() + if err != nil { + return err + } + + // Checked before buildExecutionConfig so the unsafe combination never + // opens the keyring at all. + if err := checkKeyringNeedsAuthn(os.Getenv("SEI_KEYRING_BACKEND"), authnMode); err != nil { + return err + } + + execCfg, err := buildExecutionConfig(homeDir) + if err != nil { + return err + } + + if err := tasks.EnsureDefaultConfig(homeDir); err != nil { + return fmt.Errorf("home directory init failed: %w", err) + } + + store, err := engine.NewSQLiteStore(filepath.Join(homeDir, "sidecar.db")) + if err != nil { + return fmt.Errorf("open result store: %w", err) + } + // The store also backs the pre-broadcast idempotency marker for + // sign-tx handlers (must be set before the handlers copy execCfg). + execCfg.Checkpointer = store + + snapshotRestorer, err := tasks.NewSnapshotRestorer(homeDir, snapshotBucket, snapshotRegion, chainID, nil, nil) + if err != nil { + return fmt.Errorf("creating snapshot restorer: %w", err) + } + + snapshotUploader, err := tasks.NewSnapshotUploader(homeDir, snapshotBucket, snapshotRegion, chainID, snapshotUploadInterval, nil) + if err != nil { + return fmt.Errorf("creating snapshot uploader: %w", err) + } + snapshotUploader.EmitStartupMetrics() + + handlers := map[engine.TaskType]engine.TaskHandler{ + engine.TaskSnapshotRestore: snapshotRestorer.Handler(), + engine.TaskConfigPatch: tasks.NewConfigPatcher(homeDir).Handler(), + engine.TaskConfigApply: tasks.NewConfigApplier(homeDir).Handler(), + engine.TaskConfigValidate: tasks.NewConfigValidator(homeDir).Handler(), + engine.TaskConfigReload: tasks.NewConfigReloader(homeDir).Handler(), + engine.TaskMarkReady: tasks.MarkReadyHandler(), + engine.TaskMarkNotReady: tasks.NewMarkNotReadier(store).Handler(), + engine.TaskRestartSeid: tasks.NewRestartSeider().Handler(), + engine.TaskStopSeid: tasks.NewStopSeider().Handler(), + engine.TaskResetData: tasks.NewResetDataer(homeDir).Handler(), + engine.TaskConfigureGenesis: tasks.NewGenesisFetcher(homeDir, chainID, genesisBucket, genesisRegion, nil).Handler(), + engine.TaskConfigureStateSync: tasks.NewStateSyncConfigurer(homeDir, nil).Handler(), + engine.TaskSnapshotUpload: snapshotUploader.Handler(), + engine.TaskSnapshotUploadOnce: snapshotUploader.OnceHandler(snapshotUploadTimeout), + engine.TaskResultExport: tasks.NewResultExporter(homeDir, chainID, podName, nil).Handler(), + engine.TaskAwaitCondition: tasks.NewConditionWaiter(nil).Handler(), + engine.TaskGenerateIdentity: tasks.NewIdentityGenerator(homeDir).Handler(), + engine.TaskGenerateGentx: tasks.NewGentxGenerator(homeDir).Handler(), + engine.TaskUploadGenesisArtifacts: tasks.NewGenesisArtifactUploader(homeDir, genesisBucket, genesisRegion, chainID, nil).Handler(), + engine.TaskAssembleAndUploadGenesis: tasks.NewGenesisAssembler(homeDir, genesisBucket, genesisRegion, chainID, nil, nil).Handler(), + engine.TaskSetGenesisPeers: tasks.NewGenesisPeersSetter(homeDir, genesisBucket, genesisRegion, chainID, nil).Handler(), + engine.TaskGovVote: tasks.NewGovVoter(execCfg).Handler(), + engine.TaskGovSoftwareUpgrade: tasks.NewGovSoftwareUpgrader(execCfg).Handler(), + engine.TaskGovParamChange: tasks.NewGovParamChanger(execCfg).Handler(), + engine.TaskEvmLogicalDigest: tasks.NewEvmLogicalDigester(nil).Handler(), + } + + eng := engine.NewEngine(ctx, handlers, store) + eng.Config = execCfg + // Rehydrate after Config is installed so sign-tx handlers see + // the full dep set via the goroutine-spawn happens-before edge. + eng.RehydrateStaleTasks() + + bindAddr := server.BindAddress(port, authnMode) + logArgs := []any{"authnMode", authnMode, "bind", bindAddr} + if authnMode == server.AuthnModeTrustedHeader { + logArgs = append(logArgs, "bypassPaths", server.BypassPaths()) + } + serveLog.Info("sidecar HTTP", logArgs...) + srv := server.NewServer(bindAddr, eng, homeDir, authnMode) + srvErr := srv.ListenAndServe(ctx) + + if closeErr := store.Close(); closeErr != nil { + fmt.Fprintf(os.Stderr, "warn: result store close: %v\n", closeErr) + } + + if srvErr != nil && !errors.Is(srvErr, context.Canceled) { + return fmt.Errorf("server error: %w", srvErr) + } + return nil + }, +} + +// checkKeyringNeedsAuthn refuses the one combination that hands an unauthorized +// caller a signing key: a configured operator keyring behind an unauthenticated +// listener. +// +// The two settings used to be decided independently and never compared. An unset +// SEI_SIDECAR_AUTHN_MODE resolves to unauthenticated, which binds every +// interface and installs no middleware, so a single dropped environment variable +// exposed POST /v0/tasks — gov-vote included — to any pod in the cluster while +// the keyring stayed open. Nothing failed; probes passed and tasks succeeded. +// +// The controller sets both, so this fires only on a misconfiguration. That is the +// point: it turns one into a startup error instead of a quiet opening. +func checkKeyringNeedsAuthn(keyringBackend, authnMode string) error { + if keyringBackend == "" || authnMode != server.AuthnModeUnauthenticated { + return nil + } + return fmt.Errorf("refusing to start: SEI_KEYRING_BACKEND=%q means this sidecar "+ + "holds an operator keyring and can sign transactions, but "+ + "SEI_SIDECAR_AUTHN_MODE is unauthenticated, which binds all interfaces "+ + "with no request authentication; set SEI_SIDECAR_AUTHN_MODE=%s", + keyringBackend, server.AuthnModeTrustedHeader) +} + +// buildExecutionConfig assembles the engine's runtime dependencies: +// keyring (opened from SEI_KEYRING_BACKEND, or nil) and RPC client +// (pointed at the local seid). Sign-tx tasks consume both; tasks that +// don't need them ignore the fields. +func buildExecutionConfig(homeDir string) (engine.ExecutionConfig, error) { + // Wipe passphrase before any branching so every return path leaves + // /proc//environ clean. + passphrase := os.Getenv("SEI_KEYRING_PASSPHRASE") + _ = os.Unsetenv("SEI_KEYRING_PASSPHRASE") + + rpcClient := rpc.NewClient(rpc.DefaultEndpoint, nil) + + backend := os.Getenv("SEI_KEYRING_BACKEND") + if backend == "" { + return engine.ExecutionConfig{RPC: rpcClient}, nil + } + + if !slices.Contains(server.AllowedBackends, backend) { + return engine.ExecutionConfig{}, fmt.Errorf( + "unsupported SEI_KEYRING_BACKEND %q (allowed: test|file|os)", backend) + } + + dir := os.Getenv("SEI_KEYRING_DIR") + if dir == "" { + dir = filepath.Join(homeDir, "keyring-file") + } + + if backend == server.BackendFile && passphrase == "" { + return engine.ExecutionConfig{}, fmt.Errorf( + "SEI_KEYRING_PASSPHRASE required when SEI_KEYRING_BACKEND=file") + } + + kr, err := server.OpenKeyring(backend, dir, passphrase) + if err != nil { + // Don't %w-wrap: OpenKeyring redacted err.Error(), but a typed + // field in the underlying SDK chain could resurface the secret. + return engine.ExecutionConfig{}, err + } + + if err := server.SmokeTestKeyring(kr); err != nil { + return engine.ExecutionConfig{}, err + } + + serveLog.Info("keyring opened", "backend", backend, "dir", dir) + return engine.ExecutionConfig{Keyring: kr, RPC: rpcClient}, nil +} diff --git a/sidecar/serve_test.go b/sidecar/serve_test.go new file mode 100644 index 00000000..11544e7b --- /dev/null +++ b/sidecar/serve_test.go @@ -0,0 +1,148 @@ +package main + +import ( + "os" + "strings" + "testing" +) + +// TestBuildExecutionConfig_UnsetReturnsZero verifies the Phase-1 default: +// no SEI_KEYRING_BACKEND means no keyring, no error, sidecar boots normally. +func TestBuildExecutionConfig_UnsetReturnsZero(t *testing.T) { + withEnv(t, map[string]string{ + "SEI_KEYRING_BACKEND": "", + "SEI_KEYRING_DIR": "", + "SEI_KEYRING_PASSPHRASE": "", + }) + + cfg, err := buildExecutionConfig(t.TempDir()) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if cfg.Keyring != nil { + t.Fatal("expected nil keyring when SEI_KEYRING_BACKEND is unset") + } +} + +func TestBuildExecutionConfig_UnknownBackendFailsStartup(t *testing.T) { + withEnv(t, map[string]string{ + "SEI_KEYRING_BACKEND": "kms", + "SEI_KEYRING_PASSPHRASE": "", + }) + + _, err := buildExecutionConfig(t.TempDir()) + if err == nil { + t.Fatal("expected error for unknown backend") + } + if !strings.Contains(err.Error(), "unsupported") { + t.Fatalf("error should mention unsupported, got: %v", err) + } +} + +func TestBuildExecutionConfig_FileBackend_MissingPassphraseFailsStartup(t *testing.T) { + withEnv(t, map[string]string{ + "SEI_KEYRING_BACKEND": "file", + "SEI_KEYRING_PASSPHRASE": "", + }) + + _, err := buildExecutionConfig(t.TempDir()) + if err == nil { + t.Fatal("expected error for missing passphrase") + } + if !strings.Contains(err.Error(), "SEI_KEYRING_PASSPHRASE") { + t.Fatalf("error should name the missing env, got: %v", err) + } +} + +func TestBuildExecutionConfig_FileBackend_WipesPassphrase(t *testing.T) { + const passphrase = "do-not-leak-this" + withEnv(t, map[string]string{ + "SEI_KEYRING_BACKEND": "file", + "SEI_KEYRING_DIR": t.TempDir(), + "SEI_KEYRING_PASSPHRASE": passphrase, + }) + + cfg, err := buildExecutionConfig(t.TempDir()) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if cfg.Keyring == nil { + t.Fatal("expected non-nil keyring") + } + if got := os.Getenv("SEI_KEYRING_PASSPHRASE"); got != "" { + t.Fatalf("passphrase env should be wiped, found %q", got) + } +} + +// TestBuildExecutionConfig_NoPassphraseInError asserts that for every +// path that returns a non-nil error from buildExecutionConfig, the +// passphrase is absent from the error message. Each case is a distinct +// failure mode and the test serves as a regression guard against future +// error-construction changes that might interpolate the env value. +func TestBuildExecutionConfig_NoPassphraseInError(t *testing.T) { + const passphrase = "do-not-leak-this-passphrase" + cases := []struct { + name string + backend string + }{ + {"unknown backend", "kms"}, + {"empty passphrase on file backend", "file"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + env := map[string]string{ + "SEI_KEYRING_BACKEND": tc.backend, + } + if tc.name == "unknown backend" { + env["SEI_KEYRING_PASSPHRASE"] = passphrase + } + withEnv(t, env) + + _, err := buildExecutionConfig(t.TempDir()) + if err == nil { + t.Fatal("expected error") + } + if strings.Contains(err.Error(), passphrase) { + t.Fatalf("passphrase leaked into error: %v", err) + } + }) + } +} + +func TestBuildExecutionConfig_TestBackend(t *testing.T) { + withEnv(t, map[string]string{ + "SEI_KEYRING_BACKEND": "test", + "SEI_KEYRING_DIR": t.TempDir(), + "SEI_KEYRING_PASSPHRASE": "", + }) + + cfg, err := buildExecutionConfig(t.TempDir()) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if cfg.Keyring == nil { + t.Fatal("expected non-nil keyring for test backend") + } +} + +// withEnv sets the supplied env vars for the duration of the test and +// restores prior values via t.Cleanup. Empty values are honored as +// "unset" so callers can express the Phase-1 default explicitly. +func withEnv(t *testing.T, kv map[string]string) { + t.Helper() + for k, v := range kv { + prev, had := os.LookupEnv(k) + if v == "" { + _ = os.Unsetenv(k) + } else { + _ = os.Setenv(k, v) + } + t.Cleanup(func() { + if had { + _ = os.Setenv(k, prev) + } else { + _ = os.Unsetenv(k) + } + }) + } +} diff --git a/sidecar/server/auth.go b/sidecar/server/auth.go new file mode 100644 index 00000000..53dd23e8 --- /dev/null +++ b/sidecar/server/auth.go @@ -0,0 +1,135 @@ +package server + +import ( + "fmt" + "net/http" + "os" + "sort" + "strings" + + "github.com/prometheus/client_golang/prometheus" +) + +// SECURITY POSTURE — trusted-header mode +// +// The trust boundary is the POD, not the loopback interface. Loopback +// bind keeps the sidecar off the pod network, but every container in +// the pod shares the network namespace and can reach 127.0.0.1:7777 +// directly with a forged X-Remote-User. With shareProcessNamespace: +// true (the current SeiNode default), a colocated container can also +// read /proc//mem and exfiltrate the unlocked keyring — +// memory-read is the load-bearing threat, not header forgery; this +// middleware does not defend against it. +// +// Pod-level isolation requirements (enforced by the controller-side +// PR): hostNetwork, hostPID, and hostIPC MUST all be false. +// hostNetwork would expose 127.0.0.1 to every other hostNetwork pod +// on the node. hostPID/hostIPC would let off-pod processes attach. + +const ( + // AuthnModeUnauthenticated: sidecar binds all interfaces; every + // caller is trusted. Acceptable only on validator-only pod + // networks. + AuthnModeUnauthenticated = "" + + // AuthnModeTrustedHeader pairs the sidecar with an in-pod + // kube-rbac-proxy on TLS :8443. The proxy performs TokenReview + + // SAR against the K8s API and forwards passed requests to + // 127.0.0.1:7777 with X-Remote-User naming the authenticated + // identity. + AuthnModeTrustedHeader = "trusted-header" + + remoteUserHeader = "X-Remote-User" +) + +// bypassPaths skip the X-Remote-User check in trusted-header mode. +// Each path's caller does not carry K8s auth headers, so requiring +// X-Remote-User would break the corresponding probe / scrape. +// kube-rbac-proxy must include every path here in its --allow-paths. +var bypassPaths = map[string]struct{}{ + "/v0/healthz": {}, // kubelet readiness probe + "/v0/startupz": {}, // kubelet startup probe + "/v0/livez": {}, // kubelet liveness probe + "/v0/metrics": {}, // Prometheus scrape +} + +// BypassPaths returns the set of paths exempt from the X-Remote-User +// check, sorted, so serve.go can log them at startup and the +// controller-side PR can keep --allow-paths in sync. +func BypassPaths() []string { + out := make([]string, 0, len(bypassPaths)) + for p := range bypassPaths { + out = append(out, p) + } + sort.Strings(out) + return out +} + +// authnRejections counts 401s from the trust-header check. Tagged by +// reason so a misconfigured proxy (duplicate-header) is grep-able +// apart from genuine missing-header attempts. +var authnRejections = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "seictl_sidecar_authn_rejections_total", + Help: "Count of 401 responses from the trusted-header middleware, by reason.", + }, + []string{"reason"}, +) + +func init() { + prometheus.MustRegister(authnRejections) +} + +// AuthnMode reads SEI_SIDECAR_AUTHN_MODE and returns the canonical +// value. Strict: an unrecognized non-empty value is an error, so a +// typo (e.g. "trusted_header" with underscore) cannot silently +// degrade a hardened deployment to wide-open :7777. +func AuthnMode() (string, error) { + raw := strings.ToLower(strings.TrimSpace(os.Getenv("SEI_SIDECAR_AUTHN_MODE"))) + switch raw { + case "", "unauthenticated": + return AuthnModeUnauthenticated, nil + case AuthnModeTrustedHeader: + return AuthnModeTrustedHeader, nil + default: + return "", fmt.Errorf("SEI_SIDECAR_AUTHN_MODE=%q is not recognized (allowed: \"\", \"unauthenticated\", %q)", raw, AuthnModeTrustedHeader) + } +} + +// BindAddress returns the listen address for the given mode. The +// loopback bind in trusted-header mode is load-bearing — it confines +// the listen socket to the pod's network namespace so the only path +// to :7777 is through the in-pod proxy. +func BindAddress(port, mode string) string { + if mode == AuthnModeTrustedHeader { + return "127.0.0.1:" + port + } + return ":" + port +} + +// trustedHeaderMiddleware enforces X-Remote-User on every path +// outside bypassPaths. The header check requires EXACTLY one +// non-empty value: a misconfigured proxy that appends instead of +// overwriting would let an attacker-supplied value arrive first, so +// len != 1 fails closed rather than silently trusting the first. +func trustedHeaderMiddleware(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if _, ok := bypassPaths[r.URL.Path]; ok { + next.ServeHTTP(w, r) + return + } + switch vals := r.Header.Values(remoteUserHeader); { + case len(vals) == 0: + authnRejections.WithLabelValues("missing_header").Inc() + writeError(w, http.StatusUnauthorized, "missing "+remoteUserHeader+" header") + case len(vals) > 1: + authnRejections.WithLabelValues("duplicate_header").Inc() + writeError(w, http.StatusUnauthorized, "expected exactly one "+remoteUserHeader+" header — proxy must overwrite, not append") + case vals[0] == "": + authnRejections.WithLabelValues("empty_header").Inc() + writeError(w, http.StatusUnauthorized, "empty "+remoteUserHeader+" header") + default: + next.ServeHTTP(w, r) + } + }) +} diff --git a/sidecar/server/auth_test.go b/sidecar/server/auth_test.go new file mode 100644 index 00000000..121629b2 --- /dev/null +++ b/sidecar/server/auth_test.go @@ -0,0 +1,145 @@ +package server + +import ( + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" +) + +func TestAuthnMode(t *testing.T) { + cases := []struct { + name string + env string + want string + wantErr string + }{ + {"unset", "", AuthnModeUnauthenticated, ""}, + {"explicit unauthenticated", "unauthenticated", AuthnModeUnauthenticated, ""}, + {"trusted-header", "trusted-header", AuthnModeTrustedHeader, ""}, + {"case-insensitive", "TRUSTED-HEADER", AuthnModeTrustedHeader, ""}, + {"whitespace tolerated", " trusted-header ", AuthnModeTrustedHeader, ""}, + {"typo with underscore", "trusted_header", "", "not recognized"}, + {"missing hyphen", "trustedheader", "", "not recognized"}, + {"future mode not yet supported", "mtls", "", "not recognized"}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + t.Setenv("SEI_SIDECAR_AUTHN_MODE", c.env) + got, err := AuthnMode() + if c.wantErr != "" { + if err == nil { + t.Fatalf("want err containing %q, got nil (mode=%q)", c.wantErr, got) + } + if !strings.Contains(err.Error(), c.wantErr) { + t.Fatalf("err = %q, want substring %q", err.Error(), c.wantErr) + } + return + } + if err != nil { + t.Fatalf("unexpected err: %v", err) + } + if got != c.want { + t.Errorf("AuthnMode() = %q, want %q", got, c.want) + } + }) + } +} + +func TestBindAddress(t *testing.T) { + if got := BindAddress("7777", AuthnModeUnauthenticated); got != ":7777" { + t.Errorf("unauthenticated: %q, want :7777", got) + } + if got := BindAddress("7777", AuthnModeTrustedHeader); got != "127.0.0.1:7777" { + t.Errorf("trusted-header: %q, want 127.0.0.1:7777", got) + } +} + +func TestTrustedHeaderMiddleware(t *testing.T) { + called := false + inner := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + called = true + w.WriteHeader(http.StatusOK) + }) + wrapped := trustedHeaderMiddleware(inner) + + cases := []struct { + name string + path string + headers map[string][]string + want int + called bool + }{ + {"healthz bypasses (readiness probe)", "/v0/healthz", nil, http.StatusOK, true}, + {"startupz bypasses (startup probe)", "/v0/startupz", nil, http.StatusOK, true}, + {"livez bypasses (liveness probe)", "/v0/livez", nil, http.StatusOK, true}, + {"metrics bypasses (Prometheus scrape)", "/v0/metrics", nil, http.StatusOK, true}, + {"node-id requires auth", "/v0/node-id", nil, http.StatusUnauthorized, false}, + {"missing header on tasks", "/v0/tasks", nil, http.StatusUnauthorized, false}, + {"empty header on tasks", "/v0/tasks", map[string][]string{"X-Remote-User": {""}}, http.StatusUnauthorized, false}, + // Defense against an APPEND-misconfigured proxy: an + // attacker-supplied value would arrive first. + {"duplicate header rejected", "/v0/tasks", map[string][]string{"X-Remote-User": {"attacker", "system:serviceaccount:platform:bot"}}, http.StatusUnauthorized, false}, + {"single header passes", "/v0/tasks", map[string][]string{"X-Remote-User": {"system:serviceaccount:platform:bot"}}, http.StatusOK, true}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + called = false + req := httptest.NewRequest(http.MethodGet, c.path, nil) + req.Header = http.Header(c.headers) + rr := httptest.NewRecorder() + wrapped.ServeHTTP(rr, req) + if rr.Code != c.want { + body, _ := io.ReadAll(rr.Body) + t.Fatalf("status = %d (body: %s), want %d", rr.Code, body, c.want) + } + if called != c.called { + t.Fatalf("inner called = %v, want %v", called, c.called) + } + }) + } +} + +func TestNewServerAppliesMiddlewareInTrustedHeaderMode(t *testing.T) { + s := NewServer(":0", nil, t.TempDir(), AuthnModeTrustedHeader) + + t.Run("healthz bypasses", func(t *testing.T) { + defer func() { _ = recover() }() // handleHealthz panics on nil engine; the auth gate is what we're testing. + req := httptest.NewRequest(http.MethodGet, "/v0/healthz", nil) + rr := httptest.NewRecorder() + s.handler.ServeHTTP(rr, req) + if rr.Code == http.StatusUnauthorized { + t.Fatalf("healthz hit 401 unexpectedly: %s", rr.Body.String()) + } + }) + + t.Run("status without header 401s", func(t *testing.T) { + req := httptest.NewRequest(http.MethodGet, "/v0/status", nil) + rr := httptest.NewRecorder() + s.handler.ServeHTTP(rr, req) + if rr.Code != http.StatusUnauthorized { + t.Fatalf("status = %d, want 401", rr.Code) + } + }) +} + +func TestNewServerSkipsMiddlewareInUnauthenticatedMode(t *testing.T) { + s := NewServer(":0", nil, t.TempDir(), AuthnModeUnauthenticated) + if s.handler != s.mux { + t.Fatal("unauthenticated mode wrapped the mux in middleware") + } +} + +func TestBypassPaths(t *testing.T) { + got := BypassPaths() + want := []string{"/v0/healthz", "/v0/livez", "/v0/metrics", "/v0/startupz"} + if len(got) != len(want) { + t.Fatalf("got %d paths, want %d: %v", len(got), len(want), got) + } + for i, p := range want { + if got[i] != p { + t.Errorf("path %d: %q, want %q", i, got[i], p) + } + } +} diff --git a/sidecar/server/keyring.go b/sidecar/server/keyring.go new file mode 100644 index 00000000..92257b05 --- /dev/null +++ b/sidecar/server/keyring.go @@ -0,0 +1,108 @@ +package server + +import ( + "errors" + "fmt" + "io" + "path/filepath" + "strings" + "time" + + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/keyring" + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" +) + +// Values accepted by SEI_KEYRING_BACKEND. Aliased so the env-contract +// surface lives in one place. +const ( + BackendTest = keyring.BackendTest + BackendFile = keyring.BackendFile + BackendOS = keyring.BackendOS +) + +// AllowedBackends is the narrow set supported today; KMS / Vault are deferred. +var AllowedBackends = []string{BackendTest, BackendFile, BackendOS} + +// Smoke-test retry window absorbs the kubelet Secret-mount race; +// beyond this the keyring is genuinely broken and the pod fails fast. +const ( + smokeTestAttempts = 3 + smokeTestBackoff = 2 * time.Second +) + +// Override hook for tests so failure paths don't wait the full 6 seconds. +var smokeTestBackoffTestHook = smokeTestBackoff + +// OpenKeyring constructs a Cosmos SDK keyring for the given backend. +// For file backend, the passphrase is fed twice because the underlying +// 99designs/keyring asks for it twice on key-creation paths. +// Caller is responsible for unsetting SEI_KEYRING_PASSPHRASE post-return. +func OpenKeyring(backend, dir, passphrase string) (keyring.Keyring, error) { + var input io.Reader + rootDir := dir + switch backend { + case BackendTest, BackendOS: + // rootDir honored as-is; no passphrase prompt. + case BackendFile: + if passphrase == "" { + return nil, fmt.Errorf("keyring backend %q requires a passphrase", backend) + } + input = strings.NewReader(passphrase + "\n" + passphrase + "\n") + // SDK appends "keyring-file" internally; strip a trailing match + // so callers passing either /sei or /sei/keyring-file converge. + if filepath.Base(dir) == "keyring-file" { + rootDir = filepath.Dir(dir) + } + default: + return nil, fmt.Errorf("unsupported keyring backend %q (allowed: %s)", + backend, strings.Join(AllowedBackends, "|")) + } + + kr, err := keyring.New(sdk.KeyringServiceName(), backend, rootDir, input) + if err != nil { + // errors.New severs the chain so a typed field embedding the + // passphrase cannot resurface via a caller's %w or %v of a wrap. + return nil, errors.New("open keyring: " + redactPassphrase(err.Error(), passphrase)) + } + return kr, nil +} + +// redactPassphrase strips verbatim occurrences of the passphrase. +// Defensive: the SDK isn't known to leak, but the guard is cheap. +func redactPassphrase(s, passphrase string) string { + if passphrase == "" { + return s + } + return strings.ReplaceAll(s, passphrase, "[redacted]") +} + +// SmokeTestKeyring verifies the keyring is structurally usable. +// An empty keyring is permitted; first sign-tx surfaces missing keys. +// Panic recovery exists so the retry loop runs even if the underlying +// lib panics on a malformed config. +func SmokeTestKeyring(kr keyring.Keyring) error { + var lastErr error + for attempt := 1; attempt <= smokeTestAttempts; attempt++ { + err := smokeTestAttempt(kr) + if err == nil { + return nil + } + lastErr = err + if attempt < smokeTestAttempts { + time.Sleep(smokeTestBackoffTestHook) + } + } + return fmt.Errorf("keyring smoke test failed after %d attempts: %w", + smokeTestAttempts, lastErr) +} + +func smokeTestAttempt(kr keyring.Keyring) (err error) { + defer func() { + if r := recover(); r != nil { + err = fmt.Errorf("keyring backend panicked during smoke test: %v", r) + } + }() + // List decrypts only the index — strongest non-destructive check. + _, err = kr.List() + return err +} diff --git a/sidecar/server/keyring_test.go b/sidecar/server/keyring_test.go new file mode 100644 index 00000000..a98d0ae2 --- /dev/null +++ b/sidecar/server/keyring_test.go @@ -0,0 +1,124 @@ +package server + +import ( + "errors" + "strings" + "testing" + + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/keyring" +) + +func TestOpenKeyring_TestBackend(t *testing.T) { + kr, err := OpenKeyring(BackendTest, t.TempDir(), "") + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if kr == nil { + t.Fatal("expected non-nil keyring") + } +} + +func TestOpenKeyring_OSBackend(t *testing.T) { + // The OS backend on macOS prompts the Security framework on first + // open; we only assert the factory dispatches without rejecting the + // backend name. Actual OS-backed key storage is exercised + // out-of-band by operators. + kr, err := OpenKeyring(BackendOS, t.TempDir(), "") + if err != nil { + t.Skipf("OS backend not available in this environment: %v", err) + } + if kr == nil { + t.Fatal("expected non-nil keyring") + } +} + +func TestOpenKeyring_FileBackend(t *testing.T) { + kr, err := OpenKeyring(BackendFile, t.TempDir(), "correct-horse-battery-staple") + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if kr == nil { + t.Fatal("expected non-nil keyring") + } +} + +func TestOpenKeyring_FileBackend_EmptyPassphrase(t *testing.T) { + _, err := OpenKeyring(BackendFile, t.TempDir(), "") + if err == nil { + t.Fatal("expected error for empty passphrase") + } + if !strings.Contains(err.Error(), "passphrase") { + t.Fatalf("error should mention passphrase, got: %v", err) + } +} + +func TestOpenKeyring_UnknownBackend(t *testing.T) { + _, err := OpenKeyring("kms", t.TempDir(), "") + if err == nil { + t.Fatal("expected error for unknown backend") + } + if !strings.Contains(err.Error(), "unsupported keyring backend") { + t.Fatalf("error should name the unsupported backend, got: %v", err) + } +} + +func TestRedactPassphrase(t *testing.T) { + cases := []struct { + name string + in string + passphrase string + want string + }{ + {"empty passphrase is a no-op", "no secret here", "", "no secret here"}, + {"verbatim occurrence is replaced", "open failed: pw=hunter2", "hunter2", "open failed: pw=[redacted]"}, + {"absent passphrase is unchanged", "open failed: io error", "hunter2", "open failed: io error"}, + {"multiple occurrences are all replaced", "hunter2 then hunter2", "hunter2", "[redacted] then [redacted]"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := redactPassphrase(tc.in, tc.passphrase) + if got != tc.want { + t.Fatalf("got %q, want %q", got, tc.want) + } + }) + } +} + +func TestSmokeTestKeyring_HappyPath(t *testing.T) { + kr, err := OpenKeyring(BackendTest, t.TempDir(), "") + if err != nil { + t.Fatalf("setup: %v", err) + } + if err := SmokeTestKeyring(kr); err != nil { + t.Fatalf("smoke test failed on healthy keyring: %v", err) + } +} + +func TestSmokeTestKeyring_FailurePath(t *testing.T) { + // Shrink the retry backoff so the failure path completes in + // milliseconds instead of seconds. + prev := smokeTestBackoffTestHook + smokeTestBackoffTestHook = 0 + t.Cleanup(func() { smokeTestBackoffTestHook = prev }) + + err := SmokeTestKeyring(&brokenKeyring{}) + if err == nil { + t.Fatal("expected smoke test to fail") + } + if !strings.Contains(err.Error(), "smoke test failed") { + t.Fatalf("error should mention smoke test, got: %v", err) + } + if !strings.Contains(err.Error(), "keyring backend is sick") { + t.Fatalf("error should wrap underlying cause, got: %v", err) + } +} + +// brokenKeyring satisfies keyring.Keyring for the smoke-test failure case. +// Only List() is exercised by SmokeTestKeyring; the embedded interface +// gives us nil methods that will panic if anything else touches it, +// which would surface a refactor that broadens the smoke-test surface. +type brokenKeyring struct{ keyring.Keyring } + +func (b *brokenKeyring) List() ([]keyring.Info, error) { + return nil, errors.New("keyring backend is sick") +} diff --git a/sidecar/server/server.go b/sidecar/server/server.go new file mode 100644 index 00000000..89d66c6d --- /dev/null +++ b/sidecar/server/server.go @@ -0,0 +1,249 @@ +package server + +import ( + "context" + "crypto/sha256" + "encoding/base64" + "encoding/hex" + "encoding/json" + "errors" + "net/http" + "os" + "path/filepath" + "time" + + "github.com/prometheus/client_golang/prometheus/promhttp" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +var serverLog = seilog.NewLogger("seictl", "sidecar", "server") + +const ( + ed25519PrivKeyLen = 64 // seed (32) + public key (32) + ed25519PubKeyOffset = 32 + cometbftAddressLen = 20 // hex(SHA256(pubkey)[:20]) +) + +// Server is the HTTP API for the sidecar. +type Server struct { + addr string + homeDir string + engine *engine.Engine + mux *http.ServeMux + handler http.Handler // mux, possibly wrapped by trustedHeaderMiddleware +} + +// TaskRequest is the JSON body for POST /v0/tasks. When ID is provided, +// the engine uses it as the task's canonical identifier; otherwise a +// random UUID is generated. +type TaskRequest struct { + ID string `json:"id,omitempty"` + Type string `json:"type"` + Params map[string]any `json:"params,omitempty"` +} + +// ErrorResponse is a standard JSON error envelope. +type ErrorResponse struct { + Error string `json:"error"` +} + +// NewServer wires a Server to the engine. authnMode must come from +// AuthnMode() so the env read and validation happen once at startup. +func NewServer(addr string, eng *engine.Engine, homeDir, authnMode string) *Server { + s := &Server{ + addr: addr, + homeDir: homeDir, + engine: eng, + mux: http.NewServeMux(), + } + s.mux.HandleFunc("GET /v0/healthz", s.handleHealthz) + s.mux.HandleFunc("GET /v0/startupz", s.handleHealthz) + s.mux.HandleFunc("GET /v0/livez", s.handleLivez) + s.mux.HandleFunc("GET /v0/status", s.handleStatus) + s.mux.Handle("GET /v0/metrics", promhttp.Handler()) + s.mux.HandleFunc("GET /v0/node-id", s.handleNodeID) + s.mux.HandleFunc("POST /v0/tasks", s.handlePostTask) + s.mux.HandleFunc("GET /v0/tasks", s.handleListTasks) + s.mux.HandleFunc("GET /v0/tasks/{id}", s.handleGetTask) + s.mux.HandleFunc("DELETE /v0/tasks/{id}", s.handleDeleteTask) + + s.handler = s.mux + if authnMode == AuthnModeTrustedHeader { + s.handler = trustedHeaderMiddleware(s.mux) + } + return s +} + +func writeJSON(w http.ResponseWriter, status int, v any) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + json.NewEncoder(w).Encode(v) //nolint:errcheck // best-effort response +} + +func writeError(w http.ResponseWriter, status int, msg string) { + writeJSON(w, status, ErrorResponse{Error: msg}) +} + +func (s *Server) handleHealthz(w http.ResponseWriter, _ *http.Request) { + if !s.engine.Healthz() { + w.WriteHeader(http.StatusServiceUnavailable) + return + } + w.WriteHeader(http.StatusOK) +} + +// handleLivez follows the kube-apiserver convention: bare 200/503 +// status for kubelet probes. ?verbose=1 includes the underlying +// store error for human triage. +func (s *Server) handleLivez(w http.ResponseWriter, r *http.Request) { + err := s.engine.Livez() + if err == nil { + w.WriteHeader(http.StatusOK) + return + } + if r.URL.Query().Get("verbose") == "1" { + writeError(w, http.StatusServiceUnavailable, err.Error()) + return + } + w.WriteHeader(http.StatusServiceUnavailable) +} + +func (s *Server) handleStatus(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusOK, s.engine.Status()) +} + +func (s *Server) handlePostTask(w http.ResponseWriter, r *http.Request) { + var req TaskRequest + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "invalid request body") + return + } + if req.Type == "" { + writeError(w, http.StatusBadRequest, "type is required") + return + } + + task := engine.Task{ID: req.ID, Type: engine.TaskType(req.Type), Params: req.Params} + + id, err := s.engine.Submit(task) + if err != nil { + writeError(w, http.StatusBadRequest, err.Error()) + return + } + writeJSON(w, http.StatusCreated, map[string]string{"id": id}) +} + +func (s *Server) handleListTasks(w http.ResponseWriter, _ *http.Request) { + results := s.engine.RecentResults() + if results == nil { + results = []engine.TaskResult{} + } + writeJSON(w, http.StatusOK, results) +} + +func (s *Server) handleGetTask(w http.ResponseWriter, r *http.Request) { + id := r.PathValue("id") + if id == "" { + writeError(w, http.StatusBadRequest, "missing task ID") + return + } + result := s.engine.GetResult(id) + if result == nil { + writeError(w, http.StatusNotFound, "task not found") + return + } + writeJSON(w, http.StatusOK, result) +} + +func (s *Server) handleDeleteTask(w http.ResponseWriter, r *http.Request) { + id := r.PathValue("id") + if id == "" { + writeError(w, http.StatusBadRequest, "missing task ID") + return + } + deleted, err := s.engine.RemoveResult(id) + if err != nil { + // A failed delete leaves the row recoverable (RemoveResult cancels the + // work but keeps the row and its registry entry), so the store hiccup is + // transient and the DELETE is safe to retry. 503 + Retry-After signals + // that to automated clients, which treat a bare 500 as fatal. + w.Header().Set("Retry-After", "1") + writeError(w, http.StatusServiceUnavailable, "failed to delete task; retry") + return + } + if !deleted { + writeError(w, http.StatusNotFound, "task not found") + return + } + w.WriteHeader(http.StatusNoContent) +} + +// handleNodeID reads node_key.json from the home directory and returns the +// Tendermint node ID. The node ID is hex(SHA256(ed25519_pubkey)[:20]), matching +// CometBFT's p2p.PubKeyToID derivation. +func (s *Server) handleNodeID(w http.ResponseWriter, _ *http.Request) { + keyPath := filepath.Join(s.homeDir, "config", "node_key.json") + data, err := os.ReadFile(keyPath) + if err != nil { + writeError(w, http.StatusInternalServerError, + "node_key.json not found — generate-identity may not have run yet") + return + } + + var keyFile struct { + PrivKey struct { + Value string `json:"value"` + } `json:"priv_key"` + } + if err := json.Unmarshal(data, &keyFile); err != nil { + writeError(w, http.StatusInternalServerError, "failed to parse node_key.json") + return + } + + keyBytes, err := base64.StdEncoding.DecodeString(keyFile.PrivKey.Value) + if err != nil || len(keyBytes) != ed25519PrivKeyLen { + writeError(w, http.StatusInternalServerError, "invalid Ed25519 key in node_key.json") + return + } + + pubKey := keyBytes[ed25519PubKeyOffset:] + hash := sha256.Sum256(pubKey) + nodeID := hex.EncodeToString(hash[:cometbftAddressLen]) + + writeJSON(w, http.StatusOK, map[string]string{"nodeId": nodeID}) +} + +// ListenAndServe starts the HTTP server and blocks until ctx is cancelled. +func (s *Server) ListenAndServe(ctx context.Context) error { + srv := &http.Server{ + Addr: s.addr, + Handler: s.handler, + ReadHeaderTimeout: 5 * time.Second, + ReadTimeout: 10 * time.Second, + WriteTimeout: 30 * time.Second, + IdleTimeout: 60 * time.Second, + // Cap impersonation-header amplification in trusted-header + // mode (proxy injects X-Remote-User / Group / Extra-*). + // Go's default is 1MB. + MaxHeaderBytes: 32 * 1024, + } + + go func() { + <-ctx.Done() + // 25s leaves a 5s buffer under K8s's default 30s + // terminationGracePeriodSeconds. + shutdownCtx, cancel := context.WithTimeout(context.Background(), 25*time.Second) + defer cancel() + if err := srv.Shutdown(shutdownCtx); err != nil { + serverLog.Warn("graceful shutdown failed", "err", err) + } + }() + + if err := srv.ListenAndServe(); !errors.Is(err, http.ErrServerClosed) { + return err + } + return nil +} diff --git a/sidecar/server/server_test.go b/sidecar/server/server_test.go new file mode 100644 index 00000000..90b41ed2 --- /dev/null +++ b/sidecar/server/server_test.go @@ -0,0 +1,519 @@ +package server + +import ( + "context" + "crypto/ed25519" + "crypto/sha256" + "encoding/base64" + "encoding/hex" + "encoding/json" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +func newTestEngine(t *testing.T, handlers map[engine.TaskType]engine.TaskHandler) *engine.Engine { + t.Helper() + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + store, err := engine.NewMemoryStore() + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { store.Close() }) + return engine.NewEngine(ctx, handlers, store) +} + +func serveHTTP(srv *Server, method, path string, body string) *httptest.ResponseRecorder { + var req *http.Request + if body != "" { + req = httptest.NewRequest(method, path, strings.NewReader(body)) + } else { + req = httptest.NewRequest(method, path, nil) + } + rec := httptest.NewRecorder() + srv.mux.ServeHTTP(rec, req) + return rec +} + +func waitForReady(eng *engine.Engine) { + deadline := time.Now().Add(2 * time.Second) + for time.Now().Before(deadline) { + if eng.Healthz() { + return + } + time.Sleep(5 * time.Millisecond) + } +} + +func waitForTaskResult(eng *engine.Engine, id string) *engine.TaskResult { + deadline := time.Now().Add(2 * time.Second) + for time.Now().Before(deadline) { + if r := eng.GetResult(id); r != nil && r.CompletedAt != nil { + return r + } + time.Sleep(5 * time.Millisecond) + } + return nil +} + +func TestLivezReturns200WhenStoreHealthy(t *testing.T) { + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + rec := serveHTTP(srv, http.MethodGet, "/v0/livez", "") + + if rec.Code != http.StatusOK { + t.Fatalf("expected 200, got %d", rec.Code) + } +} + +func TestLivezReturns200BeforeReady(t *testing.T) { + // Livez should pass even before mark-ready (healthz would return 503). + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + if eng.Healthz() { + t.Fatal("expected healthz=false before mark-ready") + } + rec := serveHTTP(srv, http.MethodGet, "/v0/livez", "") + if rec.Code != http.StatusOK { + t.Fatalf("expected livez=200 even before mark-ready, got %d", rec.Code) + } +} + +func TestLivezReturns503WhenStoreDown(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + store, err := engine.NewMemoryStore() + if err != nil { + t.Fatal(err) + } + eng := engine.NewEngine(ctx, nil, store) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + // Close the backing store to simulate SQLite failure. + store.Close() + + rec := serveHTTP(srv, http.MethodGet, "/v0/livez", "") + if rec.Code != http.StatusServiceUnavailable { + t.Fatalf("expected 503 when store is down, got %d", rec.Code) + } +} + +func TestHealthzReturns503BeforeReady(t *testing.T) { + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + rec := serveHTTP(srv, http.MethodGet, "/v0/healthz", "") + + if rec.Code != http.StatusServiceUnavailable { + t.Fatalf("expected 503, got %d", rec.Code) + } +} + +func TestHealthzReturns200AfterMarkReady(t *testing.T) { + eng := newTestEngine(t, map[engine.TaskType]engine.TaskHandler{ + engine.TaskMarkReady: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + _, _ = eng.Submit(engine.Task{Type: engine.TaskMarkReady}) + waitForReady(eng) + + rec := serveHTTP(srv, http.MethodGet, "/v0/healthz", "") + if rec.Code != http.StatusOK { + t.Fatalf("expected 200, got %d", rec.Code) + } +} + +func TestStatusResponse(t *testing.T) { + eng := newTestEngine(t, map[engine.TaskType]engine.TaskHandler{ + engine.TaskMarkReady: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + rec := serveHTTP(srv, http.MethodGet, "/v0/status", "") + if rec.Code != http.StatusOK { + t.Fatalf("expected 200, got %d", rec.Code) + } + + var resp engine.StatusResponse + if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil { + t.Fatalf("failed to decode: %v", err) + } + if resp.Status != "Initializing" { + t.Fatalf("expected Initializing initially, got %q", resp.Status) + } + + _, _ = eng.Submit(engine.Task{Type: engine.TaskMarkReady}) + waitForReady(eng) + + rec = serveHTTP(srv, http.MethodGet, "/v0/status", "") + if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil { + t.Fatalf("failed to decode: %v", err) + } + if resp.Status != "Ready" { + t.Fatalf("expected Ready after mark-ready, got %q", resp.Status) + } +} + +func TestPostTaskReturnsID(t *testing.T) { + eng := newTestEngine(t, map[engine.TaskType]engine.TaskHandler{ + engine.TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + body := `{"type":"config-patch","params":{"peers":["a@1.2.3.4:26656"]}}` + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", body) + if rec.Code != http.StatusCreated { + t.Fatalf("expected 201, got %d", rec.Code) + } + + var resp map[string]string + if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil { + t.Fatalf("failed to decode: %v", err) + } + if resp["id"] == "" { + t.Fatal("expected non-empty id") + } +} + +func TestPostTaskWithCallerID(t *testing.T) { + eng := newTestEngine(t, map[engine.TaskType]engine.TaskHandler{ + engine.TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + body := `{"id":"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee","type":"config-patch"}` + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", body) + if rec.Code != http.StatusCreated { + t.Fatalf("expected 201, got %d: %s", rec.Code, rec.Body.String()) + } + + var resp map[string]string + if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil { + t.Fatalf("failed to decode: %v", err) + } + if resp["id"] != "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" { + t.Fatalf("expected caller-provided ID, got %q", resp["id"]) + } +} + +func TestPostTaskDedupReturnsExistingID(t *testing.T) { + started := make(chan struct{}) + blocked := make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + store, _ := engine.NewMemoryStore() + t.Cleanup(func() { store.Close() }) + eng := engine.NewEngine(ctx, map[engine.TaskType]engine.TaskHandler{ + engine.TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + close(started) + <-blocked + return nil, nil + }, + }, store) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + body := `{"id":"ffffffff-1111-2222-3333-444444444444","type":"config-patch"}` + rec1 := serveHTTP(srv, http.MethodPost, "/v0/tasks", body) + if rec1.Code != http.StatusCreated { + t.Fatalf("first submit: expected 201, got %d: %s", rec1.Code, rec1.Body.String()) + } + + select { + case <-started: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for task to start") + } + + rec2 := serveHTTP(srv, http.MethodPost, "/v0/tasks", body) + if rec2.Code != http.StatusCreated { + t.Fatalf("second submit: expected 201, got %d", rec2.Code) + } + + var resp1, resp2 map[string]string + _ = json.NewDecoder(rec1.Body).Decode(&resp1) + _ = json.NewDecoder(rec2.Body).Decode(&resp2) + if resp1["id"] != resp2["id"] { + t.Fatalf("dedup should return same ID: %q vs %q", resp1["id"], resp2["id"]) + } + + close(blocked) +} + +func TestPostTaskInvalidIDReturns400(t *testing.T) { + eng := newTestEngine(t, map[engine.TaskType]engine.TaskHandler{ + engine.TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + body := `{"id":"not-a-valid-uuid","type":"config-patch"}` + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", body) + if rec.Code != http.StatusBadRequest { + t.Fatalf("expected 400, got %d: %s", rec.Code, rec.Body.String()) + } + + var resp ErrorResponse + if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil { + t.Fatalf("failed to decode error: %v", err) + } + if resp.Error == "" { + t.Fatal("expected non-empty error message") + } +} + +func TestPostTaskInvalidJSON(t *testing.T) { + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", `{not json}`) + if rec.Code != http.StatusBadRequest { + t.Fatalf("expected 400, got %d", rec.Code) + } +} + +func TestPostTaskMissingType(t *testing.T) { + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", `{"params":{}}`) + if rec.Code != http.StatusBadRequest { + t.Fatalf("expected 400, got %d", rec.Code) + } +} + +func TestPostTaskUnknownType(t *testing.T) { + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", `{"type":"nonexistent"}`) + if rec.Code != http.StatusBadRequest { + t.Fatalf("expected 400, got %d", rec.Code) + } +} + +func TestListTasksEmpty(t *testing.T) { + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + rec := serveHTTP(srv, http.MethodGet, "/v0/tasks", "") + + var results []engine.TaskResult + if err := json.NewDecoder(rec.Body).Decode(&results); err != nil { + t.Fatalf("failed to decode: %v", err) + } + if len(results) != 0 { + t.Fatalf("expected empty array, got %d", len(results)) + } +} + +func TestListTasksAfterSubmit(t *testing.T) { + eng := newTestEngine(t, map[engine.TaskType]engine.TaskHandler{ + engine.TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", `{"type":"config-patch"}`) + var resp map[string]string + _ = json.NewDecoder(rec.Body).Decode(&resp) + waitForTaskResult(eng, resp["id"]) + + rec = serveHTTP(srv, http.MethodGet, "/v0/tasks", "") + var results []engine.TaskResult + if err := json.NewDecoder(rec.Body).Decode(&results); err != nil { + t.Fatalf("failed to decode: %v", err) + } + if len(results) != 1 { + t.Fatalf("expected 1 result, got %d", len(results)) + } +} + +func TestGetTask(t *testing.T) { + eng := newTestEngine(t, map[engine.TaskType]engine.TaskHandler{ + engine.TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", `{"type":"config-patch"}`) + var resp map[string]string + _ = json.NewDecoder(rec.Body).Decode(&resp) + id := resp["id"] + waitForTaskResult(eng, id) + + rec = serveHTTP(srv, http.MethodGet, "/v0/tasks/"+id, "") + if rec.Code != http.StatusOK { + t.Fatalf("expected 200, got %d", rec.Code) + } + + var result engine.TaskResult + _ = json.NewDecoder(rec.Body).Decode(&result) + if result.ID != id { + t.Fatalf("expected ID %q, got %q", id, result.ID) + } +} + +func TestGetTaskInProgress(t *testing.T) { + started := make(chan struct{}) + blocked := make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + store, _ := engine.NewMemoryStore() + t.Cleanup(func() { store.Close() }) + eng := engine.NewEngine(ctx, map[engine.TaskType]engine.TaskHandler{ + engine.TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { + close(started) + <-blocked + return nil, nil + }, + }, store) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", `{"type":"config-patch"}`) + if rec.Code != http.StatusCreated { + t.Fatalf("expected 201, got %d", rec.Code) + } + var resp map[string]string + _ = json.NewDecoder(rec.Body).Decode(&resp) + id := resp["id"] + + select { + case <-started: + case <-time.After(2 * time.Second): + t.Fatal("timed out waiting for task to start") + } + + rec = serveHTTP(srv, http.MethodGet, "/v0/tasks/"+id, "") + if rec.Code != http.StatusOK { + t.Fatalf("expected 200 for in-progress task, got %d", rec.Code) + } + + var result engine.TaskResult + _ = json.NewDecoder(rec.Body).Decode(&result) + if result.Status != engine.TaskStatusRunning { + t.Fatalf("expected status %q, got %q", engine.TaskStatusRunning, result.Status) + } + if result.CompletedAt != nil { + t.Fatal("expected CompletedAt to be nil for running task") + } + + close(blocked) + waitForTaskResult(eng, id) + + rec = serveHTTP(srv, http.MethodGet, "/v0/tasks/"+id, "") + if rec.Code != http.StatusOK { + t.Fatalf("expected 200 for completed task, got %d", rec.Code) + } + _ = json.NewDecoder(rec.Body).Decode(&result) + if result.Status != engine.TaskStatusCompleted { + t.Fatalf("expected status %q after completion, got %q", engine.TaskStatusCompleted, result.Status) + } +} + +func TestGetTaskNotFound(t *testing.T) { + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + rec := serveHTTP(srv, http.MethodGet, "/v0/tasks/nonexistent", "") + if rec.Code != http.StatusNotFound { + t.Fatalf("expected 404, got %d", rec.Code) + } +} + +func TestDeleteTask(t *testing.T) { + eng := newTestEngine(t, map[engine.TaskType]engine.TaskHandler{ + engine.TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, + }) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + rec := serveHTTP(srv, http.MethodPost, "/v0/tasks", `{"type":"config-patch"}`) + var resp map[string]string + _ = json.NewDecoder(rec.Body).Decode(&resp) + id := resp["id"] + waitForTaskResult(eng, id) + + rec = serveHTTP(srv, http.MethodDelete, "/v0/tasks/"+id, "") + if rec.Code != http.StatusNoContent { + t.Fatalf("expected 204, got %d", rec.Code) + } + + rec = serveHTTP(srv, http.MethodDelete, "/v0/tasks/"+id, "") + if rec.Code != http.StatusNotFound { + t.Fatalf("expected 404 after delete, got %d", rec.Code) + } +} + +// writeTestNodeKey generates a real Ed25519 keypair, writes it as a CometBFT +// node_key.json, and returns the expected node ID (hex(SHA256(pubkey)[:20])). +func writeTestNodeKey(t *testing.T, homeDir string) string { + t.Helper() + + pub, priv, err := ed25519.GenerateKey(nil) + if err != nil { + t.Fatal(err) + } + + // CometBFT stores the full 64-byte key (seed || pubkey) base64-encoded. + keyFile := struct { + PrivKey struct { + Type string `json:"type"` + Value string `json:"value"` + } `json:"priv_key"` + }{ + PrivKey: struct { + Type string `json:"type"` + Value string `json:"value"` + }{ + Type: "tendermint/PrivKeyEd25519", + Value: base64.StdEncoding.EncodeToString(priv), + }, + } + data, err := json.Marshal(keyFile) + if err != nil { + t.Fatal(err) + } + + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(configDir, "node_key.json"), data, 0o644); err != nil { + t.Fatal(err) + } + + hash := sha256.Sum256(pub) + return hex.EncodeToString(hash[:20]) +} + +func TestNodeID_ReturnsCorrectID(t *testing.T) { + homeDir := t.TempDir() + want := writeTestNodeKey(t, homeDir) + + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, homeDir, AuthnModeUnauthenticated) + + rec := serveHTTP(srv, http.MethodGet, "/v0/node-id", "") + if rec.Code != http.StatusOK { + t.Fatalf("expected 200, got %d: %s", rec.Code, rec.Body.String()) + } + + var result struct { + NodeID string `json:"nodeId"` + } + if err := json.Unmarshal(rec.Body.Bytes(), &result); err != nil { + t.Fatalf("invalid JSON: %v", err) + } + if result.NodeID != want { + t.Errorf("nodeId = %q, want %q", result.NodeID, want) + } +} + +func TestNodeID_MissingKeyFile(t *testing.T) { + eng := newTestEngine(t, nil) + srv := NewServer(":0", eng, t.TempDir(), AuthnModeUnauthenticated) + + rec := serveHTTP(srv, http.MethodGet, "/v0/node-id", "") + if rec.Code != http.StatusInternalServerError { + t.Fatalf("expected 500, got %d", rec.Code) + } +} diff --git a/sidecar/shadow/close_test.go b/sidecar/shadow/close_test.go new file mode 100644 index 00000000..33d9e2be --- /dev/null +++ b/sidecar/shadow/close_test.go @@ -0,0 +1,50 @@ +package shadow + +import "testing" + +// closeableState is a StateReader with a no-return Close() (the shape +// *ethclient.Client uses), recording whether Close was called. +type closeableState struct { + *mockState + closed *bool +} + +func (c closeableState) Close() { *c.closed = true } + +// closeableKeySource is a KeySource with a no-return Close(). +type closeableKeySource struct { + mockKeySource + closed *bool +} + +func (c closeableKeySource) Close() { *c.closed = true } + +// Comparator.Close must close the configured readers/key source. *ethclient.Client +// and *rpc.Client expose Close() with no return (not io.Closer), so this guards +// against the regression where an io.Closer assertion silently skipped them. +func TestComparator_CloseClosesReaders(t *testing.T) { + var shadowClosed, canonClosed, ksClosed bool + shadow := closeableState{mockState: newMockState(), closed: &shadowClosed} + canon := closeableState{mockState: newMockState(), closed: &canonClosed} + ks := closeableKeySource{closed: &ksClosed} + + comp := NewComparator("http://shadow", "http://canon", WithLayer2(shadow, canon, ks)) + comp.Close() + + if !shadowClosed || !canonClosed { + t.Errorf("state readers not closed: shadow=%v canon=%v", shadowClosed, canonClosed) + } + if !ksClosed { + t.Error("key source not closed") + } +} + +// Close must be safe when Layer 2 was never configured (nil readers). +func TestComparator_CloseNoLayer2(t *testing.T) { + comp := NewComparator("http://shadow", "http://canon") + comp.Close() // must not panic +} + +// compile-time guard: TraceKeySource matches the no-return Close() shape that +// Comparator.Close asserts (the same shape *ethclient.Client / *rpc.Client use). +var _ interface{ Close() } = (*TraceKeySource)(nil) diff --git a/sidecar/shadow/comparator.go b/sidecar/shadow/comparator.go new file mode 100644 index 00000000..6390bff5 --- /dev/null +++ b/sidecar/shadow/comparator.go @@ -0,0 +1,197 @@ +package shadow + +import ( + "context" + "fmt" + "time" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +var log = seilog.NewLogger("seictl", "shadow") + +// layer2Timeout bounds the per-block Layer 2 RPC fan-out (touched-key trace plus +// state reads on both chains) so one slow endpoint cannot stall the compare loop. +const layer2Timeout = 30 * time.Second + +// Comparator performs block-by-block comparison between a shadow node and +// a canonical chain node via their RPC endpoints. +type Comparator struct { + shadowClient *rpc.Client + canonicalClient *rpc.Client + + // migrationMode tunes the verdict for an AppHash-breaking migration shadow + // (e.g. memiavl->flatkv): the shadow's AppHash diverges from canonical by + // design every block, so AppHash mismatch is expected and informational. + // The correctness signals become LastResultsHash + gas + per-tx receipts + // (execution equivalence), and Layer 1 always runs. + migrationMode bool + + // Layer 2 (logical state diff) runs only when all three are configured. + shadowState StateReader + canonicalState StateReader + keySource KeySource +} + +// Option configures a Comparator. +type Option func(*Comparator) + +// WithMigrationMode treats AppHash divergence as expected (not a mismatch) and +// keys the verdict on execution-results equivalence. Use for a shadow running +// an AppHash-breaking state migration against an un-migrated canonical chain. +func WithMigrationMode() Option { + return func(c *Comparator) { c.migrationMode = true } +} + +// WithLayer2 enables logical state-diff comparison: the keySource yields the +// accounts/slots each block touched, and the two StateReaders (EVM RPC on the +// shadow and canonical chains) supply their logical values to compare. +func WithLayer2(shadowState, canonicalState StateReader, keySource KeySource) Option { + return func(c *Comparator) { + c.shadowState = shadowState + c.canonicalState = canonicalState + c.keySource = keySource + } +} + +// NewComparator creates a Comparator that queries shadowRPC for the local +// shadow node and canonicalRPC for the reference chain. +func NewComparator(shadowRPC, canonicalRPC string, opts ...Option) *Comparator { + c := &Comparator{ + shadowClient: rpc.NewClient(shadowRPC, nil), + canonicalClient: rpc.NewClient(canonicalRPC, nil), + } + for _, opt := range opts { + opt(c) + } + return c +} + +// CompareBlock performs a layered comparison for the given block height. +// Layer 0 (block headers) always runs. Layer 1 (transaction receipts) runs when +// a real divergence is detected, and always in migration mode — where AppHash, +// the cheap Layer 0 signal, is expected to differ, so the receipt check is the +// real correctness signal. +func (c *Comparator) CompareBlock(ctx context.Context, height int64) (*CompareResult, error) { + result := &CompareResult{ + Height: height, + Timestamp: time.Now().UTC().Format(time.RFC3339), + Match: true, + MigrationMode: c.migrationMode, + } + + // --- Layer 0: block header comparison --- + l0, err := c.compareLayer0(ctx, height) + if err != nil { + return nil, err + } + result.Layer0 = *l0 + + // In migration mode AppHash mismatch is expected; a real Layer 0 divergence + // is a LastResultsHash mismatch (execution results differ). Per-tx gas is + // compared in Layer 1; Layer 0 GasUsedMatch is not yet wired (stubbed true), + // so it is deliberately not part of this verdict. Otherwise any Layer 0 field + // mismatch (including AppHash) counts. + // Note: LastResultsHash at height N reflects N-1 execution; the per-tx Layer 1 + // signal lands on the correct height, so attribution stays accurate. + realL0Divergence := !l0.Match() + if c.migrationMode { + realL0Divergence = !l0.LastResultsHashMatch + } + + // --- Layer 1: transaction receipt comparison --- + if realL0Divergence || c.migrationMode { + l1, err := c.compareLayer1(ctx, height) + if err != nil { + // In migration mode Layer 1 is load-bearing (AppHash is expected to + // differ), so an error must fail closed, not silently pass. Outside + // migration mode Layer 1 only runs after a confirmed Layer 0 + // divergence, so the block is already not clean and the error is + // merely missing detail. + log.Warn("layer 1 comparison failed", "height", height, "err", err) + if c.migrationMode { + result.Layer1 = &Layer1Result{Indeterminate: true, Error: err.Error()} + } + } else { + result.Layer1 = l1 + } + } + l1Diverged := result.Layer1 != nil && len(result.Layer1.Divergences) > 0 + l1Indeterminate := result.Layer1 != nil && result.Layer1.Indeterminate + + // --- Layer 2: logical state diff (when configured) --- + if c.layer2Enabled() && (realL0Divergence || c.migrationMode) { + l2, err := c.compareLayer2(ctx, height) + if err != nil { + // Fail closed: the load-bearing check could not run, so this block is + // NOT clean. Record it as indeterminate (forces Match=false below) + // rather than silently passing on the layer that actually validates + // the migration. + log.Warn("layer 2 state comparison could not run; marking indeterminate", + "height", height, "err", err) + result.Layer2 = &Layer2Result{Indeterminate: true, Error: err.Error()} + } else { + result.Layer2 = l2 + } + } + l2Diverged := result.Layer2 != nil && len(result.Layer2.Divergences) > 0 + l2Indeterminate := result.Layer2 != nil && result.Layer2.Indeterminate + + // Attribute to the deepest (most specific) layer that fired: a Layer 1/2 + // divergence or indeterminate is more actionable than the Layer 0 header + // mismatch that triggered the descent. + switch { + case l2Diverged || l2Indeterminate: + result.Match = false + layer := 2 + result.DivergenceLayer = &layer + case l1Diverged || l1Indeterminate: + result.Match = false + layer := 1 + result.DivergenceLayer = &layer + case realL0Divergence: + result.Match = false + layer := 0 + result.DivergenceLayer = &layer + } + + return result, nil +} + +func (c *Comparator) layer2Enabled() bool { + return c.keySource != nil && c.shadowState != nil && c.canonicalState != nil +} + +// compareLayer2 fetches the accounts a block touched and compares their logical +// state (balance/code/nonce/storage) between the shadow and canonical chains. It +// bounds the per-block RPC fan-out with a timeout so one slow endpoint cannot +// stall the compare loop. +// +// The CometBFT height is used directly as the EVM block number for the trace and +// the state reads. This holds because sei-chain maps an explicit EVM block number +// straight to the tendermint height (evmrpc getBlockNumber: identity, no offset). +// If a future sei-chain reintroduces an offset, that identity breaks and this +// must translate height -> EVM block number before the EVM calls. +func (c *Comparator) compareLayer2(ctx context.Context, height int64) (*Layer2Result, error) { + ctx, cancel := context.WithTimeout(ctx, layer2Timeout) + defer cancel() + touched, err := c.keySource.TouchedAccounts(ctx, height) + if err != nil { + return nil, fmt.Errorf("resolving touched accounts at height %d: %w", height, err) + } + return compareState(ctx, height, touched, c.shadowState, c.canonicalState) +} + +// Close releases resources held by configured Layer 2 readers / key source. +// go-ethereum's *ethclient.Client and *rpc.Client expose Close() with NO return, +// so they do not satisfy io.Closer — assert the no-return shape instead, or the +// connections leak silently. +func (c *Comparator) Close() { + for _, r := range []any{c.shadowState, c.canonicalState, c.keySource} { + if cl, ok := r.(interface{ Close() }); ok { + cl.Close() + } + } +} diff --git a/sidecar/shadow/comparator_migration_test.go b/sidecar/shadow/comparator_migration_test.go new file mode 100644 index 00000000..20bf7a77 --- /dev/null +++ b/sidecar/shadow/comparator_migration_test.go @@ -0,0 +1,142 @@ +package shadow + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// In migration mode the shadow's AppHash diverges from canonical by design, so an +// AppHash-only mismatch must NOT count as a divergence; the verdict keys on +// execution-results equivalence (LastResultsHash + gas + per-tx receipts). +func TestCompareBlock_MigrationMode_AppHashExpected(t *testing.T) { + txs := []rpc.TxResult{ + {Code: 0, GasUsed: "100", GasWanted: "200", Log: "ok", Events: json.RawMessage(`[]`)}, + } + shadowSrv := rpcServer("SHADOW_APPHASH", "SAME_RESULTS", txs) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CANON_APPHASH", "SAME_RESULTS", txs) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL, WithMigrationMode()) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if !result.Match { + t.Error("expected match: AppHash divergence is expected in migration mode") + } + if !result.MigrationMode { + t.Error("expected result to record migration mode") + } + if result.Layer0.AppHashMatch { + t.Error("expected AppHash mismatch") + } + if !result.Layer0.LastResultsHashMatch { + t.Error("expected LastResultsHash match") + } + if result.Layer1 == nil { + t.Error("migration mode must always run Layer 1, even when results match") + } + if result.DivergenceLayer != nil { + t.Errorf("expected nil divergence layer, got %d", *result.DivergenceLayer) + } +} + +// In migration mode Layer 1 is load-bearing (AppHash is expected to differ). If +// the receipt comparison cannot run, the block must fail closed (indeterminate, +// attributed to layer 1) — never a silent clean pass. +func TestCompareBlock_MigrationMode_Layer1ErrorFailsClosed(t *testing.T) { + // /block returns differing AppHash but matching LastResultsHash (so Layer 0 is + // not a real divergence); /block_results errors, so Layer 1 cannot run. + handler := func(appHash string) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + switch r.URL.Path { + case "/block": + _, _ = w.Write(blockJSON(appHash, "SAME_RESULTS")) + case "/block_results": + http.Error(w, "block_results unavailable", http.StatusInternalServerError) + default: + http.NotFound(w, r) + } + } + } + shadowSrv := httptest.NewServer(handler("SHADOW_APPHASH")) + defer shadowSrv.Close() + canonicalSrv := httptest.NewServer(handler("CANON_APPHASH")) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL, WithMigrationMode()) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if result.Match { + t.Error("expected NOT clean: Layer 1 could not run in migration mode, must fail closed") + } + if result.DivergenceLayer == nil || *result.DivergenceLayer != 1 { + t.Errorf("expected divergence layer 1, got %v", result.DivergenceLayer) + } + if result.Layer1 == nil || !result.Layer1.Indeterminate { + t.Errorf("expected Layer1 marked indeterminate, got %+v", result.Layer1) + } +} + +// A LastResultsHash mismatch is a real execution divergence even in migration +// mode, attributed to Layer 0. +func TestCompareBlock_MigrationMode_ResultsDivergence(t *testing.T) { + shadowSrv := rpcServer("SHADOW_APPHASH", "SHADOW_RESULTS", nil) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CANON_APPHASH", "CANON_RESULTS", nil) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL, WithMigrationMode()) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if result.Match { + t.Error("expected divergence: LastResultsHash mismatch is a real execution divergence") + } + if result.DivergenceLayer == nil || *result.DivergenceLayer != 0 { + t.Errorf("expected divergence layer 0, got %v", result.DivergenceLayer) + } +} + +// When results hashes agree but a per-tx receipt differs, migration mode still +// catches it at Layer 1 (which always runs in migration mode). +func TestCompareBlock_MigrationMode_ReceiptDivergence(t *testing.T) { + shadowTxs := []rpc.TxResult{ + {Code: 0, GasUsed: "100", GasWanted: "200", Log: "ok", Events: json.RawMessage(`[]`)}, + } + canonicalTxs := []rpc.TxResult{ + {Code: 1, GasUsed: "150", GasWanted: "200", Log: "reverted", Events: json.RawMessage(`[]`)}, + } + shadowSrv := rpcServer("SHADOW_APPHASH", "SAME_RESULTS", shadowTxs) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CANON_APPHASH", "SAME_RESULTS", canonicalTxs) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL, WithMigrationMode()) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if result.Match { + t.Error("expected divergence: a per-tx receipt differs") + } + if result.DivergenceLayer == nil || *result.DivergenceLayer != 1 { + t.Errorf("expected divergence layer 1, got %v", result.DivergenceLayer) + } + if result.Layer1 == nil || len(result.Layer1.Divergences) != 1 { + t.Errorf("expected 1 tx divergence, got %+v", result.Layer1) + } +} diff --git a/sidecar/shadow/comparator_test.go b/sidecar/shadow/comparator_test.go new file mode 100644 index 00000000..6334b33a --- /dev/null +++ b/sidecar/shadow/comparator_test.go @@ -0,0 +1,441 @@ +package shadow + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "testing" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// blockJSON builds a minimal /block JSON-RPC response with the given header fields. +func blockJSON(appHash, lastResultsHash string) []byte { + resp := map[string]any{ + "jsonrpc": "2.0", + "id": -1, + "result": map[string]any{ + "block_id": map[string]any{"hash": ""}, + "block": map[string]any{ + "header": map[string]any{ + "app_hash": appHash, + "last_results_hash": lastResultsHash, + }, + }, + }, + } + b, _ := json.Marshal(resp) + return b +} + +// blockResultsJSON builds a minimal /block_results JSON-RPC response. +func blockResultsJSON(txs []rpc.TxResult) []byte { + resp := map[string]any{ + "jsonrpc": "2.0", + "id": -1, + "result": map[string]any{ + "txs_results": txs, + }, + } + b, _ := json.Marshal(resp) + return b +} + +// rpcServer creates an httptest server that responds to /block and /block_results. +func rpcServer(appHash, lastResultsHash string, txs []rpc.TxResult) *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case r.URL.Path == "/block": + w.Write(blockJSON(appHash, lastResultsHash)) + case r.URL.Path == "/block_results": + w.Write(blockResultsJSON(txs)) + default: + http.NotFound(w, r) + } + })) +} + +func TestCompareBlock_Match(t *testing.T) { + srv := rpcServer("AABB", "CCDD", nil) + defer srv.Close() + + comp := NewComparator(srv.URL, srv.URL) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if !result.Match { + t.Error("expected match when both endpoints return identical data") + } + if result.DivergenceLayer != nil { + t.Errorf("expected nil divergence layer, got %d", *result.DivergenceLayer) + } + if !result.Layer0.AppHashMatch { + t.Error("expected AppHash match") + } + if result.Layer1 != nil { + t.Error("expected nil Layer1 when Layer0 matches") + } +} + +func TestCompareBlock_Layer0Divergence(t *testing.T) { + shadowSrv := rpcServer("SHADOW_HASH", "RESULTS_HASH", nil) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CANONICAL_HASH", "RESULTS_HASH", nil) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if result.Match { + t.Error("expected divergence") + } + if result.DivergenceLayer == nil || *result.DivergenceLayer != 0 { + t.Errorf("expected divergence layer 0, got %v", result.DivergenceLayer) + } + if result.Layer0.AppHashMatch { + t.Error("expected AppHash mismatch") + } + if result.Layer0.ShadowAppHash != "SHADOW_HASH" { + t.Errorf("shadow app hash = %q, want SHADOW_HASH", result.Layer0.ShadowAppHash) + } + if result.Layer0.CanonicalAppHash != "CANONICAL_HASH" { + t.Errorf("canonical app hash = %q, want CANONICAL_HASH", result.Layer0.CanonicalAppHash) + } +} + +func TestCompareBlock_Layer1TxDivergence(t *testing.T) { + shadowTxs := []rpc.TxResult{ + {Code: 0, GasUsed: "100", GasWanted: "200", Log: "ok", Events: json.RawMessage(`[]`)}, + } + canonicalTxs := []rpc.TxResult{ + {Code: 1, GasUsed: "150", GasWanted: "200", Log: "reverted", Events: json.RawMessage(`[]`)}, + } + + shadowSrv := rpcServer("AAA", "BBB", shadowTxs) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CCC", "BBB", canonicalTxs) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL) + result, err := comp.CompareBlock(context.Background(), 50) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if result.Match { + t.Error("expected divergence") + } + if result.Layer1 == nil { + t.Fatal("expected Layer1 to be populated") + } + if len(result.Layer1.Divergences) != 1 { + t.Fatalf("expected 1 tx divergence, got %d", len(result.Layer1.Divergences)) + } + + div := result.Layer1.Divergences[0] + if div.TxIndex != 0 { + t.Errorf("tx index = %d, want 0", div.TxIndex) + } + + // Should have divergences for code, gasUsed, and log. + fieldNames := make(map[string]bool) + for _, f := range div.Fields { + fieldNames[f.Field] = true + } + for _, expected := range []string{"code", "gasUsed", "log"} { + if !fieldNames[expected] { + t.Errorf("expected field %q in divergences", expected) + } + } +} + +func TestCompareBlock_Layer1TxCountMismatch(t *testing.T) { + shadowTxs := []rpc.TxResult{ + {Code: 0, GasUsed: "100", GasWanted: "200"}, + {Code: 0, GasUsed: "100", GasWanted: "200"}, + } + canonicalTxs := []rpc.TxResult{ + {Code: 0, GasUsed: "100", GasWanted: "200"}, + } + + shadowSrv := rpcServer("AAA", "BBB", shadowTxs) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CCC", "BBB", canonicalTxs) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL) + result, err := comp.CompareBlock(context.Background(), 50) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if result.Layer1 == nil { + t.Fatal("expected Layer1 result") + } + if result.Layer1.TxCountMatch { + t.Error("expected tx count mismatch") + } + + // Extra tx on shadow side should be reported as "presence" divergence. + found := false + for _, d := range result.Layer1.Divergences { + for _, f := range d.Fields { + if f.Field == "presence" { + found = true + } + } + } + if !found { + t.Error("expected a presence divergence for the extra tx") + } +} + +func TestCompareResult_Diverged(t *testing.T) { + match := CompareResult{Match: true} + if match.Diverged() { + t.Error("expected Diverged()=false for matching result") + } + + mismatch := CompareResult{Match: false} + if !mismatch.Diverged() { + t.Error("expected Diverged()=true for mismatching result") + } +} + +func TestLayer0Result_Match(t *testing.T) { + cases := []struct { + name string + r Layer0Result + want bool + }{ + {"all match", Layer0Result{AppHashMatch: true, LastResultsHashMatch: true, GasUsedMatch: true}, true}, + {"app hash mismatch", Layer0Result{AppHashMatch: false, LastResultsHashMatch: true, GasUsedMatch: true}, false}, + {"results hash mismatch", Layer0Result{AppHashMatch: true, LastResultsHashMatch: false, GasUsedMatch: true}, false}, + {"gas mismatch", Layer0Result{AppHashMatch: true, LastResultsHashMatch: true, GasUsedMatch: false}, false}, + {"all mismatch", Layer0Result{}, false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := tc.r.Match(); got != tc.want { + t.Errorf("Match() = %v, want %v", got, tc.want) + } + }) + } +} + +func TestCompareBlock_AllMatch_NoLayer1(t *testing.T) { + srv := rpcServer("SAME", "SAME", nil) + defer srv.Close() + + comp := NewComparator(srv.URL, srv.URL) + result, err := comp.CompareBlock(context.Background(), 1) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + if !result.Match { + t.Error("expected match") + } + if result.Layer0.ShadowAppHash != "" { + t.Error("expected empty ShadowAppHash when matching") + } + if result.Layer0.CanonicalAppHash != "" { + t.Error("expected empty CanonicalAppHash when matching") + } + if result.Layer1 != nil { + t.Error("expected nil Layer1 when all hashes match") + } +} + +func TestCompareBlock_LastResultsHashDivergence(t *testing.T) { + shadowSrv := rpcServer("SAME_APP", "SHADOW_RES", nil) + defer shadowSrv.Close() + canonicalSrv := rpcServer("SAME_APP", "CANONICAL_RES", nil) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL) + result, err := comp.CompareBlock(context.Background(), 1) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + if result.Match { + t.Error("expected divergence on LastResultsHash mismatch") + } + if !result.Layer0.AppHashMatch { + t.Error("expected AppHash to match") + } + if result.Layer0.LastResultsHashMatch { + t.Error("expected LastResultsHash mismatch") + } + if result.Layer0.ShadowLastResultsHash != "SHADOW_RES" { + t.Errorf("ShadowLastResultsHash = %q, want SHADOW_RES", result.Layer0.ShadowLastResultsHash) + } +} + +// --- DivergenceReport tests --- + +func TestBuildDivergenceReport_CapturesBothChains(t *testing.T) { + shadowSrv := rpcServer("SHADOW_HASH", "RESULTS", nil) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CANONICAL_HASH", "RESULTS", nil) + defer canonicalSrv.Close() + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL) + comparison, err := comp.CompareBlock(context.Background(), 42) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + report, err := comp.BuildDivergenceReport(context.Background(), 42, *comparison) + if err != nil { + t.Fatalf("BuildDivergenceReport: %v", err) + } + + if report.Height != 42 { + t.Errorf("Height = %d, want 42", report.Height) + } + if report.Timestamp == "" { + t.Error("expected non-empty Timestamp") + } + if report.Comparison.Match { + t.Error("expected divergent comparison in report") + } + if len(report.Shadow.Block) == 0 { + t.Error("expected non-empty Shadow.Block") + } + if len(report.Shadow.BlockResults) == 0 { + t.Error("expected non-empty Shadow.BlockResults") + } + if len(report.Canonical.Block) == 0 { + t.Error("expected non-empty Canonical.Block") + } + if len(report.Canonical.BlockResults) == 0 { + t.Error("expected non-empty Canonical.BlockResults") + } +} + +func TestBuildDivergenceReport_RPCFailure(t *testing.T) { + goodSrv := rpcServer("AA", "BB", nil) + defer goodSrv.Close() + badSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusInternalServerError) + })) + defer badSrv.Close() + + comp := NewComparator(badSrv.URL, goodSrv.URL) + comparison := CompareResult{Height: 1, Match: false} + + _, err := comp.BuildDivergenceReport(context.Background(), 1, comparison) + if err == nil { + t.Error("expected error when shadow RPC fails during report capture") + } +} + +// --- RenderMarkdown tests --- + +func TestRenderMarkdown_DivergentReport(t *testing.T) { + layer := 0 + report := &DivergenceReport{ + Height: 198740042, + Timestamp: "2026-03-28T01:00:00Z", + Comparison: CompareResult{ + Height: 198740042, Match: false, DivergenceLayer: &layer, + Layer0: Layer0Result{ + AppHashMatch: false, LastResultsHashMatch: true, GasUsedMatch: true, + ShadowAppHash: "AABBCCDD11223344AABBCCDD11223344", CanonicalAppHash: "11223344AABBCCDD11223344AABBCCDD", + }, + Layer1: &Layer1Result{ + TotalTxs: 4, TxCountMatch: true, + Divergences: []TxDivergence{ + {TxIndex: 3, Fields: []FieldDivergence{ + {Field: "code", Shadow: float64(0), Canonical: float64(1)}, + {Field: "gasUsed", Shadow: "142000", Canonical: "154000"}, + }}, + }, + }, + }, + } + + md := RenderMarkdown(report) + + checks := []string{ + "# Divergence Report — Height 198740042", + "2026-03-28T01:00:00Z", + "AppHash", + "AABBCCDD...3344", + "11223344...CCDD", + "❌", + "LastResultsHash", + "✅", + "## Layer 1", + "Transaction 3", + "code", + "gasUsed", + "142000", + "154000", + } + + for _, want := range checks { + found := false + for i := 0; i <= len(md)-len(want); i++ { + if md[i:i+len(want)] == want { + found = true + break + } + } + if !found { + t.Errorf("markdown missing %q", want) + } + } +} + +func TestRenderMarkdown_MatchingReport(t *testing.T) { + report := &DivergenceReport{ + Height: 100, + Timestamp: "2026-03-28T00:00:00Z", + Comparison: CompareResult{ + Height: 100, Match: true, + Layer0: Layer0Result{AppHashMatch: true, LastResultsHashMatch: true, GasUsedMatch: true}, + }, + } + + md := RenderMarkdown(report) + if !contains(md, "✅") { + t.Error("expected checkmarks for matching report") + } + if contains(md, "Layer 1") { + t.Error("should not include Layer 1 section when Layer0 matches") + } +} + +func contains(s, sub string) bool { + for i := 0; i <= len(s)-len(sub); i++ { + if s[i:i+len(sub)] == sub { + return true + } + } + return false +} + +func TestCompareBlock_RPCError(t *testing.T) { + badSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusInternalServerError) + fmt.Fprint(w, "internal error") + })) + defer badSrv.Close() + + goodSrv := rpcServer("AA", "BB", nil) + defer goodSrv.Close() + + comp := NewComparator(badSrv.URL, goodSrv.URL) + _, err := comp.CompareBlock(context.Background(), 1) + if err == nil { + t.Error("expected error when shadow RPC fails") + } +} diff --git a/sidecar/shadow/fetch.go b/sidecar/shadow/fetch.go new file mode 100644 index 00000000..4a95efaa --- /dev/null +++ b/sidecar/shadow/fetch.go @@ -0,0 +1,43 @@ +package shadow + +import ( + "compress/gzip" + "context" + "encoding/json" + "fmt" + "io" + "strings" + + "github.com/aws/aws-sdk-go-v2/service/s3" + + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +// FetchReport downloads and decodes a DivergenceReport from S3. +func FetchReport(ctx context.Context, downloader seis3.Downloader, bucket, key string) (*DivergenceReport, error) { + resp, err := downloader.GetObject(ctx, &s3.GetObjectInput{ + Bucket: &bucket, + Key: &key, + }) + if err != nil { + return nil, fmt.Errorf("downloading s3://%s/%s: %w", bucket, key, err) + } + defer resp.Body.Close() + + var reader io.Reader = resp.Body + if strings.HasSuffix(key, ".gz") { + gz, err := gzip.NewReader(resp.Body) + if err != nil { + return nil, fmt.Errorf("decompressing report: %w", err) + } + defer gz.Close() + reader = gz + } + + var report DivergenceReport + if err := json.NewDecoder(reader).Decode(&report); err != nil { + return nil, fmt.Errorf("decoding report: %w", err) + } + + return &report, nil +} diff --git a/sidecar/shadow/fetch_test.go b/sidecar/shadow/fetch_test.go new file mode 100644 index 00000000..11d1375c --- /dev/null +++ b/sidecar/shadow/fetch_test.go @@ -0,0 +1,243 @@ +package shadow + +import ( + "bytes" + "compress/gzip" + "context" + "encoding/json" + "io" + "testing" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/service/s3" + s3types "github.com/aws/aws-sdk-go-v2/service/s3/types" +) + +type mockDownloader struct { + objects map[string][]byte + err error +} + +func (m *mockDownloader) GetObject(_ context.Context, input *s3.GetObjectInput, _ ...func(*s3.Options)) (*s3.GetObjectOutput, error) { + if m.err != nil { + return nil, m.err + } + key := aws.ToString(input.Key) + data, ok := m.objects[key] + if !ok { + return nil, &s3types.NoSuchKey{} + } + return &s3.GetObjectOutput{ + Body: io.NopCloser(bytes.NewReader(data)), + }, nil +} + +func gzipJSON(t *testing.T, v any) []byte { + t.Helper() + var buf bytes.Buffer + gw := gzip.NewWriter(&buf) + if err := json.NewEncoder(gw).Encode(v); err != nil { + t.Fatalf("encoding JSON: %v", err) + } + if err := gw.Close(); err != nil { + t.Fatalf("closing gzip: %v", err) + } + return buf.Bytes() +} + +func rawJSON(t *testing.T, v any) []byte { + t.Helper() + data, err := json.Marshal(v) + if err != nil { + t.Fatalf("encoding JSON: %v", err) + } + return data +} + +func TestFetchReport_GzippedReport(t *testing.T) { + report := DivergenceReport{ + Height: 198032451, + Timestamp: "2026-04-02T00:00:00Z", + Comparison: CompareResult{ + Height: 198032451, + Match: false, + Layer0: Layer0Result{ + AppHashMatch: false, + LastResultsHashMatch: true, + GasUsedMatch: true, + ShadowAppHash: "aaa", + CanonicalAppHash: "bbb", + }, + }, + } + + dl := &mockDownloader{objects: map[string][]byte{ + "shadow-results/divergence-198032451.report.json.gz": gzipJSON(t, report), + }} + + got, err := FetchReport(context.Background(), dl, "bucket", "shadow-results/divergence-198032451.report.json.gz") + if err != nil { + t.Fatalf("FetchReport: %v", err) + } + + if got.Height != 198032451 { + t.Errorf("Height = %d, want 198032451", got.Height) + } + if got.Timestamp != "2026-04-02T00:00:00Z" { + t.Errorf("Timestamp = %q, want 2026-04-02T00:00:00Z", got.Timestamp) + } + if got.Comparison.Match { + t.Error("expected divergent comparison") + } + if got.Comparison.Layer0.AppHashMatch { + t.Error("expected AppHash mismatch") + } + if got.Comparison.Layer0.ShadowAppHash != "aaa" { + t.Errorf("ShadowAppHash = %q, want aaa", got.Comparison.Layer0.ShadowAppHash) + } +} + +func TestFetchReport_UncompressedReport(t *testing.T) { + report := DivergenceReport{ + Height: 100, + Timestamp: "2026-04-02T00:00:00Z", + Comparison: CompareResult{ + Height: 100, + Match: true, + Layer0: Layer0Result{AppHashMatch: true, LastResultsHashMatch: true, GasUsedMatch: true}, + }, + } + + dl := &mockDownloader{objects: map[string][]byte{ + "reports/divergence-100.report.json": rawJSON(t, report), + }} + + got, err := FetchReport(context.Background(), dl, "bucket", "reports/divergence-100.report.json") + if err != nil { + t.Fatalf("FetchReport: %v", err) + } + if got.Height != 100 { + t.Errorf("Height = %d, want 100", got.Height) + } +} + +func TestFetchReport_S3NotFound(t *testing.T) { + dl := &mockDownloader{objects: map[string][]byte{}} + + _, err := FetchReport(context.Background(), dl, "bucket", "nonexistent-key.json.gz") + if err == nil { + t.Fatal("expected error for missing key") + } +} + +func TestFetchReport_CorruptGzip(t *testing.T) { + dl := &mockDownloader{objects: map[string][]byte{ + "corrupt.json.gz": {0x00, 0x01, 0x02, 0x03}, + }} + + _, err := FetchReport(context.Background(), dl, "bucket", "corrupt.json.gz") + if err == nil { + t.Fatal("expected error for corrupt gzip data") + } +} + +func TestFetchReport_InvalidJSON(t *testing.T) { + var buf bytes.Buffer + gw := gzip.NewWriter(&buf) + gw.Write([]byte("not valid json")) + gw.Close() + + dl := &mockDownloader{objects: map[string][]byte{ + "bad.json.gz": buf.Bytes(), + }} + + _, err := FetchReport(context.Background(), dl, "bucket", "bad.json.gz") + if err == nil { + t.Fatal("expected error for invalid JSON") + } +} + +func TestFetchReport_WithLayer1Data(t *testing.T) { + layer := 0 + report := DivergenceReport{ + Height: 42, + Timestamp: "2026-04-02T00:00:00Z", + Comparison: CompareResult{ + Height: 42, + Match: false, + DivergenceLayer: &layer, + Layer0: Layer0Result{ + AppHashMatch: false, + LastResultsHashMatch: true, + GasUsedMatch: true, + ShadowAppHash: "shadow", + CanonicalAppHash: "canonical", + }, + Layer1: &Layer1Result{ + TotalTxs: 5, + TxCountMatch: true, + Divergences: []TxDivergence{ + { + TxIndex: 2, + Fields: []FieldDivergence{ + {Field: "code", Shadow: 0, Canonical: 1}, + {Field: "gasUsed", Shadow: "100000", Canonical: "120000"}, + }, + }, + }, + }, + }, + } + + dl := &mockDownloader{objects: map[string][]byte{ + "divergence-42.report.json.gz": gzipJSON(t, report), + }} + + got, err := FetchReport(context.Background(), dl, "bucket", "divergence-42.report.json.gz") + if err != nil { + t.Fatalf("FetchReport: %v", err) + } + + if got.Comparison.Layer1 == nil { + t.Fatal("expected Layer1 data") + } + if got.Comparison.Layer1.TotalTxs != 5 { + t.Errorf("TotalTxs = %d, want 5", got.Comparison.Layer1.TotalTxs) + } + if len(got.Comparison.Layer1.Divergences) != 1 { + t.Fatalf("expected 1 tx divergence, got %d", len(got.Comparison.Layer1.Divergences)) + } + div := got.Comparison.Layer1.Divergences[0] + if div.TxIndex != 2 { + t.Errorf("TxIndex = %d, want 2", div.TxIndex) + } + if len(div.Fields) != 2 { + t.Errorf("expected 2 field divergences, got %d", len(div.Fields)) + } +} + +func TestFetchReport_WithChainSnapshots(t *testing.T) { + report := DivergenceReport{ + Height: 99, + Timestamp: "2026-04-02T00:00:00Z", + Comparison: CompareResult{Height: 99, Match: false, Layer0: Layer0Result{AppHashMatch: false}}, + Shadow: ChainSnapshot{Block: json.RawMessage(`{"shadow":"block"}`), BlockResults: json.RawMessage(`{"shadow":"results"}`)}, + Canonical: ChainSnapshot{Block: json.RawMessage(`{"canonical":"block"}`), BlockResults: json.RawMessage(`{"canonical":"results"}`)}, + } + + dl := &mockDownloader{objects: map[string][]byte{ + "report.json.gz": gzipJSON(t, report), + }} + + got, err := FetchReport(context.Background(), dl, "bucket", "report.json.gz") + if err != nil { + t.Fatalf("FetchReport: %v", err) + } + + if len(got.Shadow.Block) == 0 { + t.Error("expected non-empty Shadow.Block") + } + if len(got.Canonical.Block) == 0 { + t.Error("expected non-empty Canonical.Block") + } +} diff --git a/sidecar/shadow/keysource.go b/sidecar/shadow/keysource.go new file mode 100644 index 00000000..bc501801 --- /dev/null +++ b/sidecar/shadow/keysource.go @@ -0,0 +1,129 @@ +package shadow + +import ( + "bytes" + "context" + "fmt" + "sort" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/common/hexutil" + gethrpc "github.com/ethereum/go-ethereum/rpc" +) + +// TraceKeySource derives a block's touched accounts and storage slots from a +// diff-mode prestate trace (debug_traceBlockByNumber, prestateTracer with +// diffMode) on an EVM JSON-RPC endpoint. diffMode reports both the pre-state +// (slots read) and post-state (slots written), so the union covers keys the +// block read OR modified — not just those read. Requires the debug_ namespace +// enabled on the endpoint (a non-public, operator-owned node). +// +// Coverage boundary: this is the per-block TOUCHED set. Keys migrated but never +// touched by any transaction (cold state), and non-EVM Cosmos-module state, are +// not covered here — that breadth is the corpus harness's job (Arm A) plus a +// periodic StaticKeySource sweep. Layer 2 over a trace source is a hot-state +// sampling oracle, not a full-keyspace check. +type TraceKeySource struct { + client *gethrpc.Client +} + +// NewTraceKeySource dials the EVM JSON-RPC endpoint used to fetch prestate +// traces. The same touched set applies to both chains (identical transactions), +// so a single endpoint (typically the canonical node) is sufficient. +func NewTraceKeySource(evmRPC string) (*TraceKeySource, error) { + c, err := gethrpc.Dial(evmRPC) + if err != nil { + return nil, fmt.Errorf("dialing EVM RPC %q: %w", evmRPC, err) + } + return &TraceKeySource{client: c}, nil +} + +// Close releases the underlying RPC connection. No return value, matching +// go-ethereum's *ethclient.Client / *rpc.Client Close() so Comparator.Close can +// treat all closeables uniformly. +func (t *TraceKeySource) Close() { + if t.client != nil { + t.client.Close() + } +} + +// prestateAccount is the subset of the prestateTracer's per-account output the +// key source needs: which slots were accessed, and whether the account carries +// code (a contract). +type prestateAccount struct { + Storage map[common.Hash]common.Hash `json:"storage"` + Code hexutil.Bytes `json:"code"` +} + +// prestateResult is one transaction's diff-mode prestate output: pre-state +// (read) and post-state (written) per account. +type prestateResult struct { + Pre map[common.Address]prestateAccount `json:"pre"` + Post map[common.Address]prestateAccount `json:"post"` +} + +// prestateTxTrace wraps one transaction's tracer result. +type prestateTxTrace struct { + Result prestateResult `json:"result"` +} + +func (t *TraceKeySource) TouchedAccounts(ctx context.Context, height int64) ([]TouchedAccount, error) { + var traces []prestateTxTrace + blockArg := hexutil.EncodeUint64(uint64(height)) + cfg := map[string]any{"tracer": "prestateTracer", "tracerConfig": map[string]any{"diffMode": true}} + if err := t.client.CallContext(ctx, &traces, "debug_traceBlockByNumber", blockArg, cfg); err != nil { + return nil, fmt.Errorf("debug_traceBlockByNumber at height %d: %w", height, err) + } + return mergePrestateTraces(traces), nil +} + +// mergePrestateTraces unions the per-transaction diff-mode results into one +// touched-account set per address: slots unioned across pre and post (reads and +// writes), code checked when either side shows code, balance and nonce checked +// for every touched account. Output is sorted (accounts by address, slots by +// hash) so the same block yields a byte-reproducible report. +func mergePrestateTraces(traces []prestateTxTrace) []TouchedAccount { + slots := map[common.Address]map[common.Hash]struct{}{} + hasCode := map[common.Address]bool{} + + absorb := func(accts map[common.Address]prestateAccount) { + for addr, acct := range accts { + if slots[addr] == nil { + slots[addr] = map[common.Hash]struct{}{} + } + for slot := range acct.Storage { + slots[addr][slot] = struct{}{} + } + if len(acct.Code) > 0 { + hasCode[addr] = true + } + } + } + for _, tx := range traces { + absorb(tx.Result.Pre) + absorb(tx.Result.Post) + } + + out := make([]TouchedAccount, 0, len(slots)) + for addr, slotSet := range slots { + ta := TouchedAccount{Addr: addr, CheckCode: hasCode[addr], CheckNonce: true, CheckBalance: true} + for slot := range slotSet { + ta.Slots = append(ta.Slots, slot) + } + sort.Slice(ta.Slots, func(i, j int) bool { return bytes.Compare(ta.Slots[i][:], ta.Slots[j][:]) < 0 }) + out = append(out, ta) + } + sort.Slice(out, func(i, j int) bool { return bytes.Compare(out[i].Addr[:], out[j].Addr[:]) < 0 }) + return out +} + +// StaticKeySource compares a fixed, curated set of accounts on every block — the +// fallback when prestate tracing (debug_) is unavailable, and the hook for a +// periodic cold-key / hot-contract sweep that the trace source does not cover. +type StaticKeySource struct { + Accounts []TouchedAccount +} + +func (s StaticKeySource) TouchedAccounts(_ context.Context, _ int64) ([]TouchedAccount, error) { + return s.Accounts, nil +} diff --git a/sidecar/shadow/keysource_test.go b/sidecar/shadow/keysource_test.go new file mode 100644 index 00000000..eb249964 --- /dev/null +++ b/sidecar/shadow/keysource_test.go @@ -0,0 +1,115 @@ +package shadow + +import ( + "context" + "encoding/json" + "testing" + + "github.com/ethereum/go-ethereum/common" +) + +// A realistic diff-mode debug_traceBlockByNumber response: an array with one +// entry per transaction, each {result: {pre: {...}, post: {...}}}. The union of +// pre (read) and post (written) slots is the touched set. +const samplePrestateJSON = `[ + {"result": { + "pre": { + "0x00000000000000000000000000000000000000aa": { + "balance": "0x1", "nonce": 7, + "storage": { + "0x0000000000000000000000000000000000000000000000000000000000000001": "0x000000000000000000000000000000000000000000000000000000000000002a", + "0x0000000000000000000000000000000000000000000000000000000000000002": "0x0000000000000000000000000000000000000000000000000000000000000000" + } + }, + "0x00000000000000000000000000000000000000bb": { + "balance": "0x0", "code": "0x6060604052", + "storage": { + "0x0000000000000000000000000000000000000000000000000000000000000005": "0x0000000000000000000000000000000000000000000000000000000000000007" + } + } + }, + "post": { + "0x00000000000000000000000000000000000000aa": { + "storage": { + "0x0000000000000000000000000000000000000000000000000000000000000003": "0x0000000000000000000000000000000000000000000000000000000000000009" + } + } + } + }} +]` + +func TestMergePrestateTraces(t *testing.T) { + var traces []prestateTxTrace + if err := json.Unmarshal([]byte(samplePrestateJSON), &traces); err != nil { + t.Fatalf("unmarshal prestate: %v", err) + } + + touched := mergePrestateTraces(traces) + byAddr := map[common.Address]TouchedAccount{} + for _, ta := range touched { + byAddr[ta.Addr] = ta + } + + addrAA := common.HexToAddress("0x00000000000000000000000000000000000000aa") + addrBB := common.HexToAddress("0x00000000000000000000000000000000000000bb") + + aa, ok := byAddr[addrAA] + if !ok { + t.Fatal("missing account aa") + } + // aa's slots unioned across pre (0x01, 0x02) and post (0x03) = 3. + if len(aa.Slots) != 3 { + t.Errorf("aa slots = %d, want 3 (%v)", len(aa.Slots), aa.Slots) + } + if aa.CheckCode { + t.Error("aa has no code; CheckCode should be false") + } + if !aa.CheckNonce || !aa.CheckBalance { + t.Error("every touched account should check nonce and balance") + } + + bb, ok := byAddr[addrBB] + if !ok { + t.Fatal("missing account bb") + } + if !bb.CheckCode { + t.Error("bb carries code; CheckCode should be true") + } + if len(bb.Slots) != 1 { + t.Errorf("bb slots = %d, want 1", len(bb.Slots)) + } +} + +// mergePrestateTraces output must be sorted (accounts by address, slots by hash) +// so the same block renders a byte-reproducible report. +func TestMergePrestateTraces_Sorted(t *testing.T) { + var traces []prestateTxTrace + if err := json.Unmarshal([]byte(samplePrestateJSON), &traces); err != nil { + t.Fatalf("unmarshal: %v", err) + } + touched := mergePrestateTraces(traces) + for i := 1; i < len(touched); i++ { + if common.Bytes2Hex(touched[i-1].Addr[:]) > common.Bytes2Hex(touched[i].Addr[:]) { + t.Errorf("accounts not sorted: %s before %s", touched[i-1].Addr.Hex(), touched[i].Addr.Hex()) + } + } + for _, ta := range touched { + for i := 1; i < len(ta.Slots); i++ { + if common.Bytes2Hex(ta.Slots[i-1][:]) > common.Bytes2Hex(ta.Slots[i][:]) { + t.Errorf("slots not sorted for %s", ta.Addr.Hex()) + } + } + } +} + +func TestStaticKeySource(t *testing.T) { + want := []TouchedAccount{{Addr: testAddr, Slots: []common.Hash{testSlot}, CheckNonce: true}} + src := StaticKeySource{Accounts: want} + got, err := src.TouchedAccounts(context.Background(), 1) + if err != nil { + t.Fatalf("TouchedAccounts: %v", err) + } + if len(got) != 1 || got[0].Addr != testAddr { + t.Errorf("got %+v, want %+v", got, want) + } +} diff --git a/sidecar/shadow/layer0.go b/sidecar/shadow/layer0.go new file mode 100644 index 00000000..5ad82282 --- /dev/null +++ b/sidecar/shadow/layer0.go @@ -0,0 +1,66 @@ +package shadow + +import ( + "context" + "encoding/json" + "fmt" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// compareLayer0 fetches the block header from both chains at the given +// height and compares AppHash, LastResultsHash, and total gas used. +func (c *Comparator) compareLayer0(ctx context.Context, height int64) (*Layer0Result, error) { + shadowBlock, err := queryBlock(ctx, c.shadowClient, height) + if err != nil { + return nil, fmt.Errorf("querying shadow block at height %d: %w", height, err) + } + canonicalBlock, err := queryBlock(ctx, c.canonicalClient, height) + if err != nil { + return nil, fmt.Errorf("querying canonical block at height %d: %w", height, err) + } + + sAppHash := shadowBlock.Block.Header.AppHash + cAppHash := canonicalBlock.Block.Header.AppHash + sLastResults := shadowBlock.Block.Header.LastResultsHash + cLastResults := canonicalBlock.Block.Header.LastResultsHash + + // Gas is summed from block_results; the block header doesn't carry it + // directly. For L0 we compare what the header gives us. Gas comparison + // via block_results happens implicitly in L1. + // For now, mark gas as matching at L0; a future enhancement can pull + // gas from the block_results endpoint at this layer. + gasMatch := true + + result := &Layer0Result{ + AppHashMatch: sAppHash == cAppHash, + LastResultsHashMatch: sLastResults == cLastResults, + GasUsedMatch: gasMatch, + } + + if !result.AppHashMatch { + result.ShadowAppHash = sAppHash + result.CanonicalAppHash = cAppHash + } + if !result.LastResultsHashMatch { + result.ShadowLastResultsHash = sLastResults + result.CanonicalLastResultsHash = cLastResults + } + + return result, nil +} + +// queryBlock fetches the block at the given height from a CometBFT RPC endpoint +// and returns the header fields needed for comparison. +func queryBlock(ctx context.Context, client *rpc.Client, height int64) (*rpc.BlockResult, error) { + raw, err := client.Get(ctx, fmt.Sprintf("/block?height=%d", height)) + if err != nil { + return nil, err + } + + var result rpc.BlockResult + if err := json.Unmarshal(raw, &result); err != nil { + return nil, fmt.Errorf("decoding /block response: %w", err) + } + return &result, nil +} diff --git a/sidecar/shadow/layer1.go b/sidecar/shadow/layer1.go new file mode 100644 index 00000000..0db819ec --- /dev/null +++ b/sidecar/shadow/layer1.go @@ -0,0 +1,119 @@ +package shadow + +import ( + "context" + "encoding/json" + "fmt" + "reflect" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// compareLayer1 fetches block_results from both chains and compares +// individual transaction receipts to identify which transactions diverged. +func (c *Comparator) compareLayer1(ctx context.Context, height int64) (*Layer1Result, error) { + shadowResults, err := queryBlockResults(ctx, c.shadowClient, height) + if err != nil { + return nil, fmt.Errorf("querying shadow block_results at height %d: %w", height, err) + } + canonicalResults, err := queryBlockResults(ctx, c.canonicalClient, height) + if err != nil { + return nil, fmt.Errorf("querying canonical block_results at height %d: %w", height, err) + } + + sTxs := shadowResults.TxsResults + cTxs := canonicalResults.TxsResults + + result := &Layer1Result{ + TotalTxs: max(len(sTxs), len(cTxs)), + TxCountMatch: len(sTxs) == len(cTxs), + } + + // Compare the overlapping transactions. + minLen := min(len(sTxs), len(cTxs)) + for i := 0; i < minLen; i++ { + divergence := compareTxReceipts(i, sTxs[i], cTxs[i]) + if divergence != nil { + result.Divergences = append(result.Divergences, *divergence) + } + } + + // Any extra transactions on either side are divergences by definition. + if len(sTxs) > minLen { + for i := minLen; i < len(sTxs); i++ { + result.Divergences = append(result.Divergences, TxDivergence{ + TxIndex: i, + Fields: []FieldDivergence{{ + Field: "presence", + Shadow: "present", + Canonical: "missing", + }}, + }) + } + } + if len(cTxs) > minLen { + for i := minLen; i < len(cTxs); i++ { + result.Divergences = append(result.Divergences, TxDivergence{ + TxIndex: i, + Fields: []FieldDivergence{{ + Field: "presence", + Shadow: "missing", + Canonical: "present", + }}, + }) + } + } + + return result, nil +} + +// compareTxReceipts compares critical fields from two transaction results. +// Returns nil when the receipts match. +func compareTxReceipts(idx int, shadow, canonical rpc.TxResult) *TxDivergence { + var fields []FieldDivergence + + if shadow.Code != canonical.Code { + fields = append(fields, FieldDivergence{ + Field: "code", Shadow: shadow.Code, Canonical: canonical.Code, + }) + } + if shadow.GasUsed != canonical.GasUsed { + fields = append(fields, FieldDivergence{ + Field: "gasUsed", Shadow: shadow.GasUsed, Canonical: canonical.GasUsed, + }) + } + if shadow.GasWanted != canonical.GasWanted { + fields = append(fields, FieldDivergence{ + Field: "gasWanted", Shadow: shadow.GasWanted, Canonical: canonical.GasWanted, + }) + } + if shadow.Log != canonical.Log { + fields = append(fields, FieldDivergence{ + Field: "log", Shadow: shadow.Log, Canonical: canonical.Log, + }) + } + if !reflect.DeepEqual(shadow.Events, canonical.Events) { + fields = append(fields, FieldDivergence{ + Field: "events", Shadow: shadow.Events, Canonical: canonical.Events, + }) + } + + if len(fields) == 0 { + return nil + } + return &TxDivergence{TxIndex: idx, Fields: fields} +} + +// queryBlockResults fetches /block_results at the given height. +func queryBlockResults(ctx context.Context, client *rpc.Client, height int64) (*rpc.BlockResultsResult, error) { + raw, err := client.Get(ctx, fmt.Sprintf("/block_results?height=%d", height)) + if err != nil { + return nil, err + } + + var result rpc.BlockResultsResult + if err := json.Unmarshal(raw, &result); err != nil { + return nil, fmt.Errorf("decoding /block_results response: %w", err) + } + return &result, nil +} diff --git a/sidecar/shadow/layer2.go b/sidecar/shadow/layer2.go new file mode 100644 index 00000000..7c95455b --- /dev/null +++ b/sidecar/shadow/layer2.go @@ -0,0 +1,123 @@ +package shadow + +import ( + "bytes" + "context" + "fmt" + "math/big" + + "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/common/hexutil" +) + +// StateReader reads logical EVM state at a height. Its method set matches +// go-ethereum's *ethclient.Client, so a real client satisfies it directly; the +// shadow and canonical sides are two instances. A nil blockNumber means latest. +type StateReader interface { + StorageAt(ctx context.Context, account common.Address, key common.Hash, blockNumber *big.Int) ([]byte, error) + CodeAt(ctx context.Context, account common.Address, blockNumber *big.Int) ([]byte, error) + NonceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (uint64, error) + BalanceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (*big.Int, error) +} + +// TouchedAccount is the set of state a block touched for one address: the +// storage slots to compare, and whether the account's balance, code, and nonce +// should be checked. A KeySource produces these per block (e.g. from a trace). +type TouchedAccount struct { + Addr common.Address + Slots []common.Hash + CheckCode bool + CheckNonce bool + CheckBalance bool +} + +// KeySource yields the accounts (and their slots) a block touched, so Layer 2 +// compares exactly the state real transactions read or wrote at that height. +type KeySource interface { + TouchedAccounts(ctx context.Context, height int64) ([]TouchedAccount, error) +} + +// compareState reads each touched key's logical value from both chains at the +// given height and records every mismatch. It fails closed: a read error on any +// key aborts with that error rather than reporting a partial (and so falsely +// clean) result. +func compareState(ctx context.Context, height int64, touched []TouchedAccount, shadow, canonical StateReader) (*Layer2Result, error) { + blockNum := big.NewInt(height) + res := &Layer2Result{} + + for _, acct := range touched { + res.AccountsChecked++ + + for _, slot := range acct.Slots { + res.KeysChecked++ + s, err := shadow.StorageAt(ctx, acct.Addr, slot, blockNum) + if err != nil { + return nil, fmt.Errorf("shadow storage %s/%s: %w", acct.Addr.Hex(), slot.Hex(), err) + } + c, err := canonical.StorageAt(ctx, acct.Addr, slot, blockNum) + if err != nil { + return nil, fmt.Errorf("canonical storage %s/%s: %w", acct.Addr.Hex(), slot.Hex(), err) + } + if !bytes.Equal(common.LeftPadBytes(s, 32), common.LeftPadBytes(c, 32)) { + res.Divergences = append(res.Divergences, StateDivergence{ + Kind: "storage", Addr: acct.Addr.Hex(), Slot: slot.Hex(), + Shadow: hexutil.Encode(s), Canonical: hexutil.Encode(c), + }) + } + } + + if acct.CheckBalance { + res.KeysChecked++ + s, err := shadow.BalanceAt(ctx, acct.Addr, blockNum) + if err != nil { + return nil, fmt.Errorf("shadow balance %s: %w", acct.Addr.Hex(), err) + } + c, err := canonical.BalanceAt(ctx, acct.Addr, blockNum) + if err != nil { + return nil, fmt.Errorf("canonical balance %s: %w", acct.Addr.Hex(), err) + } + if s.Cmp(c) != 0 { + res.Divergences = append(res.Divergences, StateDivergence{ + Kind: "balance", Addr: acct.Addr.Hex(), Shadow: s.String(), Canonical: c.String(), + }) + } + } + + if acct.CheckCode { + res.KeysChecked++ + s, err := shadow.CodeAt(ctx, acct.Addr, blockNum) + if err != nil { + return nil, fmt.Errorf("shadow code %s: %w", acct.Addr.Hex(), err) + } + c, err := canonical.CodeAt(ctx, acct.Addr, blockNum) + if err != nil { + return nil, fmt.Errorf("canonical code %s: %w", acct.Addr.Hex(), err) + } + if !bytes.Equal(s, c) { + res.Divergences = append(res.Divergences, StateDivergence{ + Kind: "code", Addr: acct.Addr.Hex(), Shadow: hexutil.Encode(s), Canonical: hexutil.Encode(c), + }) + } + } + + if acct.CheckNonce { + res.KeysChecked++ + s, err := shadow.NonceAt(ctx, acct.Addr, blockNum) + if err != nil { + return nil, fmt.Errorf("shadow nonce %s: %w", acct.Addr.Hex(), err) + } + c, err := canonical.NonceAt(ctx, acct.Addr, blockNum) + if err != nil { + return nil, fmt.Errorf("canonical nonce %s: %w", acct.Addr.Hex(), err) + } + if s != c { + res.Divergences = append(res.Divergences, StateDivergence{ + Kind: "nonce", Addr: acct.Addr.Hex(), + Shadow: fmt.Sprintf("%d", s), Canonical: fmt.Sprintf("%d", c), + }) + } + } + } + + return res, nil +} diff --git a/sidecar/shadow/layer2_test.go b/sidecar/shadow/layer2_test.go new file mode 100644 index 00000000..c3d1845a --- /dev/null +++ b/sidecar/shadow/layer2_test.go @@ -0,0 +1,244 @@ +package shadow + +import ( + "context" + "encoding/json" + "errors" + "math/big" + "testing" + + "github.com/ethereum/go-ethereum/common" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// mockKeySource returns a fixed set of touched accounts (or an error). +type mockKeySource struct { + accts []TouchedAccount + err error +} + +func (m mockKeySource) TouchedAccounts(_ context.Context, _ int64) ([]TouchedAccount, error) { + return m.accts, m.err +} + +// mockState is a controllable StateReader: maps hold the value each side returns +// for a key, errOn forces a read error for a kind so the fail-closed path can be +// exercised. +type mockState struct { + storage map[string][]byte // addr.Hex()|slot.Hex() -> value + code map[string][]byte // addr.Hex() -> bytecode + nonce map[string]uint64 // addr.Hex() -> nonce + balance map[string]*big.Int // addr.Hex() -> balance + errOn string // "storage" | "code" | "nonce" | "balance" +} + +func newMockState() *mockState { + return &mockState{ + storage: map[string][]byte{}, + code: map[string][]byte{}, + nonce: map[string]uint64{}, + balance: map[string]*big.Int{}, + } +} + +func (m *mockState) StorageAt(_ context.Context, account common.Address, key common.Hash, _ *big.Int) ([]byte, error) { + if m.errOn == "storage" { + return nil, errors.New("injected storage error") + } + return m.storage[account.Hex()+"|"+key.Hex()], nil +} + +func (m *mockState) CodeAt(_ context.Context, account common.Address, _ *big.Int) ([]byte, error) { + if m.errOn == "code" { + return nil, errors.New("injected code error") + } + return m.code[account.Hex()], nil +} + +func (m *mockState) NonceAt(_ context.Context, account common.Address, _ *big.Int) (uint64, error) { + if m.errOn == "nonce" { + return 0, errors.New("injected nonce error") + } + return m.nonce[account.Hex()], nil +} + +func (m *mockState) BalanceAt(_ context.Context, account common.Address, _ *big.Int) (*big.Int, error) { + if m.errOn == "balance" { + return nil, errors.New("injected balance error") + } + if b, ok := m.balance[account.Hex()]; ok { + return b, nil + } + return big.NewInt(0), nil +} + +var ( + testAddr = common.HexToAddress("0x00000000000000000000000000000000000000aa") + testSlot = common.HexToHash("0x01") +) + +func storageKey(a common.Address, s common.Hash) string { return a.Hex() + "|" + s.Hex() } + +func pad32(b []byte) []byte { return common.LeftPadBytes(b, 32) } + +func TestCompareState_AllMatch(t *testing.T) { + shadow, canonical := newMockState(), newMockState() + shadow.storage[storageKey(testAddr, testSlot)] = pad32([]byte{0x2a}) + canonical.storage[storageKey(testAddr, testSlot)] = []byte{0x2a} // trimmed; must normalize-equal + shadow.code[testAddr.Hex()] = []byte{0x60, 0x60} + canonical.code[testAddr.Hex()] = []byte{0x60, 0x60} + shadow.nonce[testAddr.Hex()] = 5 + canonical.nonce[testAddr.Hex()] = 5 + shadow.balance[testAddr.Hex()] = big.NewInt(1000) + canonical.balance[testAddr.Hex()] = big.NewInt(1000) + + touched := []TouchedAccount{{Addr: testAddr, Slots: []common.Hash{testSlot}, CheckCode: true, CheckNonce: true, CheckBalance: true}} + res, err := compareState(context.Background(), 100, touched, shadow, canonical) + if err != nil { + t.Fatalf("compareState: %v", err) + } + if len(res.Divergences) != 0 { + t.Errorf("expected no divergences, got %+v", res.Divergences) + } + if res.AccountsChecked != 1 || res.KeysChecked != 4 { + t.Errorf("counts: accounts=%d keys=%d, want 1/4", res.AccountsChecked, res.KeysChecked) + } +} + +func TestCompareState_Mismatches(t *testing.T) { + shadow, canonical := newMockState(), newMockState() + shadow.storage[storageKey(testAddr, testSlot)] = pad32([]byte{0x2a}) + canonical.storage[storageKey(testAddr, testSlot)] = pad32([]byte{0x2b}) // storage differs + shadow.code[testAddr.Hex()] = []byte{0x60} + canonical.code[testAddr.Hex()] = []byte{0x61} // code differs + shadow.nonce[testAddr.Hex()] = 7 + canonical.nonce[testAddr.Hex()] = 8 // nonce differs + shadow.balance[testAddr.Hex()] = big.NewInt(1) + canonical.balance[testAddr.Hex()] = big.NewInt(2) // balance differs + + touched := []TouchedAccount{{Addr: testAddr, Slots: []common.Hash{testSlot}, CheckCode: true, CheckNonce: true, CheckBalance: true}} + res, err := compareState(context.Background(), 100, touched, shadow, canonical) + if err != nil { + t.Fatalf("compareState: %v", err) + } + if len(res.Divergences) != 4 { + t.Fatalf("expected 4 divergences, got %d: %+v", len(res.Divergences), res.Divergences) + } + kinds := map[string]bool{} + for _, d := range res.Divergences { + kinds[d.Kind] = true + if d.Addr != testAddr.Hex() { + t.Errorf("divergence addr = %s, want %s", d.Addr, testAddr.Hex()) + } + } + for _, want := range []string{"storage", "balance", "code", "nonce"} { + if !kinds[want] { + t.Errorf("missing %s divergence", want) + } + } +} + +func TestCompareState_FailsClosed(t *testing.T) { + shadow, canonical := newMockState(), newMockState() + canonical.errOn = "storage" + touched := []TouchedAccount{{Addr: testAddr, Slots: []common.Hash{testSlot}}} + if _, err := compareState(context.Background(), 100, touched, shadow, canonical); err == nil { + t.Error("expected fail-closed error when a side cannot be read") + } +} + +// End-to-end through CompareBlock: migration mode (AppHash expected to differ), +// Layer 1 receipts match, but Layer 2 logical state diverges -> DivergenceLayer 2. +func TestCompareBlock_Layer2_StateDivergence(t *testing.T) { + txs := []rpc.TxResult{ + {Code: 0, GasUsed: "100", GasWanted: "200", Log: "ok", Events: json.RawMessage(`[]`)}, + } + shadowSrv := rpcServer("SHADOW_APPHASH", "SAME_RESULTS", txs) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CANON_APPHASH", "SAME_RESULTS", txs) + defer canonicalSrv.Close() + + shadowState, canonState := newMockState(), newMockState() + shadowState.storage[storageKey(testAddr, testSlot)] = pad32([]byte{0x01}) + canonState.storage[storageKey(testAddr, testSlot)] = pad32([]byte{0x02}) + ks := mockKeySource{accts: []TouchedAccount{{Addr: testAddr, Slots: []common.Hash{testSlot}}}} + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL, + WithMigrationMode(), WithLayer2(shadowState, canonState, ks)) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if result.Match { + t.Error("expected divergence: Layer 2 logical state differs") + } + if result.DivergenceLayer == nil || *result.DivergenceLayer != 2 { + t.Errorf("expected divergence layer 2, got %v", result.DivergenceLayer) + } + if result.Layer2 == nil || len(result.Layer2.Divergences) != 1 { + t.Errorf("expected 1 Layer2 divergence, got %+v", result.Layer2) + } +} + +// When Layer 2 logical state matches, a migration shadow is a clean match even +// though its AppHash differs (Layer 0) by design. +func TestCompareBlock_Layer2_CleanMatch(t *testing.T) { + txs := []rpc.TxResult{ + {Code: 0, GasUsed: "100", GasWanted: "200", Log: "ok", Events: json.RawMessage(`[]`)}, + } + shadowSrv := rpcServer("SHADOW_APPHASH", "SAME_RESULTS", txs) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CANON_APPHASH", "SAME_RESULTS", txs) + defer canonicalSrv.Close() + + shadowState, canonState := newMockState(), newMockState() + shadowState.storage[storageKey(testAddr, testSlot)] = pad32([]byte{0x42}) + canonState.storage[storageKey(testAddr, testSlot)] = pad32([]byte{0x42}) + ks := mockKeySource{accts: []TouchedAccount{{Addr: testAddr, Slots: []common.Hash{testSlot}}}} + + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL, + WithMigrationMode(), WithLayer2(shadowState, canonState, ks)) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if !result.Match { + t.Errorf("expected clean match, got divergence at layer %v", result.DivergenceLayer) + } + if result.Layer2 == nil || result.Layer2.AccountsChecked != 1 { + t.Errorf("expected Layer2 populated with 1 account checked, got %+v", result.Layer2) + } +} + +// A Layer 2 that cannot run (key-source error) must NOT be a silent clean pass: +// it is marked indeterminate and forces a divergence at layer 2 (fail-closed). +func TestCompareBlock_Layer2_IndeterminateFailsClosed(t *testing.T) { + txs := []rpc.TxResult{ + {Code: 0, GasUsed: "100", GasWanted: "200", Log: "ok", Events: json.RawMessage(`[]`)}, + } + shadowSrv := rpcServer("SHADOW_APPHASH", "SAME_RESULTS", txs) + defer shadowSrv.Close() + canonicalSrv := rpcServer("CANON_APPHASH", "SAME_RESULTS", txs) + defer canonicalSrv.Close() + + ks := mockKeySource{err: errors.New("trace endpoint unavailable")} + comp := NewComparator(shadowSrv.URL, canonicalSrv.URL, + WithMigrationMode(), WithLayer2(newMockState(), newMockState(), ks)) + result, err := comp.CompareBlock(context.Background(), 100) + if err != nil { + t.Fatalf("CompareBlock: %v", err) + } + + if result.Match { + t.Error("expected NOT clean: Layer 2 could not run, must fail closed") + } + if result.DivergenceLayer == nil || *result.DivergenceLayer != 2 { + t.Errorf("expected divergence layer 2, got %v", result.DivergenceLayer) + } + if result.Layer2 == nil || !result.Layer2.Indeterminate { + t.Errorf("expected Layer2 marked indeterminate, got %+v", result.Layer2) + } +} diff --git a/sidecar/shadow/metrics.go b/sidecar/shadow/metrics.go new file mode 100644 index 00000000..d8592383 --- /dev/null +++ b/sidecar/shadow/metrics.go @@ -0,0 +1,35 @@ +package shadow + +import "github.com/prometheus/client_golang/prometheus" + +var ( + // BlocksCompared counts blocks the comparator has processed. + // rate(...)==0 indicates the comparator has stopped advancing — typically + // shadow RPC unreachable or the local node has stopped producing blocks. + // pod_name differentiates two shadow candidates for the same chain so + // alerts can route to a specific candidate image. + BlocksCompared = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "seictl_shadow_blocks_compared_total", + Help: "Total blocks compared between the shadow node and the canonical chain.", + }, + []string{"chain_id", "pod_name"}, + ) + + // Divergences counts app-hash divergences detected. Increments at most + // once per process lifetime — the comparison loop exits on first divergence. + // divergence_layer is "0" for header-hash mismatch, "1" when Layer 1 + // isolated specific tx-receipt mismatches. + Divergences = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "seictl_shadow_divergences_total", + Help: "App-hash divergences detected by the shadow comparator. Increments once per process lifetime since the loop exits on first divergence.", + }, + []string{"chain_id", "pod_name", "divergence_layer"}, + ) +) + +func init() { + prometheus.MustRegister(BlocksCompared) + prometheus.MustRegister(Divergences) +} diff --git a/sidecar/shadow/render.go b/sidecar/shadow/render.go new file mode 100644 index 00000000..6040e086 --- /dev/null +++ b/sidecar/shadow/render.go @@ -0,0 +1,152 @@ +package shadow + +import ( + "fmt" + "strings" +) + +// RenderMarkdown produces a human-readable investigation report from a +// DivergenceReport. The output is designed to be consumed by engineers +// or LLM agents analyzing why a shadow node diverged from the canonical chain. +func RenderMarkdown(r *DivergenceReport) string { + var b strings.Builder + + writeHeader(&b, r) + writeLayer0(&b, r.Comparison.Layer0) + + if r.Comparison.Layer1 != nil { + writeLayer1(&b, r.Comparison.Layer1) + } + + if r.Comparison.Layer2 != nil { + writeLayer2(&b, r.Comparison.Layer2) + } + + writeRawDataNote(&b, r.Height) + return b.String() +} + +func writeHeader(b *strings.Builder, r *DivergenceReport) { + fmt.Fprintf(b, "# Divergence Report — Height %d\n\n", r.Height) + fmt.Fprintf(b, "**Detected at:** %s\n\n", r.Timestamp) + fmt.Fprintf(b, "App-hash divergence detected. The shadow node and canonical chain ") + fmt.Fprintf(b, "produce different execution results at this block.\n\n") +} + +func writeLayer0(b *strings.Builder, l0 Layer0Result) { + fmt.Fprintf(b, "## Layer 0: Block Header Comparison\n\n") + fmt.Fprintf(b, "| Field | Shadow | Canonical | Match |\n") + fmt.Fprintf(b, "|-------|--------|-----------|-------|\n") + + writeL0Row(b, "AppHash", l0.AppHashMatch, l0.ShadowAppHash, l0.CanonicalAppHash) + writeL0Row(b, "LastResultsHash", l0.LastResultsHashMatch, l0.ShadowLastResultsHash, l0.CanonicalLastResultsHash) + writeL0GasRow(b, l0) + fmt.Fprintf(b, "\n") +} + +func writeL0Row(b *strings.Builder, field string, match bool, shadow, canonical string) { + icon := "✅" + if !match { + icon = "❌" + } + s := truncateHash(shadow) + c := truncateHash(canonical) + if match { + s = "—" + c = "—" + } + fmt.Fprintf(b, "| %s | %s | %s | %s |\n", field, s, c, icon) +} + +func writeL0GasRow(b *strings.Builder, l0 Layer0Result) { + icon := "✅" + s := "—" + c := "—" + if !l0.GasUsedMatch { + icon = "❌" + s = fmt.Sprintf("%d", l0.ShadowGasUsed) + c = fmt.Sprintf("%d", l0.CanonicalGasUsed) + } + fmt.Fprintf(b, "| GasUsed | %s | %s | %s |\n", s, c, icon) +} + +func writeLayer1(b *strings.Builder, l1 *Layer1Result) { + fmt.Fprintf(b, "## Layer 1: Transaction Receipt Comparison\n\n") + if l1.Indeterminate { + fmt.Fprintf(b, "**Indeterminate** — receipt comparison could not run, so this block is not validated: %s\n\n", l1.Error) + return + } + fmt.Fprintf(b, "**Total transactions:** %d\n", l1.TotalTxs) + + if !l1.TxCountMatch { + fmt.Fprintf(b, "**Transaction count mismatch** — chains have different numbers of transactions in this block.\n") + } + + fmt.Fprintf(b, "**Divergent transactions:** %d\n\n", len(l1.Divergences)) + + for _, div := range l1.Divergences { + writeTxDivergence(b, div) + } +} + +func writeLayer2(b *strings.Builder, l2 *Layer2Result) { + fmt.Fprintf(b, "## Layer 2: Logical State Comparison\n\n") + if l2.Indeterminate { + fmt.Fprintf(b, "**Indeterminate** — the logical state check could not run, so this block is not validated: %s\n\n", l2.Error) + return + } + fmt.Fprintf(b, "**Accounts checked:** %d    **Keys checked:** %d\n", l2.AccountsChecked, l2.KeysChecked) + fmt.Fprintf(b, "**Divergent keys:** %d\n\n", len(l2.Divergences)) + + if len(l2.Divergences) == 0 { + return + } + + fmt.Fprintf(b, "| Kind | Address | Slot | Shadow | Canonical |\n") + fmt.Fprintf(b, "|------|---------|------|--------|----------|\n") + for _, d := range l2.Divergences { + slot := d.Slot + if slot == "" { + slot = "—" + } + fmt.Fprintf(b, "| %s | %s | %s | %s | %s |\n", + d.Kind, truncateHash(d.Addr), truncateHash(slot), + truncateHash(d.Shadow), truncateHash(d.Canonical)) + } + fmt.Fprintf(b, "\n") +} + +func writeTxDivergence(b *strings.Builder, div TxDivergence) { + fmt.Fprintf(b, "### Transaction %d\n\n", div.TxIndex) + fmt.Fprintf(b, "| Field | Shadow | Canonical |\n") + fmt.Fprintf(b, "|-------|--------|----------|\n") + + for _, f := range div.Fields { + fmt.Fprintf(b, "| %s | %s | %s |\n", + f.Field, + truncateValue(f.Shadow), + truncateValue(f.Canonical)) + } + fmt.Fprintf(b, "\n") +} + +func writeRawDataNote(b *strings.Builder, height int64) { + fmt.Fprintf(b, "## Raw Data\n\n") + fmt.Fprintf(b, "Full block and block_results JSON from both chains is included in this report.\n") + fmt.Fprintf(b, "Use `--json` flag to output the raw DivergenceReport for programmatic analysis.\n") +} + +func truncateHash(h string) string { + if len(h) <= 16 { + return h + } + return h[:8] + "..." + h[len(h)-4:] +} + +func truncateValue(v any) string { + s := fmt.Sprintf("%v", v) + if len(s) > 80 { + return s[:77] + "..." + } + return s +} diff --git a/sidecar/shadow/render_layer2_test.go b/sidecar/shadow/render_layer2_test.go new file mode 100644 index 00000000..d02f2bec --- /dev/null +++ b/sidecar/shadow/render_layer2_test.go @@ -0,0 +1,41 @@ +package shadow + +import ( + "strings" + "testing" +) + +func TestRenderMarkdown_Layer2(t *testing.T) { + layer := 2 + report := &DivergenceReport{ + Height: 1000, + Timestamp: "2026-06-17T00:00:00Z", + Comparison: CompareResult{ + Height: 1000, + Match: false, + MigrationMode: true, + DivergenceLayer: &layer, + Layer2: &Layer2Result{ + AccountsChecked: 3, + KeysChecked: 12, + Divergences: []StateDivergence{ + {Kind: "storage", Addr: "0xabc", Slot: "0x01", Shadow: "0x2a", Canonical: "0x2b"}, + {Kind: "nonce", Addr: "0xdef", Shadow: "7", Canonical: "8"}, + }, + }, + }, + } + + md := RenderMarkdown(report) + for _, want := range []string{ + "## Layer 2: Logical State Comparison", + "Accounts checked:", + "Divergent keys:", + "storage", + "nonce", + } { + if !strings.Contains(md, want) { + t.Errorf("rendered report missing %q\n---\n%s", want, md) + } + } +} diff --git a/sidecar/shadow/report.go b/sidecar/shadow/report.go new file mode 100644 index 00000000..06552f95 --- /dev/null +++ b/sidecar/shadow/report.go @@ -0,0 +1,57 @@ +package shadow + +import ( + "context" + "fmt" + "time" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// BuildDivergenceReport captures a complete investigation artifact at the +// given height. It pairs the comparison result with the full raw RPC +// responses from both chains so engineers can diagnose offline. +func (c *Comparator) BuildDivergenceReport(ctx context.Context, height int64, comparison CompareResult) (*DivergenceReport, error) { + shadowSnap, err := captureChainSnapshot(ctx, c.shadowClient, height) + if err != nil { + return nil, fmt.Errorf("capturing shadow snapshot at height %d: %w", height, err) + } + + canonicalSnap, err := captureChainSnapshot(ctx, c.canonicalClient, height) + if err != nil { + return nil, fmt.Errorf("capturing canonical snapshot at height %d: %w", height, err) + } + + return &DivergenceReport{ + Height: height, + Timestamp: time.Now().UTC().Format(time.RFC3339), + Comparison: comparison, + Shadow: *shadowSnap, + Canonical: *canonicalSnap, + }, nil +} + +func captureChainSnapshot(ctx context.Context, client *rpc.Client, height int64) (*ChainSnapshot, error) { + block, err := fetchRawBlock(ctx, client, height) + if err != nil { + return nil, err + } + + blockResults, err := fetchRawBlockResults(ctx, client, height) + if err != nil { + return nil, err + } + + return &ChainSnapshot{ + Block: block, + BlockResults: blockResults, + }, nil +} + +func fetchRawBlock(ctx context.Context, client *rpc.Client, height int64) ([]byte, error) { + return client.GetRaw(ctx, fmt.Sprintf("/block?height=%d", height)) +} + +func fetchRawBlockResults(ctx context.Context, client *rpc.Client, height int64) ([]byte, error) { + return client.GetRaw(ctx, fmt.Sprintf("/block_results?height=%d", height)) +} diff --git a/sidecar/shadow/types.go b/sidecar/shadow/types.go new file mode 100644 index 00000000..ef4ad6f9 --- /dev/null +++ b/sidecar/shadow/types.go @@ -0,0 +1,164 @@ +// Package shadow provides block-by-block comparison between a shadow +// chain node and a canonical chain node. The comparison is layered: +// +// - Layer 0: Block header hashes (AppHash, LastResultsHash, gas). +// If these match, the block is identical and deeper layers are skipped. +// - Layer 1: Transaction receipt comparison (status, gas, logs, etc.). +// Run only when Layer 0 fails, to isolate which transactions diverged. +// - Layer 2: State diff comparison — logical EVM state (storage/code/nonce) +// for the keys a block touched, read via EVM RPC on both sides. The +// load-bearing check for an AppHash-breaking migration shadow, where the +// committed root diverges by design and only logical state can be compared. +// - Layer 3: Execution trace comparison (future). +package shadow + +import "encoding/json" + +// CompareResult holds the comparison output for a single block. +type CompareResult struct { + // Height is the block height that was compared. + Height int64 `json:"height"` + + // Timestamp is the UTC time the comparison was performed. + Timestamp string `json:"timestamp"` + + // Match is true when all checked layers agree between shadow and canonical. + // In migration mode an expected AppHash divergence does not clear Match. + Match bool `json:"match"` + + // MigrationMode records that this comparison treated AppHash divergence as + // expected (an AppHash-breaking migration shadow), keying the verdict on + // execution-results equivalence instead. + MigrationMode bool `json:"migrationMode,omitempty"` + + // DivergenceLayer is the first layer that detected a mismatch. + // Nil when Match is true. + DivergenceLayer *int `json:"divergenceLayer,omitempty"` + + // Layer0 holds the block-level hash comparison. Always populated. + Layer0 Layer0Result `json:"layer0"` + + // Layer1 holds the transaction receipt comparison. + // Only populated when Layer 0 detected a divergence. + Layer1 *Layer1Result `json:"layer1,omitempty"` + + // Layer2 holds the logical state-diff comparison. Populated only when a + // state reader and key source are configured (see WithLayer2). + Layer2 *Layer2Result `json:"layer2,omitempty"` +} + +// Diverged returns true when the comparison detected a mismatch at any layer. +func (r *CompareResult) Diverged() bool { + return !r.Match +} + +// Layer0Result compares block-level hashes. This is the cheapest check; +// if all fields match, the block is identical and no further comparison +// is needed. +type Layer0Result struct { + AppHashMatch bool `json:"appHashMatch"` + LastResultsHashMatch bool `json:"lastResultsHashMatch"` + GasUsedMatch bool `json:"gasUsedMatch"` + + // Raw values are included when there is a mismatch, for diagnostics. + ShadowAppHash string `json:"shadowAppHash,omitempty"` + CanonicalAppHash string `json:"canonicalAppHash,omitempty"` + + ShadowLastResultsHash string `json:"shadowLastResultsHash,omitempty"` + CanonicalLastResultsHash string `json:"canonicalLastResultsHash,omitempty"` + + ShadowGasUsed int64 `json:"shadowGasUsed,omitempty"` + CanonicalGasUsed int64 `json:"canonicalGasUsed,omitempty"` +} + +// Match returns true when all Layer 0 fields agree. +func (r Layer0Result) Match() bool { + return r.AppHashMatch && r.LastResultsHashMatch && r.GasUsedMatch +} + +// Layer1Result compares individual transaction receipts within a block. +// Only populated when Layer 0 fails. +type Layer1Result struct { + // TotalTxs is the number of transactions in the block. + TotalTxs int `json:"totalTxs"` + + // TxCountMatch is true when both chains have the same number of txs. + TxCountMatch bool `json:"txCountMatch"` + + // Divergences lists the per-transaction differences found. + Divergences []TxDivergence `json:"divergences,omitempty"` + + // Indeterminate is set when the receipt comparison could not run (RPC error). + // In migration mode Layer 1 is a load-bearing check, so an indeterminate + // Layer 1 forces the block to fail closed rather than pass silently. + Indeterminate bool `json:"indeterminate,omitempty"` + Error string `json:"error,omitempty"` +} + +// TxDivergence records a mismatch for a single transaction within a block. +type TxDivergence struct { + // TxIndex is the position of the transaction within the block. + TxIndex int `json:"txIndex"` + + // Fields lists which receipt fields diverged. + Fields []FieldDivergence `json:"fields"` +} + +// FieldDivergence records a single field-level mismatch in a tx receipt. +type FieldDivergence struct { + Field string `json:"field"` + Shadow any `json:"shadow"` + Canonical any `json:"canonical"` +} + +// Layer2Result compares logical EVM state (storage slots, code, nonce) for the +// keys a block touched, read via EVM RPC on both chains. This is the logical- +// content truth check; it never compares the committed root, which is +// schedule-dependent for a migration shadow and so not a correctness oracle. +type Layer2Result struct { + // AccountsChecked is the number of touched accounts compared. + AccountsChecked int `json:"accountsChecked"` + + // KeysChecked is the number of individual state reads compared (storage + // slots plus per-account balance/code/nonce checks). + KeysChecked int `json:"keysChecked"` + + // Divergences lists the logical-state mismatches found. + Divergences []StateDivergence `json:"divergences,omitempty"` + + // Indeterminate is set when the layer could not be evaluated (a key source + // or state read failed). A migration shadow's load-bearing check could not + // run, so the block must NOT be counted clean — Error carries the cause. + Indeterminate bool `json:"indeterminate,omitempty"` + Error string `json:"error,omitempty"` +} + +// StateDivergence records a single logical-state mismatch between the shadow +// and canonical chains. Values are hex for legibility in reports. +type StateDivergence struct { + Kind string `json:"kind"` // storage | code | nonce + Addr string `json:"addr"` + Slot string `json:"slot,omitempty"` // set only for storage + Shadow string `json:"shadow"` + Canonical string `json:"canonical"` +} + +// DivergenceReport is a self-contained investigation artifact for a single +// app-hash divergence event. It includes the layered comparison result plus +// the full block and block_results from both chains, giving engineers all +// the context needed to diagnose why the shadow node diverged without +// querying external systems. +type DivergenceReport struct { + Height int64 `json:"height"` + Timestamp string `json:"timestamp"` + Comparison CompareResult `json:"comparison"` + Shadow ChainSnapshot `json:"shadow"` + Canonical ChainSnapshot `json:"canonical"` +} + +// ChainSnapshot captures the raw RPC responses from one chain at a +// specific height. The JSON is preserved verbatim for offline analysis. +type ChainSnapshot struct { + Block json.RawMessage `json:"block"` + BlockResults json.RawMessage `json:"blockResults"` +} diff --git a/sidecar/startup_guard_test.go b/sidecar/startup_guard_test.go new file mode 100644 index 00000000..5a1e2990 --- /dev/null +++ b/sidecar/startup_guard_test.go @@ -0,0 +1,88 @@ +package main + +import ( + "strings" + "testing" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/server" +) + +// The refusal is the only thing standing between a dropped environment variable +// and an unauthenticated signing API, so pin every combination rather than just +// the failing one — a future edit that widens the guard's escape hatch should +// break a test, not a cluster. +func TestCheckKeyringNeedsAuthn(t *testing.T) { + cases := []struct { + name string + keyringBackend string + authnMode string + wantErr bool + }{ + { + name: "keyring with no authn is refused", + keyringBackend: server.BackendFile, + authnMode: server.AuthnModeUnauthenticated, + wantErr: true, + }, + { + name: "the test backend is refused too — it is an unencrypted " + + "operator keyring, so an open listener is worse, not better", + keyringBackend: server.BackendTest, + authnMode: server.AuthnModeUnauthenticated, + wantErr: true, + }, + { + name: "keyring behind trusted-header authn is allowed", + keyringBackend: server.BackendFile, + authnMode: server.AuthnModeTrustedHeader, + wantErr: false, + }, + { + name: "no keyring with no authn is allowed — nothing to sign with, " + + "which is the non-validator node case", + keyringBackend: "", + authnMode: server.AuthnModeUnauthenticated, + wantErr: false, + }, + { + name: "no keyring behind authn is allowed", + keyringBackend: "", + authnMode: server.AuthnModeTrustedHeader, + wantErr: false, + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + err := checkKeyringNeedsAuthn(tc.keyringBackend, tc.authnMode) + if tc.wantErr { + if err == nil { + t.Fatal("expected a refusal, got nil") + } + // The message is what an operator debugging a CrashLoopBackOff + // reads, so keep it pointing at both halves of the problem. + for _, want := range []string{"SEI_KEYRING_BACKEND", "SEI_SIDECAR_AUTHN_MODE"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error should name %s; got %q", want, err.Error()) + } + } + return + } + if err != nil { + t.Errorf("expected nil, got %v", err) + } + }) + } +} + +// AuthnModeUnauthenticated is the empty string, so an unset SEI_SIDECAR_AUTHN_MODE +// and an explicit "unauthenticated" reach the guard identically. Pinned because +// the guard's correctness depends on it: if the constant ever gained a non-empty +// value, an unset variable would stop matching and the refusal would go quiet. +func TestUnauthenticatedIsTheZeroValue(t *testing.T) { + if server.AuthnModeUnauthenticated != "" { + t.Fatalf("AuthnModeUnauthenticated = %q, want the empty string; "+ + "checkKeyringNeedsAuthn relies on an unset env var resolving to it", + server.AuthnModeUnauthenticated) + } +} diff --git a/sidecar/tasks/assemble_genesis.go b/sidecar/tasks/assemble_genesis.go new file mode 100644 index 00000000..4b7c2468 --- /dev/null +++ b/sidecar/tasks/assemble_genesis.go @@ -0,0 +1,702 @@ +package tasks + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + + tmcfg "github.com/sei-protocol/sei-chain/sei-tendermint/config" + tmtypes "github.com/sei-protocol/sei-chain/sei-tendermint/types" + + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + authtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/types" + vestingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/vesting/types" + banktypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/bank/types" + "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil" + genutiltypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil/types" + stakingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/staking/types" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + "github.com/aws/aws-sdk-go-v2/service/s3" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/wire" +) + +var assembleLog = seilog.NewLogger("seictl", "task", "assemble-genesis") + +const assembleMarkerFile = ".sei-sidecar-assemble-done" + +// assembledGentxSubdir holds the assembler's downloaded gentxs, kept separate +// from config/gentx/ (which generate-gentx and upload-genesis-artifacts use) so +// the assembler can't collect its own gentx-.json twice. +const assembledGentxSubdir = "gentx-assembled" + +// maxGentxBytes caps a single gentx download (a gentx is a few KB) so a wrong or +// oversized S3 object can't be read wholesale into memory. +const maxGentxBytes = 1 << 20 // 1 MiB + +// GenesisAssembler downloads per-node gentx files from S3, calls +// genutil.GenAppStateFromConfig (the same function as seid collect-gentxs) +// to produce the final genesis.json, and uploads it back to S3 for all +// validators to download. +type GenesisAssembler struct { + homeDir string + bucket string + region string + chainID string + s3ClientFactory S3ClientFactory + s3UploaderFactory seis3.UploaderFactory +} + +// AssembleNodeEntry represents a single node in the "nodes" list param. +type AssembleNodeEntry struct { + Name string `json:"name"` +} + +// GenesisAccountEntry and GenesisAccountVesting are the wire contract, aliased +// here so handler code keeps writing the bare names. sidecar/client aliases the +// same definitions, so the payload this package unmarshals and the request the +// client builds are one type — their json tags cannot drift apart. +type ( + GenesisAccountEntry = wire.GenesisAccountEntry + GenesisAccountVesting = wire.GenesisAccountVesting +) + +// AssembleGenesisResult is the task's structured result, emitted in-band over +// the trusted controller↔sidecar task-result channel. GenesisHash is the bare +// SHA-256 hex digest (no "sha256:" prefix) of the exact uploaded genesis.json +// bytes; the controller stamps status.genesisHash from it and plumbs it into +// followers' ConfigureGenesisTask.ExpectedGenesisHash. +type AssembleGenesisResult struct { + GenesisHash string `json:"genesisHash"` +} + +// AssembleGenesisRequest holds the typed parameters for the assemble-and-upload-genesis task. +// S3 bucket, region, and prefix are derived from the sidecar's environment. +// +// Overrides is a flat map of dotted-path keys into genesis.app_state to +// raw JSON values. The first dotted token is the cosmos module name (a key +// in app_state); subsequent tokens walk into that module's JSON tree. The +// leaf value is replaced verbatim with the supplied json.RawMessage. The +// controller enforces immutability of these keys post-bootstrap via CEL; +// the sidecar applies them once during the genesis ceremony. +type AssembleGenesisRequest struct { + AccountBalance string `json:"accountBalance"` + Namespace string `json:"namespace"` + Nodes []AssembleNodeEntry `json:"nodes"` + Accounts []GenesisAccountEntry `json:"accounts,omitempty"` + Overrides map[string]json.RawMessage `json:"overrides,omitempty"` +} + +// nodeNames returns the list of node name strings from the Nodes entries. +func (c AssembleGenesisRequest) nodeNames() []string { + names := make([]string, len(c.Nodes)) + for i, n := range c.Nodes { + names[i] = n.Name + } + return names +} + +// NewGenesisAssembler creates an assembler targeting the given home directory. +func NewGenesisAssembler(homeDir, bucket, region, chainID string, s3Factory S3ClientFactory, uploaderFactory seis3.UploaderFactory) *GenesisAssembler { + if s3Factory == nil { + s3Factory = DefaultS3ClientFactory + } + if uploaderFactory == nil { + uploaderFactory = seis3.DefaultUploaderFactory + } + return &GenesisAssembler{ + homeDir: homeDir, + bucket: bucket, + region: region, + chainID: chainID, + s3ClientFactory: s3Factory, + s3UploaderFactory: uploaderFactory, + } +} + +// Handler returns an engine.TaskHandler for the assemble-and-upload-genesis task type. +// S3 coordinates are derived from the sidecar's environment. +func (a *GenesisAssembler) Handler() engine.TaskHandler { + return engine.TypedHandlerWithResult(func(ctx context.Context, cfg AssembleGenesisRequest) (*AssembleGenesisResult, error) { + if markerExists(a.homeDir, assembleMarkerFile) { + assembleLog.Debug("already completed, skipping") + return nil, nil + } + + if cfg.AccountBalance == "" { + return nil, fmt.Errorf("assemble-genesis: missing required param 'accountBalance'") + } + if cfg.Namespace == "" { + return nil, fmt.Errorf("assemble-genesis: missing required param 'namespace'") + } + if len(cfg.Nodes) == 0 { + return nil, fmt.Errorf("assemble-genesis: 'nodes' list is empty") + } + for i, n := range cfg.Nodes { + if n.Name == "" { + return nil, fmt.Errorf("assemble-genesis: nodes[%d] missing required field 'name'", i) + } + } + + nodes := cfg.nodeNames() + + if err := a.downloadGentxFiles(ctx, cfg, nodes); err != nil { + return nil, err + } + + if err := a.verifyAssembledGentxs(nodes); err != nil { + return nil, err + } + + if err := a.addMissingGenesisAccounts(cfg.AccountBalance); err != nil { + return nil, err + } + + if err := a.addExternalGenesisAccounts(cfg.Accounts); err != nil { + return nil, err + } + + if err := a.collectGentxs(); err != nil { + return nil, err + } + + if err := a.applyOverrides(cfg.Overrides); err != nil { + return nil, err + } + + if err := a.populateGenesisValidators(); err != nil { + return nil, err + } + + genesisHash, err := a.uploadGenesis(ctx, cfg) + if err != nil { + return nil, err + } + + if err := a.uploadPeers(ctx, cfg, nodes); err != nil { + return nil, err + } + + if err := writeMarker(a.homeDir, assembleMarkerFile); err != nil { + return nil, err + } + + // Hand the hash to the controller over the trusted task-result + // channel (GET /v0/tasks/{id}); never via shared S3. + assembleLog.Info("genesis assembled and uploaded", "nodes", len(nodes), "genesisHash", genesisHash) + return &AssembleGenesisResult{GenesisHash: genesisHash}, nil + }) +} + +// assembledGentxDir is the isolated dir the assembler collects from; see +// assembledGentxSubdir. +func (a *GenesisAssembler) assembledGentxDir() string { + return filepath.Join(a.homeDir, "config", assembledGentxSubdir) +} + +// downloadGentxFiles wipes the assemble dir and refills it with exactly one +// gentx per node from S3, so collect reads precisely the downloaded set — never +// this node's own generate-gentx output or a leftover from a prior run. +func (a *GenesisAssembler) downloadGentxFiles(ctx context.Context, cfg AssembleGenesisRequest, nodes []string) error { + s3Client, err := a.s3ClientFactory(ctx, a.region) + if err != nil { + return fmt.Errorf("assemble-genesis: building S3 client: %w", err) + } + + gentxDir := a.assembledGentxDir() + if err := os.RemoveAll(gentxDir); err != nil { + return fmt.Errorf("assemble-genesis: clearing assemble dir: %w", err) + } + if err := os.MkdirAll(gentxDir, 0o755); err != nil { + return fmt.Errorf("assemble-genesis: creating assemble dir: %w", err) + } + + prefix := a.chainID + "/" + + for _, nodeName := range nodes { + key := fmt.Sprintf("%s%s/gentx.json", prefix, nodeName) + assembleLog.Info("downloading gentx", "node", nodeName, "key", key) + + output, err := s3Client.GetObject(ctx, &s3.GetObjectInput{ + Bucket: aws.String(a.bucket), + Key: aws.String(key), + }) + if err != nil { + return seis3.ClassifyS3Error("assemble-and-upload-genesis", a.bucket, key, a.region, err) + } + + data, err := io.ReadAll(io.LimitReader(output.Body, maxGentxBytes+1)) + _ = output.Body.Close() + if err != nil { + return fmt.Errorf("assemble-genesis: reading %s: %w", key, err) + } + if len(data) > maxGentxBytes { + return fmt.Errorf("assemble-genesis: gentx %s exceeds %d bytes", key, maxGentxBytes) + } + + destPath := filepath.Join(gentxDir, fmt.Sprintf("gentx-%s.json", nodeName)) + if err := os.WriteFile(destPath, data, 0o644); err != nil { + return fmt.Errorf("assemble-genesis: writing %s: %w", destPath, err) + } + } + + assembleLog.Info("all gentx files downloaded", "count", len(nodes)) + return nil +} + +// verifyAssembledGentxs requires exactly one MsgCreateValidator per expected +// node, all for distinct validators, before genesis is mutated. Iterating node +// names also catches a missing gentx. gentxs are self-delegating, so a validator +// is identified equally by delegator, operator address, and consensus pubkey; +// any collision means it'd be created twice, which panics/aborts InitChain and +// wedges the chain. Rejecting here yields a clear error instead. +func (a *GenesisAssembler) verifyAssembledGentxs(nodes []string) error { + _, txCfg := makeCodec() + ensureBech32() + + gentxDir := a.assembledGentxDir() + + // Each map records identity -> node name, so a collision names both nodes. + seenDelegator := make(map[string]string, len(nodes)) + seenValidator := make(map[string]string, len(nodes)) + seenPubKey := make(map[string]string, len(nodes)) + for _, nodeName := range nodes { + data, err := os.ReadFile(filepath.Join(gentxDir, fmt.Sprintf("gentx-%s.json", nodeName))) + if err != nil { + return fmt.Errorf("assemble-genesis: reading gentx for node %s: %w", nodeName, err) + } + tx, err := txCfg.TxJSONDecoder()(data) + if err != nil { + return fmt.Errorf("assemble-genesis: decoding gentx for node %s: %w", nodeName, err) + } + msgs := tx.GetMsgs() + if len(msgs) != 1 { + return fmt.Errorf("assemble-genesis: gentx for node %s has %d messages, want exactly 1 MsgCreateValidator", nodeName, len(msgs)) + } + msg, ok := msgs[0].(*stakingtypes.MsgCreateValidator) + if !ok { + return fmt.Errorf("assemble-genesis: gentx for node %s is not a MsgCreateValidator", nodeName) + } + + // Guard nil so a malformed gentx can't panic before x/staking rejects it. + pubKey := "" + if msg.Pubkey != nil { + pubKey = msg.Pubkey.String() + } + + dedupe := func(kind, key string, seen map[string]string) error { + if key == "" { + return nil + } + if prev, dup := seen[key]; dup { + return fmt.Errorf("assemble-genesis: duplicate %s %q (nodes %s and %s); "+ + "the same validator would be created twice and abort InitChain", + kind, key, prev, nodeName) + } + seen[key] = nodeName + return nil + } + if err := dedupe("delegator", msg.DelegatorAddress, seenDelegator); err != nil { + return err + } + if err := dedupe("validator operator address", msg.ValidatorAddress, seenValidator); err != nil { + return err + } + if err := dedupe("consensus pubkey", pubKey, seenPubKey); err != nil { + return err + } + } + + assembleLog.Info("assembled gentxs verified", "count", len(nodes)) + return nil +} + +// addMissingGenesisAccounts parses each downloaded gentx to extract the +// delegator address, then adds any accounts not already present in the +// assembler's local genesis.json. This is necessary because each node only +// adds its own account during generate-gentx, but collect-gentxs validates +// that every gentx's delegator exists in genesis state. +func (a *GenesisAssembler) addMissingGenesisAccounts(accountBalance string) error { + cdc, txCfg := makeCodec() + ensureBech32() + + gentxDir := a.assembledGentxDir() + entries, err := os.ReadDir(gentxDir) + if err != nil { + return fmt.Errorf("assemble-genesis: reading assemble dir: %w", err) + } + + genFile := filepath.Join(a.homeDir, "config", "genesis.json") + appState, genDoc, err := genutiltypes.GenesisStateFromGenFile(genFile) + if err != nil { + return fmt.Errorf("assemble-genesis: reading genesis: %w", err) + } + + authGenState := authtypes.GetGenesisStateFromAppState(cdc, appState) + accs, err := authtypes.UnpackAccounts(authGenState.Accounts) + if err != nil { + return fmt.Errorf("assemble-genesis: unpacking accounts: %w", err) + } + + bankGenState := banktypes.GetGenesisStateFromAppState(cdc, appState) + + coins, err := sdk.ParseCoinsNormalized(accountBalance) + if err != nil { + return fmt.Errorf("assemble-genesis: parsing account balance: %w", err) + } + + added := 0 + for _, entry := range entries { + if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" { + continue + } + + data, err := os.ReadFile(filepath.Join(gentxDir, entry.Name())) + if err != nil { + return fmt.Errorf("assemble-genesis: reading %s: %w", entry.Name(), err) + } + + tx, err := txCfg.TxJSONDecoder()(data) + if err != nil { + return fmt.Errorf("assemble-genesis: decoding %s: %w", entry.Name(), err) + } + + msgs := tx.GetMsgs() + if len(msgs) == 0 { + continue + } + msg, ok := msgs[0].(*stakingtypes.MsgCreateValidator) + if !ok { + continue + } + + addr, err := sdk.AccAddressFromBech32(msg.DelegatorAddress) + if err != nil { + return fmt.Errorf("assemble-genesis: parsing delegator address from %s: %w", entry.Name(), err) + } + + if accs.Contains(addr) { + continue + } + + accs = append(accs, authtypes.NewBaseAccount(addr, nil, 0, 0)) + bankGenState.Balances = append(bankGenState.Balances, banktypes.Balance{ + Address: addr.String(), + Coins: coins.Sort(), + }) + added++ + assembleLog.Info("added missing genesis account", "address", addr.String()) + } + + if added == 0 { + return nil + } + + if err := writeBackAuthAndBank(cdc, genFile, genDoc, appState, authGenState, accs, bankGenState); err != nil { + return fmt.Errorf("assemble-genesis: %w", err) + } + + assembleLog.Info("genesis accounts reconciled", "added", added, "total", len(accs)) + return nil +} + +// Run after addMissingGenesisAccounts so collisions catch validator- +// derived addresses. Non-fork's empty Supply lets bank.InitGenesis +// recompute from balances; the fork mirror updates Supply explicitly. +func (a *GenesisAssembler) addExternalGenesisAccounts(accounts []GenesisAccountEntry) error { + if len(accounts) == 0 { + return nil + } + + cdc, _ := makeCodec() + ensureBech32() + + genFile := filepath.Join(a.homeDir, "config", "genesis.json") + appState, genDoc, err := genutiltypes.GenesisStateFromGenFile(genFile) + if err != nil { + return fmt.Errorf("assemble-genesis: reading genesis: %w", err) + } + + authGenState := authtypes.GetGenesisStateFromAppState(cdc, appState) + accs, err := authtypes.UnpackAccounts(authGenState.Accounts) + if err != nil { + return fmt.Errorf("assemble-genesis: unpacking accounts: %w", err) + } + + bankGenState := banktypes.GetGenesisStateFromAppState(cdc, appState) + + for _, entry := range accounts { + addr, err := sdk.AccAddressFromBech32(entry.Address) + if err != nil { + return fmt.Errorf("assemble-genesis: external account %q: %w", entry.Address, err) + } + if accs.Contains(addr) { + return fmt.Errorf("assemble-genesis: external account %s collides with an existing genesis account", addr.String()) + } + coins, err := sdk.ParseCoinsNormalized(entry.Balance) + if err != nil { + return fmt.Errorf("assemble-genesis: external account %s balance %q: %w", addr.String(), entry.Balance, err) + } + + baseAccount := authtypes.NewBaseAccount(addr, nil, 0, 0) + var acc authtypes.GenesisAccount = baseAccount + + if entry.Vesting != nil { + vestingCoins, err := sdk.ParseCoinsNormalized(entry.Vesting.Amount) + if err != nil { + return fmt.Errorf("assemble-genesis: external account %s vesting amount %q: %w", addr.String(), entry.Vesting.Amount, err) + } + // IsAllPositive (not Empty): ParseCoinsNormalized truncates a + // fractional base-denom amount ("0.4usei") to {usei:0}, which is + // non-empty yet locks nothing — Empty() would miss it. IsAllPositive + // is false for both the empty and the zero-valued case. + if !vestingCoins.IsAllPositive() { + return fmt.Errorf("assemble-genesis: external account %s vesting amount %q must be a positive coin amount (fractional base-denom values truncate to zero)", addr.String(), entry.Vesting.Amount) + } + if !coins.IsAllGTE(vestingCoins) { + return fmt.Errorf("assemble-genesis: external account %s vesting amount %s exceeds balance %s", addr.String(), vestingCoins, coins) + } + // EndTime must be strictly after genesis time for both + // schedules: a continuous account with EndTime <= StartTime + // is fully vested from block 1 (ContinuousVestingAccount's own + // Validate rejects StartTime >= EndTime, but only if something + // calls Validate — see below); a delayed account has no + // start/end ordering check at all, so an EndTime in the past + // silently unlocks everything at genesis with no error either + // way. Both would defeat the entire purpose of a vesting + // fixture: a balance that's supposed to be locked. + if entry.Vesting.EndTime <= genDoc.GenesisTime.Unix() { + return fmt.Errorf("assemble-genesis: external account %s vesting end time %d must be after genesis time %d", + addr.String(), entry.Vesting.EndTime, genDoc.GenesisTime.Unix()) + } + // Mirrors msg_server.go's (deprecated-for-live-tx) CreateVestingAccount + // handler, with the genesis timestamp standing in for + // ctx.BlockTime() (no live "now" at genesis-assembly time) and no + // admin (no live tx to have named one). + baseVestingAccount := vestingtypes.NewBaseVestingAccount(baseAccount, vestingCoins.Sort(), entry.Vesting.EndTime, nil) + if entry.Vesting.Delayed { + acc = vestingtypes.NewDelayedVestingAccountRaw(baseVestingAccount) + } else { + acc = vestingtypes.NewContinuousVestingAccountRaw(baseVestingAccount, genDoc.GenesisTime.Unix()) + } + // Belt-and-suspenders: auth.InitGenesis never calls Validate() + // on genesis accounts (only the separate validate-genesis CLI + // path does), so an invalid vesting schedule would otherwise + // reach InitChain unguarded. + if err := acc.Validate(); err != nil { + return fmt.Errorf("assemble-genesis: external account %s vesting schedule: %w", addr.String(), err) + } + } + + accs = append(accs, acc) + bankGenState.Balances = append(bankGenState.Balances, banktypes.Balance{ + Address: addr.String(), + Coins: coins.Sort(), + }) + assembleLog.Info("added external genesis account", "address", addr.String(), "balance", entry.Balance, "vesting", entry.Vesting != nil) + } + + if err := writeBackAuthAndBank(cdc, genFile, genDoc, appState, authGenState, accs, bankGenState); err != nil { + return fmt.Errorf("assemble-genesis: %w", err) + } + return nil +} + +// collectGentxs calls genutil.GenAppStateFromConfig — the exact same +// function behind seid collect-gentxs. It decodes each gentx through +// the proto codec, validates balances, extracts persistent peers, +// and writes the final genesis.json. +func (a *GenesisAssembler) collectGentxs() error { + assembleLog.Info("running collect-gentxs via SDK") + + cdc, txCfg := makeCodec() + ensureBech32() + + cfg := tmcfg.DefaultConfig() + cfg.SetRoot(a.homeDir) + + nodeID, valPubKey, err := genutil.InitializeNodeValidatorFiles(cfg) + if err != nil { + return fmt.Errorf("assemble-genesis: loading validator files: %w", err) + } + + genDoc, err := tmtypes.GenesisDocFromFile(cfg.GenesisFile()) + if err != nil { + return fmt.Errorf("assemble-genesis: reading genesis: %w", err) + } + + gentxsDir := a.assembledGentxDir() + initCfg := genutiltypes.NewInitConfig(genDoc.ChainID, gentxsDir, nodeID, valPubKey) + + genBalIterator := banktypes.GenesisBalancesIterator{} + + _, err = genutil.GenAppStateFromConfig(cdc, txCfg, cfg, initCfg, *genDoc, genBalIterator) + if err != nil { + return fmt.Errorf("assemble-genesis: collect-gentxs: %w", err) + } + + return nil +} + +// applyOverrides re-reads the assembled genesis.json, applies the +// caller-supplied app_state overrides, and writes the file back. This runs +// after collectGentxs so the dispatched MsgCreateValidator data and +// derived persistent peers are already baked into app_state — overrides +// are an in-place patch on the final assembled doc. +func (a *GenesisAssembler) applyOverrides(overrides map[string]json.RawMessage) error { + if len(overrides) == 0 { + return nil + } + + genFile := filepath.Join(a.homeDir, "config", "genesis.json") + genDoc, err := tmtypes.GenesisDocFromFile(genFile) + if err != nil { + return fmt.Errorf("assemble-genesis: reading genesis for overrides: %w", err) + } + + var appState map[string]json.RawMessage + if err := json.Unmarshal(genDoc.AppState, &appState); err != nil { + return fmt.Errorf("assemble-genesis: parsing app_state for overrides: %w", err) + } + + if err := applyGenesisOverrides(appState, overrides); err != nil { + return fmt.Errorf("assemble-genesis: %w", err) + } + + appStateJSON, err := json.Marshal(appState) + if err != nil { + return fmt.Errorf("assemble-genesis: marshaling overridden app_state: %w", err) + } + genDoc.AppState = appStateJSON + + if err := genutil.ExportGenesisFile(genDoc, genFile); err != nil { + return fmt.Errorf("assemble-genesis: writing genesis after overrides: %w", err) + } + + assembleLog.Info("applied genesis overrides", "count", len(overrides)) + return nil +} + +// uploadGenesis reads the assembled genesis.json and uploads it to S3 +// at /genesis.json where all validators will fetch it from. It +// returns the bare SHA-256 hex digest (no "sha256:" prefix) computed over the +// exact bytes uploaded — the same []byte handed to PutObject, not a re-read or +// re-serialized form — so the digest matches what a follower will download and +// verify. The digest travels to the controller only in-band, over the trusted +// task-result channel; it is never written to S3, where the prefix is +// attacker-writable and a sibling hash would let a poisoned genesis carry its +// own matching digest. +func (a *GenesisAssembler) uploadGenesis(ctx context.Context, cfg AssembleGenesisRequest) (string, error) { + genesisPath := filepath.Join(a.homeDir, "config", "genesis.json") + data, err := os.ReadFile(genesisPath) + if err != nil { + return "", fmt.Errorf("assemble-genesis: reading genesis.json: %w", err) + } + + sum := sha256.Sum256(data) + genesisHash := hex.EncodeToString(sum[:]) + + uploader, err := a.s3UploaderFactory(ctx, a.region) + if err != nil { + return "", fmt.Errorf("assemble-genesis: building S3 uploader: %w", err) + } + + key := a.chainID + "/" + "genesis.json" + assembleLog.Info("uploading assembled genesis", "key", key, "sha256", genesisHash) + + _, err = uploader.UploadObject(ctx, &transfermanager.UploadObjectInput{ + Bucket: aws.String(a.bucket), + Key: aws.String(key), + Body: bytes.NewReader(data), + ContentType: aws.String("application/json"), + }) + if err != nil { + return "", seis3.ClassifyS3Error("assemble-and-upload-genesis", a.bucket, key, a.region, err) + } + + return genesisHash, nil +} + +// uploadPeers builds a peers.json from each node's identity.json and uploads +// it to S3 alongside genesis.json. Each entry is a full Tendermint peer address +// using in-cluster DNS: @-0...svc.cluster.local:26656 +func (a *GenesisAssembler) uploadPeers(ctx context.Context, cfg AssembleGenesisRequest, nodes []string) error { + s3Client, err := a.s3ClientFactory(ctx, a.region) + if err != nil { + return fmt.Errorf("assemble-genesis: building S3 client for peers: %w", err) + } + + prefix := a.chainID + "/" + var peers []string + + for _, nodeName := range nodes { + key := fmt.Sprintf("%s%s/identity.json", prefix, nodeName) + output, err := s3Client.GetObject(ctx, &s3.GetObjectInput{ + Bucket: aws.String(a.bucket), + Key: aws.String(key), + }) + if err != nil { + return seis3.ClassifyS3Error("assemble-and-upload-genesis", a.bucket, key, a.region, err) + } + data, err := io.ReadAll(output.Body) + _ = output.Body.Close() + if err != nil { + return fmt.Errorf("assemble-genesis: reading identity for %s: %w", nodeName, err) + } + + var identity struct { + NodeKey json.RawMessage `json:"node_key"` + } + if err := json.Unmarshal(data, &identity); err != nil { + return fmt.Errorf("assemble-genesis: parsing identity for %s: %w", nodeName, err) + } + + var nodeKey struct { + ID string `json:"id"` + } + if err := json.Unmarshal(identity.NodeKey, &nodeKey); err != nil { + return fmt.Errorf("assemble-genesis: parsing node_key for %s: %w", nodeName, err) + } + if nodeKey.ID == "" { + return fmt.Errorf("assemble-genesis: empty node ID for %s", nodeName) + } + + dns := fmt.Sprintf("%s-0.%s.%s.svc.cluster.local", nodeName, nodeName, cfg.Namespace) + peers = append(peers, fmt.Sprintf("%s@%s:26656", nodeKey.ID, dns)) + } + + peersJSON, err := json.Marshal(peers) + if err != nil { + return fmt.Errorf("assemble-genesis: marshaling peers.json: %w", err) + } + + uploader, err := a.s3UploaderFactory(ctx, a.region) + if err != nil { + return fmt.Errorf("assemble-genesis: building S3 uploader for peers: %w", err) + } + + peersKey := prefix + "peers.json" + assembleLog.Info("uploading peers.json", "key", peersKey, "count", len(peers)) + + _, err = uploader.UploadObject(ctx, &transfermanager.UploadObjectInput{ + Bucket: aws.String(a.bucket), + Key: aws.String(peersKey), + Body: bytes.NewReader(peersJSON), + ContentType: aws.String("application/json"), + }) + if err != nil { + return seis3.ClassifyS3Error("assemble-and-upload-genesis", a.bucket, peersKey, a.region, err) + } + return nil +} diff --git a/sidecar/tasks/assemble_genesis_dedup_test.go b/sidecar/tasks/assemble_genesis_dedup_test.go new file mode 100644 index 00000000..8669ec5c --- /dev/null +++ b/sidecar/tasks/assemble_genesis_dedup_test.go @@ -0,0 +1,149 @@ +package tasks + +import ( + "os" + "path/filepath" + "strings" + "testing" + + codectypes "github.com/sei-protocol/sei-chain/sei-cosmos/codec/types" + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/keys/secp256k1" + cryptotypes "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/types" + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + stakingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/staking/types" +) + +// randomSeiAddr returns a fresh bech32 "sei" account address. +func randomSeiAddr(t *testing.T) string { + t.Helper() + ensureBech32() + return sdk.AccAddress(secp256k1.GenPrivKey().PubKey().Address()).String() +} + +// writeGentxFixture writes a minimal-but-decodable MsgCreateValidator gentx for +// the given delegator into dir/filename with a fresh, unique consensus pubkey. +func writeGentxFixture(t *testing.T, dir, filename, delegator string) { + t.Helper() + writeGentxFixtureWithPubKey(t, dir, filename, delegator, secp256k1.GenPrivKey().PubKey()) +} + +// writeGentxFixtureWithPubKey is writeGentxFixture with an explicit consensus +// pubkey, so a test can force two gentxs to share one — the copied-validator-key +// case the dedup guard must reject. Signatures are irrelevant to the invariant +// (it inspects only delegator, operator address, and pubkey), so the tx is left +// unsigned. +func writeGentxFixtureWithPubKey(t *testing.T, dir, filename, delegator string, pk cryptotypes.PubKey) { + t.Helper() + ensureBech32() + _, txCfg := makeCodec() + + addr, err := sdk.AccAddressFromBech32(delegator) + if err != nil { + t.Fatalf("parsing delegator %q: %v", delegator, err) + } + + pkAny, err := codectypes.NewAnyWithValue(pk) + if err != nil { + t.Fatalf("packing pubkey: %v", err) + } + + msg := &stakingtypes.MsgCreateValidator{ + Description: stakingtypes.NewDescription("moniker", "", "", "", ""), + Commission: stakingtypes.NewCommissionRates(sdk.ZeroDec(), sdk.ZeroDec(), sdk.ZeroDec()), + MinSelfDelegation: sdk.OneInt(), + DelegatorAddress: addr.String(), + ValidatorAddress: sdk.ValAddress(addr).String(), + Pubkey: pkAny, + Value: sdk.NewCoin("usei", sdk.NewInt(1)), + } + + txb := txCfg.NewTxBuilder() + if err := txb.SetMsgs(msg); err != nil { + t.Fatalf("set msgs: %v", err) + } + bz, err := txCfg.TxJSONEncoder()(txb.GetTx()) + if err != nil { + t.Fatalf("encoding gentx: %v", err) + } + + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatalf("mkdir: %v", err) + } + if err := os.WriteFile(filepath.Join(dir, filename), bz, 0o644); err != nil { + t.Fatalf("write gentx: %v", err) + } +} + +// TestVerifyAssembledGentxs_DuplicateDelegator: two gentxs for the same +// delegator (the failure that panicked InitChain with "account sequence +// mismatch, expected 1, got 0") must be rejected up front rather than producing +// a chain-wedging genesis. +func TestVerifyAssembledGentxs_DuplicateDelegator(t *testing.T) { + homeDir := t.TempDir() + a := NewGenesisAssembler(homeDir, "b", "r", "chain", nil, nil) + dir := a.assembledGentxDir() + + dup := randomSeiAddr(t) + writeGentxFixture(t, dir, "gentx-val-0.json", dup) + writeGentxFixture(t, dir, "gentx-val-1.json", dup) // same delegator → duplicate + + err := a.verifyAssembledGentxs([]string{"val-0", "val-1"}) + if err == nil { + t.Fatal("expected duplicate-delegator error, got nil") + } + if !strings.Contains(err.Error(), "duplicate delegator") { + t.Errorf("error = %q, want substring 'duplicate delegator'", err.Error()) + } +} + +// TestVerifyAssembledGentxs_DuplicateConsensusPubKey covers the copied-key case: +// distinct delegators that share one consensus pubkey would abort InitChain in +// x/staking, so the guard must reject it even though the delegators differ. +func TestVerifyAssembledGentxs_DuplicateConsensusPubKey(t *testing.T) { + homeDir := t.TempDir() + a := NewGenesisAssembler(homeDir, "b", "r", "chain", nil, nil) + dir := a.assembledGentxDir() + + shared := secp256k1.GenPrivKey().PubKey() + writeGentxFixtureWithPubKey(t, dir, "gentx-val-0.json", randomSeiAddr(t), shared) + writeGentxFixtureWithPubKey(t, dir, "gentx-val-1.json", randomSeiAddr(t), shared) + + err := a.verifyAssembledGentxs([]string{"val-0", "val-1"}) + if err == nil { + t.Fatal("expected duplicate-consensus-pubkey error, got nil") + } + if !strings.Contains(err.Error(), "duplicate consensus pubkey") { + t.Errorf("error = %q, want substring 'duplicate consensus pubkey'", err.Error()) + } +} + +func TestVerifyAssembledGentxs_DistinctOK(t *testing.T) { + homeDir := t.TempDir() + a := NewGenesisAssembler(homeDir, "b", "r", "chain", nil, nil) + dir := a.assembledGentxDir() + + writeGentxFixture(t, dir, "gentx-val-0.json", randomSeiAddr(t)) + writeGentxFixture(t, dir, "gentx-val-1.json", randomSeiAddr(t)) + + if err := a.verifyAssembledGentxs([]string{"val-0", "val-1"}); err != nil { + t.Fatalf("expected distinct gentxs to pass, got %v", err) + } +} + +// TestVerifyAssembledGentxs_MissingGentx covers the by-name 1:1 assertion: an +// expected node with no gentx in the assemble dir must fail before mutation. +func TestVerifyAssembledGentxs_MissingGentx(t *testing.T) { + homeDir := t.TempDir() + a := NewGenesisAssembler(homeDir, "b", "r", "chain", nil, nil) + dir := a.assembledGentxDir() + + writeGentxFixture(t, dir, "gentx-val-0.json", randomSeiAddr(t)) // val-1 absent + + err := a.verifyAssembledGentxs([]string{"val-0", "val-1"}) + if err == nil { + t.Fatal("expected missing-gentx error, got nil") + } + if !strings.Contains(err.Error(), "reading gentx for node val-1") { + t.Errorf("error = %q, want substring 'reading gentx for node val-1'", err.Error()) + } +} diff --git a/sidecar/tasks/assemble_genesis_external_test.go b/sidecar/tasks/assemble_genesis_external_test.go new file mode 100644 index 00000000..a2eb07e9 --- /dev/null +++ b/sidecar/tasks/assemble_genesis_external_test.go @@ -0,0 +1,467 @@ +package tasks + +import ( + "os" + "path/filepath" + "strings" + "testing" + "time" + + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + authtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/types" + vestingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/vesting/types" + banktypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/bank/types" + genutiltypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil/types" +) + +// minimalGenesis writes a Tendermint genesis.json with empty app_state; +// GetGenesisStateFromAppState defaults absent module keys, so this is +// enough for auth+bank append tests. +func minimalGenesis(t *testing.T, homeDir string) string { + t.Helper() + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatalf("mkdir: %v", err) + } + genFile := filepath.Join(configDir, "genesis.json") + body := `{ + "chain_id": "test-chain-1", + "genesis_time": "2026-01-01T00:00:00Z", + "initial_height": "1", + "consensus_params": { + "block": {"max_bytes": "22020096", "max_gas": "-1"}, + "evidence": {"max_age_num_blocks": "100000", "max_age_duration": "172800000000000", "max_bytes": "1048576"}, + "validator": {"pub_key_types": ["ed25519"]}, + "version": {} + }, + "validators": [], + "app_hash": "", + "app_state": {} + }` + if err := os.WriteFile(genFile, []byte(body), 0o644); err != nil { + t.Fatalf("write genesis: %v", err) + } + return genFile +} + +func readBankBalances(t *testing.T, genFile string) []banktypes.Balance { + t.Helper() + cdc, _ := makeCodec() + appState, _, err := genutiltypes.GenesisStateFromGenFile(genFile) + if err != nil { + t.Fatalf("reading genesis: %v", err) + } + bank := banktypes.GetGenesisStateFromAppState(cdc, appState) + return bank.Balances +} + +func readAuthAccountAddrs(t *testing.T, genFile string) []string { + t.Helper() + cdc, _ := makeCodec() + appState, _, err := genutiltypes.GenesisStateFromGenFile(genFile) + if err != nil { + t.Fatalf("reading genesis: %v", err) + } + auth := authtypes.GetGenesisStateFromAppState(cdc, appState) + accs, err := authtypes.UnpackAccounts(auth.Accounts) + if err != nil { + t.Fatalf("unpacking accounts: %v", err) + } + out := make([]string, 0, len(accs)) + for _, a := range accs { + out = append(out, a.GetAddress().String()) + } + return out +} + +func TestAddExternalGenesisAccounts_NoOpOnEmpty(t *testing.T) { + homeDir := t.TempDir() + genFile := minimalGenesis(t, homeDir) + mtimeBefore := mustModTime(t, genFile) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + if err := a.addExternalGenesisAccounts(nil); err != nil { + t.Fatalf("nil accounts: %v", err) + } + if err := a.addExternalGenesisAccounts([]GenesisAccountEntry{}); err != nil { + t.Fatalf("empty accounts: %v", err) + } + + if mustModTime(t, genFile) != mtimeBefore { + t.Errorf("genesis.json was rewritten on no-op input") + } +} + +func TestAddExternalGenesisAccounts_AppendsBalanceAndAccount(t *testing.T) { + homeDir := t.TempDir() + genFile := minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + addr := "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9" + bal := "1000000000usei" + + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{Address: addr, Balance: bal}}) + if err != nil { + t.Fatalf("add: %v", err) + } + + balances := readBankBalances(t, genFile) + found := false + for _, b := range balances { + if b.Address == addr { + found = true + if b.Coins.String() != "1000000000usei" { + t.Errorf("balance: got %s, want %s", b.Coins.String(), bal) + } + break + } + } + if !found { + t.Errorf("address %s not found in bank.balances; got %v", addr, balances) + } + + addrs := readAuthAccountAddrs(t, genFile) + if !containsAddr(addrs, addr) { + t.Errorf("address %s not found in auth.accounts; got %v", addr, addrs) + } +} + +func TestAddExternalGenesisAccounts_CollisionHardFails(t *testing.T) { + homeDir := t.TempDir() + _ = minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + addr := "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9" + + // First add succeeds. + if err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{Address: addr, Balance: "1usei"}}); err != nil { + t.Fatalf("first add: %v", err) + } + // Second add of the same address must error. + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{Address: addr, Balance: "999usei"}}) + if err == nil { + t.Fatal("expected collision error, got nil") + } + if !strings.Contains(err.Error(), "collides") { + t.Errorf("collision error message: got %q, want substring 'collides'", err.Error()) + } +} + +func TestAddExternalGenesisAccounts_RejectsDuplicateInSameBatch(t *testing.T) { + homeDir := t.TempDir() + _ = minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + addr := "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9" + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{ + {Address: addr, Balance: "1usei"}, + {Address: addr, Balance: "2usei"}, + }) + if err == nil { + t.Fatal("expected duplicate-in-batch error, got nil") + } + if !strings.Contains(err.Error(), "collides") { + t.Errorf("error: got %q, want substring 'collides'", err.Error()) + } +} + +func TestAddExternalGenesisAccounts_CollidesWithPreSeeded(t *testing.T) { + // Mirrors the production case where a validator-derived account + // (added by addMissingGenesisAccounts) collides with an external + // account on the same address. Seed via the codec API directly, + // not via the function under test, to prove collision detection + // does not depend on which path populated auth.accounts. + homeDir := t.TempDir() + genFile := minimalGenesis(t, homeDir) + addr := "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9" + seedAuthAccount(t, genFile, addr) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{Address: addr, Balance: "1usei"}}) + if err == nil { + t.Fatal("expected collision against pre-seeded account, got nil") + } + if !strings.Contains(err.Error(), "collides") { + t.Errorf("error: got %q, want substring 'collides'", err.Error()) + } +} + +// seedAuthAccount writes a single bech32 entry into auth.accounts on the +// genesis file via the same codec path the production code uses. +func seedAuthAccount(t *testing.T, genFile, bech32 string) { + t.Helper() + cdc, _ := makeCodec() + ensureBech32() + appState, genDoc, err := genutiltypes.GenesisStateFromGenFile(genFile) + if err != nil { + t.Fatalf("reading genesis: %v", err) + } + authGenState := authtypes.GetGenesisStateFromAppState(cdc, appState) + accs, err := authtypes.UnpackAccounts(authGenState.Accounts) + if err != nil { + t.Fatalf("unpacking: %v", err) + } + addr, err := sdk.AccAddressFromBech32(bech32) + if err != nil { + t.Fatalf("parsing addr: %v", err) + } + accs = append(accs, authtypes.NewBaseAccount(addr, nil, 0, 0)) + bank := banktypes.GetGenesisStateFromAppState(cdc, appState) + if err := writeBackAuthAndBank(cdc, genFile, genDoc, appState, authGenState, accs, bank); err != nil { + t.Fatalf("seed: %v", err) + } +} + +func TestAddExternalGenesisAccounts_RejectsBadBech32(t *testing.T) { + homeDir := t.TempDir() + _ = minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{Address: "not-bech32", Balance: "1usei"}}) + if err == nil { + t.Fatal("expected bech32 error, got nil") + } +} + +func TestAddExternalGenesisAccounts_RejectsBadBalance(t *testing.T) { + homeDir := t.TempDir() + _ = minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{ + Address: "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9", + Balance: "not-a-coin", + }}) + if err == nil { + t.Fatal("expected balance parse error, got nil") + } +} + +func mustModTime(t *testing.T, path string) int64 { + t.Helper() + st, err := os.Stat(path) + if err != nil { + t.Fatalf("stat: %v", err) + } + return st.ModTime().UnixNano() +} + +func containsAddr(haystack []string, needle string) bool { + for _, h := range haystack { + if h == needle { + return true + } + } + return false +} + +// findAuthAccount returns the genesis account at addr, or nil if absent. +func findAuthAccount(t *testing.T, genFile, addr string) authtypes.GenesisAccount { + t.Helper() + cdc, _ := makeCodec() + appState, _, err := genutiltypes.GenesisStateFromGenFile(genFile) + if err != nil { + t.Fatalf("reading genesis: %v", err) + } + auth := authtypes.GetGenesisStateFromAppState(cdc, appState) + accs, err := authtypes.UnpackAccounts(auth.Accounts) + if err != nil { + t.Fatalf("unpacking accounts: %v", err) + } + for _, a := range accs { + if a.GetAddress().String() == addr { + return a + } + } + return nil +} + +func TestAddExternalGenesisAccounts_VestingContinuousByDefault(t *testing.T) { + homeDir := t.TempDir() + genFile := minimalGenesis(t, homeDir) + addr := "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9" + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{ + Address: addr, + Balance: "2000000usei", + Vesting: &GenesisAccountVesting{Amount: "1000000usei", EndTime: 1893456000}, + }}) + if err != nil { + t.Fatalf("add: %v", err) + } + + // Bank balance carries the full Balance, not just the vesting portion — + // vesting locks part of an existing balance, it isn't a separate pot. + balances := readBankBalances(t, genFile) + found := false + for _, b := range balances { + if b.Address == addr { + found = true + if b.Coins.String() != "2000000usei" { + t.Errorf("balance: got %s, want 2000000usei", b.Coins.String()) + } + } + } + if !found { + t.Fatalf("address %s not found in bank.balances", addr) + } + + acc := findAuthAccount(t, genFile, addr) + if acc == nil { + t.Fatalf("address %s not found in auth.accounts", addr) + } + cva, ok := acc.(*vestingtypes.ContinuousVestingAccount) + if !ok { + t.Fatalf("account type: got %T, want *vestingtypes.ContinuousVestingAccount", acc) + } + if cva.OriginalVesting.String() != "1000000usei" { + t.Errorf("OriginalVesting: got %s, want 1000000usei", cva.OriginalVesting.String()) + } + if cva.EndTime != 1893456000 { + t.Errorf("EndTime: got %d, want 1893456000", cva.EndTime) + } + // StartTime must be the genesis timestamp, not wall-clock "now" — there + // is no live block-time at genesis-assembly time. minimalGenesis sets + // genesis_time to 2026-01-01T00:00:00Z (1767225600). + const wantStartTime = 1767225600 + if cva.StartTime != wantStartTime { + t.Errorf("StartTime: got %d, want genesis time %d", cva.StartTime, wantStartTime) + } + // The point of the fixture: assert the coins are actually LOCKED at the + // chain's first block (block time == genesis time), not merely that the + // account is the right type. At StartTime nothing has vested, so the full + // OriginalVesting is locked; well before EndTime spendable is zero. + genesisBlock := time.Unix(wantStartTime, 0) + if locked := cva.LockedCoins(genesisBlock); locked.String() != "1000000usei" { + t.Errorf("LockedCoins at genesis block: got %s, want 1000000usei", locked.String()) + } +} + +func TestAddExternalGenesisAccounts_VestingDelayed(t *testing.T) { + homeDir := t.TempDir() + genFile := minimalGenesis(t, homeDir) + addr := "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9" + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{ + Address: addr, + Balance: "1000000usei", + Vesting: &GenesisAccountVesting{Amount: "1000000usei", EndTime: 1893456000, Delayed: true}, + }}) + if err != nil { + t.Fatalf("add: %v", err) + } + + acc := findAuthAccount(t, genFile, addr) + if acc == nil { + t.Fatalf("address %s not found in auth.accounts", addr) + } + dva, ok := acc.(*vestingtypes.DelayedVestingAccount) + if !ok { + t.Fatalf("account type: got %T, want *vestingtypes.DelayedVestingAccount", acc) + } + if dva.OriginalVesting.String() != "1000000usei" { + t.Errorf("OriginalVesting: got %s, want 1000000usei", dva.OriginalVesting.String()) + } + if dva.EndTime != 1893456000 { + t.Errorf("EndTime: got %d, want 1893456000", dva.EndTime) + } +} + +func TestAddExternalGenesisAccounts_VestingAmountExceedsBalanceRejected(t *testing.T) { + homeDir := t.TempDir() + _ = minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{ + Address: "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9", + Balance: "1000000usei", + Vesting: &GenesisAccountVesting{Amount: "2000000usei", EndTime: 1893456000}, + }}) + if err == nil { + t.Fatal("expected vesting-exceeds-balance error, got nil") + } + if !strings.Contains(err.Error(), "exceeds balance") { + t.Errorf("error: got %q, want substring 'exceeds balance'", err.Error()) + } +} + +func TestAddExternalGenesisAccounts_VestingRejectsBadAmount(t *testing.T) { + homeDir := t.TempDir() + _ = minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{ + Address: "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9", + Balance: "1000000usei", + Vesting: &GenesisAccountVesting{Amount: "not-a-coin", EndTime: 1893456000}, + }}) + if err == nil { + t.Fatal("expected vesting amount parse error, got nil") + } +} + +func TestAddExternalGenesisAccounts_VestingRejectsDegenerateAmount(t *testing.T) { + // Two ways a vesting amount can lock nothing yet pass the CRD MinLength=1 + // string check: a literal zero ("0usei", which ParseCoinsNormalized + // normalizes to empty) and a fractional base-denom amount ("0.4usei", + // which is non-empty but TRUNCATES to {usei:0}). The IsAllPositive guard + // must reject both — Empty() alone would miss the truncation case. + for _, amount := range []string{"0usei", "0.4usei"} { + t.Run(amount, func(t *testing.T) { + homeDir := t.TempDir() + _ = minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{ + Address: "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9", + Balance: "1000000usei", + Vesting: &GenesisAccountVesting{Amount: amount, EndTime: 1893456000}, + }}) + if err == nil { + t.Fatalf("amount %q: expected positive-amount error, got nil", amount) + } + if !strings.Contains(err.Error(), "must be a positive coin amount") { + t.Errorf("amount %q: error got %q, want substring 'must be a positive coin amount'", amount, err.Error()) + } + }) + } +} + +func TestAddExternalGenesisAccounts_VestingRejectsEndTimeBeforeGenesis(t *testing.T) { + // EndTime at/before genesis time yields a schedule that's already fully + // vested at chain start — the account is unlocked, defeating the fixture. + // minimalGenesis sets genesis_time to 2026-01-01T00:00:00Z (1767225600). + homeDir := t.TempDir() + _ = minimalGenesis(t, homeDir) + + a := NewGenesisAssembler(homeDir, "bucket", "region", "test-chain-1", nil, nil) + + // Continuous: EndTime exactly at genesis time. + err := a.addExternalGenesisAccounts([]GenesisAccountEntry{{ + Address: "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9", + Balance: "1000000usei", + Vesting: &GenesisAccountVesting{Amount: "1000000usei", EndTime: 1767225600}, + }}) + if err == nil { + t.Fatal("continuous: expected end-time-before-genesis error, got nil") + } + if !strings.Contains(err.Error(), "after genesis time") { + t.Errorf("continuous: error got %q, want substring 'after genesis time'", err.Error()) + } + + // Delayed with a past EndTime: no start/end ordering check in the account + // type itself, so this guard is the only thing catching it. + err = a.addExternalGenesisAccounts([]GenesisAccountEntry{{ + Address: "sei1zg69v7y6hn00qy352euf40x77qfrg4nclsjzp9", + Balance: "1000000usei", + Vesting: &GenesisAccountVesting{Amount: "1000000usei", EndTime: 1, Delayed: true}, + }}) + if err == nil { + t.Fatal("delayed: expected end-time-before-genesis error, got nil") + } + if !strings.Contains(err.Error(), "after genesis time") { + t.Errorf("delayed: error got %q, want substring 'after genesis time'", err.Error()) + } +} diff --git a/sidecar/tasks/assemble_genesis_test.go b/sidecar/tasks/assemble_genesis_test.go new file mode 100644 index 00000000..e21eb1f2 --- /dev/null +++ b/sidecar/tasks/assemble_genesis_test.go @@ -0,0 +1,379 @@ +package tasks + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/aws/aws-sdk-go-v2/service/s3" +) + +type mockS3GetObject struct { + objects map[string][]byte +} + +func (m *mockS3GetObject) GetObject(_ context.Context, input *s3.GetObjectInput, _ ...func(*s3.Options)) (*s3.GetObjectOutput, error) { + key := *input.Key + data, ok := m.objects[key] + if !ok { + return nil, fmt.Errorf("NoSuchKey: %s", key) + } + return &s3.GetObjectOutput{ + Body: io.NopCloser(bytes.NewReader(data)), + }, nil +} + +func TestAssembler_DownloadsGentxFiles(t *testing.T) { + homeDir := t.TempDir() + configDir := filepath.Join(homeDir, "config") + os.MkdirAll(configDir, 0o755) + + s3Objects := &mockS3GetObject{objects: map[string][]byte{ + "genesis/val-0/gentx.json": []byte(`{"gentx":"val0"}`), + "genesis/val-1/gentx.json": []byte(`{"gentx":"val1"}`), + }} + s3Factory := func(_ context.Context, _ string) (S3GetObjectAPI, error) { + return s3Objects, nil + } + + assembler := NewGenesisAssembler(homeDir, "my-bucket", "us-west-2", "genesis", s3Factory, mockUploaderFactory(newMockS3Uploader())) + + cfg := AssembleGenesisRequest{ + AccountBalance: "10000000usei", + Namespace: "default", + Nodes: []AssembleNodeEntry{{Name: "val-0"}, {Name: "val-1"}}, + } + + nodes := cfg.nodeNames() + if err := assembler.downloadGentxFiles(context.Background(), cfg, nodes); err != nil { + t.Fatalf("unexpected error: %v", err) + } + + gentxDir := filepath.Join(homeDir, "config", assembledGentxSubdir) + for _, node := range []string{"val-0", "val-1"} { + path := filepath.Join(gentxDir, fmt.Sprintf("gentx-%s.json", node)) + if _, err := os.Stat(path); err != nil { + t.Errorf("expected gentx file %s to exist", path) + } + } + + // Isolation guarantee: the assembler must never touch config/gentx/, which + // generate-gentx and upload-genesis-artifacts use. + if _, err := os.Stat(filepath.Join(homeDir, "config", "gentx")); !os.IsNotExist(err) { + t.Errorf("assembler must not create or populate config/gentx/; stat err = %v (want IsNotExist)", err) + } +} + +// TestAssembler_DownloadClearsStaleFiles verifies the assemble dir is wiped on +// every run, so a leftover gentx from a crashed prior attempt can't inflate the +// collected set on a retry. +func TestAssembler_DownloadClearsStaleFiles(t *testing.T) { + homeDir := t.TempDir() + + s3Objects := &mockS3GetObject{objects: map[string][]byte{ + "genesis/val-0/gentx.json": []byte(`{"gentx":"val0"}`), + }} + s3Factory := func(_ context.Context, _ string) (S3GetObjectAPI, error) { + return s3Objects, nil + } + assembler := NewGenesisAssembler(homeDir, "my-bucket", "us-west-2", "genesis", s3Factory, mockUploaderFactory(newMockS3Uploader())) + + // Pre-seed a stale file in the assemble dir as if a prior run had crashed. + gentxDir := assembler.assembledGentxDir() + if err := os.MkdirAll(gentxDir, 0o755); err != nil { + t.Fatalf("mkdir: %v", err) + } + stale := filepath.Join(gentxDir, "gentx-stale.json") + if err := os.WriteFile(stale, []byte(`{"stale":true}`), 0o644); err != nil { + t.Fatalf("write stale: %v", err) + } + + cfg := AssembleGenesisRequest{AccountBalance: "10000000usei", Namespace: "default", Nodes: []AssembleNodeEntry{{Name: "val-0"}}} + if err := assembler.downloadGentxFiles(context.Background(), cfg, cfg.nodeNames()); err != nil { + t.Fatalf("downloadGentxFiles: %v", err) + } + + if _, err := os.Stat(stale); !os.IsNotExist(err) { + t.Errorf("stale gentx file survived download; stat err = %v (want IsNotExist)", err) + } + if _, err := os.Stat(filepath.Join(gentxDir, "gentx-val-0.json")); err != nil { + t.Errorf("expected downloaded gentx-val-0.json to exist: %v", err) + } +} + +func TestAssembler_MissingParams(t *testing.T) { + handler := NewGenesisAssembler(t.TempDir(), "b", "r", "chain", nil, nil).Handler() + + tests := []struct { + name string + params map[string]any + }{ + {"missing accountBalance", map[string]any{"namespace": "ns", "nodes": []any{map[string]any{"name": "n"}}}}, + {"missing namespace", map[string]any{"accountBalance": "10usei", "nodes": []any{map[string]any{"name": "n"}}}}, + {"missing nodes", map[string]any{"accountBalance": "10usei", "namespace": "ns"}}, + {"empty nodes", map[string]any{"accountBalance": "10usei", "namespace": "ns", "nodes": []any{}}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if _, err := handler(context.Background(), tt.params); err == nil { + t.Fatal("expected error") + } + }) + } +} + +func TestAssembler_S3DownloadFailure(t *testing.T) { + homeDir := t.TempDir() + s3Factory := func(_ context.Context, _ string) (S3GetObjectAPI, error) { + return &mockS3GetObject{objects: map[string][]byte{}}, nil + } + + handler := NewGenesisAssembler(homeDir, "b", "r", "c", s3Factory, nil).Handler() + _, err := handler(context.Background(), map[string]any{ + "accountBalance": "10000000usei", "namespace": "default", + "nodes": []any{map[string]any{"name": "missing-node"}}, + }) + if err == nil { + t.Fatal("expected error when S3 download fails") + } +} + +func TestParseAssembleNodes(t *testing.T) { + // Test that AssembleNodeEntry JSON round-trips correctly. + raw := `[{"name":"val-0"},{"name":"val-1"}]` + var nodes []AssembleNodeEntry + if err := json.Unmarshal([]byte(raw), &nodes); err != nil { + t.Fatalf("unexpected error: %v", err) + } + if len(nodes) != 2 || nodes[0].Name != "val-0" || nodes[1].Name != "val-1" { + t.Errorf("nodes = %v, want [{val-0} {val-1}]", nodes) + } +} + +func TestParseAssembleNodes_MissingName(t *testing.T) { + // Test that empty names are caught by the handler validation. + raw := `[{"other":"field"}]` + var nodes []AssembleNodeEntry + if err := json.Unmarshal([]byte(raw), &nodes); err != nil { + t.Fatalf("unexpected error: %v", err) + } + // The node should unmarshal but with empty Name — the handler validates this. + if nodes[0].Name != "" { + t.Fatalf("expected empty name, got %q", nodes[0].Name) + } +} + +func TestAssembler_UploadGenesis_ReturnsHashOfUploadedBytes(t *testing.T) { + homeDir := t.TempDir() + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatalf("mkdir: %v", err) + } + body := []byte(`{"chain_id":"test-chain-1","app_state":{"staking":{"params":{}}}}`) + if err := os.WriteFile(filepath.Join(configDir, "genesis.json"), body, 0o644); err != nil { + t.Fatalf("write genesis: %v", err) + } + + mock := newMockS3Uploader() + a := NewGenesisAssembler(homeDir, "my-bucket", "us-west-2", "test-chain-1", nil, mockUploaderFactory(mock)) + + gotHash, err := a.uploadGenesis(context.Background(), AssembleGenesisRequest{}) + if err != nil { + t.Fatalf("uploadGenesis: %v", err) + } + + // The returned hash is the SHA-256 of the exact uploaded bytes. + uploaded, ok := mock.uploads["my-bucket/test-chain-1/genesis.json"] + if !ok { + t.Fatal("genesis.json was not uploaded") + } + sum := sha256.Sum256(uploaded) + wantHash := hex.EncodeToString(sum[:]) + if gotHash != wantHash { + t.Errorf("returned hash = %q, want sha256(uploaded bytes) = %q", gotHash, wantHash) + } + // Byte-exactness: the uploaded bytes equal the on-disk genesis (no re-marshal). + if !bytes.Equal(uploaded, body) { + t.Errorf("uploaded bytes differ from on-disk genesis; got %q", uploaded) + } + // No "sha256:" prefix — bare hex. + if strings.Contains(gotHash, ":") { + t.Errorf("hash %q must be bare hex (no algorithm prefix)", gotHash) + } + // The hash travels only in-band: no sibling .sha256 object is written + // to the attacker-writable prefix. + if _, ok := mock.uploads["my-bucket/test-chain-1/genesis.json.sha256"]; ok { + t.Error("sibling genesis.json.sha256 object must not be uploaded; the hash travels in-band only") + } + + // The in-band result the handler emits is {"genesisHash":""} + // carrying exactly sha256(uploaded bytes). + resultJSON, err := json.Marshal(AssembleGenesisResult{GenesisHash: gotHash}) + if err != nil { + t.Fatalf("marshal result: %v", err) + } + var decoded AssembleGenesisResult + if err := json.Unmarshal(resultJSON, &decoded); err != nil { + t.Fatalf("unmarshal result: %v", err) + } + if decoded.GenesisHash != wantHash { + t.Errorf("in-band result genesisHash = %q, want sha256(uploaded) = %q", decoded.GenesisHash, wantHash) + } +} + +// genesisWithAppState writes a Tendermint genesis.json containing the given +// JSON-encoded app_state body. Used by override tests that need real +// module-shaped data to walk into. +func genesisWithAppState(t *testing.T, homeDir, appStateJSON string) string { + t.Helper() + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatalf("mkdir: %v", err) + } + genFile := filepath.Join(configDir, "genesis.json") + body := `{ + "chain_id": "test-chain-1", + "genesis_time": "2026-01-01T00:00:00Z", + "initial_height": "1", + "consensus_params": { + "block": {"max_bytes": "22020096", "max_gas": "-1"}, + "evidence": {"max_age_num_blocks": "100000", "max_age_duration": "172800000000000", "max_bytes": "1048576"}, + "validator": {"pub_key_types": ["ed25519"]}, + "version": {} + }, + "validators": [], + "app_hash": "", + "app_state": ` + appStateJSON + ` + }` + if err := os.WriteFile(genFile, []byte(body), 0o644); err != nil { + t.Fatalf("write genesis: %v", err) + } + return genFile +} + +func TestAssembler_ApplyOverrides_NoOpOnEmpty(t *testing.T) { + homeDir := t.TempDir() + genFile := genesisWithAppState(t, homeDir, `{"staking":{"params":{"unbonding_time":"1814400s"}}}`) + mtimeBefore := mustModTime(t, genFile) + + a := NewGenesisAssembler(homeDir, "b", "r", "test-chain-1", nil, nil) + if err := a.applyOverrides(nil); err != nil { + t.Fatalf("nil overrides: %v", err) + } + if err := a.applyOverrides(map[string]json.RawMessage{}); err != nil { + t.Fatalf("empty overrides: %v", err) + } + + if mustModTime(t, genFile) != mtimeBefore { + t.Errorf("genesis.json was rewritten on no-op overrides") + } +} + +func TestAssembler_ApplyOverrides_PatchesGenesisFile(t *testing.T) { + homeDir := t.TempDir() + genFile := genesisWithAppState(t, homeDir, `{ + "staking": {"params": {"unbonding_time": "1814400s", "max_validators": 100}}, + "gov": {"params": {"max_deposit_period": "172800s"}} + }`) + + a := NewGenesisAssembler(homeDir, "b", "r", "test-chain-1", nil, nil) + err := a.applyOverrides(map[string]json.RawMessage{ + "staking.params.unbonding_time": json.RawMessage(`"600s"`), + "gov.params.max_deposit_period": json.RawMessage(`"60s"`), + }) + if err != nil { + t.Fatalf("applyOverrides: %v", err) + } + + // Read the file back and confirm the leaves changed. + body, err := os.ReadFile(genFile) + if err != nil { + t.Fatalf("reading genesis back: %v", err) + } + var doc struct { + AppState map[string]json.RawMessage `json:"app_state"` + } + if err := json.Unmarshal(body, &doc); err != nil { + t.Fatalf("parsing genesis: %v", err) + } + var staking struct { + Params struct { + UnbondingTime string `json:"unbonding_time"` + MaxValidators int `json:"max_validators"` + } `json:"params"` + } + if err := json.Unmarshal(doc.AppState["staking"], &staking); err != nil { + t.Fatalf("parsing staking: %v", err) + } + if staking.Params.UnbondingTime != "600s" { + t.Errorf("unbonding_time = %q, want 600s", staking.Params.UnbondingTime) + } + if staking.Params.MaxValidators != 100 { + t.Errorf("max_validators = %d, want preserved 100", staking.Params.MaxValidators) + } + + var gov struct { + Params struct { + MaxDepositPeriod string `json:"max_deposit_period"` + } `json:"params"` + } + if err := json.Unmarshal(doc.AppState["gov"], &gov); err != nil { + t.Fatalf("parsing gov: %v", err) + } + if gov.Params.MaxDepositPeriod != "60s" { + t.Errorf("max_deposit_period = %q, want 60s", gov.Params.MaxDepositPeriod) + } +} + +func TestAssembler_ApplyOverrides_BubblesBadKeyError(t *testing.T) { + homeDir := t.TempDir() + _ = genesisWithAppState(t, homeDir, `{"staking":{"params":{"unbonding_time":"1814400s"}}}`) + + a := NewGenesisAssembler(homeDir, "b", "r", "test-chain-1", nil, nil) + err := a.applyOverrides(map[string]json.RawMessage{ + "nope.params.x": json.RawMessage(`"y"`), + }) + if err == nil { + t.Fatal("expected error for unknown module override") + } + if !strings.Contains(err.Error(), "unknown module") { + t.Errorf("error = %q, want substring 'unknown module'", err.Error()) + } +} + +// TestAssembleGenesisRequest_OverridesRoundTrip verifies the new Overrides +// field deserializes from the wire shape the controller emits. +func TestAssembleGenesisRequest_OverridesRoundTrip(t *testing.T) { + wire := `{ + "accountBalance": "10000000usei", + "namespace": "default", + "nodes": [{"name": "val-0"}], + "overrides": { + "staking.params.unbonding_time": "600s", + "staking.params.max_validators": 50 + } + }` + var got AssembleGenesisRequest + if err := json.Unmarshal([]byte(wire), &got); err != nil { + t.Fatalf("unmarshal: %v", err) + } + if len(got.Overrides) != 2 { + t.Fatalf("overrides len = %d, want 2", len(got.Overrides)) + } + if string(got.Overrides["staking.params.unbonding_time"]) != `"600s"` { + t.Errorf("unbonding_time raw = %q, want %q", + string(got.Overrides["staking.params.unbonding_time"]), `"600s"`) + } + if string(got.Overrides["staking.params.max_validators"]) != "50" { + t.Errorf("max_validators raw = %q, want %q", + string(got.Overrides["staking.params.max_validators"]), "50") + } +} diff --git a/sidecar/tasks/assemble_genesis_validators.go b/sidecar/tasks/assemble_genesis_validators.go new file mode 100644 index 00000000..7687531b --- /dev/null +++ b/sidecar/tasks/assemble_genesis_validators.go @@ -0,0 +1,162 @@ +package tasks + +import ( + "encoding/json" + "fmt" + "path/filepath" + + tmtypes "github.com/sei-protocol/sei-chain/sei-tendermint/types" + + "github.com/sei-protocol/sei-chain/sei-cosmos/client" + "github.com/sei-protocol/sei-chain/sei-cosmos/codec" + cryptocodec "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/codec" + cryptotypes "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/types" + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil" + genutiltypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil/types" + stakingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/staking/types" +) + +// populateGenesisValidators derives the initial CometBFT validator set from the +// collected gentxs and writes it into the top-level genDoc.Validators, making +// the assembled genesis self-contained: its validator set is loadable without +// InitChain (e.g. by a state-sync boot), matching canonical chains. A +// collect-gentxs genesis otherwise leaves genDoc.Validators empty and +// materializes the set only at InitChain (height 0). +// +// The derivation mirrors staking's InitChain math exactly, so a normal +// from-genesis boot still passes CometBFT's genesis-vs-InitChain equality +// assertion: consensus power = TokensToConsensusPower(self-bond, +// DefaultPowerReduction) and the pubkey is the gentx's consensus pubkey. +// +// It guards only the divergences that are a pure function of a well-formed +// gentx set — where a per-gentx mapping would silently disagree with the set +// InitChain actually bonds — turning each into a loud, assembly-time failure +// rather than a genesis that hard-errors every founding validator at boot: +// - a validator whose stake yields consensus power 0 (InitChain drops it), +// - more validators than the staking MaxValidators cap (InitChain bonds only +// the top N by power), and +// - two gentxs sharing a consensus key (CometBFT's NewValidatorSet panics on a +// duplicate entry). +// +// It does NOT re-validate gentx admissibility — commission below the params +// minimum, a duplicate operator/owner, a self-bond in the wrong denom, an +// unsupported consensus pubkey type. Those are rejected upstream by genutil +// ValidateGenesis and, failing that, by DeliverGenTxs panicking during +// InitChain; this derivation trusts that gate rather than reimplementing the +// staking handler/ante checks. +// +// Runs after collectGentxs/applyOverrides and before uploadGenesis, so the +// validators are part of the bytes that get hashed and distributed. Deriving +// from the assembled genesis's genutil.gen_txs (not the raw gentx files) uses +// the exact input InitChain consumes, so any override that rewrote gen_txs is +// reflected here too. +func (a *GenesisAssembler) populateGenesisValidators() error { + cdc, txCfg := makeCodec() + ensureBech32() + + genFile := filepath.Join(a.homeDir, "config", "genesis.json") + genDoc, err := tmtypes.GenesisDocFromFile(genFile) + if err != nil { + return fmt.Errorf("assemble-genesis: reading genesis for validators: %w", err) + } + + var appState map[string]json.RawMessage + if err := json.Unmarshal(genDoc.AppState, &appState); err != nil { + return fmt.Errorf("assemble-genesis: parsing app_state for validators: %w", err) + } + + maxValidators := stakingtypes.GetGenesisStateFromAppState(cdc, appState).Params.MaxValidators + genUtilState := genutiltypes.GetGenesisStateFromAppState(cdc, appState) + + validators, err := deriveGenesisValidators(cdc, txCfg, genUtilState.GenTxs) + if err != nil { + return err + } + if len(validators) == 0 { + return fmt.Errorf("assemble-genesis: no gentxs to derive genesis validators from") + } + if uint32(len(validators)) > maxValidators { + return fmt.Errorf( + "assemble-genesis: %d validators exceed staking max_validators %d; InitChain would bond "+ + "only the top %d by power, so genDoc.validators would diverge from the app's set and "+ + "every validator would fail the boot-time genesis/InitChain equality check", + len(validators), maxValidators, maxValidators, + ) + } + + genDoc.Validators = validators + if err := genutil.ExportGenesisFile(genDoc, genFile); err != nil { + return fmt.Errorf("assemble-genesis: writing genesis with validators: %w", err) + } + + assembleLog.Info("populated genesis validators", "count", len(validators), "maxValidators", maxValidators) + return nil +} + +// deriveGenesisValidators converts each collected gentx's MsgCreateValidator +// into a GenesisValidator using the same consensus-power conversion InitChain +// applies. It fails loud on the zero-power edge and on a duplicate consensus key +// so the assembled set can never silently diverge from what InitChain would bond +// nor panic CometBFT's NewValidatorSet at boot. +func deriveGenesisValidators(cdc codec.Codec, txCfg client.TxConfig, genTxs []json.RawMessage) ([]tmtypes.GenesisValidator, error) { + validators := make([]tmtypes.GenesisValidator, 0, len(genTxs)) + seenConsAddr := make(map[string]string, len(genTxs)) + for i, raw := range genTxs { + tx, err := txCfg.TxJSONDecoder()(raw) + if err != nil { + return nil, fmt.Errorf("assemble-genesis: decoding gen_tx %d: %w", i, err) + } + msgs := tx.GetMsgs() + if len(msgs) != 1 { + return nil, fmt.Errorf("assemble-genesis: gen_tx %d has %d messages, want exactly 1 MsgCreateValidator", i, len(msgs)) + } + msg, ok := msgs[0].(*stakingtypes.MsgCreateValidator) + if !ok { + return nil, fmt.Errorf("assemble-genesis: gen_tx %d is not a MsgCreateValidator", i) + } + + var pk cryptotypes.PubKey + if err := cdc.UnpackAny(msg.Pubkey, &pk); err != nil { + return nil, fmt.Errorf("assemble-genesis: unpacking consensus pubkey for validator %s: %w", msg.ValidatorAddress, err) + } + tmPk, err := cryptocodec.ToTmPubKeyInterface(pk) + if err != nil { + return nil, fmt.Errorf("assemble-genesis: converting consensus pubkey for validator %s: %w", msg.ValidatorAddress, err) + } + + consAddr := tmPk.Address().String() + if prev, dup := seenConsAddr[consAddr]; dup { + return nil, fmt.Errorf( + "assemble-genesis: validators %s and %s share consensus key %s; CometBFT's "+ + "NewValidatorSet panics on a duplicate entry, bricking every founding validator at boot", + prev, msg.ValidatorAddress, consAddr, + ) + } + seenConsAddr[consAddr] = msg.ValidatorAddress + + // Same conversion as staking InitChain: k.PowerReduction(ctx) returns + // sdk.DefaultPowerReduction, and a genesis validator's bonded tokens are + // its self-delegation (msg.Value). sei-cosmos x/staking/keeper/params.go:58-60 + // returns DefaultPowerReduction unconditionally — it is not a gov param — so + // hardcoding the constant is safe. If it ever becomes parameterized, this + // power stops matching InitChain and every validator fails the boot-time + // genesis/InitChain equality check (replay.go:211-228). + power := sdk.TokensToConsensusPower(msg.Value.Amount, sdk.DefaultPowerReduction) + if power == 0 { + return nil, fmt.Errorf( + "assemble-genesis: validator %s self-bond %s yields consensus power 0 (below power "+ + "reduction %s); InitChain would drop it, so genDoc.validators would diverge from "+ + "the app's set", + msg.ValidatorAddress, msg.Value.String(), sdk.DefaultPowerReduction, + ) + } + + validators = append(validators, tmtypes.GenesisValidator{ + Address: tmPk.Address(), + PubKey: tmPk, + Power: power, + }) + } + return validators, nil +} diff --git a/sidecar/tasks/assemble_genesis_validators_test.go b/sidecar/tasks/assemble_genesis_validators_test.go new file mode 100644 index 00000000..a904686e --- /dev/null +++ b/sidecar/tasks/assemble_genesis_validators_test.go @@ -0,0 +1,210 @@ +package tasks + +import ( + "bytes" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + "time" + + tmtypes "github.com/sei-protocol/sei-chain/sei-tendermint/types" + + "github.com/sei-protocol/sei-chain/sei-cosmos/client" + "github.com/sei-protocol/sei-chain/sei-cosmos/codec" + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/keys/ed25519" + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/keys/secp256k1" + cryptotypes "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/types" + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil" + genutiltypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil/types" + stakingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/staking/types" +) + +// buildTestGentx builds an (unsigned) gentx carrying a single MsgCreateValidator +// with the given consensus pubkey and self-bond. Signatures are irrelevant to +// validator derivation, which only decodes the message. +func buildTestGentx(t *testing.T, txCfg client.TxConfig, consPub cryptotypes.PubKey, selfBond sdk.Coin) json.RawMessage { + t.Helper() + valAddr := sdk.ValAddress(secp256k1.GenPrivKey().PubKey().Address()) + msg, err := stakingtypes.NewMsgCreateValidator( + valAddr, consPub, selfBond, + stakingtypes.Description{Moniker: "val"}, + stakingtypes.NewCommissionRates(sdk.ZeroDec(), sdk.ZeroDec(), sdk.ZeroDec()), + sdk.OneInt(), + ) + if err != nil { + t.Fatalf("NewMsgCreateValidator: %v", err) + } + + b := txCfg.NewTxBuilder() + if err := b.SetMsgs(msg); err != nil { + t.Fatalf("SetMsgs: %v", err) + } + bz, err := txCfg.TxJSONEncoder()(b.GetTx()) + if err != nil { + t.Fatalf("encoding gentx: %v", err) + } + return bz +} + +// writeAssembledGenesis writes a homeDir/config/genesis.json whose app_state +// carries the given gentxs and staking max_validators — the post-collect, +// pre-populate shape the assembler operates on. +func writeAssembledGenesis(t *testing.T, cdc codec.Codec, homeDir string, maxValidators uint32, genTxs []json.RawMessage) { + t.Helper() + if err := os.MkdirAll(filepath.Join(homeDir, "config"), 0o755); err != nil { + t.Fatalf("mkdir config: %v", err) + } + + stakingGen := stakingtypes.DefaultGenesisState() + stakingGen.Params.MaxValidators = maxValidators + + appState := map[string]json.RawMessage{ + stakingtypes.ModuleName: cdc.MustMarshalJSON(stakingGen), + genutiltypes.ModuleName: cdc.MustMarshalJSON(genutiltypes.NewGenesisState(genTxs)), + } + appStateBz, err := json.Marshal(appState) + if err != nil { + t.Fatalf("marshal app_state: %v", err) + } + + genDoc := &tmtypes.GenesisDoc{ + ChainID: "assemble-validators-test", + GenesisTime: time.Now(), + ConsensusParams: tmtypes.DefaultConsensusParams(), + AppState: appStateBz, + } + if err := genutil.ExportGenesisFile(genDoc, filepath.Join(homeDir, "config", "genesis.json")); err != nil { + t.Fatalf("export genesis: %v", err) + } +} + +func TestPopulateGenesisValidators_NValidators(t *testing.T) { + ensureBech32() + cdc, txCfg := makeCodec() + + // Distinct ed25519 consensus keys and distinct powers. Stakes are exact + // multiples of the power reduction (1e6) so truncation is unambiguous. + type spec struct { + pub cryptotypes.PubKey + stake int64 + } + specs := []spec{ + {ed25519.GenPrivKey().PubKey(), 3_000_000}, + {ed25519.GenPrivKey().PubKey(), 7_000_000}, + } + genTxs := make([]json.RawMessage, len(specs)) + for i, s := range specs { + genTxs[i] = buildTestGentx(t, txCfg, s.pub, sdk.NewCoin("usei", sdk.NewInt(s.stake))) + } + + homeDir := t.TempDir() + writeAssembledGenesis(t, cdc, homeDir, 100, genTxs) + + a := NewGenesisAssembler(homeDir, "b", "r", "assemble-validators-test", nil, nil) + if err := a.populateGenesisValidators(); err != nil { + t.Fatalf("populateGenesisValidators: %v", err) + } + + genDoc, err := tmtypes.GenesisDocFromFile(filepath.Join(homeDir, "config", "genesis.json")) + if err != nil { + t.Fatalf("reading genesis: %v", err) + } + if got, want := len(genDoc.Validators), len(specs); got != want { + t.Fatalf("len(genDoc.Validators) = %d, want %d", got, want) + } + + for i, s := range specs { + gotVal := genDoc.Validators[i] + + // Independent power oracle: reimplement the power reduction as literal + // integer division rather than calling sdk.TokensToConsensusPower, so a + // change to Sei's power math breaks this test instead of tracking the + // derivation that also calls it. + wantPower := s.stake / 1_000_000 + if got := gotVal.Power; got != wantPower { + t.Errorf("validator %d power = %d, want %d", i, got, wantPower) + } + + // Independent pubkey oracle: the cosmos ed25519 pubkey's 32 raw key bytes + // must survive the proto oneof roundtrip and equal the derived CometBFT + // pubkey's bytes, without going back through ToTmPubKeyInterface. + wantPubKey := s.pub.Bytes() + if got := gotVal.PubKey.Bytes(); !bytes.Equal(got, wantPubKey) { + t.Errorf("validator %d pubkey = %x, want %x", i, got, wantPubKey) + } + if got := len(wantPubKey); got != 32 { + t.Errorf("validator %d source pubkey length = %d, want 32", i, got) + } + } +} + +func TestPopulateGenesisValidators_ZeroPowerFailsLoud(t *testing.T) { + ensureBech32() + cdc, txCfg := makeCodec() + + // Stake below the power reduction (1e6) => consensus power 0. + genTxs := []json.RawMessage{ + buildTestGentx(t, txCfg, ed25519.GenPrivKey().PubKey(), sdk.NewCoin("usei", sdk.NewInt(999_999))), + } + homeDir := t.TempDir() + writeAssembledGenesis(t, cdc, homeDir, 100, genTxs) + + a := NewGenesisAssembler(homeDir, "b", "r", "assemble-validators-test", nil, nil) + err := a.populateGenesisValidators() + if err == nil { + t.Fatal("expected zero-power error, got nil") + } + if !strings.Contains(err.Error(), "consensus power 0") { + t.Errorf("error = %q, want substring 'consensus power 0'", err.Error()) + } +} + +func TestPopulateGenesisValidators_ExceedsMaxValidatorsFailsLoud(t *testing.T) { + ensureBech32() + cdc, txCfg := makeCodec() + + genTxs := []json.RawMessage{ + buildTestGentx(t, txCfg, ed25519.GenPrivKey().PubKey(), sdk.NewCoin("usei", sdk.NewInt(1_000_000))), + buildTestGentx(t, txCfg, ed25519.GenPrivKey().PubKey(), sdk.NewCoin("usei", sdk.NewInt(1_000_000))), + } + homeDir := t.TempDir() + writeAssembledGenesis(t, cdc, homeDir, 1, genTxs) // max_validators=1, but 2 gentxs + + a := NewGenesisAssembler(homeDir, "b", "r", "assemble-validators-test", nil, nil) + err := a.populateGenesisValidators() + if err == nil { + t.Fatal("expected max-validators error, got nil") + } + if !strings.Contains(err.Error(), "max_validators") { + t.Errorf("error = %q, want substring 'max_validators'", err.Error()) + } +} + +// TestPopulateGenesisValidators_DuplicateConsensusKeyFailsLoud: two gentxs (with +// distinct operators, so the delegator/owner guards do not fire) sharing one +// consensus key would panic CometBFT's NewValidatorSet at boot. The derivation +// must reject it at assembly instead. +func TestPopulateGenesisValidators_DuplicateConsensusKeyFailsLoud(t *testing.T) { + ensureBech32() + cdc, txCfg := makeCodec() + + shared := ed25519.GenPrivKey().PubKey() + genTxs := []json.RawMessage{ + buildTestGentx(t, txCfg, shared, sdk.NewCoin("usei", sdk.NewInt(1_000_000))), + buildTestGentx(t, txCfg, shared, sdk.NewCoin("usei", sdk.NewInt(1_000_000))), + } + homeDir := t.TempDir() + writeAssembledGenesis(t, cdc, homeDir, 100, genTxs) + + a := NewGenesisAssembler(homeDir, "b", "r", "assemble-validators-test", nil, nil) + err := a.populateGenesisValidators() + if err == nil { + t.Fatal("expected duplicate-consensus-key error, got nil") + } + if !strings.Contains(err.Error(), "share consensus key") { + t.Errorf("error = %q, want substring 'share consensus key'", err.Error()) + } +} diff --git a/sidecar/tasks/await_condition.go b/sidecar/tasks/await_condition.go new file mode 100644 index 00000000..e110e6d3 --- /dev/null +++ b/sidecar/tasks/await_condition.go @@ -0,0 +1,164 @@ +package tasks + +import ( + "context" + "fmt" + "time" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/actions" + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +var awaitLog = seilog.NewLogger("seictl", "task", "await-condition") + +const ( + conditionHeight = "height" + conditionCatchingUp = "catchingUp" + actionSIGTERM = "SIGTERM_SEID" + + heightPollInterval = 100 * time.Millisecond +) + +// AwaitConditionRequest holds the typed parameters for the await-condition task. +type AwaitConditionRequest struct { + Condition string `json:"condition"` + Action string `json:"action"` + TargetHeight int64 `json:"targetHeight"` +} + +// ConditionWaiter polls a local node until a condition is met, then +// optionally executes a post-condition action. +type ConditionWaiter struct { + rpc *rpc.StatusClient +} + +// NewConditionWaiter creates a ConditionWaiter. Pass nil for the default RPC client. +func NewConditionWaiter(rpcClient *rpc.StatusClient) *ConditionWaiter { + if rpcClient == nil { + rpcClient = rpc.NewStatusClient("", nil) + } + return &ConditionWaiter{rpc: rpcClient} +} + +// Handler returns an engine.TaskHandler for the await-condition task type. +func (w *ConditionWaiter) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, params AwaitConditionRequest) error { + if params.Condition == "" { + return fmt.Errorf("condition is required") + } + + switch params.Condition { + case conditionHeight: + if params.TargetHeight <= 0 { + return fmt.Errorf("targetHeight must be > 0, got %d", params.TargetHeight) + } + if err := w.awaitHeight(ctx, params.TargetHeight); err != nil { + return err + } + case conditionCatchingUp: + if err := w.awaitCaughtUp(ctx); err != nil { + return err + } + default: + return fmt.Errorf("unknown condition %q", params.Condition) + } + + if params.Action == "" { + return nil + } + return w.executeAction(ctx, params.Action) + }) +} + +func (w *ConditionWaiter) awaitHeight(ctx context.Context, targetHeight int64) error { + awaitLog.Info("awaiting height", "target", targetHeight, "rpc", w.rpc.Endpoint()) + + var rpcHealthy bool + var loggedInitialWait bool + ticker := time.NewTicker(heightPollInterval) + defer ticker.Stop() + + for { + select { + case <-ctx.Done(): + return ctx.Err() + case <-ticker.C: + } + + height, err := w.rpc.LatestHeight(ctx) + if err != nil { + if rpcHealthy { + awaitLog.Warn("rpc became unavailable", "err", err) + rpcHealthy = false + } else if !loggedInitialWait { + awaitLog.Info("waiting for rpc to become available", "err", err) + loggedInitialWait = true + } + continue + } + if !rpcHealthy { + awaitLog.Info("rpc available", "height", height) + rpcHealthy = true + } + + if height >= targetHeight { + awaitLog.Info("target height reached", "current", height, "target", targetHeight) + return nil + } + } +} + +// awaitCaughtUp polls the local node until it reports catching_up=false at a +// height past genesis (>1). A freshly state-synced node reports catching_up +// while it applies the snapshot and backfills; the height>1 floor rejects the +// degenerate window where a just-started node reads catching_up=false before it +// has synced. This matches the sdk/sei readiness semantics the controller uses. +func (w *ConditionWaiter) awaitCaughtUp(ctx context.Context) error { + awaitLog.Info("awaiting caught up", "rpc", w.rpc.Endpoint()) + + var rpcHealthy bool + var loggedInitialWait bool + ticker := time.NewTicker(heightPollInterval) + defer ticker.Stop() + + for { + select { + case <-ctx.Done(): + return ctx.Err() + case <-ticker.C: + } + + status, err := w.rpc.Status(ctx) + if err != nil { + if rpcHealthy { + awaitLog.Warn("rpc became unavailable", "err", err) + rpcHealthy = false + } else if !loggedInitialWait { + awaitLog.Info("waiting for rpc to become available", "err", err) + loggedInitialWait = true + } + continue + } + if !rpcHealthy { + awaitLog.Info("rpc available", "height", status.LatestBlockHeight, "catchingUp", status.CatchingUp) + rpcHealthy = true + } + + if !status.CatchingUp && status.LatestBlockHeight > 1 { + awaitLog.Info("node caught up", "height", status.LatestBlockHeight) + return nil + } + } +} + +func (w *ConditionWaiter) executeAction(ctx context.Context, action string) error { + switch action { + case actionSIGTERM: + return actions.GracefulStop(ctx, nil, "seid", actions.DefaultGracePeriod) + default: + return fmt.Errorf("unknown action %q", action) + } +} diff --git a/sidecar/tasks/await_condition_test.go b/sidecar/tasks/await_condition_test.go new file mode 100644 index 00000000..5d9a532d --- /dev/null +++ b/sidecar/tasks/await_condition_test.go @@ -0,0 +1,326 @@ +package tasks + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "sync" + "testing" + "time" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// heightServer returns an httptest.Server that serves a height sequence. +// After the sequence is exhausted it keeps returning the last value. +func heightServer(heights ...int64) *httptest.Server { + var mu sync.Mutex + idx := 0 + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + mu.Lock() + i := idx + if i < len(heights)-1 { + idx++ + } + mu.Unlock() + fmt.Fprintf(w, `{"jsonrpc":"2.0","id":-1,"result":{"sync_info":{"latest_block_height":"%d","catching_up":false}}}`, heights[i]) + })) +} + +func rpcClient(url string) *rpc.StatusClient { + return rpc.NewStatusClient(url, nil) +} + +func TestAwaitHeight_ReachesTarget(t *testing.T) { + srv := heightServer(100, 200, 500) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": float64(500), + } + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + + if _, err := handler(ctx, params); err != nil { + t.Fatalf("expected success, got %v", err) + } +} + +func TestAwaitHeight_AlreadyPastTarget(t *testing.T) { + srv := heightServer(1000) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": float64(500), + } + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + + if _, err := handler(ctx, params); err != nil { + t.Fatalf("expected success, got %v", err) + } +} + +func TestAwaitHeight_MissingTargetHeight(t *testing.T) { + srv := heightServer(100) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{"condition": "height"} + if _, err := handler(context.Background(), params); err == nil { + t.Fatal("expected error for missing targetHeight") + } +} + +func TestAwaitHeight_ZeroTargetHeight(t *testing.T) { + srv := heightServer(100) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": float64(0), + } + if _, err := handler(context.Background(), params); err == nil { + t.Fatal("expected error for zero targetHeight") + } +} + +func TestAwaitHeight_NegativeTargetHeight(t *testing.T) { + srv := heightServer(100) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": float64(-5), + } + if _, err := handler(context.Background(), params); err == nil { + t.Fatal("expected error for negative targetHeight") + } +} + +func TestAwaitHeight_TransientRPCErrors(t *testing.T) { + errSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusInternalServerError) + })) + defer errSrv.Close() + + goodSrv := heightServer(500) + defer goodSrv.Close() + + var mu sync.Mutex + calls := 0 + proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + mu.Lock() + calls++ + n := calls + mu.Unlock() + var target string + if n <= 2 { + target = errSrv.URL + } else { + target = goodSrv.URL + } + resp, err := http.Get(target + r.URL.Path) + if err != nil { + w.WriteHeader(http.StatusBadGateway) + return + } + defer func() { _ = resp.Body.Close() }() + w.WriteHeader(resp.StatusCode) + buf := make([]byte, 4096) + for { + n, readErr := resp.Body.Read(buf) + if n > 0 { + _, _ = w.Write(buf[:n]) + } + if readErr != nil { + break + } + } + })) + defer proxy.Close() + + handler := NewConditionWaiter(rpcClient(proxy.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": float64(500), + } + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + + if _, err := handler(ctx, params); err != nil { + t.Fatalf("expected success after transient errors, got %v", err) + } +} + +func TestAwaitHeight_ContextCancellation(t *testing.T) { + srv := heightServer(100) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": float64(99999), + } + ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond) + defer cancel() + + _, err := handler(ctx, params) + if err == nil { + t.Fatal("expected context error") + } + if err != context.DeadlineExceeded { + t.Fatalf("expected DeadlineExceeded, got %v", err) + } +} + +func TestAwaitHeight_MissingCondition(t *testing.T) { + handler := NewConditionWaiter(rpcClient("http://unused")).Handler() + params := map[string]any{} + if _, err := handler(context.Background(), params); err == nil { + t.Fatal("expected error for missing condition") + } +} + +func TestAwaitHeight_UnknownCondition(t *testing.T) { + handler := NewConditionWaiter(rpcClient("http://unused")).Handler() + params := map[string]any{"condition": "unknown"} + if _, err := handler(context.Background(), params); err == nil { + t.Fatal("expected error for unknown condition") + } +} + +// syncStep is one (height, catchingUp) sample for statusServer. +type syncStep struct { + height int64 + catchingUp bool +} + +// statusServer serves a /status sequence; after exhaustion it repeats the last. +func statusServer(steps ...syncStep) *httptest.Server { + var mu sync.Mutex + idx := 0 + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + mu.Lock() + i := idx + if i < len(steps)-1 { + idx++ + } + mu.Unlock() + _, _ = fmt.Fprintf(w, + `{"jsonrpc":"2.0","id":-1,"result":{"sync_info":{"latest_block_height":"%d","catching_up":%t}}}`, + steps[i].height, steps[i].catchingUp) + })) +} + +func TestAwaitCatchingUp_ReachesCaughtUp(t *testing.T) { + // catching_up flips false only after height climbs past 1. + srv := statusServer( + syncStep{0, true}, + syncStep{500, true}, + syncStep{1200, false}, + ) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + + if _, err := handler(ctx, map[string]any{"condition": "catchingUp"}); err != nil { + t.Fatalf("expected success, got %v", err) + } +} + +func TestAwaitCatchingUp_IgnoresNotCaughtUpAtGenesisHeight(t *testing.T) { + // A just-started node can report catching_up=false at height<=1 before it + // has synced; the height>1 floor must not treat that as caught up. The + // server holds height 1 for the first two samples, then jumps past it. + srv := statusServer( + syncStep{1, false}, + syncStep{1, false}, + syncStep{3000, false}, + ) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + + if _, err := handler(ctx, map[string]any{"condition": "catchingUp"}); err != nil { + t.Fatalf("expected success once height passes 1, got %v", err) + } +} + +func TestAwaitCatchingUp_BlocksWhileCatchingUp(t *testing.T) { + srv := statusServer(syncStep{5000, true}) // never stops catching up + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + + _, err := handler(ctx, map[string]any{"condition": "catchingUp"}) + if err != context.DeadlineExceeded { + t.Fatalf("expected DeadlineExceeded while catching up, got %v", err) + } +} + +func TestAwaitHeight_UnknownAction(t *testing.T) { + srv := heightServer(500) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": float64(500), + "action": "UNKNOWN", + } + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + + if _, err := handler(ctx, params); err == nil { + t.Fatal("expected error for unknown action") + } +} + +func TestAwaitHeight_Int64TargetHeight(t *testing.T) { + srv := heightServer(1000) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": int64(500), + } + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + + if _, err := handler(ctx, params); err != nil { + t.Fatalf("expected success with int64 targetHeight, got %v", err) + } +} + +func TestAwaitHeight_JSONNumberTargetHeight(t *testing.T) { + srv := heightServer(5000) + defer srv.Close() + + handler := NewConditionWaiter(rpcClient(srv.URL)).Handler() + params := map[string]any{ + "condition": "height", + "targetHeight": json.Number("5000"), + } + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + + if _, err := handler(ctx, params); err != nil { + t.Fatalf("expected success with json.Number targetHeight, got %v", err) + } +} diff --git a/sidecar/tasks/config.go b/sidecar/tasks/config.go new file mode 100644 index 00000000..3117dbbb --- /dev/null +++ b/sidecar/tasks/config.go @@ -0,0 +1,118 @@ +package tasks + +import ( + "context" + "fmt" + "os" + "path/filepath" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/tasks/defaults" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/tomlpatch" +) + +var patchLog = seilog.NewLogger("seictl", "task", "config-patch") + +// ConfigPatchRequest holds the typed parameters for the config-patch task. +// Files is intentionally map[string]map[string]any because TOML patches +// are inherently untyped. +type ConfigPatchRequest struct { + Files map[string]map[string]any `json:"files"` +} + +// ConfigPatcher applies generic TOML merge-patches to seid configuration files. +type ConfigPatcher struct { + homeDir string +} + +// NewConfigPatcher creates a patcher targeting the given home directory. +func NewConfigPatcher(homeDir string) *ConfigPatcher { + return &ConfigPatcher{homeDir: homeDir} +} + +// Handler returns an engine.TaskHandler that reads a "files" map from params +// and merge-patches each named file under homeDir/config/. +// +// Expected params format: +// +// { +// "files": { +// "config.toml": {"p2p": {"persistent-peers": "..."}}, +// "app.toml": {"pruning": "nothing"} +// } +// } +func (p *ConfigPatcher) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, params ConfigPatchRequest) error { + if len(params.Files) == 0 { + return fmt.Errorf("config-patch: at least one file is required") + } + // Convert to map[string]any for PatchFiles (public API). + files := make(map[string]any, len(params.Files)) + for k, v := range params.Files { + files[k] = v + } + return p.PatchFiles(ctx, files) + }) +} + +// PatchFiles merge-patches each named TOML file under homeDir/config/. +func (p *ConfigPatcher) PatchFiles(_ context.Context, files map[string]any) error { + for filename, rawPatch := range files { + patchMap, ok := rawPatch.(map[string]any) + if !ok { + return fmt.Errorf("config-patch: value for %q must be a map", filename) + } + filePath := filepath.Join(p.homeDir, "config", filename) + patchLog.Debug("patching file", "file", filename) + if err := mergeAndWrite(filePath, patchMap); err != nil { + return fmt.Errorf("config-patch %s: %w", filename, err) + } + } + patchLog.Info("files patched", "count", len(files)) + return nil +} + +func mergeAndWrite(filePath string, patchMap map[string]any) error { + doc, err := tomlpatch.ReadTOML(filePath) + if err != nil { + return fmt.Errorf("reading %s: %w", filepath.Base(filePath), err) + } + merged, ok := tomlpatch.Merge(doc, patchMap).(map[string]any) + if !ok { + return fmt.Errorf("merge produced non-map result for %s", filepath.Base(filePath)) + } + return tomlpatch.WriteTOML(filePath, merged) +} + +// EnsureDefaultConfig creates the seid home directory structure and writes a +// minimal default config.toml if one does not already exist. The default is +// embedded from defaults/config.toml. +func EnsureDefaultConfig(homeDir string) error { + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + return fmt.Errorf("creating config directory: %w", err) + } + + dataDir := filepath.Join(homeDir, "data") + if err := os.MkdirAll(dataDir, 0o755); err != nil { + return fmt.Errorf("creating data directory: %w", err) + } + + configPath := filepath.Join(configDir, "config.toml") + if _, err := os.Stat(configPath); err == nil { + return nil + } + + defaultConfig, err := defaults.FS.ReadFile("config.toml") + if err != nil { + return fmt.Errorf("reading embedded default config: %w", err) + } + + if err := os.WriteFile(configPath, defaultConfig, 0o644); err != nil { + return fmt.Errorf("writing default config.toml: %w", err) + } + + return nil +} diff --git a/sidecar/tasks/config_apply.go b/sidecar/tasks/config_apply.go new file mode 100644 index 00000000..0efe15e7 --- /dev/null +++ b/sidecar/tasks/config_apply.go @@ -0,0 +1,138 @@ +package tasks + +import ( + "context" + "encoding/json" + "fmt" + "path/filepath" + + seiconfig "github.com/sei-protocol/sei-config" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +var applyLog = seilog.NewLogger("seictl", "task", "config-apply") + +// ConfigApplier generates or patches node config using sei-config's +// intent resolution pipeline. The handler deserializes a ConfigIntent from +// task params, calls the appropriate resolver, and writes the result to disk. +type ConfigApplier struct { + homeDir string +} + +// NewConfigApplier creates an applier targeting the given home directory. +func NewConfigApplier(homeDir string) *ConfigApplier { + return &ConfigApplier{homeDir: homeDir} +} + +// Handler returns an engine.TaskHandler for the config-apply task type. +func (a *ConfigApplier) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, intent seiconfig.ConfigIntent) error { + if intent.Incremental { + return a.applyIncremental(ctx, intent) + } + return a.applyFull(ctx, intent) + }) +} + +// applyFull resolves an intent from mode defaults and writes the result. +func (a *ConfigApplier) applyFull(_ context.Context, intent seiconfig.ConfigIntent) error { + applyLog.Info("resolving config intent", "mode", intent.Mode, "targetVersion", intent.TargetVersion) + + result, err := seiconfig.ResolveIntent(intent) + if err != nil { + return fmt.Errorf("config-apply: %w", err) + } + + if !result.Valid { + return resultError(result) + } + + for _, d := range result.Diagnostics { + if d.Severity == seiconfig.SeverityWarning { + applyLog.Warn("config warning", "field", d.Field, "message", d.Message) + } + } + + configDir := filepath.Join(a.homeDir, "config") + if err := seiconfig.WriteConfigToDir(result.Config, a.homeDir); err != nil { + return fmt.Errorf("config-apply: writing config to %s: %w", configDir, err) + } + + applyLog.Info("config written", + "mode", result.Mode, + "version", result.Version, + "overrides", len(intent.Overrides), + ) + return nil +} + +// applyIncremental reads current on-disk config and resolves the intent +// incrementally against it. +func (a *ConfigApplier) applyIncremental(_ context.Context, intent seiconfig.ConfigIntent) error { + current, err := seiconfig.ReadConfigFromDir(a.homeDir) + if err != nil { + return fmt.Errorf("config-apply incremental: reading current config: %w", err) + } + + applyLog.Info("resolving incremental intent", "overrides", len(intent.Overrides)) + + result, err := seiconfig.ResolveIncrementalIntent(intent, current) + if err != nil { + return fmt.Errorf("config-apply incremental: %w", err) + } + + if !result.Valid { + return resultError(result) + } + + for _, d := range result.Diagnostics { + if d.Severity == seiconfig.SeverityWarning { + applyLog.Warn("config warning", "field", d.Field, "message", d.Message) + } + } + + if err := seiconfig.WriteConfigToDir(result.Config, a.homeDir); err != nil { + return fmt.Errorf("config-apply incremental: writing config: %w", err) + } + + applyLog.Info("incremental config applied", + "mode", result.Mode, + "version", result.Version, + "overrides", len(intent.Overrides), + ) + return nil +} + +// resultError formats a ConfigResult's diagnostics as a structured JSON error +// so the controller can parse them from the task result. +func resultError(result *seiconfig.ConfigResult) error { + return diagnosticsError(result.Diagnostics) +} + +// validationError formats ValidationResult diagnostics as a structured JSON +// error. Used by handlers that call seiconfig.Validate() directly (e.g. reload). +func validationError(vr *seiconfig.ValidationResult) error { + return diagnosticsError(vr.Diagnostics) +} + +// diagnosticsError converts a slice of Diagnostic findings into a structured +// JSON error string suitable for returning to the controller. +func diagnosticsError(diags []seiconfig.Diagnostic) error { + type diagJSON struct { + Severity string `json:"severity"` + Field string `json:"field"` + Message string `json:"message"` + } + out := make([]diagJSON, len(diags)) + for i, d := range diags { + out[i] = diagJSON{ + Severity: d.Severity.String(), + Field: d.Field, + Message: d.Message, + } + } + data, _ := json.Marshal(out) + return fmt.Errorf("config validation failed: %s", data) +} diff --git a/sidecar/tasks/config_apply_test.go b/sidecar/tasks/config_apply_test.go new file mode 100644 index 00000000..56536979 --- /dev/null +++ b/sidecar/tasks/config_apply_test.go @@ -0,0 +1,257 @@ +package tasks + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + + seiconfig "github.com/sei-protocol/sei-config" +) + +func writeDefaultConfig(t *testing.T, homeDir string, mode seiconfig.NodeMode) { + t.Helper() + cfg := seiconfig.DefaultForMode(mode) + if err := seiconfig.WriteConfigToDir(cfg, homeDir); err != nil { + t.Fatalf("writing default config: %v", err) + } +} + +func TestConfigApplier_FullGeneration(t *testing.T) { + homeDir := t.TempDir() + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + _, err := handler(context.Background(), map[string]any{ + "mode": "validator", + "incremental": false, + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + cfg, err := seiconfig.ReadConfigFromDir(homeDir) + if err != nil { + t.Fatalf("reading back config: %v", err) + } + if cfg.Mode != seiconfig.ModeValidator { + t.Errorf("mode: got %q, want %q", cfg.Mode, seiconfig.ModeValidator) + } + if cfg.EVM.HTTPEnabled { + t.Error("validator should have EVM HTTP disabled") + } +} + +func TestConfigApplier_FullWithOverrides(t *testing.T) { + homeDir := t.TempDir() + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + _, err := handler(context.Background(), map[string]any{ + "mode": "full", + "incremental": false, + "overrides": map[string]any{ + "evm.http_port": "9545", + }, + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + cfg, err := seiconfig.ReadConfigFromDir(homeDir) + if err != nil { + t.Fatalf("reading back config: %v", err) + } + if cfg.EVM.HTTPPort != 9545 { + t.Errorf("evm.http_port: got %d, want 9545", cfg.EVM.HTTPPort) + } +} + +func TestConfigApplier_FullMissingMode(t *testing.T) { + homeDir := t.TempDir() + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + _, err := handler(context.Background(), map[string]any{ + "incremental": false, + }) + if err == nil { + t.Fatal("expected error for missing mode") + } + if !strings.Contains(err.Error(), "mode is required") { + t.Errorf("error should mention mode: %v", err) + } +} + +func TestConfigApplier_FullInvalidMode(t *testing.T) { + homeDir := t.TempDir() + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + _, err := handler(context.Background(), map[string]any{ + "mode": "bogus", + "incremental": false, + }) + if err == nil { + t.Fatal("expected error for invalid mode") + } + if !strings.Contains(err.Error(), "invalid mode") { + t.Errorf("error should mention invalid mode: %v", err) + } +} + +func TestConfigApplier_Incremental(t *testing.T) { + homeDir := t.TempDir() + writeDefaultConfig(t, homeDir, seiconfig.ModeFull) + + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + _, err := handler(context.Background(), map[string]any{ + "incremental": true, + "overrides": map[string]any{ + "evm.http_port": "9999", + }, + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + cfg, err := seiconfig.ReadConfigFromDir(homeDir) + if err != nil { + t.Fatalf("reading back config: %v", err) + } + if cfg.EVM.HTTPPort != 9999 { + t.Errorf("evm.http_port: got %d, want 9999", cfg.EVM.HTTPPort) + } + if cfg.Mode != seiconfig.ModeFull { + t.Errorf("mode should be preserved: got %q", cfg.Mode) + } +} + +func TestConfigApplier_IncrementalPreservesExisting(t *testing.T) { + homeDir := t.TempDir() + writeDefaultConfig(t, homeDir, seiconfig.ModeFull) + + origCfg, _ := seiconfig.ReadConfigFromDir(homeDir) + origMoniker := origCfg.Chain.Moniker + + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + _, err := handler(context.Background(), map[string]any{ + "incremental": true, + "overrides": map[string]any{ + "evm.http_port": "7777", + }, + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + cfg, _ := seiconfig.ReadConfigFromDir(homeDir) + if cfg.Chain.Moniker != origMoniker { + t.Errorf("moniker changed: got %q, want %q", cfg.Chain.Moniker, origMoniker) + } +} + +func TestConfigApplier_WritesFiles(t *testing.T) { + homeDir := t.TempDir() + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + _, err := handler(context.Background(), map[string]any{ + "mode": "full", + "incremental": false, + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + for _, f := range []string{"config.toml", "app.toml"} { + path := filepath.Join(homeDir, "config", f) + info, err := os.Stat(path) + if err != nil { + t.Errorf("%s not created: %v", f, err) + continue + } + if info.Size() == 0 { + t.Errorf("%s is empty", f) + } + } +} + +func TestConfigApplier_AllModes(t *testing.T) { + modes := []string{"validator", "full", "seed", "archive"} + for _, mode := range modes { + t.Run(mode, func(t *testing.T) { + homeDir := t.TempDir() + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + _, err := handler(context.Background(), map[string]any{ + "mode": mode, + "incremental": false, + }) + if err != nil { + t.Fatalf("mode %s failed: %v", mode, err) + } + + cfg, err := seiconfig.ReadConfigFromDir(homeDir) + if err != nil { + t.Fatalf("reading back %s config: %v", mode, err) + } + if string(cfg.Mode) != mode { + t.Errorf("mode: got %q, want %q", cfg.Mode, mode) + } + }) + } +} + +// The controller expresses a seed only as ConfigIntent{Mode: "seed"}; everything +// else resolves here. This pins the two properties a seed cannot work without, +// at the boundary where they are actually produced: a loopback P2P bind accepts +// no peers, and seid will not construct a seed node without pex. +func TestConfigApplier_SeedIsReachable(t *testing.T) { + homeDir := t.TempDir() + handler := NewConfigApplier(homeDir).Handler() + + if _, err := handler(context.Background(), map[string]any{ + "mode": string(seiconfig.ModeSeed), + "incremental": false, + }); err != nil { + t.Fatalf("applying seed config: %v", err) + } + + cfg, err := seiconfig.ReadConfigFromDir(homeDir) + if err != nil { + t.Fatalf("reading back seed config: %v", err) + } + if got, want := cfg.Network.P2P.ListenAddress, "tcp://0.0.0.0:26656"; got != want { + t.Errorf("seed p2p listen_address: got %q, want %q", got, want) + } + if !cfg.Network.P2P.PexReactor { + t.Error("seed must have pex enabled") + } +} + +// The sidecar is the last gate before seid boots, so a seed config that reaches +// it with pex stripped must fail the task rather than write a config seid +// rejects at startup. +func TestConfigApplier_SeedWithoutPexRejected(t *testing.T) { + homeDir := t.TempDir() + handler := NewConfigApplier(homeDir).Handler() + + _, err := handler(context.Background(), map[string]any{ + "mode": string(seiconfig.ModeSeed), + "incremental": false, + "overrides": map[string]any{"network.p2p.pex": "false"}, + }) + if err == nil { + t.Fatal("a seed with pex disabled must fail config-apply") + } + if !strings.Contains(err.Error(), "pex") { + t.Errorf("error should name the offending key, got: %v", err) + } +} diff --git a/sidecar/tasks/config_reload.go b/sidecar/tasks/config_reload.go new file mode 100644 index 00000000..fc7494f2 --- /dev/null +++ b/sidecar/tasks/config_reload.go @@ -0,0 +1,82 @@ +package tasks + +import ( + "context" + "fmt" + + seiconfig "github.com/sei-protocol/sei-config" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +var reloadLog = seilog.NewLogger("seictl", "task", "config-reload") + +// ConfigReloadRequest holds the typed parameters for the config-reload task. +type ConfigReloadRequest struct { + Fields map[string]string `json:"fields"` +} + +// ConfigReloader patches hot-reloadable fields on disk and signals seid +// to re-read its configuration. The signal mechanism is deferred to a future +// release; for now only the on-disk write is performed. +type ConfigReloader struct { + homeDir string +} + +// NewConfigReloader creates a reloader targeting the given home directory. +func NewConfigReloader(homeDir string) *ConfigReloader { + return &ConfigReloader{homeDir: homeDir} +} + +// Handler returns an engine.TaskHandler for the config-reload task type. +func (r *ConfigReloader) Handler() engine.TaskHandler { + return engine.TypedHandler(func(_ context.Context, params ConfigReloadRequest) error { + if len(params.Fields) == 0 { + return fmt.Errorf("config-reload: at least one field is required") + } + + registry := seiconfig.BuildRegistry() + registry.EnrichAll(seiconfig.DefaultEnrichments()) + + var nonHotReload []string + for key := range params.Fields { + f := registry.Field(key) + if f == nil { + return fmt.Errorf("config-reload: unknown field %q", key) + } + if !f.HotReload { + nonHotReload = append(nonHotReload, key) + } + } + if len(nonHotReload) > 0 { + return fmt.Errorf( + "config-reload: fields %v are not hot-reloadable and require a restart", + nonHotReload) + } + + cfg, err := seiconfig.ReadConfigFromDir(r.homeDir) + if err != nil { + return fmt.Errorf("config-reload: reading config: %w", err) + } + + if err := seiconfig.ApplyOverrides(cfg, params.Fields); err != nil { + return fmt.Errorf("config-reload: applying fields: %w", err) + } + + vr := seiconfig.Validate(cfg) + if vr.HasErrors() { + return validationError(vr) + } + + if err := seiconfig.WriteConfigToDir(cfg, r.homeDir); err != nil { + return fmt.Errorf("config-reload: writing config: %w", err) + } + + // TODO: signal seid to re-read config (SIGHUP or API call) + reloadLog.Info("hot-reloadable fields written, seid signal pending implementation", + "fields", len(params.Fields)) + + return nil + }) +} diff --git a/sidecar/tasks/config_reload_test.go b/sidecar/tasks/config_reload_test.go new file mode 100644 index 00000000..25d85111 --- /dev/null +++ b/sidecar/tasks/config_reload_test.go @@ -0,0 +1,88 @@ +package tasks + +import ( + "context" + "strings" + "testing" + + seiconfig "github.com/sei-protocol/sei-config" +) + +func TestConfigReloader_HotReloadableField(t *testing.T) { + homeDir := t.TempDir() + writeDefaultConfig(t, homeDir, seiconfig.ModeFull) + + reloader := NewConfigReloader(homeDir) + handler := reloader.Handler() + + _, err := handler(context.Background(), map[string]any{ + "fields": map[string]any{ + "logging.level": "debug", + }, + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + cfg, err := seiconfig.ReadConfigFromDir(homeDir) + if err != nil { + t.Fatalf("reading config: %v", err) + } + if cfg.Logging.Level != "debug" { + t.Errorf("logging.level: got %q, want %q", cfg.Logging.Level, "debug") + } +} + +func TestConfigReloader_NonHotReloadableField(t *testing.T) { + homeDir := t.TempDir() + writeDefaultConfig(t, homeDir, seiconfig.ModeFull) + + reloader := NewConfigReloader(homeDir) + handler := reloader.Handler() + + _, err := handler(context.Background(), map[string]any{ + "fields": map[string]any{ + "storage.db_backend": "rocksdb", + }, + }) + if err == nil { + t.Fatal("expected error for non-hot-reloadable field") + } + if !strings.Contains(err.Error(), "not hot-reloadable") { + t.Errorf("error should mention hot-reloadable: %v", err) + } +} + +func TestConfigReloader_UnknownField(t *testing.T) { + homeDir := t.TempDir() + writeDefaultConfig(t, homeDir, seiconfig.ModeFull) + + reloader := NewConfigReloader(homeDir) + handler := reloader.Handler() + + _, err := handler(context.Background(), map[string]any{ + "fields": map[string]any{ + "nonexistent.field": "value", + }, + }) + if err == nil { + t.Fatal("expected error for unknown field") + } + if !strings.Contains(err.Error(), "unknown field") { + t.Errorf("error should mention unknown field: %v", err) + } +} + +func TestConfigReloader_EmptyFields(t *testing.T) { + homeDir := t.TempDir() + reloader := NewConfigReloader(homeDir) + handler := reloader.Handler() + + _, err := handler(context.Background(), map[string]any{}) + if err == nil { + t.Fatal("expected error for empty fields") + } + if !strings.Contains(err.Error(), "at least one field") { + t.Errorf("error should mention at least one field: %v", err) + } +} diff --git a/sidecar/tasks/config_test.go b/sidecar/tasks/config_test.go new file mode 100644 index 00000000..b75d131e --- /dev/null +++ b/sidecar/tasks/config_test.go @@ -0,0 +1,327 @@ +package tasks + +import ( + "context" + "os" + "path/filepath" + "testing" + + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/tomlpatch" +) + +func setupConfigFile(t *testing.T, homeDir, content string) string { + t.Helper() + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatalf("creating config dir: %v", err) + } + configPath := filepath.Join(configDir, "config.toml") + if err := os.WriteFile(configPath, []byte(content), 0o644); err != nil { + t.Fatalf("writing config.toml: %v", err) + } + return configPath +} + +func setupAppFile(t *testing.T, homeDir, content string) string { + t.Helper() + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatalf("creating config dir: %v", err) + } + appPath := filepath.Join(configDir, "app.toml") + if err := os.WriteFile(appPath, []byte(content), 0o644); err != nil { + t.Fatalf("writing app.toml: %v", err) + } + return appPath +} + +func readTOML(t *testing.T, path string) map[string]any { + t.Helper() + doc, err := tomlpatch.ReadTOML(path) + if err != nil { + t.Fatalf("reading TOML %s: %v", path, err) + } + return doc +} + +func TestConfigPatcherMergesConfigToml(t *testing.T) { + homeDir := t.TempDir() + configPath := setupConfigFile(t, homeDir, ` +[p2p] +persistent-peers = "" +laddr = "tcp://0.0.0.0:26656" +`) + + patcher := NewConfigPatcher(homeDir) + err := patcher.PatchFiles(context.Background(), map[string]any{ + "config.toml": map[string]any{ + "p2p": map[string]any{ + "persistent-peers": "abc@1.2.3.4:26656,def@5.6.7.8:26656", + }, + }, + }) + if err != nil { + t.Fatalf("PatchFiles failed: %v", err) + } + + doc := readTOML(t, configPath) + p2p := doc["p2p"].(map[string]any) + if p2p["persistent-peers"] != "abc@1.2.3.4:26656,def@5.6.7.8:26656" { + t.Fatalf("expected peers, got %q", p2p["persistent-peers"]) + } + if p2p["laddr"] != "tcp://0.0.0.0:26656" { + t.Fatalf("expected laddr preserved, got %q", p2p["laddr"]) + } +} + +func TestConfigPatcherMergesMultipleFiles(t *testing.T) { + homeDir := t.TempDir() + configPath := setupConfigFile(t, homeDir, ` +[statesync] +enable = false +`) + appPath := setupAppFile(t, homeDir, ` +pruning = "default" +snapshot-interval = 0 +`) + + patcher := NewConfigPatcher(homeDir) + err := patcher.PatchFiles(context.Background(), map[string]any{ + "config.toml": map[string]any{ + "statesync": map[string]any{ + "use-local-snapshot": true, + "backfill-blocks": int64(0), + "trust-period": "9999h0m0s", + }, + }, + "app.toml": map[string]any{ + "pruning": "nothing", + "snapshot-interval": int64(2000), + "snapshot-keep-recent": int64(5), + }, + }) + if err != nil { + t.Fatalf("PatchFiles failed: %v", err) + } + + configDoc := readTOML(t, configPath) + ss := configDoc["statesync"].(map[string]any) + if ss["use-local-snapshot"] != true { + t.Fatal("expected use-local-snapshot = true") + } + if ss["trust-period"] != "9999h0m0s" { + t.Fatalf("expected trust-period = 9999h0m0s, got %v", ss["trust-period"]) + } + + appDoc := readTOML(t, appPath) + if appDoc["pruning"] != "nothing" { + t.Fatalf("expected pruning = nothing, got %v", appDoc["pruning"]) + } + if appDoc["snapshot-interval"] != int64(2000) { + t.Fatalf("expected snapshot-interval = 2000, got %v", appDoc["snapshot-interval"]) + } +} + +func TestConfigPatcherPreservesUnrelatedFields(t *testing.T) { + homeDir := t.TempDir() + configPath := setupConfigFile(t, homeDir, ` +[consensus] +timeout-commit = "5s" + +[p2p] +persistent-peers = "" +max-num-inbound-peers = 40 + +[mempool] +size = 5000 +`) + + patcher := NewConfigPatcher(homeDir) + err := patcher.PatchFiles(context.Background(), map[string]any{ + "config.toml": map[string]any{ + "p2p": map[string]any{ + "persistent-peers": "abc@1.2.3.4:26656", + }, + }, + }) + if err != nil { + t.Fatalf("PatchFiles failed: %v", err) + } + + doc := readTOML(t, configPath) + consensus := doc["consensus"].(map[string]any) + if consensus["timeout-commit"] != "5s" { + t.Fatal("consensus.timeout-commit not preserved") + } + p2p := doc["p2p"].(map[string]any) + if p2p["max-num-inbound-peers"] != int64(40) { + t.Fatal("p2p.max-num-inbound-peers not preserved") + } + mempool := doc["mempool"].(map[string]any) + if mempool["size"] != int64(5000) { + t.Fatal("mempool.size not preserved") + } +} + +func TestConfigPatcherCreatesNewSections(t *testing.T) { + homeDir := t.TempDir() + configPath := setupConfigFile(t, homeDir, ` +[p2p] +persistent-peers = "" +`) + + patcher := NewConfigPatcher(homeDir) + err := patcher.PatchFiles(context.Background(), map[string]any{ + "config.toml": map[string]any{ + "statesync": map[string]any{ + "enable": true, + }, + }, + }) + if err != nil { + t.Fatalf("PatchFiles failed: %v", err) + } + + doc := readTOML(t, configPath) + ss := doc["statesync"].(map[string]any) + if ss["enable"] != true { + t.Fatal("expected statesync.enable = true") + } +} + +func TestConfigPatcherHandlerRejectsEmptyFiles(t *testing.T) { + patcher := NewConfigPatcher(t.TempDir()) + handler := patcher.Handler() + _, err := handler(context.Background(), map[string]any{}) + if err == nil { + t.Fatal("expected error for empty files, got nil") + } +} + +func TestConfigPatcherRejectsNonMapValue(t *testing.T) { + homeDir := t.TempDir() + setupConfigFile(t, homeDir, ` +[p2p] +persistent-peers = "" +`) + + patcher := NewConfigPatcher(homeDir) + err := patcher.PatchFiles(context.Background(), map[string]any{ + "config.toml": "not a map", + }) + if err == nil { + t.Fatal("expected error for non-map file value") + } +} + +func TestConfigPatcherHandlerRoundTrip(t *testing.T) { + homeDir := t.TempDir() + configPath := setupConfigFile(t, homeDir, ` +[p2p] +persistent-peers = "" +`) + + patcher := NewConfigPatcher(homeDir) + handler := patcher.Handler() + _, err := handler(context.Background(), map[string]any{ + "files": map[string]any{ + "config.toml": map[string]any{ + "p2p": map[string]any{ + "persistent-peers": "node1@1.2.3.4:26656", + }, + }, + }, + }) + if err != nil { + t.Fatalf("Handler failed: %v", err) + } + + doc := readTOML(t, configPath) + p2p := doc["p2p"].(map[string]any) + if p2p["persistent-peers"] != "node1@1.2.3.4:26656" { + t.Fatalf("expected peers via handler, got %q", p2p["persistent-peers"]) + } +} + +func TestConfigPatcherCreatesFileIfMissing(t *testing.T) { + homeDir := t.TempDir() + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatal(err) + } + + patcher := NewConfigPatcher(homeDir) + err := patcher.PatchFiles(context.Background(), map[string]any{ + "app.toml": map[string]any{ + "pruning": "nothing", + }, + }) + if err != nil { + t.Fatalf("PatchFiles failed on missing file: %v", err) + } + + appPath := filepath.Join(configDir, "app.toml") + doc := readTOML(t, appPath) + if doc["pruning"] != "nothing" { + t.Fatalf("expected pruning=nothing in newly created file, got %v", doc["pruning"]) + } +} + +func TestWritePeersToConfig(t *testing.T) { + homeDir := t.TempDir() + configPath := setupConfigFile(t, homeDir, ` +[p2p] +persistent-peers = "" +laddr = "tcp://0.0.0.0:26656" +`) + + err := writePeersToConfig(homeDir, []string{"abc@1.2.3.4:26656", "def@5.6.7.8:26656"}) + if err != nil { + t.Fatalf("writePeersToConfig failed: %v", err) + } + + doc := readTOML(t, configPath) + p2p := doc["p2p"].(map[string]any) + if p2p["persistent-peers"] != "abc@1.2.3.4:26656,def@5.6.7.8:26656" { + t.Fatalf("expected joined peers, got %q", p2p["persistent-peers"]) + } + if p2p["laddr"] != "tcp://0.0.0.0:26656" { + t.Fatal("laddr not preserved") + } +} + +func TestWriteStateSyncToConfig(t *testing.T) { + homeDir := t.TempDir() + configPath := setupConfigFile(t, homeDir, ` +[statesync] +enable = false +trust-height = 0 +trust-hash = "" +rpc-servers = "" +`) + + cfg := StateSyncConfig{ + TrustHeight: 500000, + TrustHash: "ABCDEF1234567890", + RpcServers: "1.2.3.4:26657,5.6.7.8:26657", + } + err := writeStateSyncToConfig(homeDir, cfg) + if err != nil { + t.Fatalf("writeStateSyncToConfig failed: %v", err) + } + + doc := readTOML(t, configPath) + ss := doc["statesync"].(map[string]any) + if ss["enable"] != true { + t.Fatal("expected enable=true") + } + if ss["trust-height"] != int64(500000) { + t.Fatalf("expected trust-height=500000, got %v", ss["trust-height"]) + } + if ss["trust-hash"] != "ABCDEF1234567890" { + t.Fatalf("expected trust-hash, got %v", ss["trust-hash"]) + } + if ss["rpc-servers"] != "1.2.3.4:26657,5.6.7.8:26657" { + t.Fatalf("expected rpc-servers, got %v", ss["rpc-servers"]) + } +} diff --git a/sidecar/tasks/config_validate.go b/sidecar/tasks/config_validate.go new file mode 100644 index 00000000..0e86842f --- /dev/null +++ b/sidecar/tasks/config_validate.go @@ -0,0 +1,77 @@ +package tasks + +import ( + "context" + "encoding/json" + "fmt" + + seiconfig "github.com/sei-protocol/sei-config" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +var validateLog = seilog.NewLogger("seictl", "task", "config-validate") + +// ConfigValidator reads on-disk config and returns validation diagnostics. +type ConfigValidator struct { + homeDir string +} + +// NewConfigValidator creates a validator targeting the given home directory. +func NewConfigValidator(homeDir string) *ConfigValidator { + return &ConfigValidator{homeDir: homeDir} +} + +// Handler returns an engine.TaskHandler for the config-validate task type. +func (v *ConfigValidator) Handler() engine.TaskHandler { + return engine.TypedHandler(func(_ context.Context, _ struct{}) error { + cfg, err := seiconfig.ReadConfigFromDir(v.homeDir) + if err != nil { + return fmt.Errorf("config-validate: reading config: %w", err) + } + + vr := seiconfig.Validate(cfg) + + type resultJSON struct { + Valid bool `json:"valid"` + Version int `json:"version"` + Mode string `json:"mode"` + Diagnostics []struct { + Severity string `json:"severity"` + Field string `json:"field"` + Message string `json:"message"` + } `json:"diagnostics"` + } + + result := resultJSON{ + Valid: !vr.HasErrors(), + Version: cfg.Version, + Mode: string(cfg.Mode), + } + for _, d := range vr.Diagnostics { + result.Diagnostics = append(result.Diagnostics, struct { + Severity string `json:"severity"` + Field string `json:"field"` + Message string `json:"message"` + }{ + Severity: d.Severity.String(), + Field: d.Field, + Message: d.Message, + }) + } + + if vr.HasErrors() { + data, _ := json.Marshal(result) + return fmt.Errorf("config validation failed: %s", data) + } + + validateLog.Info("config validated", + "valid", result.Valid, + "version", result.Version, + "mode", result.Mode, + "diagnostics", len(result.Diagnostics), + ) + return nil + }) +} diff --git a/sidecar/tasks/config_validate_test.go b/sidecar/tasks/config_validate_test.go new file mode 100644 index 00000000..3591f787 --- /dev/null +++ b/sidecar/tasks/config_validate_test.go @@ -0,0 +1,75 @@ +package tasks + +import ( + "context" + "strings" + "testing" + + seiconfig "github.com/sei-protocol/sei-config" +) + +func TestConfigValidator_ValidConfig(t *testing.T) { + homeDir := t.TempDir() + writeDefaultConfig(t, homeDir, seiconfig.ModeValidator) + + validator := NewConfigValidator(homeDir) + handler := validator.Handler() + + _, err := handler(context.Background(), nil) + if err != nil { + t.Fatalf("unexpected error for valid config: %v", err) + } +} + +func TestConfigValidator_AllModes(t *testing.T) { + modes := []seiconfig.NodeMode{ + seiconfig.ModeValidator, seiconfig.ModeFull, seiconfig.ModeSeed, + seiconfig.ModeArchive, + } + for _, mode := range modes { + t.Run(string(mode), func(t *testing.T) { + homeDir := t.TempDir() + writeDefaultConfig(t, homeDir, mode) + + validator := NewConfigValidator(homeDir) + _, err := validator.Handler()(context.Background(), nil) + if err != nil { + t.Fatalf("mode %s validation failed: %v", mode, err) + } + }) + } +} + +func TestConfigValidator_MissingFiles(t *testing.T) { + homeDir := t.TempDir() + validator := NewConfigValidator(homeDir) + handler := validator.Handler() + + _, err := handler(context.Background(), nil) + if err == nil { + t.Fatal("expected error for missing config files") + } + if !strings.Contains(err.Error(), "reading config") { + t.Errorf("error should mention reading: %v", err) + } +} + +func TestConfigValidator_InvalidConfig(t *testing.T) { + homeDir := t.TempDir() + cfg := seiconfig.DefaultForMode(seiconfig.ModeValidator) + cfg.Chain.MinGasPrices = "" // triggers validation error + if err := seiconfig.WriteConfigToDir(cfg, homeDir); err != nil { + t.Fatal(err) + } + + validator := NewConfigValidator(homeDir) + handler := validator.Handler() + + _, err := handler(context.Background(), nil) + if err == nil { + t.Fatal("expected error for invalid config") + } + if !strings.Contains(err.Error(), "validation failed") { + t.Errorf("error should mention validation: %v", err) + } +} diff --git a/sidecar/tasks/defaults/config.toml b/sidecar/tasks/defaults/config.toml new file mode 100644 index 00000000..81c8724a --- /dev/null +++ b/sidecar/tasks/defaults/config.toml @@ -0,0 +1,18 @@ +[base] +mode = "full" + +[p2p] +persistent-peers = "" +laddr = "tcp://0.0.0.0:26656" + +[statesync] +enable = false +trust-height = 0 +trust-hash = "" +rpc-servers = "" + +[consensus] +timeout-commit = "5s" + +[mempool] +size = 5000 diff --git a/sidecar/tasks/defaults/defaults.go b/sidecar/tasks/defaults/defaults.go new file mode 100644 index 00000000..5ecef941 --- /dev/null +++ b/sidecar/tasks/defaults/defaults.go @@ -0,0 +1,6 @@ +package defaults + +import "embed" + +//go:embed config.toml +var FS embed.FS diff --git a/sidecar/tasks/evm_logical_digest.go b/sidecar/tasks/evm_logical_digest.go new file mode 100644 index 00000000..5fba6ce2 --- /dev/null +++ b/sidecar/tasks/evm_logical_digest.go @@ -0,0 +1,337 @@ +package tasks + +import ( + "bufio" + "context" + "errors" + "fmt" + "os/exec" + "strconv" + "strings" + "time" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +var evmDigestLog = seilog.NewLogger("seictl", "task", "evm-logical-digest") + +const defaultSeidbPath = "seidb" + +// defaultNormalizations are the memiavl normalization modes proved per run. +// "semantic" decodes raw memiavl EVM leaves independently; "translator" runs +// them through flatkv.ImportTranslator. Proving both guards against a bug in +// either decoder masking a real divergence. +var defaultNormalizations = []string{"semantic", "translator"} + +// EvmLogicalDigestRequest parameters the evm-logical-digest task. It shells out +// to seidb's evm-logical-digest for a flatkv clone and a memiavl snapshot at the +// same height, compares the backend-independent logical digests, and publishes +// the verdict to S3. +type EvmLogicalDigestRequest struct { + FlatKVDir string `json:"flatkvDir"` + MemIAVLDir string `json:"memiavlDir"` + Height int64 `json:"height"` + Bucket string `json:"bucket"` + Prefix string `json:"prefix"` + Region string `json:"region"` + + // Normalizations is the set of memiavl normalization modes to prove. + // Defaults to ["semantic","translator"] when empty. + Normalizations []string `json:"normalizations"` + + // SeidbPath is the seidb binary to exec. Defaults to "seidb" (PATH lookup). + SeidbPath string `json:"seidbPath"` +} + +// bucketDigests holds the per-bucket and final logical digests parsed from one +// seidb run. All values are lowercase hex as printed by seidb. +type bucketDigests struct { + Account string `json:"account"` + Code string `json:"code"` + Storage string `json:"storage"` + Legacy string `json:"legacy"` + Final string `json:"final"` + Version int64 `json:"version"` +} + +// EndpointDigestRecord is the published verdict for one (height, normalization) +// comparison — the durable artifact a reader trusts. Its own SHA-256 seal lives +// out-of-band (the s3 helper's EmitResult, logged + in the TaskResult): a record +// cannot carry the hash of its own published bytes. +type EndpointDigestRecord struct { + Height int64 `json:"height"` + Normalization string `json:"normalization"` + FlatKVDigest string `json:"flatkv_digest"` + MemIAVLDigest string `json:"memiavl_digest"` + PerBucket map[string]bucket `json:"per_bucket"` + Match bool `json:"match"` + AxesProved []string `json:"axes_proved"` + GeneratedAt string `json:"generated_at"` +} + +// bucket pairs the flatkv and memiavl digest for one canonical bucket. +type bucket struct { + FlatKV string `json:"flatkv"` + MemIAVL string `json:"memiavl"` + Match bool `json:"match"` +} + +// EvmLogicalDigester runs the comparison. It holds no per-request state. +type EvmLogicalDigester struct { + s3UploaderFactory seis3.UploaderFactory +} + +// NewEvmLogicalDigester builds the task handler dependency. A nil factory uses +// the default real-S3 uploader. +func NewEvmLogicalDigester(factory seis3.UploaderFactory) *EvmLogicalDigester { + if factory == nil { + factory = seis3.DefaultUploaderFactory + } + return &EvmLogicalDigester{s3UploaderFactory: factory} +} + +func (d *EvmLogicalDigester) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, req EvmLogicalDigestRequest) error { + return d.run(ctx, req) + }) +} + +func (d *EvmLogicalDigester) run(ctx context.Context, req EvmLogicalDigestRequest) error { + if err := validateDigestRequest(&req); err != nil { + return err + } + + uploader, err := d.s3UploaderFactory(ctx, req.Region) + if err != nil { + return fmt.Errorf("evm-logical-digest: building S3 uploader: %w", err) + } + prefix := normalizePrefix(req.Prefix) + + // FlatKV is normalization-independent (native physical keyspace), so it is + // digested once and compared against each memiavl normalization. + flatkv, err := d.runSeidb(ctx, req.SeidbPath, "flatkv", req.FlatKVDir, req.Height, "") + if err != nil { + return fmt.Errorf("evm-logical-digest: flatkv digest: %w", err) + } + // Trust the run, not the request: a digest taken at the wrong height is a + // silent false match. seidb WAL-replays flatkv to --height, so the opened + // version MUST equal the requested height. + if flatkv.Version != req.Height { + return fmt.Errorf("evm-logical-digest: flatkv opened version %d != requested height %d", flatkv.Version, req.Height) + } + + for _, norm := range req.Normalizations { + memiavl, err := d.runSeidb(ctx, req.SeidbPath, "memiavl", req.MemIAVLDir, req.Height, norm) + if err != nil { + return fmt.Errorf("evm-logical-digest: memiavl digest (%s): %w", norm, err) + } + // Symmetric with the flatkv check: if seidb clamps to the nearest + // available snapshot instead of erroring, the comparison would run at + // the wrong height — a silent false match in the degenerate case. + if memiavl.Version != req.Height { + return fmt.Errorf("evm-logical-digest: memiavl opened version %d != requested height %d (%s)", memiavl.Version, req.Height, norm) + } + + record := buildEndpointDigest(req.Height, norm, flatkv, memiavl) + key := fmt.Sprintf("%sendpoint-digest-%d-%s.json.gz", prefix, req.Height, norm) + + emit, err := seis3.StreamGzipJSON(ctx, uploader, req.Bucket, key, record) + if err != nil { + return seis3.ClassifyS3Error("evm-logical-digest", req.Bucket, key, req.Region, err) + } + + evmDigestLog.Info("published endpoint digest", + "height", req.Height, + "normalization", norm, + "match", record.Match, + "flatkv-digest", record.FlatKVDigest, + "memiavl-digest", record.MemIAVLDigest, + "key", key, + "sha256", emit.UncompressedSHA256) + } + + return nil +} + +func validateDigestRequest(req *EvmLogicalDigestRequest) error { + if req.Bucket == "" { + return fmt.Errorf("evm-logical-digest: missing required param 'bucket'") + } + if req.Region == "" { + return fmt.Errorf("evm-logical-digest: missing required param 'region'") + } + if req.Height <= 0 { + return fmt.Errorf("evm-logical-digest: 'height' must be a positive block height, got %d", req.Height) + } + if req.FlatKVDir == "" { + return fmt.Errorf("evm-logical-digest: missing required param 'flatkvDir'") + } + if req.MemIAVLDir == "" { + return fmt.Errorf("evm-logical-digest: missing required param 'memiavlDir'") + } + if req.SeidbPath == "" { + req.SeidbPath = defaultSeidbPath + } + if len(req.Normalizations) == 0 { + req.Normalizations = defaultNormalizations + } + for _, n := range req.Normalizations { + if n != "semantic" && n != "translator" { + return fmt.Errorf("evm-logical-digest: unknown normalization %q (want semantic|translator)", n) + } + } + return nil +} + +// runSeidb execs `seidb evm-logical-digest` for one backend and parses the +// per-bucket + FINAL_DIGEST lines and the opened version from its stdout. +func (d *EvmLogicalDigester) runSeidb(ctx context.Context, seidbPath, backend, dir string, height int64, normalization string) (bucketDigests, error) { + args := []string{ + "evm-logical-digest", + "--backend", backend, + "-d", dir, + "--height", strconv.FormatInt(height, 10), + } + if backend == "memiavl" && normalization != "" { + args = append(args, "--memiavl-normalization", normalization) + } + + cmd := exec.CommandContext(ctx, seidbPath, args...) + // Stderr is left nil so Output() captures it into ExitError.Stderr, which + // stderrTail surfaces on failure. + out, err := cmd.Output() + if err != nil { + return bucketDigests{}, fmt.Errorf("running %s %s: %w%s", seidbPath, strings.Join(args, " "), err, stderrTail(err)) + } + return parseDigestOutput(string(out)) +} + +// stderrTail surfaces the captured stderr from an *exec.ExitError so a seidb +// failure is actionable instead of a bare "exit status 1". +func stderrTail(err error) string { + var exitErr *exec.ExitError + if errors.As(err, &exitErr) && len(exitErr.Stderr) > 0 { + return fmt.Sprintf(" (stderr: %s)", strings.TrimSpace(string(exitErr.Stderr))) + } + return "" +} + +// parseDigestOutput extracts the per-bucket digests, the final digest, and the +// opened version from one seidb run's stdout. It is fail-closed: a missing +// FINAL_DIGEST, a missing bucket, or a missing version is an error, never a +// zero-valued "match". +func parseDigestOutput(out string) (bucketDigests, error) { + var bd bucketDigests + var ( + haveVersion bool + haveAccount bool + haveCode bool + haveStorage bool + haveLegacy bool + haveFinal bool + ) + + scanner := bufio.NewScanner(strings.NewReader(out)) + // seidb prints raw bytecode/values nowhere on stdout, but bump the buffer + // so a long context/progress line never trips the default 64 KiB cap. + scanner.Buffer(make([]byte, 0, 1024*1024), 4*1024*1024) + for scanner.Scan() { + fields := strings.Fields(scanner.Text()) + if len(fields) == 0 { + continue + } + switch fields[0] { + case "version:": + if len(fields) < 2 { + return bucketDigests{}, fmt.Errorf("seidb output: malformed 'version:' line %q", scanner.Text()) + } + v, err := strconv.ParseInt(fields[1], 10, 64) + if err != nil { + return bucketDigests{}, fmt.Errorf("parsing version %q: %w", fields[1], err) + } + bd.Version = v + haveVersion = true + case "account": + if v, ok := digestField(fields, "bucket_digest="); ok { + bd.Account = v + haveAccount = true + } + case "code": + if v, ok := digestField(fields, "bucket_digest="); ok { + bd.Code = v + haveCode = true + } + case "storage": + if v, ok := digestField(fields, "bucket_digest="); ok { + bd.Storage = v + haveStorage = true + } + case "legacy": + if v, ok := digestField(fields, "bucket_digest="); ok { + bd.Legacy = v + haveLegacy = true + } + case "FINAL_DIGEST": + if v, ok := digestField(fields, "digest="); ok { + bd.Final = v + haveFinal = true + } + } + } + if err := scanner.Err(); err != nil { + return bucketDigests{}, fmt.Errorf("scanning seidb output: %w", err) + } + + switch { + case !haveVersion: + return bucketDigests{}, fmt.Errorf("seidb output missing 'version:' line") + case !haveFinal: + return bucketDigests{}, fmt.Errorf("seidb output missing FINAL_DIGEST line") + case !haveAccount || !haveCode || !haveStorage || !haveLegacy: + return bucketDigests{}, fmt.Errorf("seidb output missing a per-bucket digest (account=%t code=%t storage=%t legacy=%t)", + haveAccount, haveCode, haveStorage, haveLegacy) + } + return bd, nil +} + +// digestField returns the hex value of the field with the given prefix +// (e.g. "bucket_digest="), and false if no such field is present. +func digestField(fields []string, prefix string) (string, bool) { + for _, f := range fields { + if rest, ok := strings.CutPrefix(f, prefix); ok { + return rest, true + } + } + return "", false +} + +func buildEndpointDigest(height int64, normalization string, flatkv, memiavl bucketDigests) EndpointDigestRecord { + perBucket := map[string]bucket{ + "account": {FlatKV: flatkv.Account, MemIAVL: memiavl.Account, Match: flatkv.Account == memiavl.Account}, + "code": {FlatKV: flatkv.Code, MemIAVL: memiavl.Code, Match: flatkv.Code == memiavl.Code}, + "storage": {FlatKV: flatkv.Storage, MemIAVL: memiavl.Storage, Match: flatkv.Storage == memiavl.Storage}, + "legacy": {FlatKV: flatkv.Legacy, MemIAVL: memiavl.Legacy, Match: flatkv.Legacy == memiavl.Legacy}, + } + return EndpointDigestRecord{ + Height: height, + Normalization: normalization, + FlatKVDigest: flatkv.Final, + MemIAVLDigest: memiavl.Final, + PerBucket: perBucket, + Match: flatkv.Final == memiavl.Final, + AxesProved: axesProved(normalization), + GeneratedAt: time.Now().UTC().Format(time.RFC3339), + } +} + +// axesProved records which logical axes this digest actually proves. The +// memiavl EVM keyspace carries no balance, and the semantic account decoder +// zeroes the balance field of the account payload, so neither normalization's +// account digest attests balance equivalence — that is the per-block +// comparator's job. We never claim "balance" here; a reader must not infer it. +func axesProved(_ string) []string { + return []string{"nonce", "code", "code_hash", "storage", "legacy"} +} diff --git a/sidecar/tasks/evm_logical_digest_test.go b/sidecar/tasks/evm_logical_digest_test.go new file mode 100644 index 00000000..066447cc --- /dev/null +++ b/sidecar/tasks/evm_logical_digest_test.go @@ -0,0 +1,329 @@ +package tasks + +import ( + "bytes" + "compress/gzip" + "context" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + "runtime" + "strings" + "sync" + "testing" + + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +// digestRecordingUploader captures every put body keyed by S3 key. +type digestRecordingUploader struct { + mu sync.Mutex + objects map[string][]byte +} + +func (u *digestRecordingUploader) UploadObject(_ context.Context, in *transfermanager.UploadObjectInput, _ ...func(*transfermanager.Options)) (*transfermanager.UploadObjectOutput, error) { + var buf bytes.Buffer + if in.Body != nil { + if _, err := io.Copy(&buf, in.Body); err != nil { + return nil, err + } + } + u.mu.Lock() + defer u.mu.Unlock() + if u.objects == nil { + u.objects = map[string][]byte{} + } + u.objects[*in.Key] = buf.Bytes() + return &transfermanager.UploadObjectOutput{}, nil +} + +func (u *digestRecordingUploader) factory() seis3.UploaderFactory { + return func(context.Context, string) (seis3.Uploader, error) { return u, nil } +} + +func (u *digestRecordingUploader) record(t *testing.T, key string) EndpointDigestRecord { + t.Helper() + u.mu.Lock() + raw, ok := u.objects[key] + u.mu.Unlock() + if !ok { + t.Fatalf("no object published at key %q; have %v", key, u.keys()) + } + gr, err := gzip.NewReader(bytes.NewReader(raw)) + if err != nil { + t.Fatalf("gzip: %v", err) + } + dec := json.NewDecoder(gr) + var rec EndpointDigestRecord + if err := dec.Decode(&rec); err != nil { + t.Fatalf("decode record: %v", err) + } + return rec +} + +func (u *digestRecordingUploader) keys() []string { + u.mu.Lock() + defer u.mu.Unlock() + ks := make([]string, 0, len(u.objects)) + for k := range u.objects { + ks = append(ks, k) + } + return ks +} + +// fakeSeidb writes an executable shim that prints canned digest output. The +// shim branches on --backend and --memiavl-normalization and echoes the version +// it was asked for via --height so version-assertion paths can be exercised. +// flatkvVersionOverride / memiavlVersionOverride, when non-empty, replace the +// printed version for that backend's run to simulate a height mismatch (e.g. a +// snapshot tool clamping to the nearest available height). +func fakeSeidb(t *testing.T, flatkvVersionOverride, memiavlVersionOverride string) string { + t.Helper() + if runtime.GOOS == "windows" { + t.Skip("shell shim not portable to windows") + } + dir := t.TempDir() + path := filepath.Join(dir, "seidb") + flatkvVer := `$height` + if flatkvVersionOverride != "" { + flatkvVer = flatkvVersionOverride + } + memiavlVer := `$height` + if memiavlVersionOverride != "" { + memiavlVer = memiavlVersionOverride + } + // The shim parses --backend, --height, --memiavl-normalization from argv and + // prints a digest block whose values depend on backend+normalization. + script := fmt.Sprintf(`#!/bin/sh +backend="" +height="" +norm="" +while [ $# -gt 0 ]; do + case "$1" in + --backend) backend="$2"; shift 2;; + --height) height="$2"; shift 2;; + --memiavl-normalization) norm="$2"; shift 2;; + *) shift;; + esac +done +acct=aaaa +code=cccc +stor=5555 +leg=1111 +if [ "$backend" = "memiavl" ] && [ "$norm" = "translator" ]; then + stor=9999 +fi +final=ffff +if [ "$backend" = "memiavl" ] && [ "$norm" = "translator" ]; then + final=eeee +fi +if [ "$backend" = "memiavl" ]; then ver=%s; else ver=%s; fi +echo "EVM logical digest start" +echo "backend: $backend" +echo "version: $ver" +echo "" +echo "Bucket digests (final digest inputs)" +echo "account count=10 bucket_digest=$acct" +echo "code count=2 bucket_digest=$code" +echo "storage count=99 bucket_digest=$stor" +echo "legacy count=3 bucket_digest=$leg" +echo "" +echo "FINAL_DIGEST account+code+storage+legacy count=114 digest=$final" +`, memiavlVer, flatkvVer) + if err := os.WriteFile(path, []byte(script), 0o755); err != nil { + t.Fatalf("write shim: %v", err) + } + return path +} + +func baseRequest(seidbPath string) EvmLogicalDigestRequest { + return EvmLogicalDigestRequest{ + FlatKVDir: "/data/flatkv", + MemIAVLDir: "/data/memiavl", + Height: 213200000, + Bucket: "digest-bucket", + Prefix: "digests", + Region: "us-east-1", + SeidbPath: seidbPath, + } +} + +func TestEvmLogicalDigest_SemanticMatchTranslatorMismatch(t *testing.T) { + up := &digestRecordingUploader{} + d := NewEvmLogicalDigester(up.factory()) + req := baseRequest(fakeSeidb(t, "", "")) + + if err := d.run(context.Background(), req); err != nil { + t.Fatalf("run: %v", err) + } + + // semantic: flatkv final ffff == memiavl final ffff -> match. + sem := up.record(t, "digests/endpoint-digest-213200000-semantic.json.gz") + if !sem.Match { + t.Errorf("semantic should match, got %+v", sem) + } + if !sem.PerBucket["storage"].Match { + t.Error("semantic storage should match") + } + + // translator: memiavl storage/final diverge -> no match. This is the + // fail-closed proof that a real divergence is reported, not masked. + tr := up.record(t, "digests/endpoint-digest-213200000-translator.json.gz") + if tr.Match { + t.Errorf("translator should NOT match, got %+v", tr) + } + if tr.PerBucket["storage"].Match { + t.Error("translator storage should not match") + } + if tr.PerBucket["account"].Match != true { + t.Error("translator account should still match") + } + + // axes_proved must never claim balance. + for _, axis := range sem.AxesProved { + if axis == "balance" { + t.Fatal("axes_proved must not claim balance for the semantic path") + } + } +} + +func TestEvmLogicalDigest_VersionMismatchFailsClosed(t *testing.T) { + up := &digestRecordingUploader{} + d := NewEvmLogicalDigester(up.factory()) + // flatkv shim prints version 999, but the request asks for 213200000. + req := baseRequest(fakeSeidb(t, "999", "")) + + err := d.run(context.Background(), req) + if err == nil { + t.Fatal("expected version-mismatch error, got nil") + } + if !strings.Contains(err.Error(), "version") { + t.Errorf("error should mention version, got: %v", err) + } + if len(up.keys()) != 0 { + t.Errorf("nothing should be published on version mismatch, got %v", up.keys()) + } +} + +func TestEvmLogicalDigest_MemiavlVersionMismatchFailsClosed(t *testing.T) { + up := &digestRecordingUploader{} + d := NewEvmLogicalDigester(up.factory()) + // flatkv opens at the requested height, but the memiavl snapshot resolves + // to 888 (e.g. seidb clamped to the nearest available snapshot). The + // comparison must not publish — a wrong-height memiavl side is a silent + // false match in the degenerate case. + req := baseRequest(fakeSeidb(t, "", "888")) + + err := d.run(context.Background(), req) + if err == nil { + t.Fatal("expected memiavl version-mismatch error, got nil") + } + if !strings.Contains(err.Error(), "memiavl opened version") { + t.Errorf("error should name the memiavl version mismatch, got: %v", err) + } + if len(up.keys()) != 0 { + t.Errorf("nothing should be published on memiavl version mismatch, got %v", up.keys()) + } +} + +func TestEvmLogicalDigest_Validation(t *testing.T) { + d := NewEvmLogicalDigester((&digestRecordingUploader{}).factory()) + cases := map[string]func(*EvmLogicalDigestRequest){ + "missing bucket": func(r *EvmLogicalDigestRequest) { r.Bucket = "" }, + "missing region": func(r *EvmLogicalDigestRequest) { r.Region = "" }, + "missing flatkvDir": func(r *EvmLogicalDigestRequest) { r.FlatKVDir = "" }, + "missing memiavlDir": func(r *EvmLogicalDigestRequest) { r.MemIAVLDir = "" }, + "non-positive height": func(r *EvmLogicalDigestRequest) { r.Height = 0 }, + "bad normalization": func(r *EvmLogicalDigestRequest) { r.Normalizations = []string{"bogus"} }, + } + for name, mutate := range cases { + t.Run(name, func(t *testing.T) { + req := baseRequest("seidb") + mutate(&req) + if err := d.run(context.Background(), req); err == nil { + t.Fatalf("%s should fail validation", name) + } + }) + } +} + +func TestEvmLogicalDigest_DefaultsApplied(t *testing.T) { + req := EvmLogicalDigestRequest{ + FlatKVDir: "/a", MemIAVLDir: "/b", Height: 1, Bucket: "x", Region: "y", + } + if err := validateDigestRequest(&req); err != nil { + t.Fatalf("validate: %v", err) + } + if req.SeidbPath != defaultSeidbPath { + t.Errorf("SeidbPath default = %q, want %q", req.SeidbPath, defaultSeidbPath) + } + if len(req.Normalizations) != 2 || req.Normalizations[0] != "semantic" || req.Normalizations[1] != "translator" { + t.Errorf("Normalizations default = %v", req.Normalizations) + } +} + +func TestParseDigestOutput_HappyPath(t *testing.T) { + out := strings.Join([]string{ + "EVM logical digest start", + "backend: flatkv", + "version: 213200000", + "", + "account count=10 bucket_digest=aabb", + "code count=2 bucket_digest=ccdd", + "storage count=99 bucket_digest=eeff", + "legacy count=3 bucket_digest=1122", + "", + "FINAL_DIGEST account+code+storage+legacy count=114 digest=deadbeef", + }, "\n") + bd, err := parseDigestOutput(out) + if err != nil { + t.Fatalf("parse: %v", err) + } + if bd.Version != 213200000 { + t.Errorf("version = %d", bd.Version) + } + if bd.Account != "aabb" || bd.Code != "ccdd" || bd.Storage != "eeff" || bd.Legacy != "1122" { + t.Errorf("bucket digests = %+v", bd) + } + if bd.Final != "deadbeef" { + t.Errorf("final = %q", bd.Final) + } +} + +func TestParseDigestOutput_FailClosed(t *testing.T) { + cases := map[string]string{ + "missing final": strings.Join([]string{ + "version: 5", + "account count=1 bucket_digest=aa", + "code count=1 bucket_digest=bb", + "storage count=1 bucket_digest=cc", + "legacy count=1 bucket_digest=dd", + }, "\n"), + "missing version": strings.Join([]string{ + "account count=1 bucket_digest=aa", + "code count=1 bucket_digest=bb", + "storage count=1 bucket_digest=cc", + "legacy count=1 bucket_digest=dd", + "FINAL_DIGEST account+code+storage+legacy count=4 digest=ee", + }, "\n"), + "missing a bucket": strings.Join([]string{ + "version: 5", + "account count=1 bucket_digest=aa", + "storage count=1 bucket_digest=cc", + "legacy count=1 bucket_digest=dd", + "FINAL_DIGEST account+code+storage+legacy count=3 digest=ee", + }, "\n"), + "empty": "", + } + for name, out := range cases { + t.Run(name, func(t *testing.T) { + if _, err := parseDigestOutput(out); err == nil { + t.Fatalf("%s should be a parse error", name) + } + }) + } +} diff --git a/sidecar/tasks/generate_gentx.go b/sidecar/tasks/generate_gentx.go new file mode 100644 index 00000000..7c12ad98 --- /dev/null +++ b/sidecar/tasks/generate_gentx.go @@ -0,0 +1,459 @@ +package tasks + +import ( + "bytes" + "context" + "crypto/ecdsa" + "encoding/hex" + "encoding/json" + "fmt" + "os" + "path/filepath" + + ethcrypto "github.com/ethereum/go-ethereum/crypto" + + tmcfg "github.com/sei-protocol/sei-chain/sei-tendermint/config" + tmtypes "github.com/sei-protocol/sei-chain/sei-tendermint/types" + + "github.com/sei-protocol/sei-chain/sei-cosmos/client" + "github.com/sei-protocol/sei-chain/sei-cosmos/client/tx" + "github.com/sei-protocol/sei-chain/sei-cosmos/codec" + "github.com/sei-protocol/sei-chain/sei-cosmos/codec/legacy" + codectypes "github.com/sei-protocol/sei-chain/sei-cosmos/codec/types" + cryptocodec "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/codec" + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/hd" + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/keyring" + "github.com/sei-protocol/sei-chain/sei-cosmos/server" + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + authclient "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/client" + authtx "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/tx" + authtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/types" + vestingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/vesting/types" + banktypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/bank/types" + stakingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/staking/types" + + "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil" + genutiltypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil/types" + stakingcli "github.com/sei-protocol/sei-chain/sei-cosmos/x/staking/client/cli" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +var gentxLog = seilog.NewLogger("seictl", "task", "generate-gentx") + +const ( + gentxMarkerFile = ".sei-sidecar-gentx-done" + validatorKeyName = "validator" +) + +// GenerateGentxRequest holds the typed parameters for the generate-gentx task. +type GenerateGentxRequest struct { + ChainID string `json:"chainId"` + StakingAmount string `json:"stakingAmount"` + AccountBalance string `json:"accountBalance"` +} + +// GentxGenerator produces a genesis transaction by calling the same SDK +// functions as seid keys add -> seid add-genesis-account -> seid gentx. +type GentxGenerator struct { + homeDir string +} + +// NewGentxGenerator creates a generator targeting the given home directory. +func NewGentxGenerator(homeDir string) *GentxGenerator { + return &GentxGenerator{homeDir: homeDir} +} + +// Handler returns an engine.TaskHandler for the generate-gentx task type. +// +// Expected params: +// +// { +// "chainId": "my-chain", +// "stakingAmount": "1000000usei", +// "accountBalance": "10000000usei" +// } +func (g *GentxGenerator) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, params GenerateGentxRequest) error { + if markerExists(g.homeDir, gentxMarkerFile) { + gentxLog.Debug("already completed, skipping") + return nil + } + + if params.ChainID == "" { + return fmt.Errorf("generate-gentx: missing required param 'chainId'") + } + if params.StakingAmount == "" { + return fmt.Errorf("generate-gentx: missing required param 'stakingAmount'") + } + if params.AccountBalance == "" { + return fmt.Errorf("generate-gentx: missing required param 'accountBalance'") + } + + cdc, txCfg := makeCodec() + ensureBech32() + + address, err := g.addValidatorKey(cdc) + if err != nil { + return err + } + + if err := g.addGenesisAccount(cdc, address, params.AccountBalance); err != nil { + return err + } + + if err := g.generateGentx(cdc, txCfg, params.ChainID, params.StakingAmount); err != nil { + return err + } + + gentxLog.Info("gentx generated", "address", address) + return writeMarker(g.homeDir, gentxMarkerFile) + }) +} + +// addValidatorKey creates a local key and returns its bech32 address. +// Same path as: seid keys add validator --keyring-backend test +// +// Genesis ceremonies use throwaway keys; the production-keyring backend +// configured via SEI_KEYRING_BACKEND is intentionally not consumed here. +// Mixing the two would let a gentx run mutate an operator's production +// keyring (or surface its passphrase prompt) — both unacceptable. +func (g *GentxGenerator) addValidatorKey(cdc codec.Codec) (string, error) { + gentxLog.Info("creating validator key") + + kb, err := keyring.New(sdk.KeyringServiceName(), keyring.BackendTest, g.homeDir, os.Stdin) + if err != nil { + return "", fmt.Errorf("generate-gentx: creating keyring: %w", err) + } + + info, _, err := kb.NewMnemonic( + validatorKeyName, + keyring.English, + sdk.GetConfig().GetFullBIP44Path(), + "", + hd.Secp256k1, + ) + if err != nil { + return "", fmt.Errorf("generate-gentx: keys add: %w", err) + } + + addr := info.GetAddress().String() + gentxLog.Info("validator key created", "address", addr) + return addr, nil +} + +// addGenesisAccount adds the validator's account and balance to genesis. +// Same path as: seid add-genesis-account +func (g *GentxGenerator) addGenesisAccount(cdc codec.Codec, address, balance string) error { + gentxLog.Info("adding genesis account", "address", address, "balance", balance) + + addr, err := sdk.AccAddressFromBech32(address) + if err != nil { + return fmt.Errorf("generate-gentx: parsing address: %w", err) + } + + coins, err := sdk.ParseCoinsNormalized(balance) + if err != nil { + return fmt.Errorf("generate-gentx: parsing balance: %w", err) + } + + genFile := filepath.Join(g.homeDir, "config", "genesis.json") + appState, genDoc, err := genutiltypes.GenesisStateFromGenFile(genFile) + if err != nil { + return fmt.Errorf("generate-gentx: reading genesis: %w", err) + } + + // auth module: add account (same as seid add-genesis-account) + authGenState := authtypes.GetGenesisStateFromAppState(cdc, appState) + accs, err := authtypes.UnpackAccounts(authGenState.Accounts) + if err != nil { + return fmt.Errorf("generate-gentx: unpacking accounts: %w", err) + } + + if accs.Contains(addr) { + return fmt.Errorf("generate-gentx: account already exists at %s", addr) + } + + accs = append(accs, authtypes.NewBaseAccount(addr, nil, 0, 0)) + accs = authtypes.SanitizeGenesisAccounts(accs) + + genAccs, err := authtypes.PackAccounts(accs) + if err != nil { + return fmt.Errorf("generate-gentx: packing accounts: %w", err) + } + authGenState.Accounts = genAccs + + authStateBz, err := cdc.MarshalAsJSON(&authGenState) + if err != nil { + return fmt.Errorf("generate-gentx: marshaling auth state: %w", err) + } + appState[authtypes.ModuleName] = authStateBz + + // bank module: add balance (same as seid add-genesis-account) + bankGenState := banktypes.GetGenesisStateFromAppState(cdc, appState) + bankGenState.Balances = append(bankGenState.Balances, banktypes.Balance{ + Address: addr.String(), + Coins: coins.Sort(), + }) + bankGenState.Balances = banktypes.SanitizeGenesisBalances(bankGenState.Balances) + + bankStateBz, err := cdc.MarshalAsJSON(bankGenState) + if err != nil { + return fmt.Errorf("generate-gentx: marshaling bank state: %w", err) + } + appState[banktypes.ModuleName] = bankStateBz + + // evm module: associate sei address with eth address. + // Same as seid add-genesis-account when --keyring-backend=test. + if err := g.addEVMAddressAssociation(cdc, appState, addr); err != nil { + return fmt.Errorf("generate-gentx: adding EVM association: %w", err) + } + + appStateJSON, err := json.Marshal(appState) + if err != nil { + return fmt.Errorf("generate-gentx: marshaling app state: %w", err) + } + genDoc.AppState = appStateJSON + + return genutil.ExportGenesisFile(genDoc, genFile) +} + +// generateGentx builds and signs a MsgCreateValidator. +// Same path as: seid gentx validator --chain-id +func (g *GentxGenerator) generateGentx(cdc codec.Codec, txCfg client.TxConfig, chainID, stakingAmount string) error { + gentxLog.Info("generating gentx", "chainId", chainID, "stakingAmount", stakingAmount) + + cfg := tmcfg.DefaultConfig() + cfg.SetRoot(g.homeDir) + + nodeID, valPubKey, err := genutil.InitializeNodeValidatorFiles(cfg) + if err != nil { + return fmt.Errorf("generate-gentx: loading validator files: %w", err) + } + + kb, err := keyring.New(sdk.KeyringServiceName(), keyring.BackendTest, g.homeDir, os.Stdin) + if err != nil { + return fmt.Errorf("generate-gentx: opening keyring: %w", err) + } + + keyInfo, err := kb.Key(validatorKeyName) + if err != nil { + return fmt.Errorf("generate-gentx: looking up key: %w", err) + } + + // Read and validate genesis + genDoc, err := tmtypes.GenesisDocFromFile(cfg.GenesisFile()) + if err != nil { + return fmt.Errorf("generate-gentx: reading genesis: %w", err) + } + + var genesisState map[string]json.RawMessage + if err := json.Unmarshal(genDoc.AppState, &genesisState); err != nil { + return fmt.Errorf("generate-gentx: parsing app_state: %w", err) + } + + // Validate account has sufficient balance (same check seid gentx does) + coins, err := sdk.ParseCoinsNormalized(stakingAmount) + if err != nil { + return fmt.Errorf("generate-gentx: parsing staking amount: %w", err) + } + + genBalIterator := banktypes.GenesisBalancesIterator{} + if err := genutil.ValidateAccountInGenesis( + genesisState, genBalIterator, keyInfo.GetAddress(), coins, cdc, + ); err != nil { + return fmt.Errorf("generate-gentx: %w", err) + } + + // Resolve the pod's IP for the gentx memo (nodeID@ip:port). + // collect-gentxs requires a non-empty memo for peer discovery. + // The memo IP is vestigial — the controller overwrites persistent + // peers with DNS-based addresses — but it must be non-empty. + // Same call as seid gentx: server.ExternalIP(). + ip, ipErr := server.ExternalIP() + if ipErr != nil { + gentxLog.Warn("ExternalIP resolution failed, memo will contain empty IP", "error", ipErr) + } + + // Build MsgCreateValidator (same struct seid gentx populates) + createValCfg := stakingcli.TxCreateValidatorConfig{ + ChainID: chainID, + NodeID: nodeID, + Moniker: cfg.Moniker, + Amount: stakingAmount, + CommissionRate: "0.1", + CommissionMaxRate: "0.2", + CommissionMaxChangeRate: "0.01", + MinSelfDelegation: "1", + PubKey: valPubKey, + IP: ip, + P2PPort: "26656", + } + + clientCtx := client.Context{}. + WithKeyring(kb). + WithCodec(cdc). + WithTxConfig(txCfg). + WithFromAddress(keyInfo.GetAddress()) + + txFactory := tx.Factory{}. + WithChainID(chainID). + WithKeybase(kb). + WithTxConfig(txCfg) + + txBldr, msg, err := stakingcli.BuildCreateValidatorMsg(clientCtx, createValCfg, txFactory, true) + if err != nil { + return fmt.Errorf("generate-gentx: building MsgCreateValidator: %w", err) + } + + if err := msg.ValidateBasic(); err != nil { + return fmt.Errorf("generate-gentx: invalid MsgCreateValidator: %w", err) + } + + // Round-trip through the codec — same flow as seid gentx + w := &bytes.Buffer{} + clientCtx = clientCtx.WithOutput(w) + + if err := authclient.PrintUnsignedStdTx(txBldr, clientCtx, []sdk.Msg{msg}); err != nil { + return fmt.Errorf("generate-gentx: generating unsigned tx: %w", err) + } + + stdTx, err := txCfg.TxJSONDecoder()(w.Bytes()) + if err != nil { + return fmt.Errorf("generate-gentx: decoding unsigned tx: %w", err) + } + + txBuilder, err := txCfg.WrapTxBuilder(stdTx) + if err != nil { + return fmt.Errorf("generate-gentx: wrapping tx builder: %w", err) + } + + if err := authclient.SignTx(txFactory, clientCtx, validatorKeyName, txBuilder, true, true); err != nil { + return fmt.Errorf("generate-gentx: signing: %w", err) + } + + signedJSON, err := txCfg.TxJSONEncoder()(txBuilder.GetTx()) + if err != nil { + return fmt.Errorf("generate-gentx: encoding signed tx: %w", err) + } + + // Write gentx file + gentxDir := filepath.Join(g.homeDir, "config", "gentx") + if err := os.MkdirAll(gentxDir, 0o700); err != nil { + return fmt.Errorf("generate-gentx: creating gentx dir: %w", err) + } + + gentxFile := filepath.Join(gentxDir, fmt.Sprintf("gentx-%s.json", nodeID)) + return os.WriteFile(gentxFile, append(signedJSON, '\n'), 0o600) +} + +// addEVMAddressAssociation derives the Ethereum address from the +// validator's secp256k1 private key and writes it into the EVM module's +// genesis state. This mirrors seid add-genesis-account lines 136-148. +// +// We avoid importing x/evm (which transitively pulls in wasmvm/duckdb +// and breaks CGO_ENABLED=0) by operating on the genesis JSON directly. +// The result is identical: an AddressAssociation entry is appended. +func (g *GentxGenerator) addEVMAddressAssociation(cdc codec.Codec, appState map[string]json.RawMessage, addr sdk.AccAddress) error { + evmRaw, ok := appState["evm"] + if !ok { + // No evm module in genesis — skip silently. + return nil + } + + kb, err := keyring.New(sdk.KeyringServiceName(), keyring.BackendTest, g.homeDir, nil) + if err != nil { + return err + } + + pk, err := getPrivateKeyOfAddr(kb, addr) + if err != nil { + return fmt.Errorf("deriving ETH address for EVM association: %w", err) + } + + ethAddr := ethcrypto.PubkeyToAddress(pk.PublicKey) + + // Unmarshal the evm genesis state as generic JSON, append the + // association, and marshal back. This avoids importing x/evm/types + // which pulls in wasmvm via its transitive dependency chain. + var evmState map[string]json.RawMessage + if err := json.Unmarshal(evmRaw, &evmState); err != nil { + return fmt.Errorf("parsing evm genesis state: %w", err) + } + + var associations []json.RawMessage + if raw, ok := evmState["address_associations"]; ok && string(raw) != "null" { + if err := json.Unmarshal(raw, &associations); err != nil { + return fmt.Errorf("parsing evm address_associations: %w", err) + } + } + + entry, _ := json.Marshal(map[string]string{ + "sei_address": addr.String(), + "eth_address": ethAddr.Hex(), + }) + associations = append(associations, entry) + + evmState["address_associations"], _ = json.Marshal(associations) + appState["evm"], _ = json.Marshal(evmState) + return nil +} + +// getPrivateKeyOfAddr extracts the secp256k1 private key for the given +// address from the test keyring. Mirrors seid's getPrivateKeyOfAddr +// in cmd/seid/cmd/genaccounts.go:212-238. +func getPrivateKeyOfAddr(kb keyring.Keyring, addr sdk.Address) (*ecdsa.PrivateKey, error) { + keys, err := kb.List() + if err != nil { + return nil, err + } + for _, key := range keys { + localInfo, ok := key.(keyring.LocalInfo) + if !ok { + continue + } + if localInfo.GetAddress().Equals(addr) { + priv, err := legacy.PrivKeyFromBytes([]byte(localInfo.PrivKeyArmor)) + if err != nil { + return nil, err + } + privKey, err := ethcrypto.HexToECDSA(hex.EncodeToString(priv.Bytes())) + if err != nil { + return nil, err + } + return privKey, nil + } + } + return nil, fmt.Errorf("key not found for address %s", addr) +} + +// makeCodec builds a proto codec with the minimum interface types +// registered for genesis ceremony operations. vestingtypes is required to +// marshal/unmarshal a ContinuousVestingAccount or DelayedVestingAccount — +// addExternalGenesisAccounts can produce either when a GenesisAccountEntry +// sets Vesting, so this codec must resolve their type URLs even though no +// gentx itself is ever a vesting account. +func makeCodec() (codec.Codec, client.TxConfig) { + registry := codectypes.NewInterfaceRegistry() + cryptocodec.RegisterInterfaces(registry) + authtypes.RegisterInterfaces(registry) + vestingtypes.RegisterInterfaces(registry) + banktypes.RegisterInterfaces(registry) + stakingtypes.RegisterInterfaces(registry) + cdc := codec.NewProtoCodec(registry) + + txConfig := authtx.NewTxConfig(cdc, authtx.DefaultSignModes) + return cdc, txConfig +} + +// ensureBech32 sets the Sei bech32 address prefixes if not already set. +func ensureBech32() { + cfg := sdk.GetConfig() + if cfg.GetBech32AccountAddrPrefix() != "sei" { + cfg.SetBech32PrefixForAccount("sei", "seipub") + cfg.SetBech32PrefixForValidator("seivaloper", "seivaloperpub") + cfg.SetBech32PrefixForConsensusNode("seivalcons", "seivalconspub") + } +} diff --git a/sidecar/tasks/generate_gentx_test.go b/sidecar/tasks/generate_gentx_test.go new file mode 100644 index 00000000..ae2ca8e5 --- /dev/null +++ b/sidecar/tasks/generate_gentx_test.go @@ -0,0 +1,44 @@ +package tasks + +import ( + "context" + "os" + "path/filepath" + "testing" +) + +func TestGentxGenerator_MissingParams(t *testing.T) { + handler := NewGentxGenerator(t.TempDir()).Handler() + + tests := []struct { + name string + params map[string]any + }{ + {"missing chainId", map[string]any{"stakingAmount": "1", "accountBalance": "1"}}, + {"missing stakingAmount", map[string]any{"chainId": "c", "accountBalance": "1"}}, + {"missing accountBalance", map[string]any{"chainId": "c", "stakingAmount": "1"}}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, err := handler(context.Background(), tt.params) + if err == nil { + t.Fatal("expected error") + } + }) + } +} + +func TestGentxGenerator_NoMarkerOnFailure(t *testing.T) { + homeDir := t.TempDir() + handler := NewGentxGenerator(homeDir).Handler() + + // This will fail because there's no genesis.json to work with + _, _ = handler(context.Background(), map[string]any{ + "chainId": "c", "stakingAmount": "1000usei", "accountBalance": "10000usei", + }) + + if _, err := os.Stat(filepath.Join(homeDir, gentxMarkerFile)); err == nil { + t.Fatal("marker file should not exist after failure") + } +} diff --git a/sidecar/tasks/generate_identity.go b/sidecar/tasks/generate_identity.go new file mode 100644 index 00000000..03243ccd --- /dev/null +++ b/sidecar/tasks/generate_identity.go @@ -0,0 +1,96 @@ +package tasks + +import ( + "context" + "fmt" + "os" + + tmcfg "github.com/sei-protocol/sei-chain/sei-tendermint/config" + tmtypes "github.com/sei-protocol/sei-chain/sei-tendermint/types" + + "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +var identityLog = seilog.NewLogger("seictl", "task", "generate-identity") + +const identityMarkerFile = ".sei-sidecar-identity-done" + +// GenerateIdentityRequest holds the typed parameters for the generate-identity task. +type GenerateIdentityRequest struct { + ChainID string `json:"chainId"` + Moniker string `json:"moniker"` +} + +// IdentityGenerator creates the validator identity by calling the same +// SDK functions as seid init: genutil.InitializeNodeValidatorFilesFromMnemonic +// for keys, tmcfg.WriteConfigFile for config.toml, and genutil.ExportGenesisFile +// for genesis.json. +type IdentityGenerator struct { + homeDir string +} + +// NewIdentityGenerator creates a generator targeting the given home directory. +func NewIdentityGenerator(homeDir string) *IdentityGenerator { + return &IdentityGenerator{homeDir: homeDir} +} + +// Handler returns an engine.TaskHandler for the generate-identity task type. +// +// Expected params: {"chainId": "...", "moniker": "..."} +func (g *IdentityGenerator) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, params GenerateIdentityRequest) error { + if markerExists(g.homeDir, identityMarkerFile) { + identityLog.Debug("already completed, skipping") + return nil + } + + if params.ChainID == "" { + return fmt.Errorf("generate-identity: missing required param 'chainId'") + } + if params.Moniker == "" { + return fmt.Errorf("generate-identity: missing required param 'moniker'") + } + + identityLog.Info("generating identity", "chainId", params.ChainID, "moniker", params.Moniker) + + cfg := tmcfg.DefaultConfig() + cfg.SetRoot(g.homeDir) + tmcfg.EnsureRoot(g.homeDir) + + // Same call as seid init — generates node_key.json, + // priv_validator_key.json, priv_validator_state.json. + nodeID, _, err := genutil.InitializeNodeValidatorFilesFromMnemonic(cfg, "") + if err != nil { + return fmt.Errorf("generate-identity: initializing validator files: %w", err) + } + + cfg.Moniker = params.Moniker + + if err := tmcfg.WriteConfigFile(cfg.RootDir, cfg); err != nil { + return fmt.Errorf("generate-identity: writing config.toml: %w", err) + } + + // If no genesis.json exists yet (seid-init container didn't run + // or this is a standalone invocation), write a minimal one. + // The seid-init container normally creates the full genesis with + // all module defaults; this fallback produces a bare genesis that + // will be populated by subsequent ceremony steps. + genFile := cfg.GenesisFile() + if _, err := os.Stat(genFile); os.IsNotExist(err) { + genDoc := &tmtypes.GenesisDoc{ + ChainID: params.ChainID, + AppState: []byte("{}"), + } + if err := genutil.ExportGenesisFile(genDoc, genFile); err != nil { + return fmt.Errorf("generate-identity: writing genesis: %w", err) + } + } + + identityLog.Info("identity generated", "nodeId", nodeID, "moniker", params.Moniker) + return writeMarker(g.homeDir, identityMarkerFile) + }) +} diff --git a/sidecar/tasks/generate_identity_test.go b/sidecar/tasks/generate_identity_test.go new file mode 100644 index 00000000..4be1fcba --- /dev/null +++ b/sidecar/tasks/generate_identity_test.go @@ -0,0 +1,73 @@ +package tasks + +import ( + "context" + "os" + "path/filepath" + "testing" +) + +func TestIdentityGenerator_CreatesNodeKey(t *testing.T) { + homeDir := t.TempDir() + os.MkdirAll(filepath.Join(homeDir, "config"), 0o755) + + handler := NewIdentityGenerator(homeDir).Handler() + _, err := handler(context.Background(), map[string]any{ + "chainId": "test-chain-1", + "moniker": "val-0", + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + // Verify identity files were created + for _, f := range []string{ + "config/node_key.json", + "config/config.toml", + } { + path := filepath.Join(homeDir, f) + if _, err := os.Stat(path); os.IsNotExist(err) { + t.Errorf("expected %s to exist", f) + } + } +} + +func TestIdentityGenerator_Idempotent(t *testing.T) { + homeDir := t.TempDir() + + handler := NewIdentityGenerator(homeDir).Handler() + params := map[string]any{"chainId": "test-chain-1", "moniker": "val-0"} + + if _, err := handler(context.Background(), params); err != nil { + t.Fatalf("first call: %v", err) + } + + // Read node_key.json after first call + nodeKeyBefore, _ := os.ReadFile(filepath.Join(homeDir, "config", "node_key.json")) + + if _, err := handler(context.Background(), params); err != nil { + t.Fatalf("second call: %v", err) + } + + // Verify node_key.json wasn't regenerated (marker file skips) + nodeKeyAfter, _ := os.ReadFile(filepath.Join(homeDir, "config", "node_key.json")) + if string(nodeKeyBefore) != string(nodeKeyAfter) { + t.Error("node_key.json changed on second call — idempotency broken") + } +} + +func TestIdentityGenerator_MissingChainID(t *testing.T) { + handler := NewIdentityGenerator(t.TempDir()).Handler() + _, err := handler(context.Background(), map[string]any{"moniker": "val-0"}) + if err == nil { + t.Fatal("expected error for missing chainId") + } +} + +func TestIdentityGenerator_MissingMoniker(t *testing.T) { + handler := NewIdentityGenerator(t.TempDir()).Handler() + _, err := handler(context.Background(), map[string]any{"chainId": "test-chain-1"}) + if err == nil { + t.Fatal("expected error for missing moniker") + } +} diff --git a/sidecar/tasks/genesis.go b/sidecar/tasks/genesis.go new file mode 100644 index 00000000..5dc89931 --- /dev/null +++ b/sidecar/tasks/genesis.go @@ -0,0 +1,220 @@ +package tasks + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "fmt" + "io" + "os" + "path/filepath" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/config" + "github.com/aws/aws-sdk-go-v2/service/s3" + seiconfig "github.com/sei-protocol/sei-config" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +var genesisLog = seilog.NewLogger("seictl", "task", "genesis") + +const genesisMarkerFile = ".sei-sidecar-genesis-done" + +// S3GetObjectAPI abstracts a single-object S3 download for small files. +type S3GetObjectAPI interface { + GetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error) +} + +// S3ClientFactory builds an S3GetObjectAPI for a given region. +type S3ClientFactory func(ctx context.Context, region string) (S3GetObjectAPI, error) + +// DefaultS3ClientFactory creates a real S3 client using default credentials. +func DefaultS3ClientFactory(ctx context.Context, region string) (S3GetObjectAPI, error) { + cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion(region)) + if err != nil { + return nil, fmt.Errorf("loading AWS config: %w", err) + } + return s3.NewFromConfig(cfg), nil +} + +// GenesisS3Config holds S3 coordinates for genesis.json download. +type GenesisS3Config struct { + Bucket string + Key string + Region string +} + +// ConfigureGenesisRequest holds the typed parameters for the configure-genesis task. +// The fetcher resolves genesis from the chain ID using embedded config or S3 fallback. +// +// ExpectedGenesisHash is the bare SHA-256 hex digest (no "sha256:" prefix) the +// downloaded genesis.json must match. When non-empty it gates the S3 download: +// a mismatch fails closed. When empty (what the current controller sends) the +// download is unverified, preserving today's behavior. +type ConfigureGenesisRequest struct { + ExpectedGenesisHash string `json:"expectedGenesisHash,omitempty"` +} + +// GenesisFetcher writes genesis.json to the config directory. It first checks +// for an embedded genesis in sei-config for the chain ID. If not found, it +// falls back to downloading from S3 at {bucket}/{chainID}/genesis.json. +type GenesisFetcher struct { + homeDir string + chainID string + genesisBucket string + genesisRegion string + s3ClientFactory S3ClientFactory +} + +// NewGenesisFetcher creates a fetcher targeting the given home directory. +// chainID is the chain this sidecar is running for (typically from SEI_CHAIN_ID). +// genesisBucket and genesisRegion configure the S3 fallback location when the +// chain is not embedded in sei-config. +func NewGenesisFetcher(homeDir, chainID, genesisBucket, genesisRegion string, factory S3ClientFactory) *GenesisFetcher { + if factory == nil { + factory = DefaultS3ClientFactory + } + return &GenesisFetcher{ + homeDir: homeDir, + chainID: chainID, + genesisBucket: genesisBucket, + genesisRegion: genesisRegion, + s3ClientFactory: factory, + } +} + +// Handler returns an engine.TaskHandler that resolves genesis from embedded +// config or S3 fallback. No task parameters are required. +func (g *GenesisFetcher) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, req ConfigureGenesisRequest) error { + if markerExists(g.homeDir, genesisMarkerFile) { + genesisLog.Debug("already completed, skipping") + return nil + } + + // Try embedded genesis first. + if _, err := seiconfig.GenesisForChain(g.chainID); err == nil { + return g.writeEmbeddedGenesis() + } + + // Fall back to S3. + if g.genesisBucket == "" || g.genesisRegion == "" { + return fmt.Errorf("configure-genesis: chain %q is not embedded and SEI_GENESIS_BUCKET/SEI_GENESIS_REGION are not set", g.chainID) + } + + key := g.chainID + "/genesis.json" + genesisLog.Info("chain not embedded, fetching from S3", "chainId", g.chainID, "bucket", g.genesisBucket, "key", key) + return g.fetchFromS3(ctx, GenesisS3Config{Bucket: g.genesisBucket, Key: key, Region: g.genesisRegion}, req.ExpectedGenesisHash) + }) +} + +// Fetch downloads genesis.json from S3, skipping if the marker file exists. +// Retained for backward compatibility with callers that build GenesisS3Config +// directly; such callers do not verify a hash (empty expected hash). +func (g *GenesisFetcher) Fetch(ctx context.Context, cfg GenesisS3Config) error { + if markerExists(g.homeDir, genesisMarkerFile) { + genesisLog.Debug("already completed, skipping") + return nil + } + return g.fetchFromS3(ctx, cfg, "") +} + +// fetchFromS3 downloads genesis.json, tee-ing the bytes through SHA-256 as they +// land on disk. When expectedHash is non-empty the digest is verified BEFORE the +// completion marker is written: a mismatch deletes the partial file, skips the +// marker, and returns a terminal (non-retryable) error so a poisoned-then-retried +// node always re-verifies and never skips via a stale marker. +func (g *GenesisFetcher) fetchFromS3(ctx context.Context, cfg GenesisS3Config, expectedHash string) error { + genesisLog.Info("downloading genesis.json from S3", "bucket", cfg.Bucket, "key", cfg.Key) + s3Client, err := g.s3ClientFactory(ctx, cfg.Region) + if err != nil { + return fmt.Errorf("building S3 client: %w", err) + } + + output, err := s3Client.GetObject(ctx, &s3.GetObjectInput{ + Bucket: aws.String(cfg.Bucket), + Key: aws.String(cfg.Key), + }) + if err != nil { + return seis3.ClassifyS3Error("configure-genesis", cfg.Bucket, cfg.Key, cfg.Region, err) + } + defer func() { _ = output.Body.Close() }() + + // Hash the exact downloaded bytes inline with the copy — no re-read, no + // JSON re-marshal between download and digest. + hasher := sha256.New() + if err := g.writeGenesisFile(func(f *os.File) error { + _, err := io.Copy(io.MultiWriter(f, hasher), output.Body) + return err + }); err != nil { + return err + } + gotHash := hex.EncodeToString(hasher.Sum(nil)) + + if expectedHash != "" && gotHash != expectedHash { + destPath := filepath.Join(g.homeDir, "config", "genesis.json") + _ = os.Remove(destPath) + genesisLog.Error("genesis hash mismatch — failing closed", + "bucket", cfg.Bucket, "key", cfg.Key, "expected", expectedHash, "got", gotHash) + return &engine.TaskError{ + Task: "configure-genesis", + Operation: "verify-hash", + Message: fmt.Sprintf("downloaded genesis.json from s3://%s/%s has SHA-256 %s, expected %s", + cfg.Bucket, cfg.Key, gotHash, expectedHash), + Hint: "the genesis trust root does not match the expected hash; the S3 object may have been replaced — refusing to use it", + Retryable: false, + } + } + + if expectedHash != "" { + genesisLog.Info("genesis hash verified", "sha256", gotHash) + } + genesisLog.Info("genesis download complete (S3)") + return writeMarker(g.homeDir, genesisMarkerFile) +} + +func (g *GenesisFetcher) writeEmbeddedGenesis() error { + if g.chainID == "" { + return fmt.Errorf("configure-genesis: no S3 params and SEI_CHAIN_ID is not set") + } + + genesisLog.Info("writing embedded genesis", "chainId", g.chainID) + data, err := seiconfig.GenesisForChain(g.chainID) + if err != nil { + return fmt.Errorf("configure-genesis: %w", err) + } + + if err := g.writeGenesisFile(func(f *os.File) error { + _, err := f.Write(data) + return err + }); err != nil { + return err + } + + genesisLog.Info("genesis written from embedded data", "chainId", g.chainID) + return writeMarker(g.homeDir, genesisMarkerFile) +} + +// writeGenesisFile creates the config directory and genesis.json file, calling +// writeFn to populate its contents. +func (g *GenesisFetcher) writeGenesisFile(writeFn func(*os.File) error) error { + destDir := filepath.Join(g.homeDir, "config") + if err := os.MkdirAll(destDir, 0o755); err != nil { + return fmt.Errorf("creating config directory: %w", err) + } + + destPath := filepath.Join(destDir, "genesis.json") + f, err := os.Create(destPath) + if err != nil { + return fmt.Errorf("creating %s: %w", destPath, err) + } + defer func() { _ = f.Close() }() + + if err := writeFn(f); err != nil { + return fmt.Errorf("writing %s: %w", destPath, err) + } + return nil +} diff --git a/sidecar/tasks/genesis_overrides.go b/sidecar/tasks/genesis_overrides.go new file mode 100644 index 00000000..935ec90c --- /dev/null +++ b/sidecar/tasks/genesis_overrides.go @@ -0,0 +1,106 @@ +package tasks + +import ( + "encoding/json" + "fmt" + "strings" +) + +// applyGenesisOverrides applies a flat map of dotted-path overrides to a +// cosmos-sdk genesis app_state map. Keys take the shape +// ".[....]" — the first segment must be a key in +// appState (the cosmos module name), and remaining segments walk into the +// module's JSON tree. The leaf JSON value is replaced verbatim. +// +// Intermediate path segments must resolve to JSON objects. A missing +// intermediate key is created as an empty object so callers can add new +// nested paths. A non-object intermediate is a hard error — we never +// silently overwrite scalars or arrays. +// +// The function fails loudly on malformed input (empty keys, single-token +// keys, unknown modules, non-object intermediates) so misconfiguration +// surfaces at ceremony time via the task's failure path rather than as +// silently-ignored fields in the final genesis. +func applyGenesisOverrides(appState map[string]json.RawMessage, overrides map[string]json.RawMessage) error { + if len(overrides) == 0 { + return nil + } + + for key, value := range overrides { + if key == "" { + return fmt.Errorf("genesis-overrides: empty key") + } + parts := strings.Split(key, ".") + if len(parts) < 2 { + return fmt.Errorf("genesis-overrides: key %q must be of the form module.field[.field...]", key) + } + for i, p := range parts { + if p == "" { + return fmt.Errorf("genesis-overrides: key %q has empty segment at index %d", key, i) + } + } + if len(value) == 0 { + return fmt.Errorf("genesis-overrides: key %q has empty value", key) + } + + module := parts[0] + moduleRaw, ok := appState[module] + if !ok { + return fmt.Errorf("genesis-overrides: unknown module %q (key %q); app_state has no such top-level key", module, key) + } + + var moduleState map[string]json.RawMessage + if err := json.Unmarshal(moduleRaw, &moduleState); err != nil { + return fmt.Errorf("genesis-overrides: module %q is not a JSON object: %w", module, err) + } + if moduleState == nil { + moduleState = map[string]json.RawMessage{} + } + + if err := setNestedRaw(moduleState, parts[1:], value, key); err != nil { + return err + } + + moduleBz, err := json.Marshal(moduleState) + if err != nil { + return fmt.Errorf("genesis-overrides: re-marshaling module %q: %w", module, err) + } + appState[module] = moduleBz + } + return nil +} + +// setNestedRaw walks path into state, creating empty objects for missing +// intermediates, and writes value at the leaf. The originalKey is carried +// only for error messages. +func setNestedRaw(state map[string]json.RawMessage, path []string, value json.RawMessage, originalKey string) error { + if len(path) == 0 { + return fmt.Errorf("genesis-overrides: key %q has no field path under module", originalKey) + } + if len(path) == 1 { + state[path[0]] = value + return nil + } + + head, rest := path[0], path[1:] + var child map[string]json.RawMessage + if raw, ok := state[head]; ok && string(raw) != "null" { + if err := json.Unmarshal(raw, &child); err != nil { + return fmt.Errorf("genesis-overrides: key %q traverses non-object at segment %q: %w", originalKey, head, err) + } + } + if child == nil { + child = map[string]json.RawMessage{} + } + + if err := setNestedRaw(child, rest, value, originalKey); err != nil { + return err + } + + childBz, err := json.Marshal(child) + if err != nil { + return fmt.Errorf("genesis-overrides: re-marshaling intermediate %q for key %q: %w", head, originalKey, err) + } + state[head] = childBz + return nil +} diff --git a/sidecar/tasks/genesis_overrides_test.go b/sidecar/tasks/genesis_overrides_test.go new file mode 100644 index 00000000..d11196ba --- /dev/null +++ b/sidecar/tasks/genesis_overrides_test.go @@ -0,0 +1,329 @@ +package tasks + +import ( + "encoding/json" + "reflect" + "strings" + "testing" +) + +// mustAppState builds an app_state seed map from a JSON literal so tests +// read like the on-the-wire structure they're verifying. +func mustAppState(t *testing.T, body string) map[string]json.RawMessage { + t.Helper() + var out map[string]json.RawMessage + if err := json.Unmarshal([]byte(body), &out); err != nil { + t.Fatalf("seed app_state: %v", err) + } + return out +} + +func TestApplyGenesisOverrides_NilAndEmpty(t *testing.T) { + state := mustAppState(t, `{"staking":{"params":{"unbonding_time":"21 days"}}}`) + before, _ := json.Marshal(state) + + if err := applyGenesisOverrides(state, nil); err != nil { + t.Fatalf("nil overrides: %v", err) + } + if err := applyGenesisOverrides(state, map[string]json.RawMessage{}); err != nil { + t.Fatalf("empty overrides: %v", err) + } + after, _ := json.Marshal(state) + if string(before) != string(after) { + t.Errorf("app_state mutated on no-op input:\nbefore=%s\nafter=%s", before, after) + } +} + +func TestApplyGenesisOverrides_StringLeaf(t *testing.T) { + state := mustAppState(t, `{"staking":{"params":{"unbonding_time":"21 days"}}}`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "staking.params.unbonding_time": json.RawMessage(`"600s"`), + }) + if err != nil { + t.Fatalf("apply: %v", err) + } + + got := unbondingTime(t, state) + if got != "600s" { + t.Errorf("unbonding_time = %q, want %q", got, "600s") + } +} + +func TestApplyGenesisOverrides_NumberLeaf(t *testing.T) { + state := mustAppState(t, `{"staking":{"params":{"max_validators":100}}}`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "staking.params.max_validators": json.RawMessage(`50`), + }) + if err != nil { + t.Fatalf("apply: %v", err) + } + + stakingState := unmarshalObj(t, state["staking"]) + params := unmarshalObj(t, stakingState["params"]) + if got := string(params["max_validators"]); got != "50" { + t.Errorf("max_validators raw = %q, want %q", got, "50") + } +} + +func TestApplyGenesisOverrides_ObjectLeaf(t *testing.T) { + state := mustAppState(t, `{"gov":{"params":{"voting_params":{"voting_period":"172800s"}}}}`) + newParams := json.RawMessage(`{"voting_period":"60s","quorum":"0.4"}`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "gov.params.voting_params": newParams, + }) + if err != nil { + t.Fatalf("apply: %v", err) + } + + govState := unmarshalObj(t, state["gov"]) + params := unmarshalObj(t, govState["params"]) + vp := unmarshalObj(t, params["voting_params"]) + if got := string(vp["voting_period"]); got != `"60s"` { + t.Errorf("voting_period = %q, want %q", got, `"60s"`) + } + if got := string(vp["quorum"]); got != `"0.4"` { + t.Errorf("quorum = %q, want %q", got, `"0.4"`) + } +} + +func TestApplyGenesisOverrides_MultipleKeysAcrossModules(t *testing.T) { + state := mustAppState(t, `{ + "staking": {"params": {"unbonding_time": "21 days", "max_validators": 100}}, + "gov": {"params": {"max_deposit_period": "172800s"}} + }`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "staking.params.unbonding_time": json.RawMessage(`"600s"`), + "staking.params.max_validators": json.RawMessage(`50`), + "gov.params.max_deposit_period": json.RawMessage(`"60s"`), + }) + if err != nil { + t.Fatalf("apply: %v", err) + } + + if got := unbondingTime(t, state); got != "600s" { + t.Errorf("unbonding_time = %q, want 600s", got) + } + + stakingState := unmarshalObj(t, state["staking"]) + params := unmarshalObj(t, stakingState["params"]) + if got := string(params["max_validators"]); got != "50" { + t.Errorf("max_validators = %q, want 50", got) + } + + govState := unmarshalObj(t, state["gov"]) + govParams := unmarshalObj(t, govState["params"]) + if got := string(govParams["max_deposit_period"]); got != `"60s"` { + t.Errorf("max_deposit_period = %q, want \"60s\"", got) + } +} + +func TestApplyGenesisOverrides_DeepNestedPath(t *testing.T) { + state := mustAppState(t, `{"mod":{"a":{"b":{"c":{"d":"old"}}}}}`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "mod.a.b.c.d": json.RawMessage(`"new"`), + }) + if err != nil { + t.Fatalf("apply: %v", err) + } + + mod := unmarshalObj(t, state["mod"]) + a := unmarshalObj(t, mod["a"]) + b := unmarshalObj(t, a["b"]) + c := unmarshalObj(t, b["c"]) + if got := string(c["d"]); got != `"new"` { + t.Errorf("deep leaf = %q, want \"new\"", got) + } +} + +func TestApplyGenesisOverrides_CreatesMissingIntermediate(t *testing.T) { + // Override under an existing module that doesn't yet have the + // intermediate path. The helper auto-creates empty objects so + // new sub-fields can be added without seeding the path first. + state := mustAppState(t, `{"staking":{"params":{"unbonding_time":"21 days"}}}`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "staking.future.new_field": json.RawMessage(`true`), + }) + if err != nil { + t.Fatalf("apply: %v", err) + } + + stakingState := unmarshalObj(t, state["staking"]) + future := unmarshalObj(t, stakingState["future"]) + if got := string(future["new_field"]); got != "true" { + t.Errorf("future.new_field = %q, want true", got) + } +} + +func TestApplyGenesisOverrides_Idempotent(t *testing.T) { + overrides := map[string]json.RawMessage{ + "staking.params.unbonding_time": json.RawMessage(`"600s"`), + "staking.params.max_validators": json.RawMessage(`50`), + } + + stateA := mustAppState(t, `{"staking":{"params":{"unbonding_time":"21 days","max_validators":100}}}`) + if err := applyGenesisOverrides(stateA, overrides); err != nil { + t.Fatalf("first apply: %v", err) + } + afterFirst, _ := json.Marshal(stateA) + + if err := applyGenesisOverrides(stateA, overrides); err != nil { + t.Fatalf("second apply: %v", err) + } + afterSecond, _ := json.Marshal(stateA) + + if string(afterFirst) != string(afterSecond) { + t.Errorf("not idempotent:\nfirst=%s\nsecond=%s", afterFirst, afterSecond) + } +} + +func TestApplyGenesisOverrides_RejectsBadKey(t *testing.T) { + state := mustAppState(t, `{"staking":{"params":{"unbonding_time":"21 days"}}}`) + + cases := []struct { + name string + overrides map[string]json.RawMessage + want string + }{ + { + name: "empty key", + overrides: map[string]json.RawMessage{"": json.RawMessage(`"x"`)}, + want: "empty key", + }, + { + name: "single token", + overrides: map[string]json.RawMessage{"staking": json.RawMessage(`"x"`)}, + want: "module.field", + }, + { + name: "trailing dot", + overrides: map[string]json.RawMessage{"staking.params.": json.RawMessage(`"x"`)}, + want: "empty segment", + }, + { + name: "double dot", + overrides: map[string]json.RawMessage{"staking..unbonding_time": json.RawMessage(`"x"`)}, + want: "empty segment", + }, + { + name: "empty value", + overrides: map[string]json.RawMessage{"staking.params.unbonding_time": json.RawMessage(``)}, + want: "empty value", + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + err := applyGenesisOverrides(copyAppState(t, state), c.overrides) + if err == nil { + t.Fatal("expected error, got nil") + } + if !strings.Contains(err.Error(), c.want) { + t.Errorf("error %q does not contain %q", err.Error(), c.want) + } + }) + } +} + +func TestApplyGenesisOverrides_UnknownModule(t *testing.T) { + state := mustAppState(t, `{"staking":{"params":{"unbonding_time":"21 days"}}}`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "nope.params.x": json.RawMessage(`"y"`), + }) + if err == nil { + t.Fatal("expected error for unknown module") + } + if !strings.Contains(err.Error(), "unknown module") { + t.Errorf("error = %q, want substring 'unknown module'", err.Error()) + } +} + +func TestApplyGenesisOverrides_TraverseScalar(t *testing.T) { + state := mustAppState(t, `{"staking":{"params":"this-is-a-string"}}`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "staking.params.unbonding_time": json.RawMessage(`"600s"`), + }) + if err == nil { + t.Fatal("expected error traversing scalar intermediate") + } + if !strings.Contains(err.Error(), "non-object") { + t.Errorf("error = %q, want substring 'non-object'", err.Error()) + } +} + +func TestApplyGenesisOverrides_TraverseArray(t *testing.T) { + state := mustAppState(t, `{"staking":{"params":["a","b"]}}`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "staking.params.unbonding_time": json.RawMessage(`"600s"`), + }) + if err == nil { + t.Fatal("expected error traversing array intermediate") + } + if !strings.Contains(err.Error(), "non-object") { + t.Errorf("error = %q, want substring 'non-object'", err.Error()) + } +} + +func TestApplyGenesisOverrides_LeavesUnrelatedKeysIntact(t *testing.T) { + state := mustAppState(t, `{ + "staking": {"params": {"unbonding_time": "21 days", "max_validators": 100}}, + "gov": {"params": {"max_deposit_period": "172800s"}} + }`) + err := applyGenesisOverrides(state, map[string]json.RawMessage{ + "staking.params.unbonding_time": json.RawMessage(`"600s"`), + }) + if err != nil { + t.Fatalf("apply: %v", err) + } + + // staking.params.max_validators preserved + stakingState := unmarshalObj(t, state["staking"]) + params := unmarshalObj(t, stakingState["params"]) + if got := string(params["max_validators"]); got != "100" { + t.Errorf("max_validators = %q, want preserved 100", got) + } + + // entire gov module preserved + gov := unmarshalObj(t, state["gov"]) + govParams := unmarshalObj(t, gov["params"]) + if got := string(govParams["max_deposit_period"]); got != `"172800s"` { + t.Errorf("gov.params.max_deposit_period = %q, want preserved", got) + } +} + +// unbondingTime is a convenience helper used by multiple tests to drill +// into staking.params.unbonding_time and return the unquoted string. +func unbondingTime(t *testing.T, state map[string]json.RawMessage) string { + t.Helper() + staking := unmarshalObj(t, state["staking"]) + params := unmarshalObj(t, staking["params"]) + var s string + if err := json.Unmarshal(params["unbonding_time"], &s); err != nil { + t.Fatalf("decoding unbonding_time: %v", err) + } + return s +} + +func unmarshalObj(t *testing.T, raw json.RawMessage) map[string]json.RawMessage { + t.Helper() + var out map[string]json.RawMessage + if err := json.Unmarshal(raw, &out); err != nil { + t.Fatalf("decoding object: %v", err) + } + return out +} + +func copyAppState(t *testing.T, in map[string]json.RawMessage) map[string]json.RawMessage { + t.Helper() + bz, err := json.Marshal(in) + if err != nil { + t.Fatalf("marshal: %v", err) + } + var out map[string]json.RawMessage + if err := json.Unmarshal(bz, &out); err != nil { + t.Fatalf("unmarshal: %v", err) + } + if reflect.DeepEqual(in, out) == false { + // Sanity guard — not a test assertion, just paranoia about the helper. + t.Logf("note: marshal/unmarshal round-trip diverged for state; tests should still be valid") + } + return out +} diff --git a/sidecar/tasks/genesis_peers.go b/sidecar/tasks/genesis_peers.go new file mode 100644 index 00000000..7edb757e --- /dev/null +++ b/sidecar/tasks/genesis_peers.go @@ -0,0 +1,135 @@ +package tasks + +import ( + "context" + "crypto/sha256" + "encoding/base64" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + "strings" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/service/s3" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +var genesisPeersLog = seilog.NewLogger("seictl", "task", "set-genesis-peers") + +const ( + ed25519PrivKeyLen = 64 + ed25519PubKeyOffset = 32 + cometbftAddressLen = 20 +) + +// SetGenesisPeersRequest holds the typed parameters for the set-genesis-peers task. +// S3 coordinates are derived from the sidecar's environment. +type SetGenesisPeersRequest struct{} + +// GenesisPeersSetter downloads a peers.json file produced by the genesis +// assembler and writes the entries into config.toml as persistent_peers, +// filtering out the current node's own entry. +type GenesisPeersSetter struct { + homeDir string + bucket string + region string + chainID string + s3ClientFactory S3ClientFactory +} + +// NewGenesisPeersSetter creates a setter targeting the given home directory. +func NewGenesisPeersSetter(homeDir, bucket, region, chainID string, s3Factory S3ClientFactory) *GenesisPeersSetter { + if s3Factory == nil { + s3Factory = DefaultS3ClientFactory + } + return &GenesisPeersSetter{ + homeDir: homeDir, + bucket: bucket, + region: region, + chainID: chainID, + s3ClientFactory: s3Factory, + } +} + +// Handler returns an engine.TaskHandler for the set-genesis-peers task. +// The peers.json key is derived from the chain ID: {chainID}/peers.json. +func (g *GenesisPeersSetter) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, _ SetGenesisPeersRequest) error { + key := g.chainID + "/peers.json" + + s3Client, err := g.s3ClientFactory(ctx, g.region) + if err != nil { + return fmt.Errorf("set-genesis-peers: building S3 client: %w", err) + } + + output, err := s3Client.GetObject(ctx, &s3.GetObjectInput{ + Bucket: aws.String(g.bucket), + Key: aws.String(key), + }) + if err != nil { + return seis3.ClassifyS3Error("set-genesis-peers", g.bucket, key, g.region, err) + } + data, err := io.ReadAll(output.Body) + _ = output.Body.Close() + if err != nil { + return fmt.Errorf("set-genesis-peers: reading peers.json: %w", err) + } + + var allPeers []string + if err := json.Unmarshal(data, &allPeers); err != nil { + return fmt.Errorf("set-genesis-peers: parsing peers.json: %w", err) + } + + selfID, err := readLocalNodeID(g.homeDir) + if err != nil { + return err + } + + var filtered []string + for _, peer := range allPeers { + if !strings.HasPrefix(peer, selfID+"@") { + filtered = append(filtered, peer) + } + } + + genesisPeersLog.Info("applying genesis peers", + "total", len(allPeers), "self", selfID, "peers", len(filtered)) + + return writePeersToConfig(g.homeDir, filtered) + }) +} + +// readLocalNodeID derives the Tendermint node ID from the Ed25519 key in +// node_key.json. The ID is hex(SHA256(pubkey)[:20]), matching CometBFT's +// p2p.PubKeyToID derivation. +func readLocalNodeID(homeDir string) (string, error) { + path := filepath.Join(homeDir, "config", "node_key.json") + data, err := os.ReadFile(path) + if err != nil { + return "", fmt.Errorf("set-genesis-peers: reading node_key.json: %w", err) + } + + var keyFile struct { + PrivKey struct { + Value string `json:"value"` + } `json:"priv_key"` + } + if err := json.Unmarshal(data, &keyFile); err != nil { + return "", fmt.Errorf("set-genesis-peers: parsing node_key.json: %w", err) + } + + keyBytes, err := base64.StdEncoding.DecodeString(keyFile.PrivKey.Value) + if err != nil || len(keyBytes) != ed25519PrivKeyLen { + return "", fmt.Errorf("set-genesis-peers: invalid Ed25519 key in node_key.json") + } + + pubKey := keyBytes[ed25519PubKeyOffset:] + hash := sha256.Sum256(pubKey) + return hex.EncodeToString(hash[:cometbftAddressLen]), nil +} diff --git a/sidecar/tasks/genesis_test.go b/sidecar/tasks/genesis_test.go new file mode 100644 index 00000000..00aae5ba --- /dev/null +++ b/sidecar/tasks/genesis_test.go @@ -0,0 +1,186 @@ +package tasks + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "testing" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +// genesisFetchFixture wires a GenesisFetcher to a mock S3 holding a single +// genesis.json under the unknown chain's key, returning the fetcher plus the +// bytes' true SHA-256 hex digest. The chain ID is intentionally not embedded so +// the handler takes the S3 fallback path. +func genesisFetchFixture(t *testing.T, body []byte) (*GenesisFetcher, string, string) { + t.Helper() + homeDir := t.TempDir() + const chainID = "custom-devnet-1" + key := chainID + "/genesis.json" + s3 := &mockS3GetObject{objects: map[string][]byte{key: body}} + factory := func(_ context.Context, _ string) (S3GetObjectAPI, error) { return s3, nil } + fetcher := NewGenesisFetcher(homeDir, chainID, "genesis-bucket", "us-east-2", factory) + sum := sha256.Sum256(body) + return fetcher, hex.EncodeToString(sum[:]), homeDir +} + +func TestGenesisFetcher_S3_MatchingHashSucceeds(t *testing.T) { + body := []byte(`{"chain_id":"custom-devnet-1","app_state":{}}`) + fetcher, wantHash, homeDir := genesisFetchFixture(t, body) + + _, err := fetcher.Handler()(context.Background(), map[string]any{"expectedGenesisHash": wantHash}) + if err != nil { + t.Fatalf("matching hash should succeed, got: %v", err) + } + + got, err := os.ReadFile(filepath.Join(homeDir, "config", "genesis.json")) + if err != nil { + t.Fatalf("reading genesis.json: %v", err) + } + if string(got) != string(body) { + t.Errorf("genesis bytes = %q, want %q", got, body) + } + if !markerExists(homeDir, genesisMarkerFile) { + t.Error("expected completion marker to be written on verified download") + } +} + +func TestGenesisFetcher_S3_MismatchedHashFailsClosed(t *testing.T) { + body := []byte(`{"chain_id":"custom-devnet-1","app_state":{}}`) + fetcher, _, homeDir := genesisFetchFixture(t, body) + + _, err := fetcher.Handler()(context.Background(), map[string]any{ + "expectedGenesisHash": "0000000000000000000000000000000000000000000000000000000000000000", + }) + if err == nil { + t.Fatal("mismatched hash must fail closed, got nil error") + } + + var te *engine.TaskError + if !errors.As(err, &te) { + t.Fatalf("error type = %T, want *engine.TaskError", err) + } + if te.Retryable { + t.Error("hash-mismatch error must be terminal (non-retryable)") + } + + if _, statErr := os.Stat(filepath.Join(homeDir, "config", "genesis.json")); !os.IsNotExist(statErr) { + t.Error("partial genesis.json must be deleted on mismatch") + } + if markerExists(homeDir, genesisMarkerFile) { + t.Error("completion marker must NOT be written on mismatch (re-verify safety)") + } +} + +func TestGenesisFetcher_S3_EmptyHashPreservesBehavior(t *testing.T) { + body := []byte(`{"chain_id":"custom-devnet-1","app_state":{}}`) + fetcher, _, homeDir := genesisFetchFixture(t, body) + + // No expectedGenesisHash in params — the current controller's wire shape. + if _, err := fetcher.Handler()(context.Background(), map[string]any{}); err != nil { + t.Fatalf("empty expected hash should download unverified, got: %v", err) + } + + got, err := os.ReadFile(filepath.Join(homeDir, "config", "genesis.json")) + if err != nil { + t.Fatalf("reading genesis.json: %v", err) + } + if string(got) != string(body) { + t.Errorf("genesis bytes = %q, want %q", got, body) + } + if !markerExists(homeDir, genesisMarkerFile) { + t.Error("expected completion marker on unverified download") + } +} + +func TestGenesisFetcher_EmbeddedChain(t *testing.T) { + homeDir := t.TempDir() + fetcher := NewGenesisFetcher(homeDir, "pacific-1", "test-bucket", "us-east-2", nil) + handler := fetcher.Handler() + + _, err := handler(context.Background(), map[string]any{}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + data, err := os.ReadFile(filepath.Join(homeDir, "config", "genesis.json")) + if err != nil { + t.Fatalf("reading genesis.json: %v", err) + } + if len(data) == 0 { + t.Fatal("genesis.json is empty") + } + + var doc struct { + ChainID string `json:"chain_id"` + } + if err := json.Unmarshal(data, &doc); err != nil { + t.Fatalf("genesis.json is not valid JSON: %v", err) + } + if doc.ChainID != "pacific-1" { + t.Errorf("chain_id = %q, want %q", doc.ChainID, "pacific-1") + } +} + +func TestGenesisFetcher_EmbeddedChain_Idempotent(t *testing.T) { + homeDir := t.TempDir() + fetcher := NewGenesisFetcher(homeDir, "atlantic-2", "test-bucket", "us-east-2", nil) + handler := fetcher.Handler() + + if _, err := handler(context.Background(), map[string]any{}); err != nil { + t.Fatalf("first call: %v", err) + } + if _, err := handler(context.Background(), map[string]any{}); err != nil { + t.Fatalf("second call (should skip via marker): %v", err) + } +} + +func TestGenesisFetcher_UnknownChainFallsBackToS3(t *testing.T) { + homeDir := t.TempDir() + called := false + mockFactory := func(ctx context.Context, region string) (S3GetObjectAPI, error) { + called = true + if region != "us-east-2" { + t.Errorf("region = %q, want %q", region, "us-east-2") + } + return nil, fmt.Errorf("mock: intentional S3 error") + } + fetcher := NewGenesisFetcher(homeDir, "custom-devnet-1", "my-genesis-bucket", "us-east-2", mockFactory) + handler := fetcher.Handler() + + _, err := handler(context.Background(), map[string]any{}) + if !called { + t.Fatal("expected S3 fallback for unknown chain") + } + if err == nil { + t.Fatal("expected error from mock S3 factory") + } +} + +func TestGenesisFetcher_UnknownChainNoBucket(t *testing.T) { + homeDir := t.TempDir() + fetcher := NewGenesisFetcher(homeDir, "custom-devnet-1", "", "", nil) + handler := fetcher.Handler() + + _, err := handler(context.Background(), map[string]any{}) + if err == nil { + t.Fatal("expected error for unknown chain with no bucket configured") + } +} + +func TestGenesisFetcher_NoChainID(t *testing.T) { + homeDir := t.TempDir() + fetcher := NewGenesisFetcher(homeDir, "", "bucket", "region", nil) + handler := fetcher.Handler() + + _, err := handler(context.Background(), map[string]any{}) + if err == nil { + t.Fatal("expected error when chainID is empty") + } +} diff --git a/sidecar/tasks/genesis_writeback.go b/sidecar/tasks/genesis_writeback.go new file mode 100644 index 00000000..33d6435a --- /dev/null +++ b/sidecar/tasks/genesis_writeback.go @@ -0,0 +1,55 @@ +package tasks + +import ( + "encoding/json" + "fmt" + + tmtypes "github.com/sei-protocol/sei-chain/sei-tendermint/types" + + "github.com/sei-protocol/sei-chain/sei-cosmos/codec" + authtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/types" + banktypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/bank/types" + "github.com/sei-protocol/sei-chain/sei-cosmos/x/genutil" +) + +// writeBackAuthAndBank serializes mutated auth + bank state and exports +// the genesis file. Callers wrap the returned error with task context. +func writeBackAuthAndBank( + cdc codec.Codec, + genFile string, + genDoc *tmtypes.GenesisDoc, + appState map[string]json.RawMessage, + authGenState authtypes.GenesisState, + accs authtypes.GenesisAccounts, + bankGenState *banktypes.GenesisState, +) error { + accs = authtypes.SanitizeGenesisAccounts(accs) + genAccs, err := authtypes.PackAccounts(accs) + if err != nil { + return fmt.Errorf("packing accounts: %w", err) + } + authGenState.Accounts = genAccs + authStateBz, err := cdc.MarshalAsJSON(&authGenState) + if err != nil { + return fmt.Errorf("marshaling auth state: %w", err) + } + appState[authtypes.ModuleName] = authStateBz + + bankGenState.Balances = banktypes.SanitizeGenesisBalances(bankGenState.Balances) + bankStateBz, err := cdc.MarshalAsJSON(bankGenState) + if err != nil { + return fmt.Errorf("marshaling bank state: %w", err) + } + appState[banktypes.ModuleName] = bankStateBz + + appStateJSON, err := json.Marshal(appState) + if err != nil { + return fmt.Errorf("marshaling app state: %w", err) + } + genDoc.AppState = appStateJSON + + if err := genutil.ExportGenesisFile(genDoc, genFile); err != nil { + return fmt.Errorf("writing genesis: %w", err) + } + return nil +} diff --git a/sidecar/tasks/gov_param_change.go b/sidecar/tasks/gov_param_change.go new file mode 100644 index 00000000..8a8f745a --- /dev/null +++ b/sidecar/tasks/gov_param_change.go @@ -0,0 +1,181 @@ +// Package tasks — gov-param-change handler. +// +// This handler signs ParameterChangeProposals as the validator's +// operator account. API authentication is controlled by +// SEI_SIDECAR_AUTHN_MODE; see sidecar/server/auth.go. +// +// REHYDRATION — sei-protocol/seictl#174 +// +// MsgSubmitProposal is NOT chain-idempotent, and param-change has no "applies +// once" safety net (unlike gov-software-upgrade). Crash-idempotency is provided +// by the pre-broadcast TxMarker + rehydrate-adopt in SignAndBroadcast: a re-run +// adopts the in-flight tx rather than re-signing, so a crash no longer produces +// a duplicate proposal. + +package tasks + +import ( + "context" + "encoding/json" + "errors" + "fmt" + + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + govtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/gov/types" + proposal "github.com/sei-protocol/sei-chain/sei-cosmos/x/params/types/proposal" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/wire" +) + +var govParamChangeLog = seilog.NewLogger("seictl", "task", "gov-param-change") + +// paramChange is one (subspace, key, value) entry. Value is raw JSON of +// whatever shape the param's registered type expects (scalar, string, +// bool, or object). It is stringified exactly ONCE — see buildParamChangeMsg. +// +// Integer-valued params MUST be passed as JSON strings (e.g. "100"), not +// bare numbers: the sidecar request decode routes values through a +// map[string]any (float64 numbers), which silently loses precision above +// 2^53. Sei's large-integer params (durations, windows) are string-encoded +// by convention, so this is the natural form anyway. +type paramChange struct { + Subspace string `json:"subspace"` + Key string `json:"key"` + Value json.RawMessage `json:"value"` +} + +// GovParamChangeRequest holds gov-param-change params. Idempotency is +// NOT handled — see the REHYDRATION WARNING at the top of this file. +type GovParamChangeRequest struct { + ChainID string `json:"chainId"` + KeyName string `json:"keyName"` + + Title string `json:"title"` + Description string `json:"description"` + + Changes []paramChange `json:"changes"` + + InitialDeposit string `json:"initialDeposit"` + + Memo string `json:"memo,omitempty"` + Fees string `json:"fees"` + Gas uint64 `json:"gas"` +} + +// GovParamChanger captures cfg by value at construction; engine.Config +// is documented read-only after startup, so the copy is safe. +type GovParamChanger struct { + cfg engine.ExecutionConfig +} + +func NewGovParamChanger(cfg engine.ExecutionConfig) *GovParamChanger { + return &GovParamChanger{cfg: cfg} +} + +// Handler delegates to SignAndBroadcast (which owns the crash-idempotency +// marker — see the REHYDRATION note at the top of this file) and classifies +// the outcome via classifyGovResult. +func (g *GovParamChanger) Handler() engine.TaskHandler { + return engine.TypedHandlerWithResult(func(ctx context.Context, params GovParamChangeRequest) (*wire.GovTxResult, error) { + msg, err := buildParamChangeMsg(g.cfg, params) + if err != nil { + return nil, err + } + result, err := SignAndBroadcast(ctx, g.cfg, SignAndBroadcastInput{ + ChainID: params.ChainID, + KeyName: params.KeyName, + Msg: msg, + Fees: params.Fees, + Gas: params.Gas, + Memo: params.Memo, + TaskID: engine.TaskIDFromContext(ctx), + }) + if err != nil { + return nil, err + } + out, cerr := classifyGovResult(engine.TaskGovParamChange, result) + cerr = requireProposalID(out, cerr) + keys := make([]string, 0, len(params.Changes)) + for _, c := range params.Changes { + keys = append(keys, c.Subspace+"/"+c.Key) + } + govParamChangeLog.Info("proposal broadcast", + "taskId", engine.TaskIDFromContext(ctx), + "chainId", params.ChainID, + "changes", keys, + "txHash", out.TxHash, + "height", out.Height, + "proposalId", out.ProposalID, + "inclusionStatus", out.InclusionStatus) + return out, cerr + }) +} + +func buildParamChangeMsg(cfg engine.ExecutionConfig, params GovParamChangeRequest) (*govtypes.MsgSubmitProposal, error) { + if cfg.Keyring == nil { + return nil, Terminal(errors.New("keyring not configured: set SEI_KEYRING_BACKEND/SEI_KEYRING_PASSPHRASE on the sidecar")) + } + if params.KeyName == "" { + return nil, Terminal(errors.New("keyName required")) + } + if params.Title == "" { + return nil, Terminal(errors.New("title required")) + } + if params.Description == "" { + return nil, Terminal(errors.New("description required")) + } + if len(params.Changes) == 0 { + return nil, Terminal(errors.New("at least one change required")) + } + changes := make([]proposal.ParamChange, 0, len(params.Changes)) + for i, c := range params.Changes { + if c.Subspace == "" { + return nil, Terminal(fmt.Errorf("changes[%d].subspace required", i)) + } + if c.Key == "" { + return nil, Terminal(fmt.Errorf("changes[%d].key required", i)) + } + if len(c.Value) == 0 { + return nil, Terminal(fmt.Errorf("changes[%d].value required", i)) + } + // The ONLY string() conversion: c.Value is the raw JSON of the + // param's registered type (scalar/string/bool/object). The chain + // runs UnmarshalAsJSON(value, registeredType) at apply time, so + // the bytes must be valid JSON for that type — NOT a re-escaped + // string (which would double-encode and fail at apply). + changes = append(changes, proposal.NewParamChange(c.Subspace, c.Key, string(c.Value))) + } + info, err := cfg.Keyring.Key(params.KeyName) + if err != nil { + return nil, Terminal(fmt.Errorf("keyring entry %q: %w", params.KeyName, err)) + } + deposit, err := sdk.ParseCoinsNormalized(params.InitialDeposit) + if err != nil { + return nil, Terminal(fmt.Errorf("parse initialDeposit %q: %w", params.InitialDeposit, err)) + } + if len(deposit) == 0 { + return nil, Terminal(fmt.Errorf("initialDeposit %q resolves to zero coins", params.InitialDeposit)) + } + if !deposit.IsAllPositive() { + return nil, Terminal(fmt.Errorf("initialDeposit %q contains non-positive amounts", params.InitialDeposit)) + } + // Symmetric with checkFeesDenom: deposit denom is fixed by gov params + // on Sei (usei); a wrong denom would CheckTx-reject anyway, but + // rejecting here saves the sign + broadcast roundtrip. + for _, c := range deposit { + if c.Denom != feeDenom { + return nil, Terminal(fmt.Errorf("initialDeposit %q: denom %q not permitted (only %q)", params.InitialDeposit, c.Denom, feeDenom)) + } + } + // isExpedited=false: expedited is deferred (it is honored only via + // NewMsgSubmitProposalWithExpedite, not the content field). See LLD. + content := proposal.NewParameterChangeProposal(params.Title, params.Description, changes, false) + msg, err := govtypes.NewMsgSubmitProposal(content, deposit, info.GetAddress()) + if err != nil { + return nil, Terminal(fmt.Errorf("build MsgSubmitProposal: %w", err)) + } + return msg, nil +} diff --git a/sidecar/tasks/gov_param_change_test.go b/sidecar/tasks/gov_param_change_test.go new file mode 100644 index 00000000..ce8123c5 --- /dev/null +++ b/sidecar/tasks/gov_param_change_test.go @@ -0,0 +1,166 @@ +package tasks + +import ( + "context" + "strings" + "testing" + + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + proposal "github.com/sei-protocol/sei-chain/sei-cosmos/x/params/types/proposal" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +func validParamChangeRequest() GovParamChangeRequest { + return GovParamChangeRequest{ + ChainID: "arctic-1", + KeyName: "node_admin", + Title: "Update Consensus Timeout Params", + Description: "Tighten consensus timeouts.", + Changes: []paramChange{ + // struct-valued param (object) + {Subspace: "baseapp", Key: "TimeoutParams", Value: []byte(`{"propose":"300000000","commit":"200000000"}`)}, + }, + InitialDeposit: "10000000usei", + Fees: "8000usei", + Gas: 300_000, + } +} + +func TestBuildParamChangeMsg(t *testing.T) { + kr, addr := testKeyring(t) + cfg := engine.ExecutionConfig{Keyring: kr} + + t.Run("happy path", func(t *testing.T) { + msg, err := buildParamChangeMsg(cfg, validParamChangeRequest()) + if err != nil { + t.Fatalf("unexpected err: %v", err) + } + if msg.Proposer != addr.String() { + t.Errorf("proposer = %q, want %q", msg.Proposer, addr.String()) + } + want, _ := sdk.ParseCoinsNormalized("10000000usei") + if !msg.InitialDeposit.IsEqual(want) { + t.Errorf("InitialDeposit = %v, want %v", msg.InitialDeposit, want) + } + if err := msg.ValidateBasic(); err != nil { + t.Errorf("ValidateBasic on returned msg: %v", err) + } + if _, ok := msg.GetContent().(*proposal.ParameterChangeProposal); !ok { + t.Errorf("content type = %T, want *ParameterChangeProposal", msg.GetContent()) + } + }) + + // Regression guard for the prop-252 double-encode bug: the raw JSON + // value must reach ParamChange.Value stringified exactly ONCE, for any + // JSON shape — an object or a bare scalar. A value of {"a":"b"} must + // become {"a":"b"}, never "{\"a\":\"b\"}". + t.Run("value single-encoded for object and scalar", func(t *testing.T) { + cases := []struct { + name, raw string + }{ + {"object", `{"propose":"300000000","commit":"200000000"}`}, + {"scalar-string", `"86400000000000"`}, + {"scalar-number", `100`}, + {"scalar-bool", `true`}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + req := validParamChangeRequest() + req.Changes = []paramChange{{Subspace: "staking", Key: "K", Value: []byte(tc.raw)}} + msg, err := buildParamChangeMsg(cfg, req) + if err != nil { + t.Fatalf("unexpected err: %v", err) + } + pcp := msg.GetContent().(*proposal.ParameterChangeProposal) + if got := pcp.Changes[0].Value; got != tc.raw { + t.Errorf("ParamChange.Value = %q, want %q (double-encoded?)", got, tc.raw) + } + }) + } + }) + + t.Run("non-usei deposit rejected", func(t *testing.T) { + req := validParamChangeRequest() + req.InitialDeposit = "10000000uatom" + if _, err := buildParamChangeMsg(cfg, req); err == nil { + t.Fatal("expected error for non-usei deposit") + } else if !strings.Contains(err.Error(), "not permitted") { + t.Errorf("err = %v, want denom-not-permitted", err) + } + }) + + t.Run("validation failures are Terminal", func(t *testing.T) { + cases := []struct { + name string + mut func(*GovParamChangeRequest) + }{ + {"missing keyName", func(r *GovParamChangeRequest) { r.KeyName = "" }}, + {"missing title", func(r *GovParamChangeRequest) { r.Title = "" }}, + {"missing description", func(r *GovParamChangeRequest) { r.Description = "" }}, + {"empty changes", func(r *GovParamChangeRequest) { r.Changes = nil }}, + {"empty subspace", func(r *GovParamChangeRequest) { r.Changes[0].Subspace = "" }}, + {"empty key", func(r *GovParamChangeRequest) { r.Changes[0].Key = "" }}, + {"empty value", func(r *GovParamChangeRequest) { r.Changes[0].Value = nil }}, + {"non-usei deposit", func(r *GovParamChangeRequest) { r.InitialDeposit = "1uatom" }}, + {"zero-coin deposit", func(r *GovParamChangeRequest) { r.InitialDeposit = "0usei" }}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + req := validParamChangeRequest() + tc.mut(&req) + _, err := buildParamChangeMsg(cfg, req) + if err == nil { + t.Fatalf("expected error") + } + if !IsTerminal(err) { + t.Errorf("err = %v, want Terminal", err) + } + }) + } + }) +} + +// TestGovParamChangeHandler_HappyPath threads the handler end-to-end +// through signAndBroadcast with a fake txClient. The MsgSubmitProposal +// content (a ParameterChangeProposal) is packed as an Any, so reaching +// the broadcast step proves newSignTxInterfaceRegistry registers the +// x/params proposal interfaces; a missing registration fails in +// txCfg.TxEncoder before the fakeTxClient ever sees bytes. +func TestGovParamChangeHandler_HappyPath(t *testing.T) { + cfg, _ := newGuardCfg(t, "arctic-1") + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h", Height: 0}, + queryDefault: &sdk.TxResponse{Code: 0, Height: 7}, + } + + req := validParamChangeRequest() + msg, err := buildParamChangeMsg(cfg, req) + if err != nil { + t.Fatalf("buildParamChangeMsg: %v", err) + } + info, err := cfg.Keyring.Key("node_admin") + if err != nil { + t.Fatalf("keyring: %v", err) + } + + result, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "arctic-1", + KeyName: "node_admin", + Msg: msg, + Fees: req.Fees, + Gas: req.Gas, + TaskID: "00000000-0000-0000-0000-0000000000aa", + }, info.GetAddress()) + if err != nil { + t.Fatalf("signAndBroadcast: %v", err) + } + if result.TxHash != "h" { + t.Errorf("TxHash = %q, want %q", result.TxHash, "h") + } + if tc.broadcasts != 1 { + t.Errorf("broadcasts = %d, want 1", tc.broadcasts) + } +} diff --git a/sidecar/tasks/gov_result.go b/sidecar/tasks/gov_result.go new file mode 100644 index 00000000..41910721 --- /dev/null +++ b/sidecar/tasks/gov_result.go @@ -0,0 +1,122 @@ +// Package tasks — gov completion contract. Gov sign-tx handlers surface a +// structured GovTxResult (committed_ok / committed_failed / pending) so the +// controller isn't left inferring success from a bare "task Complete". +package tasks + +import ( + "fmt" + + "github.com/prometheus/client_golang/prometheus" + + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + govtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/gov/types" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/wire" +) + +var govResultLog = seilog.NewLogger("seictl", "task", "gov-result") + +// submitProposalMsgType is the message type URL baseapp stamps on a +// MsgSubmitProposal's TxMsgData entry — guards parseProposalID against +// decoding a different message's response (e.g. a vote's) as a +// proposal-submit result. +var submitProposalMsgType = sdk.MsgTypeURL(&govtypes.MsgSubmitProposal{}) + +var txBroadcastTotal = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "seictl_tx_broadcast_total", + Help: "Gov sign-tx broadcasts by task type and inclusion outcome.", + }, + []string{"type", "outcome"}, +) + +func init() { prometheus.MustRegister(txBroadcastTotal) } + +// classifyGovResult maps a broadcast result to the gov completion contract and +// records the outcome metric. Returns (result, err): +// - committed_ok (included, code 0) → (result, nil) → task Completed +// - committed_failed (included, code≠0) → (result, Terminal) → task Failed, terminal +// - pending (inclusion undetermined) → (result, non-terminal) → task Failed; +// the controller re-submits (same task ID → re-run → marker re-check) +func classifyGovResult(taskType engine.TaskType, r *SignAndBroadcastResult) (*wire.GovTxResult, error) { + out := &wire.GovTxResult{ + TxHash: r.TxHash, + Height: r.Height, + ProposalID: r.ProposalID, + Code: r.Code, + Codespace: r.Codespace, + RawLog: r.RawLog, + } + switch { + case r.Unverifiable: + // Broadcast accepted but the node's tx index is off, so the outcome is + // unobservable. Terminal (retrying this node is futile) but NOT + // committed_failed — the operator must verify via an indexed RPC. + out.InclusionStatus = wire.InclusionUnverifiable + txBroadcastTotal.WithLabelValues(string(taskType), wire.InclusionUnverifiable).Inc() + return out, Terminal(fmt.Errorf("tx %s inclusion unverifiable: %w", r.TxHash, errTxIndexingDisabled)) + case r.IncludedAt == nil: + out.InclusionStatus = wire.InclusionPending + txBroadcastTotal.WithLabelValues(string(taskType), wire.InclusionPending).Inc() + return out, fmt.Errorf("tx %s inclusion undetermined; re-check pending", r.TxHash) + case r.Code != 0: + out.InclusionStatus = wire.InclusionCommittedFailed + txBroadcastTotal.WithLabelValues(string(taskType), wire.InclusionCommittedFailed).Inc() + return out, Terminal(fmt.Errorf("tx %s committed but failed: code=%d codespace=%q log=%s", + r.TxHash, r.Code, r.Codespace, r.RawLog)) + default: + out.InclusionStatus = wire.InclusionCommittedOK + txBroadcastTotal.WithLabelValues(string(taskType), wire.InclusionCommittedOK).Inc() + return out, nil + } +} + +// parseProposalID extracts the minted proposal ID from a committed +// MsgSubmitProposal tx's result data, or 0 if none is present: a +// not-yet-included tx (empty result data), a non-submit-proposal task such +// as a vote (index 0 is a different message's response), or a malformed +// result. SignAndBroadcastInput carries exactly one Msg per tx (never +// batched), so a MsgSubmitProposal response is always index 0 of the +// baseapp-encoded TxMsgData when present, regardless of the proposal's +// Content (software-upgrade or param-change) — the MsgType check guards +// that assumption rather than trusting the index blindly. +// This fork's gov keeper emits the proposal ID on the proposal_deposit +// event, not submit_proposal, so the tx's own result data is the one place +// the ID is unconditionally present. +func parseProposalID(resp *sdk.TxResponse) uint64 { + var txMsgData sdk.TxMsgData + if err := txMsgData.Unmarshal([]byte(resp.Data)); err != nil { + govResultLog.Warn("decode tx result data", "txHash", resp.TxHash, "height", resp.Height, "err", err) + return 0 + } + if len(txMsgData.Data) == 0 { + return 0 // not-yet-included (CheckTx-only response) or a non-Msg-service tx + } + if txMsgData.Data[0].MsgType != submitProposalMsgType { + return 0 // a different message's response (e.g. a vote) — not ours to parse + } + var msgResp govtypes.MsgSubmitProposalResponse + if err := msgResp.Unmarshal(txMsgData.Data[0].Data); err != nil { + govResultLog.Warn("decode MsgSubmitProposalResponse", "txHash", resp.TxHash, "height", resp.Height, "err", err) + return 0 + } + return msgResp.ProposalId +} + +// requireProposalID escalates a committed-ok gov result with no minted +// proposal ID to a Terminal error. A committed MsgSubmitProposal tx always +// mints an ID >= 1 (DefaultStartingProposalID); a 0 there is parseProposalID +// failing to decode it, not a legitimate outcome, and must not silently +// latch the task Complete. Callers whose result never carries a proposal ID +// (gov-vote) must not call this — cerr is returned unchanged for any +// inclusion status other than committed-ok, so it composes safely with the +// pending/committed-failed paths classifyGovResult already produces. +func requireProposalID(out *wire.GovTxResult, cerr error) error { + if cerr == nil && out.InclusionStatus == wire.InclusionCommittedOK && out.ProposalID == 0 { + return Terminal(fmt.Errorf("tx %s committed but minted no proposal ID (parse failure)", out.TxHash)) + } + return cerr +} diff --git a/sidecar/tasks/gov_result_test.go b/sidecar/tasks/gov_result_test.go new file mode 100644 index 00000000..d7e3d56a --- /dev/null +++ b/sidecar/tasks/gov_result_test.go @@ -0,0 +1,231 @@ +package tasks + +import ( + "errors" + "fmt" + "testing" + "time" + + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + govtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/gov/types" + + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/wire" +) + +// mustMarshalTxMsgData builds the baseapp-encoded TxMsgData bytes a +// committed tx carries in its result Data, wrapping one message response +// under the given type URL — the shape parseProposalID decodes. Mirrors +// baseapp's runMsgs (sei-cosmos abci.pb.go's TxMsgData/MsgData wrapping a +// raw, non-Any response). +func mustMarshalTxMsgData(t *testing.T, msgType string, respBytes []byte) string { + t.Helper() + dataBytes, err := (&sdk.TxMsgData{Data: []*sdk.MsgData{{ + MsgType: msgType, + Data: respBytes, + }}}).Marshal() + if err != nil { + t.Fatalf("marshal TxMsgData: %v", err) + } + return string(dataBytes) +} + +func mustMarshalSubmitProposalResponse(t *testing.T, proposalID uint64) string { + t.Helper() + respBytes, err := (&govtypes.MsgSubmitProposalResponse{ProposalId: proposalID}).Marshal() + if err != nil { + t.Fatalf("marshal MsgSubmitProposalResponse: %v", err) + } + return mustMarshalTxMsgData(t, submitProposalMsgType, respBytes) +} + +// An unverifiable result (broadcast accepted, outcome unobservable because the +// node's tx index is off) classifies as a TERMINAL InclusionUnverifiable +// wrapping errTxIndexingDisabled — the honest "outcome unknown, stop retrying", +// distinct from both committed_failed and the retryable pending case. +func TestClassifyGovResult_Unverifiable_Terminal(t *testing.T) { + out, err := classifyGovResult("gov-param-change", &SignAndBroadcastResult{ + TxHash: "ABC", Unverifiable: true, + }) + if out.InclusionStatus != wire.InclusionUnverifiable || out.TxHash != "ABC" { + t.Fatalf("unexpected result: %+v", out) + } + if !IsTerminal(err) { + t.Fatalf("unverifiable must be terminal (retrying is futile), got %v", err) + } + if !errors.Is(err, errTxIndexingDisabled) { + t.Fatalf("terminal error should wrap errTxIndexingDisabled, got %v", err) + } +} + +func TestClassifyGovResult_CommittedOK(t *testing.T) { + now := time.Now().UTC() + out, err := classifyGovResult("gov-software-upgrade", &SignAndBroadcastResult{ + TxHash: "ABC", Height: 10, Code: 0, IncludedAt: &now, ProposalID: 5, + }) + if err != nil { + t.Fatalf("committed-ok should not error, got %v", err) + } + if out.InclusionStatus != wire.InclusionCommittedOK || out.ProposalID != 5 || out.TxHash != "ABC" { + t.Fatalf("unexpected result: %+v", out) + } +} + +func TestClassifyGovResult_CommittedFailed_Terminal(t *testing.T) { + now := time.Now().UTC() + out, err := classifyGovResult("gov-param-change", &SignAndBroadcastResult{ + TxHash: "ABC", Height: 10, Code: 11, Codespace: "sdk", RawLog: "insufficient funds", IncludedAt: &now, + }) + if err == nil || !IsTerminal(err) { + t.Fatalf("committed-failed should be a terminal error, got %v", err) + } + if out.InclusionStatus != wire.InclusionCommittedFailed || out.Code != 11 { + t.Fatalf("unexpected result: %+v", out) + } +} + +func TestClassifyGovResult_Pending_NonTerminal(t *testing.T) { + out, err := classifyGovResult("gov-vote", &SignAndBroadcastResult{ + TxHash: "ABC", IncludedAt: nil, + }) + if err == nil || IsTerminal(err) { + t.Fatalf("pending should be a non-terminal error (controller re-submits), got %v", err) + } + if out.InclusionStatus != wire.InclusionPending || out.TxHash != "ABC" { + t.Fatalf("unexpected result: %+v", out) + } +} + +func TestParseProposalID(t *testing.T) { + resp := &sdk.TxResponse{ + TxHash: "ABC", + Height: 10, + Data: mustMarshalSubmitProposalResponse(t, 42), + } + if got := parseProposalID(resp); got != 42 { + t.Fatalf("proposal id = %d, want 42", got) + } +} + +// A tx result carrying no message responses (empty Data) yields 0, not a panic +// — e.g. a vote or non-Msg-service tx passed through the same helper. +func TestParseProposalID_EmptyData(t *testing.T) { + if got := parseProposalID(&sdk.TxResponse{}); got != 0 { + t.Fatalf("empty Data should yield 0, got %d", got) + } +} + +// Garbage Data (not a valid TxMsgData) yields 0, not a panic or error. +func TestParseProposalID_MalformedData(t *testing.T) { + resp := &sdk.TxResponse{TxHash: "ABC", Height: 10, Data: "not a valid protobuf message"} + if got := parseProposalID(resp); got != 0 { + t.Fatalf("malformed Data should yield 0, got %d", got) + } +} + +// TxMsgData with zero message responses yields 0, not an index panic. +func TestParseProposalID_NoMessageResponses(t *testing.T) { + dataBytes, err := (&sdk.TxMsgData{}).Marshal() + if err != nil { + t.Fatalf("marshal empty TxMsgData: %v", err) + } + resp := &sdk.TxResponse{TxHash: "ABC", Height: 10, Data: string(dataBytes)} + if got := parseProposalID(resp); got != 0 { + t.Fatalf("zero message responses should yield 0, got %d", got) + } +} + +// A vote's response at index 0 must not be decoded as a MsgSubmitProposal +// response: MsgVoteResponse is an empty message, so a naive decode would +// succeed and silently return 0 for the wrong reason — the MsgType guard +// must reject it before the decode is even attempted. Pins the guard +// against a future message type whose response DOES have a field 1, which +// would otherwise decode into a plausible-looking, wrong proposal ID. +func TestParseProposalID_WrongMsgType(t *testing.T) { + respBytes, err := (&govtypes.MsgVoteResponse{}).Marshal() + if err != nil { + t.Fatalf("marshal MsgVoteResponse: %v", err) + } + resp := &sdk.TxResponse{ + TxHash: "ABC", Height: 10, + Data: mustMarshalTxMsgData(t, sdk.MsgTypeURL(&govtypes.MsgVote{}), respBytes), + } + if got := parseProposalID(resp); got != 0 { + t.Fatalf("a vote response must not be decoded as MsgSubmitProposalResponse, got %d", got) + } +} + +// Index 0 is the MsgSubmitProposal response even when the tx result carries +// additional message responses after it — pins the index-0, not-last +// semantics the fix relies on (SignAndBroadcastInput never actually batches +// today, but this documents why index 0 specifically is correct). +func TestParseProposalID_MultipleMessageResponses(t *testing.T) { + submitBytes, err := (&govtypes.MsgSubmitProposalResponse{ProposalId: 7}).Marshal() + if err != nil { + t.Fatalf("marshal MsgSubmitProposalResponse: %v", err) + } + voteBytes, err := (&govtypes.MsgVoteResponse{}).Marshal() + if err != nil { + t.Fatalf("marshal MsgVoteResponse: %v", err) + } + dataBytes, err := (&sdk.TxMsgData{Data: []*sdk.MsgData{ + {MsgType: submitProposalMsgType, Data: submitBytes}, + {MsgType: sdk.MsgTypeURL(&govtypes.MsgVote{}), Data: voteBytes}, + }}).Marshal() + if err != nil { + t.Fatalf("marshal TxMsgData: %v", err) + } + resp := &sdk.TxResponse{TxHash: "ABC", Height: 10, Data: string(dataBytes)} + if got := parseProposalID(resp); got != 7 { + t.Fatalf("proposal id = %d, want 7 (from index 0)", got) + } +} + +func TestRequireProposalID(t *testing.T) { + t.Run("committed-ok with zero ID is escalated to Terminal", func(t *testing.T) { + out := &wire.GovTxResult{TxHash: "ABC", InclusionStatus: wire.InclusionCommittedOK, ProposalID: 0} + err := requireProposalID(out, nil) + if err == nil || !IsTerminal(err) { + t.Fatalf("want a Terminal error, got %v", err) + } + }) + t.Run("committed-ok with a real ID passes through nil", func(t *testing.T) { + out := &wire.GovTxResult{TxHash: "ABC", InclusionStatus: wire.InclusionCommittedOK, ProposalID: 5} + if err := requireProposalID(out, nil); err != nil { + t.Fatalf("want nil, got %v", err) + } + }) + t.Run("pending with zero ID is not escalated — a vote or not-yet-included tx legitimately has no ID", func(t *testing.T) { + out := &wire.GovTxResult{TxHash: "ABC", InclusionStatus: wire.InclusionPending, ProposalID: 0} + if err := requireProposalID(out, nil); err != nil { + t.Fatalf("pending must not be escalated, got %v", err) + } + }) + t.Run("an existing cerr is preserved, not overridden", func(t *testing.T) { + out := &wire.GovTxResult{TxHash: "ABC", InclusionStatus: wire.InclusionCommittedFailed, ProposalID: 0} + want := Terminal(fmt.Errorf("committed but failed")) + if got := requireProposalID(out, want); got != want { + t.Fatalf("want the original error preserved, got %v", got) + } + }) +} + +// TestVoteOptionValuesMatchGovtypes guards the wire.VoteOption consts against +// drift from the cosmos govtypes enum the server casts to +// (govtypes.VoteOption(wire.Option*)). A renumber would silently mismap a vote. +func TestVoteOptionValuesMatchGovtypes(t *testing.T) { + cases := []struct { + w wire.VoteOption + g govtypes.VoteOption + }{ + {wire.OptionEmpty, govtypes.OptionEmpty}, + {wire.OptionYes, govtypes.OptionYes}, + {wire.OptionAbstain, govtypes.OptionAbstain}, + {wire.OptionNo, govtypes.OptionNo}, + {wire.OptionNoWithVeto, govtypes.OptionNoWithVeto}, + } + for _, c := range cases { + if int(c.w) != int(c.g) { + t.Fatalf("wire.VoteOption %d != govtypes %d — ParseVoteOption cast would mismap", c.w, c.g) + } + } +} diff --git a/sidecar/tasks/gov_software_upgrade.go b/sidecar/tasks/gov_software_upgrade.go new file mode 100644 index 00000000..d90034cc --- /dev/null +++ b/sidecar/tasks/gov_software_upgrade.go @@ -0,0 +1,154 @@ +// Package tasks — gov-software-upgrade handler. +// +// This handler signs software-upgrade proposals as the validator's +// operator account. API authentication is controlled by +// SEI_SIDECAR_AUTHN_MODE; see sidecar/server/auth.go for the +// deployment guidance. +// +// REHYDRATION — sei-protocol/seictl#174 +// +// MsgSubmitProposal is NOT chain-idempotent. Crash-idempotency is provided by +// the pre-broadcast TxMarker + rehydrate-adopt in SignAndBroadcast: a re-run +// adopts the in-flight tx (re-broadcasting the identical signed bytes) rather +// than re-signing, so a crash between broadcast and result-persist no longer +// produces a second proposal. + +package tasks + +import ( + "context" + "errors" + "fmt" + + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + govtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/gov/types" + upgradetypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/upgrade/types" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/wire" +) + +var govSoftwareUpgradeLog = seilog.NewLogger("seictl", "task", "gov-software-upgrade") + +// GovSoftwareUpgradeRequest holds gov-software-upgrade params. +// Idempotency is NOT handled — see the REHYDRATION WARNING at the +// top of this file. +type GovSoftwareUpgradeRequest struct { + ChainID string `json:"chainId"` + KeyName string `json:"keyName"` + + Title string `json:"title"` + Description string `json:"description"` + + UpgradeName string `json:"upgradeName"` + UpgradeHeight int64 `json:"upgradeHeight"` + UpgradeInfo string `json:"upgradeInfo,omitempty"` + + InitialDeposit string `json:"initialDeposit"` + + Memo string `json:"memo,omitempty"` + Fees string `json:"fees"` + Gas uint64 `json:"gas"` +} + +// GovSoftwareUpgrader captures cfg by value at construction; +// engine.Config is documented read-only after startup, so the copy +// is safe. +type GovSoftwareUpgrader struct { + cfg engine.ExecutionConfig +} + +func NewGovSoftwareUpgrader(cfg engine.ExecutionConfig) *GovSoftwareUpgrader { + return &GovSoftwareUpgrader{cfg: cfg} +} + +// Handler delegates to SignAndBroadcast (which owns the crash-idempotency +// marker — see the REHYDRATION note at the top of this file) and classifies +// the outcome via classifyGovResult. +func (g *GovSoftwareUpgrader) Handler() engine.TaskHandler { + return engine.TypedHandlerWithResult(func(ctx context.Context, params GovSoftwareUpgradeRequest) (*wire.GovTxResult, error) { + msg, err := buildSoftwareUpgradeMsg(g.cfg, params) + if err != nil { + return nil, err + } + result, err := SignAndBroadcast(ctx, g.cfg, SignAndBroadcastInput{ + ChainID: params.ChainID, + KeyName: params.KeyName, + Msg: msg, + Fees: params.Fees, + Gas: params.Gas, + Memo: params.Memo, + TaskID: engine.TaskIDFromContext(ctx), + }) + if err != nil { + return nil, err + } + out, cerr := classifyGovResult(engine.TaskGovSoftwareUpgrade, result) + cerr = requireProposalID(out, cerr) + govSoftwareUpgradeLog.Info("proposal broadcast", + "taskId", engine.TaskIDFromContext(ctx), + "chainId", params.ChainID, + "upgradeName", params.UpgradeName, + "upgradeHeight", params.UpgradeHeight, + "txHash", out.TxHash, + "height", out.Height, + "proposalId", out.ProposalID, + "inclusionStatus", out.InclusionStatus) + return out, cerr + }) +} + +func buildSoftwareUpgradeMsg(cfg engine.ExecutionConfig, params GovSoftwareUpgradeRequest) (*govtypes.MsgSubmitProposal, error) { + if cfg.Keyring == nil { + return nil, Terminal(errors.New("keyring not configured: set SEI_KEYRING_BACKEND/SEI_KEYRING_PASSPHRASE on the sidecar")) + } + if params.KeyName == "" { + return nil, Terminal(errors.New("keyName required")) + } + if params.Title == "" { + return nil, Terminal(errors.New("title required")) + } + if params.Description == "" { + return nil, Terminal(errors.New("description required")) + } + if params.UpgradeName == "" { + return nil, Terminal(errors.New("upgradeName required")) + } + if params.UpgradeHeight <= 0 { + return nil, Terminal(errors.New("upgradeHeight required (must be > 0)")) + } + info, err := cfg.Keyring.Key(params.KeyName) + if err != nil { + return nil, Terminal(fmt.Errorf("keyring entry %q: %w", params.KeyName, err)) + } + deposit, err := sdk.ParseCoinsNormalized(params.InitialDeposit) + if err != nil { + return nil, Terminal(fmt.Errorf("parse initialDeposit %q: %w", params.InitialDeposit, err)) + } + if len(deposit) == 0 { + return nil, Terminal(fmt.Errorf("initialDeposit %q resolves to zero coins", params.InitialDeposit)) + } + if !deposit.IsAllPositive() { + return nil, Terminal(fmt.Errorf("initialDeposit %q contains non-positive amounts", params.InitialDeposit)) + } + // Symmetric with checkFeesDenom: deposit denom is fixed by gov + // params on Sei (usei); a wrong denom would CheckTx-reject anyway, + // but rejecting here saves the sign + broadcast roundtrip. + for _, c := range deposit { + if c.Denom != feeDenom { + return nil, Terminal(fmt.Errorf("initialDeposit %q: denom %q not permitted (only %q)", params.InitialDeposit, c.Denom, feeDenom)) + } + } + content := upgradetypes.NewSoftwareUpgradeProposal(params.Title, params.Description, upgradetypes.Plan{ + Name: params.UpgradeName, + Height: params.UpgradeHeight, + Info: params.UpgradeInfo, + }) + msg, err := govtypes.NewMsgSubmitProposal(content, deposit, info.GetAddress()) + if err != nil { + return nil, Terminal(fmt.Errorf("build MsgSubmitProposal: %w", err)) + } + return msg, nil +} diff --git a/sidecar/tasks/gov_software_upgrade_test.go b/sidecar/tasks/gov_software_upgrade_test.go new file mode 100644 index 00000000..940a66ab --- /dev/null +++ b/sidecar/tasks/gov_software_upgrade_test.go @@ -0,0 +1,145 @@ +package tasks + +import ( + "context" + "strings" + "testing" + + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + upgradetypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/upgrade/types" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +func validSoftwareUpgradeRequest() GovSoftwareUpgradeRequest { + return GovSoftwareUpgradeRequest{ + ChainID: "pacific-1", + KeyName: "node_admin", + Title: "Upgrade to v0.42", + Description: "Routine release; binaries published in release notes.", + UpgradeName: "v0.42", + UpgradeHeight: 10_000_000, + UpgradeInfo: "https://github.com/sei-protocol/sei-chain/releases/tag/v0.42", + InitialDeposit: "10000000usei", + Fees: "4000usei", + Gas: 300_000, + } +} + +func TestBuildSoftwareUpgradeMsg(t *testing.T) { + kr, addr := testKeyring(t) + cfg := engine.ExecutionConfig{Keyring: kr} + + t.Run("happy path", func(t *testing.T) { + msg, err := buildSoftwareUpgradeMsg(cfg, validSoftwareUpgradeRequest()) + if err != nil { + t.Fatalf("unexpected err: %v", err) + } + if msg.Proposer != addr.String() { + t.Errorf("proposer = %q, want %q", msg.Proposer, addr.String()) + } + want, _ := sdk.ParseCoinsNormalized("10000000usei") + if !msg.InitialDeposit.IsEqual(want) { + t.Errorf("InitialDeposit = %v, want %v", msg.InitialDeposit, want) + } + // Lock that the SDK's MsgSubmitProposal.ValidateBasic and the + // wrapped SoftwareUpgradeProposal.ValidateBasic both accept what + // we produce — signAndBroadcast runs them next. + if err := msg.ValidateBasic(); err != nil { + t.Errorf("ValidateBasic on returned msg: %v", err) + } + content := msg.GetContent() + if _, ok := content.(*upgradetypes.SoftwareUpgradeProposal); !ok { + t.Errorf("content type = %T, want *SoftwareUpgradeProposal", content) + } + }) + + mutations := []struct { + name string + mutate func(*GovSoftwareUpgradeRequest) + wantInErr string + }{ + {"missing keyName", func(r *GovSoftwareUpgradeRequest) { r.KeyName = "" }, "keyName required"}, + {"missing title", func(r *GovSoftwareUpgradeRequest) { r.Title = "" }, "title required"}, + {"missing description", func(r *GovSoftwareUpgradeRequest) { r.Description = "" }, "description required"}, + {"missing upgradeName", func(r *GovSoftwareUpgradeRequest) { r.UpgradeName = "" }, "upgradeName required"}, + {"zero upgradeHeight", func(r *GovSoftwareUpgradeRequest) { r.UpgradeHeight = 0 }, "upgradeHeight required"}, + {"negative upgradeHeight", func(r *GovSoftwareUpgradeRequest) { r.UpgradeHeight = -1 }, "upgradeHeight required"}, + {"unparseable deposit", func(r *GovSoftwareUpgradeRequest) { r.InitialDeposit = "not-a-coin" }, "parse initialDeposit"}, + {"zero deposit", func(r *GovSoftwareUpgradeRequest) { r.InitialDeposit = "0usei" }, "zero coins"}, + {"non-usei deposit", func(r *GovSoftwareUpgradeRequest) { r.InitialDeposit = "10sei" }, "usei"}, + {"mixed-denom deposit", func(r *GovSoftwareUpgradeRequest) { r.InitialDeposit = "10000000usei,1uatom" }, "uatom"}, + } + for _, m := range mutations { + t.Run(m.name+" is Terminal", func(t *testing.T) { + req := validSoftwareUpgradeRequest() + m.mutate(&req) + _, err := buildSoftwareUpgradeMsg(cfg, req) + if !IsTerminal(err) { + t.Fatalf("want Terminal, got %v", err) + } + if !strings.Contains(err.Error(), m.wantInErr) { + t.Fatalf("err = %q, want substring %q", err.Error(), m.wantInErr) + } + }) + } + + t.Run("nil keyring is Terminal", func(t *testing.T) { + _, err := buildSoftwareUpgradeMsg(engine.ExecutionConfig{}, validSoftwareUpgradeRequest()) + if !IsTerminal(err) { + t.Fatalf("want Terminal, got %v", err) + } + }) + + t.Run("missing key is Terminal", func(t *testing.T) { + req := validSoftwareUpgradeRequest() + req.KeyName = "ghost" + _, err := buildSoftwareUpgradeMsg(cfg, req) + if !IsTerminal(err) { + t.Fatalf("want Terminal, got %v", err) + } + }) +} + +// TestGovSoftwareUpgradeHandler_HappyPath threads the handler +// end-to-end through signAndBroadcast with a fake txClient. The +// MsgSubmitProposal content is packed as an Any, so reaching the +// broadcast step at all proves newSignTxInterfaceRegistry has the +// upgrade-types registration; a missing registration fails in +// txCfg.TxEncoder before the fakeTxClient ever sees bytes. +func TestGovSoftwareUpgradeHandler_HappyPath(t *testing.T) { + cfg, _ := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h", Height: 0}, + queryDefault: &sdk.TxResponse{Code: 0, Height: 7}, + } + + msg, err := buildSoftwareUpgradeMsg(cfg, validSoftwareUpgradeRequest()) + if err != nil { + t.Fatalf("buildSoftwareUpgradeMsg: %v", err) + } + info, err := cfg.Keyring.Key("node_admin") + if err != nil { + t.Fatalf("keyring: %v", err) + } + + result, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: msg, + Fees: "4000usei", + Gas: 300_000, + TaskID: "00000000-0000-0000-0000-0000000000aa", + }, info.GetAddress()) + if err != nil { + t.Fatalf("signAndBroadcast: %v", err) + } + if result.TxHash != "h" { + t.Errorf("TxHash = %q, want %q", result.TxHash, "h") + } + if tc.broadcasts != 1 { + t.Errorf("broadcasts = %d, want 1", tc.broadcasts) + } +} diff --git a/sidecar/tasks/gov_vote.go b/sidecar/tasks/gov_vote.go new file mode 100644 index 00000000..8f7686ae --- /dev/null +++ b/sidecar/tasks/gov_vote.go @@ -0,0 +1,109 @@ +// Package tasks — gov-vote handler. +// +// This handler signs votes as the validator's operator account. +// API authentication is controlled by SEI_SIDECAR_AUTHN_MODE; see +// sidecar/server/auth.go for the deployment guidance. + +package tasks + +import ( + "context" + "errors" + "fmt" + + govtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/gov/types" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/wire" +) + +var govVoteLog = seilog.NewLogger("seictl", "task", "gov-vote") + +// GovVoteRequest holds gov-vote params. Idempotency is handled by the +// chain — last-write-wins on (proposalID, voter); no pre-broadcast +// chain query. +type GovVoteRequest struct { + ChainID string `json:"chainId"` + KeyName string `json:"keyName"` + ProposalID uint64 `json:"proposalId"` + Option string `json:"option"` // yes | no | abstain | no_with_veto + Memo string `json:"memo,omitempty"` + Fees string `json:"fees"` + Gas uint64 `json:"gas"` +} + +// GovVoter captures cfg by value at construction; engine.Config is +// documented read-only after startup, so the copy is safe. +type GovVoter struct { + cfg engine.ExecutionConfig +} + +func NewGovVoter(cfg engine.ExecutionConfig) *GovVoter { + return &GovVoter{cfg: cfg} +} + +// Handler delegates to SignAndBroadcast after MsgVote construction. +// +// Rehydration: a crash after BroadcastSync but before result persist +// re-runs this handler. The rehydrated run signs at sequence+1 and +// broadcasts a second tx; chain last-write-wins on (proposalID, voter) +// keeps governance state correct and the operator pays fees twice. +// Safe ONLY because MsgVote is chain-idempotent — non-idempotent Msg +// types (MsgSend, MsgWithdraw…) would double-spend. Future sign-tx +// handlers must evaluate per-Msg idempotency before reusing this shape. +// +// Stale proposals are rejected by CheckTx and surface as Terminal. We +// do not pre-check via chain query — that opens a TOCTOU window. +func (g *GovVoter) Handler() engine.TaskHandler { + return engine.TypedHandlerWithResult(func(ctx context.Context, params GovVoteRequest) (*wire.GovTxResult, error) { + msg, err := buildVoteMsg(g.cfg, params) + if err != nil { + return nil, err + } + result, err := SignAndBroadcast(ctx, g.cfg, SignAndBroadcastInput{ + ChainID: params.ChainID, + KeyName: params.KeyName, + Msg: msg, + Fees: params.Fees, + Gas: params.Gas, + Memo: params.Memo, + TaskID: engine.TaskIDFromContext(ctx), + }) + if err != nil { + return nil, err + } + out, cerr := classifyGovResult(engine.TaskGovVote, result) + govVoteLog.Info("vote broadcast", + "taskId", engine.TaskIDFromContext(ctx), + "chainId", params.ChainID, + "proposalId", params.ProposalID, + "option", params.Option, + "txHash", out.TxHash, + "height", out.Height, + "inclusionStatus", out.InclusionStatus) + return out, cerr + }) +} + +func buildVoteMsg(cfg engine.ExecutionConfig, params GovVoteRequest) (*govtypes.MsgVote, error) { + if params.ProposalID == 0 { + return nil, Terminal(errors.New("proposalId required (must be > 0)")) + } + option, err := wire.ParseVoteOption(params.Option) + if err != nil { + return nil, Terminal(err) + } + if cfg.Keyring == nil { + return nil, Terminal(errors.New("keyring not configured: set SEI_KEYRING_BACKEND/SEI_KEYRING_PASSPHRASE on the sidecar")) + } + if params.KeyName == "" { + return nil, Terminal(errors.New("keyName required")) + } + info, err := cfg.Keyring.Key(params.KeyName) + if err != nil { + return nil, Terminal(fmt.Errorf("keyring entry %q: %w", params.KeyName, err)) + } + return govtypes.NewMsgVote(info.GetAddress(), params.ProposalID, govtypes.VoteOption(option)), nil +} diff --git a/sidecar/tasks/gov_vote_test.go b/sidecar/tasks/gov_vote_test.go new file mode 100644 index 00000000..7e592d65 --- /dev/null +++ b/sidecar/tasks/gov_vote_test.go @@ -0,0 +1,100 @@ +package tasks + +import ( + "errors" + "testing" + + govtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/gov/types" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +func TestBuildVoteMsg(t *testing.T) { + kr, addr := testKeyring(t) + cfg := engine.ExecutionConfig{Keyring: kr} + + t.Run("happy path", func(t *testing.T) { + msg, err := buildVoteMsg(cfg, GovVoteRequest{ + KeyName: "node_admin", + ProposalID: 42, + Option: "yes", + }) + if err != nil { + t.Fatalf("unexpected err: %v", err) + } + if msg.Voter != addr.String() { + t.Errorf("voter = %q, want %q", msg.Voter, addr.String()) + } + if msg.ProposalId != 42 { + t.Errorf("proposalId = %d, want 42", msg.ProposalId) + } + if msg.Option != govtypes.OptionYes { + t.Errorf("option = %v, want OptionYes", msg.Option) + } + // Guard: signAndBroadcast runs ValidateBasic immediately; lock + // that it accepts the message we produce here. + if err := msg.ValidateBasic(); err != nil { + t.Errorf("ValidateBasic on returned msg: %v", err) + } + }) + + t.Run("zero proposalId is Terminal", func(t *testing.T) { + _, err := buildVoteMsg(cfg, GovVoteRequest{ + KeyName: "node_admin", + ProposalID: 0, + Option: "yes", + }) + if !IsTerminal(err) { + t.Fatalf("want Terminal, got %v", err) + } + }) + + t.Run("invalid option is Terminal", func(t *testing.T) { + _, err := buildVoteMsg(cfg, GovVoteRequest{ + KeyName: "node_admin", + ProposalID: 7, + Option: "bogus", + }) + if !IsTerminal(err) { + t.Fatalf("want Terminal, got %v", err) + } + }) + + t.Run("nil keyring is Terminal", func(t *testing.T) { + _, err := buildVoteMsg(engine.ExecutionConfig{}, GovVoteRequest{ + KeyName: "node_admin", + ProposalID: 7, + Option: "yes", + }) + if !IsTerminal(err) { + t.Fatalf("want Terminal, got %v", err) + } + }) + + t.Run("empty keyName is Terminal", func(t *testing.T) { + _, err := buildVoteMsg(cfg, GovVoteRequest{ + KeyName: "", + ProposalID: 7, + Option: "yes", + }) + if !IsTerminal(err) { + t.Fatalf("want Terminal, got %v", err) + } + }) + + t.Run("missing key in keyring is Terminal", func(t *testing.T) { + _, err := buildVoteMsg(cfg, GovVoteRequest{ + KeyName: "does-not-exist", + ProposalID: 7, + Option: "yes", + }) + if !IsTerminal(err) { + t.Fatalf("want Terminal, got %v", err) + } + // Make sure the underlying keyring error is preserved. + var terr *TerminalError + if !errors.As(err, &terr) || terr.Unwrap() == nil { + t.Fatalf("expected wrapped keyring error: %v", err) + } + }) +} diff --git a/sidecar/tasks/mark_not_ready.go b/sidecar/tasks/mark_not_ready.go new file mode 100644 index 00000000..03673648 --- /dev/null +++ b/sidecar/tasks/mark_not_ready.go @@ -0,0 +1,52 @@ +package tasks + +import ( + "context" + "fmt" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +var markNotReadyLog = seilog.NewLogger("seictl", "task", "mark-not-ready") + +// markReadyPurger deletes recorded mark-ready results. The concrete +// implementation is the engine's ResultStore; a narrow interface keeps the +// handler testable without a full store. +type markReadyPurger interface { + DeleteByType(taskType string) (int, error) +} + +// MarkNotReadier re-arms the seid start gate for a node hold. Its handler +// purges recorded mark-ready results from the task store; the engine then +// flips the readiness flag false on success (the sole false-writer, see +// engine.execute). The purge is what closes the rehydration release path: a +// mark-ready left running by an ungraceful shutdown would otherwise re-run on +// restart and mark the engine ready again, releasing seid onto a wiped data +// directory. +type MarkNotReadier struct { + purger markReadyPurger +} + +// NewMarkNotReadier builds a MarkNotReadier over the given store. +func NewMarkNotReadier(purger markReadyPurger) *MarkNotReadier { + return &MarkNotReadier{purger: purger} +} + +// Handler returns an engine.TaskHandler for the mark-not-ready task type. +// Params are empty. The handler purges mark-ready records and returns; the +// engine's completion hook performs the readiness flip. On purge failure it +// returns an error so the engine skips the flip (fail-safe: readiness is left +// untouched rather than flipped over a store that still holds a releasable +// mark-ready). +func (m *MarkNotReadier) Handler() engine.TaskHandler { + return engine.TypedHandler(func(_ context.Context, _ struct{}) error { + n, err := m.purger.DeleteByType(string(engine.TaskMarkReady)) + if err != nil { + return fmt.Errorf("mark-not-ready: purging mark-ready records: %w", err) + } + markNotReadyLog.Info("purged mark-ready records before hold", "count", n) + return nil + }) +} diff --git a/sidecar/tasks/mark_not_ready_test.go b/sidecar/tasks/mark_not_ready_test.go new file mode 100644 index 00000000..b442a911 --- /dev/null +++ b/sidecar/tasks/mark_not_ready_test.go @@ -0,0 +1,39 @@ +package tasks + +import ( + "context" + "fmt" + "testing" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +// fakePurger records DeleteByType calls and can inject a failure. +type fakePurger struct { + deleted []string + n int + err error +} + +func (p *fakePurger) DeleteByType(taskType string) (int, error) { + p.deleted = append(p.deleted, taskType) + return p.n, p.err +} + +func TestMarkNotReady_PurgesMarkReadyRecords(t *testing.T) { + p := &fakePurger{n: 2} + if _, err := NewMarkNotReadier(p).Handler()(context.Background(), nil); err != nil { + t.Fatalf("expected success, got %v", err) + } + if len(p.deleted) != 1 || p.deleted[0] != string(engine.TaskMarkReady) { + t.Errorf("expected a single purge of %q, got %v", engine.TaskMarkReady, p.deleted) + } +} + +func TestMarkNotReady_PurgeFailurePropagates(t *testing.T) { + p := &fakePurger{err: fmt.Errorf("store offline")} + _, err := NewMarkNotReadier(p).Handler()(context.Background(), nil) + if err == nil { + t.Fatal("expected purge failure to propagate so the engine skips the readiness flip") + } +} diff --git a/sidecar/tasks/peers.go b/sidecar/tasks/peers.go new file mode 100644 index 00000000..b9892133 --- /dev/null +++ b/sidecar/tasks/peers.go @@ -0,0 +1,16 @@ +package tasks + +import ( + "path/filepath" + "strings" +) + +func writePeersToConfig(homeDir string, peers []string) error { + configPath := filepath.Join(homeDir, "config", "config.toml") + peersPatch := map[string]any{ + "p2p": map[string]any{ + "persistent-peers": strings.Join(peers, ","), + }, + } + return mergeAndWrite(configPath, peersPatch) +} diff --git a/sidecar/tasks/ready.go b/sidecar/tasks/ready.go new file mode 100644 index 00000000..22b10e7c --- /dev/null +++ b/sidecar/tasks/ready.go @@ -0,0 +1,15 @@ +package tasks + +import ( + "context" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" +) + +// MarkReadyHandler returns a no-op TaskHandler. When it succeeds, the engine +// marks itself as ready. +func MarkReadyHandler() engine.TaskHandler { + return engine.TypedHandler(func(_ context.Context, _ struct{}) error { + return nil + }) +} diff --git a/sidecar/tasks/reset_data.go b/sidecar/tasks/reset_data.go new file mode 100644 index 00000000..688217fd --- /dev/null +++ b/sidecar/tasks/reset_data.go @@ -0,0 +1,176 @@ +package tasks + +import ( + "context" + "fmt" + "io/fs" + "os" + "path/filepath" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +var resetDataLog = seilog.NewLogger("seictl", "task", "reset-data") + +// privValidatorStateFile is CometBFT's last-sign-state file. It lives inside +// data/ (unlike node_key.json / priv_validator_key.json, which live in config/ +// and are therefore outside the wipe), so the reset rewrites it fresh. +const privValidatorStateFile = "priv_validator_state.json" + +// emptyPrivValidatorState is the reset last-sign-state (unsafe-reset-all +// semantics): height is a JSON string, round and step are numbers. RPC nodes +// do not sign, and validators are excluded from the recipe, so writing a fresh +// zero state is always safe here. +const emptyPrivValidatorState = `{"height":"0","round":0,"step":0}` + "\n" + +// ResetDataResult is the reset-data task's structured result. WipedBytes is the +// pre-wipe on-disk size of data/ (regular files only), surfaced so the +// workflow's hold event can report how much was cleared. It is -1 when the +// measurement failed: the count is observability, never a gate, so a failed +// measurement does not block the reset. Symlinked entries are counted by their +// own (link) size rather than their target, so a data dir that uses symlinks +// may undercount — acceptable for an observability figure. +type ResetDataResult struct { + WipedBytes int64 `json:"wipedBytes"` +} + +// ResetDataer clears the chain data directory for a state-sync re-bootstrap. +// The wipe is scoped to /data/ and nothing else: the home root holds +// config/ (node identity), the sidecar's task ledger (sidecar.db — the database +// that resumes this very wipe after a crash), and the hold sentinel/markers. +// Wiping the home root would destroy the machinery mid-flight; this is the +// design's most important correctness rule. +// +// The reset needs no atomicity of its own. A partially deleted data directory +// is only dangerous if seid starts on it, and the node hold guarantees it does +// not. As defense-in-depth the handler refuses to run while seid's local RPC is +// serving (i.e. the node is not actually held). It is content-idempotent: an +// already-wiped directory is success. +type ResetDataer struct { + homeDir string + probeUp func(ctx context.Context) bool + // measure returns the pre-wipe size of data/. A test seam; defaults to + // dirSize when nil. + measure func(dir string) (int64, error) +} + +// NewResetDataer builds a ResetDataer rooted at homeDir with the real local-RPC +// serving probe. +func NewResetDataer(homeDir string) *ResetDataer { + statusClient := rpc.NewStatusClient("", nil) + return &ResetDataer{ + homeDir: homeDir, + probeUp: func(ctx context.Context) bool { return seidRPCUp(ctx, statusClient) }, + measure: dirSize, + } +} + +// Handler returns an engine.TaskHandler for the reset-data task type. Params +// are empty; the result carries the pre-wipe byte count. +func (d *ResetDataer) Handler() engine.TaskHandler { + return engine.TypedHandlerWithResult(func(ctx context.Context, _ struct{}) (ResetDataResult, error) { + return d.reset(ctx) + }) +} + +func (d *ResetDataer) reset(ctx context.Context) (ResetDataResult, error) { + // Defense-in-depth: a serving RPC means seid is running, so the node is not + // held and a wipe would race a live process. Refuse rather than wipe under + // it (mirrors restart-seid's refusal to report a stop that did not happen). + if d.probeUp(ctx) { + return ResetDataResult{}, fmt.Errorf("reset-data: seid RPC is serving; node is not held — refusing to wipe a live data directory") + } + + dataDir := filepath.Join(d.homeDir, "data") + + // Measurement is observability only — never let it gate the wipe. On any + // non-ENOENT failure, log and proceed with an unknown (-1) size. + measure := d.measure + if measure == nil { + measure = dirSize + } + size, err := measure(dataDir) + if err != nil { + resetDataLog.Warn("measuring data dir failed; proceeding with unknown size", "dir", dataDir, "err", err) + size = -1 + } + + if err := wipeDirContents(dataDir); err != nil { + return ResetDataResult{}, fmt.Errorf("reset-data: wiping %s: %w", dataDir, err) + } + + // Recreate data/ (the wipe may have removed it if it was empty of anything + // but itself) and drop a fresh zero sign-state. + if err := os.MkdirAll(dataDir, 0o750); err != nil { + return ResetDataResult{}, fmt.Errorf("reset-data: recreating %s: %w", dataDir, err) + } + statePath := filepath.Join(dataDir, privValidatorStateFile) + if err := os.WriteFile(statePath, []byte(emptyPrivValidatorState), 0o600); err != nil { + return ResetDataResult{}, fmt.Errorf("reset-data: writing %s: %w", statePath, err) + } + + // Clear the state-sync completion marker (home root, outside data/) so the + // downstream configure-state-sync reruns instead of short-circuiting. + markerPath := filepath.Join(d.homeDir, stateSyncMarkerFile) + if err := os.Remove(markerPath); err != nil && !os.IsNotExist(err) { + return ResetDataResult{}, fmt.Errorf("reset-data: removing marker %s: %w", markerPath, err) + } + + resetDataLog.Info("data directory reset", "dir", dataDir, "wipedBytes", size) + return ResetDataResult{WipedBytes: size}, nil +} + +// wipeDirContents removes every entry under dir, leaving dir itself. A missing +// dir is success (content-idempotent: already-empty is the goal state), and a +// concurrent peer removing an entry first (ENOENT) is tolerated so a rehydrated +// re-run cannot fail on a half-wiped tree. +func wipeDirContents(dir string) error { + entries, err := os.ReadDir(dir) + if err != nil { + if os.IsNotExist(err) { + return nil + } + return err + } + for _, e := range entries { + p := filepath.Join(dir, e.Name()) + if err := os.RemoveAll(p); err != nil && !os.IsNotExist(err) { + return fmt.Errorf("removing %s: %w", p, err) + } + } + return nil +} + +// dirSize sums the on-disk size of regular files under dir. A missing dir is +// zero. Errors from entries that vanish mid-walk (ENOENT) are ignored — the +// count is observability, not a correctness signal. +func dirSize(dir string) (int64, error) { + var total int64 + err := filepath.WalkDir(dir, func(_ string, entry fs.DirEntry, err error) error { + if err != nil { + if os.IsNotExist(err) { + return nil + } + return err + } + if !entry.Type().IsRegular() { + return nil + } + info, err := entry.Info() + if err != nil { + if os.IsNotExist(err) { + return nil + } + return err + } + total += info.Size() + return nil + }) + if os.IsNotExist(err) { + return 0, nil + } + return total, err +} diff --git a/sidecar/tasks/reset_data_test.go b/sidecar/tasks/reset_data_test.go new file mode 100644 index 00000000..89a6f6b1 --- /dev/null +++ b/sidecar/tasks/reset_data_test.go @@ -0,0 +1,177 @@ +package tasks + +import ( + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "testing" +) + +// newResetDataer builds a ResetDataer over homeDir whose RPC probe reports the +// given serving state (true = seid up, which the reset must refuse). +func newResetDataer(homeDir string, rpcUp bool) *ResetDataer { + return &ResetDataer{ + homeDir: homeDir, + probeUp: func(context.Context) bool { return rpcUp }, + } +} + +// seedHome lays out a realistic home root: data/ with chain files, config/ with +// identity, the sidecar task ledger, and the state-sync marker. +func seedHome(t *testing.T) string { + t.Helper() + home := t.TempDir() + + mustWrite(t, filepath.Join(home, "data", "blockstore.db", "000001.log"), "blocks") + mustWrite(t, filepath.Join(home, "data", "application.db", "CURRENT"), "app") + mustWrite(t, filepath.Join(home, "data", privValidatorStateFile), `{"height":"987","round":0,"step":3}`) + mustWrite(t, filepath.Join(home, "config", "config.toml"), "cfg") + mustWrite(t, filepath.Join(home, "config", "node_key.json"), "nodekey") + mustWrite(t, filepath.Join(home, "config", "priv_validator_key.json"), "conskey") + mustWrite(t, filepath.Join(home, "sidecar.db"), "ledger") + mustWrite(t, filepath.Join(home, stateSyncMarkerFile), "") + return home +} + +func mustWrite(t *testing.T, path, content string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o750); err != nil { + t.Fatalf("mkdir %s: %v", filepath.Dir(path), err) + } + if err := os.WriteFile(path, []byte(content), 0o600); err != nil { + t.Fatalf("write %s: %v", path, err) + } +} + +func exists(path string) bool { + _, err := os.Stat(path) + return err == nil +} + +func runReset(t *testing.T, d *ResetDataer) ResetDataResult { + t.Helper() + raw, err := d.Handler()(context.Background(), nil) + if err != nil { + t.Fatalf("reset-data: %v", err) + } + var res ResetDataResult + if len(raw) > 0 { + if err := json.Unmarshal(raw, &res); err != nil { + t.Fatalf("decoding result: %v", err) + } + } + return res +} + +func TestResetData_WipesDataPreservesHomeRoot(t *testing.T) { + home := seedHome(t) + res := runReset(t, newResetDataer(home, false)) + + // data/ chain files gone. + if exists(filepath.Join(home, "data", "blockstore.db")) { + t.Error("blockstore.db survived the wipe") + } + if exists(filepath.Join(home, "data", "application.db")) { + t.Error("application.db survived the wipe") + } + // Home-root siblings untouched — the design's core correctness rule. + for _, p := range []string{ + filepath.Join(home, "config", "config.toml"), + filepath.Join(home, "config", "node_key.json"), + filepath.Join(home, "config", "priv_validator_key.json"), + filepath.Join(home, "sidecar.db"), + } { + if !exists(p) { + t.Errorf("home-root file destroyed by wipe: %s", p) + } + } + if res.WipedBytes == 0 { + t.Error("expected non-zero wipedBytes for a seeded data dir") + } +} + +func TestResetData_RemovesStateSyncMarker(t *testing.T) { + home := seedHome(t) + runReset(t, newResetDataer(home, false)) + if exists(filepath.Join(home, stateSyncMarkerFile)) { + t.Error("state-sync marker survived the reset") + } +} + +func TestResetData_WritesEmptyPrivValidatorState(t *testing.T) { + home := seedHome(t) + runReset(t, newResetDataer(home, false)) + + got, err := os.ReadFile(filepath.Join(home, "data", privValidatorStateFile)) + if err != nil { + t.Fatalf("reading priv_validator_state: %v", err) + } + if string(got) != emptyPrivValidatorState { + t.Errorf("priv_validator_state = %q, want %q", got, emptyPrivValidatorState) + } +} + +func TestResetData_IdempotentOverAlreadyWiped(t *testing.T) { + home := seedHome(t) + runReset(t, newResetDataer(home, false)) + + // Second run over the already-wiped dir: success, and the sign-state is + // the same fresh-empty content. + res := runReset(t, newResetDataer(home, false)) + got, err := os.ReadFile(filepath.Join(home, "data", privValidatorStateFile)) + if err != nil { + t.Fatalf("reading priv_validator_state after re-run: %v", err) + } + if string(got) != emptyPrivValidatorState { + t.Errorf("priv_validator_state after re-run = %q, want %q", got, emptyPrivValidatorState) + } + // Only the small state file remains, so the second wipe measures little. + if res.WipedBytes >= int64(len(emptyPrivValidatorState))+64 { + t.Errorf("second-run wipedBytes unexpectedly large: %d", res.WipedBytes) + } +} + +func TestResetData_MissingDataDirIsSuccess(t *testing.T) { + home := t.TempDir() // no data/ at all + res := runReset(t, newResetDataer(home, false)) + if res.WipedBytes != 0 { + t.Errorf("expected 0 wipedBytes for absent data dir, got %d", res.WipedBytes) + } + if !exists(filepath.Join(home, "data", privValidatorStateFile)) { + t.Error("expected a fresh priv_validator_state to be created") + } +} + +func TestResetData_MeasurementFailureYieldsUnknownSize(t *testing.T) { + home := seedHome(t) + d := newResetDataer(home, false) + d.measure = func(string) (int64, error) { return 0, fmt.Errorf("simulated measurement failure") } + + res := runReset(t, d) + + if res.WipedBytes != -1 { + t.Errorf("WipedBytes = %d, want -1 (unknown) on measurement failure", res.WipedBytes) + } + // Measurement must not gate the wipe: the reset still cleared data/ and + // wrote a fresh sign-state. + if exists(filepath.Join(home, "data", "blockstore.db")) { + t.Error("data not wiped after measurement failure — measurement gated the reset") + } + if !exists(filepath.Join(home, "data", privValidatorStateFile)) { + t.Error("fresh priv_validator_state not written after measurement failure") + } +} + +func TestResetData_RefusesWhenRPCServing(t *testing.T) { + home := seedHome(t) + _, err := newResetDataer(home, true).Handler()(context.Background(), nil) + if err == nil { + t.Fatal("expected refusal when seid RPC is serving") + } + // Data must be untouched on refusal. + if !exists(filepath.Join(home, "data", "blockstore.db")) { + t.Error("blockstore.db was wiped despite RPC-serving refusal") + } +} diff --git a/sidecar/tasks/restart_seid.go b/sidecar/tasks/restart_seid.go new file mode 100644 index 00000000..74b86b5a --- /dev/null +++ b/sidecar/tasks/restart_seid.go @@ -0,0 +1,201 @@ +package tasks + +import ( + "context" + "fmt" + "os" + "strconv" + "strings" + "syscall" + "time" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/actions" + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +var restartSeidLog = seilog.NewLogger("seictl", "task", "restart-seid") + +const ( + // restartSeidProcess is the comm/argv[0] of the validator process. + restartSeidProcess = "seid" + + // restartSeidGracePeriod bounds the SIGTERM→exit window. A loaded validator's + // graceful shutdown (consensus WAL flush + PebbleDB/IAVL close, possibly + // mid-compaction) can run well past the idle-only ~3s figure, so the window + // is sized for the loaded-shutdown tail. If seid is still alive at the + // deadline the task fails loud and leaves the process running for an + // operator — it is never force-killed (a stuck-but-alive validator is safer + // than a SIGKILL mid-commit). + restartSeidGracePeriod = 90 * time.Second + + // restartSeidUpTimeout bounds the wait for seid's local RPC to serve + // /status again after the restart. Cold-start replay can be slow on a + // loaded node; the engine has no retry, so this is the full budget. With + // graceful-only shutdown there is no SIGKILL-induced replay blowup, so 5m + // holds; revisit for very large archive nodes if replay outgrows it. + restartSeidUpTimeout = 5 * time.Minute + + restartSeidUpPollInterval = 1 * time.Second + + // restartSeidExitPollInterval is how often gracefulStop checks whether seid + // has exited after SIGTERM. + restartSeidExitPollInterval = 100 * time.Millisecond +) + +// seidStartFinder scans /proc for the running `seid start` process. It +// corroborates argv[0]==seid with the "start" subcommand so it never matches +// seid-init or the bash wait-loop wrapper that share the PID namespace. +// +// It implements actions.ProcessSignaler so stopSeid can SIGTERM + poll it; +// FindPID ignores the name argument (the corroboration is baked in) and Signal +// / Alive delegate to the real syscall-backed package functions. +type seidStartFinder struct{} + +func (seidStartFinder) FindPID(string) (int, error) { + entries, err := os.ReadDir("/proc") + if err != nil { + return 0, fmt.Errorf("reading /proc: %w", err) + } + for _, e := range entries { + if !e.IsDir() { + continue + } + pid, err := strconv.Atoi(e.Name()) + if err != nil { + continue + } + comm, err := os.ReadFile(fmt.Sprintf("/proc/%d/comm", pid)) + if err != nil || strings.TrimSpace(string(comm)) != restartSeidProcess { + continue + } + cmdline, err := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", pid)) + if err != nil { + continue + } + if isSeidStart(cmdline) { + return pid, nil + } + } + return 0, fmt.Errorf("process %q not found in /proc", restartSeidProcess) +} + +func (seidStartFinder) Signal(pid int, sig syscall.Signal) error { return actions.SignalPID(pid, sig) } + +func (seidStartFinder) Alive(pid int) bool { return actions.PIDAlive(pid) } + +// isSeidStart reports whether a null-delimited /proc cmdline is `seid start ...`, +// matching both a bare "seid" and an absolute path ending in "/seid". +func isSeidStart(cmdline []byte) bool { + args := strings.Split(strings.TrimRight(string(cmdline), "\x00"), "\x00") + if len(args) < 2 { + return false + } + exe := args[0] + if exe != restartSeidProcess && !strings.HasSuffix(exe, "/"+restartSeidProcess) { + return false + } + return args[1] == "start" +} + +// RestartSeider restarts the co-located seid process in place: it SIGTERMs seid +// and waits for it to exit gracefully (the kubelet restarts the container), then +// waits for seid's local RPC to serve again. seid re-reads config.toml on this +// restart without bouncing the sidecar. The handler never starts seid and never +// flips the engine ready flag — it is not a readiness operation. +// +// Shutdown is graceful-only and fail-loud: if seid does not exit within the +// grace window it is left running and the task fails (never SIGKILLed). A +// force-kill opt-in is intentionally omitted until a non-validator forced +// restart needs it. +// +// Completion means "seid's RPC is serving /status again," NOT "caught up / +// voting." Callers that need in-service-and-voting must gate height / caught-up +// separately (downstream AwaitNodesAtHeight). +// +// The three OS interactions are injectable for testing: +// - signaler: process discovery + SIGTERM (defaults to a /proc + syscall +// implementation that corroborates `seid start`). +// - probeUp: returns true once seid's local RPC answers /status (defaults to +// a local CometBFT /status probe). +type RestartSeider struct { + signaler actions.ProcessSignaler + probeUp func(ctx context.Context) bool + gracePeriod time.Duration + upTimeout time.Duration + upInterval time.Duration +} + +// NewRestartSeider builds a RestartSeider with the real /proc + syscall + +// local-RPC implementations. +func NewRestartSeider() *RestartSeider { + statusClient := rpc.NewStatusClient("", nil) + return &RestartSeider{ + signaler: seidStartFinder{}, + probeUp: func(ctx context.Context) bool { return seidRPCUp(ctx, statusClient) }, + gracePeriod: restartSeidGracePeriod, + upTimeout: restartSeidUpTimeout, + upInterval: restartSeidUpPollInterval, + } +} + +// seidRPCUp reports whether seid's local RPC answers /status (latest_block_height +// parses); any successful parse counts as RPC up. A transport or parse error +// (RPC not yet listening) returns false. +func seidRPCUp(ctx context.Context, c *rpc.StatusClient) bool { + if _, err := c.Status(ctx); err != nil { + return false + } + return true +} + +// Handler returns an engine.TaskHandler for the restart-seid task type. +// Params are empty: restart-seid is a fire-and-confirm operation. +func (r *RestartSeider) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, _ struct{}) error { + if err := r.stopSeid(ctx); err != nil { + return err + } + return r.waitForUp(ctx) + }) +} + +// stopSeid SIGTERMs seid and waits for it to exit gracefully via the shared +// seidStopper (graceful-only, never SIGKILL; honesty check when /proc shows +// nothing but the RPC serves). restart-seid proceeds to waitForUp afterwards. +func (r *RestartSeider) stopSeid(ctx context.Context) error { + return seidStopper{ + signaler: r.signaler, + probeUp: r.probeUp, + gracePeriod: r.gracePeriod, + exitPollInterval: restartSeidExitPollInterval, + log: restartSeidLog, + op: "restart", + }.stop(ctx) +} + +// waitForUp polls seid's local RPC until it serves /status or the timeout +// elapses. Success here is the completion signal for the in-place restart. +func (r *RestartSeider) waitForUp(ctx context.Context) error { + deadline := time.Now().Add(r.upTimeout) + ticker := time.NewTicker(r.upInterval) + defer ticker.Stop() + + restartSeidLog.Info("waiting for seid RPC to come back up", "timeout", r.upTimeout) + for { + if r.probeUp(ctx) { + restartSeidLog.Info("seid RPC is up; restart complete") + return nil + } + select { + case <-ctx.Done(): + return ctx.Err() + case <-ticker.C: + if time.Now().After(deadline) { + return fmt.Errorf("seid RPC did not come up within %s after restart", r.upTimeout) + } + } + } +} diff --git a/sidecar/tasks/restart_seid_test.go b/sidecar/tasks/restart_seid_test.go new file mode 100644 index 00000000..e451e4fd --- /dev/null +++ b/sidecar/tasks/restart_seid_test.go @@ -0,0 +1,192 @@ +package tasks + +import ( + "context" + "errors" + "fmt" + "strings" + "sync/atomic" + "syscall" + "testing" + "time" +) + +// fakeSignaler implements actions.ProcessSignaler for restart-seid tests. +type fakeSignaler struct { + findPID int + findErr error + alive atomic.Bool + signals []syscall.Signal + signalFn func(pid int, sig syscall.Signal) error +} + +func (f *fakeSignaler) FindPID(string) (int, error) { return f.findPID, f.findErr } + +func (f *fakeSignaler) Signal(pid int, sig syscall.Signal) error { + f.signals = append(f.signals, sig) + if f.signalFn != nil { + return f.signalFn(pid, sig) + } + return nil +} + +func (f *fakeSignaler) Alive(int) bool { return f.alive.Load() } + +// upAfter returns a probe that reports down for the first n calls, then up. +func upAfter(n int) func(context.Context) bool { + var calls int32 + return func(context.Context) bool { + return atomic.AddInt32(&calls, 1) > int32(n) + } +} + +func neverUp(context.Context) bool { return false } + +func TestRestartSeider_HappyPath(t *testing.T) { + sig := &fakeSignaler{findPID: 42} + sig.alive.Store(false) // exits immediately after SIGTERM + + r := &RestartSeider{ + signaler: sig, + probeUp: upAfter(0), // up on first probe + gracePeriod: time.Second, + upTimeout: time.Second, + upInterval: time.Millisecond, + } + + if _, err := r.Handler()(context.Background(), nil); err != nil { + t.Fatalf("expected success, got %v", err) + } + if len(sig.signals) != 1 || sig.signals[0] != syscall.SIGTERM { + t.Errorf("expected single SIGTERM, got %v", sig.signals) + } +} + +func TestRestartSeider_GraceTimeoutFailsWithoutSIGKILL(t *testing.T) { + sig := &fakeSignaler{findPID: 42} + sig.alive.Store(true) // never exits on SIGTERM + + r := &RestartSeider{ + signaler: sig, + probeUp: upAfter(0), + gracePeriod: 50 * time.Millisecond, + upTimeout: time.Second, + upInterval: time.Millisecond, + } + + _, err := r.Handler()(context.Background(), nil) + if err == nil { + t.Fatal("expected grace-timeout failure, got nil") + } + if !strings.Contains(err.Error(), "still alive") { + t.Errorf("expected still-alive error, got %v", err) + } + if len(sig.signals) != 1 || sig.signals[0] != syscall.SIGTERM { + t.Errorf("expected single SIGTERM and no SIGKILL, got %v", sig.signals) + } +} + +func TestRestartSeider_NotFoundRPCDownWaitsForUp(t *testing.T) { + sig := &fakeSignaler{findErr: fmt.Errorf("process \"seid\" not found in /proc")} + + r := &RestartSeider{ + signaler: sig, + probeUp: upAfter(2), // RPC down (not-found guard + first poll), then up + gracePeriod: time.Second, + upTimeout: time.Second, + upInterval: time.Millisecond, + } + + if _, err := r.Handler()(context.Background(), nil); err != nil { + t.Fatalf("expected success when seid not found and RPC down, got %v", err) + } + if len(sig.signals) != 0 { + t.Errorf("expected no signals when seid not running, got %v", sig.signals) + } +} + +func TestRestartSeider_NotFoundRPCUpFails(t *testing.T) { + sig := &fakeSignaler{findErr: fmt.Errorf("process \"seid\" not found in /proc")} + + r := &RestartSeider{ + signaler: sig, + probeUp: upAfter(0), // RPC serving despite no /proc match + gracePeriod: time.Second, + upTimeout: time.Second, + upInterval: time.Millisecond, + } + + _, err := r.Handler()(context.Background(), nil) + if err == nil { + t.Fatal("expected error when RPC up but process not found, got nil") + } + if !strings.Contains(err.Error(), "not found in /proc") { + t.Errorf("expected not-found-in-/proc error, got %v", err) + } + if len(sig.signals) != 0 { + t.Errorf("expected no signals, got %v", sig.signals) + } +} + +func TestRestartSeider_WaitForUpContextCancelled(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + r := &RestartSeider{ + signaler: &fakeSignaler{findErr: fmt.Errorf("process \"seid\" not found in /proc")}, + probeUp: neverUp, + gracePeriod: time.Second, + upTimeout: time.Minute, + upInterval: 10 * time.Millisecond, + } + + go func() { + time.Sleep(20 * time.Millisecond) + cancel() + }() + + err := r.waitForUp(ctx) + if !errors.Is(err, context.Canceled) { + t.Fatalf("expected context.Canceled, got %v", err) + } +} + +func TestRestartSeider_RPCNeverUpTimesOut(t *testing.T) { + sig := &fakeSignaler{findPID: 42} + sig.alive.Store(false) + + r := &RestartSeider{ + signaler: sig, + probeUp: neverUp, + gracePeriod: time.Second, + upTimeout: 50 * time.Millisecond, + upInterval: time.Millisecond, + } + + _, err := r.Handler()(context.Background(), nil) + if err == nil { + t.Fatal("expected timeout error, got nil") + } +} + +func TestIsSeidStart(t *testing.T) { + tests := []struct { + name string + cmdline []byte + want bool + }{ + {"bare seid start", []byte("seid\x00start\x00--home\x00/.sei"), true}, + {"absolute path seid start", []byte("/usr/bin/seid\x00start"), true}, + {"trailing null", []byte("seid\x00start\x00"), true}, + {"seid non-start subcommand", []byte("seid\x00version"), false}, + {"seid-init", []byte("seid-init\x00start"), false}, + {"bash wrapper", []byte("bash\x00-c\x00seid start"), false}, + {"seid no args", []byte("seid"), false}, + {"empty", []byte{}, false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := isSeidStart(tt.cmdline); got != tt.want { + t.Errorf("isSeidStart(%q) = %v, want %v", tt.cmdline, got, tt.want) + } + }) + } +} diff --git a/sidecar/tasks/result_compare.go b/sidecar/tasks/result_compare.go new file mode 100644 index 00000000..20400800 --- /dev/null +++ b/sidecar/tasks/result_compare.go @@ -0,0 +1,346 @@ +package tasks + +import ( + "context" + "errors" + "fmt" + "time" + + "github.com/ethereum/go-ethereum/ethclient" + + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" + "github.com/sei-protocol/sei-k8s-controller/sidecar/shadow" +) + +const ( + comparePollInterval = 5 * time.Second + comparePageSize = 100 + + // finalFlushTimeout bounds the best-effort flush of the trailing compare + // page when a survey is stopped. The loop's context is already cancelled at + // that point, so the flush runs on a fresh deadline; the pod's termination + // grace period must accommodate it. + finalFlushTimeout = 10 * time.Second +) + +// Guard the external contract Comparator.Close relies on: *ethclient.Client must +// expose a no-return Close(). If a go-ethereum upgrade changed it to Close() +// error, the no-return assertion in Comparator.Close would silently skip it +// (the leak we already fixed once) — this fails the build instead. +var _ interface{ Close() } = (*ethclient.Client)(nil) + +// comparisonLoop holds the state for a running block comparison session. +type comparisonLoop struct { + exporter *ResultExporter + comparator *shadow.Comparator + uploader seis3.Uploader + cfg ResultExportRequest + prefix string + height int64 + pageBuf []shadow.CompareResult + pollInterval time.Duration +} + +// ExportAndCompare runs a continuous comparison between the local shadow node +// and a canonical chain. +// +// By default it completes successfully on the first divergence, uploading a +// DivergenceReport alongside the comparison pages. In survey mode +// (cfg.ContinueOnDivergence) a divergence never halts the run: the comparison +// tails the chain until the context is cancelled, and a clean cancellation +// completes the task — being stopped is the survey's natural end, not a failure. +func (e *ResultExporter) ExportAndCompare(ctx context.Context, cfg ResultExportRequest) error { + loop, err := e.newComparisonLoop(ctx, cfg) + if err != nil { + return err + } + defer loop.comparator.Close() + + exportLog.Info("starting block comparison", + "start-height", loop.height, + "canonical-rpc", cfg.CanonicalRPC, + "bucket", cfg.Bucket) + + return loop.run(ctx) +} + +func (e *ResultExporter) newComparisonLoop(ctx context.Context, cfg ResultExportRequest) (*comparisonLoop, error) { + uploader, err := e.s3UploaderFactory(ctx, cfg.Region) + if err != nil { + return nil, fmt.Errorf("building S3 uploader: %w", err) + } + + var compOpts []shadow.Option + if cfg.MigrationMode { + compOpts = append(compOpts, shadow.WithMigrationMode()) + } + + // Layer 2 (logical state diff) is enabled when both EVM JSON-RPC endpoints + // are configured. Touched keys come from a prestate trace on TraceRPC + // (defaults to the canonical endpoint). + if cfg.ShadowEVMRPC != "" && cfg.CanonicalEVMRPC != "" { + shadowState, err := ethclient.Dial(cfg.ShadowEVMRPC) + if err != nil { + return nil, fmt.Errorf("dialing shadow EVM RPC: %w", err) + } + canonicalState, err := ethclient.Dial(cfg.CanonicalEVMRPC) + if err != nil { + shadowState.Close() + return nil, fmt.Errorf("dialing canonical EVM RPC: %w", err) + } + traceRPC := cfg.TraceRPC + if traceRPC == "" { + traceRPC = cfg.CanonicalEVMRPC + } + keySource, err := shadow.NewTraceKeySource(traceRPC) + if err != nil { + shadowState.Close() + canonicalState.Close() + return nil, fmt.Errorf("building trace key source: %w", err) + } + compOpts = append(compOpts, shadow.WithLayer2(shadowState, canonicalState, keySource)) + } + + last := e.readExportState() + return &comparisonLoop{ + exporter: e, + comparator: shadow.NewComparator(cfg.RPCEndpoint, cfg.CanonicalRPC, compOpts...), + uploader: uploader, + cfg: cfg, + prefix: normalizePrefix(cfg.Prefix), + height: last.LastExportedHeight + 1, + pollInterval: comparePollInterval, + }, nil +} + +func (l *comparisonLoop) run(ctx context.Context) error { + for { + if err := ctx.Err(); err != nil { + return l.finalize(err) + } + + latestHeight, err := l.waitForBlocks(ctx) + if err != nil { + return l.finalize(err) + } + + diverged, err := l.compareBlocksUpTo(ctx, latestHeight) + if err != nil { + return l.finalize(err) + } + if diverged { + return nil + } + } +} + +// finalize maps the loop's exit error to a task verdict. Survey mode has no +// divergence-halt, so being stopped is its natural end: a clean context +// cancellation (e.g. sidecar shutdown) completes the task rather than failing +// it. Any other error — and any error in the default halt-on-divergence mode — +// propagates and fails the run. +// +// Before completing, the trailing partial page (blocks compared since the last +// full-page boundary) is flushed — otherwise those blocks would be silently +// absent from S3 while the task reports success, and a completed task is not +// re-run. If that final flush fails the run fails instead, so a restart +// re-surveys the trailing blocks from the last persisted height. +func (l *comparisonLoop) finalize(err error) error { + if l.cfg.ContinueOnDivergence && errors.Is(err, context.Canceled) { + flushCtx, cancel := context.WithTimeout(context.Background(), finalFlushTimeout) + defer cancel() + if ferr := l.flushFinalPage(flushCtx); ferr != nil { + return fmt.Errorf("flushing final survey page on shutdown: %w", ferr) + } + exportLog.Info("survey stopped; completing", "last-height", l.height) + return nil + } + return err +} + +// flushFinalPage uploads whatever remains in the page buffer (a partial page +// below comparePageSize) and persists the height of its last block, so a +// stopped survey loses no compared blocks. It keys the persisted height off the +// buffer's last entry rather than l.height, which has already advanced past it. +func (l *comparisonLoop) flushFinalPage(ctx context.Context) error { + if len(l.pageBuf) == 0 { + return nil + } + lastHeight := l.pageBuf[len(l.pageBuf)-1].Height + if err := flushComparePage(ctx, l.uploader, l.cfg.Bucket, l.prefix, l.pageBuf); err != nil { + return err + } + l.exporter.persistHeight(lastHeight) + l.pageBuf = l.pageBuf[:0] + return nil +} + +func (l *comparisonLoop) waitForBlocks(ctx context.Context) (int64, error) { + for { + latestHeight, err := queryLatestHeight(ctx, l.cfg.RPCEndpoint) + if err != nil { + exportLog.Debug("shadow RPC unavailable, will retry", "err", err) + if err := sleep(ctx, l.pollInterval); err != nil { + return 0, err + } + continue + } + + if l.height <= latestHeight { + return latestHeight, nil + } + + if err := sleep(ctx, l.pollInterval); err != nil { + return 0, err + } + } +} + +func (l *comparisonLoop) compareBlocksUpTo(ctx context.Context, latestHeight int64) (diverged bool, _ error) { + for l.height <= latestHeight { + if err := ctx.Err(); err != nil { + return false, err + } + + result, err := l.comparator.CompareBlock(ctx, l.height) + if err != nil { + exportLog.Warn("comparison failed, will retry", "height", l.height, "err", err) + return false, sleep(ctx, l.pollInterval) + } + + shadow.BlocksCompared.WithLabelValues(l.exporter.chainID, l.exporter.podName).Inc() + l.pageBuf = append(l.pageBuf, *result) + + if result.Diverged() { + if !l.cfg.ContinueOnDivergence { + return true, l.handleDivergence(ctx, *result) + } + // Survey mode: the divergent block is already appended to the compare + // page (its authentic per-block verdict); record it and fall through to + // the normal page-flush + height++ discipline. Unlike handleDivergence we + // upload NO per-block DivergenceReport — that re-fetches block + + // block_results from both chains and writes an S3 object per block, which + // would overload the sidecar over a multi-million-block sweep. The page + // (flushed and truncated at comparePageSize, so memory stays bounded) is + // the survey record; seictl report classifies benign vs real downstream. + l.recordDivergence(*result) + } + + if err := l.flushPageIfFull(ctx); err != nil { + return false, err + } + + l.height++ + } + return false, nil +} + +func (l *comparisonLoop) handleDivergence(ctx context.Context, result shadow.CompareResult) error { + layer := l.incDivergenceMetric(result) + + exportLog.Info("app-hash divergence detected", + "height", l.height, + "divergence-layer", layer, + "shadow-app-hash", result.Layer0.ShadowAppHash, + "canonical-app-hash", result.Layer0.CanonicalAppHash) + + if err := l.uploadDivergenceReport(ctx, result); err != nil { + exportLog.Warn("failed to upload divergence report, continuing", "err", err) + } + + if err := flushComparePage(ctx, l.uploader, l.cfg.Bucket, l.prefix, l.pageBuf); err != nil { + return fmt.Errorf("flushing final comparison page: %w", err) + } + + l.exporter.persistHeight(l.height) + return nil +} + +// recordDivergence notes a divergent block under ContinueOnDivergence (survey +// mode): it increments the divergence metric and logs, but — unlike +// handleDivergence — uploads no per-block DivergenceReport and forces no early +// page flush. The block is already in the page; it rides the normal +// flushPageIfFull boundary, so the in-memory buffer stays bounded over a long +// sweep and the run continues instead of halting. +func (l *comparisonLoop) recordDivergence(result shadow.CompareResult) { + layer := l.incDivergenceMetric(result) + exportLog.Info("divergence recorded; continuing (survey mode)", + "height", l.height, "divergence-layer", layer) +} + +// incDivergenceMetric increments the divergence counter under the result's +// layer label and returns that label. The label encoding (nil DivergenceLayer +// → "0", else the layer number) is one contract shared by both the +// halt-on-divergence and survey-mode paths, so it lives in a single place. +func (l *comparisonLoop) incDivergenceMetric(result shadow.CompareResult) string { + layer := "0" + if result.DivergenceLayer != nil { + layer = fmt.Sprintf("%d", *result.DivergenceLayer) + } + shadow.Divergences.WithLabelValues(l.exporter.chainID, l.exporter.podName, layer).Inc() + return layer +} + +func (l *comparisonLoop) uploadDivergenceReport(ctx context.Context, result shadow.CompareResult) error { + report, err := l.comparator.BuildDivergenceReport(ctx, l.height, result) + if err != nil { + return fmt.Errorf("building divergence report: %w", err) + } + + key := fmt.Sprintf("%sdivergence-%d.report.json.gz", l.prefix, l.height) + _, err = seis3.StreamGzipJSON(ctx, l.uploader, l.cfg.Bucket, key, report) + return err +} + +func (l *comparisonLoop) flushPageIfFull(ctx context.Context) error { + if len(l.pageBuf) < comparePageSize { + return nil + } + + // A flush failure ends the run rather than continuing: the uploader (AWS SDK) + // already retries transient faults, so an error here is a real failure, and + // swallowing it would leave pageBuf un-truncated and growing every iteration + // — unbounded under a persistent S3 fault on a never-halting survey. Failing + // lets the task restart and resume from the last persisted (flushed) height. + if err := flushComparePage(ctx, l.uploader, l.cfg.Bucket, l.prefix, l.pageBuf); err != nil { + return fmt.Errorf("flushing comparison page: %w", err) + } + + l.exporter.persistHeight(l.height) + l.pageBuf = l.pageBuf[:0] + return nil +} + +// persistHeight saves the last exported height to disk, logging on failure. +func (e *ResultExporter) persistHeight(height int64) { + if err := e.writeExportState(exportState{LastExportedHeight: height}); err != nil { + exportLog.Warn("failed to persist export state", "err", err) + } +} + +// --- S3 upload helpers --- + +func flushComparePage(ctx context.Context, uploader seis3.Uploader, bucket, prefix string, results []shadow.CompareResult) error { + if len(results) == 0 { + return nil + } + + start := results[0].Height + end := results[len(results)-1].Height + key := fmt.Sprintf("%s%d-%d.compare.ndjson.gz", prefix, start, end) + + exportLog.Info("flushing comparison page", "key", key, "blocks", len(results)) + _, err := seis3.StreamGzipNDJSON(ctx, uploader, bucket, key, results) + return err +} + +func sleep(ctx context.Context, d time.Duration) error { + t := time.NewTimer(d) + defer t.Stop() + select { + case <-ctx.Done(): + return ctx.Err() + case <-t.C: + return nil + } +} diff --git a/sidecar/tasks/result_compare_test.go b/sidecar/tasks/result_compare_test.go new file mode 100644 index 00000000..cf807daf --- /dev/null +++ b/sidecar/tasks/result_compare_test.go @@ -0,0 +1,208 @@ +package tasks + +import ( + "context" + "errors" + "strings" + "testing" + "time" + + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" + "github.com/sei-protocol/sei-k8s-controller/sidecar/shadow" +) + +// newTestComparisonLoop wires a comparison loop against two RPC servers whose +// blocks diverge at Layer 0 (different app_hash, matching results, no per-tx +// detail) so every compared block is a divergence, with a recording uploader. +// continueOnDivergence selects survey mode vs the default halt-on-first; latest +// is the height the servers report via /status (drives waitForBlocks in run()). +func newTestComparisonLoop(t *testing.T, continueOnDivergence bool, latest int64) (*comparisonLoop, *recordingUploader) { + t.Helper() + shadowSrv := fakeRPCAndBlockServer(latest, "SHADOW", "RES", nil) + canonicalSrv := fakeRPCAndBlockServer(latest, "CANONICAL", "RES", nil) + t.Cleanup(shadowSrv.Close) + t.Cleanup(canonicalSrv.Close) + + rec := &recordingUploader{} + exporter := NewResultExporter(t.TempDir(), "test-chain", "pod-0", func(_ context.Context, _ string) (seis3.Uploader, error) { + return rec, nil + }) + return &comparisonLoop{ + exporter: exporter, + comparator: shadow.NewComparator(shadowSrv.URL, canonicalSrv.URL), + uploader: rec, + cfg: ResultExportRequest{ + Bucket: "bkt", Region: "us-east-1", Prefix: "p/", + RPCEndpoint: shadowSrv.URL, + CanonicalRPC: canonicalSrv.URL, + ContinueOnDivergence: continueOnDivergence, + }, + prefix: "p/", + height: 1, + pollInterval: time.Millisecond, + }, rec +} + +func countKeys(rec *recordingUploader, substr string) int { + n := 0 + for _, k := range rec.keys { + if strings.Contains(k, substr) { + n++ + } + } + return n +} + +// TestCompareDefaultMode_HaltsOnDivergence: with ContinueOnDivergence unset the +// first divergent block trips the loop — it reports diverged, uploads a +// per-block divergence report, and does not advance past the divergent height. +func TestCompareDefaultMode_HaltsOnDivergence(t *testing.T) { + loop, rec := newTestComparisonLoop(t, false, 5) + + diverged, err := loop.compareBlocksUpTo(context.Background(), 5) + if err != nil { + t.Fatalf("compareBlocksUpTo: %v", err) + } + if !diverged { + t.Fatal("default mode must halt on the first divergence") + } + if loop.height != 1 { + t.Errorf("height advanced to %d; default-mode halt must not step past the divergent block", loop.height) + } + if n := countKeys(rec, "divergence-1.report"); n != 1 { + t.Errorf("expected exactly one per-block divergence report, got %d", n) + } +} + +// TestCompareSurveyMode_ContinuesPastDivergence: with ContinueOnDivergence set +// the loop surveys every divergent block to the end of the range, advances the +// height, and uploads NO per-block divergence report (the page is the record). +func TestCompareSurveyMode_ContinuesPastDivergence(t *testing.T) { + loop, rec := newTestComparisonLoop(t, true, 5) + + diverged, err := loop.compareBlocksUpTo(context.Background(), 5) + if err != nil { + t.Fatalf("compareBlocksUpTo: %v", err) + } + if diverged { + t.Fatal("survey mode must not halt on divergence") + } + if loop.height != 6 { + t.Errorf("height = %d, want 6 (surveyed every block past the divergences)", loop.height) + } + if n := countKeys(rec, ".report"); n != 0 { + t.Errorf("survey mode must upload no per-block divergence report, got %d", n) + } +} + +// TestCompareSurveyMode_BoundsMemory: surveying past comparePageSize divergent +// blocks must flush-and-truncate the in-memory page rather than accumulate +// every result — the bound that keeps a multi-million-block sweep from +// exhausting the sidecar. Two full pages flush; the buffer holds only the +// trailing remainder. +func TestCompareSurveyMode_BoundsMemory(t *testing.T) { + loop, rec := newTestComparisonLoop(t, true, 2*comparePageSize+50) + + const blocks = 2*comparePageSize + 50 + diverged, err := loop.compareBlocksUpTo(context.Background(), blocks) + if err != nil { + t.Fatalf("compareBlocksUpTo: %v", err) + } + if diverged { + t.Fatal("survey mode must not halt on divergence") + } + if len(loop.pageBuf) >= comparePageSize { + t.Errorf("pageBuf holds %d results; survey mode must flush+truncate at comparePageSize (%d) to bound memory", len(loop.pageBuf), comparePageSize) + } + if len(loop.pageBuf) != 50 { + t.Errorf("pageBuf = %d, want 50 (the remainder after two full-page flushes)", len(loop.pageBuf)) + } + if n := countKeys(rec, ".compare.ndjson.gz"); n != 2 { + t.Errorf("expected 2 flushed compare pages, got %d", n) + } + if n := countKeys(rec, ".report"); n != 0 { + t.Errorf("survey mode must upload no per-block divergence report, got %d", n) + } +} + +// TestCompareSurveyMode_RunCompletesOnCancel: survey mode never returns +// diverged=true, so run() only exits when stopped. A clean context cancellation +// is the survey's natural end and must complete the task (return nil), not fail it. +func TestCompareSurveyMode_RunCompletesOnCancel(t *testing.T) { + loop, _ := newTestComparisonLoop(t, true, 3) + + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan error, 1) + go func() { done <- loop.run(ctx) }() + + // Give the survey a moment to process the few available blocks and begin + // tailing, then stop it the way a sidecar shutdown would. (Reading loop + // state here would race run()'s goroutine; a cancel completes cleanly + // whether the loop is tailing or mid-survey, so a brief wait is enough — + // the assertion is on the exit verdict, not on catch-up.) + time.Sleep(20 * time.Millisecond) + cancel() + + select { + case err := <-done: + if err != nil { + t.Fatalf("survey run on clean cancel = %v, want nil (a stop completes, not fails)", err) + } + case <-time.After(2 * time.Second): + t.Fatal("run did not return after context cancellation") + } +} + +// TestCompareSurveyMode_FlushesTrailingPageOnStop: when a survey is stopped +// with a partial page buffered (fewer than comparePageSize blocks), that page +// must be flushed to S3 before the task completes — otherwise up to +// comparePageSize-1 compared blocks are silently dropped while the task reports +// success. latest < comparePageSize, so the only page is the trailing partial. +func TestCompareSurveyMode_FlushesTrailingPageOnStop(t *testing.T) { + loop, rec := newTestComparisonLoop(t, true, 50) + + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan error, 1) + go func() { done <- loop.run(ctx) }() + + // Survey the available blocks and begin tailing, then stop the survey. + time.Sleep(30 * time.Millisecond) + cancel() + + select { + case err := <-done: + if err != nil { + t.Fatalf("survey run on clean stop = %v, want nil", err) + } + case <-time.After(2 * time.Second): + t.Fatal("run did not return after context cancellation") + } + + // Safe to read rec.keys: the run goroutine's writes happen-before the done + // receive, and it has returned. + if n := countKeys(rec, ".compare.ndjson.gz"); n != 1 { + t.Errorf("trailing compare page not flushed on stop: got %d compare pages, want 1", n) + } + if n := countKeys(rec, ".report"); n != 0 { + t.Errorf("survey mode must upload no per-block divergence report, got %d", n) + } +} + +// TestCompareSurveyMode_FlushFailureFailsBounded: a persistent flush failure on +// a never-halting survey must fail the run (so the task restarts and resumes) +// rather than silently swallow the error and let pageBuf grow unbounded. +func TestCompareSurveyMode_FlushFailureFailsBounded(t *testing.T) { + loop, _ := newTestComparisonLoop(t, true, 5*comparePageSize) + loop.uploader = drainingFailingUploader{err: errors.New("s3 unavailable")} + + diverged, err := loop.compareBlocksUpTo(context.Background(), 5*comparePageSize) + if err == nil { + t.Fatal("a persistent flush failure must fail the run, not be swallowed") + } + if diverged { + t.Fatal("a flush failure is an error, not a divergence-halt") + } + if len(loop.pageBuf) > comparePageSize { + t.Errorf("pageBuf grew to %d past comparePageSize (%d) on flush failure — the buffer must stay bounded", len(loop.pageBuf), comparePageSize) + } +} diff --git a/sidecar/tasks/result_export.go b/sidecar/tasks/result_export.go new file mode 100644 index 00000000..8139da2f --- /dev/null +++ b/sidecar/tasks/result_export.go @@ -0,0 +1,311 @@ +package tasks + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "strconv" + "strings" + "time" + + seiconfig "github.com/sei-protocol/sei-config" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +var exportLog = seilog.NewLogger("seictl", "task", "result-export") + +const ( + exportStateFile = ".sei-sidecar-last-export.json" + defaultPageSize = 1000 +) + +var defaultRPCEndpoint = fmt.Sprintf("http://localhost:%d", seiconfig.PortRPC) + +// errSinkWrite tags a failure writing to the export sink (the gzip/upload pipe), +// distinguishing a downstream/S3 cause from a genuine producer fault (RPC, +// marshaling, ctx). When the sink fails mid-stream the real cause is the upload +// error, so the write error must not be returned as if the producer failed. +var errSinkWrite = errors.New("writing to export sink") + +// ResultExportRequest holds the parameters for the result-export task. +type ResultExportRequest struct { + Bucket string `json:"bucket"` + Prefix string `json:"prefix"` + Region string `json:"region"` + RPCEndpoint string `json:"rpcEndpoint"` + + // CanonicalRPC enables comparison mode. When set, the exporter compares + // local block execution against this canonical RPC endpoint and completes + // when app-hash divergence is detected. + CanonicalRPC string `json:"canonicalRpc"` + + // MigrationMode tunes comparison for an AppHash-breaking migration shadow + // (e.g. memiavl->flatkv): AppHash divergence from canonical is expected + // every block, so it is treated as informational and the verdict keys on + // execution-results equivalence (LastResultsHash + gas + per-tx receipts). + MigrationMode bool `json:"migrationMode,omitempty"` + + // ContinueOnDivergence selects survey mode: the comparison records each + // divergent block to the compare page and keeps going instead of halting on + // the first divergence. Default false preserves the production tripwire. The + // comparator's verdict is unchanged — every field is compared authentically; + // this only decides whether a divergence stops the run. Classifying benign vs + // real divergences is the downstream `seictl report` step's job. Has no + // effect outside comparison mode (it requires CanonicalRPC). + ContinueOnDivergence bool `json:"continueOnDivergence,omitempty"` + + // ShadowEVMRPC and CanonicalEVMRPC are the EVM JSON-RPC endpoints for the + // shadow and canonical chains. When both are set, Layer 2 (logical state + // diff) is enabled, comparing storage/code/nonce for the keys each block + // touched. These are EVM JSON-RPC (eth_*), distinct from the CometBFT RPC + // used for Layers 0/1. + ShadowEVMRPC string `json:"shadowEvmRpc,omitempty"` + CanonicalEVMRPC string `json:"canonicalEvmRpc,omitempty"` + + // TraceRPC is the EVM JSON-RPC endpoint used for prestate traces + // (debug_traceBlockByNumber) to derive each block's touched keys. Defaults + // to CanonicalEVMRPC. Requires the debug_ namespace enabled on that node. + TraceRPC string `json:"traceRpc,omitempty"` +} + +type exportState struct { + LastExportedHeight int64 `json:"lastExportedHeight"` +} + +// ResultExporter queries the local seid RPC for block results and uploads +// them in compressed NDJSON pages to S3. +type ResultExporter struct { + homeDir string + chainID string + podName string + s3UploaderFactory seis3.UploaderFactory +} + +// NewResultExporter creates an exporter targeting the given home directory. +// chainID and podName label shadow comparison metrics; pass empty strings +// if the exporter is only used in non-comparison mode. +func NewResultExporter(homeDir, chainID, podName string, factory seis3.UploaderFactory) *ResultExporter { + if factory == nil { + factory = seis3.DefaultUploaderFactory + } + return &ResultExporter{homeDir: homeDir, chainID: chainID, podName: podName, s3UploaderFactory: factory} +} + +func (e *ResultExporter) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, cfg ResultExportRequest) error { + if cfg.Bucket == "" { + return fmt.Errorf("result-export: missing required param 'bucket'") + } + if cfg.Region == "" { + return fmt.Errorf("result-export: missing required param 'region'") + } + if cfg.RPCEndpoint == "" { + cfg.RPCEndpoint = defaultRPCEndpoint + } + if cfg.CanonicalRPC != "" { + return e.ExportAndCompare(ctx, cfg) + } + return e.Export(ctx, cfg) + }) +} + +// Export queries the local node for block results and uploads pages to S3. +// Each invocation exports as many complete pages as are available since the +// last export height. The state file tracks progress across invocations. +func (e *ResultExporter) Export(ctx context.Context, cfg ResultExportRequest) error { + last := e.readExportState() + startHeight := last.LastExportedHeight + 1 + + latestHeight, err := queryLatestHeight(ctx, cfg.RPCEndpoint) + if err != nil { + exportLog.Info("RPC unavailable, will retry", "err", err) + return nil + } + + if startHeight > latestHeight { + exportLog.Debug("no new blocks to export", + "last-exported", last.LastExportedHeight, + "latest", latestHeight) + return nil + } + + uploader, err := e.s3UploaderFactory(ctx, cfg.Region) + if err != nil { + return fmt.Errorf("building S3 uploader: %w", err) + } + + rpcClient := rpc.NewClient(cfg.RPCEndpoint, nil) + prefix := normalizePrefix(cfg.Prefix) + + availableBlocks := latestHeight - startHeight + 1 + fullPages := int(availableBlocks) / defaultPageSize + + if fullPages == 0 { + exportLog.Debug("not enough blocks for a full page yet", + "available", availableBlocks, + "page-size", defaultPageSize) + return nil + } + + for page := 0; page < fullPages; page++ { + pageStart := startHeight + int64(page*defaultPageSize) + pageEnd := pageStart + int64(defaultPageSize) - 1 + + exportLog.Info("exporting result page", + "start", pageStart, + "end", pageEnd, + "bucket", cfg.Bucket) + + if err := e.exportPage(ctx, rpcClient, uploader, cfg.Bucket, cfg.Region, prefix, pageStart, pageEnd); err != nil { + return fmt.Errorf("exporting page %d-%d: %w", pageStart, pageEnd, err) + } + + if err := e.writeExportState(exportState{LastExportedHeight: pageEnd}); err != nil { + exportLog.Warn("failed to persist export state, deferring to next scheduled run", + "last-exported", pageEnd, "err", err) + return nil + } + } + + lastExported := startHeight + int64(fullPages*defaultPageSize) - 1 + exportLog.Info("export complete", + "pages", fullPages, + "last-exported", lastExported, + "latest-available", latestHeight) + + return nil +} + +// exportPage collects block results for [start, end] and streams a gzipped +// NDJSON file to S3. +func (e *ResultExporter) exportPage( + ctx context.Context, + client *rpc.Client, + uploader seis3.Uploader, + bucket, region, prefix string, + start, end int64, +) error { + key := fmt.Sprintf("%s%d-%d.ndjson.gz", prefix, start, end) + + var collectErr error + _, uploadErr := seis3.StreamGzipFunc(ctx, uploader, bucket, key, func(w io.Writer) error { + collectErr = e.collectResults(ctx, client, w, start, end) + return collectErr + }) + switch { + case collectErr != nil && !errors.Is(collectErr, errSinkWrite): + // Genuine producer fault (block_results RPC, marshaling, ctx cancel) — + // not an S3 problem; return it as-is, never mislabeled with S3 hints. + return collectErr + case uploadErr != nil: + // Upload failed, including mid-stream (which also shows up as a tagged + // errSinkWrite in collectErr). Keep the S3 classification + Retryable hint. + return seis3.ClassifyS3Error("result-export", bucket, key, region, uploadErr) + case collectErr != nil: + // A sink write failed without the upload reporting an error — surface it. + return collectErr + } + return nil +} + +// collectResults queries block_results for each height and writes one NDJSON +// line per block to w. Each line is a JSON object with height, time, and the +// raw block_results response. gzip/pipe/checksum are owned by the s3 helper. +func (e *ResultExporter) collectResults(ctx context.Context, client *rpc.Client, w io.Writer, start, end int64) error { + for h := start; h <= end; h++ { + if err := ctx.Err(); err != nil { + return err + } + + result, err := queryBlockResults(ctx, client, h) + if err != nil { + return fmt.Errorf("querying block_results at height %d: %w", h, err) + } + + record := map[string]any{ + "height": h, + "exported_at": time.Now().UTC().Format(time.RFC3339), + "block_results": result, + } + + line, err := json.Marshal(record) + if err != nil { + return fmt.Errorf("marshaling result at height %d: %w", h, err) + } + line = append(line, '\n') + + if _, err := w.Write(line); err != nil { + return fmt.Errorf("%w at height %d: %w", errSinkWrite, h, err) + } + } + + return nil +} + +func queryLatestHeight(ctx context.Context, rpcEndpoint string) (int64, error) { + c := rpc.NewStatusClient(rpcEndpoint, nil) + h, err := c.LatestHeight(ctx) + if err != nil { + return 0, err + } + if h <= 0 { + return 0, fmt.Errorf("latest_block_height is %d, node may still be syncing", h) + } + return h, nil +} + +func queryBlockResults(ctx context.Context, client *rpc.Client, height int64) (json.RawMessage, error) { + body, err := client.GetRaw(ctx, fmt.Sprintf("/block_results?height=%d", height)) + if err != nil { + return nil, err + } + return json.RawMessage(body), nil +} + +func (e *ResultExporter) readExportState() exportState { + data, err := os.ReadFile(filepath.Join(e.homeDir, exportStateFile)) + if err != nil { + return e.bootstrapExportState() + } + var state exportState + if err := json.Unmarshal(data, &state); err != nil { + return e.bootstrapExportState() + } + return state +} + +// bootstrapExportState reads the snapshot height file written by the restorer +// and uses it as the initial last-exported height, so the exporter begins at +// the first block after the restored snapshot rather than from block 1. +func (e *ResultExporter) bootstrapExportState() exportState { + data, err := os.ReadFile(filepath.Join(e.homeDir, SnapshotHeightFile)) + if err != nil { + return exportState{} + } + h, err := strconv.ParseInt(strings.TrimSpace(string(data)), 10, 64) + if err != nil || h <= 0 { + return exportState{} + } + exportLog.Info("bootstrapping export state from snapshot height", "height", h) + return exportState{LastExportedHeight: h} +} + +func (e *ResultExporter) writeExportState(state exportState) error { + data, err := json.Marshal(state) + if err != nil { + return fmt.Errorf("marshaling export state: %w", err) + } + path := filepath.Join(e.homeDir, exportStateFile) + if err := os.WriteFile(path, data, 0o644); err != nil { + return fmt.Errorf("writing export state: %w", err) + } + return nil +} diff --git a/sidecar/tasks/result_export_test.go b/sidecar/tasks/result_export_test.go new file mode 100644 index 00000000..18ce001b --- /dev/null +++ b/sidecar/tasks/result_export_test.go @@ -0,0 +1,572 @@ +package tasks + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + "github.com/prometheus/client_golang/prometheus/testutil" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" + "github.com/sei-protocol/sei-k8s-controller/sidecar/shadow" +) + +type mockResultUploader struct{} + +func (m *mockResultUploader) UploadObject(_ context.Context, in *transfermanager.UploadObjectInput, _ ...func(*transfermanager.Options)) (*transfermanager.UploadObjectOutput, error) { + if in.Body != nil { + _, _ = io.Copy(io.Discard, in.Body) + } + return &transfermanager.UploadObjectOutput{}, nil +} + +func mockResultUploaderFactory() seis3.UploaderFactory { + return func(_ context.Context, _ string) (seis3.Uploader, error) { + return &mockResultUploader{}, nil + } +} + +func failingUploaderFactory(errMsg string) seis3.UploaderFactory { + return func(_ context.Context, _ string) (seis3.Uploader, error) { + return nil, fmt.Errorf("%s", errMsg) + } +} + +// drainingFailingUploader drains the streamed body (so the writer never blocks) +// then fails — an S3 error after reading a valid stream. +type drainingFailingUploader struct{ err error } + +func (u drainingFailingUploader) UploadObject(_ context.Context, in *transfermanager.UploadObjectInput, _ ...func(*transfermanager.Options)) (*transfermanager.UploadObjectOutput, error) { + if in.Body != nil { + _, _ = io.Copy(io.Discard, in.Body) + } + return nil, u.err +} + +// TestExportPage_ProducerErrorNotClassifiedAsS3 proves a block_results RPC +// failure is returned as-is, not mislabeled as an S3 access problem. +func TestExportPage_ProducerErrorNotClassifiedAsS3(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/block_results" { + http.Error(w, "block results unavailable", http.StatusInternalServerError) + return + } + fmt.Fprint(w, `{"jsonrpc":"2.0","id":-1,"result":{"sync_info":{"latest_block_height":"100"}}}`) + })) + defer srv.Close() + + e := NewResultExporter(t.TempDir(), "test-1", "pod-0", mockResultUploaderFactory()) + err := e.exportPage(context.Background(), rpc.NewClient(srv.URL, nil), &mockResultUploader{}, "bkt", "us-east-1", "p/", 100, 100) + if err == nil { + t.Fatal("expected a producer error") + } + var te *engine.TaskError + if errors.As(err, &te) && te.Operation == "S3" { + t.Fatalf("producer RPC error misclassified as S3: %v", err) + } + if !strings.Contains(err.Error(), "block_results") { + t.Fatalf("expected the block_results producer error, got: %v", err) + } +} + +// TestExportPage_UploadErrorKeepsS3Classification proves a genuine upload +// failure retains its ClassifyS3Error treatment (Operation S3), not lost behind +// a wrapped sink-write error. +func TestExportPage_UploadErrorKeepsS3Classification(t *testing.T) { + srv := fakeRPCServer(100) + defer srv.Close() + + up := drainingFailingUploader{err: errors.New("connection reset by peer")} + e := NewResultExporter(t.TempDir(), "test-1", "pod-0", mockResultUploaderFactory()) + err := e.exportPage(context.Background(), rpc.NewClient(srv.URL, nil), up, "bkt", "us-east-1", "p/", 100, 100) + if err == nil { + t.Fatal("expected an upload error") + } + var te *engine.TaskError + if !errors.As(err, &te) || te.Operation != "S3" { + t.Fatalf("upload failure should be S3-classified, got: %v", err) + } +} + +// fakeRPCServer returns an httptest.Server that responds to /status and /block_results. +func fakeRPCServer(latestHeight int64) *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case r.URL.Path == "/status": + fmt.Fprintf(w, `{"jsonrpc":"2.0","id":-1,"result":{"sync_info":{"latest_block_height":"%d"}}}`, latestHeight) + case r.URL.Path == "/block_results": + fmt.Fprint(w, `{"jsonrpc":"2.0","id":-1,"result":{}}`) + default: + http.NotFound(w, r) + } + })) +} + +func TestExportBootstrapFromSnapshotHeight(t *testing.T) { + tmpDir := t.TempDir() + if err := os.WriteFile(filepath.Join(tmpDir, SnapshotHeightFile), []byte("198030000"), 0o644); err != nil { + t.Fatalf("writing snapshot height file: %v", err) + } + + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", nil) + state := e.readExportState() + + if state.LastExportedHeight != 198030000 { + t.Errorf("LastExportedHeight = %d, want 198030000", state.LastExportedHeight) + } +} + +func TestExportBootstrapNoFiles(t *testing.T) { + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", nil) + state := e.readExportState() + + if state.LastExportedHeight != 0 { + t.Errorf("LastExportedHeight = %d, want 0", state.LastExportedHeight) + } +} + +func TestExportBootstrapPreferStateFile(t *testing.T) { + tmpDir := t.TempDir() + + stateData, _ := json.Marshal(exportState{LastExportedHeight: 200000000}) + if err := os.WriteFile(filepath.Join(tmpDir, exportStateFile), stateData, 0o644); err != nil { + t.Fatalf("writing state file: %v", err) + } + if err := os.WriteFile(filepath.Join(tmpDir, SnapshotHeightFile), []byte("198030000"), 0o644); err != nil { + t.Fatalf("writing snapshot height file: %v", err) + } + + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", nil) + state := e.readExportState() + + if state.LastExportedHeight != 200000000 { + t.Errorf("LastExportedHeight = %d, want 200000000", state.LastExportedHeight) + } +} + +func TestExportRPCUnavailable(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusInternalServerError) + })) + srv.Close() + + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", mockResultUploaderFactory()) + + err := e.Export(context.Background(), ResultExportRequest{ + Bucket: "test-bucket", + Region: "us-east-1", + RPCEndpoint: srv.URL, + }) + if err != nil { + t.Fatalf("Export() returned error %v, want nil (fail-safe)", err) + } +} + +func TestExportRPCNon200Status(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusServiceUnavailable) + fmt.Fprint(w, "node is syncing") + })) + defer srv.Close() + + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", mockResultUploaderFactory()) + + err := e.Export(context.Background(), ResultExportRequest{ + Bucket: "test-bucket", + Region: "us-east-1", + RPCEndpoint: srv.URL, + }) + if err != nil { + t.Fatalf("Export() returned error %v, want nil (fail-safe on HTTP error)", err) + } +} + +func TestQueryLatestHeight_ZeroHeight(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + fmt.Fprint(w, `{"jsonrpc":"2.0","id":-1,"result":{"sync_info":{"latest_block_height":"0"}}}`) + })) + defer srv.Close() + + _, err := queryLatestHeight(context.Background(), srv.URL) + if err == nil { + t.Fatal("expected error for zero height, got nil") + } +} + +func TestExportS3UploaderFactoryError(t *testing.T) { + srv := fakeRPCServer(100000) + defer srv.Close() + + tmpDir := t.TempDir() + if err := os.WriteFile(filepath.Join(tmpDir, SnapshotHeightFile), []byte("1"), 0o644); err != nil { + t.Fatalf("writing snapshot height file: %v", err) + } + + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", failingUploaderFactory("simulated AWS error")) + + err := e.Export(context.Background(), ResultExportRequest{ + Bucket: "test-bucket", + Region: "us-east-1", + RPCEndpoint: srv.URL, + }) + if err == nil { + t.Fatal("expected error when S3 uploader factory fails") + } + if !strings.Contains(err.Error(), "simulated AWS error") { + t.Errorf("expected simulated AWS error, got: %v", err) + } +} + +func TestExportWritesStateAfterPage(t *testing.T) { + // Latest height 1001 with start at 1 gives exactly 1001 available blocks, + // which is 1 full page (heights 1–1000). The remaining 1 block is deferred. + srv := fakeRPCServer(1001) + defer srv.Close() + + tmpDir := t.TempDir() + if err := os.WriteFile(filepath.Join(tmpDir, SnapshotHeightFile), []byte("0"), 0o644); err != nil { + t.Fatalf("writing snapshot height file: %v", err) + } + + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", mockResultUploaderFactory()) + err := e.Export(context.Background(), ResultExportRequest{ + Bucket: "test-bucket", + Prefix: "results", + Region: "us-east-1", + RPCEndpoint: srv.URL, + }) + if err != nil { + t.Fatalf("Export() error = %v", err) + } + + state := e.readExportState() + if state.LastExportedHeight != 1000 { + t.Errorf("LastExportedHeight = %d, want 1000", state.LastExportedHeight) + } +} + +func TestExportHandler_MissingParams(t *testing.T) { + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", mockResultUploaderFactory()) + handler := e.Handler() + + cases := []struct { + name string + params map[string]any + }{ + {"missing bucket", map[string]any{"region": "us-east-1"}}, + {"missing region", map[string]any{"bucket": "my-bucket"}}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + _, err := handler(context.Background(), tc.params) + if err == nil { + t.Fatal("expected error, got nil") + } + }) + } +} + +func TestExportConfigJSONRoundTrip(t *testing.T) { + cfg := ResultExportRequest{ + Bucket: "my-bucket", + Region: "us-east-1", + RPCEndpoint: "http://custom:26657", + CanonicalRPC: "http://canonical:26657", + } + data, err := json.Marshal(cfg) + if err != nil { + t.Fatalf("marshaling: %v", err) + } + var decoded ResultExportRequest + if err := json.Unmarshal(data, &decoded); err != nil { + t.Fatalf("unmarshaling: %v", err) + } + if decoded.Bucket != cfg.Bucket { + t.Errorf("Bucket = %q, want %q", decoded.Bucket, cfg.Bucket) + } + if decoded.Region != cfg.Region { + t.Errorf("Region = %q, want %q", decoded.Region, cfg.Region) + } + if decoded.RPCEndpoint != cfg.RPCEndpoint { + t.Errorf("RPCEndpoint = %q, want %q", decoded.RPCEndpoint, cfg.RPCEndpoint) + } + if decoded.CanonicalRPC != cfg.CanonicalRPC { + t.Errorf("CanonicalRPC = %q, want %q", decoded.CanonicalRPC, cfg.CanonicalRPC) + } +} + +// --- Handler routing tests --- + +func TestHandlerRouting_WithCanonicalRPC_CallsExportAndCompare(t *testing.T) { + srv := fakeRPCAndBlockServer(1, "AABB", "CCDD", nil) + defer srv.Close() + + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", mockResultUploaderFactory()) + + ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond) + defer cancel() + + _, err := e.Handler()(ctx, map[string]any{ + "bucket": "test-bucket", + "region": "us-east-1", + "rpcEndpoint": srv.URL, + "canonicalRpc": srv.URL, + }) + if err == nil { + t.Fatal("expected context deadline exceeded error") + } +} + +func TestHandlerRouting_WithoutCanonicalRPC_CallsExport(t *testing.T) { + srv := fakeRPCServer(0) // 0 blocks → nothing to export + defer srv.Close() + + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", mockResultUploaderFactory()) + + _, err := e.Handler()(context.Background(), map[string]any{ + "bucket": "test-bucket", + "region": "us-east-1", + "rpcEndpoint": srv.URL, + }) + if err != nil { + t.Fatalf("Handler() error = %v, want nil for empty export", err) + } +} + +// --- ExportAndCompare tests --- + +func TestExportAndCompare_DivergenceDetected(t *testing.T) { + // Shadow returns different app hash than canonical → immediate divergence. + shadowSrv := fakeRPCAndBlockServer(5, "SHADOW_HASH", "RESULTS", nil) + defer shadowSrv.Close() + canonicalSrv := fakeRPCAndBlockServer(5, "CANONICAL_HASH", "RESULTS", nil) + defer canonicalSrv.Close() + + tmpDir := t.TempDir() + const testPodName = "shadow-test-0" + e := NewResultExporter(tmpDir, "test-1", testPodName, mockResultUploaderFactory()) + + divergenceBefore := testutil.ToFloat64(shadow.Divergences.WithLabelValues("test-1", testPodName, "0")) + + err := e.ExportAndCompare(context.Background(), ResultExportRequest{ + Bucket: "test-bucket", + Prefix: "compare/", + Region: "us-east-1", + RPCEndpoint: shadowSrv.URL, + CanonicalRPC: canonicalSrv.URL, + }) + // Divergence detected = task completes successfully (nil error). + if err != nil { + t.Fatalf("ExportAndCompare() error = %v, want nil (divergence = success)", err) + } + + state := e.readExportState() + if state.LastExportedHeight != 1 { + t.Errorf("LastExportedHeight = %d, want 1 (diverged at first block)", state.LastExportedHeight) + } + + if got := testutil.ToFloat64(shadow.Divergences.WithLabelValues("test-1", testPodName, "0")); got-divergenceBefore != 1 { + t.Errorf("seictl_shadow_divergences_total{chain_id=test-1,pod_name=%s,divergence_layer=0} delta = %v, want 1", testPodName, got-divergenceBefore) + } + if got := testutil.ToFloat64(shadow.BlocksCompared.WithLabelValues("test-1", testPodName)); got < 1 { + t.Errorf("seictl_shadow_blocks_compared_total{chain_id=test-1,pod_name=%s} = %v, want >= 1", testPodName, got) + } +} + +func TestExportAndCompare_ContextCancelled(t *testing.T) { + srv := fakeRPCAndBlockServer(1, "SAME", "SAME", nil) + defer srv.Close() + + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", mockResultUploaderFactory()) + + ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond) + defer cancel() + + err := e.ExportAndCompare(ctx, ResultExportRequest{ + Bucket: "test-bucket", + Region: "us-east-1", + RPCEndpoint: srv.URL, + CanonicalRPC: srv.URL, + }) + if err == nil { + t.Fatal("expected context deadline exceeded error") + } +} + +func TestExportAndCompare_S3UploaderError(t *testing.T) { + srv := fakeRPCAndBlockServer(5, "AA", "BB", nil) + defer srv.Close() + + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", failingUploaderFactory("AWS creds expired")) + + err := e.ExportAndCompare(context.Background(), ResultExportRequest{ + Bucket: "test-bucket", + Region: "us-east-1", + RPCEndpoint: srv.URL, + CanonicalRPC: srv.URL, + }) + if err == nil { + t.Fatal("expected error from S3 uploader factory failure") + } +} + +func TestExportAndCompare_ResumesFromExportState(t *testing.T) { + // Set export state to height 3, shadow serves up to 5. + // With matching hashes, it compares blocks 4 and 5 (no divergence). + // Set up divergence at block 4 by using different servers. + shadowSrv := fakeRPCAndBlockServer(5, "SHADOW", "RES", nil) + defer shadowSrv.Close() + canonicalSrv := fakeRPCAndBlockServer(5, "CANONICAL", "RES", nil) + defer canonicalSrv.Close() + + tmpDir := t.TempDir() + stateData, _ := json.Marshal(exportState{LastExportedHeight: 3}) + if err := os.WriteFile(filepath.Join(tmpDir, exportStateFile), stateData, 0o644); err != nil { + t.Fatalf("writing state: %v", err) + } + + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", mockResultUploaderFactory()) + err := e.ExportAndCompare(context.Background(), ResultExportRequest{ + Bucket: "test-bucket", + Region: "us-east-1", + RPCEndpoint: shadowSrv.URL, + CanonicalRPC: canonicalSrv.URL, + }) + if err != nil { + t.Fatalf("ExportAndCompare() error = %v", err) + } + + state := e.readExportState() + if state.LastExportedHeight != 4 { + t.Errorf("LastExportedHeight = %d, want 4 (first block after resume)", state.LastExportedHeight) + } +} + +// --- Divergence report tests --- + +func TestExportAndCompare_UploadsDivergenceReport(t *testing.T) { + shadowSrv := fakeRPCAndBlockServer(5, "SHADOW", "RESULTS", nil) + defer shadowSrv.Close() + canonicalSrv := fakeRPCAndBlockServer(5, "CANONICAL", "RESULTS", nil) + defer canonicalSrv.Close() + + recorder := &recordingUploader{} + tmpDir := t.TempDir() + e := NewResultExporter(tmpDir, "test-1", "test-pod-0", func(_ context.Context, _ string) (seis3.Uploader, error) { + return recorder, nil + }) + + err := e.ExportAndCompare(context.Background(), ResultExportRequest{ + Bucket: "test-bucket", + Prefix: "shadow/pacific-1/", + Region: "us-east-1", + RPCEndpoint: shadowSrv.URL, + CanonicalRPC: canonicalSrv.URL, + }) + if err != nil { + t.Fatalf("ExportAndCompare() error = %v", err) + } + + var reportKey string + var compareKey string + for _, key := range recorder.keys { + if strings.Contains(key, "divergence-") && strings.Contains(key, ".report.json.gz") { + reportKey = key + } + if strings.Contains(key, ".compare.ndjson.gz") { + compareKey = key + } + } + + if reportKey == "" { + t.Errorf("expected divergence report upload, got keys: %v", recorder.keys) + } + if compareKey == "" { + t.Errorf("expected comparison page upload, got keys: %v", recorder.keys) + } + + if reportKey != "" && reportKey != "shadow/pacific-1/divergence-1.report.json.gz" { + t.Errorf("report key = %q, want %q", reportKey, "shadow/pacific-1/divergence-1.report.json.gz") + } +} + +// --- flushComparePage tests --- + +func TestFlushComparePage_EmptyResults(t *testing.T) { + err := flushComparePage(context.Background(), &mockResultUploader{}, "bucket", "prefix/", nil) + if err != nil { + t.Errorf("flushComparePage(nil) error = %v, want nil", err) + } +} + +// --- Test helpers --- + +type recordingUploader struct { + keys []string +} + +func (r *recordingUploader) UploadObject(_ context.Context, in *transfermanager.UploadObjectInput, _ ...func(*transfermanager.Options)) (*transfermanager.UploadObjectOutput, error) { + if in.Key != nil { + r.keys = append(r.keys, *in.Key) + } + if in.Body != nil { + _, _ = io.Copy(io.Discard, in.Body) + } + return &transfermanager.UploadObjectOutput{}, nil +} + +// fakeRPCAndBlockServer responds to /status, /block, and /block_results. +func fakeRPCAndBlockServer(latestHeight int64, appHash, lastResultsHash string, txResults []json.RawMessage) *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case r.URL.Path == "/status": + fmt.Fprintf(w, `{"jsonrpc":"2.0","id":-1,"result":{"sync_info":{"latest_block_height":"%d"}}}`, latestHeight) + case r.URL.Path == "/block": + resp := map[string]any{ + "jsonrpc": "2.0", + "id": -1, + "result": map[string]any{ + "block_id": map[string]any{"hash": ""}, + "block": map[string]any{ + "header": map[string]any{ + "app_hash": appHash, + "last_results_hash": lastResultsHash, + }, + }, + }, + } + json.NewEncoder(w).Encode(resp) + case r.URL.Path == "/block_results": + resp := map[string]any{ + "jsonrpc": "2.0", + "id": -1, + "result": map[string]any{ + "txs_results": txResults, + }, + } + json.NewEncoder(w).Encode(resp) + default: + http.NotFound(w, r) + } + })) +} diff --git a/sidecar/tasks/sign_and_broadcast.go b/sidecar/tasks/sign_and_broadcast.go new file mode 100644 index 00000000..e4874674 --- /dev/null +++ b/sidecar/tasks/sign_and_broadcast.go @@ -0,0 +1,517 @@ +// Package tasks — sign-tx family helper. +// +// SignAndBroadcast signs txs as the validator's operator account. +// API-level authentication is controlled by SEI_SIDECAR_AUTHN_MODE: +// trusted-header mode pairs the sidecar with a kube-rbac-proxy +// container that gates every request via TokenReview + SAR. When the +// env var is unset, the API is unauthenticated and any actor with +// network reach to the listen port can reach this code path. + +package tasks + +import ( + "context" + "crypto/sha256" + "encoding/json" + "errors" + "fmt" + "os" + "strings" + "time" + "unicode" + + "github.com/sei-protocol/sei-chain/sei-cosmos/client/tx" + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + authtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/types" + + signingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/types/tx/signing" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +var signTxLog = seilog.NewLogger("seictl", "task", "sign-tx") + +// feeDenom is the only fee denom the sidecar will sign for. seienv +// historically used "sei" (e.g., `--fees 20sei` in vote.go:15) which +// seid rejects at parse time — we make the rejection explicit and +// chain-confusion-proof. 1 SEI = 1_000_000 usei. +const feeDenom = "usei" + +// Var (not const) so tests can shorten without a deadline context. +var ( + inclusionPollTimeout = 60 * time.Second + inclusionPollInterval = 500 * time.Millisecond +) + +// SignAndBroadcastInput is the shared input contract for every sign-tx +// handler. Msg stays as sdk.Msg so the helper never grows a per-msg switch. +type SignAndBroadcastInput struct { + ChainID string + KeyName string + Msg sdk.Msg + + // Fees is a coin-string in usei. Non-usei denoms are rejected Terminal. + Fees string + + Gas uint64 + Memo string + + // TaskID is appended to the on-chain memo so operators can grep the + // chain by task. See appendTaskIDToMemo. + TaskID string +} + +// SignAndBroadcastResult is the shared output contract. Sign-tx handlers +// extend it with type-specific fields after this function returns. +type SignAndBroadcastResult struct { + TxHash string `json:"txHash"` + Height int64 `json:"height"` + Code uint32 `json:"code"` + Codespace string `json:"codespace,omitempty"` + RawLog string `json:"rawLog,omitempty"` + GasWanted int64 `json:"gasWanted"` + GasUsed int64 `json:"gasUsed"` + Sequence uint64 `json:"sequence"` + AccountNumber uint64 `json:"accountNumber"` + ChainID string `json:"chainId"` + BroadcastedAt time.Time `json:"broadcastedAt"` + // ProposalID is decoded from the committed tx's result data (see + // parseProposalID); 0 for votes, non-gov txs, or a not-yet-included tx. + ProposalID uint64 `json:"proposalId,omitempty"` + // IncludedAt is nil when inclusion polling timed out after a + // successful broadcast. nil means UNDETERMINED — the tx may still + // land later. It does NOT mean "not included". Callers must + // re-query the chain to determine final state. + IncludedAt *time.Time `json:"includedAt,omitempty"` + // Unverifiable is set when the tx was broadcast but its outcome cannot be + // observed because the target node's tx index is off (errTxIndexingDisabled). + // classifyGovResult turns this into a terminal InclusionUnverifiable — it is + // distinct from IncludedAt==nil (pending/retryable): retrying is futile. + Unverifiable bool `json:"unverifiable,omitempty"` +} + +// TerminalError marks a sign-tx error as non-retryable (malformed input, +// chain-confusion, CheckTx rejection, missing key). The engine has no +// retry policy yet, but callers should not implement ad-hoc retry on top. +type TerminalError struct { + Err error +} + +func (e *TerminalError) Error() string { return e.Err.Error() } +func (e *TerminalError) Unwrap() error { return e.Err } + +// Terminal wraps err as TerminalError, or returns nil when err is nil. +func Terminal(err error) error { + if err == nil { + return nil + } + return &TerminalError{Err: err} +} + +// IsTerminal reports whether err (or any wrapped error) is a TerminalError. +func IsTerminal(err error) bool { + var t *TerminalError + return errors.As(err, &t) +} + +// txClient is the narrow seam for everything not covered by local files +// or rpc.Client: account fetch, broadcast, tx-by-hash query. Kept local +// to sign-tx rather than on engine.ExecutionConfig. +type txClient interface { + // AccountNumberSequence always refreshes — a stale sequence is the + // most common CheckTx rejection and could mask a prior broadcast. + AccountNumberSequence(ctx context.Context, fromAddr sdk.AccAddress) (uint64, uint64, error) + + // BroadcastSync returns (resp, err). resp.Code != 0 is a CheckTx + // rejection and is treated as Terminal. + BroadcastSync(ctx context.Context, txBytes []byte) (*sdk.TxResponse, error) + + // QueryTx returns (resp, found, err). found=false / err=nil means + // the node has no record (distinct from a transport error). A node with + // tx indexing disabled returns errTxIndexingDisabled (match via + // errors.Is), since /tx cannot answer there. + QueryTx(ctx context.Context, txHashHex string) (*sdk.TxResponse, bool, error) +} + +// SignAndBroadcast is the entry point each sign-tx handler calls. It +// resolves the signer from the in-memory keyring, wires the production +// txClient, and delegates to signAndBroadcast for the full validate + +// guard + sign + broadcast + poll cycle. +func SignAndBroadcast(ctx context.Context, cfg engine.ExecutionConfig, in SignAndBroadcastInput) (*SignAndBroadcastResult, error) { + if cfg.Keyring == nil { + return nil, Terminal(errors.New("keyring not configured: set SEI_KEYRING_BACKEND/SEI_KEYRING_PASSPHRASE on the sidecar")) + } + info, err := cfg.Keyring.Key(in.KeyName) + if err != nil { + return nil, Terminal(fmt.Errorf("keyring entry %q: %w", in.KeyName, err)) + } + fromAddr := info.GetAddress() + + tc, err := newSDKTxClient(cfg, in, fromAddr) + if err != nil { + return nil, err + } + return signAndBroadcast(ctx, cfg, tc, in, fromAddr) +} + +// signAndBroadcast does the full sign+broadcast+poll cycle. Public callers +// reach this through SignAndBroadcast which wires the production txClient; +// tests call it directly with a fake. +func signAndBroadcast(ctx context.Context, cfg engine.ExecutionConfig, tc txClient, in SignAndBroadcastInput, fromAddr sdk.AccAddress) (*SignAndBroadcastResult, error) { + // Reject caller-supplied "taskID=" before appending so on-chain + // audit greps never see a forged tag preceding the genuine one. + if strings.Contains(in.Memo, taskIDMemoPrefix) { + return nil, Terminal(fmt.Errorf("memo must not contain %q (reserved for engine-supplied task tag)", taskIDMemoPrefix)) + } + // Append taskID to memo so the byte cap is enforced against the + // effective on-chain memo. + in.Memo = appendTaskIDToMemo(in.Memo, in.TaskID) + if err := validateInput(in); err != nil { + return nil, Terminal(err) + } + if err := checkFeesDenom(in.Fees); err != nil { + return nil, Terminal(err) + } + if err := guardChainID(ctx, cfg.RPC, in.ChainID); err != nil { + return nil, err + } + + // Adopt an in-flight tx from a prior crashed run instead of signing anew. + if cfg.Checkpointer != nil && in.TaskID != "" { + marker, err := cfg.Checkpointer.GetTxMarker(in.TaskID) + if err != nil { + return nil, fmt.Errorf("read tx marker: %w", err) + } + if marker != nil { + signTxLog.Info("adopting in-flight tx from marker", "taskId", in.TaskID, "txHash", marker.TxHash) + return adoptMarker(ctx, tc, marker) + } + } + + accNum, seq, err := tc.AccountNumberSequence(ctx, fromAddr) + if err != nil { + return nil, fmt.Errorf("account retrieve %s: %w", fromAddr.String(), err) + } + + // WithFees panics on parse error; checkFeesDenom already ran. + _, _, txCfg := makeSignTxCodec() + factory := tx.Factory{}. + WithChainID(in.ChainID). + WithKeybase(cfg.Keyring). + WithTxConfig(txCfg). + WithAccountRetriever(authtypes.AccountRetriever{}). + WithAccountNumber(accNum). + WithSequence(seq). + WithGas(in.Gas). + WithFees(in.Fees). + WithMemo(in.Memo). + WithSignMode(signingtypes.SignMode_SIGN_MODE_DIRECT) + + builder, err := tx.BuildUnsignedTx(factory, in.Msg) + if err != nil { + return nil, Terminal(fmt.Errorf("build unsigned tx: %w", err)) + } + if err := tx.Sign(factory, in.KeyName, builder, true); err != nil { + return nil, fmt.Errorf("sign tx: %w", err) + } + txBytes, err := txCfg.TxEncoder()(builder.GetTx()) + if err != nil { + return nil, fmt.Errorf("encode tx: %w", err) + } + + txHash := fmt.Sprintf("%X", sha256.Sum256(txBytes)) + + // Marker must be durable before broadcast so a crash re-adopts this tx. + if cfg.Checkpointer != nil && in.TaskID != "" { + if err := cfg.Checkpointer.SaveTxMarker(&engine.TxMarker{ + TaskID: in.TaskID, + TxHash: txHash, + TxBytes: txBytes, + AccountNumber: accNum, + Sequence: seq, + ChainID: in.ChainID, + }); err != nil { + return nil, fmt.Errorf("persist tx marker: %w", err) + } + } else { + signTxLog.Warn("no checkpointer configured; broadcasting without crash-idempotency marker", + "taskId", in.TaskID) + } + + signTxLog.Info("broadcasting tx", + "taskId", in.TaskID, "chainId", in.ChainID, + "sequence", seq, "accountNumber", accNum, "gas", in.Gas, "fees", in.Fees) + return broadcastAndPoll(ctx, tc, txBytes, txHash, accNum, seq, in.ChainID) +} + +// adoptReQueryTimeout bounds the re-query after an adopt re-broadcast is +// CheckTx-rejected. Var so tests can shorten it. +var adoptReQueryTimeout = 5 * time.Second + +// adoptMarker resumes a prior run's tx: already on chain → build from it; query +// error → report undetermined (retry, don't re-broadcast blind); not indexed → +// re-broadcast the identical bytes (never re-sign; CometBFT dedups by hash). +// The /tx index lags commit, so a CheckTx rejection on that re-broadcast likely +// means the tx already landed — re-query before failing. +func adoptMarker(ctx context.Context, tc txClient, m *engine.TxMarker) (*SignAndBroadcastResult, error) { + resp, found, err := tc.QueryTx(ctx, m.TxHash) + if err != nil { + if errors.Is(err, errTxIndexingDisabled) { + // Inclusion is unobservable on this node — retrying is futile. + // Return an unverifiable result (classifyGovResult makes it + // terminal), carrying the txHash for the operator's manual check. + return unverifiableResult(m.TxHash, m.AccountNumber, m.Sequence, m.ChainID), nil + } + // Unknown state — report undetermined so the caller re-checks; the + // durable marker makes a later re-broadcast safe. + signTxLog.Warn("adopt query transport error; reporting inclusion-undetermined", + "txHash", m.TxHash, "err", err) + return &SignAndBroadcastResult{ + TxHash: m.TxHash, AccountNumber: m.AccountNumber, Sequence: m.Sequence, + ChainID: m.ChainID, BroadcastedAt: time.Now().UTC(), IncludedAt: nil, + }, nil + } + if found { + now := time.Now().UTC() + return resultFromTxResponse(resp, m.AccountNumber, m.Sequence, m.ChainID, now, &now), nil + } + + out, berr := broadcastAndPoll(ctx, tc, m.TxBytes, m.TxHash, m.AccountNumber, m.Sequence, m.ChainID) + if berr != nil && IsTerminal(berr) { + if included, qerr := pollForInclusion(ctx, tc, m.TxHash, adoptReQueryTimeout, inclusionPollInterval); qerr == nil && included != nil { + now := time.Now().UTC() + return resultFromTxResponse(included, m.AccountNumber, m.Sequence, m.ChainID, now, &now), nil + } + } + return out, berr +} + +// broadcastAndPoll broadcasts signed tx bytes and polls for inclusion; shared +// by the fresh-sign and re-broadcast paths. A CheckTx rejection is terminal; a +// transport error or poll timeout returns an undetermined result (IncludedAt +// nil) so the caller classifies it as pending and re-checks — a bare error +// would look terminal and strand a possibly-live tx. +func broadcastAndPoll(ctx context.Context, tc txClient, txBytes []byte, txHash string, accNum, seq uint64, chainID string) (*SignAndBroadcastResult, error) { + broadcastedAt := time.Now().UTC() + undetermined := &SignAndBroadcastResult{ + TxHash: txHash, Sequence: seq, AccountNumber: accNum, + ChainID: chainID, BroadcastedAt: broadcastedAt, IncludedAt: nil, + } + + resp, err := tc.BroadcastSync(ctx, txBytes) + if err != nil { + signTxLog.Warn("broadcast transport error; reporting inclusion-undetermined", + "txHash", txHash, "err", err) + return undetermined, nil + } + if resp.Code != 0 { + return nil, Terminal(fmt.Errorf("checkTx rejected: code=%d codespace=%q log=%s", + resp.Code, resp.Codespace, resp.RawLog)) + } + + included, perr := pollForInclusion(ctx, tc, resp.TxHash, inclusionPollTimeout, inclusionPollInterval) + if perr != nil { + if errors.Is(perr, errTxIndexingDisabled) { + // Inclusion is unobservable on this node — polling can never + // confirm it. Return an unverifiable result (terminal), not + // pending-until-deadline. + return unverifiableResult(resp.TxHash, accNum, seq, chainID), nil + } + // ctx cancellation (shutdown) — propagate; a poll timeout is (nil,nil). + return nil, perr + } + if included != nil { + now := time.Now().UTC() + return resultFromTxResponse(included, accNum, seq, chainID, broadcastedAt, &now), nil + } + return resultFromTxResponse(resp, accNum, seq, chainID, broadcastedAt, nil), nil +} + +// taskIDMemoPrefix is the literal tag prefix written into the on-chain +// memo by appendTaskIDToMemo. Caller-supplied memos containing this +// substring are rejected so audit greps see exactly one tag per tx. +const taskIDMemoPrefix = "taskID=" + +// appendTaskIDToMemo tags the memo with the task ID so operators can grep +// the chain by memo. taskID="" leaves base unchanged. +func appendTaskIDToMemo(base, taskID string) string { + if taskID == "" { + return base + } + tag := taskIDMemoPrefix + taskID + if base == "" { + return tag + } + return base + " " + tag +} + +func validateInput(in SignAndBroadcastInput) error { + if in.ChainID == "" { + return errors.New("chainId required") + } + if in.KeyName == "" { + return errors.New("keyName required") + } + if in.Msg == nil { + return errors.New("msg required") + } + if in.TaskID == "" { + return errors.New("taskId required (engine should always populate this)") + } + if in.Gas == 0 { + return errors.New("gas required (must be > 0)") + } + if in.Fees == "" { + return errors.New("fees required") + } + if err := validateMemo(in.Memo); err != nil { + return err + } + if err := in.Msg.ValidateBasic(); err != nil { + return fmt.Errorf("msg.ValidateBasic: %w", err) + } + return nil +} + +// maxMemoBytes matches Cosmos SDK's MaxMemoCharacters consensus cap. +const maxMemoBytes = 256 + +// validateMemo rejects oversize or non-printable memos. Memos surface +// in on-chain events and audit pipelines; control chars (tab, newline, +// ANSI) are known log/CSV injection vectors. +func validateMemo(memo string) error { + if len(memo) > maxMemoBytes { + return fmt.Errorf("memo length %d exceeds %d bytes", len(memo), maxMemoBytes) + } + for _, r := range memo { + if !unicode.IsPrint(r) { + return fmt.Errorf("memo contains non-printable character %U", r) + } + } + return nil +} + +// checkFeesDenom rejects any coin string whose denom is not "usei". +// Guards the seienv `--fees 20sei` latent bug before the SDK's +// Factory.WithFees panics deep inside. +func checkFeesDenom(fees string) error { + coins, err := sdk.ParseCoinsNormalized(fees) + if err != nil { + return fmt.Errorf("parse fees %q: %w", fees, err) + } + // ParseCoinsNormalized filters zero-amount entries; "0usei" and + // non-positive sums normalize to empty Coins. + if len(coins) == 0 { + return fmt.Errorf("fees %q resolves to zero or empty coins", fees) + } + if !coins.IsAllPositive() { + return fmt.Errorf("fees %q contains non-positive amounts", fees) + } + for _, c := range coins { + if c.Denom != feeDenom { + return fmt.Errorf("fees %q: denom %q not permitted (only %q)", fees, c.Denom, feeDenom) + } + } + return nil +} + +// guardChainID rejects sign requests whose ChainID does not match BOTH +// SEI_CHAIN_ID AND the chain the local seid reports via /status. +func guardChainID(ctx context.Context, rpcClient *rpc.Client, chainID string) error { + envChain := os.Getenv("SEI_CHAIN_ID") + if envChain == "" { + return Terminal(errors.New("SEI_CHAIN_ID not set on sidecar; refusing to sign")) + } + if chainID != envChain { + return Terminal(fmt.Errorf("chain mismatch: params.chainId=%q sidecar.SEI_CHAIN_ID=%q", chainID, envChain)) + } + if rpcClient == nil { + return Terminal(errors.New("RPC client not configured; cannot verify chain identity via /status")) + } + raw, err := rpcClient.Get(ctx, "/status") + if err != nil { + // Transport failure — refuse to sign since we cannot prove chain identity. + return fmt.Errorf("query local seid /status: %w", err) + } + var status rpc.StatusResult + if err := json.Unmarshal(raw, &status); err != nil { + return fmt.Errorf("decode /status: %w", err) + } + if status.NodeInfo.Network == "" { + return Terminal(errors.New("/status.node_info.network is empty; refusing to sign")) + } + if status.NodeInfo.Network != chainID { + return Terminal(fmt.Errorf("chain mismatch: params.chainId=%q node.network=%q", chainID, status.NodeInfo.Network)) + } + return nil +} + +// pollForInclusion polls /tx?hash=... until inclusion, ctx cancel, or +// timeout. Returns (resp, nil) on inclusion, (nil, nil) on timeout +// (caller treats as "broadcast OK, confirmation deferred"), and +// (nil, ctx.Err()) on cancellation. +func pollForInclusion(ctx context.Context, tc txClient, txHash string, timeout, interval time.Duration) (*sdk.TxResponse, error) { + deadline := time.Now().Add(timeout) + for { + // Check cancellation + deadline before each network call so a + // hung QueryTx transport does not blow past the bound. + if err := ctx.Err(); err != nil { + return nil, err + } + if time.Now().After(deadline) { + return nil, nil + } + resp, found, err := tc.QueryTx(ctx, txHash) + if err == nil && found { + return resp, nil + } + if errors.Is(err, errTxIndexingDisabled) { + // Unobservable on this node — polling can never confirm inclusion, + // so stop and surface it for the caller rather than spin until the + // deadline. + return nil, err + } + select { + case <-ctx.Done(): + return nil, ctx.Err() + case <-time.After(interval): + } + } +} + +// unverifiableResult builds the result for a broadcast tx whose on-chain +// outcome cannot be observed because the target node's tx index is off. +// IncludedAt stays nil and Unverifiable is set, so classifyGovResult reports a +// terminal InclusionUnverifiable — distinct from the retryable pending case. +func unverifiableResult(txHash string, accNum, seq uint64, chainID string) *SignAndBroadcastResult { + return &SignAndBroadcastResult{ + TxHash: txHash, + AccountNumber: accNum, + Sequence: seq, + ChainID: chainID, + BroadcastedAt: time.Now().UTC(), + IncludedAt: nil, + Unverifiable: true, + } +} + +func resultFromTxResponse(resp *sdk.TxResponse, accNum, seq uint64, chainID string, broadcastedAt time.Time, includedAt *time.Time) *SignAndBroadcastResult { + return &SignAndBroadcastResult{ + TxHash: resp.TxHash, + Height: resp.Height, + Code: resp.Code, + Codespace: resp.Codespace, + RawLog: resp.RawLog, + GasWanted: resp.GasWanted, + GasUsed: resp.GasUsed, + Sequence: seq, + AccountNumber: accNum, + ChainID: chainID, + BroadcastedAt: broadcastedAt, + IncludedAt: includedAt, + ProposalID: parseProposalID(resp), + } +} diff --git a/sidecar/tasks/sign_and_broadcast_test.go b/sidecar/tasks/sign_and_broadcast_test.go new file mode 100644 index 00000000..c8af82bd --- /dev/null +++ b/sidecar/tasks/sign_and_broadcast_test.go @@ -0,0 +1,890 @@ +package tasks + +import ( + "context" + "crypto/sha256" + "errors" + "fmt" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + "time" + + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/hd" + "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/keyring" + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + txtypes "github.com/sei-protocol/sei-chain/sei-cosmos/types/tx" + govtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/gov/types" + rpctypes "github.com/sei-protocol/sei-chain/sei-tendermint/rpc/jsonrpc/types" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// fakeTxClient is the in-memory txClient used by tests. +type fakeTxClient struct { + mu sync.Mutex + + accountNumber uint64 + sequence uint64 + accountErr error + + broadcastResp *sdk.TxResponse + broadcastErr error + broadcasts int + lastTxBytes []byte + + // queryDefault returns canned data for any hash (mirroring production + // QueryTx). Nil returns not-found. + queryDefault *sdk.TxResponse + queryErr error + queryCalls int + + // queryFoundAfter makes the first N QueryTx calls return not-found; + // calls after that return queryDefault (found). Zero preserves the + // default behavior. Used to model a committed-but-unindexed tx that + // the /tx index only surfaces on a later re-query. + queryFoundAfter int +} + +func (f *fakeTxClient) AccountNumberSequence(_ context.Context, _ sdk.AccAddress) (uint64, uint64, error) { + f.mu.Lock() + defer f.mu.Unlock() + return f.accountNumber, f.sequence, f.accountErr +} + +func (f *fakeTxClient) BroadcastSync(_ context.Context, txBytes []byte) (*sdk.TxResponse, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.broadcasts++ + f.lastTxBytes = append(f.lastTxBytes[:0], txBytes...) + if f.broadcastErr != nil { + return nil, f.broadcastErr + } + return f.broadcastResp, nil +} + +func (f *fakeTxClient) QueryTx(_ context.Context, hash string) (*sdk.TxResponse, bool, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.queryCalls++ + if f.queryErr != nil { + return nil, false, f.queryErr + } + if f.queryCalls <= f.queryFoundAfter { + return nil, false, nil + } + if f.queryDefault != nil { + resp := *f.queryDefault + resp.TxHash = hash + return &resp, true, nil + } + return nil, false, nil +} + +// fakeCheckpointer is an in-memory engine.Checkpointer test double. +type fakeCheckpointer struct { + mu sync.Mutex + markers map[string]*engine.TxMarker + saveErr error + saves int + + // tc lets SaveTxMarker record whether a broadcast had already happened + // at save time, so tests can assert the marker is persisted BEFORE the + // broadcast side effect. saveBroadcasts is tc.broadcasts captured on the + // most recent SaveTxMarker call. + tc *fakeTxClient + saveBroadcasts int +} + +func newFakeCheckpointer(tc *fakeTxClient) *fakeCheckpointer { + return &fakeCheckpointer{markers: map[string]*engine.TxMarker{}, tc: tc} +} + +func (f *fakeCheckpointer) SaveTxMarker(m *engine.TxMarker) error { + f.mu.Lock() + defer f.mu.Unlock() + f.saves++ + if f.tc != nil { + f.tc.mu.Lock() + f.saveBroadcasts = f.tc.broadcasts + f.tc.mu.Unlock() + } + if f.saveErr != nil { + return f.saveErr + } + cp := *m + f.markers[m.TaskID] = &cp + return nil +} + +func (f *fakeCheckpointer) GetTxMarker(taskID string) (*engine.TxMarker, error) { + f.mu.Lock() + defer f.mu.Unlock() + m, ok := f.markers[taskID] + if !ok { + return nil, nil + } + return m, nil +} + +// testKeyring returns a memory keyring with one entry under "node_admin". +func testKeyring(t *testing.T) (keyring.Keyring, sdk.AccAddress) { + t.Helper() + ensureBech32() + kb, err := keyring.New("seictl-tests", keyring.BackendMemory, t.TempDir(), nil) + if err != nil { + t.Fatalf("keyring: %v", err) + } + info, _, err := kb.NewMnemonic("node_admin", keyring.English, sdk.GetConfig().GetFullBIP44Path(), "", hd.Secp256k1) + if err != nil { + t.Fatalf("NewMnemonic: %v", err) + } + return kb, info.GetAddress() +} + +func fakeStatusServer(t *testing.T, network string, respCode int) *httptest.Server { + t.Helper() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/status" { + http.NotFound(w, r) + return + } + if respCode != 0 { + w.WriteHeader(respCode) + return + } + // Seid's flat response shape (no JSON-RPC envelope). + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"node_info":{"id":"abc","network":"` + network + `"},"sync_info":{"latest_block_height":"100","catching_up":false}}`)) + })) + t.Cleanup(srv.Close) + return srv +} + +func makeMsgVote(t *testing.T, voter sdk.AccAddress) sdk.Msg { + t.Helper() + return govtypes.NewMsgVote(voter, 7, govtypes.OptionYes) +} + +func newGuardCfg(t *testing.T, chainID string) (engine.ExecutionConfig, sdk.AccAddress) { + t.Helper() + t.Setenv("SEI_CHAIN_ID", chainID) + srv := fakeStatusServer(t, chainID, 0) + kb, addr := testKeyring(t) + return engine.ExecutionConfig{ + Keyring: kb, + RPC: rpc.NewClient(srv.URL, nil), + }, addr +} + +// --- Tests ----------------------------------------------------------- + +func TestChainConfusionGuard_EnvMismatch(t *testing.T) { + cfg, addr := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{} + + _, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "wrong-chain", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-000000000001", + }, addr) + if !IsTerminal(err) { + t.Fatalf("want Terminal env-mismatch error, got %v", err) + } + if !strings.Contains(err.Error(), "SEI_CHAIN_ID") { + t.Fatalf("error should reference SEI_CHAIN_ID, got %q", err.Error()) + } + if tc.broadcasts != 0 { + t.Fatalf("broadcast must not run on guard failure; saw %d", tc.broadcasts) + } +} + +func TestChainConfusionGuard_StatusMismatch(t *testing.T) { + t.Setenv("SEI_CHAIN_ID", "pacific-1") + srv := fakeStatusServer(t, "atlantic-2", 0) + kb, addr := testKeyring(t) + cfg := engine.ExecutionConfig{ + Keyring: kb, + RPC: rpc.NewClient(srv.URL, nil), + } + + _, err := signAndBroadcast(context.Background(), cfg, &fakeTxClient{}, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-000000000002", + }, addr) + if !IsTerminal(err) { + t.Fatalf("want Terminal status-mismatch error, got %v", err) + } + if !strings.Contains(err.Error(), "node.network") { + t.Fatalf("error should reference node.network, got %q", err.Error()) + } +} + +func TestChainConfusionGuard_MissingEnv(t *testing.T) { + // t.Setenv("","") auto-restores after the test, unlike os.Unsetenv. + t.Setenv("SEI_CHAIN_ID", "") + srv := fakeStatusServer(t, "pacific-1", 0) + kb, addr := testKeyring(t) + cfg := engine.ExecutionConfig{ + Keyring: kb, + RPC: rpc.NewClient(srv.URL, nil), + } + + _, err := signAndBroadcast(context.Background(), cfg, &fakeTxClient{}, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-000000000003", + }, addr) + if !IsTerminal(err) { + t.Fatalf("want Terminal env-not-set error, got %v", err) + } +} + +func TestFeesDenomGuard_RejectsNonUSei(t *testing.T) { + cfg, addr := newGuardCfg(t, "pacific-1") + + _, err := signAndBroadcast(context.Background(), cfg, &fakeTxClient{}, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "20sei", // the seienv vote.go:15 latent bug + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-000000000004", + }, addr) + if !IsTerminal(err) { + t.Fatalf("want Terminal denom error, got %v", err) + } + if !strings.Contains(err.Error(), "usei") { + t.Fatalf("error should mention usei, got %q", err.Error()) + } +} + +func TestMissingKeyring_ReturnsTerminal(t *testing.T) { + t.Setenv("SEI_CHAIN_ID", "pacific-1") + srv := fakeStatusServer(t, "pacific-1", 0) + cfg := engine.ExecutionConfig{ + Keyring: nil, // sidecar started without SEI_KEYRING_BACKEND + RPC: rpc.NewClient(srv.URL, nil), + } + + _, err := SignAndBroadcast(context.Background(), cfg, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: govtypes.NewMsgVote(makeAddr(t), 7, govtypes.OptionYes), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-000000000005", + }) + if !IsTerminal(err) { + t.Fatalf("want Terminal missing-keyring error, got %v", err) + } +} + +func TestMissingKeyEntry_ReturnsTerminal(t *testing.T) { + cfg, _ := newGuardCfg(t, "pacific-1") + + _, err := SignAndBroadcast(context.Background(), cfg, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "ghost", // not in the keyring + Msg: govtypes.NewMsgVote(makeAddr(t), 7, govtypes.OptionYes), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-000000000006", + }) + if !IsTerminal(err) { + t.Fatalf("want Terminal missing-key error, got %v", err) + } +} + +func TestAccountRetrieverFailure_Propagates(t *testing.T) { + // Account-retrieve failure is NOT Terminal — may be a transient seid restart. + cfg, addr := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{accountErr: errors.New("rpc dial: connection refused")} + + _, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-000000000007", + }, addr) + if err == nil { + t.Fatal("expected error from account retrieve") + } + if IsTerminal(err) { + t.Fatalf("account-retrieve transport errors should be retryable; got Terminal %v", err) + } + if !strings.Contains(err.Error(), "account retrieve") { + t.Fatalf("error should mention account retrieve, got %q", err.Error()) + } +} + +// TestSignedTxHasNoFeePayerOrGranter locks the signer-pays-its-own-fees +// invariant at the signed-bytes level. Without it a future contributor +// wiring WithFeePayer slips past the denom whitelist — the whitelist +// proves what denom is used, not who's paying. +func TestSignedTxHasNoFeePayerOrGranter(t *testing.T) { + cfg, addr := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h", Height: 0}, + // Return inclusion on first poll so the test doesn't wait + // the full inclusionPollTimeout. + queryDefault: &sdk.TxResponse{Code: 0, Height: 7}, + } + + _, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-0000000000fe", + }, addr) + if err != nil { + t.Fatalf("signAndBroadcast: %v", err) + } + if len(tc.lastTxBytes) == 0 { + t.Fatal("no tx bytes captured") + } + + // Assert raw proto fields, not sdk.FeeTx.FeePayer() — the interface + // method falls back to GetSigners()[0] when Fee.Payer is empty, so + // a future WithFeePayer(signer) plumbing would silently pass. + var pbTx txtypes.Tx + if err := pbTx.Unmarshal(tc.lastTxBytes); err != nil { + t.Fatalf("proto unmarshal tx: %v", err) + } + if pbTx.AuthInfo == nil || pbTx.AuthInfo.Fee == nil { + t.Fatalf("decoded tx missing AuthInfo.Fee") + } + if pbTx.AuthInfo.Fee.Payer != "" { + t.Errorf("AuthInfo.Fee.Payer = %q, want empty", pbTx.AuthInfo.Fee.Payer) + } + if pbTx.AuthInfo.Fee.Granter != "" { + t.Errorf("AuthInfo.Fee.Granter = %q, want empty", pbTx.AuthInfo.Fee.Granter) + } +} + +func TestBroadcastCheckTxFailure_ReturnsTerminal(t *testing.T) { + cfg, addr := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{ + Code: 11, + Codespace: "sdk", + RawLog: "insufficient fees", + }, + } + + _, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-000000000008", + }, addr) + if !IsTerminal(err) { + t.Fatalf("want Terminal checkTx error, got %v", err) + } + if !strings.Contains(err.Error(), "checkTx") { + t.Fatalf("error should mention checkTx, got %q", err.Error()) + } + if tc.broadcasts != 1 { + t.Fatalf("expected 1 broadcast, got %d", tc.broadcasts) + } +} + +func TestInclusionPollingCtxCancel_ReturnsCtxErr(t *testing.T) { + // ctx cancellation must surface so the engine records the task failed + // (rather than synthesizing "completed" on a truncated run). + cfg, addr := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h", Height: 0}, + } + + ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond) + defer cancel() + + start := time.Now() + _, err := signAndBroadcast(ctx, cfg, tc, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-00000000000c", + }, addr) + if !errors.Is(err, context.DeadlineExceeded) && !errors.Is(err, context.Canceled) { + t.Fatalf("expected ctx-cancellation error, got: %v", err) + } + if time.Since(start) > 2*time.Second { + t.Fatalf("test ran too long: %v (polling should bail on ctx.Done)", time.Since(start)) + } +} + +func TestInclusionPollingDeadline_ReturnsNonTerminalWithNilIncludedAt(t *testing.T) { + // Poll deadline (no ctx cancel) is "broadcast OK, inclusion deferred": + // non-Terminal success with IncludedAt=nil. + cfg, addr := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h", Height: 0}, + } + + prevTimeout, prevInterval := inclusionPollTimeout, inclusionPollInterval + inclusionPollTimeout = 50 * time.Millisecond + inclusionPollInterval = 10 * time.Millisecond + t.Cleanup(func() { + inclusionPollTimeout = prevTimeout + inclusionPollInterval = prevInterval + }) + + res, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "00000000-0000-0000-0000-00000000000d", + }, addr) + if err != nil { + t.Fatalf("poll-deadline (no ctx cancel) should NOT be an error: %v", err) + } + if res.IncludedAt != nil { + t.Fatal("IncludedAt must be nil when poll deadline elapsed without inclusion") + } +} + +func TestAppendTaskIDToMemo(t *testing.T) { + const id = "00000000-0000-0000-0000-000000000001" + cases := []struct { + base string + want string + }{ + {"", "taskID=" + id}, + {"vote-rationale", "vote-rationale taskID=" + id}, + } + for _, c := range cases { + if got := appendTaskIDToMemo(c.base, id); got != c.want { + t.Fatalf("appendTaskIDToMemo(%q,_) = %q, want %q", c.base, got, c.want) + } + } + // Empty taskID leaves base unchanged. + if got := appendTaskIDToMemo("base", ""); got != "base" { + t.Fatalf("empty taskID changed memo: %q", got) + } +} + +func TestCallerSuppliedTaskIDInMemoRejected(t *testing.T) { + // Caller cannot smuggle a "taskID=" tag into the memo — the audit + // trail must contain exactly the engine-appended tag. + cfg, addr := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{accountNumber: 17, sequence: 42} + + _, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + Memo: "rationale taskID=forged-uuid", + TaskID: "00000000-0000-0000-0000-0000000000aa", + }, addr) + if !IsTerminal(err) || !strings.Contains(err.Error(), "taskID=") { + t.Fatalf("want Terminal taskID-prefix rejection, got %v", err) + } +} + +func TestMemoCapEnforcedAfterTaskIDAppend(t *testing.T) { + // Caller's base is under 256 bytes but base + taskID exceeds the cap. + cfg, addr := newGuardCfg(t, "pacific-1") + tc := &fakeTxClient{accountNumber: 17, sequence: 42} + + base := strings.Repeat("a", 240) // leaves 16 bytes; "taskID=" is 7+36=43 + _, err := signAndBroadcast(context.Background(), cfg, tc, SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + Memo: base, + TaskID: "00000000-0000-0000-0000-000000000099", + }, addr) + if !IsTerminal(err) || !strings.Contains(err.Error(), "memo length") { + t.Fatalf("want Terminal memo-length error, got %v", err) + } +} + +// --- Idempotency marker / adopt tests -------------------------------- + +// shortenPolls shrinks the poll/re-query timeouts so marker tests that +// exercise the not-found paths stay fast, restoring them after the test. +func shortenPolls(t *testing.T) { + t.Helper() + pt, pi, aq := inclusionPollTimeout, inclusionPollInterval, adoptReQueryTimeout + inclusionPollTimeout = 50 * time.Millisecond + inclusionPollInterval = 5 * time.Millisecond + adoptReQueryTimeout = 50 * time.Millisecond + t.Cleanup(func() { + inclusionPollTimeout = pt + inclusionPollInterval = pi + adoptReQueryTimeout = aq + }) +} + +// markerCfg wires a checkpointer-backed ExecutionConfig around the given fake. +func markerCfg(t *testing.T, chainID string, tc *fakeTxClient) (engine.ExecutionConfig, sdk.AccAddress, *fakeCheckpointer) { + t.Helper() + cfg, addr := newGuardCfg(t, chainID) + cp := newFakeCheckpointer(tc) + cfg.Checkpointer = cp + return cfg, addr, cp +} + +func markerInput(t *testing.T, addr sdk.AccAddress) SignAndBroadcastInput { + t.Helper() + return SignAndBroadcastInput{ + ChainID: "pacific-1", + KeyName: "node_admin", + Msg: makeMsgVote(t, addr), + Fees: "4000usei", + Gas: 200_000, + TaskID: "task-1", + } +} + +// Test 1: adopt with the tx already on chain — build the result from the +// chain, never re-broadcast, never re-sign (broadcasts==0 is the proxy). +func TestAdopt_FoundOnChain_NoRebroadcast(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{queryDefault: &sdk.TxResponse{Code: 0, Height: 9, TxHash: "seed"}} + cfg, addr, cp := markerCfg(t, "pacific-1", tc) + cp.markers["task-1"] = &engine.TxMarker{ + TaskID: "task-1", TxHash: "ABCD", TxBytes: []byte{1, 2, 3}, ChainID: "pacific-1", + } + + res, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("adopt-found should succeed: %v", err) + } + if res.Height != 9 { + t.Fatalf("result should be built from chain (Height 9), got %d", res.Height) + } + if tc.broadcasts != 0 { + t.Fatalf("adopt-found must not re-broadcast; saw %d", tc.broadcasts) + } + if tc.queryCalls < 1 { + t.Fatalf("expected at least one QueryTx, got %d", tc.queryCalls) + } +} + +// Test 2: adopt not-found → re-broadcast the identical marker bytes. +func TestAdopt_NotFound_RebroadcastsIdenticalBytes(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{ + queryDefault: nil, // not found + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h"}, + } + cfg, addr, cp := markerCfg(t, "pacific-1", tc) + want := []byte{0xDE, 0xAD, 0xBE, 0xEF, 0x01, 0x02} + cp.markers["task-1"] = &engine.TxMarker{ + TaskID: "task-1", TxHash: "ABCD", TxBytes: want, + AccountNumber: 17, Sequence: 42, ChainID: "pacific-1", + } + + _, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("adopt not-found re-broadcast should succeed: %v", err) + } + if tc.broadcasts != 1 { + t.Fatalf("expected exactly 1 re-broadcast, got %d", tc.broadcasts) + } + if string(tc.lastTxBytes) != string(want) { + t.Fatalf("re-broadcast must use the marker's exact bytes; got %x want %x", tc.lastTxBytes, want) + } +} + +// Test 3 (M1/B1): adopt with a QueryTx transport error — report pending +// (inclusion-undetermined), never re-broadcast into uncertainty, never a bare +// error the controller can't distinguish from terminal. +func TestAdopt_QueryTransportError_ReportsPending_NoRebroadcast(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{queryErr: errors.New("rpc down")} + cfg, addr, cp := markerCfg(t, "pacific-1", tc) + cp.markers["task-1"] = &engine.TxMarker{ + TaskID: "task-1", TxHash: "ABCD", TxBytes: []byte{9, 9, 9}, ChainID: "pacific-1", + } + + res, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("transport error should report pending, not error: %v", err) + } + if res == nil || res.IncludedAt != nil { + t.Fatalf("expected an undetermined (pending) result, got %+v", res) + } + if res.TxHash != "ABCD" { + t.Fatalf("pending result must carry the marker txHash, got %q", res.TxHash) + } + if tc.broadcasts != 0 { + t.Fatalf("must NOT re-broadcast on unknown state; saw %d", tc.broadcasts) + } +} + +// txIndexDisabledErr mirrors the node's real /tx response when tx_index is off +// (tx_index.indexer = "null"): a JSON-RPC internal error whose Data names the +// missing kvEventSink. Observed on a validator with transaction indexing off. +func txIndexDisabledErr() *rpctypes.RPCError { + return &rpctypes.RPCError{ + Code: -32603, + Message: "Internal error", + Data: "transaction querying is disabled due to no kvEventSink", + } +} + +// TestIsTxIndexingDisabled pins the match against sei-tendermint's real +// disabled-index payloads — a reword would otherwise silently revert the fix to +// the retry-until-timeout bug. Both event-sink spellings must classify as +// disabled (terminal); crucially a message that says "not found" AND names the +// sink must too, since QueryTx checks this before isTxNotFound. A genuine miss +// on an indexed node and a transport error must not. +func TestIsTxIndexingDisabled(t *testing.T) { + cases := []struct { + name string + err error + want bool + }{ + {"pre-lookup guard (kvEventSink)", txIndexDisabledErr(), true}, + {"fallback guard (KV event sink)", &rpctypes.RPCError{Message: "Internal error", Data: "transaction querying is disabled on this node due to the KV event sink being disabled"}, true}, + {"not-found that also names the sink", &rpctypes.RPCError{Message: "Internal error", Data: "tx (ABCD) not found, err: no kvEventSink"}, true}, + {"genuine not-found on an indexed node", &rpctypes.RPCError{Message: "Internal error", Data: "tx (ABCD) not found, err: x"}, false}, + {"transport error", errors.New("connection refused"), false}, + {"nil", nil, false}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + if got := isTxIndexingDisabled(c.err); got != c.want { + t.Fatalf("isTxIndexingDisabled(%v) = %v, want %v", c.err, got, c.want) + } + }) + } +} + +// Adopt path: a marker exists but the node's tx index is off, so QueryTx yields +// errTxIndexingDisabled. signAndBroadcast must return an UNVERIFIABLE result — +// not a retryable pending one — without re-broadcasting. The pre-fix behavior +// reported inclusion-undetermined and retried to the task deadline, masking the +// outcome as a bare Timeout. (classifyGovResult turns the unverifiable result +// terminal — see gov_result_test.go.) +func TestAdopt_TxIndexingDisabled_Unverifiable(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{queryErr: errTxIndexingDisabled} + cfg, addr, cp := markerCfg(t, "pacific-1", tc) + cp.markers["task-1"] = &engine.TxMarker{ + TaskID: "task-1", TxHash: "ABCD", TxBytes: []byte{9, 9, 9}, ChainID: "pacific-1", + } + + res, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("unverifiable is carried on the result, not returned as an error: %v", err) + } + if res == nil || !res.Unverifiable || res.IncludedAt != nil { + t.Fatalf("expected an unverifiable result with IncludedAt nil, got %+v", res) + } + if res.TxHash != "ABCD" { + t.Fatalf("unverifiable result must carry the marker txHash, got %q", res.TxHash) + } + if tc.broadcasts != 0 { + t.Fatalf("must not re-broadcast when inclusion is unobservable; saw %d", tc.broadcasts) + } +} + +// Fresh path: no marker; broadcast accepted (CheckTx code 0) but the node's tx +// index is off, so inclusion can never be polled. Same unverifiable result — +// not pending-until-deadline. +func TestFresh_TxIndexingDisabled_Unverifiable(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{ + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h"}, + queryErr: errTxIndexingDisabled, + } + cfg, addr, _ := markerCfg(t, "pacific-1", tc) + + res, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("unverifiable is carried on the result, not an error: %v", err) + } + if res == nil || !res.Unverifiable || res.IncludedAt != nil { + t.Fatalf("expected an unverifiable result, got %+v", res) + } +} + +// B1: a broadcast transport error on the fresh path reports pending (the marker +// is durable, the tx may be in flight), not a bare error. +func TestBroadcastTransportError_ReportsPending(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{broadcastErr: errors.New("rpc down")} + cfg, addr, _ := markerCfg(t, "pacific-1", tc) + + res, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("broadcast transport error should report pending, not error: %v", err) + } + if res == nil || res.IncludedAt != nil || res.TxHash == "" { + t.Fatalf("expected an undetermined (pending) result with a txHash, got %+v", res) + } +} + +// Test 4 (H1): adopt not-found → re-broadcast is CheckTx-rejected (seq +// mismatch → Terminal), but the re-query guard finds the tx actually +// landed → the guard rescues it into a success. +func TestAdopt_RebroadcastRejectedButLanded_RequeryGuardRescues(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{ + queryFoundAfter: 1, // first QueryTx not-found, then found + queryDefault: &sdk.TxResponse{Code: 0, Height: 11, TxHash: "seed"}, + broadcastResp: &sdk.TxResponse{Code: 32}, // sequence mismatch → Terminal + } + cfg, addr, cp := markerCfg(t, "pacific-1", tc) + cp.markers["task-1"] = &engine.TxMarker{ + TaskID: "task-1", TxHash: "ABCD", TxBytes: []byte{1}, ChainID: "pacific-1", + } + + res, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("re-query guard should rescue a landed tx from a false Terminal fail: %v", err) + } + if res == nil || res.Height != 11 { + t.Fatalf("result should be built from the re-queried chain tx (Height 11), got %+v", res) + } + if tc.broadcasts != 1 { + t.Fatalf("expected exactly 1 re-broadcast, got %d", tc.broadcasts) + } +} + +// Test 5: fresh path persists the marker BEFORE broadcasting. +func TestFresh_MarkerPersistedBeforeBroadcast(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h"}, + queryDefault: &sdk.TxResponse{Code: 0, Height: 7}, // inclusion on first poll + } + cfg, addr, cp := markerCfg(t, "pacific-1", tc) + + _, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("fresh broadcast should succeed: %v", err) + } + if cp.saves != 1 { + t.Fatalf("expected exactly 1 SaveTxMarker, got %d", cp.saves) + } + if cp.saveBroadcasts != 0 { + t.Fatalf("marker must be saved before broadcast; broadcasts at save time = %d", cp.saveBroadcasts) + } + if tc.broadcasts != 1 { + t.Fatalf("expected 1 broadcast after save, got %d", tc.broadcasts) + } + if _, ok := cp.markers["task-1"]; !ok { + t.Fatal("marker for task-1 must exist in the checkpointer after broadcast") + } +} + +// Test 6: SaveTxMarker failure aborts before any broadcast. +func TestFresh_SaveMarkerFails_NoBroadcast(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h"}, + } + cfg, addr, cp := markerCfg(t, "pacific-1", tc) + cp.saveErr = errors.New("disk full") + + _, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err == nil { + t.Fatal("expected error when SaveTxMarker fails") + } + if tc.broadcasts != 0 { + t.Fatalf("must never broadcast without a durable marker; saw %d", tc.broadcasts) + } +} + +// Test 7: no checkpointer configured (back-compat) — broadcasts once, no panic. +func TestNoCheckpointer_BroadcastsOnce(t *testing.T) { + shortenPolls(t) + cfg, addr := newGuardCfg(t, "pacific-1") // no Checkpointer + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h"}, + queryDefault: &sdk.TxResponse{Code: 0, Height: 7}, + } + + res, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("back-compat path should succeed: %v", err) + } + if res == nil { + t.Fatal("expected a result") + } + if tc.broadcasts != 1 { + t.Fatalf("expected exactly 1 broadcast, got %d", tc.broadcasts) + } +} + +// Test 8 (L1): the persisted marker's TxHash equals sha256 of the exact +// bytes that were broadcast. +func TestFresh_MarkerHashMatchesBroadcastBytes(t *testing.T) { + shortenPolls(t) + tc := &fakeTxClient{ + accountNumber: 17, + sequence: 42, + broadcastResp: &sdk.TxResponse{Code: 0, TxHash: "h"}, + queryDefault: &sdk.TxResponse{Code: 0, Height: 7}, + } + cfg, addr, cp := markerCfg(t, "pacific-1", tc) + + _, err := signAndBroadcast(context.Background(), cfg, tc, markerInput(t, addr), addr) + if err != nil { + t.Fatalf("fresh broadcast should succeed: %v", err) + } + m, ok := cp.markers["task-1"] + if !ok { + t.Fatal("marker for task-1 missing") + } + want := fmt.Sprintf("%X", sha256.Sum256(tc.lastTxBytes)) + if m.TxHash != want { + t.Fatalf("marker TxHash %q != sha256 of broadcast bytes %q", m.TxHash, want) + } +} + +// makeAddr returns a syntactically valid sei-bech32 AccAddress without +// constructing a keyring entry. Used by guards that fail before any +// keyring lookup. +func makeAddr(t *testing.T) sdk.AccAddress { + t.Helper() + ensureBech32() + return sdk.AccAddress([]byte("aaaaaaaaaaaaaaaaaaaa")) +} diff --git a/sidecar/tasks/snapshot_restore.go b/sidecar/tasks/snapshot_restore.go new file mode 100644 index 00000000..adfad389 --- /dev/null +++ b/sidecar/tasks/snapshot_restore.go @@ -0,0 +1,330 @@ +package tasks + +import ( + "archive/tar" + "compress/gzip" + "context" + "fmt" + "io" + "os" + "path/filepath" + "regexp" + "strconv" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + "github.com/aws/aws-sdk-go-v2/service/s3" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +var restoreLog = seilog.NewLogger("seictl", "task", "snapshot-restore") + +const restoreMarkerFile = ".sei-sidecar-snapshot-done" + +// SnapshotHeightFile records the snapshot height the node was restored from. +// The result-export task uses this to know where to start exporting. +const SnapshotHeightFile = ".sei-sidecar-snapshot-height" + +// snapshotHeightRe extracts the block height from S3 snapshot keys of the form +// /state-sync/.tar.gz. The leading "/" anchor prevents the +// regex from picking up trailing digits embedded in other path segments. +var snapshotHeightRe = regexp.MustCompile(`/(\d+)\.tar\.gz$`) + +// SnapshotRestoreRequest holds the typed parameters for the snapshot-restore task. +// S3 bucket, region, and chain prefix are derived from the sidecar's environment. +// TargetHeight, when set, selects the highest available snapshot <= that height. +// When zero, the latest snapshot (from latest.txt) is used. +type SnapshotRestoreRequest struct { + TargetHeight int64 `json:"targetHeight,omitempty"` +} + +// SnapshotRestorer downloads and extracts a snapshot archive from S3. +type SnapshotRestorer struct { + homeDir string + bucket string + region string + chainID string + clientFactory seis3.TransferClientFactory + listerFactory seis3.ObjectListerFactory +} + +// NewSnapshotRestorer creates a restorer targeting the given home directory. +// Bucket, region, and chainID are read from environment at construction time. +func NewSnapshotRestorer(homeDir, bucket, region, chainID string, clientFactory seis3.TransferClientFactory, listerFactory seis3.ObjectListerFactory) (*SnapshotRestorer, error) { + if bucket == "" || region == "" || chainID == "" { + return nil, fmt.Errorf("snapshot-restore: bucket, region, and chainID are required") + } + if clientFactory == nil { + clientFactory = seis3.DefaultTransferClientFactory + } + if listerFactory == nil { + listerFactory = seis3.DefaultObjectListerFactory + } + return &SnapshotRestorer{ + homeDir: homeDir, + bucket: bucket, + region: region, + chainID: chainID, + clientFactory: clientFactory, + listerFactory: listerFactory, + }, nil +} + +// Handler returns an engine.TaskHandler for the snapshot-restore task. +func (r *SnapshotRestorer) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, req SnapshotRestoreRequest) error { + return r.Restore(ctx, req.TargetHeight) + }) +} + +// Restore downloads and extracts the snapshot, skipping if the marker file exists. +// It lists objects under the chain's state-sync prefix and picks the highest +// snapshot height; when targetHeight > 0, the search is capped at that height. +func (r *SnapshotRestorer) Restore(ctx context.Context, targetHeight int64) error { + if markerExists(r.homeDir, restoreMarkerFile) { + restoreLog.Debug("already completed, skipping") + return nil + } + + if targetHeight < 0 { + return fmt.Errorf("snapshot-restore: targetHeight must be >= 0, got %d", targetHeight) + } + + prefix := r.chainID + "/state-sync/" + + client, err := r.clientFactory(ctx, r.region) + if err != nil { + return fmt.Errorf("building S3 transfer client: %w", err) + } + + lister, err := r.listerFactory(ctx, r.region) + if err != nil { + return fmt.Errorf("building S3 lister: %w", err) + } + + snapshotKey, err := resolveKeyForHeight(ctx, lister, r.bucket, prefix, r.region, targetHeight) + if err != nil { + return err + } + + if snapshotKey == "" { + return fmt.Errorf("snapshot-restore: resolved snapshot key is empty for %s in s3://%s/%s", r.chainID, r.bucket, prefix) + } + + tmpDir := filepath.Join(r.homeDir, "tmp") + if err := os.MkdirAll(tmpDir, 0o755); err != nil { + return fmt.Errorf("creating temp dir: %w", err) + } + + tmpFile, err := os.CreateTemp(tmpDir, "snapshot-*.tar.gz") + if err != nil { + return fmt.Errorf("creating temp file: %w", err) + } + tmpPath := tmpFile.Name() + defer func() { _ = os.Remove(tmpPath) }() + + restoreLog.Info("downloading snapshot", "bucket", r.bucket, "key", snapshotKey, "dest", tmpPath) + _, err = client.DownloadObject(ctx, &transfermanager.DownloadObjectInput{ + Bucket: aws.String(r.bucket), + Key: aws.String(snapshotKey), + WriterAt: tmpFile, + }) + _ = tmpFile.Close() + if err != nil { + return seis3.ClassifyS3Error("snapshot-restore", r.bucket, snapshotKey, r.region, err) + } + + if h := parseHeightFromKey(snapshotKey); h > 0 { + if err := os.WriteFile( + filepath.Join(r.homeDir, SnapshotHeightFile), + []byte(strconv.FormatInt(h, 10)), + 0o644, + ); err != nil { + restoreLog.Warn("failed to write snapshot height file", "err", err) + } + } + + destDir := filepath.Join(r.homeDir, "data", "snapshots") + restoreLog.Info("extracting archive", "dest", destDir) + if err := extractArchive(ctx, tmpPath, destDir); err != nil { + return fmt.Errorf("extracting snapshot: %w", err) + } + + restoreLog.Info("restore complete") + return writeMarker(r.homeDir, restoreMarkerFile) +} + +// resolveKeyForHeight lists snapshot objects under prefix and returns the key +// with the highest parsed height. When targetHeight > 0 it caps the search at +// that height; targetHeight == 0 picks the highest available snapshot. +func resolveKeyForHeight(ctx context.Context, lister seis3.ObjectLister, bucket, prefix, region string, targetHeight int64) (string, error) { + var bestHeight int64 + var bestKey string + + var continuationToken *string + for { + output, err := lister.ListObjectsV2(ctx, &s3.ListObjectsV2Input{ + Bucket: aws.String(bucket), + Prefix: aws.String(prefix), + ContinuationToken: continuationToken, + }) + if err != nil { + return "", seis3.ClassifyS3Error("snapshot-restore", bucket, prefix, region, err) + } + + for _, obj := range output.Contents { + if obj.Key == nil { + continue + } + h := parseHeightFromKey(*obj.Key) + if h <= 0 { + continue + } + if targetHeight > 0 && h > targetHeight { + continue + } + if h > bestHeight { + bestHeight = h + bestKey = *obj.Key + } + } + + if !aws.ToBool(output.IsTruncated) { + break + } + continuationToken = output.NextContinuationToken + } + + if bestKey == "" { + if targetHeight > 0 { + return "", fmt.Errorf("no snapshot found at or below height %d in s3://%s/%s", targetHeight, bucket, prefix) + } + return "", fmt.Errorf("no snapshots found in s3://%s/%s", bucket, prefix) + } + + restoreLog.Info("resolved snapshot", + "targetHeight", targetHeight, "snapshotHeight", bestHeight, "key", bestKey) + return bestKey, nil +} + +func parseHeightFromKey(key string) int64 { + m := snapshotHeightRe.FindStringSubmatch(key) + if len(m) < 2 { + return 0 + } + h, err := strconv.ParseInt(m[1], 10, 64) + if err != nil { + return 0 + } + return h +} + +// extractArchive opens a .tar.gz file and extracts it to destDir. +func extractArchive(ctx context.Context, archivePath, destDir string) error { + f, err := os.Open(archivePath) + if err != nil { + return fmt.Errorf("opening archive: %w", err) + } + defer func() { _ = f.Close() }() + + return extractTarStream(ctx, f, destDir) +} + +func extractTarStream(ctx context.Context, r io.Reader, destDir string) error { + gzr, err := gzip.NewReader(r) + if err != nil { + return fmt.Errorf("creating gzip reader: %w", err) + } + defer func() { _ = gzr.Close() }() + tr := tar.NewReader(gzr) + for { + if err := ctx.Err(); err != nil { + return err + } + + header, err := tr.Next() + if err == io.EOF { + return nil + } + if err != nil { + return fmt.Errorf("reading tar header: %w", err) + } + + target := filepath.Join(destDir, filepath.Clean(header.Name)) + + if !isInsideDir(target, destDir) { + return fmt.Errorf("tar entry %q escapes destination directory", header.Name) + } + + switch header.Typeflag { + case tar.TypeDir: + if err := os.MkdirAll(target, os.FileMode(header.Mode)|0o700); err != nil { + return fmt.Errorf("creating directory %s: %w", target, err) + } + case tar.TypeReg: + if err := extractFile(tr, target, os.FileMode(header.Mode)); err != nil { + return err + } + case tar.TypeSymlink: + linkTarget := filepath.Join(filepath.Dir(target), header.Linkname) + if !isInsideDir(linkTarget, destDir) { + return fmt.Errorf("symlink %q points outside destination directory", header.Name) + } + if err := os.Symlink(header.Linkname, target); err != nil { + return fmt.Errorf("creating symlink %s: %w", target, err) + } + } + } +} + +func extractFile(r io.Reader, path string, mode os.FileMode) error { + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return fmt.Errorf("creating parent directory for %s: %w", path, err) + } + f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, mode|0o600) + if err != nil { + return fmt.Errorf("creating file %s: %w", path, err) + } + defer func() { _ = f.Close() }() + if _, err := io.Copy(f, r); err != nil { + return fmt.Errorf("writing file %s: %w", path, err) + } + return nil +} + +// isInsideDir checks that target is within or equal to baseDir. +func isInsideDir(target, baseDir string) bool { + absTarget, err := filepath.Abs(target) + if err != nil { + return false + } + absBase, err := filepath.Abs(baseDir) + if err != nil { + return false + } + if absTarget == absBase { + return true + } + rel, err := filepath.Rel(absBase, absTarget) + if err != nil { + return false + } + return len(rel) > 0 && rel[0] != '.' +} + +func markerExists(homeDir, name string) bool { + _, err := os.Stat(filepath.Join(homeDir, name)) + return err == nil +} + +func writeMarker(homeDir, name string) error { + path := filepath.Join(homeDir, name) + f, err := os.Create(path) + if err != nil { + return fmt.Errorf("writing marker file %s: %w", path, err) + } + return f.Close() +} diff --git a/sidecar/tasks/snapshot_restore_test.go b/sidecar/tasks/snapshot_restore_test.go new file mode 100644 index 00000000..cbc34653 --- /dev/null +++ b/sidecar/tasks/snapshot_restore_test.go @@ -0,0 +1,438 @@ +package tasks + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "context" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + "github.com/aws/aws-sdk-go-v2/service/s3" + "github.com/aws/aws-sdk-go-v2/service/s3/types" + + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +// mockTransferClient implements seis3.TransferClient for testing. +type mockTransferClient struct { + responses map[string][]byte + errDefault error +} + +func (m *mockTransferClient) DownloadObject(_ context.Context, in *transfermanager.DownloadObjectInput, _ ...func(*transfermanager.Options)) (*transfermanager.DownloadObjectOutput, error) { + key := "" + if in.Key != nil { + key = *in.Key + } + if body, ok := m.responses[key]; ok { + if _, err := in.WriterAt.WriteAt(body, 0); err != nil { + return nil, fmt.Errorf("writing to WriterAt: %w", err) + } + return &transfermanager.DownloadObjectOutput{}, nil + } + if m.errDefault != nil { + return nil, m.errDefault + } + return nil, fmt.Errorf("unexpected key: %s", key) +} + +func mockClientFactory(client seis3.TransferClient) seis3.TransferClientFactory { + return func(_ context.Context, _ string) (seis3.TransferClient, error) { + return client, nil + } +} + +// mockObjectLister implements seis3.ObjectLister for testing. +// pageSize controls pagination — 0 means return all keys in one page. +type mockObjectLister struct { + keys []string + pageSize int +} + +func (m *mockObjectLister) ListObjectsV2(_ context.Context, input *s3.ListObjectsV2Input, _ ...func(*s3.Options)) (*s3.ListObjectsV2Output, error) { + pageSize := m.pageSize + if pageSize <= 0 { + pageSize = len(m.keys) + } + + startIdx := 0 + if input.ContinuationToken != nil { + for i, k := range m.keys { + if k == *input.ContinuationToken { + startIdx = i + break + } + } + } + + end := startIdx + pageSize + if end > len(m.keys) { + end = len(m.keys) + } + + var contents []types.Object + for _, k := range m.keys[startIdx:end] { + key := k + contents = append(contents, types.Object{Key: &key}) + } + + truncated := end < len(m.keys) + var nextToken *string + if truncated { + nextToken = &m.keys[end] + } + + return &s3.ListObjectsV2Output{ + Contents: contents, + IsTruncated: &truncated, + NextContinuationToken: nextToken, + }, nil +} + +func mustNewRestorer(t *testing.T, homeDir, bucket, region, chainID string, cf seis3.TransferClientFactory, lf seis3.ObjectListerFactory) *SnapshotRestorer { + t.Helper() + r, err := NewSnapshotRestorer(homeDir, bucket, region, chainID, cf, lf) + if err != nil { + t.Fatalf("NewSnapshotRestorer: %v", err) + } + return r +} + +func mockListerFactory(lister seis3.ObjectLister) seis3.ObjectListerFactory { + return func(_ context.Context, _ string) (seis3.ObjectLister, error) { + return lister, nil + } +} + +func buildTarGzArchive(t *testing.T, files map[string]string) []byte { + t.Helper() + var buf bytes.Buffer + gzw := gzip.NewWriter(&buf) + tw := tar.NewWriter(gzw) + for name, content := range files { + if err := tw.WriteHeader(&tar.Header{ + Name: name, + Mode: 0o644, + Size: int64(len(content)), + }); err != nil { + t.Fatalf("writing tar header for %s: %v", name, err) + } + if _, err := tw.Write([]byte(content)); err != nil { + t.Fatalf("writing tar content for %s: %v", name, err) + } + } + if err := tw.Close(); err != nil { + t.Fatalf("closing tar writer: %v", err) + } + if err := gzw.Close(); err != nil { + t.Fatalf("closing gzip writer: %v", err) + } + return buf.Bytes() +} + +func TestSnapshotRestoreExtractsArchive(t *testing.T) { + homeDir := t.TempDir() + archive := buildTarGzArchive(t, map[string]string{ + "data/chain.db": "chaindata", + }) + + client := &mockTransferClient{ + responses: map[string][]byte{ + "testchain/state-sync/100000000.tar.gz": archive, + }, + } + lister := &mockObjectLister{ + keys: []string{ + "testchain/state-sync/100000000.tar.gz", + }, + } + restorer := mustNewRestorer(t, homeDir, "test-bucket", "us-east-1", "testchain", mockClientFactory(client), mockListerFactory(lister)) + if err := restorer.Restore(context.Background(), 0); err != nil { + t.Fatalf("Restore failed: %v", err) + } + + content, err := os.ReadFile(filepath.Join(homeDir, "data", "snapshots", "data", "chain.db")) + if err != nil { + t.Fatalf("reading extracted file: %v", err) + } + if string(content) != "chaindata" { + t.Fatalf("expected 'chaindata', got %q", string(content)) + } + + if !markerExists(homeDir, restoreMarkerFile) { + t.Fatal("marker file should exist after successful restore") + } +} + +func TestSnapshotRestoreSkipsWhenMarkerExists(t *testing.T) { + homeDir := t.TempDir() + if err := writeMarker(homeDir, restoreMarkerFile); err != nil { + t.Fatalf("writing marker: %v", err) + } + + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", mockClientFactory(&mockTransferClient{ + errDefault: fmt.Errorf("should not be called"), + }), nil) + + if err := restorer.Restore(context.Background(), 0); err != nil { + t.Fatalf("expected nil error when marker exists, got: %v", err) + } +} + +func TestSnapshotRestoreNoMarkerWhenBucketIsEmpty(t *testing.T) { + homeDir := t.TempDir() + lister := &mockObjectLister{keys: []string{}} + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", nil, mockListerFactory(lister)) + + if err := restorer.Restore(context.Background(), 0); err == nil { + t.Fatal("expected error when no snapshots are present") + } + + if markerExists(homeDir, restoreMarkerFile) { + t.Fatal("marker file should not exist after failed restore") + } +} + +func TestSnapshotRestoreNoMarkerOnDownloadError(t *testing.T) { + homeDir := t.TempDir() + client := &mockTransferClient{ + errDefault: fmt.Errorf("access denied"), + } + lister := &mockObjectLister{ + keys: []string{ + "c/state-sync/100000000.tar.gz", + }, + } + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", mockClientFactory(client), mockListerFactory(lister)) + + if err := restorer.Restore(context.Background(), 0); err == nil { + t.Fatal("expected error on snapshot download failure") + } + + if markerExists(homeDir, restoreMarkerFile) { + t.Fatal("marker file should not exist after failed restore") + } +} + +func TestSnapshotRestoreRejectsPathTraversal(t *testing.T) { + homeDir := t.TempDir() + archive := buildTarGzArchive(t, map[string]string{ + "../../etc/passwd": "malicious", + }) + + client := &mockTransferClient{ + responses: map[string][]byte{ + "c/state-sync/100000000.tar.gz": archive, + }, + } + lister := &mockObjectLister{ + keys: []string{"c/state-sync/100000000.tar.gz"}, + } + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", mockClientFactory(client), mockListerFactory(lister)) + if err := restorer.Restore(context.Background(), 0); err == nil { + t.Fatal("expected error for path traversal attempt") + } +} + +func TestSnapshotRestoreCleansUpTempFile(t *testing.T) { + homeDir := t.TempDir() + archive := buildTarGzArchive(t, map[string]string{ + "data/chain.db": "chaindata", + }) + + client := &mockTransferClient{ + responses: map[string][]byte{ + "c/state-sync/100000000.tar.gz": archive, + }, + } + lister := &mockObjectLister{ + keys: []string{"c/state-sync/100000000.tar.gz"}, + } + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", mockClientFactory(client), mockListerFactory(lister)) + if err := restorer.Restore(context.Background(), 0); err != nil { + t.Fatalf("Restore failed: %v", err) + } + + tmpDir := filepath.Join(homeDir, "tmp") + entries, err := os.ReadDir(tmpDir) + if err != nil && !os.IsNotExist(err) { + t.Fatalf("reading tmp dir: %v", err) + } + for _, e := range entries { + if matched, _ := filepath.Match("snapshot-*.tar.gz", e.Name()); matched { + t.Fatalf("temp file %s was not cleaned up", e.Name()) + } + } +} + +func TestSnapshotRestoreWritesHeightFile(t *testing.T) { + homeDir := t.TempDir() + archive := buildTarGzArchive(t, map[string]string{ + "data/chain.db": "chaindata", + }) + + client := &mockTransferClient{ + responses: map[string][]byte{ + "c/state-sync/100000000.tar.gz": archive, + }, + } + lister := &mockObjectLister{ + keys: []string{"c/state-sync/100000000.tar.gz"}, + } + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", mockClientFactory(client), mockListerFactory(lister)) + if err := restorer.Restore(context.Background(), 0); err != nil { + t.Fatalf("Restore failed: %v", err) + } + + heightBytes, err := os.ReadFile(filepath.Join(homeDir, SnapshotHeightFile)) + if err != nil { + t.Fatalf("reading snapshot height file: %v", err) + } + if string(heightBytes) != "100000000" { + t.Errorf("snapshot height file = %q, want %q", string(heightBytes), "100000000") + } +} + +func TestSnapshotRestoreWithTargetHeight(t *testing.T) { + homeDir := t.TempDir() + archive := buildTarGzArchive(t, map[string]string{ + "data/chain.db": "chaindata", + }) + + client := &mockTransferClient{ + responses: map[string][]byte{ + "c/state-sync/99000000.tar.gz": archive, + }, + } + lister := &mockObjectLister{ + keys: []string{ + "c/state-sync/98000000.tar.gz", + "c/state-sync/99000000.tar.gz", + "c/state-sync/100000000.tar.gz", + "c/state-sync/latest.txt", + }, + } + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", mockClientFactory(client), mockListerFactory(lister)) + // Target 99500000 — should pick 99000000 (highest <= target) + if err := restorer.Restore(context.Background(), 99500000); err != nil { + t.Fatalf("Restore failed: %v", err) + } + + heightBytes, err := os.ReadFile(filepath.Join(homeDir, SnapshotHeightFile)) + if err != nil { + t.Fatalf("reading snapshot height file: %v", err) + } + if string(heightBytes) != "99000000" { + t.Errorf("snapshot height = %q, want %q", string(heightBytes), "99000000") + } +} + +func TestSnapshotRestoreTargetHeightNoMatch(t *testing.T) { + homeDir := t.TempDir() + lister := &mockObjectLister{ + keys: []string{ + "c/state-sync/100000000.tar.gz", + "c/state-sync/200000000.tar.gz", + }, + } + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", nil, mockListerFactory(lister)) + // Target 50000000 — no snapshots at or below + err := restorer.Restore(context.Background(), 50000000) + if err == nil { + t.Fatal("expected error when no snapshot found at or below target height") + } + if !strings.Contains(err.Error(), "no snapshot found") { + t.Fatalf("expected 'no snapshot found' error, got: %v", err) + } +} + +func TestSnapshotRestoreTargetHeightPagination(t *testing.T) { + homeDir := t.TempDir() + archive := buildTarGzArchive(t, map[string]string{ + "data/chain.db": "chaindata", + }) + + client := &mockTransferClient{ + responses: map[string][]byte{ + "c/state-sync/99000000.tar.gz": archive, + }, + } + // Page size of 2 forces pagination across 3 pages + lister := &mockObjectLister{ + pageSize: 2, + keys: []string{ + "c/state-sync/97000000.tar.gz", + "c/state-sync/98000000.tar.gz", + "c/state-sync/99000000.tar.gz", + "c/state-sync/100000000.tar.gz", + "c/state-sync/latest.txt", + }, + } + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", mockClientFactory(client), mockListerFactory(lister)) + if err := restorer.Restore(context.Background(), 99500000); err != nil { + t.Fatalf("Restore failed: %v", err) + } + + heightBytes, err := os.ReadFile(filepath.Join(homeDir, SnapshotHeightFile)) + if err != nil { + t.Fatalf("reading snapshot height file: %v", err) + } + if string(heightBytes) != "99000000" { + t.Errorf("snapshot height = %q, want %q", string(heightBytes), "99000000") + } +} + +func TestSnapshotRestoreNegativeTargetHeight(t *testing.T) { + homeDir := t.TempDir() + restorer := mustNewRestorer(t, homeDir, "b", "r", "c", nil, nil) + err := restorer.Restore(context.Background(), -1) + if err == nil { + t.Fatal("expected error for negative targetHeight") + } +} + +func TestParseHeightFromKey(t *testing.T) { + cases := []struct { + name string + key string + want int64 + }{ + { + name: "canonical key parses to height", + key: "pacific-1/state-sync/205082000.tar.gz", + want: 205082000, + }, + { + name: "key without slash before digits returns zero", + key: "pacific-1/eu-central-1.tar.gz", + want: 0, + }, + { + name: "long-form publisher key from before the cutover returns zero", + key: "pacific-1/state-sync/snapshot_201525000_pacific-1_eu-central-1.tar.gz", + want: 0, + }, + { + name: "non-snapshot tar.gz returns zero", + key: "pacific-1/state-sync/some-other-file.tar.gz", + want: 0, + }, + { + name: "latest.txt returns zero", + key: "pacific-1/state-sync/latest.txt", + want: 0, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := parseHeightFromKey(tc.key); got != tc.want { + t.Errorf("parseHeightFromKey(%q) = %d, want %d", tc.key, got, tc.want) + } + }) + } +} diff --git a/sidecar/tasks/snapshot_upload.go b/sidecar/tasks/snapshot_upload.go new file mode 100644 index 00000000..b2a270b9 --- /dev/null +++ b/sidecar/tasks/snapshot_upload.go @@ -0,0 +1,494 @@ +package tasks + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "sort" + "strconv" + "strings" + "time" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/wire" +) + +var uploadLog = seilog.NewLogger("seictl", "task", "snapshot-upload") + +const ( + uploadStateFile = ".sei-sidecar-last-upload.json" + defaultUploadInterval = 7 * 24 * time.Hour // weekly + + // defaultUploadTimeout bounds a single one-shot upload. Uploads on large + // chains legitimately run 60-90 min, so the default is generous; a wedged + // S3 stream fails at the deadline rather than sitting 'running' forever. + defaultUploadTimeout = 2 * time.Hour +) + +// SnapshotUploadRequest holds the parameters for the snapshot upload task. +// S3 bucket, region, and prefix are derived from the sidecar's environment. +type SnapshotUploadRequest struct{} + +// UploadOutcome and NoopReason are the snapshot-upload result-wire contract. +// They live in sidecar/wire (the dependency-free contract home) and are aliased +// here so handler call sites and the CLI poller reference one definition. +type ( + UploadOutcome = wire.UploadOutcome + NoopReason = wire.NoopReason +) + +const ( + OutcomeUploaded = wire.OutcomeUploaded + OutcomeNoop = wire.OutcomeNoop + OutcomeError = wire.OutcomeError + + NoopFewerThanTwoSnapshots = wire.NoopFewerThanTwoSnapshots + NoopAlreadyUploaded = wire.NoopAlreadyUploaded +) + +// SnapshotUploadResult is the structured result both handlers return through the +// engine so a one-shot poller can distinguish uploaded / noop / error: an error +// return carries Outcome=OutcomeError alongside the error string. On the loop +// path it is discarded; the engine persists it on TaskResult.Result for the +// one-shot path. +type SnapshotUploadResult struct { + Outcome UploadOutcome `json:"outcome"` + NoopReason NoopReason `json:"noopReason,omitempty"` + Height int64 `json:"height,omitempty"` + Key string `json:"key,omitempty"` +} + +// uploadState tracks the last successfully uploaded snapshot height and when it +// was uploaded. LastUploadedAt (unix seconds) is persisted so the uploaded +// gauges can be re-emitted on sidecar startup, avoiding a false-stale reading +// after the uploading pod restarts. +type uploadState struct { + LastUploadedHeight int64 `json:"lastUploadedHeight"` + LastUploadedAt int64 `json:"lastUploadedAt,omitempty"` +} + +// SnapshotUploader scans for locally produced Tendermint state-sync snapshots +// and uploads new ones to S3. When submitted as a task, it runs in a loop +// at the configured interval until the context is cancelled. +type SnapshotUploader struct { + homeDir string + bucket string + region string + chainID string + uploadInterval time.Duration + s3UploaderFactory seis3.UploaderFactory +} + +// NewSnapshotUploader creates an uploader targeting the given home directory. +// Bucket, region, and chainID are read from environment at construction time +// and rejected here if empty so the caller fails fast rather than entering +// runLoop and uploading nothing forever. +func NewSnapshotUploader(homeDir, bucket, region, chainID string, uploadInterval time.Duration, factory seis3.UploaderFactory) (*SnapshotUploader, error) { + if bucket == "" || region == "" || chainID == "" { + return nil, fmt.Errorf("snapshot-upload: bucket, region, and chainID are required") + } + if factory == nil { + factory = seis3.DefaultUploaderFactory + } + if uploadInterval <= 0 { + uploadInterval = defaultUploadInterval + } + return &SnapshotUploader{ + homeDir: homeDir, + bucket: bucket, + region: region, + chainID: chainID, + uploadInterval: uploadInterval, + s3UploaderFactory: factory, + }, nil +} + +// Handler returns an engine.TaskHandler for the snapshot-upload task. +// The handler runs in a loop, attempting an upload on each tick and +// sleeping for the configured interval between attempts. It stays +// running until the context is cancelled. +func (u *SnapshotUploader) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, _ SnapshotUploadRequest) error { + return u.runLoop(ctx) + }) +} + +// OnceHandler returns an engine.TaskHandler for the one-shot snapshot-upload +// task. It runs Upload exactly once and returns the structured result so the +// task reaches a real terminal (completed with an outcome, or failed with the +// error). The execution is bounded by a handler-internal deadline so a wedged +// S3 stream fails cleanly rather than stranding the task in 'running'. The +// deadline lives on a child context: it surfaces as context.DeadlineExceeded, +// which the engine persists as Failed (its cancellation-suppression guard keys +// only on context.Canceled). +func (u *SnapshotUploader) OnceHandler(timeout time.Duration) engine.TaskHandler { + if timeout <= 0 { + timeout = defaultUploadTimeout + } + return engine.TypedHandlerWithResult(func(ctx context.Context, _ SnapshotUploadRequest) (SnapshotUploadResult, error) { + ctx, cancel := context.WithTimeout(ctx, timeout) + defer cancel() + return u.Upload(ctx) + }) +} + +func (u *SnapshotUploader) runLoop(ctx context.Context) error { + uploadLog.Info("starting snapshot upload loop", "interval", u.uploadInterval, "bucket", u.bucket) + for { + if _, err := u.Upload(ctx); err != nil { + uploadLog.Warn("upload attempt failed, will retry next interval", "error", err) + } + + select { + case <-ctx.Done(): + return ctx.Err() + case <-time.After(u.uploadInterval): + } + } +} + +// Upload finds the latest complete snapshot, archives it, and streams it to S3. +// It picks the second-to-latest snapshot height to avoid uploading an +// in-progress snapshot. If the snapshot has already been uploaded (tracked +// via a local state file), it no-ops. +// +// The archive is streamed through an io.Pipe so it never needs to be buffered +// entirely in memory; the transfermanager handles multipart upload automatically. +func (u *SnapshotUploader) Upload(ctx context.Context) (SnapshotUploadResult, error) { + snapshotsDir := filepath.Join(u.homeDir, "data", "snapshots") + + height, err := pickUploadCandidate(snapshotsDir) + if err != nil { + return u.recordError(ctx), err + } + if height == 0 { + uploadLog.Debug("fewer than 2 snapshots on disk, nothing to upload") + return u.recordTerminal(SnapshotUploadResult{Outcome: OutcomeNoop, NoopReason: NoopFewerThanTwoSnapshots}), nil + } + + last := u.readUploadState() + if last.LastUploadedHeight >= height { + uploadLog.Debug("height already uploaded", "height", height, "last-uploaded", last.LastUploadedHeight) + return u.recordTerminal(SnapshotUploadResult{Outcome: OutcomeNoop, NoopReason: NoopAlreadyUploaded, Height: height}), nil + } + + uploadLog.Info("uploading snapshot", "height", height, "bucket", u.bucket, "region", u.region) + + uploader, err := u.s3UploaderFactory(ctx, u.region) + if err != nil { + return u.recordError(ctx), fmt.Errorf("building S3 uploader: %w", err) + } + + prefix := u.chainID + "/state-sync/" + + archiveKey := fmt.Sprintf("%s%d.tar.gz", prefix, height) + uploadLog.Info("streaming archive to S3", "key", archiveKey) + if err := u.streamUpload(ctx, uploader, u.bucket, archiveKey, snapshotsDir, height); err != nil { + return u.recordError(ctx), fmt.Errorf("uploading %s: %w", archiveKey, err) + } + + latestKey := prefix + "latest.txt" + latestBody := []byte(strconv.FormatInt(height, 10)) + _, err = uploader.UploadObject(ctx, &transfermanager.UploadObjectInput{ + Bucket: aws.String(u.bucket), + Key: aws.String(latestKey), + Body: bytes.NewReader(latestBody), + }) + if err != nil { + return u.recordError(ctx), fmt.Errorf("uploading %s: %w", latestKey, err) + } + uploadLog.Info("updated latest.txt", "key", latestKey, "height", height) + + if err := u.writeUploadState(uploadState{LastUploadedHeight: height, LastUploadedAt: time.Now().Unix()}); err != nil { + return u.recordError(ctx), err + } + + return u.recordTerminal(SnapshotUploadResult{Outcome: OutcomeUploaded, Height: height, Key: archiveKey}), nil +} + +// recordTerminal emits the metrics for a clean terminal and returns the result +// unchanged so callers can `return u.recordTerminal(...), nil` in one line. Any +// clean terminal (uploaded or noop) refreshes the last-run-success gauge and the +// outcome counter; only a real upload advances the uploaded gauges. +func (u *SnapshotUploader) recordTerminal(result SnapshotUploadResult) SnapshotUploadResult { + now := time.Now() + snapshotUploadLastRunSuccess.WithLabelValues(u.chainID).Set(float64(now.Unix())) + snapshotUploadOutcomes.WithLabelValues(u.chainID, string(result.Outcome)).Inc() + if result.Outcome == OutcomeUploaded { + snapshotUploadLastUploaded.WithLabelValues(u.chainID).Set(float64(now.Unix())) + snapshotUploadLastUploadedHeight.WithLabelValues(u.chainID).Set(float64(result.Height)) + } + return result +} + +// recordError increments the chain-labeled outcome counter for a failed Upload +// and returns an error-tagged result so callers can `return u.recordError(ctx), +// err` in one line. It deliberately leaves last-run-success and the uploaded +// gauges untouched — those are clean-terminal-only signals — so a failing +// uploader is visible on the outcome counter without falsely refreshing health. +// +// A context.Canceled error does not tick the counter: it mirrors the engine's +// cancellation-suppression contract (a canceled task leaves no terminal record), +// so a deploy draining a node or an operator DELETE mid-upload is not counted as +// an upload failure. The predicate matches the engine's exactly — only +// context.Canceled is suppressed; a context.DeadlineExceeded (per-task timeout) +// or a genuine S3 failure still ticks. +func (u *SnapshotUploader) recordError(ctx context.Context) SnapshotUploadResult { + if !errors.Is(ctx.Err(), context.Canceled) { + snapshotUploadOutcomes.WithLabelValues(u.chainID, string(OutcomeError)).Inc() + } + return SnapshotUploadResult{Outcome: OutcomeError} +} + +// EmitStartupMetrics re-emits the last-uploaded gauges from persisted state so a +// restarted sidecar does not report a false-stale reading before its first run. +// The last-run-success gauge is deliberately left unset: it is the "no clean run +// in N hours" alert signal, and re-emitting a persisted timestamp there would +// mask a genuinely stalled uploader after a restart. +func (u *SnapshotUploader) EmitStartupMetrics() { + st := u.readUploadState() + if st.LastUploadedHeight <= 0 { + return + } + snapshotUploadLastUploadedHeight.WithLabelValues(u.chainID).Set(float64(st.LastUploadedHeight)) + if st.LastUploadedAt > 0 { + snapshotUploadLastUploaded.WithLabelValues(u.chainID).Set(float64(st.LastUploadedAt)) + } +} + +// streamUpload pipes a tar.gz archive directly into the transfermanager, +// avoiding in-memory buffering of the full archive. +func (u *SnapshotUploader) streamUpload(ctx context.Context, uploader seis3.Uploader, bucket, key, snapshotsDir string, height int64) error { + pr, pw := io.Pipe() + + archiveErr := make(chan error, 1) + go func() { + archiveErr <- writeArchive(ctx, pw, snapshotsDir, height) + }() + + _, uploadErr := uploader.UploadObject(ctx, &transfermanager.UploadObjectInput{ + Bucket: aws.String(bucket), + Key: aws.String(key), + Body: pr, + ContentType: aws.String("application/gzip"), + }) + + if uploadErr != nil { + // Unblock the archiver goroutine if it's still writing. + pr.CloseWithError(uploadErr) + } + + aErr := <-archiveErr + if uploadErr != nil { + return uploadErr + } + return aErr +} + +// pickUploadCandidate scans the snapshots directory and returns the +// second-to-latest height. This avoids uploading an in-progress snapshot. +// Returns 0 if fewer than 2 snapshots exist. +func pickUploadCandidate(snapshotsDir string) (int64, error) { + entries, err := os.ReadDir(snapshotsDir) + if err != nil { + if os.IsNotExist(err) { + return 0, nil + } + return 0, fmt.Errorf("reading snapshots directory: %w", err) + } + + var heights []int64 + for _, e := range entries { + if !e.IsDir() { + continue + } + h, err := strconv.ParseInt(e.Name(), 10, 64) + if err != nil { + continue // skip non-numeric directories + } + heights = append(heights, h) + } + + if len(heights) < 2 { + return 0, nil + } + + sort.Slice(heights, func(i, j int) bool { return heights[i] < heights[j] }) + return heights[len(heights)-2], nil +} + +// writeArchive streams a tar.gz archive of the snapshot at the given height +// into wc (typically the write half of an io.Pipe). It always closes wc when +// done, propagating any archiving error so the reader side sees it. +func writeArchive(ctx context.Context, wc io.WriteCloser, snapshotsDir string, height int64) (retErr error) { + defer func() { + if retErr != nil { + wc.(*io.PipeWriter).CloseWithError(retErr) + } else { + _ = wc.Close() + } + }() + + gw := gzip.NewWriter(wc) + tw := tar.NewWriter(gw) + + heightDir := filepath.Join(snapshotsDir, strconv.FormatInt(height, 10)) + if err := addDirToTar(ctx, tw, heightDir, strconv.FormatInt(height, 10)); err != nil { + return err + } + + // metadata.db has been a LevelDB directory in cosmos-sdk for several + // versions, but the API allows it to be a single file too. Dispatch + // on whichever we observe so a future revert doesn't break us either way. + metadataPath := filepath.Join(snapshotsDir, "metadata.db") + if info, err := os.Stat(metadataPath); err == nil { + var addErr error + if info.IsDir() { + addErr = addDirToTar(ctx, tw, metadataPath, "metadata.db") + } else { + addErr = addFileToTar(ctx, tw, metadataPath, "metadata.db", info) + } + if addErr != nil { + return fmt.Errorf("archiving metadata.db: %w", addErr) + } + } + + if err := tw.Close(); err != nil { + return fmt.Errorf("closing tar writer: %w", err) + } + if err := gw.Close(); err != nil { + return fmt.Errorf("closing gzip writer: %w", err) + } + return nil +} + +func addDirToTar(ctx context.Context, tw *tar.Writer, dir, base string) error { + return filepath.Walk(dir, func(path string, info os.FileInfo, err error) error { + if err != nil { + return err + } + if err := ctx.Err(); err != nil { + return err + } + + rel, err := filepath.Rel(filepath.Dir(dir), path) + if err != nil { + return err + } + + header, err := tar.FileInfoHeader(info, "") + if err != nil { + return err + } + header.Name = rel + + if err := tw.WriteHeader(header); err != nil { + return err + } + if info.IsDir() { + return nil + } + + f, err := os.Open(path) + if err != nil { + return err + } + defer func() { _ = f.Close() }() + _, err = io.Copy(tw, f) + return err + }) +} + +func addFileToTar(ctx context.Context, tw *tar.Writer, path, name string, info os.FileInfo) error { + if err := ctx.Err(); err != nil { + return err + } + header, err := tar.FileInfoHeader(info, "") + if err != nil { + return err + } + header.Name = name + if err := tw.WriteHeader(header); err != nil { + return err + } + f, err := os.Open(path) + if err != nil { + return err + } + defer func() { _ = f.Close() }() + _, err = io.Copy(tw, f) + return err +} + +func normalizePrefix(prefix string) string { + if prefix == "" { + return "" + } + if !strings.HasSuffix(prefix, "/") { + return prefix + "/" + } + return prefix +} + +func (u *SnapshotUploader) readUploadState() uploadState { + data, err := os.ReadFile(filepath.Join(u.homeDir, uploadStateFile)) + if err != nil { + return uploadState{} + } + var state uploadState + if err := json.Unmarshal(data, &state); err != nil { + return uploadState{} + } + return state +} + +// writeUploadState persists state atomically: a temp file in the same directory +// is written, synced, and renamed over the target. A crash mid-write leaves the +// previous complete state intact rather than a torn file that readUploadState +// would parse as zero, silently re-uploading everything from height 0. +func (u *SnapshotUploader) writeUploadState(state uploadState) error { + data, err := json.Marshal(state) + if err != nil { + return fmt.Errorf("marshaling upload state: %w", err) + } + + tmp, err := os.CreateTemp(u.homeDir, uploadStateFile+".tmp-*") + if err != nil { + return fmt.Errorf("creating temp upload state: %w", err) + } + tmpName := tmp.Name() + defer func() { _ = os.Remove(tmpName) }() // no-op once the rename succeeds + + if _, err := tmp.Write(data); err != nil { + _ = tmp.Close() + return fmt.Errorf("writing temp upload state: %w", err) + } + if err := tmp.Chmod(0o644); err != nil { + _ = tmp.Close() + return fmt.Errorf("chmod temp upload state: %w", err) + } + if err := tmp.Sync(); err != nil { + _ = tmp.Close() + return fmt.Errorf("syncing temp upload state: %w", err) + } + if err := tmp.Close(); err != nil { + return fmt.Errorf("closing temp upload state: %w", err) + } + + if err := os.Rename(tmpName, filepath.Join(u.homeDir, uploadStateFile)); err != nil { + return fmt.Errorf("renaming upload state: %w", err) + } + return nil +} diff --git a/sidecar/tasks/snapshot_upload_metrics.go b/sidecar/tasks/snapshot_upload_metrics.go new file mode 100644 index 00000000..e94e86be --- /dev/null +++ b/sidecar/tasks/snapshot_upload_metrics.go @@ -0,0 +1,50 @@ +package tasks + +import "github.com/prometheus/client_golang/prometheus" + +var ( + // snapshotUploadLastRunSuccess is refreshed on any clean terminal — an + // upload OR a noop. A chain that has not advanced far enough to produce a + // new snapshot is healthy, not stalled, so a noop keeping this fresh is + // deliberate: an alert fires on "no clean run in N hours", not "no upload". + snapshotUploadLastRunSuccess = prometheus.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "seictl_snapshot_upload_last_run_success_timestamp_seconds", + Help: "Unix timestamp of the last snapshot-upload run that reached a clean terminal (uploaded or noop).", + }, + []string{"chain"}, + ) + + // snapshotUploadLastUploaded records the last run that actually pushed an + // archive to S3 (set only on a real upload, never on a noop). + snapshotUploadLastUploaded = prometheus.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "seictl_snapshot_upload_last_uploaded_timestamp_seconds", + Help: "Unix timestamp of the last snapshot-upload run that uploaded an archive to S3.", + }, + []string{"chain"}, + ) + + snapshotUploadLastUploadedHeight = prometheus.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "seictl_snapshot_upload_last_uploaded_height", + Help: "Snapshot height of the last archive uploaded to S3.", + }, + []string{"chain"}, + ) + + snapshotUploadOutcomes = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "seictl_snapshot_upload_outcome_total", + Help: "Count of snapshot-upload terminals by outcome (uploaded, noop, or error).", + }, + []string{"chain", "outcome"}, + ) +) + +func init() { + prometheus.MustRegister(snapshotUploadLastRunSuccess) + prometheus.MustRegister(snapshotUploadLastUploaded) + prometheus.MustRegister(snapshotUploadLastUploadedHeight) + prometheus.MustRegister(snapshotUploadOutcomes) +} diff --git a/sidecar/tasks/snapshot_upload_test.go b/sidecar/tasks/snapshot_upload_test.go new file mode 100644 index 00000000..e65a21c9 --- /dev/null +++ b/sidecar/tasks/snapshot_upload_test.go @@ -0,0 +1,710 @@ +package tasks + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "context" + "encoding/json" + "errors" + "io" + "os" + "path/filepath" + "strconv" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + "github.com/prometheus/client_golang/prometheus/testutil" + + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +type mockS3Uploader struct { + uploads map[string][]byte +} + +func newMockS3Uploader() *mockS3Uploader { + return &mockS3Uploader{uploads: make(map[string][]byte)} +} + +func (m *mockS3Uploader) UploadObject(_ context.Context, input *transfermanager.UploadObjectInput, _ ...func(*transfermanager.Options)) (*transfermanager.UploadObjectOutput, error) { + var buf bytes.Buffer + if input.Body != nil { + _, _ = io.Copy(&buf, input.Body) + } + key := *input.Bucket + "/" + *input.Key + m.uploads[key] = buf.Bytes() + return &transfermanager.UploadObjectOutput{}, nil +} + +func mockUploaderFactory(client *mockS3Uploader) seis3.UploaderFactory { + return func(_ context.Context, _ string) (seis3.Uploader, error) { + return client, nil + } +} + +func setupSnapshotDirs(t *testing.T, homeDir string, heights []int64) { + t.Helper() + snapshotsDir := filepath.Join(homeDir, "data", "snapshots") + if err := os.MkdirAll(snapshotsDir, 0o755); err != nil { + t.Fatalf("creating snapshots dir: %v", err) + } + + for _, h := range heights { + dir := filepath.Join(snapshotsDir, strconv.FormatInt(h, 10)) + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatalf("creating height dir %d: %v", h, err) + } + chunkDir := filepath.Join(dir, "1") + if err := os.MkdirAll(chunkDir, 0o755); err != nil { + t.Fatalf("creating chunk dir: %v", err) + } + if err := os.WriteFile(filepath.Join(chunkDir, "0"), []byte("chunk-data"), 0o644); err != nil { + t.Fatalf("writing chunk file: %v", err) + } + } + + // Create metadata.db + metadataPath := filepath.Join(snapshotsDir, "metadata.db") + if err := os.WriteFile(metadataPath, []byte("metadata-content"), 0o644); err != nil { + t.Fatalf("writing metadata.db: %v", err) + } +} + +func TestPickUploadCandidate(t *testing.T) { + cases := []struct { + name string + heights []int64 + want int64 + }{ + {"two snapshots returns second to latest", []int64{1000, 2000}, 1000}, + {"three snapshots returns second to latest", []int64{1000, 3000, 2000}, 2000}, + {"single snapshot returns zero", []int64{1000}, 0}, + {"no snapshots returns zero", nil, 0}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + homeDir := t.TempDir() + if tc.heights != nil { + setupSnapshotDirs(t, homeDir, tc.heights) + } + h, err := pickUploadCandidate(filepath.Join(homeDir, "data", "snapshots")) + if err != nil { + t.Fatalf("pickUploadCandidate() error = %v", err) + } + if h != tc.want { + t.Errorf("height = %d, want %d", h, tc.want) + } + }) + } +} + +func TestUpload_UploadsArchiveAndLatestTxt(t *testing.T) { + homeDir := t.TempDir() + setupSnapshotDirs(t, homeDir, []int64{1000, 2000}) + + mock := newMockS3Uploader() + uploader, err := NewSnapshotUploader(homeDir, "my-bucket", "eu-central-1", "testchain", 0, mockUploaderFactory(mock)) + if err != nil { + t.Fatalf("NewSnapshotUploader: %v", err) + } + + result, err := uploader.Upload(context.Background()) + if err != nil { + t.Fatalf("Upload() error = %v", err) + } + if result.Outcome != OutcomeUploaded { + t.Errorf("outcome = %q, want %q", result.Outcome, OutcomeUploaded) + } + if result.NoopReason != "" { + t.Errorf("NoopReason = %q, want empty on OutcomeUploaded", result.NoopReason) + } + if result.Height != 1000 { + t.Errorf("result height = %d, want 1000", result.Height) + } + if result.Key != "testchain/state-sync/1000.tar.gz" { + t.Errorf("result key = %q, want testchain/state-sync/1000.tar.gz", result.Key) + } + + if _, ok := mock.uploads["my-bucket/testchain/state-sync/1000.tar.gz"]; !ok { + t.Error("expected archive upload at testchain/state-sync/1000.tar.gz") + } + + latest, ok := mock.uploads["my-bucket/testchain/state-sync/latest.txt"] + if !ok { + t.Fatal("expected latest.txt upload") + } + if string(latest) != "1000" { + t.Errorf("latest.txt = %q, want %q", string(latest), "1000") + } +} + +func TestUpload_SkipsWhenAlreadyUploaded(t *testing.T) { + homeDir := t.TempDir() + setupSnapshotDirs(t, homeDir, []int64{1000, 2000}) + + state := uploadState{LastUploadedHeight: 1000} + data, _ := json.Marshal(state) + _ = os.WriteFile(filepath.Join(homeDir, uploadStateFile), data, 0o644) + + mock := newMockS3Uploader() + uploader, err := NewSnapshotUploader(homeDir, "my-bucket", "eu-central-1", "testchain", 0, mockUploaderFactory(mock)) + if err != nil { + t.Fatalf("NewSnapshotUploader: %v", err) + } + + result, err := uploader.Upload(context.Background()) + if err != nil { + t.Fatalf("Upload() error = %v", err) + } + if result.Outcome != OutcomeNoop || result.NoopReason != NoopAlreadyUploaded { + t.Errorf("result = %+v, want noop/already-uploaded", result) + } + + if len(mock.uploads) != 0 { + t.Errorf("expected no uploads (already uploaded), got %d", len(mock.uploads)) + } +} + +func TestUpload_UploadsNewerSnapshot(t *testing.T) { + homeDir := t.TempDir() + setupSnapshotDirs(t, homeDir, []int64{1000, 2000, 3000}) + + state := uploadState{LastUploadedHeight: 1000} + data, _ := json.Marshal(state) + _ = os.WriteFile(filepath.Join(homeDir, uploadStateFile), data, 0o644) + + mock := newMockS3Uploader() + uploader, err := NewSnapshotUploader(homeDir, "my-bucket", "eu-central-1", "testchain", 0, mockUploaderFactory(mock)) + if err != nil { + t.Fatalf("NewSnapshotUploader: %v", err) + } + + _, err = uploader.Upload(context.Background()) + if err != nil { + t.Fatalf("Upload() error = %v", err) + } + + if _, ok := mock.uploads["my-bucket/testchain/state-sync/2000.tar.gz"]; !ok { + t.Error("expected archive upload at testchain/state-sync/2000.tar.gz") + } +} + +func TestUpload_NoOpsWhenTooFewSnapshots(t *testing.T) { + homeDir := t.TempDir() + setupSnapshotDirs(t, homeDir, []int64{1000}) + + mock := newMockS3Uploader() + uploader, err := NewSnapshotUploader(homeDir, "my-bucket", "eu-central-1", "testchain", 0, mockUploaderFactory(mock)) + if err != nil { + t.Fatalf("NewSnapshotUploader: %v", err) + } + + result, err := uploader.Upload(context.Background()) + if err != nil { + t.Fatalf("Upload() error = %v", err) + } + if result.Outcome != OutcomeNoop || result.NoopReason != NoopFewerThanTwoSnapshots { + t.Errorf("result = %+v, want noop/fewer-than-2-snapshots", result) + } + + if len(mock.uploads) != 0 { + t.Errorf("expected no uploads, got %d", len(mock.uploads)) + } +} + +func TestUpload_WritesUploadState(t *testing.T) { + homeDir := t.TempDir() + setupSnapshotDirs(t, homeDir, []int64{1000, 2000}) + + mock := newMockS3Uploader() + uploader, err := NewSnapshotUploader(homeDir, "my-bucket", "eu-central-1", "testchain", 0, mockUploaderFactory(mock)) + if err != nil { + t.Fatalf("NewSnapshotUploader: %v", err) + } + + _, err = uploader.Upload(context.Background()) + if err != nil { + t.Fatalf("Upload() error = %v", err) + } + + state := uploader.readUploadState() + if state.LastUploadedHeight != 1000 { + t.Errorf("LastUploadedHeight = %d, want 1000", state.LastUploadedHeight) + } + if state.LastUploadedAt == 0 { + t.Error("expected LastUploadedAt to be persisted on upload") + } +} + +// readTarGzNames decompresses + reads a tar.gz produced by writeArchive +// and returns the entry names + their typeflags. +func readTarGzNames(t *testing.T, body []byte) map[string]byte { + t.Helper() + gzr, err := gzip.NewReader(bytes.NewReader(body)) + if err != nil { + t.Fatalf("gzip reader: %v", err) + } + defer func() { _ = gzr.Close() }() + tr := tar.NewReader(gzr) + out := map[string]byte{} + for { + h, err := tr.Next() + if err == io.EOF { + break + } + if err != nil { + t.Fatalf("tar next: %v", err) + } + out[h.Name] = h.Typeflag + } + return out +} + +// runWriteArchive captures writeArchive output via an io.Pipe. +func runWriteArchive(t *testing.T, snapshotsDir string, height int64) []byte { + t.Helper() + pr, pw := io.Pipe() + done := make(chan error, 1) + go func() { + done <- writeArchive(context.Background(), pw, snapshotsDir, height) + }() + body, err := io.ReadAll(pr) + if err != nil { + t.Fatalf("read pipe: %v", err) + } + if err := <-done; err != nil { + t.Fatalf("writeArchive error: %v", err) + } + return body +} + +func TestWriteArchive_MetadataAsDirectory(t *testing.T) { + homeDir := t.TempDir() + snapshotsDir := filepath.Join(homeDir, "data", "snapshots") + if err := os.MkdirAll(filepath.Join(snapshotsDir, "1000", "1"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(snapshotsDir, "1000", "1", "0"), []byte("chunk"), 0o644); err != nil { + t.Fatal(err) + } + // metadata.db as a directory containing typical LevelDB files + mdDir := filepath.Join(snapshotsDir, "metadata.db") + if err := os.MkdirAll(mdDir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(mdDir, "CURRENT"), []byte("MANIFEST-000001\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(mdDir, "MANIFEST-000001"), []byte("manifest-bytes"), 0o644); err != nil { + t.Fatal(err) + } + + body := runWriteArchive(t, snapshotsDir, 1000) + names := readTarGzNames(t, body) + + // Height directory entries present and recursive + if _, ok := names["1000/1/0"]; !ok { + t.Errorf("missing height-dir file 1000/1/0 in archive; entries: %v", names) + } + // metadata.db directory header + recursive contents + if tf, ok := names["metadata.db"]; !ok || tf != tar.TypeDir { + t.Errorf("metadata.db missing or not a directory entry (typeflag=%d)", tf) + } + if _, ok := names["metadata.db/CURRENT"]; !ok { + t.Errorf("missing metadata.db/CURRENT in archive; entries: %v", names) + } + if _, ok := names["metadata.db/MANIFEST-000001"]; !ok { + t.Errorf("missing metadata.db/MANIFEST-000001 in archive; entries: %v", names) + } +} + +func TestWriteArchive_MetadataAsFile(t *testing.T) { + homeDir := t.TempDir() + snapshotsDir := filepath.Join(homeDir, "data", "snapshots") + if err := os.MkdirAll(filepath.Join(snapshotsDir, "1000", "1"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(snapshotsDir, "1000", "1", "0"), []byte("chunk"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(snapshotsDir, "metadata.db"), []byte("metadata-content"), 0o644); err != nil { + t.Fatal(err) + } + + body := runWriteArchive(t, snapshotsDir, 1000) + names := readTarGzNames(t, body) + + if tf, ok := names["metadata.db"]; !ok || tf != tar.TypeReg { + t.Errorf("metadata.db missing or not a regular file (typeflag=%d)", tf) + } + if _, ok := names["1000/1/0"]; !ok { + t.Errorf("missing height-dir file 1000/1/0 in archive; entries: %v", names) + } +} + +// blockingUploader models a wedged S3 stream: UploadObject never returns until +// the context is cancelled, then surfaces the context error. +type blockingUploader struct{} + +func (blockingUploader) UploadObject(ctx context.Context, _ *transfermanager.UploadObjectInput, _ ...func(*transfermanager.Options)) (*transfermanager.UploadObjectOutput, error) { + <-ctx.Done() + return nil, ctx.Err() +} + +func decodeUploadResult(t *testing.T, raw json.RawMessage) SnapshotUploadResult { + t.Helper() + var r SnapshotUploadResult + if err := json.Unmarshal(raw, &r); err != nil { + t.Fatalf("unmarshal result %q: %v", raw, err) + } + return r +} + +func TestOnceHandler_ReturnsDistinguishableOutcomes(t *testing.T) { + t.Run("uploaded", func(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000, 2000}) + uploader, err := NewSnapshotUploader(home, "b", "r", "once-uploaded", 0, mockUploaderFactory(newMockS3Uploader())) + if err != nil { + t.Fatal(err) + } + raw, err := uploader.OnceHandler(time.Minute)(context.Background(), nil) + if err != nil { + t.Fatalf("handler error = %v", err) + } + got := decodeUploadResult(t, raw) + if got.Outcome != OutcomeUploaded || got.Height != 1000 { + t.Fatalf("result = %+v, want uploaded @1000", got) + } + }) + + t.Run("noop fewer than 2 snapshots", func(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000}) + uploader, err := NewSnapshotUploader(home, "b", "r", "once-few", 0, mockUploaderFactory(newMockS3Uploader())) + if err != nil { + t.Fatal(err) + } + raw, err := uploader.OnceHandler(time.Minute)(context.Background(), nil) + if err != nil { + t.Fatalf("handler error = %v", err) + } + got := decodeUploadResult(t, raw) + if got.Outcome != OutcomeNoop || got.NoopReason != NoopFewerThanTwoSnapshots { + t.Fatalf("result = %+v, want noop/fewer-than-2-snapshots", got) + } + }) + + t.Run("noop already uploaded", func(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000, 2000}) + data, _ := json.Marshal(uploadState{LastUploadedHeight: 1000}) + if err := os.WriteFile(filepath.Join(home, uploadStateFile), data, 0o644); err != nil { + t.Fatal(err) + } + uploader, err := NewSnapshotUploader(home, "b", "r", "once-already", 0, mockUploaderFactory(newMockS3Uploader())) + if err != nil { + t.Fatal(err) + } + raw, err := uploader.OnceHandler(time.Minute)(context.Background(), nil) + if err != nil { + t.Fatalf("handler error = %v", err) + } + got := decodeUploadResult(t, raw) + if got.Outcome != OutcomeNoop || got.NoopReason != NoopAlreadyUploaded { + t.Fatalf("result = %+v, want noop/already-uploaded", got) + } + }) +} + +// A handler-internal deadline must surface as context.DeadlineExceeded so the +// engine persists Failed rather than stranding the task in 'running'. Unlike a +// context.Canceled, a DeadlineExceeded is a genuine upload failure and must tick +// the error outcome counter — the recordError suppression keys only on Canceled. +func TestOnceHandler_DeadlineFailsCleanly(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000, 2000}) + chain := "once-deadline" + factory := func(context.Context, string) (seis3.Uploader, error) { return blockingUploader{}, nil } + uploader, err := NewSnapshotUploader(home, "b", "r", chain, 0, factory) + if err != nil { + t.Fatal(err) + } + + _, err = uploader.OnceHandler(50*time.Millisecond)(context.Background(), nil) + if !errors.Is(err, context.DeadlineExceeded) { + t.Fatalf("handler error = %v, want context.DeadlineExceeded", err) + } + if got := testutil.ToFloat64(snapshotUploadOutcomes.WithLabelValues(chain, string(OutcomeError))); got != 1 { + t.Errorf("deadline-exceeded is a genuine failure: error outcome counter = %v, want 1", got) + } +} + +// A canceled context — an operator DELETE or a node drain mid-upload — must not +// tick the error counter. It mirrors the engine's cancellation-suppression +// contract: a canceled task leaves no terminal record, so a drain is not counted +// as an upload failure. The result still carries OutcomeError, but the engine +// suppresses it on the canceled path so its value there is inert. +func TestUpload_CanceledDoesNotTickErrorCounter(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000, 2000}) + chain := "cancel-no-tick" + factory := func(context.Context, string) (seis3.Uploader, error) { return blockingUploader{}, nil } + uploader, err := NewSnapshotUploader(home, "b", "r", chain, 0, factory) + if err != nil { + t.Fatal(err) + } + + ctx, cancel := context.WithCancel(context.Background()) + go func() { + time.Sleep(10 * time.Millisecond) + cancel() + }() + + result, err := uploader.Upload(ctx) + if !errors.Is(err, context.Canceled) { + t.Fatalf("Upload error = %v, want context.Canceled", err) + } + if result.Outcome != OutcomeError { + t.Errorf("result outcome = %q, want %q", result.Outcome, OutcomeError) + } + if got := testutil.ToFloat64(snapshotUploadOutcomes.WithLabelValues(chain, string(OutcomeError))); got != 0 { + t.Errorf("canceled upload must not tick error counter, got %v", got) + } + if got := testutil.ToFloat64(snapshotUploadLastRunSuccess.WithLabelValues(chain)); got != 0 { + t.Errorf("canceled path must not refresh last run success, got %v", got) + } +} + +// The loop must swallow per-iteration errors, keep running, and stop cleanly on +// context cancellation. +func TestRunLoop_SwallowsErrorsAndStopsOnCancel(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000, 2000}) + var calls atomic.Int64 + factory := func(context.Context, string) (seis3.Uploader, error) { + calls.Add(1) + return nil, errors.New("s3 down") + } + uploader, err := NewSnapshotUploader(home, "b", "r", "loop-chain", 5*time.Millisecond, factory) + if err != nil { + t.Fatal(err) + } + + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan error, 1) + go func() { done <- uploader.runLoop(ctx) }() + + time.Sleep(60 * time.Millisecond) // let it iterate and swallow several failures + cancel() + + select { + case err := <-done: + if !errors.Is(err, context.Canceled) { + t.Fatalf("runLoop returned %v, want context.Canceled", err) + } + case <-time.After(2 * time.Second): + t.Fatal("runLoop did not stop after cancel") + } + if calls.Load() < 2 { + t.Fatalf("expected the loop to retry after swallowing errors, got %d attempts", calls.Load()) + } +} + +// Concurrent writers exercise the temp-file + rename path: the persisted file +// must always parse to a complete written value, never a torn zero, and no temp +// files may be left behind. +func TestWriteUploadState_AtomicUnderConcurrentWriters(t *testing.T) { + home := t.TempDir() + uploader, err := NewSnapshotUploader(home, "b", "r", "atomic-chain", 0, mockUploaderFactory(newMockS3Uploader())) + if err != nil { + t.Fatal(err) + } + + var wg sync.WaitGroup + for i := int64(1); i <= 50; i++ { + wg.Add(1) + go func(h int64) { + defer wg.Done() + if err := uploader.writeUploadState(uploadState{LastUploadedHeight: h, LastUploadedAt: h}); err != nil { + t.Errorf("writeUploadState(%d): %v", h, err) + } + }(i) + } + wg.Wait() + + st := uploader.readUploadState() + if st.LastUploadedHeight < 1 || st.LastUploadedHeight > 50 { + t.Fatalf("torn/lost write: height=%d, want a complete value in 1..50", st.LastUploadedHeight) + } + if st.LastUploadedAt != st.LastUploadedHeight { + t.Fatalf("torn write: height=%d but at=%d (a single writer paired them)", st.LastUploadedHeight, st.LastUploadedAt) + } + + entries, err := os.ReadDir(home) + if err != nil { + t.Fatal(err) + } + for _, e := range entries { + if strings.Contains(e.Name(), ".tmp-") { + t.Errorf("leftover temp file after atomic write: %s", e.Name()) + } + } +} + +// A pre-existing complete state survives a subsequent failed write attempt: the +// rename is all-or-nothing, so a torn temp never clobbers the live file. +func TestWriteUploadState_PreservesPriorStateOnRoundTrip(t *testing.T) { + home := t.TempDir() + uploader, err := NewSnapshotUploader(home, "b", "r", "roundtrip-chain", 0, mockUploaderFactory(newMockS3Uploader())) + if err != nil { + t.Fatal(err) + } + if err := uploader.writeUploadState(uploadState{LastUploadedHeight: 4242, LastUploadedAt: 99}); err != nil { + t.Fatal(err) + } + + // A stray temp file (as a crashed write would leave) must be ignored by the + // reader, which keys only on the canonical filename. + if err := os.WriteFile(filepath.Join(home, uploadStateFile+".tmp-garbage"), []byte("{ torn"), 0o644); err != nil { + t.Fatal(err) + } + + st := uploader.readUploadState() + if st.LastUploadedHeight != 4242 || st.LastUploadedAt != 99 { + t.Fatalf("state = %+v, want height=4242 at=99", st) + } +} + +func TestUpload_EmitsMetrics(t *testing.T) { + t.Run("upload sets all gauges", func(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000, 2000}) + chain := "metrics-uploaded" + uploader, err := NewSnapshotUploader(home, "b", "r", chain, 0, mockUploaderFactory(newMockS3Uploader())) + if err != nil { + t.Fatal(err) + } + if _, err := uploader.Upload(context.Background()); err != nil { + t.Fatal(err) + } + if got := testutil.ToFloat64(snapshotUploadOutcomes.WithLabelValues(chain, string(OutcomeUploaded))); got != 1 { + t.Errorf("uploaded outcome counter = %v, want 1", got) + } + if got := testutil.ToFloat64(snapshotUploadLastUploadedHeight.WithLabelValues(chain)); got != 1000 { + t.Errorf("last uploaded height = %v, want 1000", got) + } + if testutil.ToFloat64(snapshotUploadLastUploaded.WithLabelValues(chain)) == 0 { + t.Error("last uploaded timestamp not set on upload") + } + if testutil.ToFloat64(snapshotUploadLastRunSuccess.WithLabelValues(chain)) == 0 { + t.Error("last run success not set on upload") + } + }) + + t.Run("noop refreshes success but not uploaded gauges", func(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000}) // fewer than 2 -> noop + chain := "metrics-noop" + uploader, err := NewSnapshotUploader(home, "b", "r", chain, 0, mockUploaderFactory(newMockS3Uploader())) + if err != nil { + t.Fatal(err) + } + if _, err := uploader.Upload(context.Background()); err != nil { + t.Fatal(err) + } + if got := testutil.ToFloat64(snapshotUploadOutcomes.WithLabelValues(chain, string(OutcomeNoop))); got != 1 { + t.Errorf("noop outcome counter = %v, want 1", got) + } + if testutil.ToFloat64(snapshotUploadLastRunSuccess.WithLabelValues(chain)) == 0 { + t.Error("noop must refresh last run success (a not-yet-advanced chain is healthy)") + } + if got := testutil.ToFloat64(snapshotUploadLastUploadedHeight.WithLabelValues(chain)); got != 0 { + t.Errorf("noop must not advance uploaded height gauge, got %v", got) + } + }) +} + +// A failed Upload must return OutcomeError (so a poller reading the persisted +// result can tell failure apart from an empty outcome) and increment the +// chain-labeled outcome counter, while leaving the clean-terminal gauges +// untouched so a real S3 outage cannot read as green. +func TestUpload_ErrorOutcomeOnFailure(t *testing.T) { + home := t.TempDir() + setupSnapshotDirs(t, home, []int64{1000, 2000}) + chain := "metrics-error" + factory := func(context.Context, string) (seis3.Uploader, error) { + return nil, errors.New("s3 down") + } + uploader, err := NewSnapshotUploader(home, "b", "r", chain, 0, factory) + if err != nil { + t.Fatal(err) + } + + result, err := uploader.Upload(context.Background()) + if err == nil { + t.Fatal("expected Upload to error when the S3 uploader cannot be built") + } + if result.Outcome != OutcomeError { + t.Errorf("result outcome = %q, want %q", result.Outcome, OutcomeError) + } + if got := testutil.ToFloat64(snapshotUploadOutcomes.WithLabelValues(chain, string(OutcomeError))); got != 1 { + t.Errorf("error outcome counter = %v, want 1", got) + } + if got := testutil.ToFloat64(snapshotUploadLastRunSuccess.WithLabelValues(chain)); got != 0 { + t.Errorf("error path must not refresh last run success, got %v", got) + } + if got := testutil.ToFloat64(snapshotUploadLastUploadedHeight.WithLabelValues(chain)); got != 0 { + t.Errorf("error path must not advance uploaded height gauge, got %v", got) + } +} + +func TestEmitStartupMetrics_RehydratesUploadedGauges(t *testing.T) { + home := t.TempDir() + chain := "startup-chain" + uploader, err := NewSnapshotUploader(home, "b", "r", chain, 0, mockUploaderFactory(newMockS3Uploader())) + if err != nil { + t.Fatal(err) + } + if err := uploader.writeUploadState(uploadState{LastUploadedHeight: 7777, LastUploadedAt: 1234567890}); err != nil { + t.Fatal(err) + } + + uploader.EmitStartupMetrics() + + if got := testutil.ToFloat64(snapshotUploadLastUploadedHeight.WithLabelValues(chain)); got != 7777 { + t.Errorf("startup height gauge = %v, want 7777", got) + } + if got := testutil.ToFloat64(snapshotUploadLastUploaded.WithLabelValues(chain)); got != 1234567890 { + t.Errorf("startup uploaded timestamp = %v, want 1234567890", got) + } + // The alert signal must stay unset on startup so a genuinely stalled uploader + // is not masked by a persisted success timestamp. + if got := testutil.ToFloat64(snapshotUploadLastRunSuccess.WithLabelValues(chain)); got != 0 { + t.Errorf("startup must not set last run success, got %v", got) + } +} + +func TestNormalizePrefix(t *testing.T) { + tests := []struct { + input string + want string + }{ + {"", ""}, + {"state-sync", "state-sync/"}, + {"state-sync/", "state-sync/"}, + {"a/b/c", "a/b/c/"}, + {"a/b/c/", "a/b/c/"}, + } + for _, tt := range tests { + got := normalizePrefix(tt.input) + if got != tt.want { + t.Errorf("normalizePrefix(%q) = %q, want %q", tt.input, got, tt.want) + } + } +} diff --git a/sidecar/tasks/statesync.go b/sidecar/tasks/statesync.go new file mode 100644 index 00000000..c9b3c24c --- /dev/null +++ b/sidecar/tasks/statesync.go @@ -0,0 +1,362 @@ +package tasks + +import ( + "context" + "encoding/json" + "fmt" + "net" + "net/http" + "os" + "path/filepath" + "strconv" + "strings" + "time" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" + "github.com/sei-protocol/sei-k8s-controller/sidecarapi/tomlpatch" +) + +var ssLog = seilog.NewLogger("seictl", "task", "state-sync") + +const ( + stateSyncMarkerFile = ".sei-sidecar-statesync-done" + trustHeightOffset = 2000 + rpcPort = "26657" + witnessProbeTimeout = 10 * time.Second + tlsPort = "443" +) + +// StateSyncConfig holds the trust point and RPC servers for Tendermint state sync. +type StateSyncConfig struct { + TrustHeight int64 + TrustHash string + TrustPeriod string + RpcServers string + UseLocalSnapshot bool + BackfillBlocks int64 +} + +// StateSyncConfigurer discovers a trust point from peers and writes the config file. +type StateSyncConfigurer struct { + homeDir string + httpClient rpc.HTTPDoer +} + +// NewStateSyncConfigurer creates a configurer targeting the given home directory. +func NewStateSyncConfigurer(homeDir string, client rpc.HTTPDoer) *StateSyncConfigurer { + if client == nil { + client = &http.Client{} + } + return &StateSyncConfigurer{homeDir: homeDir, httpClient: client} +} + +// Handler returns an engine.TaskHandler. +func (s *StateSyncConfigurer) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, params StateSyncRequest) error { + return s.Configure(ctx, params) + }) +} + +// StateSyncRequest groups the caller-provided parameters for state-sync configuration. +type StateSyncRequest struct { + UseLocalSnapshot bool `json:"useLocalSnapshot"` + TrustPeriod string `json:"trustPeriod"` + BackfillBlocks int64 `json:"backfillBlocks"` + // RpcServers are explicit light-client witness endpoints ("host:port"). + // When non-empty they are used verbatim; otherwise witnesses are derived + // from persistent-peers. + RpcServers []string `json:"rpcServers"` +} + +// Configure determines the state-sync light-client witnesses, queries one for a +// trust point, and writes the settings to config.toml. +// +// Witnesses come from p.RpcServers when provided, otherwise are derived from +// persistent-peers. Only witnesses that answer /status are written: a peer that +// serves P2P but not RPC (e.g. an external P2P NLB hostname) would otherwise make +// seid exit on "no witnesses connected" and crashloop. With UseLocalSnapshot the +// trust height comes from the restored snapshot instead of a query. +func (s *StateSyncConfigurer) Configure(ctx context.Context, p StateSyncRequest) error { + if markerExists(s.homeDir, stateSyncMarkerFile) { + ssLog.Debug("already completed, skipping") + return nil + } + + candidates, err := s.witnessCandidates(p) + if err != nil { + return fmt.Errorf("configure-state-sync: %w", err) + } + + reachable := s.reachableWitnesses(ctx, candidates) + if len(reachable) == 0 { + return fmt.Errorf("configure-state-sync: no reachable RPC witness among %v", candidates) + } + + var trustHeight int64 + if p.UseLocalSnapshot { + h, err := discoverLocalSnapshotHeight(s.homeDir) + if err != nil { + return fmt.Errorf("configure-state-sync: discovering local snapshot height: %w", err) + } + trustHeight = h + ssLog.Info("using local snapshot height as trust height", "height", trustHeight) + } else { + ssLog.Info("querying latest height", "endpoint", reachable[0]) + latestHeight, err := s.queryLatestHeight(ctx, reachable[0]) + if err != nil { + return fmt.Errorf("configure-state-sync: querying latest height: %w", err) + } + trustHeight = latestHeight - trustHeightOffset + if trustHeight < 1 { + trustHeight = 1 + } + } + + ssLog.Info("querying trust hash", "trust-height", trustHeight, "endpoint", reachable[0]) + trustHash, err := s.queryBlockHash(ctx, reachable[0], trustHeight) + if err != nil { + return fmt.Errorf("configure-state-sync: querying block hash at height %d: %w", trustHeight, err) + } + + // CometBFT's light client requires at least two witnesses; pad by + // duplicating the primary when only one reachable witness exists. + for len(reachable) < 2 { + reachable = append(reachable, reachable[0]) + } + + cfg := StateSyncConfig{ + TrustHeight: trustHeight, + TrustHash: trustHash, + TrustPeriod: p.TrustPeriod, + RpcServers: strings.Join(reachable, ","), + UseLocalSnapshot: p.UseLocalSnapshot, + BackfillBlocks: p.BackfillBlocks, + } + + ssLog.Info("writing config", "trust-height", trustHeight, "trust-hash", trustHash, + "trust-period", p.TrustPeriod, "rpc-servers", cfg.RpcServers, + "use-local-snapshot", p.UseLocalSnapshot, "backfill-blocks", p.BackfillBlocks) + if err := writeStateSyncToConfig(s.homeDir, cfg); err != nil { + return fmt.Errorf("configure-state-sync: writing config.toml: %w", err) + } + + return writeMarker(s.homeDir, stateSyncMarkerFile) +} + +// witnessCandidates returns the candidate witness endpoints ("host:port"): +// caller-provided RpcServers verbatim, otherwise each persistent-peer host with +// the RPC port attached. The peer-derived form holds for peers that serve RPC on +// the P2P host (EC2 peers, internal cluster DNS); reachableWitnesses drops those +// that don't. +func (s *StateSyncConfigurer) witnessCandidates(p StateSyncRequest) ([]string, error) { + if len(p.RpcServers) > 0 { + ssLog.Info("using caller-provided rpc witnesses", "count", len(p.RpcServers)) + return p.RpcServers, nil + } + + peers, err := readPeersFromConfig(s.homeDir) + if err != nil { + return nil, err + } + if len(peers) == 0 { + return nil, fmt.Errorf("no peers in config.toml") + } + ssLog.Debug("found peers", "count", len(peers)) + + hosts := extractRPCHosts(peers, 2) + if len(hosts) == 0 { + return nil, fmt.Errorf("could not extract RPC hosts from peers") + } + endpoints := make([]string, len(hosts)) + for i, h := range hosts { + endpoints[i] = h + ":" + rpcPort + } + return endpoints, nil +} + +// reachableWitnesses returns the candidate endpoints whose /status responds. +func (s *StateSyncConfigurer) reachableWitnesses(ctx context.Context, candidates []string) []string { + reachable := make([]string, 0, len(candidates)) + for _, ep := range candidates { + if err := s.probeWitness(ctx, ep); err != nil { + ssLog.Warn("state-sync witness unreachable, skipping", "endpoint", ep, "err", err) + continue + } + reachable = append(reachable, ep) + } + return reachable +} + +// probeWitness reports whether endpoint answers /status. The context deadline is +// load-bearing: the sidecar's http.Client has no Timeout, so a black-holed +// endpoint (TCP accepted, no response) is bounded only by this cancellation. +func (s *StateSyncConfigurer) probeWitness(ctx context.Context, endpoint string) error { + pctx, cancel := context.WithTimeout(ctx, witnessProbeTimeout) + defer cancel() + _, err := s.rpcClientForEndpoint(endpoint).Get(pctx, "/status") + return err +} + +// discoverLocalSnapshotHeight scans the Tendermint snapshots directory for the +// highest available snapshot height. Snapshots are stored as +// /data/snapshots///. +func discoverLocalSnapshotHeight(homeDir string) (int64, error) { + snapshotDir := filepath.Join(homeDir, "data", "snapshots") + entries, err := os.ReadDir(snapshotDir) + if err != nil { + return 0, fmt.Errorf("reading snapshots directory: %w", err) + } + + var maxHeight int64 + for _, e := range entries { + if !e.IsDir() { + continue + } + h, err := strconv.ParseInt(e.Name(), 10, 64) + if err != nil { + continue + } + if h > maxHeight { + maxHeight = h + } + } + + if maxHeight == 0 { + return 0, fmt.Errorf("no snapshot found in %s", snapshotDir) + } + return maxHeight, nil +} + +// extractRPCHosts extracts up to maxHosts host addresses from peer strings +// in "nodeId@host:port" format. +func extractRPCHosts(peers []string, maxHosts int) []string { + var hosts []string + for _, p := range peers { + if len(hosts) >= maxHosts { + break + } + parts := strings.SplitN(p, "@", 2) + if len(parts) != 2 { + continue + } + hostPort := parts[1] + host := hostPort + if idx := strings.LastIndex(hostPort, ":"); idx >= 0 { + host = hostPort[:idx] + } + if host != "" { + hosts = append(hosts, host) + } + } + return hosts +} + +// rpcClientForEndpoint builds an rpc.Client targeting a full "host:port" RPC +// endpoint. The scheme is derived from the port: a :443 witness is a public TLS +// gateway (Istio HTTPRoute) and must be reached over https; every other port is +// the plaintext in-cluster CometBFT RPC. Hardcoding http:// here previously made +// a :443 witness fail the /status probe with an immediate EOF (plaintext request +// to a TLS listener), which blocked every new state-syncing node. +func (s *StateSyncConfigurer) rpcClientForEndpoint(endpoint string) *rpc.Client { + return rpc.NewClient(witnessScheme(endpoint)+"://"+endpoint, s.httpClient) +} + +// witnessScheme returns the URL scheme for a "host:port" witness endpoint: +// https for :443, http otherwise. An endpoint with no parseable port defaults +// to http (the in-cluster plaintext form). +func witnessScheme(endpoint string) string { + if _, port, err := net.SplitHostPort(endpoint); err == nil && port == tlsPort { + return "https" + } + return "http" +} + +func (s *StateSyncConfigurer) queryLatestHeight(ctx context.Context, endpoint string) (int64, error) { + raw, err := s.rpcClientForEndpoint(endpoint).Get(ctx, "/status") + if err != nil { + return 0, err + } + + var status rpc.StatusResult + if err := json.Unmarshal(raw, &status); err != nil { + return 0, fmt.Errorf("parsing status response: %w", err) + } + + height, err := strconv.ParseInt(status.SyncInfo.LatestBlockHeight, 10, 64) + if err != nil { + return 0, fmt.Errorf("parsing height %q: %w", status.SyncInfo.LatestBlockHeight, err) + } + return height, nil +} + +func (s *StateSyncConfigurer) queryBlockHash(ctx context.Context, endpoint string, height int64) (string, error) { + path := fmt.Sprintf("/block?height=%d", height) + raw, err := s.rpcClientForEndpoint(endpoint).Get(ctx, path) + if err != nil { + return "", err + } + + var block rpc.BlockResult + if err := json.Unmarshal(raw, &block); err != nil { + return "", fmt.Errorf("parsing block response: %w", err) + } + hash := block.BlockID.Hash + if hash == "" { + return "", fmt.Errorf("empty block hash at height %d", height) + } + const sha256HexLen = 64 + if len(hash) != sha256HexLen { + return "", fmt.Errorf("unexpected block hash length at height %d: got %d, want %d", height, len(hash), sha256HexLen) + } + return hash, nil +} + +func writeStateSyncToConfig(homeDir string, cfg StateSyncConfig) error { + configPath := filepath.Join(homeDir, "config", "config.toml") + ss := map[string]any{ + "enable": true, + "trust-height": cfg.TrustHeight, + "trust-hash": cfg.TrustHash, + "rpc-servers": cfg.RpcServers, + "use-local-snapshot": cfg.UseLocalSnapshot, + } + if cfg.TrustPeriod != "" { + ss["trust-period"] = cfg.TrustPeriod + } + if cfg.BackfillBlocks > 0 { + ss["backfill-blocks"] = cfg.BackfillBlocks + } + return mergeAndWrite(configPath, map[string]any{"statesync": ss}) +} + +// readPeersFromConfig reads the persistent-peers value from config.toml and +// splits it into individual peer strings. +func readPeersFromConfig(homeDir string) ([]string, error) { + configPath := filepath.Join(homeDir, "config", "config.toml") + doc, err := tomlpatch.ReadTOML(configPath) + if err != nil { + return nil, fmt.Errorf("reading config.toml: %w", err) + } + + p2p, ok := doc["p2p"].(map[string]any) + if !ok { + return nil, nil + } + + raw, _ := p2p["persistent-peers"].(string) + if raw == "" { + return nil, nil + } + + var peers []string + for _, p := range strings.Split(raw, ",") { + p = strings.TrimSpace(p) + if p != "" { + peers = append(peers, p) + } + } + return peers, nil +} diff --git a/sidecar/tasks/statesync_test.go b/sidecar/tasks/statesync_test.go new file mode 100644 index 00000000..900a9f44 --- /dev/null +++ b/sidecar/tasks/statesync_test.go @@ -0,0 +1,697 @@ +package tasks + +import ( + "bytes" + "context" + "crypto/rand" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "strings" + "testing" +) + +type mockHTTPDoer struct { + responses map[string]*http.Response +} + +func (m *mockHTTPDoer) Do(req *http.Request) (*http.Response, error) { + resp, ok := m.responses[req.URL.String()] + if !ok { + return nil, fmt.Errorf("unexpected request: %s", req.URL.String()) + } + // Read and restore the body so repeated requests to the same URL (the + // witness reachability probe, then the trust-point query) both succeed. + body, _ := io.ReadAll(resp.Body) + resp.Body = io.NopCloser(bytes.NewReader(body)) + return &http.Response{StatusCode: resp.StatusCode, Body: io.NopCloser(bytes.NewReader(body))}, nil +} + +func generateBlockHash() string { + b := make([]byte, 32) + _, _ = rand.Read(b) + return fmt.Sprintf("%X", b) +} + +func jsonResponse(body string) *http.Response { + return &http.Response{ + StatusCode: http.StatusOK, + Body: io.NopCloser(strings.NewReader(body)), + } +} + +// wrapResult wraps inner JSON in the CometBFT JSON-RPC envelope. +func wrapResult(inner string) string { + return fmt.Sprintf(`{"jsonrpc":"2.0","id":-1,"result":%s}`, inner) +} + +func setupPeersInConfig(t *testing.T, homeDir string, peers []string) { + t.Helper() + configDir := filepath.Join(homeDir, "config") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatalf("creating config dir: %v", err) + } + content := fmt.Sprintf("[p2p]\npersistent-peers = %q\n", strings.Join(peers, ",")) + if err := os.WriteFile(filepath.Join(configDir, "config.toml"), []byte(content), 0o644); err != nil { + t.Fatalf("writing config.toml: %v", err) + } +} + +func TestStateSyncConfigurer_Success(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"nodeId1@1.2.3.4:26656", "nodeId2@5.6.7.8:26656"}) + + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://1.2.3.4:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://5.6.7.8:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://1.2.3.4:26657/block?height=8000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + if err := configurer.Configure(context.Background(), StateSyncRequest{}); err != nil { + t.Fatalf("Configure failed: %v", err) + } + + if !markerExists(homeDir, stateSyncMarkerFile) { + t.Fatal("marker file should exist after successful configure") + } + + configDoc := readTOML(t, filepath.Join(homeDir, "config", "config.toml")) + ss := configDoc["statesync"].(map[string]any) + if ss["enable"] != true { + t.Error("expected statesync.enable = true in config.toml") + } + if ss["trust-height"] != int64(8000) { + t.Errorf("expected statesync.trust-height = 8000, got %v", ss["trust-height"]) + } + if ss["trust-hash"] != hash { + t.Errorf("expected trust-hash = %s, got %v", hash, ss["trust-hash"]) + } + if ss["rpc-servers"] != "1.2.3.4:26657,5.6.7.8:26657" { + t.Errorf("expected rpc-servers, got %v", ss["rpc-servers"]) + } + if ss["use-local-snapshot"] != false { + t.Errorf("expected use-local-snapshot = false, got %v", ss["use-local-snapshot"]) + } +} + +func TestStateSyncConfigurer_MarkerSkips(t *testing.T) { + homeDir := t.TempDir() + + if err := writeMarker(homeDir, stateSyncMarkerFile); err != nil { + t.Fatalf("writing marker: %v", err) + } + + configurer := NewStateSyncConfigurer(homeDir, &mockHTTPDoer{}) + if err := configurer.Configure(context.Background(), StateSyncRequest{}); err != nil { + t.Fatalf("expected nil error when marker exists, got: %v", err) + } +} + +func TestStateSyncConfigurer_NoPeers(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, nil) + + configurer := NewStateSyncConfigurer(homeDir, &mockHTTPDoer{}) + err := configurer.Configure(context.Background(), StateSyncRequest{}) + if err == nil { + t.Fatal("expected error when no peers in config") + } +} + +func TestStateSyncConfigurer_LowHeight(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"nodeId1@10.0.0.1:26656"}) + + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://10.0.0.1:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "500"} + }`)), + "http://10.0.0.1:26657/block?height=1": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + if err := configurer.Configure(context.Background(), StateSyncRequest{}); err != nil { + t.Fatalf("Configure failed: %v", err) + } + + configDoc := readTOML(t, filepath.Join(homeDir, "config", "config.toml")) + ss := configDoc["statesync"].(map[string]any) + if ss["trust-height"] != int64(1) { + t.Errorf("expected trustHeight clamped to 1, got %v", ss["trust-height"]) + } + if ss["trust-hash"] != hash { + t.Errorf("expected trustHash %s, got %v", hash, ss["trust-hash"]) + } + // Single peer should be duplicated to satisfy Tendermint requirement. + if ss["rpc-servers"] != "10.0.0.1:26657,10.0.0.1:26657" { + t.Errorf("expected duplicated rpc-servers, got %v", ss["rpc-servers"]) + } +} + +func TestStateSyncConfigurer_InvalidBlockHash(t *testing.T) { + tests := []struct { + name string + hash string + wantErr string + }{ + {"empty hash", "", "empty block hash"}, + {"short hash", "ABCDEF", "unexpected block hash length"}, + {"long hash", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA00", "unexpected block hash length"}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"nodeId1@1.2.3.4:26656"}) + + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://1.2.3.4:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://1.2.3.4:26657/block?height=8000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, tt.hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + err := configurer.Configure(context.Background(), StateSyncRequest{}) + if err == nil { + t.Fatal("expected error") + } + if !strings.Contains(err.Error(), tt.wantErr) { + t.Errorf("error %q does not contain %q", err, tt.wantErr) + } + }) + } +} + +func TestExtractRPCHosts(t *testing.T) { + tests := []struct { + name string + peers []string + maxHosts int + want []string + }{ + { + name: "standard peers", + peers: []string{"nodeId1@1.2.3.4:26656", "nodeId2@5.6.7.8:26656"}, + maxHosts: 2, + want: []string{"1.2.3.4", "5.6.7.8"}, + }, + { + name: "max hosts limits output", + peers: []string{"a@1.1.1.1:26656", "b@2.2.2.2:26656", "c@3.3.3.3:26656"}, + maxHosts: 2, + want: []string{"1.1.1.1", "2.2.2.2"}, + }, + { + name: "invalid format skipped", + peers: []string{"no-at-sign", "valid@10.0.0.1:26656"}, + maxHosts: 2, + want: []string{"10.0.0.1"}, + }, + { + name: "empty peers", + peers: []string{}, + maxHosts: 2, + want: nil, + }, + { + name: "host without port", + peers: []string{"nodeId@myhost"}, + maxHosts: 1, + want: []string{"myhost"}, + }, + { + name: "IPv6-style host with port", + peers: []string{"nodeId@[::1]:26656"}, + maxHosts: 1, + want: []string{"[::1]"}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := extractRPCHosts(tt.peers, tt.maxHosts) + if len(got) != len(tt.want) { + t.Fatalf("expected %d hosts, got %d: %v", len(tt.want), len(got), got) + } + for i := range got { + if got[i] != tt.want[i] { + t.Errorf("host[%d]: expected %q, got %q", i, tt.want[i], got[i]) + } + } + }) + } +} + +func TestStateSyncConfigurer_Handler(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"nodeId1@1.2.3.4:26656"}) + + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://1.2.3.4:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "5000"} + }`)), + "http://1.2.3.4:26657/block?height=3000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + handler := configurer.Handler() + + if _, err := handler(context.Background(), nil); err != nil { + t.Fatalf("Handler failed: %v", err) + } + + configDoc := readTOML(t, filepath.Join(homeDir, "config", "config.toml")) + ss := configDoc["statesync"].(map[string]any) + if ss["trust-height"] != int64(3000) { + t.Errorf("expected trustHeight 3000, got %v", ss["trust-height"]) + } +} + +func TestStateSyncConfigurer_NetworkWithBackfill(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"nodeId1@1.2.3.4:26656", "nodeId2@5.6.7.8:26656"}) + + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://1.2.3.4:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://5.6.7.8:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://1.2.3.4:26657/block?height=8000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + err := configurer.Configure(context.Background(), StateSyncRequest{ + TrustPeriod: "168h0m0s", + BackfillBlocks: 6000, + }) + if err != nil { + t.Fatalf("Configure failed: %v", err) + } + + configDoc := readTOML(t, filepath.Join(homeDir, "config", "config.toml")) + ss := configDoc["statesync"].(map[string]any) + if ss["trust-period"] != "168h0m0s" { + t.Errorf("expected trust-period = 168h0m0s, got %v", ss["trust-period"]) + } + if ss["backfill-blocks"] != int64(6000) { + t.Errorf("expected backfill-blocks = 6000, got %v", ss["backfill-blocks"]) + } + if ss["use-local-snapshot"] != false { + t.Errorf("expected use-local-snapshot = false, got %v", ss["use-local-snapshot"]) + } +} + +func TestStateSyncConfigurer_LocalSnapshot(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"nodeId1@1.2.3.4:26656", "nodeId2@5.6.7.8:26656"}) + + snapshotHeight := int64(198030000) + snapshotDir := filepath.Join(homeDir, "data", "snapshots", "198030000", "1") + if err := os.MkdirAll(snapshotDir, 0o755); err != nil { + t.Fatalf("creating snapshot dir: %v", err) + } + + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://1.2.3.4:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "198030000"} + }`)), + "http://5.6.7.8:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "198030000"} + }`)), + "http://1.2.3.4:26657/block?height=198030000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + err := configurer.Configure(context.Background(), StateSyncRequest{ + UseLocalSnapshot: true, + TrustPeriod: "9999h0m0s", + BackfillBlocks: 0, + }) + if err != nil { + t.Fatalf("Configure with local snapshot failed: %v", err) + } + + configDoc := readTOML(t, filepath.Join(homeDir, "config", "config.toml")) + ss := configDoc["statesync"].(map[string]any) + + if ss["trust-height"] != snapshotHeight { + t.Errorf("expected trust-height = %d (snapshot height), got %v", snapshotHeight, ss["trust-height"]) + } + if ss["trust-hash"] != hash { + t.Errorf("expected trust-hash = %s, got %v", hash, ss["trust-hash"]) + } + if ss["use-local-snapshot"] != true { + t.Errorf("expected use-local-snapshot = true, got %v", ss["use-local-snapshot"]) + } + if ss["trust-period"] != "9999h0m0s" { + t.Errorf("expected trust-period = 9999h0m0s, got %v", ss["trust-period"]) + } + if ss["enable"] != true { + t.Error("expected statesync.enable = true") + } +} + +func TestStateSyncConfigurer_LocalSnapshotNoDir(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"nodeId1@1.2.3.4:26656"}) + + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://1.2.3.4:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "100"} + }`)), + }, + } + configurer := NewStateSyncConfigurer(homeDir, mock) + err := configurer.Configure(context.Background(), StateSyncRequest{UseLocalSnapshot: true}) + if err == nil { + t.Fatal("expected error when no snapshot directory exists") + } + if !strings.Contains(err.Error(), "discovering local snapshot height") { + t.Errorf("unexpected error: %v", err) + } +} + +func TestDiscoverLocalSnapshotHeight(t *testing.T) { + homeDir := t.TempDir() + + snapshotBase := filepath.Join(homeDir, "data", "snapshots") + for _, dir := range []string{"198030000/1", "198020000/1", "notaheight"} { + if err := os.MkdirAll(filepath.Join(snapshotBase, dir), 0o755); err != nil { + t.Fatalf("creating dir: %v", err) + } + } + + h, err := discoverLocalSnapshotHeight(homeDir) + if err != nil { + t.Fatalf("discoverLocalSnapshotHeight failed: %v", err) + } + if h != 198030000 { + t.Errorf("expected height 198030000, got %d", h) + } +} + +func TestDiscoverLocalSnapshotHeight_Empty(t *testing.T) { + homeDir := t.TempDir() + if err := os.MkdirAll(filepath.Join(homeDir, "data", "snapshots"), 0o755); err != nil { + t.Fatalf("creating dir: %v", err) + } + + _, err := discoverLocalSnapshotHeight(homeDir) + if err == nil { + t.Fatal("expected error for empty snapshots directory") + } +} + +func TestReadPeersFromConfig(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"abc@1.2.3.4:26656", "def@5.6.7.8:26656"}) + + peers, err := readPeersFromConfig(homeDir) + if err != nil { + t.Fatalf("readPeersFromConfig failed: %v", err) + } + if len(peers) != 2 { + t.Fatalf("expected 2 peers, got %d", len(peers)) + } + if peers[0] != "abc@1.2.3.4:26656" { + t.Errorf("peer[0] = %q, want abc@1.2.3.4:26656", peers[0]) + } +} + +func TestReadPeersFromConfig_Empty(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, nil) + + peers, err := readPeersFromConfig(homeDir) + if err != nil { + t.Fatalf("readPeersFromConfig failed: %v", err) + } + if len(peers) != 0 { + t.Fatalf("expected 0 peers, got %d", len(peers)) + } +} + +func TestReadPeersFromConfig_NoConfigFile(t *testing.T) { + homeDir := t.TempDir() + peers, err := readPeersFromConfig(homeDir) + if err != nil { + t.Fatalf("readPeersFromConfig failed: %v", err) + } + if len(peers) != 0 { + t.Fatalf("expected 0 peers for missing config, got %d", len(peers)) + } +} + +// Caller-provided RpcServers are used verbatim, independent of persistent-peers +// (here there are none). +func TestStateSyncConfigurer_ExplicitRpcServers(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, nil) + + const witness = "syncer-0-0-0.syncer-0-0.arctic-1.svc.cluster.local:26657" + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://" + witness + "/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://" + witness + "/block?height=8000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + if err := configurer.Configure(context.Background(), StateSyncRequest{ + RpcServers: []string{witness}, + }); err != nil { + t.Fatalf("Configure failed: %v", err) + } + + ss := readTOML(t, filepath.Join(homeDir, "config", "config.toml"))["statesync"].(map[string]any) + if ss["rpc-servers"] != witness+","+witness { + t.Errorf("expected explicit witness padded to two, got %v", ss["rpc-servers"]) + } + if ss["trust-height"] != int64(8000) { + t.Errorf("expected trust-height 8000, got %v", ss["trust-height"]) + } +} + +// The regression case: a peer-derived witness whose host serves P2P but not RPC +// (an external NLB hostname). The reachable witness is kept; the dead one is +// dropped instead of being written and crashlooping seid. +func TestStateSyncConfigurer_DropsUnreachableWitness(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{ + "nodeId1@1.2.3.4:26656", + "nodeId2@syncer-0-0-p2p.arctic-1.prod.platform.sei.io:26656", + }) + + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + // Only 1.2.3.4 serves RPC; the p2p NLB host has no /status entry. + "http://1.2.3.4:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://1.2.3.4:26657/block?height=8000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + if err := configurer.Configure(context.Background(), StateSyncRequest{}); err != nil { + t.Fatalf("Configure failed: %v", err) + } + + ss := readTOML(t, filepath.Join(homeDir, "config", "config.toml"))["statesync"].(map[string]any) + if ss["rpc-servers"] != "1.2.3.4:26657,1.2.3.4:26657" { + t.Errorf("expected unreachable witness dropped, got %v", ss["rpc-servers"]) + } +} + +// The production-regression ordering: the first candidate is unreachable (a +// P2P-only NLB host), so the trust query must fall through to the reachable +// second one — reachable[0] is the post-probe slice, not the raw peer list. +func TestStateSyncConfigurer_PrimaryUnreachableFallsThrough(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{ + "nodeId1@syncer-0-0-p2p.arctic-1.prod.platform.sei.io:26656", // P2P-only, no RPC + "nodeId2@1.2.3.4:26656", // serves RPC + }) + + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://1.2.3.4:26657/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://1.2.3.4:26657/block?height=8000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + if err := configurer.Configure(context.Background(), StateSyncRequest{}); err != nil { + t.Fatalf("Configure failed: %v", err) + } + + ss := readTOML(t, filepath.Join(homeDir, "config", "config.toml"))["statesync"].(map[string]any) + if ss["trust-height"] != int64(8000) { + t.Errorf("expected trust query to fall through to the reachable witness, trust-height = %v", ss["trust-height"]) + } + if ss["trust-hash"] != hash { + t.Errorf("expected trust-hash %s, got %v", hash, ss["trust-hash"]) + } + if ss["rpc-servers"] != "1.2.3.4:26657,1.2.3.4:26657" { + t.Errorf("expected only the reachable witness, got %v", ss["rpc-servers"]) + } +} + +func TestWitnessScheme(t *testing.T) { + cases := []struct { + name, endpoint, want string + }{ + {"tls gateway on 443", "archive-0-rpc.arctic-1.platform.sei.io:443", "https"}, + {"in-cluster rpc on 26657", "syncer-0-internal.arctic-1.svc.cluster.local:26657", "http"}, + {"bare ip rpc", "1.2.3.4:26657", "http"}, + {"no port defaults to http", "some-host", "http"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := witnessScheme(tc.endpoint); got != tc.want { + t.Errorf("witnessScheme(%q) = %q, want %q", tc.endpoint, got, tc.want) + } + }) + } +} + +// Regression guard for the state-sync witness scheme/port mismatch that blocked +// every new K8s state-syncing node: a canonical syncer resolved to the public +// Istio HTTPRoute hostname on :443 must be probed and queried over https, not +// the previously-hardcoded http (which EOFed against the TLS listener). The mock +// only answers the https URL — an http probe would fall through as an +// "unexpected request" and the configure would fail. +func TestStateSyncConfigurer_TLSWitnessUsesHTTPS(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, nil) + + const witness = "archive-0-rpc.arctic-1.platform.sei.io:443" + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "https://" + witness + "/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "https://" + witness + "/block?height=8000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + if err := configurer.Configure(context.Background(), StateSyncRequest{ + RpcServers: []string{witness}, + }); err != nil { + t.Fatalf("Configure failed: %v", err) + } + + ss := readTOML(t, filepath.Join(homeDir, "config", "config.toml"))["statesync"].(map[string]any) + // rpc-servers is written as the bare host:port (seid attaches the scheme). + if ss["rpc-servers"] != witness+","+witness { + t.Errorf("expected TLS witness padded to two, got %v", ss["rpc-servers"]) + } + if ss["trust-height"] != int64(8000) { + t.Errorf("expected trust-height 8000, got %v", ss["trust-height"]) + } +} + +// The in-cluster plaintext path (a syncer's internal Service on :26657) must +// stay on http — the forward-compatible counterpart to the TLS guard above. +func TestStateSyncConfigurer_InternalWitnessUsesHTTP(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, nil) + + const witness = "syncer-0-internal.arctic-1.svc.cluster.local:26657" + hash := generateBlockHash() + mock := &mockHTTPDoer{ + responses: map[string]*http.Response{ + "http://" + witness + "/status": jsonResponse(wrapResult(`{ + "sync_info": {"latest_block_height": "10000"} + }`)), + "http://" + witness + "/block?height=8000": jsonResponse(wrapResult(fmt.Sprintf(`{ + "block_id": {"hash": %q} + }`, hash))), + }, + } + + configurer := NewStateSyncConfigurer(homeDir, mock) + if err := configurer.Configure(context.Background(), StateSyncRequest{ + RpcServers: []string{witness}, + }); err != nil { + t.Fatalf("Configure failed: %v", err) + } + + ss := readTOML(t, filepath.Join(homeDir, "config", "config.toml"))["statesync"].(map[string]any) + if ss["trust-height"] != int64(8000) { + t.Errorf("expected trust-height 8000, got %v", ss["trust-height"]) + } +} + +// When no candidate witness is reachable, fail at configure time with a clear +// error rather than writing a config that makes seid exit on "no witnesses". +func TestStateSyncConfigurer_NoReachableWitness(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, []string{"nodeId1@1.2.3.4:26656"}) + + configurer := NewStateSyncConfigurer(homeDir, &mockHTTPDoer{}) + err := configurer.Configure(context.Background(), StateSyncRequest{}) + if err == nil { + t.Fatal("expected error when no witness is reachable") + } + if !strings.Contains(err.Error(), "no reachable RPC witness") { + t.Errorf("unexpected error: %v", err) + } +} diff --git a/sidecar/tasks/stop_seid.go b/sidecar/tasks/stop_seid.go new file mode 100644 index 00000000..e3bdf890 --- /dev/null +++ b/sidecar/tasks/stop_seid.go @@ -0,0 +1,110 @@ +package tasks + +import ( + "context" + "fmt" + "log/slog" + "syscall" + "time" + + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/actions" + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +var stopSeidLog = seilog.NewLogger("seictl", "task", "stop-seid") + +// seidStopper SIGTERMs the running `seid start` process and waits for it to +// exit within a grace window, never escalating to SIGKILL — a stuck-but-alive +// validator is safer than a SIGKILL mid-commit. It is the shared stop core of +// restart-seid (stop then wait-for-up) and stop-seid (stop and leave held at +// the gate). op names the calling operation for the honesty-check error only. +type seidStopper struct { + signaler actions.ProcessSignaler + probeUp func(ctx context.Context) bool + gracePeriod time.Duration + exitPollInterval time.Duration + log *slog.Logger + op string +} + +// stop finds seid and SIGTERMs it. When /proc shows no seid it disambiguates on +// the local RPC: serving means the process exists but is invisible to us (a +// non-shared-PID-namespace profile) and we refuse to report a stop that did not +// happen; down means seid is already stopped or mid-restart (including the +// entrypoint's bash wait-loop window before it execs seid), a no-op success. +func (s seidStopper) stop(ctx context.Context) error { + pid, err := s.signaler.FindPID(restartSeidProcess) + if err != nil { + if s.probeUp(ctx) { + return fmt.Errorf("seid RPC is serving but its process was not found in /proc: %w — refusing to report a %s that did not happen", err, s.op) + } + s.log.Info("seid process not running and RPC down; nothing to stop", "reason", err.Error()) + return nil + } + s.log.Info("stopping seid", "pid", pid, "grace", s.gracePeriod) + return s.gracefulStop(ctx, pid) +} + +// gracefulStop SIGTERMs pid and polls until it exits or the grace window +// elapses. It never escalates to SIGKILL: if seid is still alive at the +// deadline the task fails and the process is left running for an operator. +func (s seidStopper) gracefulStop(ctx context.Context, pid int) error { + if err := s.signaler.Signal(pid, syscall.SIGTERM); err != nil { + return fmt.Errorf("sending SIGTERM to seid pid %d: %w", pid, err) + } + + ticker := time.NewTicker(s.exitPollInterval) + defer ticker.Stop() + deadline := time.After(s.gracePeriod) + + for { + select { + case <-ctx.Done(): + return ctx.Err() + case <-deadline: + return fmt.Errorf("seid pid %d still alive %s after SIGTERM; leaving it running (not force-killing a validator mid-commit)", pid, s.gracePeriod) + case <-ticker.C: + if !s.signaler.Alive(pid) { + s.log.Info("seid exited after SIGTERM", "pid", pid) + return nil + } + } + } +} + +// StopSeider SIGTERMs the co-located seid process and confirms it exited, then +// returns — unlike restart-seid it never waits for seid to come back up. The +// kubelet restarts the container and the start gate parks it (healthz 503) once +// the readiness flag is false. This is the hold's stop step: pair it with a +// prior mark-not-ready so the restarted container blocks at the gate instead of +// booting onto the data directory reset-data is about to clear. +type StopSeider struct { + stopper seidStopper +} + +// NewStopSeider builds a StopSeider with the real /proc + syscall + local-RPC +// implementations, sharing restart-seid's graceful-stop core and grace window. +func NewStopSeider() *StopSeider { + statusClient := rpc.NewStatusClient("", nil) + return &StopSeider{ + stopper: seidStopper{ + signaler: seidStartFinder{}, + probeUp: func(ctx context.Context) bool { return seidRPCUp(ctx, statusClient) }, + gracePeriod: restartSeidGracePeriod, + exitPollInterval: restartSeidExitPollInterval, + log: stopSeidLog, + op: "stop", + }, + } +} + +// Handler returns an engine.TaskHandler for the stop-seid task type. Params are +// empty: stop-seid is a fire-and-confirm operation. +func (s *StopSeider) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, _ struct{}) error { + return s.stopper.stop(ctx) + }) +} diff --git a/sidecar/tasks/stop_seid_test.go b/sidecar/tasks/stop_seid_test.go new file mode 100644 index 00000000..48b0bafc --- /dev/null +++ b/sidecar/tasks/stop_seid_test.go @@ -0,0 +1,76 @@ +package tasks + +import ( + "context" + "fmt" + "strings" + "syscall" + "testing" + "time" +) + +// newStopSeider builds a StopSeider wired to a test signaler and probe. +func newStopSeider(sig *fakeSignaler, probeUp func(context.Context) bool, grace time.Duration) *StopSeider { + return &StopSeider{ + stopper: seidStopper{ + signaler: sig, + probeUp: probeUp, + gracePeriod: grace, + exitPollInterval: time.Millisecond, + log: stopSeidLog, + op: "stop", + }, + } +} + +func TestStopSeider_StopsAndDoesNotWaitForUp(t *testing.T) { + sig := &fakeSignaler{findPID: 42} + sig.alive.Store(false) // exits immediately after SIGTERM + + // neverUp would hang restart-seid's waitForUp; stop-seid must ignore it. + if _, err := newStopSeider(sig, neverUp, time.Second).Handler()(context.Background(), nil); err != nil { + t.Fatalf("expected success without waiting for up, got %v", err) + } + if len(sig.signals) != 1 || sig.signals[0] != syscall.SIGTERM { + t.Errorf("expected single SIGTERM, got %v", sig.signals) + } +} + +func TestStopSeider_GraceTimeoutFailsWithoutSIGKILL(t *testing.T) { + sig := &fakeSignaler{findPID: 42} + sig.alive.Store(true) // never exits + + _, err := newStopSeider(sig, neverUp, 50*time.Millisecond).Handler()(context.Background(), nil) + if err == nil || !strings.Contains(err.Error(), "still alive") { + t.Fatalf("expected still-alive failure, got %v", err) + } + if len(sig.signals) != 1 || sig.signals[0] != syscall.SIGTERM { + t.Errorf("expected single SIGTERM and no SIGKILL, got %v", sig.signals) + } +} + +func TestStopSeider_NotFoundRPCDownIsSuccess(t *testing.T) { + sig := &fakeSignaler{findErr: fmt.Errorf("process \"seid\" not found in /proc")} + + if _, err := newStopSeider(sig, neverUp, time.Second).Handler()(context.Background(), nil); err != nil { + t.Fatalf("expected success when seid absent and RPC down, got %v", err) + } + if len(sig.signals) != 0 { + t.Errorf("expected no signals when seid not running, got %v", sig.signals) + } +} + +func TestStopSeider_NotFoundRPCUpRefuses(t *testing.T) { + sig := &fakeSignaler{findErr: fmt.Errorf("process \"seid\" not found in /proc")} + + _, err := newStopSeider(sig, upAfter(0), time.Second).Handler()(context.Background(), nil) + if err == nil { + t.Fatal("expected refusal when RPC serves but process not found") + } + if !strings.Contains(err.Error(), "stop that did not happen") { + t.Errorf("expected stop-that-did-not-happen refusal, got %v", err) + } + if len(sig.signals) != 0 { + t.Errorf("expected no signals on refusal, got %v", sig.signals) + } +} diff --git a/sidecar/tasks/transactions.go b/sidecar/tasks/transactions.go new file mode 100644 index 00000000..b97af53c --- /dev/null +++ b/sidecar/tasks/transactions.go @@ -0,0 +1,211 @@ +package tasks + +import ( + "context" + "encoding/hex" + "errors" + "fmt" + "strings" + "time" + + rpchttp "github.com/sei-protocol/sei-chain/sei-tendermint/rpc/client/http" + rpctypes "github.com/sei-protocol/sei-chain/sei-tendermint/rpc/jsonrpc/types" + + "github.com/sei-protocol/sei-chain/sei-cosmos/client" + "github.com/sei-protocol/sei-chain/sei-cosmos/codec" + codectypes "github.com/sei-protocol/sei-chain/sei-cosmos/codec/types" + cryptocodec "github.com/sei-protocol/sei-chain/sei-cosmos/crypto/codec" + sdk "github.com/sei-protocol/sei-chain/sei-cosmos/types" + authtx "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/tx" + authtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/auth/types" + banktypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/bank/types" + govtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/gov/types" + proposal "github.com/sei-protocol/sei-chain/sei-cosmos/x/params/types/proposal" + stakingtypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/staking/types" + upgradetypes "github.com/sei-protocol/sei-chain/sei-cosmos/x/upgrade/types" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + "github.com/sei-protocol/sei-k8s-controller/sidecar/rpc" +) + +// tmRPCTimeout bounds the underlying HTTP client so a wedged seid does +// not pin goroutines/conns indefinitely (the ctx-cancel select in +// AccountNumberSequence returns the task but leaves the inflight call +// behind until transport completes). +const tmRPCTimeout = 30 * time.Second + +// newSDKTxClient wires the production txClient against the local seid RPC. +func newSDKTxClient(cfg engine.ExecutionConfig, in SignAndBroadcastInput, fromAddr sdk.AccAddress) (txClient, error) { + rpcURL := rpc.DefaultEndpoint + if cfg.RPC != nil && cfg.RPC.Endpoint() != "" { + rpcURL = cfg.RPC.Endpoint() + } + tmClient, err := rpchttp.NewWithTimeout(rpcURL, tmRPCTimeout) + if err != nil { + return nil, fmt.Errorf("tendermint RPC client at %s: %w", rpcURL, err) + } + registry, cdc, txCfg := makeSignTxCodec() + clientCtx := client.Context{}. + WithChainID(in.ChainID). + WithCodec(cdc). + WithInterfaceRegistry(registry). + WithTxConfig(txCfg). + WithKeyring(cfg.Keyring). + WithClient(tmClient). + WithFromAddress(fromAddr). + WithFromName(in.KeyName). + WithAccountRetriever(authtypes.AccountRetriever{}). + WithBroadcastMode("sync") + return &sdkTxClient{clientCtx: clientCtx}, nil +} + +type sdkTxClient struct { + clientCtx client.Context +} + +func (c *sdkTxClient) AccountNumberSequence(ctx context.Context, fromAddr sdk.AccAddress) (uint64, uint64, error) { + // AccountRetriever ignores context.Context; wrap in a goroutine so + // a hung seid does not outlive the engine's cancellation. + type res struct { + accNum, seq uint64 + err error + } + ch := make(chan res, 1) + go func() { + a, s, err := authtypes.AccountRetriever{}.GetAccountNumberSequence(c.clientCtx, fromAddr) + ch <- res{a, s, err} + }() + select { + case <-ctx.Done(): + return 0, 0, ctx.Err() + case r := <-ch: + return r.accNum, r.seq, r.err + } +} + +func (c *sdkTxClient) BroadcastSync(ctx context.Context, txBytes []byte) (*sdk.TxResponse, error) { + // Bypass clientCtx.BroadcastTxSync (hardcodes context.Background) so + // the engine's cancellation propagates into the broadcast HTTP call. + node, err := c.clientCtx.GetNode() + if err != nil { + return nil, err + } + resp, err := node.BroadcastTxSync(ctx, txBytes) + if err != nil { + return nil, err + } + return sdk.NewResponseFormatBroadcastTx(resp), nil +} + +func (c *sdkTxClient) QueryTx(ctx context.Context, txHashHex string) (*sdk.TxResponse, bool, error) { + if c.clientCtx.Client == nil { + return nil, false, errors.New("tx client has no Tendermint RPC client") + } + hashBytes, err := hex.DecodeString(txHashHex) + if err != nil { + return nil, false, fmt.Errorf("decode tx hash %q: %w", txHashHex, err) + } + // ctx is plumbed through to the underlying http.Request via + // NewRequestWithContext in sei-tendermint's jsonrpc HTTP client + // (sei-tendermint/rpc/jsonrpc/client/http_json_client.go) — + // cancellation aborts the in-flight call, so the 30s tmRPCTimeout + // is a redundant upper bound, not the only guardrail. + res, err := c.clientCtx.Client.Tx(ctx, hashBytes, false) + if err != nil { + // Order matters: the indexing-disabled check runs first. sei-tendermint's + // disabled-index guard fires before any lookup, but a message that names + // the event sink AND says "not found" must classify as the terminal + // disabled case, not the retryable not-found one. Normalize to the + // sentinel here (the one layer that owns /tx semantics) so callers branch + // on errors.Is rather than re-matching the message text. + if isTxIndexingDisabled(err) { + return nil, false, fmt.Errorf("query /tx?hash=%s: %w", txHashHex, errTxIndexingDisabled) + } + if isTxNotFound(err) { + return nil, false, nil + } + return nil, false, fmt.Errorf("query /tx?hash=%s: %w", txHashHex, err) + } + resp := &sdk.TxResponse{ + Height: res.Height, + TxHash: txHashHex, + Code: res.TxResult.Code, + Codespace: res.TxResult.Codespace, + Data: string(res.TxResult.Data), + RawLog: res.TxResult.Log, + Info: res.TxResult.Info, + GasWanted: res.TxResult.GasWanted, + GasUsed: res.TxResult.GasUsed, + } + return resp, true, nil +} + +// isTxNotFound discriminates "the node has no record of this tx" from +// every other failure mode of /tx?hash=. Server-side, all handler +// errors come back as JSON-RPC CodeInternalError (sei-tendermint's +// MakeError, types.go:171); the "tx not found" message lands in the +// Data field. Transport errors (DNS, connection-refused, timeout) are +// NOT *rpctypes.RPCError, so the type assertion fences them out — a +// DNS "host not found" cannot be confused with a missing tx. +func isTxNotFound(err error) bool { + var rpcErr *rpctypes.RPCError + if !errors.As(err, &rpcErr) { + return false + } + return strings.Contains(rpcErr.Data, "not found") +} + +// errTxIndexingDisabled is the sentinel QueryTx returns when the target node +// has transaction indexing turned off (tx_index.indexer = "null"), so /tx?hash= +// cannot answer. The tx may still have committed — this reports that inclusion +// is unobservable from this node, NOT that the tx failed. Retrying the same +// node is futile (unlike a transient transport error), so callers fail +// terminally; the message tells the operator how to unblock. +var errTxIndexingDisabled = errors.New( + "node transaction indexing is disabled (no kvEventSink): the tx may already be on chain — " + + "verify via an indexed RPC before re-running, or enable tx_index on the target node") + +// isTxIndexingDisabled reports whether err is the node's indexing-disabled +// response. Same *rpctypes.RPCError fence as isTxNotFound (a transport error is +// not an RPCError, so it stays transient); like there, the detail lands in +// Data. sei-tendermint's /tx handler names the missing sink in two spellings — +// the pre-lookup guard ("...disabled due to no kvEventSink") and the fallback +// ("...KV event sink being disabled") — so match both, case-insensitively. A +// genuine miss on an index-ENABLED node is "tx (…) not found" carrying neither +// token, and stays retryable via isTxNotFound. +func isTxIndexingDisabled(err error) bool { + var rpcErr *rpctypes.RPCError + if !errors.As(err, &rpcErr) { + return false + } + d := strings.ToLower(rpcErr.Data) + return strings.Contains(d, "kveventsink") || strings.Contains(d, "kv event sink") +} + +// newSignTxInterfaceRegistry registers only the proto interfaces sign-tx +// needs. Adding more pulls in transitive deps (notably x/wasm via x/evm) +// that break CGO_ENABLED=0 builds. +func newSignTxInterfaceRegistry() codectypes.InterfaceRegistry { + registry := codectypes.NewInterfaceRegistry() + cryptocodec.RegisterInterfaces(registry) + authtypes.RegisterInterfaces(registry) + banktypes.RegisterInterfaces(registry) + stakingtypes.RegisterInterfaces(registry) + govtypes.RegisterInterfaces(registry) + upgradetypes.RegisterInterfaces(registry) + // x/params ParameterChangeProposal as a gov Content impl (gov-param-change + // task). proposal does not pull x/wasm, so CGO_ENABLED=0 builds stay clean. + proposal.RegisterInterfaces(registry) + return registry +} + +// makeSignTxCodec is the single source of truth for the proto registry, +// codec, and TxConfig used by both production sdkTxClient wiring and the +// sign path. If interfaces ever diverge between the two, sign/encode +// breaks in confusing ways; sharing here is load-bearing. +func makeSignTxCodec() (codectypes.InterfaceRegistry, codec.Codec, client.TxConfig) { + registry := newSignTxInterfaceRegistry() + cdc := codec.NewProtoCodec(registry) + txCfg := authtx.NewTxConfig(cdc, authtx.DefaultSignModes) + return registry, cdc, txCfg +} diff --git a/sidecar/tasks/transactions_test.go b/sidecar/tasks/transactions_test.go new file mode 100644 index 00000000..a15dc666 --- /dev/null +++ b/sidecar/tasks/transactions_test.go @@ -0,0 +1,50 @@ +package tasks + +import ( + "errors" + "fmt" + "testing" + + rpctypes "github.com/sei-protocol/sei-chain/sei-tendermint/rpc/jsonrpc/types" +) + +func TestIsTxNotFound(t *testing.T) { + cases := []struct { + name string + err error + want bool + }{ + { + name: "server-side tx not found", + err: &rpctypes.RPCError{Code: int(rpctypes.CodeInternalError), Message: "Internal error", Data: "tx (DEADBEEF) not found, err: index not enabled"}, + want: true, + }, + { + name: "server-side other internal error", + err: &rpctypes.RPCError{Code: int(rpctypes.CodeInternalError), Message: "Internal error", Data: "kvEventSink disabled"}, + want: false, + }, + { + name: "wrapped RPCError still discriminates", + err: fmt.Errorf("query /tx: %w", &rpctypes.RPCError{Code: int(rpctypes.CodeInternalError), Data: "tx (BEEF) not found, err: x"}), + want: true, + }, + { + name: "transport error containing 'not found' is not classified as tx-not-found", + err: errors.New("Get http://seid:26657/tx: dial tcp: lookup seid: no such host (not found)"), + want: false, + }, + { + name: "nil err", + err: nil, + want: false, + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + if got := isTxNotFound(c.err); got != c.want { + t.Fatalf("isTxNotFound(%v) = %v, want %v", c.err, got, c.want) + } + }) + } +} diff --git a/sidecar/tasks/typed_handler_integration_test.go b/sidecar/tasks/typed_handler_integration_test.go new file mode 100644 index 00000000..5ec4d727 --- /dev/null +++ b/sidecar/tasks/typed_handler_integration_test.go @@ -0,0 +1,339 @@ +package tasks + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" +) + +// TestDeserialize_SnapshotRestore verifies that the snapshot-restore handler +// correctly deserializes simple string fields from the wire format. +func TestDeserialize_SnapshotRestore(t *testing.T) { + homeDir := t.TempDir() + // Pre-write a marker so the handler returns early without S3 calls. + if err := writeMarker(homeDir, restoreMarkerFile); err != nil { + t.Fatal(err) + } + + restorer, err := NewSnapshotRestorer(homeDir, "b", "r", "c", nil, nil) + if err != nil { + t.Fatal(err) + } + handler := restorer.Handler() + + // The handler should succeed (skip via marker) without a parse error. + _, err = handler(context.Background(), map[string]any{}) + if err != nil { + t.Fatalf("snapshot-restore handler returned error: %v", err) + } +} + +// TestDeserialize_ConfigPatch verifies that the config-patch handler +// correctly deserializes nested map[string]map[string]any from the wire format. +func TestDeserialize_ConfigPatch(t *testing.T) { + homeDir := t.TempDir() + setupConfigFile(t, homeDir, ` +[p2p] +persistent-peers = "" +`) + + patcher := NewConfigPatcher(homeDir) + handler := patcher.Handler() + + params := map[string]any{ + "files": map[string]any{ + "config.toml": map[string]any{ + "p2p": map[string]any{ + "persistent-peers": "node1@1.2.3.4:26656", + }, + }, + }, + } + + _, err := handler(context.Background(), params) + if err != nil { + t.Fatalf("config-patch handler returned error: %v", err) + } + + doc := readTOML(t, filepath.Join(homeDir, "config", "config.toml")) + p2p := doc["p2p"].(map[string]any) + if p2p["persistent-peers"] != "node1@1.2.3.4:26656" { + t.Errorf("peers = %q, want %q", p2p["persistent-peers"], "node1@1.2.3.4:26656") + } +} + +// TestDeserialize_AssembleGenesis verifies that the assemble-genesis handler +// correctly deserializes the nodes array with name fields from the wire format. +func TestDeserialize_AssembleGenesis(t *testing.T) { + // We only test deserialization, not the full S3 flow, so we expect + // a validation error for missing S3 bucket when bucket is empty. + handler := NewGenesisAssembler(t.TempDir(), "my-bucket", "us-east-1", "test-chain", nil, nil).Handler() + + params := map[string]any{ + "accountBalance": "10000000usei", + "namespace": "default", + "nodes": []any{ + map[string]any{"name": "val-0"}, + map[string]any{"name": "val-1"}, + }, + } + + // This will fail at the S3 download step (no real S3), but if it gets + // past param parsing without a "parsing params" error, deserialization worked. + _, err := handler(context.Background(), params) + if err == nil { + t.Fatal("expected error (no S3 client), got nil") + } + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } +} + +// TestDeserialize_AwaitCondition verifies that the await-condition handler +// correctly deserializes int64 from float64 (JSON number coercion) and +// string fields from the wire format. +func TestDeserialize_AwaitCondition(t *testing.T) { + handler := NewConditionWaiter(nil).Handler() + + // float64(500) simulates what json.Unmarshal produces for JSON numbers + // when the target is map[string]any. + params := map[string]any{ + "condition": "height", + "targetHeight": float64(500), + } + + // The handler will try to poll RPC (and fail because there's no server), + // but it should parse params without error. We use a context that + // cancels immediately to avoid blocking. + ctx, cancel := context.WithCancel(context.Background()) + cancel() + + _, err := handler(ctx, params) + if err == nil { + t.Fatal("expected context error, got nil") + } + // If we get a context error (not a parsing error), deserialization succeeded. + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } +} + +// TestDeserialize_ConfigApply verifies that the config-apply handler +// correctly deserializes the ConfigIntent (overrides map + mode string). +func TestDeserialize_ConfigApply(t *testing.T) { + homeDir := t.TempDir() + applier := NewConfigApplier(homeDir) + handler := applier.Handler() + + params := map[string]any{ + "mode": "full", + "incremental": false, + "overrides": map[string]any{ + "evm.http_port": "9545", + }, + } + + _, err := handler(context.Background(), params) + if err != nil { + t.Fatalf("config-apply handler returned error: %v", err) + } + + // Verify the config was actually written. + configPath := filepath.Join(homeDir, "config", "config.toml") + if _, err := os.Stat(configPath); os.IsNotExist(err) { + t.Fatal("config.toml was not written") + } +} + +// TestDeserialize_ConfigReload verifies that the config-reload handler +// correctly deserializes the fields map from the wire format. +func TestDeserialize_ConfigReload(t *testing.T) { + // config-reload needs a valid on-disk config to read. + // We just check that deserialization doesn't fail when fields is empty + // (it'll fail with a validation error, not a parse error). + handler := NewConfigReloader(t.TempDir()).Handler() + + params := map[string]any{ + "fields": map[string]any{}, + } + + _, err := handler(context.Background(), params) + if err == nil { + t.Fatal("expected error for empty fields, got nil") + } + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } + if !strings.Contains(err.Error(), "at least one field") { + t.Errorf("expected 'at least one field' error, got: %v", err) + } +} + +// TestNewSnapshotUploader_RejectsEmptyConfig verifies that the constructor +// fails fast when bucket, region, or chainID is empty rather than producing +// an uploader whose runLoop polls forever uploading nothing. +func TestNewSnapshotUploader_RejectsEmptyConfig(t *testing.T) { + tests := []struct { + name string + bucket, region, chainID, want string + }{ + {"empty bucket", "", "us-east-1", "c", "required"}, + {"empty region", "b", "", "c", "required"}, + {"empty chainID", "b", "us-east-1", "", "required"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, err := NewSnapshotUploader(t.TempDir(), tt.bucket, tt.region, tt.chainID, 0, nil) + if err == nil { + t.Fatal("expected constructor error, got nil") + } + if !strings.Contains(err.Error(), tt.want) { + t.Errorf("error %q does not mention %q", err.Error(), tt.want) + } + }) + } +} + +// TestDeserialize_ConfigureGenesis verifies that the configure-genesis handler +// works with empty params for an embedded chain. +func TestDeserialize_ConfigureGenesis(t *testing.T) { + homeDir := t.TempDir() + fetcher := NewGenesisFetcher(homeDir, "pacific-1", "test-bucket", "us-east-2", nil) + handler := fetcher.Handler() + + _, err := handler(context.Background(), map[string]any{}) + if err != nil { + t.Fatalf("unexpected error for embedded chain: %v", err) + } +} + +// TestDeserialize_UploadArtifacts verifies that the upload-genesis-artifacts handler +// correctly deserializes the S3 and node name params from the wire format. +func TestDeserialize_UploadArtifacts(t *testing.T) { + handler := NewGenesisArtifactUploader(t.TempDir(), "test-bucket", "us-east-2", "test-chain", nil).Handler() + + params := map[string]any{ + "nodeName": "", + } + + _, err := handler(context.Background(), params) + if err == nil { + t.Fatal("expected error for empty nodeName, got nil") + } + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } + if !strings.Contains(err.Error(), "missing required param 'nodeName'") { + t.Errorf("expected nodeName validation error, got: %v", err) + } +} + +// TestDeserialize_GenerateIdentity verifies that the generate-identity handler +// correctly deserializes chainId and moniker from the wire format. +func TestDeserialize_GenerateIdentity(t *testing.T) { + handler := NewIdentityGenerator(t.TempDir()).Handler() + + params := map[string]any{ + "chainId": "", + "moniker": "val-0", + } + + _, err := handler(context.Background(), params) + if err == nil { + t.Fatal("expected error for empty chainId, got nil") + } + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } + if !strings.Contains(err.Error(), "missing required param 'chainId'") { + t.Errorf("expected chainId validation error, got: %v", err) + } +} + +// TestDeserialize_SetGenesisPeers verifies that the set-genesis-peers handler +// correctly deserializes S3 coordinates from the wire format. +func TestDeserialize_SetGenesisPeers(t *testing.T) { + handler := NewGenesisPeersSetter(t.TempDir(), "test-bucket", "us-east-2", "test-chain", nil).Handler() + + // The handler will try to download peers.json from S3 — that will fail + // since there's no real S3 client. But it proves deserialization worked. + _, err := handler(context.Background(), map[string]any{}) + if err == nil { + t.Fatal("expected error (no S3), got nil") + } + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } +} + +// TestDeserialize_StateSync verifies that the state-sync handler +// correctly deserializes useLocalSnapshot (bool), trustPeriod (string), +// and backfillBlocks (int64 from float64) from the wire format. +func TestDeserialize_StateSync(t *testing.T) { + homeDir := t.TempDir() + setupPeersInConfig(t, homeDir, nil) + + handler := NewStateSyncConfigurer(homeDir, nil).Handler() + + params := map[string]any{ + "useLocalSnapshot": true, + "trustPeriod": "168h0m0s", + "backfillBlocks": float64(6000), + } + + // Will fail because there are no peers, but deserialization should succeed. + _, err := handler(context.Background(), params) + if err == nil { + t.Fatal("expected error (no peers), got nil") + } + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } +} + +// TestDeserialize_ResultExport verifies that the result-export handler +// correctly deserializes the bucket, region, and optional fields. +func TestDeserialize_ResultExport(t *testing.T) { + handler := NewResultExporter(t.TempDir(), "test-1", "test-pod-0", nil).Handler() + + params := map[string]any{ + "bucket": "", + "region": "us-east-1", + } + + _, err := handler(context.Background(), params) + if err == nil { + t.Fatal("expected error for empty bucket, got nil") + } + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } + if !strings.Contains(err.Error(), "missing required param 'bucket'") { + t.Errorf("expected bucket validation error, got: %v", err) + } +} + +// TestDeserialize_GenerateGentx verifies that the generate-gentx handler +// correctly deserializes all string params from the wire format. +func TestDeserialize_GenerateGentx(t *testing.T) { + handler := NewGentxGenerator(t.TempDir()).Handler() + + params := map[string]any{ + "chainId": "", + "stakingAmount": "1000usei", + "accountBalance": "10000usei", + } + + _, err := handler(context.Background(), params) + if err == nil { + t.Fatal("expected error for empty chainId, got nil") + } + if strings.Contains(err.Error(), "parsing params") { + t.Fatalf("deserialization failed: %v", err) + } + if !strings.Contains(err.Error(), "missing required param 'chainId'") { + t.Errorf("expected chainId validation error, got: %v", err) + } +} diff --git a/sidecar/tasks/upload_genesis_artifacts.go b/sidecar/tasks/upload_genesis_artifacts.go new file mode 100644 index 00000000..e49dce3d --- /dev/null +++ b/sidecar/tasks/upload_genesis_artifacts.go @@ -0,0 +1,155 @@ +package tasks + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager" + "github.com/sei-protocol/seilog" + + "github.com/sei-protocol/sei-k8s-controller/sidecar/engine" + seis3 "github.com/sei-protocol/sei-k8s-controller/sidecar/s3" +) + +var artifactLog = seilog.NewLogger("seictl", "task", "upload-genesis-artifacts") + +const artifactUploadMarkerFile = ".sei-sidecar-artifact-upload-done" + +// UploadArtifactsRequest holds the typed parameters for the upload-genesis-artifacts task. +type UploadArtifactsRequest struct { + NodeName string `json:"nodeName"` +} + +// GenesisArtifactUploader uploads the gentx file and a node identity +// manifest to S3 so the assembler can collect them. +type GenesisArtifactUploader struct { + homeDir string + bucket string + region string + chainID string + s3UploaderFactory seis3.UploaderFactory +} + +// NewGenesisArtifactUploader creates an uploader targeting the given home directory. +// Bucket, region, and chainID are read from environment at construction time. +func NewGenesisArtifactUploader(homeDir, bucket, region, chainID string, factory seis3.UploaderFactory) *GenesisArtifactUploader { + if factory == nil { + factory = seis3.DefaultUploaderFactory + } + return &GenesisArtifactUploader{ + homeDir: homeDir, + bucket: bucket, + region: region, + chainID: chainID, + s3UploaderFactory: factory, + } +} + +// Handler returns an engine.TaskHandler for the upload-genesis-artifacts task type. +func (u *GenesisArtifactUploader) Handler() engine.TaskHandler { + return engine.TypedHandler(func(ctx context.Context, cfg UploadArtifactsRequest) error { + if markerExists(u.homeDir, artifactUploadMarkerFile) { + artifactLog.Debug("already completed, skipping") + return nil + } + + if cfg.NodeName == "" { + return fmt.Errorf("upload-genesis-artifacts: missing required param 'nodeName'") + } + + uploader, err := u.s3UploaderFactory(ctx, u.region) + if err != nil { + return fmt.Errorf("upload-genesis-artifacts: building S3 uploader: %w", err) + } + + prefix := u.chainID + "/" + nodePrefix := prefix + cfg.NodeName + "/" + + if err := u.uploadGentx(ctx, uploader, u.bucket, nodePrefix); err != nil { + return err + } + + if err := u.uploadIdentity(ctx, uploader, u.bucket, nodePrefix); err != nil { + return err + } + + artifactLog.Info("artifacts uploaded", "bucket", u.bucket, "prefix", nodePrefix) + return writeMarker(u.homeDir, artifactUploadMarkerFile) + }) +} + +// uploadGentx finds the single gentx file in config/gentx/ and uploads it. +func (u *GenesisArtifactUploader) uploadGentx(ctx context.Context, uploader seis3.Uploader, bucket, nodePrefix string) error { + gentxDir := filepath.Join(u.homeDir, "config", "gentx") + entries, err := os.ReadDir(gentxDir) + if err != nil { + return fmt.Errorf("upload-genesis-artifacts: reading gentx dir: %w", err) + } + + var gentxFile string + for _, e := range entries { + if !e.IsDir() && strings.HasSuffix(e.Name(), ".json") { + gentxFile = e.Name() + break + } + } + if gentxFile == "" { + return fmt.Errorf("upload-genesis-artifacts: no gentx JSON file found in %s", gentxDir) + } + + data, err := os.ReadFile(filepath.Join(gentxDir, gentxFile)) + if err != nil { + return fmt.Errorf("upload-genesis-artifacts: reading %s: %w", gentxFile, err) + } + + key := nodePrefix + "gentx.json" + artifactLog.Info("uploading gentx", "key", key) + _, err = uploader.UploadObject(ctx, &transfermanager.UploadObjectInput{ + Bucket: aws.String(bucket), + Key: aws.String(key), + Body: bytes.NewReader(data), + ContentType: aws.String("application/json"), + }) + if err != nil { + return seis3.ClassifyS3Error("upload-genesis-artifacts", bucket, key, u.region, err) + } + return nil +} + +// uploadIdentity reads node_key.json, extracts the node ID, and uploads +// a minimal identity manifest. The assembler uses this to know which +// nodes participated. +func (u *GenesisArtifactUploader) uploadIdentity(ctx context.Context, uploader seis3.Uploader, bucket, nodePrefix string) error { + nodeKeyPath := filepath.Join(u.homeDir, "config", "node_key.json") + nodeKeyData, err := os.ReadFile(nodeKeyPath) + if err != nil { + return fmt.Errorf("upload-genesis-artifacts: reading node_key.json: %w", err) + } + + identity := map[string]any{ + "node_key": json.RawMessage(nodeKeyData), + } + data, err := json.Marshal(identity) + if err != nil { + return fmt.Errorf("upload-genesis-artifacts: marshaling identity: %w", err) + } + + key := nodePrefix + "identity.json" + artifactLog.Info("uploading identity", "key", key) + _, err = uploader.UploadObject(ctx, &transfermanager.UploadObjectInput{ + Bucket: aws.String(bucket), + Key: aws.String(key), + Body: bytes.NewReader(data), + ContentType: aws.String("application/json"), + }) + if err != nil { + return seis3.ClassifyS3Error("upload-genesis-artifacts", bucket, key, u.region, err) + } + return nil +} diff --git a/sidecar/tasks/upload_genesis_artifacts_test.go b/sidecar/tasks/upload_genesis_artifacts_test.go new file mode 100644 index 00000000..28152d46 --- /dev/null +++ b/sidecar/tasks/upload_genesis_artifacts_test.go @@ -0,0 +1,124 @@ +package tasks + +import ( + "context" + "os" + "path/filepath" + "testing" +) + +func TestArtifactUploader_UploadsGentxAndIdentity(t *testing.T) { + homeDir := t.TempDir() + + gentxDir := filepath.Join(homeDir, "config", "gentx") + if err := os.MkdirAll(gentxDir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(gentxDir, "gentx-abc123.json"), []byte(`{"gentx":"data"}`), 0o644); err != nil { + t.Fatal(err) + } + + configDir := filepath.Join(homeDir, "config") + if err := os.WriteFile(filepath.Join(configDir, "node_key.json"), []byte(`{"priv_key":{"value":"base64key"}}`), 0o644); err != nil { + t.Fatal(err) + } + + mock := newMockS3Uploader() + uploader := NewGenesisArtifactUploader(homeDir, "test-bucket", "us-east-2", "test-chain", mockUploaderFactory(mock)) + handler := uploader.Handler() + + _, err := handler(context.Background(), map[string]any{ + "nodeName": "val-0", + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + if _, ok := mock.uploads["test-bucket/test-chain/val-0/gentx.json"]; !ok { + t.Errorf("expected gentx.json upload, uploads: %v", keys(mock.uploads)) + } + if _, ok := mock.uploads["test-bucket/test-chain/val-0/identity.json"]; !ok { + t.Errorf("expected identity.json upload, uploads: %v", keys(mock.uploads)) + } +} + +func TestArtifactUploader_Idempotent(t *testing.T) { + homeDir := t.TempDir() + + gentxDir := filepath.Join(homeDir, "config", "gentx") + if err := os.MkdirAll(gentxDir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(gentxDir, "gentx-abc.json"), []byte(`{}`), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(homeDir, "config", "node_key.json"), []byte(`{}`), 0o644); err != nil { + t.Fatal(err) + } + + mock := newMockS3Uploader() + uploader := NewGenesisArtifactUploader(homeDir, "test-bucket", "us-east-2", "test-chain", mockUploaderFactory(mock)) + handler := uploader.Handler() + + params := map[string]any{ + "nodeName": "n", + } + if _, err := handler(context.Background(), params); err != nil { + t.Fatalf("first call: %v", err) + } + firstUploads := len(mock.uploads) + + if _, err := handler(context.Background(), params); err != nil { + t.Fatalf("second call: %v", err) + } + if len(mock.uploads) != firstUploads { + t.Fatal("expected no new uploads on second call") + } +} + +func TestArtifactUploader_MissingParams(t *testing.T) { + handler := NewGenesisArtifactUploader(t.TempDir(), "test-bucket", "us-east-2", "test-chain", nil).Handler() + + tests := []struct { + name string + params map[string]any + }{ + {"missing nodeName", map[string]any{}}, + {"empty nodeName", map[string]any{"nodeName": ""}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if _, err := handler(context.Background(), tt.params); err == nil { + t.Fatal("expected error") + } + }) + } +} + +func TestArtifactUploader_NoGentxFile(t *testing.T) { + homeDir := t.TempDir() + if err := os.MkdirAll(filepath.Join(homeDir, "config", "gentx"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(homeDir, "config", "node_key.json"), []byte(`{}`), 0o644); err != nil { + t.Fatal(err) + } + + mock := newMockS3Uploader() + handler := NewGenesisArtifactUploader(homeDir, "test-bucket", "us-east-2", "test-chain", mockUploaderFactory(mock)).Handler() + + _, err := handler(context.Background(), map[string]any{ + "nodeName": "n", + }) + if err == nil { + t.Fatal("expected error when no gentx file exists") + } +} + +func keys(m map[string][]byte) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + return out +} diff --git a/sidecarapi/go.mod b/sidecarapi/go.mod index a0ac09f6..5b63d44b 100644 --- a/sidecarapi/go.mod +++ b/sidecarapi/go.mod @@ -7,6 +7,7 @@ require ( github.com/google/uuid v1.6.0 github.com/leanovate/gopter v0.2.11 github.com/oapi-codegen/runtime v1.6.0 + github.com/pelletier/go-toml/v2 v2.2.2 github.com/sei-protocol/sei-config v0.0.25 ) diff --git a/sidecarapi/go.sum b/sidecarapi/go.sum index 7d117fd1..aa898816 100644 --- a/sidecarapi/go.sum +++ b/sidecarapi/go.sum @@ -210,6 +210,8 @@ github.com/oapi-codegen/runtime v1.6.0 h1:7Xx+GlueD6nRuyKoCPzL434Jfi3BetbiJOrzCH github.com/oapi-codegen/runtime v1.6.0/go.mod h1:GwV7hC2hviaMzj+ITfHVRESK5J2W/GefVwIND/bMGvU= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= +github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM= +github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= @@ -240,12 +242,19 @@ github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An github.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns= github.com/spkg/bom v0.0.0-20160624110644-59b7046e48ad/go.mod h1:qLr4V1qq6nMqFKkMo8ZTx3f+BZEkzsRUY10Xsm2mwU0= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= diff --git a/sidecarapi/tomlpatch/file.go b/sidecarapi/tomlpatch/file.go new file mode 100644 index 00000000..5019a731 --- /dev/null +++ b/sidecarapi/tomlpatch/file.go @@ -0,0 +1,51 @@ +package tomlpatch + +import ( + "os" + "path/filepath" +) + +// WriteFileAtomic writes content to path atomically by writing to a temp file +// first, then renaming. +func WriteFileAtomic(path string, content []byte, perm os.FileMode) error { + dir := filepath.Dir(path) + + tmpFile, err := os.CreateTemp(dir, ".tmp-*") + if err != nil { + return err + } + tmpName := tmpFile.Name() + defer func() { + if tmpFile != nil { + _ = tmpFile.Close() + _ = os.Remove(tmpName) + } + }() + + if _, err := tmpFile.Write(content); err != nil { + return err + } + if err := tmpFile.Sync(); err != nil { + return err + } + if err := tmpFile.Close(); err != nil { + return err + } + tmpFile = nil + + if err := os.Chmod(tmpName, perm); err != nil { + return err + } + return os.Rename(tmpName, path) +} + +// SetNestedValue sets doc[section][key] = value, creating the section map +// if needed. +func SetNestedValue(doc map[string]any, section, key string, value any) { + sec, ok := doc[section].(map[string]any) + if !ok { + sec = make(map[string]any) + doc[section] = sec + } + sec[key] = value +} diff --git a/sidecarapi/tomlpatch/json.go b/sidecarapi/tomlpatch/json.go new file mode 100644 index 00000000..45fa7324 --- /dev/null +++ b/sidecarapi/tomlpatch/json.go @@ -0,0 +1,55 @@ +package tomlpatch + +import ( + "bytes" + "encoding/json" + "os" +) + +// ReadJSON reads and parses a JSON file into a map. +// Returns an empty map if the file does not exist. +func ReadJSON(path string) (map[string]any, error) { + data, err := os.ReadFile(path) + if err != nil { + if os.IsNotExist(err) { + return make(map[string]any), nil + } + return nil, err + } + var doc map[string]any + if err := json.Unmarshal(data, &doc); err != nil { + return nil, err + } + return doc, nil +} + +// WriteJSON atomically encodes doc as indented JSON and writes it to path. +func WriteJSON(path string, doc any) error { + var buf bytes.Buffer + encoder := json.NewEncoder(&buf) + encoder.SetIndent("", " ") + if err := encoder.Encode(doc); err != nil { + return err + } + return WriteFileAtomic(path, buf.Bytes(), 0o644) +} + +// UnmarshalJSON parses raw JSON bytes into a map. +func UnmarshalJSON(data []byte) (map[string]any, error) { + var doc map[string]any + if err := json.Unmarshal(data, &doc); err != nil { + return nil, err + } + return doc, nil +} + +// MarshalJSON encodes a map as indented JSON bytes. +func MarshalJSON(doc any) ([]byte, error) { + var buf bytes.Buffer + encoder := json.NewEncoder(&buf) + encoder.SetIndent("", " ") + if err := encoder.Encode(doc); err != nil { + return nil, err + } + return buf.Bytes(), nil +} diff --git a/sidecarapi/tomlpatch/merge.go b/sidecarapi/tomlpatch/merge.go new file mode 100644 index 00000000..3e2d463e --- /dev/null +++ b/sidecarapi/tomlpatch/merge.go @@ -0,0 +1,36 @@ +// Package tomlpatch provides merge-patch utilities for TOML and JSON documents. +// +// It lives in the contract module because both the sidecar (config-patch, +// config-apply, state-sync) and the seictl CLI (config, genesis, patch) need +// it, and neither may import the other: the CLI must stay free of the chain +// graph the sidecar carries. A copy in each would put the same drift risk +// across a module boundary that this migration exists to remove. +package tomlpatch + +// Merge performs a recursive merge-patch of patch into original. +// nil values in the patch delete the corresponding key from original. +// Non-map patches replace the original entirely. +func Merge(original, patch any) any { + patchMap, patchIsMap := patch.(map[string]any) + if !patchIsMap { + return patch + } + originalMap, originalIsMap := original.(map[string]any) + if !originalIsMap { + originalMap = make(map[string]any) + } + result := make(map[string]any) + for k, v := range originalMap { + result[k] = v + } + for key, patchAt := range patchMap { + if patchAt == nil { + delete(result, key) + } else if originalAt, exists := result[key]; exists { + result[key] = Merge(originalAt, patchAt) + } else { + result[key] = patchAt + } + } + return result +} diff --git a/sidecarapi/tomlpatch/merge_test.go b/sidecarapi/tomlpatch/merge_test.go new file mode 100644 index 00000000..80f35376 --- /dev/null +++ b/sidecarapi/tomlpatch/merge_test.go @@ -0,0 +1,55 @@ +package tomlpatch + +import ( + "reflect" + "testing" +) + +func TestMerge(t *testing.T) { + tests := []struct { + name string + original any + patch any + expected any + }{ + { + name: "merge two maps", + original: map[string]any{"a": 1, "b": 2}, + patch: map[string]any{"b": 3, "c": 4}, + expected: map[string]any{"a": 1, "b": 3, "c": 4}, + }, + { + name: "patch is not a map", + original: map[string]any{"a": 1}, + patch: "string value", + expected: "string value", + }, + { + name: "original is not a map", + original: "original string", + patch: map[string]any{"a": 1}, + expected: map[string]any{"a": 1}, + }, + { + name: "null value deletes key", + original: map[string]any{"a": 1, "b": 2}, + patch: map[string]any{"b": nil}, + expected: map[string]any{"a": 1}, + }, + { + name: "nested map merge", + original: map[string]any{"obj": map[string]any{"x": 1, "y": 2}}, + patch: map[string]any{"obj": map[string]any{"y": 3, "z": 4}}, + expected: map[string]any{"obj": map[string]any{"x": 1, "y": 3, "z": 4}}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + result := Merge(tt.original, tt.patch) + if !reflect.DeepEqual(result, tt.expected) { + t.Errorf("Merge() = %v, want %v", result, tt.expected) + } + }) + } +} diff --git a/sidecarapi/tomlpatch/toml.go b/sidecarapi/tomlpatch/toml.go new file mode 100644 index 00000000..423c47d3 --- /dev/null +++ b/sidecarapi/tomlpatch/toml.go @@ -0,0 +1,54 @@ +package tomlpatch + +import ( + "bytes" + "fmt" + "os" + + "github.com/pelletier/go-toml/v2" +) + +// ReadTOML reads and parses a TOML file into a map. +// Returns an empty map if the file does not exist. +func ReadTOML(path string) (map[string]any, error) { + data, err := os.ReadFile(path) + if err != nil { + if os.IsNotExist(err) { + return make(map[string]any), nil + } + return nil, err + } + var doc map[string]any + if err := toml.Unmarshal(data, &doc); err != nil { + return nil, err + } + return doc, nil +} + +// WriteTOML atomically encodes doc as TOML and writes it to path via +// temp-file + rename. +func WriteTOML(path string, doc map[string]any) error { + var buf bytes.Buffer + if err := toml.NewEncoder(&buf).Encode(doc); err != nil { + return fmt.Errorf("encoding TOML: %w", err) + } + return WriteFileAtomic(path, buf.Bytes(), 0o644) +} + +// UnmarshalTOML parses raw TOML bytes into a map. +func UnmarshalTOML(data []byte) (map[string]any, error) { + var doc map[string]any + if err := toml.Unmarshal(data, &doc); err != nil { + return nil, err + } + return doc, nil +} + +// MarshalTOML encodes a map as TOML bytes. +func MarshalTOML(doc any) ([]byte, error) { + var buf bytes.Buffer + if err := toml.NewEncoder(&buf).Encode(doc); err != nil { + return nil, err + } + return buf.Bytes(), nil +} From ad922126899f21cafa06e23988b36297a75f7c1c Mon Sep 17 00:00:00 2001 From: bdchatham Date: Thu, 13 Aug 2026 13:40:05 -0700 Subject: [PATCH 02/13] style(sidecar): apply the mechanical lint fixes to the relocated code MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Putting sidecar/ and sidecarapi/tomlpatch/ under the lint matrix surfaced findings in code seictl never linted — it carries no golangci-lint config. This commit takes only what golangci-lint can fix on its own and what is provably behaviour-neutral: modernize's interface{}→any, integer-range loops (`for i := range n`), and maps.Copy in tomlpatch. Applied with `--fix`, then goimports to drop the `sort` import the slices.Sort rewrite orphaned — which broke the build until it was removed, so the auto-fix is not self-sufficient. Everything else is left for a decision rather than folded in here. The point of this PR is that the moved code is verifiably the same code; rewriting error handling inside a 24,000-line relocation would remove the reviewer's ability to check that, and some of it is not cosmetic: - noctx, 27 findings, entirely in sidecar/engine/sqlite_migrations.go and sqlite_store.go: database/sql calls that should take the *Context variants. A real improvement, and a behaviour change — queries become cancellable — in the store whose SetMaxOpenConns(1) and WAL-checkpoint invariants are the ones worth being careful with. - errcheck, 10, mostly unchecked tx.Rollback in the migration path, where ignoring the error is the intended pattern and wants `_ =` rather than handling. - bodyclose, 26, all in _test.go files; no response body is leaked in code that runs in the pod. Verified: gofmt, goimports, build and tests clean in both modules (3 sidecarapi + 8 sidecar packages, 0 failures). sidecarapi now reports 0 lint issues; sidecar's remaining findings are unaddressed by design. Co-Authored-By: Claude Opus 5 (1M context) --- sidecar/engine/engine_e2e_test.go | 11 ++++------- sidecar/engine/engine_test.go | 4 ++-- sidecar/engine/mark_not_ready_test.go | 4 ++-- sidecar/engine/sqlite_store_test.go | 4 ++-- sidecar/s3/client_test.go | 2 +- sidecar/s3/emit_test.go | 2 +- sidecar/shadow/layer1.go | 2 +- sidecar/tasks/assemble_genesis_external_test.go | 8 ++------ sidecar/tasks/result_export.go | 2 +- sidecar/tasks/snapshot_restore_test.go | 5 +---- sidecar/tasks/snapshot_upload.go | 4 ++-- sidecar/tasks/statesync.go | 7 ++----- sidecarapi/tomlpatch/merge.go | 6 +++--- 13 files changed, 24 insertions(+), 37 deletions(-) diff --git a/sidecar/engine/engine_e2e_test.go b/sidecar/engine/engine_e2e_test.go index 099b1d61..55b15da3 100644 --- a/sidecar/engine/engine_e2e_test.go +++ b/sidecar/engine/engine_e2e_test.go @@ -177,8 +177,7 @@ func TestE2E_TaskLifecycle(t *testing.T) { cancel() store2 := reopenStore(t, store, dbPath) - ctx2, cancel2 := context.WithCancel(context.Background()) - defer cancel2() + ctx2 := t.Context() eng2 := NewEngine(ctx2, handlers, store2) // The successful task should still be there. @@ -243,8 +242,7 @@ func TestE2E_StaleTaskRehydration(t *testing.T) { // Reopen store and create a new engine (simulates restart). store2 := reopenStore(t, store, dbPath) - ctx, cancel := context.WithCancel(context.Background()) - defer cancel() + ctx := t.Context() handlers := map[TaskType]TaskHandler{ TaskConfigPatch: func(_ context.Context, _ map[string]any) (json.RawMessage, error) { return nil, nil }, } @@ -260,8 +258,7 @@ func TestE2E_StaleTaskRehydration(t *testing.T) { func TestE2E_ConcurrentSubmit(t *testing.T) { store, _ := newFileStore(t) - ctx, cancel := context.WithCancel(context.Background()) - defer cancel() + ctx := t.Context() var execCount atomic.Int32 handlers := map[TaskType]TaskHandler{ @@ -281,7 +278,7 @@ func TestE2E_ConcurrentSubmit(t *testing.T) { var wg sync.WaitGroup wg.Add(n) - for i := 0; i < n; i++ { + for i := range n { go func(i int) { defer wg.Done() ids[i], errs[i] = eng.Submit(Task{Type: TaskConfigPatch}) diff --git a/sidecar/engine/engine_test.go b/sidecar/engine/engine_test.go index 51b55cba..52a8b00c 100644 --- a/sidecar/engine/engine_test.go +++ b/sidecar/engine/engine_test.go @@ -472,7 +472,7 @@ func TestRecentResultsReturnsAll(t *testing.T) { }) var ids []string - for i := 0; i < 7; i++ { + for range 7 { id, _ := eng.Submit(Task{Type: TaskConfigPatch}) ids = append(ids, id) } @@ -1110,7 +1110,7 @@ func TestSubmitConcurrentSameFailedID(t *testing.T) { // Two concurrent submits of the same failed ID. var wg sync.WaitGroup wg.Add(2) - for i := 0; i < 2; i++ { + for range 2 { go func() { defer wg.Done() eng.Submit(Task{ID: taskID, Type: TaskConfigPatch}) diff --git a/sidecar/engine/mark_not_ready_test.go b/sidecar/engine/mark_not_ready_test.go index 881dec4d..e8bd5a67 100644 --- a/sidecar/engine/mark_not_ready_test.go +++ b/sidecar/engine/mark_not_ready_test.go @@ -164,7 +164,7 @@ func TestMarkNotReady_PurgesThenFlipsReadyFalse(t *testing.T) { // with no dependence on goroutine scheduling. Looped to stress the ordering // under -race, where the pre-fix concurrent dispatch would flake. func TestRehydrate_StrandedHoldWinsDeterministically(t *testing.T) { - for i := 0; i < 50; i++ { + for i := range 50 { store, dbPath := newFileStore(t) seedStrandedMarkReady(t, store) seedStrandedMarkNotReady(t, store) @@ -329,7 +329,7 @@ func TestRehydrate_FailedPurgeSupersedesStrandedMarkReady(t *testing.T) { // and no stale hold. The durable supersession rule (keyed on the persisted hold // record, any status) must keep the node held across BOTH restarts. func TestRehydrate_HoldSurvivesFailedPurgeAcrossTwoRestarts(t *testing.T) { - for i := 0; i < 30; i++ { + for i := range 30 { store, dbPath := newFileStore(t) base := time.Now().UTC() readyID := seedStrandedAt(t, store, TaskMarkReady, base) // earlier lifecycle's release diff --git a/sidecar/engine/sqlite_store_test.go b/sidecar/engine/sqlite_store_test.go index fd232298..4cfd1599 100644 --- a/sidecar/engine/sqlite_store_test.go +++ b/sidecar/engine/sqlite_store_test.go @@ -115,7 +115,7 @@ func TestStoreListOrdering(t *testing.T) { s := newTestStore(t) base := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC) - for i := 0; i < 5; i++ { + for i := range 5 { r := &TaskResult{ ID: "list-" + string(rune('a'+i)) + "0000000-0000-0000-0000-000000000000", Type: "config-patch", @@ -147,7 +147,7 @@ func TestStoreListLimit(t *testing.T) { s := newTestStore(t) base := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC) - for i := 0; i < 20; i++ { + for i := range 20 { r := &TaskResult{ ID: "limit-" + string(rune('a'+i)) + "000000-0000-0000-0000-000000000000", Type: "config-patch", diff --git a/sidecar/s3/client_test.go b/sidecar/s3/client_test.go index 1a4c2baf..a3c9f49b 100644 --- a/sidecar/s3/client_test.go +++ b/sidecar/s3/client_test.go @@ -79,7 +79,7 @@ func TestWriteAtBuffer_SparseWrite(t *testing.T) { if len(got) != 6 { t.Fatalf("len(Bytes()) = %d, want 6", len(got)) } - for i := 0; i < 5; i++ { + for i := range 5 { if got[i] != 0 { t.Errorf("Bytes()[%d] = %d, want 0", i, got[i]) } diff --git a/sidecar/s3/emit_test.go b/sidecar/s3/emit_test.go index 05239117..7ca35eb7 100644 --- a/sidecar/s3/emit_test.go +++ b/sidecar/s3/emit_test.go @@ -154,7 +154,7 @@ func TestStreamGzip_UploadErrorUnblocksWriter(t *testing.T) { // backstop if it does not). up := &captureUploader{err: errors.New("s3 down")} _, err := StreamGzipFunc(context.Background(), up, "bkt", "k", func(w io.Writer) error { - for i := 0; i < 100000; i++ { + for range 100000 { if _, werr := io.WriteString(w, "padding-line\n"); werr != nil { return werr } diff --git a/sidecar/shadow/layer1.go b/sidecar/shadow/layer1.go index 0db819ec..9707fdea 100644 --- a/sidecar/shadow/layer1.go +++ b/sidecar/shadow/layer1.go @@ -31,7 +31,7 @@ func (c *Comparator) compareLayer1(ctx context.Context, height int64) (*Layer1Re // Compare the overlapping transactions. minLen := min(len(sTxs), len(cTxs)) - for i := 0; i < minLen; i++ { + for i := range minLen { divergence := compareTxReceipts(i, sTxs[i], cTxs[i]) if divergence != nil { result.Divergences = append(result.Divergences, *divergence) diff --git a/sidecar/tasks/assemble_genesis_external_test.go b/sidecar/tasks/assemble_genesis_external_test.go index a2eb07e9..fd1b3d73 100644 --- a/sidecar/tasks/assemble_genesis_external_test.go +++ b/sidecar/tasks/assemble_genesis_external_test.go @@ -3,6 +3,7 @@ package tasks import ( "os" "path/filepath" + "slices" "strings" "testing" "time" @@ -247,12 +248,7 @@ func mustModTime(t *testing.T, path string) int64 { } func containsAddr(haystack []string, needle string) bool { - for _, h := range haystack { - if h == needle { - return true - } - } - return false + return slices.Contains(haystack, needle) } // findAuthAccount returns the genesis account at addr, or nil if absent. diff --git a/sidecar/tasks/result_export.go b/sidecar/tasks/result_export.go index 8139da2f..3ba90af7 100644 --- a/sidecar/tasks/result_export.go +++ b/sidecar/tasks/result_export.go @@ -155,7 +155,7 @@ func (e *ResultExporter) Export(ctx context.Context, cfg ResultExportRequest) er return nil } - for page := 0; page < fullPages; page++ { + for page := range fullPages { pageStart := startHeight + int64(page*defaultPageSize) pageEnd := pageStart + int64(defaultPageSize) - 1 diff --git a/sidecar/tasks/snapshot_restore_test.go b/sidecar/tasks/snapshot_restore_test.go index cbc34653..f3641286 100644 --- a/sidecar/tasks/snapshot_restore_test.go +++ b/sidecar/tasks/snapshot_restore_test.go @@ -70,10 +70,7 @@ func (m *mockObjectLister) ListObjectsV2(_ context.Context, input *s3.ListObject } } - end := startIdx + pageSize - if end > len(m.keys) { - end = len(m.keys) - } + end := min(startIdx+pageSize, len(m.keys)) var contents []types.Object for _, k := range m.keys[startIdx:end] { diff --git a/sidecar/tasks/snapshot_upload.go b/sidecar/tasks/snapshot_upload.go index b2a270b9..5fafbe03 100644 --- a/sidecar/tasks/snapshot_upload.go +++ b/sidecar/tasks/snapshot_upload.go @@ -11,7 +11,7 @@ import ( "io" "os" "path/filepath" - "sort" + "slices" "strconv" "strings" "time" @@ -325,7 +325,7 @@ func pickUploadCandidate(snapshotsDir string) (int64, error) { return 0, nil } - sort.Slice(heights, func(i, j int) bool { return heights[i] < heights[j] }) + slices.Sort(heights) return heights[len(heights)-2], nil } diff --git a/sidecar/tasks/statesync.go b/sidecar/tasks/statesync.go index c9b3c24c..af01237f 100644 --- a/sidecar/tasks/statesync.go +++ b/sidecar/tasks/statesync.go @@ -109,10 +109,7 @@ func (s *StateSyncConfigurer) Configure(ctx context.Context, p StateSyncRequest) if err != nil { return fmt.Errorf("configure-state-sync: querying latest height: %w", err) } - trustHeight = latestHeight - trustHeightOffset - if trustHeight < 1 { - trustHeight = 1 - } + trustHeight = max(latestHeight-trustHeightOffset, 1) } ssLog.Info("querying trust hash", "trust-height", trustHeight, "endpoint", reachable[0]) @@ -352,7 +349,7 @@ func readPeersFromConfig(homeDir string) ([]string, error) { } var peers []string - for _, p := range strings.Split(raw, ",") { + for p := range strings.SplitSeq(raw, ",") { p = strings.TrimSpace(p) if p != "" { peers = append(peers, p) diff --git a/sidecarapi/tomlpatch/merge.go b/sidecarapi/tomlpatch/merge.go index 3e2d463e..2ec2187f 100644 --- a/sidecarapi/tomlpatch/merge.go +++ b/sidecarapi/tomlpatch/merge.go @@ -7,6 +7,8 @@ // across a module boundary that this migration exists to remove. package tomlpatch +import "maps" + // Merge performs a recursive merge-patch of patch into original. // nil values in the patch delete the corresponding key from original. // Non-map patches replace the original entirely. @@ -20,9 +22,7 @@ func Merge(original, patch any) any { originalMap = make(map[string]any) } result := make(map[string]any) - for k, v := range originalMap { - result[k] = v - } + maps.Copy(result, originalMap) for key, patchAt := range patchMap { if patchAt == nil { delete(result, key) From a4293e23d3f1e43d205dbbecb72a12214fe1f1b6 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Thu, 13 Aug 2026 13:56:37 -0700 Subject: [PATCH 03/13] fix(sidecar): put `serve` in the ENTRYPOINT, not the pod spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The image's ENTRYPOINT was the bare binary, leaving the subcommand to the controller's `Command`. That is a trap for the step where `Command` is removed and the ENTRYPOINT takes over: measured under pod conditions, with SEI_HOME set and no subcommand, sei-sidecar prints help and exits 0. Kubernetes restarts it under restartPolicy Always, so the sidecar would loop forever while reporting success — no crash, no failed probe, and the seid container beside it blocking on /v0/healthz behind a 86400-threshold StartupProbe. Quieter than a crash and harder to find. Naming the subcommand in the ENTRYPOINT makes that step a pure deletion of `Command` rather than a swap that has to land correctly. A pod spec's `command` overrides ENTRYPOINT outright, so today's `["seictl","serve"]` path is unchanged. Also records why the rename cannot be a lockstep cutover, since that is the question the shim answers: the sidecar image is chosen per node by spec.sidecar.image when set, which overrides the platform config, and every SeiNode sample in manifests/samples pins it. A controller rendering the new name would break every node holding a pinned older image, and a config rollout does not reach those — each CR would have to change in the same instant as the controller deploy. The shim replaces that with three steps that are each safe alone: ship the image, then flip or drop Command, then delete the shim. Verified: both invocation paths exercised against the built binary — `serve` reaches real env validation, bare no longer exits 0. Build and all 8 sidecar test packages unchanged. Co-Authored-By: Claude Opus 5 (1M context) --- sidecar/Dockerfile | 37 ++++++++++++++++++++++++++----------- 1 file changed, 26 insertions(+), 11 deletions(-) diff --git a/sidecar/Dockerfile b/sidecar/Dockerfile index 1d8952db..8f47e451 100644 --- a/sidecar/Dockerfile +++ b/sidecar/Dockerfile @@ -33,17 +33,32 @@ COPY --from=build /go/bin/sei-sidecar /usr/bin/sei-sidecar # Compatibility shim, deliberately temporary. The controller renders # `Command: []string{"seictl", "serve"}` into every SeiNode pod spec -# (internal/noderesource/noderesource.go, internal/task/bootstrap_resources.go). -# Controller rollouts and sidecar-image rollouts are independent, so a binary -# rename with no shim has no safe ordering: an old controller against a new -# image fails to find `seictl`, and a new controller against an old image fails -# to find the new name. Either way the pod does not crash loudly — the seid -# container blocks on a shell loop polling /v0/healthz behind a StartupProbe -# with FailureThreshold 86400 at 5s, so it hangs for days instead of alerting. +# (internal/noderesource/noderesource.go, internal/task/bootstrap_resources.go), +# and a pod spec's `command` overrides ENTRYPOINT outright — so without this +# name the container cannot start at all. # -# With the shim, adopting this image is a pure `images.sidecar` reference -# change. Remove it only after the controller stops rendering `Command` (letting -# this ENTRYPOINT decide) and every cell has rolled past that controller. +# The rename cannot be done in lockstep, which is why the shim exists rather +# than a coordinated cutover. The sidecar image is chosen per node by +# `spec.sidecar.image` when set (noderesource.go, sidecarImage), and that +# overrides the platform config's images.sidecar — every SeiNode sample in +# manifests/samples pins it. So a controller that rendered the new name would +# break every node carrying a pinned older image, and no config rollout reaches +# those; each CR would have to change in the same instant as the controller +# deploy. The failure is also quiet: the seid container blocks on a shell loop +# polling /v0/healthz behind a StartupProbe with FailureThreshold 86400 at 5s, +# so a broken sidecar hangs for days rather than alerting. +# +# With both names present, adoption is a per-node image reference change at any +# pace. Retire the shim in three steps, each safe alone: +# +# 1. this image reaches every cell; +# 2. the controller stops rendering `Command` (or renders sei-sidecar) — by +# then every image answers to either name; +# 3. delete this COPY. COPY --from=build /go/bin/sei-sidecar /usr/bin/seictl -ENTRYPOINT ["/usr/bin/sei-sidecar"] +# `serve` belongs in the ENTRYPOINT, not left to the pod spec. Step 2 above +# removes `Command`, and a bare `sei-sidecar` with no subcommand prints help and +# exits 0 — a container that restarts forever while reporting success, which is +# worse than a crash. Naming the subcommand here makes step 2 a pure deletion. +ENTRYPOINT ["/usr/bin/sei-sidecar", "serve"] From 531d7dc8ad04076109b4b9eb07c4bb03e5bf45ab Mon Sep 17 00:00:00 2001 From: bdchatham Date: Thu, 13 Aug 2026 13:59:46 -0700 Subject: [PATCH 04/13] feat(sidecar)!: let the image own its entrypoint; drop the seictl name MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The sidecar container's pod spec no longer carries a Command. The image's ENTRYPOINT names the binary and the subcommand, so the image owns its own entrypoint and renaming the binary later is an image-only change. The compatibility symlink is gone with it. This is a coordinated deploy, not a rolling one. Ship together, per cell: 1. the new sidecar image published as sei/sei-sidecar, and 2. images.sidecar in the platform app-config pointing at it, and 3. this controller. That is safe in one step because the controller renders Image and the (now absent) Command into the same container spec in the same reconcile: a single StatefulSet update carries both, and the rollout follows. It is not safe split apart. This controller against an older image runs that image's bare ENTRYPOINT with no subcommand; an older controller against the new image looks for a `seictl` binary the image no longer has. Both fail quietly rather than loudly — seid blocks on /v0/healthz behind a StartupProbe with FailureThreshold 86400 at 5s, so a mismatch hangs rather than alerts. Rollback is the same change in reverse, and equally coordinated. Note spec.sidecar.image still exists and overrides images.sidecar per node. Nothing sets it today, which is what makes the single-step cutover viable; a node pinned to a pre-rename image after this lands would fail the same quiet way, so pin only images that carry the new entrypoint. No test asserted Command, and the container was already named sei-sidecar. Verified: build clean, 14 root test packages pass, verify-generated clean. Co-Authored-By: Claude Opus 5 (1M context) --- internal/noderesource/noderesource.go | 2 +- internal/task/bootstrap_resources.go | 1 - sidecar/Dockerfile | 37 ++++++--------------------- 3 files changed, 9 insertions(+), 31 deletions(-) diff --git a/internal/noderesource/noderesource.go b/internal/noderesource/noderesource.go index 6d4fcecd..f2944e7b 100644 --- a/internal/noderesource/noderesource.go +++ b/internal/noderesource/noderesource.go @@ -855,10 +855,10 @@ func buildSidecarContainer(node *seiv1alpha1.SeiNode, p PlatformConfig) corev1.C ) mounts = append(mounts, keyringMounts...) + // No Command — the sidecar image's ENTRYPOINT is the command. c := corev1.Container{ Name: containerNameSidecar, Image: sidecarImage(node, p), - Command: []string{"seictl", "serve"}, RestartPolicy: ptr.To(corev1.ContainerRestartPolicyAlways), Env: env, Ports: []corev1.ContainerPort{ diff --git a/internal/task/bootstrap_resources.go b/internal/task/bootstrap_resources.go index ccd8d424..3ab04874 100644 --- a/internal/task/bootstrap_resources.go +++ b/internal/task/bootstrap_resources.go @@ -135,7 +135,6 @@ func buildBootstrapPodSpec(node *seiv1alpha1.SeiNode, snap *seiv1alpha1.Snapshot sidecar := corev1.Container{ Name: "sei-sidecar", Image: bootstrapSidecarImage(node, platformCfg), - Command: []string{"seictl", "serve"}, RestartPolicy: ptr.To(corev1.ContainerRestartPolicyAlways), Env: []corev1.EnvVar{ {Name: "SEI_CHAIN_ID", Value: node.Spec.ChainID}, diff --git a/sidecar/Dockerfile b/sidecar/Dockerfile index 8f47e451..e700d74b 100644 --- a/sidecar/Dockerfile +++ b/sidecar/Dockerfile @@ -31,34 +31,13 @@ RUN cd sidecar && CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} \ FROM gcr.io/distroless/static-debian12 COPY --from=build /go/bin/sei-sidecar /usr/bin/sei-sidecar -# Compatibility shim, deliberately temporary. The controller renders -# `Command: []string{"seictl", "serve"}` into every SeiNode pod spec -# (internal/noderesource/noderesource.go, internal/task/bootstrap_resources.go), -# and a pod spec's `command` overrides ENTRYPOINT outright — so without this -# name the container cannot start at all. +# The subcommand belongs here, not in the pod spec. The controller renders no +# `Command` for the sidecar container, so this ENTRYPOINT is what runs, and the +# image owns its own binary name — renaming it later is an image-only change. # -# The rename cannot be done in lockstep, which is why the shim exists rather -# than a coordinated cutover. The sidecar image is chosen per node by -# `spec.sidecar.image` when set (noderesource.go, sidecarImage), and that -# overrides the platform config's images.sidecar — every SeiNode sample in -# manifests/samples pins it. So a controller that rendered the new name would -# break every node carrying a pinned older image, and no config rollout reaches -# those; each CR would have to change in the same instant as the controller -# deploy. The failure is also quiet: the seid container blocks on a shell loop -# polling /v0/healthz behind a StartupProbe with FailureThreshold 86400 at 5s, -# so a broken sidecar hangs for days rather than alerting. -# -# With both names present, adoption is a per-node image reference change at any -# pace. Retire the shim in three steps, each safe alone: -# -# 1. this image reaches every cell; -# 2. the controller stops rendering `Command` (or renders sei-sidecar) — by -# then every image answers to either name; -# 3. delete this COPY. -COPY --from=build /go/bin/sei-sidecar /usr/bin/seictl - -# `serve` belongs in the ENTRYPOINT, not left to the pod spec. Step 2 above -# removes `Command`, and a bare `sei-sidecar` with no subcommand prints help and -# exits 0 — a container that restarts forever while reporting success, which is -# worse than a crash. Naming the subcommand here makes step 2 a pure deletion. +# `serve` is named explicitly rather than left as the root command's default: a +# bare `sei-sidecar` with SEI_HOME set prints help and exits 0, which under +# restartPolicy Always is a container that loops forever while reporting +# success. Harder to notice than a crash, and the seid container beside it would +# sit blocked on /v0/healthz behind a StartupProbe with FailureThreshold 86400. ENTRYPOINT ["/usr/bin/sei-sidecar", "serve"] From 94d400fe1412b9e9c3482502fe7452a8657ba238 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Thu, 13 Aug 2026 14:45:24 -0700 Subject: [PATCH 05/13] fix(sidecar): unbreak the image build, restore dependency floors, close SEI_HOME="" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three defects a two-lens cross review found, plus the documentation the earlier commits contradicted. The image could not be built. .dockerignore starts with `**` and re-includes only Go sources, module files and shell scripts, so sidecar/tasks/defaults/config.toml — a //go:embed target — never entered the build context: `pattern config.toml: no matching files found`. seictl had no .dockerignore, so its `COPY . .` carried the file and the move regressed it. No CI job builds either Dockerfile, so this would have reached main unseen. Reproduced by rebuilding a context under the repo's own filter rules, and confirmed fixed the same way. The relocated binary was not built from the versions the binary it replaces was built from. A fresh `go mod tidy` resolves the minimum each import allows, and seictl's explicit floors did not come along: 10 of its 24 direct requires drifted. modernc.org/sqlite went *down*, v1.21.2 → v1.18.1, which is embedded SQLite 3.41.2 → 3.39.2 — a driver that owns an on-disk format on a live PVC, where the sign-tx idempotency marker lives. aws-sdk-go-v2's s3 transfermanager went up two minor versions while still pre-1.0, on the multi-GB snapshot path where part size and concurrency set the memory ceiling. Floors restored to seictl's, so this is a relocation rather than a silent dependency bump. sidecarapi and sidecar also disagreed on go-toml/v2 (v2.2.2 vs v2.2.4), so tomlpatch's tests exercised a different TOML parser than the sidecar links; both now v2.2.4. SEI_HOME="" defeated the required-flag check. The flag's Required tests whether a value was supplied, and an empty string is a value. Measured: the sidecar started, logged `sidecar HTTP bind=:17991`, and created config/, data/ and sidecar.db relative to its working directory — exactly the silent wrong-directory failure Required was added to prevent. validateHome now rejects empty and whitespace-only, covered for both. Documentation that the entrypoint change falsified: CLAUDE.md described a `seictl` symlink and forbade removing it until every cell had rolled, two commits before removing it; main.go's package doc still described the symlink; README.md's controller-first rollout rule reads as covering the entrypoint change, which it does not. CLAUDE.md now also records what the earlier text got wrong about the rollout — StatefulSets are OnDelete, so a template change never touches a live pod, and what rolls the cell is the controller's own NodeUpdate `replace-pod` for every node with status.currentSidecarImage set. Verified: builds, gofmt, tidy-check and verify-generated clean across all three modules; 14 root + 3 sidecarapi + 8 sidecar test packages pass; the sidecar builds from a simulated build context that previously failed; SEI_HOME empty and whitespace-only both refuse with no files created. Co-Authored-By: Claude Opus 5 (1M context) --- .dockerignore | 7 ++++++ CLAUDE.md | 8 ++++++- README.md | 2 ++ sidecar/go.mod | 21 +++++++++-------- sidecar/go.sum | 44 ++++++++++++++++++++++------------- sidecar/main.go | 30 +++++++++++++++++++++--- sidecar/startup_guard_test.go | 35 ++++++++++++++++++++++++++++ sidecarapi/go.mod | 4 ++-- sidecarapi/go.sum | 19 ++++----------- 9 files changed, 124 insertions(+), 46 deletions(-) diff --git a/.dockerignore b/.dockerignore index c02e37e1..b9fb09fc 100644 --- a/.dockerignore +++ b/.dockerignore @@ -18,6 +18,13 @@ # Re-include embedded shell scripts !**/*.sh +# Re-include //go:embed targets that are not Go source. The build fails at +# compile time when one is missing from the context — `pattern config.toml: no +# matching files found` — so an addition here is required whenever a new asset +# is embedded. Currently: sidecar/tasks/defaults/config.toml, the seid config +# the sidecar writes on first start. +!**/tasks/defaults/*.toml + # Re-include the runner image's per-kind Go text templates. The runner # Dockerfile (runner/Dockerfile) bakes runner/templates/ into the image # at /templates/. Without this re-include the seitask-runner build fails diff --git a/CLAUDE.md b/CLAUDE.md index 7fd8718c..15e1c682 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,7 +26,13 @@ Two checks keep the controller tidyable, both in `make ci`. The `depguard` rule ### The sidecar binary -`sidecar/main.go` → `sei-sidecar`, published to ECR as `sei/sei-sidecar`. The image also installs a `seictl` symlink because the controller renders `Command: []string{"seictl", "serve"}` into every pod spec (`internal/noderesource/`, `internal/task/bootstrap_resources.go`). Controller and sidecar images roll independently, so a rename without the shim has no safe ordering — and the failure is silent, not loud: seid blocks on a shell loop polling `/v0/healthz` behind a StartupProbe with `FailureThreshold: 86400` at 5s. Remove the shim only after the controller stops rendering `Command` and every cell has rolled past that controller. +`sidecar/main.go` → `sei-sidecar`, published to ECR as `sei/sei-sidecar`. The controller renders **no `Command`** for the sidecar container (`internal/noderesource/`, `internal/task/bootstrap_resources.go`), so the image's ENTRYPOINT — `sei-sidecar serve` — is what runs. The image owns its entrypoint; renaming the binary is an image-only change. + +**Changing the sidecar image is a coordinated deploy.** The image, `images.sidecar` in the platform app-config, and the controller ship together per cell. `images.sidecar` is read once at startup, so a config edit alone does nothing until the controller restarts — and the two halves failing apart is silent, not loud. A controller that renders no `Command` against an image whose entrypoint lacks the subcommand gets a container that prints help and exits 0, restarting forever under `restartPolicy: Always`, while seid blocks on a shell loop polling `/v0/healthz` behind a StartupProbe with `FailureThreshold: 86400` at 5s — about five days before Kubernetes calls it failed. + +The rollout is controller-driven, not Kubernetes-driven: StatefulSets use `UpdateStrategy: OnDelete`, so a template change never touches a live pod. Sidecar-image drift builds a NodeUpdate plan whose `replace-pod` task deletes pods at the old revision, for **every** node whose `status.currentSidecarImage` is set. Expect the whole cell to roll. + +`spec.sidecar.image` still overrides `images.sidecar` per node. Pin only images whose entrypoint matches what the controller renders. Two startup refusals in `sidecar/` are load-bearing; do not soften them into defaults: diff --git a/README.md b/README.md index 8681d12b..ab88841c 100644 --- a/README.md +++ b/README.md @@ -101,6 +101,8 @@ The controller image is built and pushed to ECR by GitHub Actions on every push **Deployment ordering**: When adding new environment variables to the sidecar, the controller must be deployed first — it injects env vars into pod specs. If the sidecar image is updated before the controller, existing pods will crash because they lack the new env vars. The safe sequence is: (1) deploy the controller, (2) then update the sidecar image in SeiNode specs. +That ordering holds only for *additive* env vars. It does **not** hold for a change to the sidecar's entrypoint or binary name: the controller renders no `Command`, so the image's ENTRYPOINT is the command, and the two must move together. Split apart, the container prints help and exits 0 in a restart loop while seid waits on `/v0/healthz` behind a `FailureThreshold: 86400` probe — silent for roughly five days. See CLAUDE.md, "The sidecar binary". + The `config/` directory follows the standard [Kubebuilder](https://book.kubebuilder.io) layout: ``` diff --git a/sidecar/go.mod b/sidecar/go.mod index c393e4f2..f71081dd 100644 --- a/sidecar/go.mod +++ b/sidecar/go.mod @@ -4,9 +4,9 @@ go 1.26.0 require ( github.com/aws/aws-sdk-go-v2 v1.43.5 - github.com/aws/aws-sdk-go-v2/config v1.32.36 - github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.12 - github.com/aws/aws-sdk-go-v2/service/s3 v1.107.1 + github.com/aws/aws-sdk-go-v2/config v1.32.12 + github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.1.10 + github.com/aws/aws-sdk-go-v2/service/s3 v1.97.2 github.com/aws/smithy-go v1.27.7 github.com/ethereum/go-ethereum v1.16.8 github.com/google/uuid v1.6.0 @@ -16,7 +16,7 @@ require ( github.com/sei-protocol/sei-k8s-controller/sidecarapi v0.0.0 github.com/sei-protocol/seilog v0.0.3 github.com/urfave/cli/v3 v3.6.1 - modernc.org/sqlite v1.18.1 + modernc.org/sqlite v1.21.2 ) require ( @@ -34,6 +34,7 @@ require ( github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36 // indirect github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36 // indirect github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36 // indirect + github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 // indirect github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37 // indirect github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.16 // indirect github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.29 // indirect @@ -139,7 +140,7 @@ require ( github.com/prometheus/procfs v0.17.0 // indirect github.com/rakyll/statik v0.1.7 // indirect github.com/regen-network/cosmos-proto v0.3.1 // indirect - github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect github.com/rogpeppe/go-internal v1.14.1 // indirect github.com/rs/cors v1.11.1 // indirect github.com/sagikazarmark/locafero v0.11.0 // indirect @@ -203,14 +204,14 @@ require ( gopkg.in/yaml.v3 v3.0.1 // indirect k8s.io/component-base v0.35.0 // indirect lukechampine.com/uint128 v1.2.0 // indirect - modernc.org/cc/v3 v3.36.3 // indirect - modernc.org/ccgo/v3 v3.16.9 // indirect - modernc.org/libc v1.17.1 // indirect + modernc.org/cc/v3 v3.40.0 // indirect + modernc.org/ccgo/v3 v3.16.13 // indirect + modernc.org/libc v1.22.4 // indirect modernc.org/mathutil v1.5.0 // indirect - modernc.org/memory v1.2.1 // indirect + modernc.org/memory v1.5.0 // indirect modernc.org/opt v0.1.3 // indirect modernc.org/strutil v1.1.3 // indirect - modernc.org/token v1.0.0 // indirect + modernc.org/token v1.0.1 // indirect nhooyr.io/websocket v1.8.6 // indirect ) diff --git a/sidecar/go.sum b/sidecar/go.sum index 5959cfe4..280bd8b5 100644 --- a/sidecar/go.sum +++ b/sidecar/go.sum @@ -702,16 +702,16 @@ github.com/aws/aws-sdk-go-v2 v1.43.5/go.mod h1:wZjAJppCntyOGgVSmgVTfDyRJK5PHOasO github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.17 h1:mn+Vxb9zgz/FE/yDTcFim3DZ1qpcrxR+qBQkBrl6bzA= github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.17/go.mod h1:eDfmEFxu+BSVsUGLbzJhWjpOurv1mqczClS97yI8wdk= github.com/aws/aws-sdk-go-v2/config v1.18.45/go.mod h1:ZwDUgFnQgsazQTnWfeLWk5GjeqTQTL8lMkoE1UXzxdE= -github.com/aws/aws-sdk-go-v2/config v1.32.36 h1:mX6ietU7UlB4w/2IUaexJdsyUDvhTd+jYPjVePiyi6s= -github.com/aws/aws-sdk-go-v2/config v1.32.36/go.mod h1:rMpV4xk7ZK59edraSaHP0jsWrztWTT5tbCwWY495hug= +github.com/aws/aws-sdk-go-v2/config v1.32.12 h1:O3csC7HUGn2895eNrLytOJQdoL2xyJy0iYXhoZ1OmP0= +github.com/aws/aws-sdk-go-v2/config v1.32.12/go.mod h1:96zTvoOFR4FURjI+/5wY1vc1ABceROO4lWgWJuxgy0g= github.com/aws/aws-sdk-go-v2/credentials v1.13.43/go.mod h1:zWJBz1Yf1ZtX5NGax9ZdNjhhI4rgjfgsyk6vTY1yfVg= github.com/aws/aws-sdk-go-v2/credentials v1.19.35 h1:Cxua2RVdRwL0sfjHM/SnQoOnQ7xKng9m5EQBO8BnZlg= github.com/aws/aws-sdk-go-v2/credentials v1.19.35/go.mod h1:9XQ+RSIGPkycr+oCJYnB1uTv5kMVVR+rd2vYK0Hxj2w= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.13/go.mod h1:f/Ib/qYjhV2/qdsf79H3QP/eRE4AkVyEf6sk7XfZ1tg= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36 h1:gucL1KH/PAYbpTpBg09CiVpBdTu4qkCl8C7xOTBixUg= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36/go.mod h1:usTB+PHhNMhrx2dxUeHcM7OrT5pySvmjYI++IsefPN0= -github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.12 h1:yIZV4/Eg+W8AN0MaI+PshJQ0sfsv6Hgsos6WmaojzFk= -github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.12/go.mod h1:lBJJRVakFaZwpIFvzSBKcLkLCki8jg086seIleOq8Ic= +github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.1.10 h1:2KCL4TmeiNvpPedtC4Bey5jvjRLD74WUYqGeHJ//aco= +github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.1.10/go.mod h1:KwaiUFVO7pG8Z9F5bMGvvrRibdSDaAu8HtlKGKkjZSA= github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.43/go.mod h1:auo+PiyLl0n1l8A0e8RIeR8tOzYPfZZH/JNlrJ8igTQ= github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36 h1:5CrzwxDqf4w3x1Vs3/NiZ0nsC34Hbm3pIDMWbsLebOE= github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36/go.mod h1:A3gHdKZIvG/QXERzZwcxNS3RNDFcRCuhhTFBYp+V/nw= @@ -719,6 +719,8 @@ github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.37/go.mod h1:Qe+2KtKml+F github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36 h1:A4N2f4YPcST0v+dWtX+xrpPPCL9VTBhoIFFUWYqbacE= github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36/go.mod h1:B/Qr859uxWUEfZeGotK5KAEoof4Q9YWgNtPSwV6jcyk= github.com/aws/aws-sdk-go-v2/internal/ini v1.3.45/go.mod h1:lD5M20o09/LCuQ2mE62Mb/iSdSlCNuj6H5ci7tW7OsE= +github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 h1:qYQ4pzQ2Oz6WpQ8T3HvGHnZydA72MnLuFK9tJwmrbHw= +github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6/go.mod h1:O3h0IK87yXci+kg6flUKzJnWeziQUKciKrLjcatSNcY= github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37 h1:oyd3ke4V9AhKcRR7rRgxk1VyI+DjK2CBQtbxh3OkdaA= github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37/go.mod h1:aA9D7SqfG9IC1b7FLD7Iyc8Q4JN0a8gHhNjN4zPlIaI= github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.16 h1:iE4NGbvqUZnHDqddQAauZzCILYtFjOHwRM5MOOKLB5A= @@ -731,8 +733,8 @@ github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.36/go.mod h1:Q github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.37 h1:KGHa9iZCrgtkOsFfXb0S4ywsjostA/hau7WE9aSb43E= github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.37/go.mod h1:FV79f0DSnZIEGsQjWenENGtUycrasyAaJZO+zRanLHA= github.com/aws/aws-sdk-go-v2/service/route53 v1.30.2/go.mod h1:TQZBt/WaQy+zTHoW++rnl8JBrmZ0VO6EUbVua1+foCA= -github.com/aws/aws-sdk-go-v2/service/s3 v1.107.1 h1:VUTtUJMuRNMkb/7NIKmd8NQaeQLPGCMoTJxkYKre4qM= -github.com/aws/aws-sdk-go-v2/service/s3 v1.107.1/go.mod h1:WvUaO0lP5GNMs1R6cs6qvB3mqo16GLta8yfOuf55Rpc= +github.com/aws/aws-sdk-go-v2/service/s3 v1.97.2 h1:MRNiP6nqa20aEl8fQ6PJpEq11b2d40b16sm4WD7QgMU= +github.com/aws/aws-sdk-go-v2/service/s3 v1.97.2/go.mod h1:FrNA56srbsr3WShiaelyWYEo70x80mXnVZ17ZZfbeqg= github.com/aws/aws-sdk-go-v2/service/signin v1.5.5 h1:0VTFBfOgPJrUSpGMgzoi8qLcXF5dbmiBuxpo14eBWUw= github.com/aws/aws-sdk-go-v2/service/signin v1.5.5/go.mod h1:sNZYlBxoohYMBYl47BO/bFtAM6I8HSsPa1qwwPPRGoQ= github.com/aws/aws-sdk-go-v2/service/sso v1.15.2/go.mod h1:gsL4keucRCgW+xA85ALBpRFfdSLH4kHOVSnLMSuBECo= @@ -1514,8 +1516,9 @@ github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc= github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= github.com/mattn/go-sqlite3 v1.14.5/go.mod h1:WVKg1VTActs4Qso6iwGbiFih2UIHo0ENGwNd0Lj+XmI= -github.com/mattn/go-sqlite3 v1.14.14 h1:qZgc/Rwetq+MtyE18WhzjokPD93dNqLGNT3QJuLvBGw= github.com/mattn/go-sqlite3 v1.14.14/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU= +github.com/mattn/go-sqlite3 v1.14.16 h1:yOQRA0RpS5PFz/oikGwBEqvAWhWg5ufRz4ETLjwpU1Y= +github.com/mattn/go-sqlite3 v1.14.16/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= github.com/mediocregopher/radix/v3 v3.8.1/go.mod h1:8FL3F6UQRXHXIBSPUs5h0RybMF8i4n7wVopoX3x7Bv8= @@ -1762,8 +1765,9 @@ github.com/regen-network/cosmos-proto v0.3.1 h1:rV7iM4SSFAagvy8RiyhiACbWEGotmqzy github.com/regen-network/cosmos-proto v0.3.1/go.mod h1:jO0sVX6a1B36nmE8C9xBFXpNwWejXC7QqCOnH3O0+YM= github.com/regen-network/protobuf v1.3.3-alpha.regen.1 h1:OHEc+q5iIAXpqiqFKeLpu5NwTIkVXUs48vFMwzqpqY4= github.com/regen-network/protobuf v1.3.3-alpha.regen.1/go.mod h1:2DjTFR1HhMQhiWC5sZ4OhQ3+NtdbZ6oBDKQwq5Ou+FI= -github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 h1:OdAsTTz6OkFY5QxjkYwrChwuRruF69c169dPK26NUlk= github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ= github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= @@ -2890,15 +2894,17 @@ lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI= lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk= modernc.org/cc/v3 v3.36.0/go.mod h1:NFUHyPn4ekoC/JHeZFfZurN6ixxawE1BnVonP/oahEI= modernc.org/cc/v3 v3.36.2/go.mod h1:NFUHyPn4ekoC/JHeZFfZurN6ixxawE1BnVonP/oahEI= -modernc.org/cc/v3 v3.36.3 h1:uISP3F66UlixxWEcKuIWERa4TwrZENHSL8tWxZz8bHg= modernc.org/cc/v3 v3.36.3/go.mod h1:NFUHyPn4ekoC/JHeZFfZurN6ixxawE1BnVonP/oahEI= +modernc.org/cc/v3 v3.40.0 h1:P3g79IUS/93SYhtoeaHW+kRCIrYaxJ27MFPv+7kaTOw= +modernc.org/cc/v3 v3.40.0/go.mod h1:/bTg4dnWkSXowUO6ssQKnOV0yMVxDYNIsIrzqTFDGH0= modernc.org/ccgo/v3 v3.0.0-20220428102840-41399a37e894/go.mod h1:eI31LL8EwEBKPpNpA4bU1/i+sKOwOrQy8D87zWUcRZc= modernc.org/ccgo/v3 v3.0.0-20220430103911-bc99d88307be/go.mod h1:bwdAnOoaIt8Ax9YdWGjxWsdkPcZyRPHqrOvJxaKAKGw= modernc.org/ccgo/v3 v3.16.4/go.mod h1:tGtX0gE9Jn7hdZFeU88slbTh1UtCYKusWOoCJuvkWsQ= modernc.org/ccgo/v3 v3.16.6/go.mod h1:tGtX0gE9Jn7hdZFeU88slbTh1UtCYKusWOoCJuvkWsQ= modernc.org/ccgo/v3 v3.16.8/go.mod h1:zNjwkizS+fIFDrDjIAgBSCLkWbJuHF+ar3QRn+Z9aws= -modernc.org/ccgo/v3 v3.16.9 h1:AXquSwg7GuMk11pIdw7fmO1Y/ybgazVkMhsZWCV0mHM= modernc.org/ccgo/v3 v3.16.9/go.mod h1:zNMzC9A9xeNUepy6KuZBbugn3c0Mc9TeiJO4lgvkJDo= +modernc.org/ccgo/v3 v3.16.13 h1:Mkgdzl46i5F/CNR/Kj80Ri59hC8TKAhZrYSaqvkwzUw= +modernc.org/ccgo/v3 v3.16.13/go.mod h1:2Quk+5YgpImhPjv2Qsob1DnZ/4som1lJTodubIcoUkY= modernc.org/ccorpus v1.11.6 h1:J16RXiiqiCgua6+ZvQot4yUuUy8zxgqbqEEUuGPlISk= modernc.org/ccorpus v1.11.6/go.mod h1:2gEUTrWqdpH2pXsmTM1ZkjeSrUWDpjMu2T6m29L/ErQ= modernc.org/httpfs v1.0.6 h1:AAgIpFZRXuYnkjftxTAZwMIiwEqAfk8aVB2/oA6nAeM= @@ -2909,30 +2915,36 @@ modernc.org/libc v1.16.1/go.mod h1:JjJE0eu4yeK7tab2n4S1w8tlWd9MxXLRzheaRnAKymU= modernc.org/libc v1.16.17/go.mod h1:hYIV5VZczAmGZAnG15Vdngn5HSF5cSkbvfz2B7GRuVU= modernc.org/libc v1.16.19/go.mod h1:p7Mg4+koNjc8jkqwcoFBJx7tXkpj00G77X7A72jXPXA= modernc.org/libc v1.17.0/go.mod h1:XsgLldpP4aWlPlsjqKRdHPqCxCjISdHfM/yeWC5GyW0= -modernc.org/libc v1.17.1 h1:Q8/Cpi36V/QBfuQaFVeisEBs3WqoGAJprZzmf7TfEYI= modernc.org/libc v1.17.1/go.mod h1:FZ23b+8LjxZs7XtFMbSzL/EhPxNbfZbErxEHc7cbD9s= +modernc.org/libc v1.22.4 h1:wymSbZb0AlrjdAVX3cjreCHTPCpPARbQXNz6BHPzdwQ= +modernc.org/libc v1.22.4/go.mod h1:jj+Z7dTNX8fBScMVNRAYZ/jF91K8fdT2hYMThc3YjBY= modernc.org/mathutil v1.2.2/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E= modernc.org/mathutil v1.4.1/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E= modernc.org/mathutil v1.5.0 h1:rV0Ko/6SfM+8G+yKiyI830l3Wuz1zRutdslNoQ0kfiQ= modernc.org/mathutil v1.5.0/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E= modernc.org/memory v1.1.1/go.mod h1:/0wo5ibyrQiaoUoH7f9D8dnglAmILJ5/cxZlRECf+Nw= modernc.org/memory v1.2.0/go.mod h1:/0wo5ibyrQiaoUoH7f9D8dnglAmILJ5/cxZlRECf+Nw= -modernc.org/memory v1.2.1 h1:dkRh86wgmq/bJu2cAS2oqBCz/KsMZU7TUM4CibQ7eBs= modernc.org/memory v1.2.1/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU= +modernc.org/memory v1.5.0 h1:N+/8c5rE6EqugZwHii4IFsaJ7MUhoWX07J5tC/iI5Ds= +modernc.org/memory v1.5.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU= modernc.org/opt v0.1.1/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0= modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4= modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0= -modernc.org/sqlite v1.18.1 h1:ko32eKt3jf7eqIkCgPAeHMBXw3riNSLhl2f3loEF7o8= modernc.org/sqlite v1.18.1/go.mod h1:6ho+Gow7oX5V+OiOQ6Tr4xeqbx13UZ6t+Fw9IRUG4d4= +modernc.org/sqlite v1.21.2 h1:ixuUG0QS413Vfzyx6FWx6PYTmHaOegTY+hjzhn7L+a0= +modernc.org/sqlite v1.21.2/go.mod h1:cxbLkB5WS32DnQqeH4h4o1B0eMr8W/y8/RGuxQ3JsC0= modernc.org/strutil v1.1.1/go.mod h1:DE+MQQ/hjKBZS2zNInV5hhcipt5rLPWkmpbGeW5mmdw= modernc.org/strutil v1.1.3 h1:fNMm+oJklMGYfU9Ylcywl0CO5O6nTfaowNsh2wpPjzY= modernc.org/strutil v1.1.3/go.mod h1:MEHNA7PdEnEwLvspRMtWTNnp2nnyvMfkimT1NKNAGbw= -modernc.org/tcl v1.13.1 h1:npxzTwFTZYM8ghWicVIX1cRWzj7Nd8i6AqqX2p+IYao= modernc.org/tcl v1.13.1/go.mod h1:XOLfOwzhkljL4itZkK6T72ckMgvj0BDsnKNdZVUOecw= -modernc.org/token v1.0.0 h1:a0jaWiNMDhDUtqOj09wvjWWAqd3q7WpBulmL9H2egsk= +modernc.org/tcl v1.15.1 h1:mOQwiEK4p7HruMZcwKTZPw/aqtGM4aY00uzWhlKKYws= +modernc.org/tcl v1.15.1/go.mod h1:aEjeGJX2gz1oWKOLDVZ2tnEWLUrIn8H+GFu+akoDhqs= modernc.org/token v1.0.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= -modernc.org/z v1.5.1 h1:RTNHdsrOpeoSeOF4FbzTo8gBYByaJ5xT7NgZ9ZqRiJM= +modernc.org/token v1.0.1 h1:A3qvTqOwexpfZZeyI0FeGPDlSWX5pjZu9hF4lU+EKWg= +modernc.org/token v1.0.1/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= modernc.org/z v1.5.1/go.mod h1:eWFB510QWW5Th9YGZT81s+LwvaAs3Q2yr4sP0rmLkv8= +modernc.org/z v1.7.0 h1:xkDw/KepgEjeizO2sNco+hqYkU12taxQFqPEmgm1GWE= +modernc.org/z v1.7.0/go.mod h1:hVdgNMh8ggTuRG1rGU8x+xGRFfiQUIAw0ZqlPy8+HyQ= moul.io/http2curl/v2 v2.3.0/go.mod h1:RW4hyBjTWSYDOxapodpNEtX0g5Eb16sxklBqmd2RHcE= nhooyr.io/websocket v1.8.6 h1:s+C3xAMLwGmlI31Nyn/eAehUlZPwfYZu2JXM621Q5/k= nhooyr.io/websocket v1.8.6/go.mod h1:B70DZP8IakI65RVQ51MsWP/8jndNma26DVA/nFSCgW0= diff --git a/sidecar/main.go b/sidecar/main.go index 84a8e640..614ec390 100644 --- a/sidecar/main.go +++ b/sidecar/main.go @@ -2,15 +2,20 @@ // that the controller drives over kube-rbac-proxy. It runs as a native // restartable sidecar container beside seid in every SeiNode pod. // -// It ships as `serve` under a root command rather than as a bare binary because -// the controller renders `Command: []string{"seictl", "serve"}` into every pod -// spec. The image keeps a `seictl` symlink for that reason; see Dockerfile. +// The controller renders no `Command` for the sidecar container, so the image's +// ENTRYPOINT is what runs it: `sei-sidecar serve`. `serve` is a subcommand +// rather than the root action so the binary keeps room for operator +// subcommands, and it is named in the ENTRYPOINT rather than left as a default +// because a bare invocation prints help and exits 0 — under +// restartPolicy: Always that is a container looping while reporting success. package main import ( "context" + "errors" "fmt" "os" + "strings" "github.com/sei-protocol/seilog" "github.com/urfave/cli/v3" @@ -50,9 +55,28 @@ func main() { Commands: []*cli.Command{&serveCmd}, } + cmd.Before = func(ctx context.Context, _ *cli.Command) (context.Context, error) { + return ctx, validateHome(destinations.home) + } + if err := cmd.Run(context.Background(), os.Args); err != nil { fmt.Fprintf(os.Stderr, "error: %v\n", err) _ = seilog.Close() os.Exit(1) } } + +// validateHome rejects a set-but-empty home directory. +// +// The flag's Required only tests whether a value was supplied, and SEI_HOME="" +// supplies one. Without this check the sidecar starts and resolves every path +// relative to its working directory — measured, it creates config/, data/ and +// sidecar.db there and serves normally — which is the silent wrong-directory +// failure Required is there to prevent. +func validateHome(home string) error { + if strings.TrimSpace(home) == "" { + return errors.New("SEI_HOME is set but empty; it must name the node's data " + + "volume, or every path resolves relative to the working directory") + } + return nil +} diff --git a/sidecar/startup_guard_test.go b/sidecar/startup_guard_test.go index 5a1e2990..e3c60e6b 100644 --- a/sidecar/startup_guard_test.go +++ b/sidecar/startup_guard_test.go @@ -86,3 +86,38 @@ func TestUnauthenticatedIsTheZeroValue(t *testing.T) { server.AuthnModeUnauthenticated) } } + +// The flag's Required only checks that a value was supplied, and SEI_HOME="" +// supplies one. Before validateHome existed, an empty value started the sidecar +// and it created config/, data/ and sidecar.db relative to its working +// directory — a running, probe-passing sidecar operating off the data volume. +func TestValidateHome(t *testing.T) { + cases := []struct { + name string + home string + wantErr bool + }{ + {name: "empty is refused", home: "", wantErr: true}, + {name: "whitespace-only is refused", home: " ", wantErr: true}, + {name: "tab is refused", home: "\t", wantErr: true}, + {name: "a real path is accepted", home: "/home/nonroot/.sei", wantErr: false}, + {name: "a relative path is accepted — odd but the operator's choice", home: "sei", wantErr: false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + err := validateHome(tc.home) + if tc.wantErr { + if err == nil { + t.Fatal("expected a refusal, got nil") + } + if !strings.Contains(err.Error(), "SEI_HOME") { + t.Errorf("error should name SEI_HOME; got %q", err.Error()) + } + return + } + if err != nil { + t.Errorf("expected nil, got %v", err) + } + }) + } +} diff --git a/sidecarapi/go.mod b/sidecarapi/go.mod index 5b63d44b..6db70050 100644 --- a/sidecarapi/go.mod +++ b/sidecarapi/go.mod @@ -6,8 +6,8 @@ require ( github.com/cosmos/btcutil v1.0.5 github.com/google/uuid v1.6.0 github.com/leanovate/gopter v0.2.11 - github.com/oapi-codegen/runtime v1.6.0 - github.com/pelletier/go-toml/v2 v2.2.2 + github.com/oapi-codegen/runtime v1.2.0 + github.com/pelletier/go-toml/v2 v2.2.4 github.com/sei-protocol/sei-config v0.0.25 ) diff --git a/sidecarapi/go.sum b/sidecarapi/go.sum index aa898816..6df8762e 100644 --- a/sidecarapi/go.sum +++ b/sidecarapi/go.sum @@ -204,14 +204,12 @@ github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lN github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo= github.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM= -github.com/oapi-codegen/nullable v1.1.0 h1:eAh8JVc5430VtYVnq00Hrbpag9PFRGWLjxR1/3KntMs= -github.com/oapi-codegen/nullable v1.1.0/go.mod h1:KUZ3vUzkmEKY90ksAmit2+5juDIhIZhfDl+0PwOQlFY= -github.com/oapi-codegen/runtime v1.6.0 h1:7Xx+GlueD6nRuyKoCPzL434Jfi3BetbiJOrzCHp/VPU= -github.com/oapi-codegen/runtime v1.6.0/go.mod h1:GwV7hC2hviaMzj+ITfHVRESK5J2W/GefVwIND/bMGvU= +github.com/oapi-codegen/runtime v1.2.0 h1:RvKc1CVS1QeKSNzO97FBQbSMZyQ8s6rZd+LpmzwHMP4= +github.com/oapi-codegen/runtime v1.2.0/go.mod h1:Y7ZhmmlE8ikZOmuHRRndiIm7nf3xcVv+YMweKgG1DT0= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= -github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM= -github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs= +github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4= +github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= @@ -242,21 +240,14 @@ github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An github.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns= github.com/spkg/bom v0.0.0-20160624110644-59b7046e48ad/go.mod h1:qLr4V1qq6nMqFKkMo8ZTx3f+BZEkzsRUY10Xsm2mwU0= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= -github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= -github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= -github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= -github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= -github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= -github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= -github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= From 0145a9b1a51425d29fa5c1011a0545ee47fc48e3 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Fri, 14 Aug 2026 11:09:40 -0700 Subject: [PATCH 06/13] ci(sidecar): publish the sidecar image, and build both images on every PR MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the publish path for the sidecar image and the PR-time check that would have caught its build being broken. `publish-sidecar` is a separate job rather than a third step in `publish`. ECR sets image_tag_mutability = IMMUTABLE, so re-running the workflow on a sha whose controller image already exists fails that step and aborts every step after it — the sidecar would silently never publish. Independent jobs also stop a sidecar build failure from blocking the controller image. It gets its own cache ref for the reason the integration-harness already has one, and more so: this build pulls the whole sei-chain graph, and the image holds an operator keyring at runtime, so a poisoned layer must not reach the controller image's build. arm64 is built as well as amd64, matching what the GHCR build being replaced produced rather than quietly dropping an architecture. That needed a Dockerfile change: the build stage is now pinned with --platform=$BUILDPLATFORM so it stays on the runner's native arch and cross-compiles via the GOOS/GOARCH it already passes. Without the pin, buildx emulates that stage under QEMU per non-native target, which means compiling sei-chain through emulation. Both arches verified building. The new `docker` job in CI builds both images with push: false. Nothing in CI compiled a Dockerfile before, and the publish workflow only runs on main, so a build-context problem reached main unseen — which is exactly what happened: `.dockerignore` starts with `**`, and an embedded asset outside its re-include list failed the sidecar build with `pattern config.toml: no matching files found` while every other check stayed green. amd64 only there; doubling a sei-chain-sized build on every PR is not worth an arch-specific break the cross-compile makes unlikely. Also `docker-build-sidecar` now passes --platform linux/amd64 like `docker-build`. Without it the target silently produces an arm64 image on an Apple Silicon machine, which no cluster node runs. Nothing publishes until sei/sei-sidecar exists — platform#1535. Verified: both workflow files parse; the sidecar builds from a context filtered by the repo's own .dockerignore rules; linux/amd64 and linux/arm64 both cross-compile; tidy-check, verify-generated and all 25 test packages across the three modules unchanged. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 35 +++++++++++++++++++++++++++++++++ .github/workflows/ecr.yml | 41 +++++++++++++++++++++++++++++++++++++++ Makefile | 4 +++- sidecar/Dockerfile | 7 ++++++- 4 files changed, 85 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b4612a71..75699288 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -44,6 +44,41 @@ jobs: # `go mod tidy` cannot resolve the graph at all. - run: make tidy-check + # Build both images without pushing. Nothing else in CI compiles a Dockerfile, + # and the publish workflow only runs on main — so a build-context problem + # reaches main unseen. That is not hypothetical: `.dockerignore` starts with + # `**`, and an embedded asset outside its re-include list + # (sidecar/tasks/defaults/config.toml, a //go:embed target) failed the sidecar + # build with `pattern config.toml: no matching files found` while every other + # check stayed green. + docker: + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - uses: actions/checkout@v4 + - uses: docker/setup-buildx-action@v3 + - name: Build controller image + uses: docker/build-push-action@v6 + with: + context: . + push: false + platforms: linux/amd64 + cache-from: type=gha,scope=controller + cache-to: type=gha,scope=controller,mode=max + - name: Build sidecar image + uses: docker/build-push-action@v6 + with: + context: . + file: sidecar/Dockerfile + push: false + # amd64 only here — the publish job builds arm64 too, but doubling a + # sei-chain-sized build on every PR is not worth catching an + # arch-specific break that the Dockerfile's cross-compile makes + # unlikely. + platforms: linux/amd64 + cache-from: type=gha,scope=sidecar + cache-to: type=gha,scope=sidecar,mode=max + test: runs-on: ubuntu-latest steps: diff --git a/.github/workflows/ecr.yml b/.github/workflows/ecr.yml index 5192a748..1972142b 100644 --- a/.github/workflows/ecr.yml +++ b/.github/workflows/ecr.yml @@ -56,3 +56,44 @@ jobs: # controller image's build. cache-from: type=registry,ref=${{ steps.ecr-login.outputs.registry }}/sei/build-cache:integration-harness cache-to: type=registry,ref=${{ steps.ecr-login.outputs.registry }}/sei/build-cache:integration-harness,mode=max + + # A separate job, not a third step in `publish`. ECR sets + # image_tag_mutability = IMMUTABLE, so re-running this workflow on a sha whose + # controller image is already pushed fails that step and would abort every + # step after it — the sidecar would silently never publish. Independent jobs + # also mean a sidecar build failure does not block the controller image. + publish-sidecar: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: arn:aws:iam::189176372795:role/common/gha + role-duration-seconds: 900 + aws-region: us-east-2 + + - id: ecr-login + uses: aws-actions/amazon-ecr-login@v2 + + - uses: docker/setup-buildx-action@v3 + + # The per-node sidecar. Built from the repo root so the sidecarapi module + # it resolves through a filesystem `replace` is in the context. + - name: Build and push sidecar image + uses: docker/build-push-action@v6 + with: + context: . + file: sidecar/Dockerfile + push: true + # arm64 as well as amd64, matching what the GHCR build this replaces + # produced. The Dockerfile pins its build stage to $BUILDPLATFORM and + # cross-compiles, so neither target needs QEMU. + platforms: linux/amd64,linux/arm64 + tags: ${{ steps.ecr-login.outputs.registry }}/sei/sei-sidecar:${{ inputs.tag || github.sha }} + # Dedicated cache ref, for the same reason the integration-harness has + # one — and more so here: this build pulls the whole sei-chain graph, + # and the image holds an operator keyring at runtime, so a poisoned + # layer must not be able to reach the controller image's build. + cache-from: type=registry,ref=${{ steps.ecr-login.outputs.registry }}/sei/build-cache:sei-sidecar + cache-to: type=registry,ref=${{ steps.ecr-login.outputs.registry }}/sei/build-cache:sei-sidecar,mode=max diff --git a/Makefile b/Makefile index 0a9da6e3..88835428 100644 --- a/Makefile +++ b/Makefile @@ -40,7 +40,9 @@ build-sidecar: ## Build the sidecar binary. docker-build-sidecar: ## Build the sidecar container image. @# Build context is the repo root — sidecar/ resolves sidecarapi/ through a @# filesystem replace, so both module trees must be in the context. - $(CONTAINER_TOOL) build -f sidecar/Dockerfile -t $(SIDECAR_IMG) . + @# --platform matches docker-build: without it this silently produces an + @# arm64 image on an Apple Silicon machine, which no cluster node runs. + $(CONTAINER_TOOL) build --platform linux/amd64 -f sidecar/Dockerfile -t $(SIDECAR_IMG) . test: test-modules ## Run tests (root module with coverage, then every other module). go test $$(go list ./... | grep -v /e2e) -coverprofile cover.out diff --git a/sidecar/Dockerfile b/sidecar/Dockerfile index e700d74b..7657796d 100644 --- a/sidecar/Dockerfile +++ b/sidecar/Dockerfile @@ -8,7 +8,12 @@ # and google.golang.org/grpc *down* relative to the controller module. Under a # workspace those replaces would be promoted to main-module status and leak into # the root module's resolution; off keeps this build identical to a consumer's. -FROM docker.io/golang:1.26 AS build +# +# --platform=$BUILDPLATFORM keeps the build stage on the runner's native +# architecture and cross-compiles with GOOS/GOARCH below. Without it, buildx +# emulates this stage under QEMU for every non-native target, which means +# compiling the whole sei-chain graph through emulation. +FROM --platform=$BUILDPLATFORM docker.io/golang:1.26 AS build ENV GOWORK=off WORKDIR /workspace From c21941f5b93c32b4cfc4ed71ab79c5f4dc1d354d Mon Sep 17 00:00:00 2001 From: bdchatham Date: Fri, 14 Aug 2026 11:12:01 -0700 Subject: [PATCH 07/13] fix(samples): stop pinning the sidecar image in the sample manifests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All five samples set spec.sidecar.image to a ghcr.io/sei-protocol/seictl digest. That was harmless while the controller rendered `Command: ["seictl","serve"]` — the subcommand came from the pod spec, so any image with that binary worked. It is not harmless now. The controller renders no Command, so the image's ENTRYPOINT is the command, and an older image reached through a pin gives a container that prints help and exits 0, restarting forever under restartPolicy: Always while seid blocks on /v0/healthz behind a StartupProbe with FailureThreshold 86400 — about five days before Kubernetes calls it failed. Removing the field rather than repointing it. spec.sidecar is +optional and the blocks carried nothing but the image, so the samples now resolve the sidecar from images.sidecar in the platform app-config — which is what every real node does; no live node sets this field. The samples were teaching a pin nobody uses, and the pin is the thing that outlives a coordinated entrypoint change. README gains the reason, so the omission does not read as an oversight, and its sidecar description no longer points at the seictl repo for an image this repo now builds. Verified: all sample manifests still parse; build, verify-generated and the 14 root test packages unchanged. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 12 +++++++++--- manifests/samples/seinetwork/genesis-ceremony.yaml | 3 --- manifests/samples/seinode/pacific-1-full-node.yaml | 3 --- .../samples/seinode/pacific-1-shadow-replayer.yaml | 3 --- manifests/samples/seinode/pacific-1-snapshotter.yaml | 3 --- .../samples/seinode/pacific-1-state-syncer.yaml | 3 --- 6 files changed, 9 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index ab88841c..f71afc1b 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ A Kubernetes operator for managing the full lifecycle of [Sei](https://sei.io) b - **One StatefulSet per node** — each `SeiNode` gets its own single-replica StatefulSet rather than pooling nodes. Groups exist for fleet coordination. - **Dedicated node scheduling** — pods require `karpenter.sh/nodepool=sei-node` and tolerate `sei.io/workload=sei-node:NoSchedule`, keeping blockchain workloads off general-purpose nodes. -- **Sidecar architecture** — every node runs a [seictl](https://github.com/sei-protocol/seictl) sidecar as a restartable init container that drives bootstrap tasks before seid starts and handles runtime operations afterward. +- **Sidecar architecture** — every node runs a `sei-sidecar` container (built from `sidecar/` in this repo) as a restartable init container that drives bootstrap tasks before seid starts and handles runtime operations afterward. The controller renders no `command` for it; the image's entrypoint is the command. - **Plan model** — bootstrap is driven by a `TaskPlan` stored in `status.plan`. The controller builds a task sequence based on the node's mode, submits tasks to the sidecar one at a time, and advances through the plan. - **Environment-driven genesis** — genesis resolution is handled by the sidecar autonomously. Embedded sei-config is checked first for well-known chains (pacific-1, atlantic-2, arctic-1), then S3 fallback at `{SEI_GENESIS_BUCKET}/{chainID}/genesis.json`. @@ -37,10 +37,16 @@ spec: genesis: chainId: my-devnet stakingAmount: "10000000usei" - sidecar: - image: ghcr.io/sei-protocol/seictl:v0.0.29 ``` +`spec.sidecar.image` is omitted here on purpose, as in every sample under +`manifests/samples/`: the sidecar image comes from `images.sidecar` in the +platform app-config, so one value governs a whole cell. Setting it per node +overrides that, and a pin left behind on an older image outlives a coordinated +entrypoint change — the container then exits 0 in a restart loop while seid +waits on `/v0/healthz` behind a five-day startup probe. Pin only to debug, and +only to an image whose entrypoint matches what the controller renders. + ### SeiNode Manages a single Sei node. Supports full nodes, validators, archivers, and replayers. diff --git a/manifests/samples/seinetwork/genesis-ceremony.yaml b/manifests/samples/seinetwork/genesis-ceremony.yaml index de29b8dc..f8558bbc 100644 --- a/manifests/samples/seinetwork/genesis-ceremony.yaml +++ b/manifests/samples/seinetwork/genesis-ceremony.yaml @@ -30,7 +30,4 @@ spec: chainId: genesis-test-1 image: "ghcr.io/sei-protocol/sei:v6.3.0" - sidecar: - image: ghcr.io/sei-protocol/seictl@sha256:2cb320dd583000765520293d4e28ecc79fed4432c76760923d3aaa47803a4dbf - validator: {} diff --git a/manifests/samples/seinode/pacific-1-full-node.yaml b/manifests/samples/seinode/pacific-1-full-node.yaml index 843897f8..7b541e75 100644 --- a/manifests/samples/seinode/pacific-1-full-node.yaml +++ b/manifests/samples/seinode/pacific-1-full-node.yaml @@ -12,9 +12,6 @@ spec: chainId: pacific-1 image: "ghcr.io/sei-protocol/sei:v6.3.0" - sidecar: - image: ghcr.io/sei-protocol/seictl@sha256:2cb320dd583000765520293d4e28ecc79fed4432c76760923d3aaa47803a4dbf - peers: - ec2Tags: region: eu-central-1 diff --git a/manifests/samples/seinode/pacific-1-shadow-replayer.yaml b/manifests/samples/seinode/pacific-1-shadow-replayer.yaml index 136773c0..bb465999 100644 --- a/manifests/samples/seinode/pacific-1-shadow-replayer.yaml +++ b/manifests/samples/seinode/pacific-1-shadow-replayer.yaml @@ -13,9 +13,6 @@ spec: chainId: pacific-1 image: ghcr.io/bdchatham/sei-shadow@sha256:5dee59eaf2d0841a6e6c0f155bbafa5519a283295dd36e4271c6012f751afcd6 - sidecar: - image: ghcr.io/sei-protocol/seictl@sha256:2cb320dd583000765520293d4e28ecc79fed4432c76760923d3aaa47803a4dbf - overrides: giga_executor.enabled: "true" giga_executor.occ_enabled: "true" diff --git a/manifests/samples/seinode/pacific-1-snapshotter.yaml b/manifests/samples/seinode/pacific-1-snapshotter.yaml index 4983ed17..40e0c962 100644 --- a/manifests/samples/seinode/pacific-1-snapshotter.yaml +++ b/manifests/samples/seinode/pacific-1-snapshotter.yaml @@ -12,9 +12,6 @@ spec: chainId: pacific-1 image: "ghcr.io/sei-protocol/sei:v6.3.0" - sidecar: - image: ghcr.io/sei-protocol/seictl@sha256:2cb320dd583000765520293d4e28ecc79fed4432c76760923d3aaa47803a4dbf - peers: - ec2Tags: region: eu-central-1 diff --git a/manifests/samples/seinode/pacific-1-state-syncer.yaml b/manifests/samples/seinode/pacific-1-state-syncer.yaml index 701b128c..1350f211 100644 --- a/manifests/samples/seinode/pacific-1-state-syncer.yaml +++ b/manifests/samples/seinode/pacific-1-state-syncer.yaml @@ -11,9 +11,6 @@ spec: chainId: pacific-1 image: "ghcr.io/sei-protocol/sei:v6.3.0" - sidecar: - image: ghcr.io/sei-protocol/seictl@sha256:2cb320dd583000765520293d4e28ecc79fed4432c76760923d3aaa47803a4dbf - genesis: s3: uri: s3://sei-testnet-genesis-config/pacific-1/genesis.json From a6f6ead73dfef84e3c28375bd659e0da9341e1d5 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Fri, 14 Aug 2026 13:17:09 -0700 Subject: [PATCH 08/13] ci(sidecar): give the publish job credentials that outlast its build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The publish-sidecar job assumed the GHA role for 900 seconds and then compiled the sei-chain graph for linux/amd64 and linux/arm64. On a cold cache that overruns 15 minutes, so the credentials expire before the push and the image never reaches ECR — a failure that only appears once the cache is cold, which is exactly when a release is being cut. Raised to 3600, inside the role's max_session_duration of 7200 (terraform/aws/189176372795/us-east-2/common/gha.tf). The controller job keeps 900: it builds one architecture over a much smaller graph. Also bounds the job at 60 minutes so a stuck build fails instead of holding a runner. Also drops two history-in-source comments from sidecar/main.go that described where the home flag used to live in seictl. The invariant they were protecting is stated as present-state and enforced by validateHome. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ecr.yml | 10 +++++++++- sidecar/main.go | 11 +++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/.github/workflows/ecr.yml b/.github/workflows/ecr.yml index 1972142b..98fffa1d 100644 --- a/.github/workflows/ecr.yml +++ b/.github/workflows/ecr.yml @@ -64,13 +64,21 @@ jobs: # also mean a sidecar build failure does not block the controller image. publish-sidecar: runs-on: ubuntu-latest + # Bounded so a stuck build fails rather than burning a runner for six hours; + # generous because a cold-cache two-arch sei-chain compile is slow. + timeout-minutes: 60 steps: - uses: actions/checkout@v4 - uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: arn:aws:iam::189176372795:role/common/gha - role-duration-seconds: 900 + # Longer than the controller job's 900s, and deliberately: this build + # compiles the sei-chain graph for two architectures, which overruns + # 15 minutes on a cold cache — the credentials would expire before the + # push and the image would never reach ECR. 3600 is inside the role's + # max_session_duration of 7200 (terraform .../common/gha.tf). + role-duration-seconds: 3600 aws-region: us-east-2 - id: ecr-login diff --git a/sidecar/main.go b/sidecar/main.go index 614ec390..976ad6d2 100644 --- a/sidecar/main.go +++ b/sidecar/main.go @@ -21,13 +21,12 @@ import ( "github.com/urfave/cli/v3" ) -// destinations holds flag-bound values shared with the subcommands, mirroring -// the shape the seictl CLI used so serve.go's call sites are unchanged. +// destinations holds flag-bound values the subcommands read. // -// home is the one that matters. It binds SEI_HOME, which the controller sets to -// the node's data-PVC mount. In seictl this flag lived on the root command and -// serve.go read it from here; the wiring is reproduced deliberately, because -// losing it does not fail — it silently relocates every write off the PVC. +// home binds SEI_HOME, which the controller sets to the node's data-PVC mount. +// It lives on the root command so every subcommand resolves the same directory, +// and losing that binding does not fail — it silently relocates every write off +// the PVC, which is why validateHome guards it. var destinations = struct { home string }{} From 46b625efc19ee20842273218e1e102df62313f50 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Fri, 14 Aug 2026 13:44:37 -0700 Subject: [PATCH 09/13] fix(sidecar): close the review findings on floors, home validation, and publish MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A three-lens re-review (all DISSENT) plus its own evidence reversed three calls made earlier in this branch. Each fix below is a correction, not a refinement. Dependency floors were 3/19 restored, not restored. The previous commit claimed "floors restored to seictl's" after pinning six direct requires. Measured against the shipping build list, 35 linked modules still differed. Sixteen were AWS SDK modules ahead of what seictl shipped — the whole credential chain plus service/internal/checksum, which owns S3 request-checksum defaults on the multi-GB snapshot path. Nineteen were *behind*: x/crypto, x/net, grpc, protobuf, otel and genproto, because seictl's module also linked the controller through sdk/sei and MVS pulled them up, while a standalone sidecar module settles lower. Those are security patches, silently given back. All 35 are now pinned to what seictl shipped; drift is zero and a re-tidy is a no-op. The sqlite pin carries the reason it exists, since a bare version number is indistinguishable from drift on the next `go get -u`. The home guard shadowed the check it claimed to rely on. urfave runs Before hooks ahead of its own required-flag check, so validateHome in a root Before made `Required` unreachable: an *unset* SEI_HOME reported "is set but empty", sending an operator to look for a value that does not exist. Moved to the flag's Action, which runs only for flags actually set — so the unset case reaches the required-flag check that describes it correctly. Verified across every input shape. Non-empty was also the wrong test. The stated harm is that paths resolve against the working directory, and a relative SEI_HOME does exactly that; the previous test pinned `"sei"` as acceptable. Now refused, with filepath.Clean restored on the way through — seictl normalized the value in this same Action and the relocation had dropped it, which matters for the two consumers that take home raw rather than through filepath.Join. The credential widening was wrong, and so was the reason given for it. The push authenticates with the ECR token amazon-ecr-login mints, valid twelve hours, not with the STS session — and ecr-seid-musl.yml builds a heavier image at 900s and publishes. Reverted to 900; 3600 was a gratuitous 4x on a role that can PutImage to every sei/* repository. arm64 is dropped. The sei-node Karpenter pool pins kubernetes.io/arch In [amd64], so no node can schedule an arm64 sidecar, and all three existing publishers are single-arch. Publishing one would have made this repository the registry's first multi-arch index under an IMMUTABLE tag policy whose only lifecycle rule expires untagged manifests — an index's children — at seven days. Untested, and unrepairable at the same tag. Also: a concurrency group on CI, since push and pull_request both fire on every branch and each run now builds two images; contents: read on the docker job, which is the one that builds from branch-controlled Dockerfiles and was the only job without it; and the sidecar image field's API doc now carries the entrypoint constraint, because that description is what `kubectl explain` prints. Verified: gofmt, goimports, builds, tidy-check and verify-generated clean across three modules; 25 test packages pass; module drift against the shipping build list is zero; SEI_HOME unset, empty, whitespace and relative all refuse with no files created, and an absolute path is accepted and normalized. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 12 +++ .github/workflows/ecr.yml | 20 ++-- api/v1alpha1/common_types.go | 11 ++- config/crd/sei.io_seinetworks.yaml | 12 ++- config/crd/sei.io_seinodes.yaml | 12 ++- manifests/sei.io_seinetworks.yaml | 12 ++- manifests/sei.io_seinodes.yaml | 12 ++- sidecar/go.mod | 81 ++++++++-------- sidecar/go.sum | 148 ++++++++++++++--------------- sidecar/main.go | 66 +++++++------ sidecar/serve.go | 6 +- sidecar/startup_guard_test.go | 43 +++++---- 12 files changed, 255 insertions(+), 180 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 75699288..eae083f9 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -6,6 +6,13 @@ on: pull_request: branches: ["*"] +# `push` and `pull_request` both fire on every branch, so a PR commit runs this +# workflow twice. Without a group, superseded runs keep going — and each one now +# builds two container images. +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + jobs: lint: runs-on: ubuntu-latest @@ -54,6 +61,11 @@ jobs: docker: runs-on: ubuntu-latest timeout-minutes: 30 + # This job builds container images from Dockerfiles the branch controls, so + # it gets the narrowest token in the workflow. The repository default is + # write. + permissions: + contents: read steps: - uses: actions/checkout@v4 - uses: docker/setup-buildx-action@v3 diff --git a/.github/workflows/ecr.yml b/.github/workflows/ecr.yml index 98fffa1d..1aa15d84 100644 --- a/.github/workflows/ecr.yml +++ b/.github/workflows/ecr.yml @@ -73,12 +73,7 @@ jobs: - uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: arn:aws:iam::189176372795:role/common/gha - # Longer than the controller job's 900s, and deliberately: this build - # compiles the sei-chain graph for two architectures, which overruns - # 15 minutes on a cold cache — the credentials would expire before the - # push and the image would never reach ECR. 3600 is inside the role's - # max_session_duration of 7200 (terraform .../common/gha.tf). - role-duration-seconds: 3600 + role-duration-seconds: 900 aws-region: us-east-2 - id: ecr-login @@ -94,10 +89,15 @@ jobs: context: . file: sidecar/Dockerfile push: true - # arm64 as well as amd64, matching what the GHCR build this replaces - # produced. The Dockerfile pins its build stage to $BUILDPLATFORM and - # cross-compiles, so neither target needs QEMU. - platforms: linux/amd64,linux/arm64 + # amd64 only. The sei-node Karpenter pool pins + # kubernetes.io/arch In [amd64] (platform repo, + # clusters/prod/default/nodepool-default.yaml), so no node can schedule + # an arm64 sidecar. Publishing one would also make this repository the + # registry's first multi-arch image index, under an IMMUTABLE tag + # policy whose only lifecycle rule expires *untagged* manifests — the + # child manifests of an index — at 7 days. Untested, and a broken index + # cannot be replaced at the same tag. + platforms: linux/amd64 tags: ${{ steps.ecr-login.outputs.registry }}/sei/sei-sidecar:${{ inputs.tag || github.sha }} # Dedicated cache ref, for the same reason the integration-harness has # one — and more so here: this build pulls the whole sei-chain graph, diff --git a/api/v1alpha1/common_types.go b/api/v1alpha1/common_types.go index bcc5e823..d90fe8dc 100644 --- a/api/v1alpha1/common_types.go +++ b/api/v1alpha1/common_types.go @@ -207,7 +207,16 @@ type ShadowResultConfig struct { // SidecarConfig configures the sei-sidecar container. type SidecarConfig struct { - // Image overrides the sidecar container image. + // Image overrides the sidecar container image for this node, in place of the + // platform config's images.sidecar. Prefer leaving it unset so one value + // governs the whole cell. + // + // The controller renders no command for the sidecar container, so the image's + // entrypoint is the command. An image whose entrypoint does not match what + // the controller expects fails quietly rather than loudly: the container + // exits 0 and restarts while seid waits on /v0/healthz behind a startup + // probe that tolerates roughly five days. Pin only to debug, and only to an + // image built alongside the running controller. // +optional Image string `json:"image,omitempty"` diff --git a/config/crd/sei.io_seinetworks.yaml b/config/crd/sei.io_seinetworks.yaml index 6bb439dd..f4c98e12 100644 --- a/config/crd/sei.io_seinetworks.yaml +++ b/config/crd/sei.io_seinetworks.yaml @@ -274,7 +274,17 @@ spec: genesis validator. properties: image: - description: Image overrides the sidecar container image. + description: |- + Image overrides the sidecar container image for this node, in place of the + platform config's images.sidecar. Prefer leaving it unset so one value + governs the whole cell. + + The controller renders no command for the sidecar container, so the image's + entrypoint is the command. An image whose entrypoint does not match what + the controller expects fails quietly rather than loudly: the container + exits 0 and restarts while seid waits on /v0/healthz behind a startup + probe that tolerates roughly five days. Pin only to debug, and only to an + image built alongside the running controller. type: string port: default: 7777 diff --git a/config/crd/sei.io_seinodes.yaml b/config/crd/sei.io_seinodes.yaml index 62ba6dc7..bccd5a33 100644 --- a/config/crd/sei.io_seinodes.yaml +++ b/config/crd/sei.io_seinodes.yaml @@ -504,7 +504,17 @@ spec: description: Sidecar configures the sei-sidecar container. properties: image: - description: Image overrides the sidecar container image. + description: |- + Image overrides the sidecar container image for this node, in place of the + platform config's images.sidecar. Prefer leaving it unset so one value + governs the whole cell. + + The controller renders no command for the sidecar container, so the image's + entrypoint is the command. An image whose entrypoint does not match what + the controller expects fails quietly rather than loudly: the container + exits 0 and restarts while seid waits on /v0/healthz behind a startup + probe that tolerates roughly five days. Pin only to debug, and only to an + image built alongside the running controller. type: string port: default: 7777 diff --git a/manifests/sei.io_seinetworks.yaml b/manifests/sei.io_seinetworks.yaml index 6bb439dd..f4c98e12 100644 --- a/manifests/sei.io_seinetworks.yaml +++ b/manifests/sei.io_seinetworks.yaml @@ -274,7 +274,17 @@ spec: genesis validator. properties: image: - description: Image overrides the sidecar container image. + description: |- + Image overrides the sidecar container image for this node, in place of the + platform config's images.sidecar. Prefer leaving it unset so one value + governs the whole cell. + + The controller renders no command for the sidecar container, so the image's + entrypoint is the command. An image whose entrypoint does not match what + the controller expects fails quietly rather than loudly: the container + exits 0 and restarts while seid waits on /v0/healthz behind a startup + probe that tolerates roughly five days. Pin only to debug, and only to an + image built alongside the running controller. type: string port: default: 7777 diff --git a/manifests/sei.io_seinodes.yaml b/manifests/sei.io_seinodes.yaml index 62ba6dc7..bccd5a33 100644 --- a/manifests/sei.io_seinodes.yaml +++ b/manifests/sei.io_seinodes.yaml @@ -504,7 +504,17 @@ spec: description: Sidecar configures the sei-sidecar container. properties: image: - description: Image overrides the sidecar container image. + description: |- + Image overrides the sidecar container image for this node, in place of the + platform config's images.sidecar. Prefer leaving it unset so one value + governs the whole cell. + + The controller renders no command for the sidecar container, so the image's + entrypoint is the command. An image whose entrypoint does not match what + the controller expects fails quietly rather than loudly: the container + exits 0 and restarts while seid waits on /v0/healthz behind a startup + probe that tolerates roughly five days. Pin only to debug, and only to an + image built alongside the running controller. type: string port: default: 7777 diff --git a/sidecar/go.mod b/sidecar/go.mod index f71081dd..cd51c012 100644 --- a/sidecar/go.mod +++ b/sidecar/go.mod @@ -3,11 +3,11 @@ module github.com/sei-protocol/sei-k8s-controller/sidecar go 1.26.0 require ( - github.com/aws/aws-sdk-go-v2 v1.43.5 + github.com/aws/aws-sdk-go-v2 v1.41.6 github.com/aws/aws-sdk-go-v2/config v1.32.12 github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.1.10 github.com/aws/aws-sdk-go-v2/service/s3 v1.97.2 - github.com/aws/smithy-go v1.27.7 + github.com/aws/smithy-go v1.25.0 github.com/ethereum/go-ethereum v1.16.8 github.com/google/uuid v1.6.0 github.com/prometheus/client_golang v1.23.2 @@ -16,6 +16,13 @@ require ( github.com/sei-protocol/sei-k8s-controller/sidecarapi v0.0.0 github.com/sei-protocol/seilog v0.0.3 github.com/urfave/cli/v3 v3.6.1 + // Pinned, not floated. This driver embeds the SQLite engine that reads and + // writes sidecar.db on the node's data volume — the pre-broadcast idempotency + // marker for sign-tx tasks lives there. A `go mod tidy` in a fresh module + // resolves the lowest version each import allows, which selected v1.18.1 + // (engine 3.39.2) against the v1.21.2 (engine 3.41.2) the shipping binary + // used. Every version here matches what seictl shipped, so this relocation + // links what it replaced; do not bump without deciding the on-disk question. modernc.org/sqlite v1.21.2 ) @@ -29,21 +36,21 @@ require ( github.com/RaduBerinde/btreemap v0.0.0-20250419174037-3d62b7205d54 // indirect github.com/alitto/pond v1.8.3 // indirect github.com/armon/go-metrics v0.4.1 // indirect - github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.17 // indirect - github.com/aws/aws-sdk-go-v2/credentials v1.19.35 // indirect - github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36 // indirect - github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36 // indirect - github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36 // indirect + github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 // indirect + github.com/aws/aws-sdk-go-v2/credentials v1.19.12 // indirect + github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20 // indirect + github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 // indirect + github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22 // indirect github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 // indirect - github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37 // indirect - github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.16 // indirect - github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.29 // indirect - github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.36 // indirect - github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.37 // indirect - github.com/aws/aws-sdk-go-v2/service/signin v1.5.5 // indirect - github.com/aws/aws-sdk-go-v2/service/sso v1.33.5 // indirect - github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.5 // indirect - github.com/aws/aws-sdk-go-v2/service/sts v1.45.5 // indirect + github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20 // indirect + github.com/aws/aws-sdk-go-v2/service/signin v1.0.8 // indirect + github.com/aws/aws-sdk-go-v2/service/sso v1.30.13 // indirect + github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17 // indirect + github.com/aws/aws-sdk-go-v2/service/sts v1.41.9 // indirect github.com/benbjohnson/immutable v0.4.3 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/bgentry/speakeasy v0.2.0 // indirect @@ -100,7 +107,7 @@ require ( github.com/google/orderedcode v0.0.1 // indirect github.com/gorilla/handlers v1.5.2 // indirect github.com/gorilla/mux v1.8.1 // indirect - github.com/gorilla/websocket v1.5.3 // indirect + github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect github.com/grafana/pyroscope-go/godeltaprof v0.1.8 // indirect github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect @@ -136,7 +143,7 @@ require ( github.com/pkg/errors v0.9.1 // indirect github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect github.com/prometheus/client_model v0.6.2 // indirect - github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/common v0.67.5 // indirect github.com/prometheus/procfs v0.17.0 // indirect github.com/rakyll/statik v0.1.7 // indirect github.com/regen-network/cosmos-proto v0.3.1 // indirect @@ -176,30 +183,30 @@ require ( go.etcd.io/bbolt v1.4.0-alpha.0.0.20240404170359-43604f3112c5 // indirect go.opencensus.io v0.24.0 // indirect go.opentelemetry.io/auto/sdk v1.2.1 // indirect - go.opentelemetry.io/otel v1.39.0 // indirect + go.opentelemetry.io/otel v1.43.0 // indirect go.opentelemetry.io/otel/exporters/jaeger v1.9.0 // indirect - go.opentelemetry.io/otel/metric v1.39.0 // indirect - go.opentelemetry.io/otel/sdk v1.39.0 // indirect - go.opentelemetry.io/otel/trace v1.39.0 // indirect + go.opentelemetry.io/otel/metric v1.43.0 // indirect + go.opentelemetry.io/otel/sdk v1.43.0 // indirect + go.opentelemetry.io/otel/trace v1.43.0 // indirect go.uber.org/multierr v1.11.0 // indirect go.uber.org/zap v1.27.0 // indirect - go.yaml.in/yaml/v2 v2.4.3 // indirect + go.yaml.in/yaml/v2 v2.4.4 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect - golang.org/x/crypto v0.47.0 // indirect + golang.org/x/crypto v0.49.0 // indirect golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect - golang.org/x/mod v0.32.0 // indirect - golang.org/x/net v0.49.0 // indirect - golang.org/x/sync v0.19.0 // indirect - golang.org/x/sys v0.40.0 // indirect - golang.org/x/term v0.39.0 // indirect - golang.org/x/text v0.34.0 // indirect - golang.org/x/time v0.13.0 // indirect - golang.org/x/tools v0.41.0 // indirect - google.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect - google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 // indirect - google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c // indirect - google.golang.org/grpc v1.75.0 // indirect - google.golang.org/protobuf v1.36.11 // indirect + golang.org/x/mod v0.33.0 // indirect + golang.org/x/net v0.52.0 // indirect + golang.org/x/sync v0.20.0 // indirect + golang.org/x/sys v0.42.0 // indirect + golang.org/x/term v0.41.0 // indirect + golang.org/x/text v0.35.0 // indirect + golang.org/x/time v0.14.0 // indirect + golang.org/x/tools v0.42.0 // indirect + google.golang.org/genproto v0.0.0-20260128011058-8636f8732409 // indirect + google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20260401024825-9d38bb4040a9 // indirect + google.golang.org/grpc v1.80.0 // indirect + google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af // indirect gopkg.in/yaml.v2 v2.4.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect k8s.io/component-base v0.35.0 // indirect diff --git a/sidecar/go.sum b/sidecar/go.sum index 280bd8b5..fd956b64 100644 --- a/sidecar/go.sum +++ b/sidecar/go.sum @@ -697,58 +697,58 @@ github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQ github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g= github.com/aws/aws-sdk-go-v2 v1.21.2/go.mod h1:ErQhvNuEMhJjweavOYhxVkn2RUx7kQXVATHrjKtxIpM= -github.com/aws/aws-sdk-go-v2 v1.43.5 h1:yKT5GYnFWhuDo+DqKvE5ZPwVn3RjC4MAeBtZGlh6AVM= -github.com/aws/aws-sdk-go-v2 v1.43.5/go.mod h1:wZjAJppCntyOGgVSmgVTfDyRJK5PHOasO6Wsy8U7Axk= -github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.17 h1:mn+Vxb9zgz/FE/yDTcFim3DZ1qpcrxR+qBQkBrl6bzA= -github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.17/go.mod h1:eDfmEFxu+BSVsUGLbzJhWjpOurv1mqczClS97yI8wdk= +github.com/aws/aws-sdk-go-v2 v1.41.6 h1:1AX0AthnBQzMx1vbmir3Y4WsnJgiydmnJjiLu+LvXOg= +github.com/aws/aws-sdk-go-v2 v1.41.6/go.mod h1:dy0UzBIfwSeot4grGvY1AqFWN5zgziMmWGzysDnHFcQ= +github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 h1:eBMB84YGghSocM7PsjmmPffTa+1FBUeNvGvFou6V/4o= +github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8/go.mod h1:lyw7GFp3qENLh7kwzf7iMzAxDn+NzjXEAGjKS2UOKqI= github.com/aws/aws-sdk-go-v2/config v1.18.45/go.mod h1:ZwDUgFnQgsazQTnWfeLWk5GjeqTQTL8lMkoE1UXzxdE= github.com/aws/aws-sdk-go-v2/config v1.32.12 h1:O3csC7HUGn2895eNrLytOJQdoL2xyJy0iYXhoZ1OmP0= github.com/aws/aws-sdk-go-v2/config v1.32.12/go.mod h1:96zTvoOFR4FURjI+/5wY1vc1ABceROO4lWgWJuxgy0g= github.com/aws/aws-sdk-go-v2/credentials v1.13.43/go.mod h1:zWJBz1Yf1ZtX5NGax9ZdNjhhI4rgjfgsyk6vTY1yfVg= -github.com/aws/aws-sdk-go-v2/credentials v1.19.35 h1:Cxua2RVdRwL0sfjHM/SnQoOnQ7xKng9m5EQBO8BnZlg= -github.com/aws/aws-sdk-go-v2/credentials v1.19.35/go.mod h1:9XQ+RSIGPkycr+oCJYnB1uTv5kMVVR+rd2vYK0Hxj2w= +github.com/aws/aws-sdk-go-v2/credentials v1.19.12 h1:oqtA6v+y5fZg//tcTWahyN9PEn5eDU/Wpvc2+kJ4aY8= +github.com/aws/aws-sdk-go-v2/credentials v1.19.12/go.mod h1:U3R1RtSHx6NB0DvEQFGyf/0sbrpJrluENHdPy1j/3TE= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.13/go.mod h1:f/Ib/qYjhV2/qdsf79H3QP/eRE4AkVyEf6sk7XfZ1tg= -github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36 h1:gucL1KH/PAYbpTpBg09CiVpBdTu4qkCl8C7xOTBixUg= -github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36/go.mod h1:usTB+PHhNMhrx2dxUeHcM7OrT5pySvmjYI++IsefPN0= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20 h1:zOgq3uezl5nznfoK3ODuqbhVg1JzAGDUhXOsU0IDCAo= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20/go.mod h1:z/MVwUARehy6GAg/yQ1GO2IMl0k++cu1ohP9zo887wE= github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.1.10 h1:2KCL4TmeiNvpPedtC4Bey5jvjRLD74WUYqGeHJ//aco= github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.1.10/go.mod h1:KwaiUFVO7pG8Z9F5bMGvvrRibdSDaAu8HtlKGKkjZSA= github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.43/go.mod h1:auo+PiyLl0n1l8A0e8RIeR8tOzYPfZZH/JNlrJ8igTQ= -github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36 h1:5CrzwxDqf4w3x1Vs3/NiZ0nsC34Hbm3pIDMWbsLebOE= -github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.36/go.mod h1:A3gHdKZIvG/QXERzZwcxNS3RNDFcRCuhhTFBYp+V/nw= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 h1:GmLa5Kw1ESqtFpXsx5MmC84QWa/ZrLZvlJGa2y+4kcQ= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22/go.mod h1:6sW9iWm9DK9YRpRGga/qzrzNLgKpT2cIxb7Vo2eNOp0= github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.37/go.mod h1:Qe+2KtKml+FEsQF/DHmDV+xjtche/hwoF75EG4UlHW8= -github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36 h1:A4N2f4YPcST0v+dWtX+xrpPPCL9VTBhoIFFUWYqbacE= -github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.36/go.mod h1:B/Qr859uxWUEfZeGotK5KAEoof4Q9YWgNtPSwV6jcyk= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22 h1:dY4kWZiSaXIzxnKlj17nHnBcXXBfac6UlsAx2qL6XrU= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22/go.mod h1:KIpEUx0JuRZLO7U6cbV204cWAEco2iC3l061IxlwLtI= github.com/aws/aws-sdk-go-v2/internal/ini v1.3.45/go.mod h1:lD5M20o09/LCuQ2mE62Mb/iSdSlCNuj6H5ci7tW7OsE= github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 h1:qYQ4pzQ2Oz6WpQ8T3HvGHnZydA72MnLuFK9tJwmrbHw= github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6/go.mod h1:O3h0IK87yXci+kg6flUKzJnWeziQUKciKrLjcatSNcY= -github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37 h1:oyd3ke4V9AhKcRR7rRgxk1VyI+DjK2CBQtbxh3OkdaA= -github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.37/go.mod h1:aA9D7SqfG9IC1b7FLD7Iyc8Q4JN0a8gHhNjN4zPlIaI= -github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.16 h1:iE4NGbvqUZnHDqddQAauZzCILYtFjOHwRM5MOOKLB5A= -github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.16/go.mod h1:VsjEgrP+ibcou8TlWA4tYaB+0OojuhirsmCe+U60hTA= -github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.29 h1:E65Hj648dOV6FuUfI0mYXXhQRHbsi7n+B9h6fZPJO/E= -github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.29/go.mod h1:xLrF9yNTCs92VZSpdEd68EJbgcdw3SMR74RO6QDzWHE= +github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21 h1:SwGMTMLIlvDNyhMteQ6r8IJSBPlRdXX5d4idhIGbkXA= +github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21/go.mod h1:UUxgWxofmOdAMuqEsSppbDtGKLfR04HGsD0HXzvhI1k= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 h1:5EniKhLZe4xzL7a+fU3C2tfUN4nWIqlLesfrjkuPFTY= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7/go.mod h1:x0nZssQ3qZSnIcePWLvcoFisRXJzcTVvYpAAdYX8+GI= +github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12 h1:qtJZ70afD3ISKWnoX3xB0J2otEqu3LqicRcDBqsj0hQ= +github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12/go.mod h1:v2pNpJbRNl4vEUWEh5ytQok0zACAKfdmKS51Hotc3pQ= github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.37/go.mod h1:vBmDnwWXWxNPFRMmG2m/3MKOe+xEcMDo1tanpaWCcck= -github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.36 h1:fx2ujmozWn+C/GtfXfz5k6Ckzza40ElOpIW7d92fLWQ= -github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.36/go.mod h1:QT2ufGVJ+xTRxtXPHTQ1kHkAdWIKPCmD+BqYAXWv8/4= -github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.37 h1:KGHa9iZCrgtkOsFfXb0S4ywsjostA/hau7WE9aSb43E= -github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.37/go.mod h1:FV79f0DSnZIEGsQjWenENGtUycrasyAaJZO+zRanLHA= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20 h1:2HvVAIq+YqgGotK6EkMf+KIEqTISmTYh5zLpYyeTo1Y= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20/go.mod h1:V4X406Y666khGa8ghKmphma/7C0DAtEQYhkq9z4vpbk= +github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20 h1:siU1A6xjUZ2N8zjTHSXFhB9L/2OY8Dqs0xXiLjF30jA= +github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20/go.mod h1:4TLZCmVJDM3FOu5P5TJP0zOlu9zWgDWU7aUxWbr+rcw= github.com/aws/aws-sdk-go-v2/service/route53 v1.30.2/go.mod h1:TQZBt/WaQy+zTHoW++rnl8JBrmZ0VO6EUbVua1+foCA= github.com/aws/aws-sdk-go-v2/service/s3 v1.97.2 h1:MRNiP6nqa20aEl8fQ6PJpEq11b2d40b16sm4WD7QgMU= github.com/aws/aws-sdk-go-v2/service/s3 v1.97.2/go.mod h1:FrNA56srbsr3WShiaelyWYEo70x80mXnVZ17ZZfbeqg= -github.com/aws/aws-sdk-go-v2/service/signin v1.5.5 h1:0VTFBfOgPJrUSpGMgzoi8qLcXF5dbmiBuxpo14eBWUw= -github.com/aws/aws-sdk-go-v2/service/signin v1.5.5/go.mod h1:sNZYlBxoohYMBYl47BO/bFtAM6I8HSsPa1qwwPPRGoQ= +github.com/aws/aws-sdk-go-v2/service/signin v1.0.8 h1:0GFOLzEbOyZABS3PhYfBIx2rNBACYcKty+XGkTgw1ow= +github.com/aws/aws-sdk-go-v2/service/signin v1.0.8/go.mod h1:LXypKvk85AROkKhOG6/YEcHFPoX+prKTowKnVdcaIxE= github.com/aws/aws-sdk-go-v2/service/sso v1.15.2/go.mod h1:gsL4keucRCgW+xA85ALBpRFfdSLH4kHOVSnLMSuBECo= -github.com/aws/aws-sdk-go-v2/service/sso v1.33.5 h1:jDQARFp1mJ2PEnllQf01nfFXGfWMJ59e0/HCHUTTZCk= -github.com/aws/aws-sdk-go-v2/service/sso v1.33.5/go.mod h1:OcT2AhgTuxGAwZk5hgxaNLGpS33W8s8dUQadGVDVY9I= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.13 h1:kiIDLZ005EcKomYYITtfsjn7dtOwHDOFy7IbPXKek2o= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.13/go.mod h1:2h/xGEowcW/g38g06g3KpRWDlT+OTfxxI0o1KqayAB8= github.com/aws/aws-sdk-go-v2/service/ssooidc v1.17.3/go.mod h1:a7bHA82fyUXOm+ZSWKU6PIoBxrjSprdLoM8xPYvzYVg= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.5 h1:8xo1q9ttkYqMJ6vOXX67FPSpVEI7BWKVTKh77g82w+8= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.5/go.mod h1:hbBeEUrZg6VddXYZpbKPyF0tl4XEnM+Dbx92RW3vmZI= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17 h1:jzKAXIlhZhJbnYwHbvUQZEB8KfgAEuG0dc08Bkda7NU= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17/go.mod h1:Al9fFsXjv4KfbzQHGe6V4NZSZQXecFcvaIF4e70FoRA= github.com/aws/aws-sdk-go-v2/service/sts v1.23.2/go.mod h1:Eows6e1uQEsc4ZaHANmsPRzAKcVDrcmjjWiih2+HUUQ= -github.com/aws/aws-sdk-go-v2/service/sts v1.45.5 h1:eQ5BtXDrPg2wK0AjtVPzeBhUpYPeqHE/ptiH7xJRGek= -github.com/aws/aws-sdk-go-v2/service/sts v1.45.5/go.mod h1:f9ImhnOISY7BuTZLM8qHepCYnglHBVLk5wVzatmP++w= +github.com/aws/aws-sdk-go-v2/service/sts v1.41.9 h1:Cng+OOwCHmFljXIxpEVXAGMnBia8MSU6Ch5i9PgBkcU= +github.com/aws/aws-sdk-go-v2/service/sts v1.41.9/go.mod h1:LrlIndBDdjA/EeXeyNBle+gyCwTlizzW5ycgWnvIxkk= github.com/aws/smithy-go v1.15.0/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA= -github.com/aws/smithy-go v1.27.7 h1:Zgj5z4LfcDYoQIVk+n/yGdTkP/2y6ZT5vYxe0fp7bqE= -github.com/aws/smithy-go v1.27.7/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc= +github.com/aws/smithy-go v1.25.0 h1:Sz/XJ64rwuiKtB6j98nDIPyYrV1nVNJ4YU74gttcl5U= +github.com/aws/smithy-go v1.25.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc= github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4= github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/benbjohnson/immutable v0.4.3 h1:GYHcksoJ9K6HyAUpGxwZURrbTkXA0Dh4otXGqbhdrjA= @@ -1287,8 +1287,8 @@ github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoA github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= -github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= -github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5THxAzdVpqr6/geYxZytqFMBCOtn/ujyeo= +github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA= github.com/grafana/pyroscope-go/godeltaprof v0.1.8 h1:iwOtYXeeVSAeYefJNaxDytgjKtUuKQbJqgAIjlnicKg= github.com/grafana/pyroscope-go/godeltaprof v0.1.8/go.mod h1:2+l7K7twW49Ct4wFluZD3tZ6e0SjanjcUUBPVD/UuGU= github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc h1:GN2Lv3MGO7AS6PrRoT6yV5+wkrOpcszoIsO4+4ds248= @@ -1737,8 +1737,8 @@ github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB8 github.com/prometheus/common v0.15.0/go.mod h1:U+gB1OBLb1lF3O42bTCL+FK18tX9Oar16Clt/msog/s= github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= -github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= -github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4= +github.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw= github.com/prometheus/otlptranslator v0.0.2 h1:+1CdeLVrRQ6Psmhnobldo0kTp96Rj80DRXRd5OSnMEQ= github.com/prometheus/otlptranslator v0.0.2/go.mod h1:P8AwMgdD7XEr6QRUJ2QWLpiAZTgTE2UYgjlu3svompI= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= @@ -2042,20 +2042,20 @@ go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= -go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48= -go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8= +go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I= +go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0= go.opentelemetry.io/otel/exporters/jaeger v1.9.0 h1:gAEgEVGDWwFjcis9jJTOJqZNxDzoZfR12WNIxr7g9Ww= go.opentelemetry.io/otel/exporters/jaeger v1.9.0/go.mod h1:hquezOLVAybNW6vanIxkdLXTXvzlj2Vn3wevSP15RYs= go.opentelemetry.io/otel/exporters/prometheus v0.60.0 h1:cGtQxGvZbnrWdC2GyjZi0PDKVSLWP/Jocix3QWfXtbo= go.opentelemetry.io/otel/exporters/prometheus v0.60.0/go.mod h1:hkd1EekxNo69PTV4OWFGZcKQiIqg0RfuWExcPKFvepk= -go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0= -go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs= -go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18= -go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE= -go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8= -go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew= -go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI= -go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA= +go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM= +go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY= +go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg= +go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg= +go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw= +go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A= +go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A= +go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0= go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= go.opentelemetry.io/proto/otlp v0.15.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U= go.opentelemetry.io/proto/otlp v0.19.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U= @@ -2080,8 +2080,8 @@ go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= go.uber.org/zap v1.18.1/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8= go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= -go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= -go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= +go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ= +go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= @@ -2155,8 +2155,8 @@ golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= golang.org/x/mod v0.22.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY= -golang.org/x/mod v0.32.0 h1:9F4d3PHLljb6x//jOyokMv3eX+YDeepZSEo3mFJy93c= -golang.org/x/mod v0.32.0/go.mod h1:SgipZ/3h2Ci89DlEtEXWUk/HteuRin+HHhN+WbNhguU= +golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8= +golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w= golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -2242,8 +2242,8 @@ golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg= golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM= golang.org/x/net v0.34.0/go.mod h1:di0qlW3YNM5oh6GqDGQr92MyTozJPmybPK4Ev/Gm31k= golang.org/x/net v0.36.0/go.mod h1:bFmbeoIPfrw4sMHNhb4J9f6+tPziuGjq7Jk/38fxi1I= -golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o= -golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8= +golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0= +golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw= golang.org/x/oauth2 v0.0.0-20170207211851-4464e7848382/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -2296,8 +2296,8 @@ golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= -golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= -golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= +golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -2438,12 +2438,12 @@ golang.org/x/sys v0.24.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/sys v0.40.0 h1:DBZZqJ2Rkml6QMQsZywtnjnnGvHza6BTfYFWY9kjEWQ= -golang.org/x/sys v0.40.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo= +golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE= golang.org/x/telemetry v0.0.0-20240521205824-bda55230c457/go.mod h1:pRgIJT+bRLFKnoM1ldnzKoxTIn14Yxz928LQRYYgIN0= -golang.org/x/telemetry v0.0.0-20260109210033-bd525da824e2 h1:O1cMQHRfwNpDfDJerqRoE2oD+AFlyid87D40L/OkkJo= -golang.org/x/telemetry v0.0.0-20260109210033-bd525da824e2/go.mod h1:b7fPSJ0pKZ3ccUh8gnTONJxhn3c/PS6tyzQvyqw4iA8= +golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4 h1:bTLqdHv7xrGlFbvf5/TXNxy/iUwwdkjhqQTJDjW7aj0= +golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4/go.mod h1:g5NllXBEermZrmR51cJDQxmJUHUOfRAaNyWBM+R+548= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= @@ -2463,8 +2463,8 @@ golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY= golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM= golang.org/x/term v0.28.0/go.mod h1:Sw/lC2IAUZ92udQNf3WodGtn4k/XoLyZoh8v/8uiwek= golang.org/x/term v0.29.0/go.mod h1:6bl4lRlvVuDgSf3179VpIxBF0o10JUpXWOnI7nErv7s= -golang.org/x/term v0.39.0 h1:RclSuaJf32jOqZz74CkPA9qFuVTX7vhLlpfj/IGWlqY= -golang.org/x/term v0.39.0/go.mod h1:yxzUCTP/U+FzoxfdKmLaA0RV1WgE0VY7hXBwKtY/4ww= +golang.org/x/term v0.41.0 h1:QCgPso/Q3RTJx2Th4bDLqML4W6iJiaXFq2/ftQF13YU= +golang.org/x/term v0.41.0/go.mod h1:3pfBgksrReYfZ5lvYM0kSO0LIkAl4Yl2bXOkKP7Ec2A= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -2488,8 +2488,8 @@ golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY= -golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk= -golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA= +golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8= +golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA= golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -2501,8 +2501,8 @@ golang.org/x/time v0.1.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.2.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.9.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= -golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= -golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= +golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= +golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= golang.org/x/tools v0.0.0-20180525024113-a5b4c53f6e8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190206041539-40960b6deb8e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= @@ -2581,8 +2581,8 @@ golang.org/x/tools v0.11.0/go.mod h1:anzJrxPjNtfgiYQYirP2CPGzGLxrH2u2QBhn6Bf3qY8 golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58= golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= golang.org/x/tools v0.29.0/go.mod h1:KMQVMRsVxU6nHCFXrBPhDB8XncLNLM0lIy/F14RP588= -golang.org/x/tools v0.41.0 h1:a9b8iMweWG+S0OBnlU36rzLp20z1Rp10w+IY2czHTQc= -golang.org/x/tools v0.41.0/go.mod h1:XSY6eDqxVNiYgezAVqqCeihT4j1U2CCsqvH3WhQpnlg= +golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k= +golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -2809,16 +2809,16 @@ google.golang.org/genproto v0.0.0-20230331144136-dcfb400f0633/go.mod h1:UUQDJDOl google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1/go.mod h1:nKE/iIaLqn2bQwXBg8f1g2Ylh6r5MN5CmZvuzZCgsCU= google.golang.org/genproto v0.0.0-20230525234025-438c736192d0/go.mod h1:9ExIQyXL5hZrHzQceCwuSYwZZ5QZBazOcprJ5rgs3lY= google.golang.org/genproto v0.0.0-20230526161137-0005af68ea54/go.mod h1:zqTuNwFlFRsw5zIts5VnzLQxSRqh+CGOTVMlYbY0Eyk= -google.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4= -google.golang.org/genproto v0.0.0-20250603155806-513f23925822/go.mod h1:HubltRL7rMh0LfnQPkMH4NPDFEWp0jw3vixw7jEM53s= +google.golang.org/genproto v0.0.0-20260128011058-8636f8732409 h1:VQZ/yAbAtjkHgH80teYd2em3xtIkkHd7ZhqfH2N9CsM= +google.golang.org/genproto v0.0.0-20260128011058-8636f8732409/go.mod h1:rxKD3IEILWEu3P44seeNOAwZN4SaoKaQ/2eTg4mM6EM= google.golang.org/genproto/googleapis/api v0.0.0-20230525234020-1aefcd67740a/go.mod h1:ts19tUU+Z0ZShN1y3aPyq2+O3d5FUNNgT6FtOzmrNn8= google.golang.org/genproto/googleapis/api v0.0.0-20230525234035-dd9d682886f9/go.mod h1:vHYtlOoi6TsQ3Uk2yxR7NI5z8uoV+3pZtR4jmHIkRig= -google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 h1:FiusG7LWj+4byqhbvmB+Q93B/mOxJLN2DTozDuZm4EU= -google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7/go.mod h1:kXqgZtrWaf6qS3jZOCnCH7WYfrvFjkC51bM8fz3RsCA= +google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 h1:VPWxll4HlMw1Vs/qXtN7BvhZqsS9cdAittCNvVENElA= +google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9/go.mod h1:7QBABkRtR8z+TEnmXTqIqwJLlzrZKVfAUm7tY3yGv0M= google.golang.org/genproto/googleapis/rpc v0.0.0-20230525234015-3fc162c6f38a/go.mod h1:xURIpW9ES5+/GZhnV6beoEtxQrnkRGIfP5VQG2tCBLc= google.golang.org/genproto/googleapis/rpc v0.0.0-20230525234030-28d5490b6b19/go.mod h1:66JfowdXAEgad5O9NnYcsNPLCPZJD++2L9X0PCMODrA= -google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c h1:qXWI/sQtv5UKboZ/zUk7h+mrf/lXORyI+n9DKDAusdg= -google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c/go.mod h1:gw1tLEfykwDz2ET4a12jcXt4couGAm7IwsVaTy0Sflo= +google.golang.org/genproto/googleapis/rpc v0.0.0-20260401024825-9d38bb4040a9 h1:m8qni9SQFH0tJc1X0vmnpw/0t+AImlSvp30sEupozUg= +google.golang.org/genproto/googleapis/rpc v0.0.0-20260401024825-9d38bb4040a9/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8= google.golang.org/grpc v1.57.1 h1:upNTNqv0ES+2ZOOqACwVtS3Il8M12/+Hz41RCPzAjQg= google.golang.org/grpc v1.57.1/go.mod h1:Sd+9RMTACXwmub0zcNY2c4arhtrbBYD1AUHI/dt16Mo= google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw= @@ -2842,8 +2842,8 @@ google.golang.org/protobuf v1.29.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqw google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw= -google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= -google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af h1:+5/Sw3GsDNlEmu7TfklWKPdQ0Ykja5VEmq2i817+jbI= +google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= diff --git a/sidecar/main.go b/sidecar/main.go index 976ad6d2..2f1b3cce 100644 --- a/sidecar/main.go +++ b/sidecar/main.go @@ -15,22 +15,13 @@ import ( "errors" "fmt" "os" + "path/filepath" "strings" "github.com/sei-protocol/seilog" "github.com/urfave/cli/v3" ) -// destinations holds flag-bound values the subcommands read. -// -// home binds SEI_HOME, which the controller sets to the node's data-PVC mount. -// It lives on the root command so every subcommand resolves the same directory, -// and losing that binding does not fail — it silently relocates every write off -// the PVC, which is why validateHome guards it. -var destinations = struct { - home string -}{} - func main() { cmd := &cli.Command{ Name: "sei-sidecar", @@ -39,25 +30,33 @@ func main() { &cli.StringFlag{ Name: "home", Sources: cli.EnvVars("SEI_HOME"), - // No Value: an unset home must fail, not default. The previous - // fallback was "/sei" while the controller mounts the data PVC - // at $HOME/.sei, so a dropped SEI_HOME produced a running, - // probe-passing sidecar writing genesis and config into an - // empty directory. Required turns that into a startup error. - Required: true, - Destination: &destinations.home, - TakesFile: true, - Config: cli.StringConfig{TrimSpace: true}, - Usage: "seid home directory (the node's data volume)", + // No Value: a wrong home is silent, because every path the + // sidecar writes is resolved against it. + Required: true, + TakesFile: true, + Config: cli.StringConfig{TrimSpace: true}, + Usage: "seid home directory (the node's data volume)", + // Validation belongs here, not in a root Before hook. urfave + // runs Before hooks ahead of its own required-flag check, so a + // hook would shadow that check and report an unset SEI_HOME as + // though it were set-but-empty. A flag Action runs only for + // flags that were actually set, which leaves the unset case to + // the required-flag check that describes it correctly. + Action: func(_ context.Context, cmd *cli.Command, home string) error { + if err := validateHome(home); err != nil { + return err + } + // Normalize before any subcommand reads it: two consumers + // take the value raw rather than through filepath.Join — + // tasks.NewSnapshotUploader's os.CreateTemp root and the + // gentx generator's cfg.SetRoot. + return cmd.Set("home", filepath.Clean(home)) + }, }, }, Commands: []*cli.Command{&serveCmd}, } - cmd.Before = func(ctx context.Context, _ *cli.Command) (context.Context, error) { - return ctx, validateHome(destinations.home) - } - if err := cmd.Run(context.Background(), os.Args); err != nil { fmt.Fprintf(os.Stderr, "error: %v\n", err) _ = seilog.Close() @@ -65,17 +64,22 @@ func main() { } } -// validateHome rejects a set-but-empty home directory. +// validateHome rejects a home directory that would silently relocate every +// write off the node's data volume. // -// The flag's Required only tests whether a value was supplied, and SEI_HOME="" -// supplies one. Without this check the sidecar starts and resolves every path -// relative to its working directory — measured, it creates config/, data/ and -// sidecar.db there and serves normally — which is the silent wrong-directory -// failure Required is there to prevent. +// Two shapes get past the flag's Required, which only tests that a value was +// supplied. An empty value resolves every path against the working directory, +// and so does any relative value — the harm is identical, so both are refused +// rather than only the one that prompted the check. func validateHome(home string) error { - if strings.TrimSpace(home) == "" { + trimmed := strings.TrimSpace(home) + if trimmed == "" { return errors.New("SEI_HOME is set but empty; it must name the node's data " + "volume, or every path resolves relative to the working directory") } + if !filepath.IsAbs(trimmed) { + return fmt.Errorf("SEI_HOME must be an absolute path; got %q, which resolves "+ + "relative to the working directory rather than the node's data volume", home) + } return nil } diff --git a/sidecar/serve.go b/sidecar/serve.go index f907a74e..e2e8b6ca 100644 --- a/sidecar/serve.go +++ b/sidecar/serve.go @@ -39,11 +39,7 @@ var serveCmd = cli.Command{ ctx, stop := signal.NotifyContext(ctx, syscall.SIGTERM, syscall.SIGINT) defer stop() - // destinations.home is bound to SEI_HOME on the root command and marked - // Required there, so an unset value fails before we reach this point. - // It used to fall back to "/sei" while the controller mounts the data - // PVC elsewhere, which made a dropped SEI_HOME silent. - homeDir := destinations.home + homeDir := cmd.String("home") port := cmd.String("port") chainID := os.Getenv("SEI_CHAIN_ID") genesisBucket := os.Getenv("SEI_GENESIS_BUCKET") diff --git a/sidecar/startup_guard_test.go b/sidecar/startup_guard_test.go index e3c60e6b..f8e01e33 100644 --- a/sidecar/startup_guard_test.go +++ b/sidecar/startup_guard_test.go @@ -87,36 +87,43 @@ func TestUnauthenticatedIsTheZeroValue(t *testing.T) { } } -// The flag's Required only checks that a value was supplied, and SEI_HOME="" -// supplies one. Before validateHome existed, an empty value started the sidecar -// and it created config/, data/ and sidecar.db relative to its working -// directory — a running, probe-passing sidecar operating off the data volume. +// Required only tests that a value was supplied. Both an empty value and a +// relative one resolve every path against the working directory instead of the +// node's data volume, and the sidecar serves normally either way — it creates +// config/, data/ and sidecar.db wherever it happens to be running. func TestValidateHome(t *testing.T) { cases := []struct { name string home string - wantErr bool + wantErr string }{ - {name: "empty is refused", home: "", wantErr: true}, - {name: "whitespace-only is refused", home: " ", wantErr: true}, - {name: "tab is refused", home: "\t", wantErr: true}, - {name: "a real path is accepted", home: "/home/nonroot/.sei", wantErr: false}, - {name: "a relative path is accepted — odd but the operator's choice", home: "sei", wantErr: false}, + {name: "empty", home: "", wantErr: "set but empty"}, + {name: "whitespace only", home: " ", wantErr: "set but empty"}, + {name: "tab only", home: "\t", wantErr: "set but empty"}, + // A relative path carries the same consequence as an empty one. + {name: "bare relative", home: "sei", wantErr: "absolute path"}, + {name: "dot relative", home: "./sei", wantErr: "absolute path"}, + {name: "parent relative", home: "../sei", wantErr: "absolute path"}, + {name: "absolute", home: "/home/nonroot/.sei"}, + {name: "absolute unnormalized", home: "/home/nonroot/.sei///"}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { err := validateHome(tc.home) - if tc.wantErr { - if err == nil { - t.Fatal("expected a refusal, got nil") - } - if !strings.Contains(err.Error(), "SEI_HOME") { - t.Errorf("error should name SEI_HOME; got %q", err.Error()) + if tc.wantErr == "" { + if err != nil { + t.Errorf("expected nil, got %v", err) } return } - if err != nil { - t.Errorf("expected nil, got %v", err) + if err == nil { + t.Fatalf("expected a refusal containing %q, got nil", tc.wantErr) + } + if !strings.Contains(err.Error(), tc.wantErr) { + t.Errorf("error: got %q, want substring %q", err.Error(), tc.wantErr) + } + if !strings.Contains(err.Error(), "SEI_HOME") { + t.Errorf("error should name SEI_HOME; got %q", err.Error()) } }) } From c1f9f3f251af1f3c8d283a6c2eef5ba7f5acbfa2 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Fri, 14 Aug 2026 14:30:05 -0700 Subject: [PATCH 10/13] ci(lint): restore the diff filter, and scope the relocated module's debt MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three lint legs were red for two different reasons, and only one of them was the known-open sidecar backlog. `lint (.)` reported 82 findings in files this branch never touches — internal/planner/validator_test.go, internal/controller/node/peers_test.go and others. That is the root module's pre-existing v2.12.1 backlog, the one only-new-issues exists to hold back. The filter diffs against the merge base, which a shallow clone does not contain, so it silently passed everything through; the rebase and force-push are what exposed it. `fetch-depth: 0` on the lint checkout restores it. Same cause for the single sidecarapi finding. `sidecar/` is the real backlog. It arrived from a repo carrying no golangci-lint config, so none of it had ever been linted, and only-new-issues cannot help because every file is new here. Scoped in .golangci.yml rather than left red: a permanently failing check trains people to ignore it, and an exclusion is greppable and reviewable. lll and dupl match what internal/, api/, sdk/ and sidecarapi/ already get. Test files get the set that fires almost exclusively there — bodyclose is 26-for-26 in _test.go, staticcheck 3-for-3. noctx is scoped to sidecar/engine/sqlite_{migrations,store}.go alone, so it stays live across the rest of the module. Those 27 findings want the database/sql *Context variants throughout the store, which is a real improvement and a behaviour change: it makes queries cancellable in the component whose SetMaxOpenConns(1) and WAL-checkpoint invariants are the ones to be careful with. That belongs in its own PR, not inside a 24,000-line relocation. Fixed rather than excluded, since each is zero-risk: ten errcheck findings were the deliberate-ignore Close and Rollback pattern, now written as `_ =` so the intent is on the page — a Rollback after Commit is a no-op and the Close paths are already failing. The em dash repeated across five table cells in shadow/render.go is now `emptyCell`, which says what it means. Excluded with the reason on the page: unparam flags cfg and cdc parameters a few unexported methods do not read, kept for symmetry with sibling methods that do; prealloc's last hit is a json.Unmarshal destination, which Unmarshal allocates itself. Verified: sidecar and sidecarapi both report 0 issues locally. Note the local linter is v2.8.0 and CI pins v2.12.1, which finds strictly more — goconst 182 versus 8 on the same tree — so CI is the authority on whether the exclusions are complete. Builds, gofmt, tidy-check, verify-generated and all 25 test packages across three modules unchanged. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 6 ++++++ .golangci.yml | 30 +++++++++++++++++++++++++++++ sidecar/engine/sqlite_migrations.go | 10 +++++----- sidecar/engine/sqlite_store.go | 6 +++--- sidecar/shadow/fetch.go | 4 ++-- sidecar/shadow/render.go | 13 ++++++++----- 6 files changed, 54 insertions(+), 15 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index eae083f9..e150d044 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -25,6 +25,12 @@ jobs: module: [".", "sidecarapi", "sidecar"] steps: - uses: actions/checkout@v4 + with: + # only-new-issues below diffs against the merge base, which a shallow + # clone does not contain — the filter then silently passes and the + # module's whole pre-existing backlog is reported as new. A rebase or + # force-push is what exposes it. + fetch-depth: 0 - uses: actions/setup-go@v5 with: go-version-file: go.mod diff --git a/.golangci.yml b/.golangci.yml index 0f26bc92..3b7dcdef 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -89,6 +89,36 @@ linters: - linters: - goconst path: sidecarapi/client/* + # sidecar/ arrived from a repo with no golangci-lint config, so none of it + # has ever been linted. Everything below is pre-existing debt made visible + # by the move, not new code — and only-new-issues cannot help, because every + # file is new to this repo. Scoped here rather than left red: a permanently + # failing check trains people to ignore it, and this way the debt is + # greppable and reviewable. + # + # lll and dupl match what internal/, api/, sdk/ and sidecarapi/ already get. + - linters: [dupl, lll] + path: sidecar/* + # Test code carries different standards than the binary. bodyclose and + # staticcheck fire only in _test.go here, and errcheck almost entirely. + - linters: [bodyclose, errcheck, goconst, gocyclo, noctx, prealloc, staticcheck, unparam] + path: sidecar/.*_test\.go + # unparam flags cfg/cdc parameters that a few unexported methods do not + # read. They are there for symmetry across a family of sibling methods that + # do, so dropping them from some and not others makes the family harder to + # read — and changing signatures is churn inside a relocation. prealloc's + # one remaining hit is a json.Unmarshal destination, which Unmarshal + # allocates itself, so preallocating changes nothing. + - linters: [prealloc, unparam] + path: sidecar/* + # noctx wants the database/sql *Context variants throughout the store. That + # is a real improvement and a behaviour change — it makes queries + # cancellable in the component whose SetMaxOpenConns(1) and WAL-checkpoint + # invariants are the ones to be careful with — so it gets its own PR rather + # than riding along inside a 24,000-line relocation. Scoped to the two files + # that hold it so noctx stays live everywhere else in the module. + - linters: [noctx] + path: sidecar/engine/sqlite_(migrations|store)\.go paths: - third_party$ - builtin$ diff --git a/sidecar/engine/sqlite_migrations.go b/sidecar/engine/sqlite_migrations.go index 2db1f2e0..f75cc4f6 100644 --- a/sidecar/engine/sqlite_migrations.go +++ b/sidecar/engine/sqlite_migrations.go @@ -13,7 +13,7 @@ func migrate(db *sql.DB) error { if err != nil { return err } - defer tx.Rollback() + defer func() { _ = tx.Rollback() }() if _, err := tx.Exec(` CREATE TABLE IF NOT EXISTS task_results ( @@ -49,7 +49,7 @@ func migrate(db *sql.DB) error { if err != nil { return err } - defer tx.Rollback() + defer func() { _ = tx.Rollback() }() if _, err := tx.Exec(` DROP INDEX IF EXISTS idx_task_results_schedule; @@ -73,7 +73,7 @@ func migrate(db *sql.DB) error { if err != nil { return err } - defer tx.Rollback() + defer func() { _ = tx.Rollback() }() if _, err := tx.Exec(` ALTER TABLE task_results ADD COLUMN run INTEGER NOT NULL DEFAULT 1; @@ -95,7 +95,7 @@ func migrate(db *sql.DB) error { if err != nil { return err } - defer tx.Rollback() + defer func() { _ = tx.Rollback() }() // result holds a handler's structured output as raw JSON; NULL // for the common case of a handler that emits no result. @@ -119,7 +119,7 @@ func migrate(db *sql.DB) error { if err != nil { return err } - defer tx.Rollback() + defer func() { _ = tx.Rollback() }() // tx_markers: pre-broadcast signed-tx bytes so a crashed sign-tx task // re-adopts the identical tx instead of re-signing. Engine-owned. A diff --git a/sidecar/engine/sqlite_store.go b/sidecar/engine/sqlite_store.go index bfb84b0d..c492d8ca 100644 --- a/sidecar/engine/sqlite_store.go +++ b/sidecar/engine/sqlite_store.go @@ -49,13 +49,13 @@ func openStore(dsn string) (*SQLiteStore, error) { "PRAGMA synchronous=NORMAL", } { if _, err := db.Exec(pragma); err != nil { - db.Close() + _ = db.Close() return nil, fmt.Errorf("%s: %w", pragma, err) } } if err := migrate(db); err != nil { - db.Close() + _ = db.Close() return nil, fmt.Errorf("migrate: %w", err) } @@ -192,7 +192,7 @@ func (s *SQLiteStore) queryMany(query string, args ...any) ([]TaskResult, error) if err != nil { return nil, err } - defer rows.Close() + defer func() { _ = rows.Close() }() var results []TaskResult for rows.Next() { diff --git a/sidecar/shadow/fetch.go b/sidecar/shadow/fetch.go index 4a95efaa..42e424d2 100644 --- a/sidecar/shadow/fetch.go +++ b/sidecar/shadow/fetch.go @@ -22,7 +22,7 @@ func FetchReport(ctx context.Context, downloader seis3.Downloader, bucket, key s if err != nil { return nil, fmt.Errorf("downloading s3://%s/%s: %w", bucket, key, err) } - defer resp.Body.Close() + defer func() { _ = resp.Body.Close() }() var reader io.Reader = resp.Body if strings.HasSuffix(key, ".gz") { @@ -30,7 +30,7 @@ func FetchReport(ctx context.Context, downloader seis3.Downloader, bucket, key s if err != nil { return nil, fmt.Errorf("decompressing report: %w", err) } - defer gz.Close() + defer func() { _ = gz.Close() }() reader = gz } diff --git a/sidecar/shadow/render.go b/sidecar/shadow/render.go index 6040e086..d3a6305c 100644 --- a/sidecar/shadow/render.go +++ b/sidecar/shadow/render.go @@ -5,6 +5,9 @@ import ( "strings" ) +// emptyCell is what a table cell shows when the value is absent. +const emptyCell = "—" + // RenderMarkdown produces a human-readable investigation report from a // DivergenceReport. The output is designed to be consumed by engineers // or LLM agents analyzing why a shadow node diverged from the canonical chain. @@ -52,16 +55,16 @@ func writeL0Row(b *strings.Builder, field string, match bool, shadow, canonical s := truncateHash(shadow) c := truncateHash(canonical) if match { - s = "—" - c = "—" + s = emptyCell + c = emptyCell } fmt.Fprintf(b, "| %s | %s | %s | %s |\n", field, s, c, icon) } func writeL0GasRow(b *strings.Builder, l0 Layer0Result) { icon := "✅" - s := "—" - c := "—" + s := emptyCell + c := emptyCell if !l0.GasUsedMatch { icon = "❌" s = fmt.Sprintf("%d", l0.ShadowGasUsed) @@ -107,7 +110,7 @@ func writeLayer2(b *strings.Builder, l2 *Layer2Result) { for _, d := range l2.Divergences { slot := d.Slot if slot == "" { - slot = "—" + slot = emptyCell } fmt.Fprintf(b, "| %s | %s | %s | %s | %s |\n", d.Kind, truncateHash(d.Addr), truncateHash(slot), From 36ee9161e1dcf1955332dcdd96ace8cbd9f1de35 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Fri, 14 Aug 2026 14:36:54 -0700 Subject: [PATCH 11/13] test(sidecar): drive the home guard through the real CLI wiring MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bugbot argued the home validation never runs in production: it lives in the --home flag's Action, and nothing passes that flag — the controller sets SEI_HOME and the image runs `sei-sidecar serve` — so a non-local parent flag Action might not fire for an environment-sourced value on a subcommand. Measured against the built binary in exactly that shape, it does fire: empty, whitespace, "sei" and "./sei" are all refused with no files created, and an unset variable reports the required-flag error rather than claiming emptiness. So the claim is wrong. It was still worth a test, because the reason nobody could answer it quickly is that no test crossed the CLI wiring — TestValidateHome calls the function directly. TestHomeValidationReachesEnvSourcedValues now drives the real cli.Command with SEI_HOME from the environment and a no-op subcommand, asserting both the message and that the subcommand never ran. main() and the test build the root command from one newRootCommand, so the test cannot drift from what production wires. Also fixes the goconst that lint (sidecarapi) reported: the nested-merge case in tomlpatch repeated a map key across its three maps, and the key being identical in all three is the whole point of the case, so it is named rather than repeated. That finding was real and new — tomlpatch arrives in this PR, so only-new-issues correctly reports it. Verified: sidecar and sidecarapi both 0 lint issues locally; builds, vet, gofmt, goimports, tidy-check, verify-generated and all 25 test packages clean. Co-Authored-By: Claude Opus 5 (1M context) --- sidecar/main.go | 22 +++++++----- sidecar/startup_guard_test.go | 54 ++++++++++++++++++++++++++++++ sidecarapi/tomlpatch/merge_test.go | 11 ++++-- 3 files changed, 76 insertions(+), 11 deletions(-) diff --git a/sidecar/main.go b/sidecar/main.go index 2f1b3cce..0dc3db5d 100644 --- a/sidecar/main.go +++ b/sidecar/main.go @@ -23,7 +23,20 @@ import ( ) func main() { - cmd := &cli.Command{ + cmd := newRootCommand() + cmd.Commands = []*cli.Command{&serveCmd} + + if err := cmd.Run(context.Background(), os.Args); err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + _ = seilog.Close() + os.Exit(1) + } +} + +// newRootCommand builds the root command without its subcommands, so a test can +// drive the same flag wiring production runs. +func newRootCommand() *cli.Command { + return &cli.Command{ Name: "sei-sidecar", Usage: "Sei node sidecar: task executor and HTTP API", Flags: []cli.Flag{ @@ -54,13 +67,6 @@ func main() { }, }, }, - Commands: []*cli.Command{&serveCmd}, - } - - if err := cmd.Run(context.Background(), os.Args); err != nil { - fmt.Fprintf(os.Stderr, "error: %v\n", err) - _ = seilog.Close() - os.Exit(1) } } diff --git a/sidecar/startup_guard_test.go b/sidecar/startup_guard_test.go index f8e01e33..1d9e75d8 100644 --- a/sidecar/startup_guard_test.go +++ b/sidecar/startup_guard_test.go @@ -1,9 +1,13 @@ package main import ( + "context" + "os" "strings" "testing" + "github.com/urfave/cli/v3" + "github.com/sei-protocol/sei-k8s-controller/sidecar/server" ) @@ -128,3 +132,53 @@ func TestValidateHome(t *testing.T) { }) } } + +// validateHome is wired into the --home flag's Action, and in production nothing +// passes that flag: the controller sets SEI_HOME and the image runs +// `sei-sidecar serve`. A unit test on validateHome alone cannot tell whether the +// wiring reaches an environment-sourced value on a subcommand, which is the only +// path that runs in a pod. This drives the real cli.Command to find out. +func TestHomeValidationReachesEnvSourcedValues(t *testing.T) { + cases := []struct { + name string + home string + set bool + wantErr string + }{ + {name: "unset reports the required flag, not emptiness", set: false, wantErr: `Required flag "home" not set`}, + {name: "empty", home: "", set: true, wantErr: "set but empty"}, + {name: "whitespace only", home: " ", set: true, wantErr: "set but empty"}, + {name: "relative", home: "sei", set: true, wantErr: "absolute path"}, + {name: "dot relative", home: "./sei", set: true, wantErr: "absolute path"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if tc.set { + t.Setenv("SEI_HOME", tc.home) + } else { + t.Setenv("SEI_HOME", "") + os.Unsetenv("SEI_HOME") + } + + // A no-op subcommand: this asserts the guard runs before any + // subcommand body, so it must not depend on serve's own env. + var ran bool + cmd := newRootCommand() + cmd.Commands = []*cli.Command{{ + Name: "serve", + Action: func(context.Context, *cli.Command) error { ran = true; return nil }, + }} + + err := cmd.Run(context.Background(), []string{"sei-sidecar", "serve"}) + if err == nil { + t.Fatalf("expected a refusal, got nil (subcommand ran: %v)", ran) + } + if !strings.Contains(err.Error(), tc.wantErr) { + t.Errorf("error: got %q, want substring %q", err.Error(), tc.wantErr) + } + if ran { + t.Error("subcommand ran despite an invalid SEI_HOME") + } + }) + } +} diff --git a/sidecarapi/tomlpatch/merge_test.go b/sidecarapi/tomlpatch/merge_test.go index 80f35376..92b8b46f 100644 --- a/sidecarapi/tomlpatch/merge_test.go +++ b/sidecarapi/tomlpatch/merge_test.go @@ -5,6 +5,9 @@ import ( "testing" ) +// nestedKey is the map key the nested-merge case shares across its three maps. +const nestedKey = "obj" + func TestMerge(t *testing.T) { tests := []struct { name string @@ -37,10 +40,12 @@ func TestMerge(t *testing.T) { expected: map[string]any{"a": 1}, }, { + // The key is the same in all three maps on purpose — that is what + // makes this a merge rather than a replacement. name: "nested map merge", - original: map[string]any{"obj": map[string]any{"x": 1, "y": 2}}, - patch: map[string]any{"obj": map[string]any{"y": 3, "z": 4}}, - expected: map[string]any{"obj": map[string]any{"x": 1, "y": 3, "z": 4}}, + original: map[string]any{nestedKey: map[string]any{"x": 1, "y": 2}}, + patch: map[string]any{nestedKey: map[string]any{"y": 3, "z": 4}}, + expected: map[string]any{nestedKey: map[string]any{"x": 1, "y": 3, "z": 4}}, }, } From 4bfb93ebe8814fbd69874d03c6bc0d13032afad5 Mon Sep 17 00:00:00 2001 From: bdchatham Date: Fri, 14 Aug 2026 15:00:46 -0700 Subject: [PATCH 12/13] ci(lint): filter by merge base, and name the bypass paths MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The root leg was still reporting 82 findings, 23 of the 25 files in files this branch does not touch. only-new-issues builds its filter from the pull-request patch, and on a 144-file diff that yields nothing usable, so every pre-existing finding surfaced as new — which is why the same config passes on main and fails here. fetch-depth alone did not help. Replaced with --new-from-merge-base, which has git compute the comparison locally, so diff size cannot defeat it. Verified the mechanism on a throwaway repo: a pre-existing finding on the base is filtered, a finding added on the branch is kept. A fetch step makes the base ref available, since checkout does not create it for a pull request. The sidecar leg went 414 → 25 with the previous exclusions. Of the 25, four were worth fixing rather than excluding, and they are the ones a security review already asked for: /v0/healthz, /v0/startupz, /v0/livez and /v0/metrics are the paths that bypass the X-Remote-User check, and they were spelled out separately in auth.go's bypass list and in server.go's route registrations. Now named constants used by both, so those two cannot drift. The proxy's --ignore-paths and openapi.yaml still carry their own copies; collapsing all three is the follow-up. Also switched a test counter to atomic.Int32 from manual atomic.AddInt32 on a plain int32. The remaining goconst hits are excluded with the reason on the page. They are wire and config literals — JSON field names in the EVM digest and shadow comparators, seid config.toml keys that are hyphenated by convention and must read exactly as written, and log attribute names. Reading the literal in place is what makes them checkable against the thing they mirror. One hit suggests reusing conditionHeight for a JSON key and a log attribute that merely share the spelling "height"; that constant names an await-condition kind, so taking the suggestion would conflate two unrelated things. Verified: sidecar and sidecarapi both 0 issues locally; builds, vet, gofmt, goimports, tidy-check, verify-generated and all 25 test packages clean. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 17 ++++++++++++++--- .golangci.yml | 12 ++++++++++++ sidecar/server/auth.go | 18 ++++++++++++++---- sidecar/server/server.go | 8 ++++---- sidecar/tasks/restart_seid_test.go | 4 ++-- 5 files changed, 46 insertions(+), 13 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e150d044..6862c663 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -34,13 +34,24 @@ jobs: - uses: actions/setup-go@v5 with: go-version-file: go.mod + # only-new-issues builds its filter from the pull-request patch, and on a + # diff this size that silently yields nothing — every pre-existing finding + # in an untouched file is then reported as new. --new-from-merge-base has + # git compute the same thing locally, so PR size cannot defeat it. The + # filter exists because of pre-existing debt from the v2.8.0 → v2.12.1 + # bump, tracked in #163; remove it once that is paid down. + - name: Make the merge base available to the filter + run: | + git fetch --no-tags --quiet origin \ + "+refs/heads/${BASE}:refs/remotes/origin/${BASE}" + env: + BASE: ${{ github.base_ref || github.event.repository.default_branch }} - uses: golangci/golangci-lint-action@v8 with: version: v2.12.1 working-directory: ${{ matrix.module }} - # Temporary override — pre-existing lint debt surfaced by the - # v2.8.0 → v2.12.1 bump. Tracked in #163; remove once paid down. - only-new-issues: true + args: >- + --new-from-merge-base=origin/${{ github.base_ref || github.event.repository.default_branch }} hygiene: runs-on: ubuntu-latest diff --git a/.golangci.yml b/.golangci.yml index 3b7dcdef..d02c77d5 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -111,6 +111,18 @@ linters: # allocates itself, so preallocating changes nothing. - linters: [prealloc, unparam] path: sidecar/* + # goconst's remaining hits are wire and config literals: JSON field names + # in the EVM digest and shadow comparators (code, type, nonce, balance, + # gasUsed, storage), seid config.toml keys, which are hyphenated by + # convention and must read exactly as written (p2p, persistent-peers, + # statesync), and log attribute names. Same call as sidecarapi/client — + # reading the literal in place is what makes these checkable against the + # thing they mirror, and hoisting them trades that for a DRY score. One hit + # suggests reusing conditionHeight for a JSON key and a log attribute that + # merely share the spelling "height"; that constant names an + # await-condition kind, so reusing it would conflate two unrelated things. + - linters: [goconst] + path: sidecar/* # noctx wants the database/sql *Context variants throughout the store. That # is a real improvement and a behaviour change — it makes queries # cancellable in the component whose SetMaxOpenConns(1) and WAL-checkpoint diff --git a/sidecar/server/auth.go b/sidecar/server/auth.go index 53dd23e8..2dbd9730 100644 --- a/sidecar/server/auth.go +++ b/sidecar/server/auth.go @@ -46,11 +46,21 @@ const ( // Each path's caller does not carry K8s auth headers, so requiring // X-Remote-User would break the corresponding probe / scrape. // kube-rbac-proxy must include every path here in its --allow-paths. +// The bypass paths, named so the route registrations in server.go and this list +// cannot drift apart. kube-rbac-proxy's --ignore-paths and openapi.yaml carry +// their own copies; those are still hand-synced. +const ( + PathHealthz = "/v0/healthz" // kubelet readiness probe + PathStartupz = "/v0/startupz" // kubelet startup probe + PathLivez = "/v0/livez" // kubelet liveness probe + PathMetrics = "/v0/metrics" // Prometheus scrape +) + var bypassPaths = map[string]struct{}{ - "/v0/healthz": {}, // kubelet readiness probe - "/v0/startupz": {}, // kubelet startup probe - "/v0/livez": {}, // kubelet liveness probe - "/v0/metrics": {}, // Prometheus scrape + PathHealthz: {}, + PathStartupz: {}, + PathLivez: {}, + PathMetrics: {}, } // BypassPaths returns the set of paths exempt from the X-Remote-User diff --git a/sidecar/server/server.go b/sidecar/server/server.go index 89d66c6d..e9d332b2 100644 --- a/sidecar/server/server.go +++ b/sidecar/server/server.go @@ -59,11 +59,11 @@ func NewServer(addr string, eng *engine.Engine, homeDir, authnMode string) *Serv engine: eng, mux: http.NewServeMux(), } - s.mux.HandleFunc("GET /v0/healthz", s.handleHealthz) - s.mux.HandleFunc("GET /v0/startupz", s.handleHealthz) - s.mux.HandleFunc("GET /v0/livez", s.handleLivez) + s.mux.HandleFunc("GET "+PathHealthz, s.handleHealthz) + s.mux.HandleFunc("GET "+PathStartupz, s.handleHealthz) + s.mux.HandleFunc("GET "+PathLivez, s.handleLivez) s.mux.HandleFunc("GET /v0/status", s.handleStatus) - s.mux.Handle("GET /v0/metrics", promhttp.Handler()) + s.mux.Handle("GET "+PathMetrics, promhttp.Handler()) s.mux.HandleFunc("GET /v0/node-id", s.handleNodeID) s.mux.HandleFunc("POST /v0/tasks", s.handlePostTask) s.mux.HandleFunc("GET /v0/tasks", s.handleListTasks) diff --git a/sidecar/tasks/restart_seid_test.go b/sidecar/tasks/restart_seid_test.go index e451e4fd..a09b0603 100644 --- a/sidecar/tasks/restart_seid_test.go +++ b/sidecar/tasks/restart_seid_test.go @@ -34,9 +34,9 @@ func (f *fakeSignaler) Alive(int) bool { return f.alive.Load() } // upAfter returns a probe that reports down for the first n calls, then up. func upAfter(n int) func(context.Context) bool { - var calls int32 + var calls atomic.Int32 return func(context.Context) bool { - return atomic.AddInt32(&calls, 1) > int32(n) + return calls.Add(1) > int32(n) } } From b98df1d393bf1302311f1b8aba95bdc8bf2d82df Mon Sep 17 00:00:00 2001 From: bdchatham Date: Fri, 14 Aug 2026 15:10:46 -0700 Subject: [PATCH 13/13] =?UTF-8?q?fix(lint):=20anchor=20the=20exclusion=20p?= =?UTF-8?q?aths=20=E2=80=94=20they=20were=20leaking=20into=20sidecarapi?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `path` in a golangci-lint exclusion is a regexp, not a glob. `sidecar/*` reads as "sidecar" followed by zero or more slashes, and unanchored, so it matches every sidecarapi/ path too. The goconst exclusion added for the relocated module was therefore switching goconst off across the contract module — wire and tomlpatch included, one commit after a real goconst finding there was fixed. Demonstrated rather than reasoned: a function with three occurrences of one string dropped into sidecarapi/wire/ was reported as 0 issues before this change and is reported correctly after it. With the probe removed, sidecarapi is genuinely clean rather than silently unchecked, and sidecar is unchanged at 0. `api/*` has the same defect and predates this branch — it matches sidecarapi/ for the same reason, so lll was already off there. Every pattern is now anchored with ^, and the exclusions block carries a note, since the glob reading is the natural one and this is easy to reintroduce. Co-Authored-By: Claude Opus 5 (1M context) --- .golangci.yml | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/.golangci.yml b/.golangci.yml index d02c77d5..539a5f8f 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -64,22 +64,26 @@ linters: - omitzero exclusions: generated: lax + # `path` is a REGEXP, not a glob. An unanchored `sidecar/*` means "sidecar" + # followed by zero or more slashes, which also matches every sidecarapi/ + # path — it silently turned goconst off for the contract module. Anchor with + # ^ so a rule applies to the tree it names. rules: - linters: - lll - path: api/* + path: ^api/ - linters: - dupl - lll - path: internal/* + path: ^internal/ - linters: - dupl - lll - path: sdk/* + path: ^sdk/ - linters: - dupl - lll - path: sidecarapi/* + path: ^sidecarapi/ # goconst flags the JSON keys inside the request-params maps — chainId, # keyName, fees, gas, title, initialDeposit, address. Those literals are # the wire contract: reading a params map and seeing the JSON it produces @@ -88,7 +92,7 @@ linters: # DRY score. The remaining hits are test fixtures (1usei, config.toml). - linters: - goconst - path: sidecarapi/client/* + path: ^sidecarapi/client/ # sidecar/ arrived from a repo with no golangci-lint config, so none of it # has ever been linted. Everything below is pre-existing debt made visible # by the move, not new code — and only-new-issues cannot help, because every @@ -98,11 +102,11 @@ linters: # # lll and dupl match what internal/, api/, sdk/ and sidecarapi/ already get. - linters: [dupl, lll] - path: sidecar/* + path: ^sidecar/ # Test code carries different standards than the binary. bodyclose and # staticcheck fire only in _test.go here, and errcheck almost entirely. - linters: [bodyclose, errcheck, goconst, gocyclo, noctx, prealloc, staticcheck, unparam] - path: sidecar/.*_test\.go + path: ^sidecar/.*_test\.go # unparam flags cfg/cdc parameters that a few unexported methods do not # read. They are there for symmetry across a family of sibling methods that # do, so dropping them from some and not others makes the family harder to @@ -110,7 +114,7 @@ linters: # one remaining hit is a json.Unmarshal destination, which Unmarshal # allocates itself, so preallocating changes nothing. - linters: [prealloc, unparam] - path: sidecar/* + path: ^sidecar/ # goconst's remaining hits are wire and config literals: JSON field names # in the EVM digest and shadow comparators (code, type, nonce, balance, # gasUsed, storage), seid config.toml keys, which are hyphenated by @@ -122,7 +126,7 @@ linters: # merely share the spelling "height"; that constant names an # await-condition kind, so reusing it would conflate two unrelated things. - linters: [goconst] - path: sidecar/* + path: ^sidecar/ # noctx wants the database/sql *Context variants throughout the store. That # is a real improvement and a behaviour change — it makes queries # cancellable in the component whose SetMaxOpenConns(1) and WAL-checkpoint @@ -130,7 +134,7 @@ linters: # than riding along inside a 24,000-line relocation. Scoped to the two files # that hold it so noctx stays live everywhere else in the module. - linters: [noctx] - path: sidecar/engine/sqlite_(migrations|store)\.go + path: ^sidecar/engine/sqlite_(migrations|store)\.go paths: - third_party$ - builtin$