diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..2d00261 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +Contracts are documented in docs/architecture/contracts.md. Changing a contract updates that file in the same PR. diff --git a/README.md b/README.md index b9700e3..5a20cd2 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,7 @@ Models can be loaded from HuggingFace (default) or mounted as OCI/modelcar image - Envoy AI Gateway 1.1.0 installed. **Note**: the `envoyAIGateway.install` flag in this chart is not yet implemented - tracked in [#44](https://github.com/nebari-dev/llm-serving-pack/issues/44). Until that is done, install it manually as an ArgoCD app (see [examples/envoy-ai-gateway.yaml](examples/envoy-ai-gateway.yaml)). - [Gateway API Inference Extension](https://github.com/kubernetes-sigs/gateway-api-inference-extension) (GIE) CRDs installed - v1.5.0 validated. The pack needs `InferencePool`; the llm-d End-Point Picker crashloops without it. - A cert-manager `ClusterIssuer` the operator can use for the shared-hostname Certificate. Default expected name is `letsencrypt-issuer`, the one NIC creates; override with `platform.tls.clusterIssuer` in the chart values. +- A cert-manager `ClusterIssuer` for the operator's webhook certificate. Default expected name is `nebari-ca-issuer`, which NIC creates in releases that include [nebari-infrastructure-core#787](https://github.com/nebari-dev/nebari-infrastructure-core/pull/787); otherwise set `operator.webhook.clusterIssuer`. See the [Installation guide](https://packs.nebari.dev/llm-serving-pack/installation/#1-what-this-runbook-assumes). - DNS for `llm.` and `llm-internal.` resolving to the shared Gateway's load balancer (a wildcard CNAME on the base domain is the simplest way). Required for HTTP-01 issuance on the shared Certificate. - A StorageClass for model storage (EFS, EBS gp3, or any CSI-backed storage that can provision PVCs large enough for your models). diff --git a/charts/nebari-llm-serving/templates/operator-webhook-cert.yaml b/charts/nebari-llm-serving/templates/operator-webhook-cert.yaml index f7d58e2..ff76d54 100644 --- a/charts/nebari-llm-serving/templates/operator-webhook-cert.yaml +++ b/charts/nebari-llm-serving/templates/operator-webhook-cert.yaml @@ -11,5 +11,8 @@ spec: - {{ include "nebari-llm-serving.fullname" . }}-webhook-service.{{ include "nebari-llm-serving.operatorNamespace" . }}.svc - {{ include "nebari-llm-serving.fullname" . }}-webhook-service.{{ include "nebari-llm-serving.operatorNamespace" . }}.svc.cluster.local issuerRef: - name: selfsigned-issuer + {{- /* The dig fallback covers values reused from releases without + operator.webhook. Keep it in sync with operator.webhook.clusterIssuer in + values.yaml. */}} + name: {{ required "operator.webhook.clusterIssuer is required: name an existing cert-manager ClusterIssuer to sign the operator webhook certificate" (dig "webhook" "clusterIssuer" "nebari-ca-issuer" .Values.operator) | quote }} kind: ClusterIssuer diff --git a/charts/nebari-llm-serving/values.yaml b/charts/nebari-llm-serving/values.yaml index 6c40588..e535646 100644 --- a/charts/nebari-llm-serving/values.yaml +++ b/charts/nebari-llm-serving/values.yaml @@ -351,6 +351,17 @@ operator: # Floating on main; pinned to a build sha at release time (see cicd-and-releasing). tag: "latest" pullPolicy: Always + webhook: + # clusterIssuer - the cert-manager ClusterIssuer that signs the + # operator's admission webhook serving Certificate (DNS names + # ..svc and ..svc.cluster.local). NIC creates + # nebari-ca-issuer, its internal CA for in-cluster and webhook + # certificates, in releases that include nebari-infrastructure-core#787. + # On a cluster without it, set this to an existing ClusterIssuer, for + # example a self-signed one named selfsigned-issuer. Must not be empty. + # Keep in sync with the dig fallback in + # templates/operator-webhook-cert.yaml. + clusterIssuer: nebari-ca-issuer # modelDownloader - the init/job image the operator uses to fetch model # weights before a serving Deployment starts. Rendered into the operator's diff --git a/docs/architecture/contracts.md b/docs/architecture/contracts.md new file mode 100644 index 0000000..63b7592 --- /dev/null +++ b/docs/architecture/contracts.md @@ -0,0 +1,8 @@ +# Contracts + +Contracts the operator owns or depends on. Changing one updates its entry in the same PR. + +### Chart value `operator.webhook.clusterIssuer` +**Owner:** `charts/nebari-llm-serving` (`values.yaml`, `templates/operator-webhook-cert.yaml`) ยท **Consumers:** every chart install, through the webhook serving Certificate's `issuerRef`; the docs site (`configuration.mdx`, `installation.md`, `quickstart.mdx`, `upgrading.mdx`) and `README.md` (verified: `rg -n "operator\.webhook\.clusterIssuer|\.Values\.operator\.webhook" --glob '!node_modules' --glob '!.worktrees' .`). The kind dev cluster installs from `dev/manifests/`, not the chart, and is unaffected. +**Promises:** A string naming a cert-manager `ClusterIssuer` (the `kind` stays `ClusterIssuer`). It signs the operator's admission webhook serving Certificate, whose DNS names are `..svc` and `..svc.cluster.local`. Default `nebari-ca-issuer`, which NIC creates in releases that include nebari-infrastructure-core#787; before this value existed the issuer was hardcoded to `selfsigned-issuer`. A missing key or `operator.webhook` block (for example values reused from an older release with `helm upgrade --reuse-values`) renders the default. An explicitly empty value fails the render with `operator.webhook.clusterIssuer is required: ...`. The chart does not create the issuer. +**Since:** 2026-11-20 diff --git a/docs/install-production-screenshots/outsider-landing-page.png b/docs/install-production-screenshots/outsider-landing-page.png new file mode 100644 index 0000000..c7b1422 Binary files /dev/null and b/docs/install-production-screenshots/outsider-landing-page.png differ diff --git a/docs/src/content/docs/configuration.mdx b/docs/src/content/docs/configuration.mdx index 3d5109a..642ce52 100644 --- a/docs/src/content/docs/configuration.mdx +++ b/docs/src/content/docs/configuration.mdx @@ -149,6 +149,7 @@ The `openmeter` values configure the `nebari-openmeter` subchart; every key unde | `operator.image.repository` | `ghcr.io/nebari-dev/llm-serving-pack/operator` | Operator container image repository. | | `operator.image.tag` | `""` | Image tag. Defaults to `.Chart.AppVersion` when empty. Override only when testing a specific build. | | `operator.image.pullPolicy` | `Always` | Image pull policy. | +| `operator.webhook.clusterIssuer` | `nebari-ca-issuer` | The cert-manager `ClusterIssuer` that signs the operator's admission webhook serving certificate. NIC creates `nebari-ca-issuer` in releases that include [nebari-infrastructure-core#787](https://github.com/nebari-dev/nebari-infrastructure-core/pull/787). On a cluster without it, set this to an existing `ClusterIssuer`, for example a self-signed one named `selfsigned-issuer`. Must not be empty: the chart will not render. | ### Defaults (cluster-wide fallbacks for LLMModel resources) diff --git a/docs/src/content/docs/installation.md b/docs/src/content/docs/installation.md index bf8ca79..ab8a0c7 100644 --- a/docs/src/content/docs/installation.md +++ b/docs/src/content/docs/installation.md @@ -30,6 +30,7 @@ This runbook starts from a NIC-foundational Nebari cluster. > next `nic deploy --regen-apps` silently reverts. [nic499]: https://github.com/nebari-dev/nebari-infrastructure-core/pull/499 +[nic787]: https://github.com/nebari-dev/nebari-infrastructure-core/pull/787 NIC ships, out of the box, the components below; if any are missing or in a different namespace on your cluster, adjust the commands @@ -71,6 +72,24 @@ In addition you need: ships `letsencrypt-issuer`, which is the chart's default `platform.tls.clusterIssuer`. If your cluster uses a different issuer, set it in section 8. +- **A ClusterIssuer for the operator's webhook certificate.** The + chart's `operator.webhook.clusterIssuer` defaults to + `nebari-ca-issuer`, NIC's internal CA for in-cluster and webhook + certificates. NIC creates it in releases that include + [nebari-infrastructure-core#787][nic787]. Without it, the operator + pod stays in `ContainerCreating` because its webhook certificate is + never issued. On a cluster without `nebari-ca-issuer`, set + `operator.webhook.clusterIssuer` in section 8 to an existing + ClusterIssuer, for example a self-signed one: + + ```yaml + apiVersion: cert-manager.io/v1 + kind: ClusterIssuer + metadata: + name: selfsigned-issuer + spec: + selfSigned: {} + ``` - **A cluster-config git repo** that ArgoCD's AppProject is configured to read from. New `Application` manifests in this runbook will be committed there, under a path of your own choosing (shown throughout @@ -150,7 +169,9 @@ kubectl get clusterissuer Expected: at least one ClusterIssuer with `READY=True`. Capture its exact name; you will set `platform.tls.clusterIssuer` to that value in -section 8. +section 8. Also check for `nebari-ca-issuer` with `READY=True`; if it +is not listed, pick the ClusterIssuer you will set as +`operator.webhook.clusterIssuer` in section 8 (see section 1). ### 2.5 Confirm the Gateway HTTPS:443 listener has a usable cert @@ -1073,6 +1094,13 @@ spec: tls: # NIC's ClusterIssuer, and the chart default. clusterIssuer: letsencrypt-issuer + operator: + webhook: + # Signs the operator's webhook certificate. The default, + # nebari-ca-issuer, exists on NIC releases that include + # nebari-infrastructure-core#787. On other clusters, name + # an existing ClusterIssuer here (see section 1). + clusterIssuer: nebari-ca-issuer defaults: storage: # NIC's default StorageClass is longhorn; the vLLM model @@ -1112,6 +1140,12 @@ different hostname (`llm.` and `llm-internal.`) and a different SecurityPolicy (API-key-protected on external, JWT-protected on internal). +The `operator.webhook.clusterIssuer` value names the ClusterIssuer +that signs the operator's admission webhook certificate +(`nebari-llm-serving-webhook-cert`). Leave it at `nebari-ca-issuer` +when your cluster has that issuer; otherwise set it to the issuer you +picked in section 2.4. An empty value stops the chart from rendering. + If you want to split traffic across two physical Gateways instead (e.g. one with a public LB, one with an internal-only LB), point `platform.gateway.internal` at a different Gateway resource. diff --git a/docs/src/content/docs/quickstart.mdx b/docs/src/content/docs/quickstart.mdx index 077d71f..d471f69 100644 --- a/docs/src/content/docs/quickstart.mdx +++ b/docs/src/content/docs/quickstart.mdx @@ -17,6 +17,7 @@ For a full production-grade setup (air-gapped clusters, HuggingFace token secret - **Envoy AI Gateway** v1.1.0 installed. **Note**: the `envoyAIGateway.install` chart flag is not yet implemented ([#44](https://github.com/nebari-dev/llm-serving-pack/issues/44)). Install manually via ArgoCD (see [`examples/envoy-ai-gateway.yaml`](https://github.com/nebari-dev/llm-serving-pack/blob/main/examples/envoy-ai-gateway.yaml)). ([Installation §5](/installation/#5-install-the-envoy-ai-gateway-controller)) - [Gateway API Inference Extension](https://github.com/kubernetes-sigs/gateway-api-inference-extension) CRDs, v1.5.0 validated - `InferencePool` is the one the pack requires ([Installation §4](/installation/#4-install-ai-gateway-and-inference-extension-crds)) - A cert-manager `ClusterIssuer` for the shared TLS certificate (default name: `letsencrypt-issuer`, the one NIC creates; override with `platform.tls.clusterIssuer`) ([Installation §2.4](/installation/#24-confirm-the-cluster-issuer-is-ready)) +- A cert-manager `ClusterIssuer` for the operator's webhook certificate (default name: `nebari-ca-issuer`, created by NIC releases that include [nebari-infrastructure-core#787](https://github.com/nebari-dev/nebari-infrastructure-core/pull/787); otherwise set `operator.webhook.clusterIssuer`) ([Installation §1](/installation/#1-what-this-runbook-assumes)) - DNS for `llm.` and `llm-internal.` pointing at the shared Gateway load balancer ([Installation §2.3](/installation/#23-confirm-dns-resolves-to-the-gateway-lb)) - A StorageClass that can provision PVCs large enough for your models (EFS, EBS gp3, or equivalent) ([Installation §1](/installation/#1-what-this-runbook-assumes); [sizing guidance](/shared-storage/)) diff --git a/docs/src/content/docs/upgrading.mdx b/docs/src/content/docs/upgrading.mdx index 8399d39..4a2d756 100644 --- a/docs/src/content/docs/upgrading.mdx +++ b/docs/src/content/docs/upgrading.mdx @@ -84,6 +84,44 @@ A release that changes a behavior listed on its heading here, says what changed, and says what a deployment that relies on it must do. +### Upgrading from 0.1.4 or earlier: the webhook certificate issuer + +The ClusterIssuer that signs the operator's admission webhook certificate +is now the chart value `operator.webhook.clusterIssuer`, and its default +changed from `selfsigned-issuer` to `nebari-ca-issuer`. NIC creates +`nebari-ca-issuer` in releases that include +[nebari-infrastructure-core#787](https://github.com/nebari-dev/nebari-infrastructure-core/pull/787). + +Check which issuers the cluster has: + +```bash +kubectl get clusterissuer +``` + +If `nebari-ca-issuer` is listed with `READY=True`, no change is needed: +cert-manager reissues the webhook certificate from it right away and +updates the webhook's CA bundle. The operator picks up the new certificate +only when the kubelet next syncs its Secret volume, about a minute later. +In that window, creating or updating an `LLMModel` or `PassthroughModel` +fails with `x509: certificate signed by unknown authority`; retry after +a minute. + +Otherwise, **keep the old issuer before upgrading** by adding it to your +values: + +```yaml +operator: + webhook: + clusterIssuer: selfsigned-issuer +``` + +If you upgrade with neither, cert-manager leaves the existing certificate +Secret untouched, the Certificate goes `Ready=False` with reason +`IncorrectIssuer`, and its CertificateRequest stays `Pending`. Admission +keeps working with the old certificate, so the upgrade looks healthy, but +the certificate cannot be renewed and admission fails once it expires. +Create the issuer or set the value to fix it. + ### Upgrading from 0.1.4 or earlier: access groups compared as group paths `access.groups` is now compared with the token's groups claim as Keycloak