Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Contracts are documented in docs/architecture/contracts.md. Changing a contract updates that file in the same PR.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Models can be loaded from HuggingFace (default) or mounted as OCI/modelcar image
- Envoy AI Gateway 1.1.0 installed. **Note**: the `envoyAIGateway.install` flag in this chart is not yet implemented - tracked in [#44](https://github.com/nebari-dev/llm-serving-pack/issues/44). Until that is done, install it manually as an ArgoCD app (see [examples/envoy-ai-gateway.yaml](examples/envoy-ai-gateway.yaml)).
- [Gateway API Inference Extension](https://github.com/kubernetes-sigs/gateway-api-inference-extension) (GIE) CRDs installed - v1.5.0 validated. The pack needs `InferencePool`; the llm-d End-Point Picker crashloops without it.
- A cert-manager `ClusterIssuer` the operator can use for the shared-hostname Certificate. Default expected name is `letsencrypt-issuer`, the one NIC creates; override with `platform.tls.clusterIssuer` in the chart values.
- A cert-manager `ClusterIssuer` for the operator's webhook certificate. Default expected name is `nebari-ca-issuer`, which NIC creates in releases that include [nebari-infrastructure-core#787](https://github.com/nebari-dev/nebari-infrastructure-core/pull/787); otherwise set `operator.webhook.clusterIssuer`. See the [Installation guide](https://packs.nebari.dev/llm-serving-pack/installation/#1-what-this-runbook-assumes).
- DNS for `llm.<baseDomain>` and `llm-internal.<baseDomain>` resolving to the shared Gateway's load balancer (a wildcard CNAME on the base domain is the simplest way). Required for HTTP-01 issuance on the shared Certificate.
- A StorageClass for model storage (EFS, EBS gp3, or any CSI-backed storage that can provision PVCs large enough for your models).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,8 @@ spec:
- {{ include "nebari-llm-serving.fullname" . }}-webhook-service.{{ include "nebari-llm-serving.operatorNamespace" . }}.svc
- {{ include "nebari-llm-serving.fullname" . }}-webhook-service.{{ include "nebari-llm-serving.operatorNamespace" . }}.svc.cluster.local
issuerRef:
name: selfsigned-issuer
{{- /* The dig fallback covers values reused from releases without
operator.webhook. Keep it in sync with operator.webhook.clusterIssuer in
values.yaml. */}}
name: {{ required "operator.webhook.clusterIssuer is required: name an existing cert-manager ClusterIssuer to sign the operator webhook certificate" (dig "webhook" "clusterIssuer" "nebari-ca-issuer" .Values.operator) | quote }}
kind: ClusterIssuer
11 changes: 11 additions & 0 deletions charts/nebari-llm-serving/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -351,6 +351,17 @@ operator:
# Floating on main; pinned to a build sha at release time (see cicd-and-releasing).
tag: "latest"
pullPolicy: Always
webhook:
# clusterIssuer - the cert-manager ClusterIssuer that signs the
# operator's admission webhook serving Certificate (DNS names
# <svc>.<ns>.svc and <svc>.<ns>.svc.cluster.local). NIC creates
# nebari-ca-issuer, its internal CA for in-cluster and webhook
# certificates, in releases that include nebari-infrastructure-core#787.
# On a cluster without it, set this to an existing ClusterIssuer, for
# example a self-signed one named selfsigned-issuer. Must not be empty.
# Keep in sync with the dig fallback in
# templates/operator-webhook-cert.yaml.
clusterIssuer: nebari-ca-issuer

# modelDownloader - the init/job image the operator uses to fetch model
# weights before a serving Deployment starts. Rendered into the operator's
Expand Down
8 changes: 8 additions & 0 deletions docs/architecture/contracts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Contracts

Contracts the operator owns or depends on. Changing one updates its entry in the same PR.

### Chart value `operator.webhook.clusterIssuer`
**Owner:** `charts/nebari-llm-serving` (`values.yaml`, `templates/operator-webhook-cert.yaml`) · **Consumers:** every chart install, through the webhook serving Certificate's `issuerRef`; the docs site (`configuration.mdx`, `installation.md`, `quickstart.mdx`, `upgrading.mdx`) and `README.md` (verified: `rg -n "operator\.webhook\.clusterIssuer|\.Values\.operator\.webhook" --glob '!node_modules' --glob '!.worktrees' .`). The kind dev cluster installs from `dev/manifests/`, not the chart, and is unaffected.
**Promises:** A string naming a cert-manager `ClusterIssuer` (the `kind` stays `ClusterIssuer`). It signs the operator's admission webhook serving Certificate, whose DNS names are `<svc>.<ns>.svc` and `<svc>.<ns>.svc.cluster.local`. Default `nebari-ca-issuer`, which NIC creates in releases that include nebari-infrastructure-core#787; before this value existed the issuer was hardcoded to `selfsigned-issuer`. A missing key or `operator.webhook` block (for example values reused from an older release with `helm upgrade --reuse-values`) renders the default. An explicitly empty value fails the render with `operator.webhook.clusterIssuer is required: ...`. The chart does not create the issuer.
**Since:** 2026-11-20

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This screenshot looks unrelated to the issuer change. Nothing links to it, and it sits outside docs/src/assets/, so the site never shows it. Please drop it from the PR.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/src/content/docs/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ The `openmeter` values configure the `nebari-openmeter` subchart; every key unde
| `operator.image.repository` | `ghcr.io/nebari-dev/llm-serving-pack/operator` | Operator container image repository. |
| `operator.image.tag` | `""` | Image tag. Defaults to `.Chart.AppVersion` when empty. Override only when testing a specific build. |
| `operator.image.pullPolicy` | `Always` | Image pull policy. |
| `operator.webhook.clusterIssuer` | `nebari-ca-issuer` | The cert-manager `ClusterIssuer` that signs the operator's admission webhook serving certificate. NIC creates `nebari-ca-issuer` in releases that include [nebari-infrastructure-core#787](https://github.com/nebari-dev/nebari-infrastructure-core/pull/787). On a cluster without it, set this to an existing `ClusterIssuer`, for example a self-signed one named `selfsigned-issuer`. Must not be empty: the chart will not render. |

### Defaults (cluster-wide fallbacks for LLMModel resources)

Expand Down
36 changes: 35 additions & 1 deletion docs/src/content/docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ This runbook starts from a NIC-foundational Nebari cluster.
> next `nic deploy --regen-apps` silently reverts.

[nic499]: https://github.com/nebari-dev/nebari-infrastructure-core/pull/499
[nic787]: https://github.com/nebari-dev/nebari-infrastructure-core/pull/787

NIC ships, out of the box, the components below; if any are missing or
in a different namespace on your cluster, adjust the commands
Expand Down Expand Up @@ -71,6 +72,24 @@ In addition you need:
ships `letsencrypt-issuer`, which is the chart's default
`platform.tls.clusterIssuer`. If your cluster uses a different
issuer, set it in section 8.
- **A ClusterIssuer for the operator's webhook certificate.** The
chart's `operator.webhook.clusterIssuer` defaults to
`nebari-ca-issuer`, NIC's internal CA for in-cluster and webhook
certificates. NIC creates it in releases that include
[nebari-infrastructure-core#787][nic787]. Without it, the operator
pod stays in `ContainerCreating` because its webhook certificate is
never issued. On a cluster without `nebari-ca-issuer`, set
`operator.webhook.clusterIssuer` in section 8 to an existing
ClusterIssuer, for example a self-signed one:

```yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned-issuer
spec:
selfSigned: {}
```
- **A cluster-config git repo** that ArgoCD's AppProject is configured
to read from. New `Application` manifests in this runbook will be
committed there, under a path of your own choosing (shown throughout
Expand Down Expand Up @@ -150,7 +169,9 @@ kubectl get clusterissuer

Expected: at least one ClusterIssuer with `READY=True`. Capture its
exact name; you will set `platform.tls.clusterIssuer` to that value in
section 8.
section 8. Also check for `nebari-ca-issuer` with `READY=True`; if it
is not listed, pick the ClusterIssuer you will set as
`operator.webhook.clusterIssuer` in section 8 (see section 1).

### 2.5 Confirm the Gateway HTTPS:443 listener has a usable cert

Expand Down Expand Up @@ -1073,6 +1094,13 @@ spec:
tls:
# NIC's ClusterIssuer, and the chart default.
clusterIssuer: letsencrypt-issuer
operator:
webhook:
# Signs the operator's webhook certificate. The default,
# nebari-ca-issuer, exists on NIC releases that include
# nebari-infrastructure-core#787. On other clusters, name
# an existing ClusterIssuer here (see section 1).
clusterIssuer: nebari-ca-issuer
defaults:
storage:
# NIC's default StorageClass is longhorn; the vLLM model
Expand Down Expand Up @@ -1112,6 +1140,12 @@ different hostname (`llm.<baseDomain>` and
`llm-internal.<baseDomain>`) and a different SecurityPolicy
(API-key-protected on external, JWT-protected on internal).

The `operator.webhook.clusterIssuer` value names the ClusterIssuer
that signs the operator's admission webhook certificate
(`nebari-llm-serving-webhook-cert`). Leave it at `nebari-ca-issuer`
when your cluster has that issuer; otherwise set it to the issuer you
picked in section 2.4. An empty value stops the chart from rendering.

If you want to split traffic across two physical Gateways instead
(e.g. one with a public LB, one with an internal-only LB), point
`platform.gateway.internal` at a different Gateway resource.
Expand Down
1 change: 1 addition & 0 deletions docs/src/content/docs/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ For a full production-grade setup (air-gapped clusters, HuggingFace token secret
- **Envoy AI Gateway** v1.1.0 installed. **Note**: the `envoyAIGateway.install` chart flag is not yet implemented ([#44](https://github.com/nebari-dev/llm-serving-pack/issues/44)). Install manually via ArgoCD (see [`examples/envoy-ai-gateway.yaml`](https://github.com/nebari-dev/llm-serving-pack/blob/main/examples/envoy-ai-gateway.yaml)). ([Installation &sect;5](/installation/#5-install-the-envoy-ai-gateway-controller))
- [Gateway API Inference Extension](https://github.com/kubernetes-sigs/gateway-api-inference-extension) CRDs, v1.5.0 validated - `InferencePool` is the one the pack requires ([Installation &sect;4](/installation/#4-install-ai-gateway-and-inference-extension-crds))
- A cert-manager `ClusterIssuer` for the shared TLS certificate (default name: `letsencrypt-issuer`, the one NIC creates; override with `platform.tls.clusterIssuer`) ([Installation &sect;2.4](/installation/#24-confirm-the-cluster-issuer-is-ready))
- A cert-manager `ClusterIssuer` for the operator's webhook certificate (default name: `nebari-ca-issuer`, created by NIC releases that include [nebari-infrastructure-core#787](https://github.com/nebari-dev/nebari-infrastructure-core/pull/787); otherwise set `operator.webhook.clusterIssuer`) ([Installation &sect;1](/installation/#1-what-this-runbook-assumes))
- DNS for `llm.<baseDomain>` and `llm-internal.<baseDomain>` pointing at the shared Gateway load balancer ([Installation &sect;2.3](/installation/#23-confirm-dns-resolves-to-the-gateway-lb))
- A StorageClass that can provision PVCs large enough for your models (EFS, EBS gp3, or equivalent) ([Installation &sect;1](/installation/#1-what-this-runbook-assumes); [sizing guidance](/shared-storage/))

Expand Down
38 changes: 38 additions & 0 deletions docs/src/content/docs/upgrading.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,44 @@ A release that changes a behavior listed on
its heading here, says what changed, and says what a deployment that relies on
it must do.

### Upgrading from 0.1.4 or earlier: the webhook certificate issuer

The ClusterIssuer that signs the operator's admission webhook certificate
is now the chart value `operator.webhook.clusterIssuer`, and its default
changed from `selfsigned-issuer` to `nebari-ca-issuer`. NIC creates
`nebari-ca-issuer` in releases that include
[nebari-infrastructure-core#787](https://github.com/nebari-dev/nebari-infrastructure-core/pull/787).

Check which issuers the cluster has:

```bash
kubectl get clusterissuer
```

If `nebari-ca-issuer` is listed with `READY=True`, no change is needed:
cert-manager reissues the webhook certificate from it right away and
updates the webhook's CA bundle. The operator picks up the new certificate
only when the kubelet next syncs its Secret volume, about a minute later.
In that window, creating or updating an `LLMModel` or `PassthroughModel`
fails with `x509: certificate signed by unknown authority`; retry after
a minute.

Otherwise, **keep the old issuer before upgrading** by adding it to your
values:

```yaml
operator:
webhook:
clusterIssuer: selfsigned-issuer
```

If you upgrade with neither, cert-manager leaves the existing certificate
Secret untouched, the Certificate goes `Ready=False` with reason
`IncorrectIssuer`, and its CertificateRequest stays `Pending`. Admission
keeps working with the old certificate, so the upgrade looks healthy, but
the certificate cannot be renewed and admission fails once it expires.
Create the issuer or set the value to fix it.

### Upgrading from 0.1.4 or earlier: access groups compared as group paths

`access.groups` is now compared with the token's groups claim as Keycloak
Expand Down
Loading