Skip to content

Commit d121800

Browse files
committed
feat(helm): support image registry overrides
Signed-off-by: Bobbins228 <mcampbel@redhat.com>
1 parent e89f35b commit d121800

15 files changed

Lines changed: 179 additions & 69 deletions

‎TESTING.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -359,10 +359,10 @@ mise run e2e:kubernetes
359359
```
360360

361361
`GATEWAY_IMAGE` applies to the Kubernetes gateway container. `SUPERVISOR_IMAGE`
362-
applies to the supervisor image selected by the Kubernetes, Docker, and Podman
363-
wrappers. `SANDBOX_IMAGE` applies to the standalone sandbox image selected by
364-
all test wrappers. A repository-only value inherits `IMAGE_TAG` for gateway and
365-
supervisor images; a value with an explicit tag or `@sha256:` digest is used
362+
applies to the trusted supervisor image selected by the Kubernetes, Docker, and
363+
Podman wrappers. `SANDBOX_IMAGE` applies to the trusted workload-side runtime
364+
image that stages the `openshell-sandbox` binary. A repository-only value
365+
inherits `IMAGE_TAG`; a value with an explicit tag or `@sha256:` digest is used
366366
as-is. When these variables are unset, the existing `OPENSHELL_REGISTRY` plus
367367
`IMAGE_TAG` behavior is retained.
368368
The Docker and Podman wrappers continue to give
@@ -404,7 +404,7 @@ Kubernetes e2e environment variables:
404404
| `OPENSHELL_REGISTRY` | Image registry prefix (default: `ghcr.io/nvidia/openshell`) |
405405
| `GATEWAY_IMAGE` | Kubernetes gateway image repository or complete tagged/digest-pinned image reference; digests require `OPENSHELL_E2E_KUBE_BUILD_IMAGES=0` |
406406
| `SUPERVISOR_IMAGE` | Gateway/supervisor image repository or complete tagged/digest-pinned image reference; Kubernetes digests require `OPENSHELL_E2E_KUBE_BUILD_IMAGES=0` |
407-
| `SANDBOX_IMAGE` | Standalone sandbox image repository or complete tagged/digest-pinned image reference |
407+
| `SANDBOX_IMAGE` | Trusted sandbox runtime image repository or complete tagged/digest-pinned image reference |
408408

409409
Run a single test directly with cargo:
410410

‎deploy/helm/openshell/README.md‎

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -187,11 +187,12 @@ discovery endpoint or its TLS CA.
187187
| certManager.serverIssuerRef | object | `{"group":"","kind":"","name":""}` | Override the issuerRef for the external server Certificate (e.g. a real LetsEncrypt/ACME ClusterIssuer for a publicly-trusted cert on an external hostname). When set, the chart creates a second server certificate from this issuer with only the hostnames in serverDnsNames; the internal server certificate is always signed by the chart's own CA. Leave name empty to use the chart CA for all server certificates (default). Requires certManager.enabled=true. |
188188
| fullnameOverride | string | `""` | Override the full generated resource name. |
189189
| gateway.image.digest | string | `""` | Gateway image digest. When set, this takes precedence over tag. |
190-
| gateway.image.pullPolicy | string | `nil` | Gateway image pull policy. |
191-
| gateway.image.repository | string | `""` | Gateway image repository. |
190+
| gateway.image.pullPolicy | string | `nil` | Gateway image pull policy. Empty uses global.image.pullPolicy. |
191+
| gateway.image.registry | string | `""` | Gateway image registry. Empty uses global.image.registry. |
192+
| gateway.image.repository | string | `"openshell/gateway"` | Gateway image repository. |
192193
| gateway.image.tag | string | `""` | Gateway image tag. Defaults to the chart appVersion when empty. |
193194
| global.image.pullPolicy | string | `"IfNotPresent"` | Shared OpenShell image pull policy. Individual image pull policies take precedence. |
194-
| global.image.repository | string | `"ghcr.io/nvidia/openshell"` | Shared OpenShell image repository. Individual image repositories take precedence. |
195+
| global.image.registry | string | `"ghcr.io/nvidia"` | Shared OpenShell image registry. Individual image registries take precedence. |
195196
| global.image.tag | string | `""` | Shared OpenShell image tag. Defaults to the chart appVersion when empty. |
196197
| grpcRoute.backendTLSPolicy.caCertificateConfigMapName | string | `""` | Name of the ConfigMap containing the CA certificate (key: ca.crt) used to validate the gateway pod's TLS certificate. Defaults to `<fullname>-backend-ca` when empty. The certgen hook auto-creates this: with pkiInitJob (default), immediately on install/upgrade; with cert-manager, the hook polls for pkiInitJob.timeoutSeconds seconds waiting for cert-manager to issue the server certificate, then creates the ConfigMap. A single install usually succeeds; if cert-manager takes longer, increase pkiInitJob.timeoutSeconds. By default (pkiInitJob.failOnTimeout=true), the install fails if the timeout is reached; set failOnTimeout=false to allow the install to succeed and run `helm upgrade` after the certificate is issued. |
197198
| grpcRoute.backendTLSPolicy.enabled | bool | `false` | Create a BackendTLSPolicy resource for end-to-end TLS between the Gateway proxy and the OpenShell gateway pod. The traffic flow is: client → HTTPS → Gateway (terminate) → TLS (re-encrypt) → gateway pod. Requires server.disableTls=false and server.tls.enableMtls=false. The certgen hook auto-creates the backend CA ConfigMap. |
@@ -239,9 +240,11 @@ discovery endpoint or its TLS CA.
239240
| sandbox.image.pullPolicy | string | `nil` | Sandbox image pull policy. Leave unset to use the Kubernetes image default. |
240241
| sandbox.image.repository | string | `"ghcr.io/nvidia/openshell-community/sandboxes/base"` | Default standalone sandbox image repository. |
241242
| sandbox.image.tag | string | `"latest"` | Sandbox image tag. Defaults to latest when empty. |
242-
| sandboxRuntime.image.pullPolicy | string | `nil` | Sandbox runtime image pull policy. Defaults to the gateway image pull policy when empty. |
243-
| sandboxRuntime.image.repository | string | `""` | Sandbox runtime image repository. Changing it uses the effective gateway image tag unless tag is also set. |
244-
| sandboxRuntime.image.tag | string | `""` | Sandbox runtime image tag override. Empty uses the version pinned into the gateway unless repository is changed. |
243+
| sandboxRuntime.image.digest | string | `""` | Sandbox runtime image digest. When set, this takes precedence over tag. |
244+
| sandboxRuntime.image.pullPolicy | string | `nil` | Sandbox runtime image pull policy. Empty uses global.image.pullPolicy. |
245+
| sandboxRuntime.image.registry | string | `""` | Sandbox runtime image registry. Empty uses global.image.registry. |
246+
| sandboxRuntime.image.repository | string | `"openshell/sandbox"` | Sandbox runtime image repository. |
247+
| sandboxRuntime.image.tag | string | `""` | Sandbox runtime image tag. Defaults to the chart appVersion when empty. |
245248
| sandboxServiceAccount.annotations | object | `{}` | Annotations to add to the generated sandbox service account. |
246249
| sandboxServiceAccount.create | bool | `true` | Create a service account for sandbox pods. |
247250
| sandboxServiceAccount.name | string | `""` | Existing service account name for sandbox pods when sandboxServiceAccount.create is false. |
@@ -320,9 +323,10 @@ discovery endpoint or its TLS CA.
320323
| serviceAccount.create | bool | `true` | Create a service account for the gateway. |
321324
| serviceAccount.name | string | `""` | Existing service account name to use when serviceAccount.create is false. |
322325
| supervisor.image.digest | string | `""` | Supervisor image digest. When set, this takes precedence over tag. |
323-
| supervisor.image.pullPolicy | string | `nil` | Sandbox supervisor pull policy. Leave unset to use the Kubernetes image default. Prefer always, if_not_present, or never; the chart also accepts legacy Kubernetes spellings Always, IfNotPresent, and Never. |
324-
| supervisor.image.repository | string | `""` | Supervisor image repository. Changing it uses the effective gateway image tag unless tag is also set. |
325-
| supervisor.image.tag | string | `""` | Supervisor image tag override. Empty uses the version pinned into the gateway unless repository is changed. |
326+
| supervisor.image.pullPolicy | string | `nil` | Supervisor image pull policy. Empty uses global.image.pullPolicy. Prefer always, if_not_present, or never; the chart also accepts legacy Kubernetes spellings Always, IfNotPresent, and Never. |
327+
| supervisor.image.registry | string | `""` | Supervisor image registry. Empty uses global.image.registry. |
328+
| supervisor.image.repository | string | `"openshell/supervisor"` | Supervisor image repository. |
329+
| supervisor.image.tag | string | `""` | Supervisor image tag. Defaults to the chart appVersion when empty. |
326330
| supervisor.sandboxRuntime.boundaryPort | int | `5500` | Workload boundary TLS listener port. |
327331
| supervisor.sandboxRuntime.networkPolicyEnforced | bool | `false` | Required operator acknowledgement that the cluster CNI enforces NetworkPolicy. |
328332
| tolerations | list | `[]` | Tolerations for the gateway pod. |

‎deploy/helm/openshell/skaffold.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,9 @@ deploy:
139139
# To test multi-replica external PostgreSQL behavior:
140140
#- ci/values-high-availability.yaml
141141
setValueTemplates:
142+
# Skaffold supplies complete image repositories. Clear the chart's
143+
# shared registry so these values are used verbatim.
144+
global.image.registry: ''
142145
gateway.image.repository: '{{.IMAGE_REPO_openshell_gateway}}'
143146
gateway.image.tag: '{{.IMAGE_TAG_openshell_gateway}}'
144147
supervisor.image.repository: '{{.IMAGE_REPO_openshell_supervisor}}'

‎deploy/helm/openshell/templates/_helpers.tpl‎

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,8 @@ default to enabled so upgrades with --reuse-values preserve the old topology.
8787
{{- define "openshell.image" -}}
8888
{{- $image := .Values.gateway.image -}}
8989
{{- $global := .Values.global.image -}}
90-
{{- $repository := $image.repository | default (printf "%s/gateway" $global.repository) -}}
90+
{{- $registry := $image.registry | default $global.registry -}}
91+
{{- $repository := ternary (printf "%s/%s" $registry $image.repository) $image.repository (ne $registry "") -}}
9192
{{- if $image.digest -}}
9293
{{- printf "%s@%s" $repository $image.digest -}}
9394
{{- else -}}
@@ -114,16 +115,22 @@ ghcr.io/nvidia/openshell/sandbox
114115
{{- define "openshell.sandboxRuntimeImageOverrideEnabled" -}}
115116
{{- $defaultRepository := include "openshell.defaultSandboxRuntimeRepository" . -}}
116117
{{- $global := .Values.global.image -}}
117-
{{- $repository := .Values.sandboxRuntime.image.repository | default (printf "%s/sandbox" $global.repository) -}}
118-
{{- if or (ne $repository $defaultRepository) .Values.sandboxRuntime.image.tag .Values.global.image.tag -}}true{{- end -}}
118+
{{- $registry := .Values.sandboxRuntime.image.registry | default $global.registry -}}
119+
{{- $repository := ternary (printf "%s/%s" $registry .Values.sandboxRuntime.image.repository) .Values.sandboxRuntime.image.repository (ne $registry "") -}}
120+
{{- if or (ne $repository $defaultRepository) .Values.sandboxRuntime.image.tag .Values.sandboxRuntime.image.digest .Values.global.image.tag -}}true{{- end -}}
119121
{{- end }}
120122

121123
{{/* Sandbox runtime image override. */}}
122124
{{- define "openshell.sandboxRuntimeImage" -}}
123125
{{- $global := .Values.global.image -}}
124-
{{- $repository := .Values.sandboxRuntime.image.repository | default (printf "%s/sandbox" $global.repository) -}}
126+
{{- $registry := .Values.sandboxRuntime.image.registry | default $global.registry -}}
127+
{{- $repository := ternary (printf "%s/%s" $registry .Values.sandboxRuntime.image.repository) .Values.sandboxRuntime.image.repository (ne $registry "") -}}
128+
{{- if .Values.sandboxRuntime.image.digest -}}
129+
{{- printf "%s@%s" $repository .Values.sandboxRuntime.image.digest -}}
130+
{{- else -}}
125131
{{- $tag := .Values.sandboxRuntime.image.tag | default $global.tag | default .Chart.AppVersion -}}
126-
{{- printf "%s:%s" $repository $tag }}
132+
{{- printf "%s:%s" $repository $tag -}}
133+
{{- end -}}
127134
{{- end }}
128135

129136
{{/* Official supervisor repository used by the gateway's built-in default. */}}
@@ -154,13 +161,11 @@ The chart's documented repository and empty tag are the gateway-owned default.
154161
true
155162
{{- end }}
156163

157-
{{/*
158-
Supervisor image override. A tag-only override uses the official repository;
159-
a repository-only override uses the effective gateway image tag.
160-
*/}}
164+
{{/* Supervisor image override. */}}
161165
{{- define "openshell.supervisorImage" -}}
162166
{{- $global := .Values.global.image -}}
163-
{{- $repository := .Values.supervisor.image.repository | default (printf "%s/supervisor" $global.repository) -}}
167+
{{- $registry := .Values.supervisor.image.registry | default $global.registry -}}
168+
{{- $repository := ternary (printf "%s/%s" $registry .Values.supervisor.image.repository) .Values.supervisor.image.repository (ne $registry "") -}}
164169
{{- if .Values.supervisor.image.digest -}}
165170
{{- printf "%s@%s" $repository .Values.supervisor.image.digest -}}
166171
{{- else -}}

‎deploy/helm/openshell/tests/gateway_config_test.yaml‎

Lines changed: 51 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,8 @@ tests:
4040
- it: preserves a complete tagged gateway image reference
4141
template: templates/statefulset.yaml
4242
set:
43-
gateway.image.repository: registry.example/gateway
43+
gateway.image.registry: registry.example
44+
gateway.image.repository: gateway
4445
gateway.image.tag: branch
4546
asserts:
4647
- equal:
@@ -50,7 +51,8 @@ tests:
5051
- it: defaults a repository-only gateway image to the chart appVersion
5152
template: templates/statefulset.yaml
5253
set:
53-
gateway.image.repository: registry.example/gateway
54+
gateway.image.registry: registry.example
55+
gateway.image.repository: gateway
5456
asserts:
5557
- equal:
5658
path: spec.template.spec.containers[0].image
@@ -59,7 +61,7 @@ tests:
5961
- it: applies global image values to gateway, supervisor, and sandbox runtime
6062
template: templates/gateway-config.yaml
6163
set:
62-
global.image.repository: registry.example.com/openshell
64+
global.image.registry: registry.example.com
6365
global.image.tag: v0.1.0
6466
global.image.pullPolicy: IfNotPresent
6567
asserts:
@@ -70,6 +72,28 @@ tests:
7072
path: data["gateway.toml"]
7173
pattern: 'sandbox_runtime_image\s*=\s*"registry\.example\.com/openshell/sandbox:v0\.1\.0"'
7274

75+
- it: lets an individual registry override the global registry
76+
template: templates/statefulset.yaml
77+
set:
78+
global.image.registry: registry.example.com
79+
gateway.image.registry: mirror.example.com
80+
gateway.image.repository: custom/gateway
81+
global.image.tag: v0.1.0
82+
asserts:
83+
- equal:
84+
path: spec.template.spec.containers[0].image
85+
value: mirror.example.com/custom/gateway:v0.1.0
86+
87+
- it: applies the global registry to the gateway image
88+
template: templates/statefulset.yaml
89+
set:
90+
global.image.registry: registry.example.com
91+
global.image.tag: v0.1.0
92+
asserts:
93+
- equal:
94+
path: spec.template.spec.containers[0].image
95+
value: registry.example.com/openshell/gateway:v0.1.0
96+
7397
- it: defaults a repository-only sandbox image to latest
7498
template: templates/gateway-config.yaml
7599
set:
@@ -82,7 +106,8 @@ tests:
82106
- it: gives a gateway digest precedence over its tag
83107
template: templates/statefulset.yaml
84108
set:
85-
gateway.image.repository: registry.example/gateway
109+
gateway.image.registry: registry.example
110+
gateway.image.repository: gateway
86111
gateway.image.tag: ignored
87112
gateway.image.digest: sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
88113
asserts:
@@ -93,7 +118,8 @@ tests:
93118
- it: gives supervisor and sandbox digests precedence over their tags
94119
template: templates/gateway-config.yaml
95120
set:
96-
supervisor.image.repository: registry.example/supervisor
121+
supervisor.image.registry: registry.example
122+
supervisor.image.repository: supervisor
97123
supervisor.image.tag: ignored
98124
supervisor.image.digest: sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
99125
sandbox.image.repository: registry.example/sandbox
@@ -367,10 +393,12 @@ tests:
367393
- it: renders independent sandbox runtime and supervisor image overrides
368394
template: templates/gateway-config.yaml
369395
set:
370-
sandboxRuntime.image.repository: registry.example.com/openshell/sandbox
396+
sandboxRuntime.image.registry: registry.example.com
397+
sandboxRuntime.image.repository: openshell/sandbox
371398
sandboxRuntime.image.tag: sandbox-build
372399
sandboxRuntime.image.pullPolicy: Always
373-
supervisor.image.repository: registry.example.com/openshell/supervisor
400+
supervisor.image.registry: registry.example.com
401+
supervisor.image.repository: openshell/supervisor
374402
supervisor.image.tag: supervisor-build
375403
asserts:
376404
- matchRegex:
@@ -383,6 +411,18 @@ tests:
383411
path: data["gateway.toml"]
384412
pattern: 'supervisor_image\s*=\s*"registry\.example\.com/openshell/supervisor:supervisor-build"'
385413

414+
- it: gives a sandbox runtime digest precedence over its tag
415+
template: templates/gateway-config.yaml
416+
set:
417+
sandboxRuntime.image.registry: registry.example.com
418+
sandboxRuntime.image.repository: openshell/sandbox
419+
sandboxRuntime.image.tag: ignored
420+
sandboxRuntime.image.digest: sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
421+
asserts:
422+
- matchRegex:
423+
path: data["gateway.toml"]
424+
pattern: 'sandbox_runtime_image\s*=\s*"registry\.example\.com/openshell/sandbox@sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"'
425+
386426
- it: renders a supervisor tag override with the official repository
387427
template: templates/gateway-config.yaml
388428
set:
@@ -395,7 +435,8 @@ tests:
395435
- it: renders a supervisor repository override with the default chart tag
396436
template: templates/gateway-config.yaml
397437
set:
398-
supervisor.image.repository: registry.example.com/openshell/supervisor
438+
supervisor.image.registry: registry.example.com
439+
supervisor.image.repository: openshell/supervisor
399440
asserts:
400441
- matchRegex:
401442
path: data["gateway.toml"]
@@ -404,7 +445,8 @@ tests:
404445
- it: renders complete supervisor repository and tag overrides
405446
template: templates/gateway-config.yaml
406447
set:
407-
supervisor.image.repository: registry.example.com/openshell/supervisor
448+
supervisor.image.registry: registry.example.com
449+
supervisor.image.repository: openshell/supervisor
408450
supervisor.image.tag: supervisor-build
409451
asserts:
410452
- matchRegex:

0 commit comments

Comments
 (0)