diff --git a/docs/en/solutions/How_to_Preserve_a_Non_Default_Backend_Port_in_the_Upstream_Host_Header_with_Envoy_Gateway_on_ACP.md b/docs/en/solutions/How_to_Preserve_a_Non_Default_Backend_Port_in_the_Upstream_Host_Header_with_Envoy_Gateway_on_ACP.md index a60db1b2f..ae5afc665 100644 --- a/docs/en/solutions/How_to_Preserve_a_Non_Default_Backend_Port_in_the_Upstream_Host_Header_with_Envoy_Gateway_on_ACP.md +++ b/docs/en/solutions/How_to_Preserve_a_Non_Default_Backend_Port_in_the_Upstream_Host_Header_with_Envoy_Gateway_on_ACP.md @@ -14,7 +14,7 @@ id: KB260800008 ## Issue -An Envoy Gateway route forwards traffic to an HTTPS backend that listens on a non-default port: +The following `HTTPRoute` forwards traffic to an HTTPS backend that listens on a non-default port: ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 @@ -28,11 +28,7 @@ spec: - fqdn: hostname: model.example.com port: 7448 -``` - -The route also uses backend hostname rewriting, either directly or through a resource generated by Envoy AI Gateway: - -```yaml +--- apiVersion: gateway.envoyproxy.io/v1alpha1 kind: HTTPRouteFilter metadata: @@ -42,8 +38,40 @@ spec: urlRewrite: hostname: type: Backend +--- +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: model-route + namespace: model-serving +spec: + parentRefs: + - group: gateway.networking.k8s.io + kind: Gateway + name: model-gateway + sectionName: http + rules: + - matches: + - path: + type: PathPrefix + value: /v1 + - path: + type: PathPrefix + value: /openai + filters: + - type: ExtensionRef + extensionRef: + group: gateway.envoyproxy.io + kind: HTTPRouteFilter + name: rewrite-host-to-backend + backendRefs: + - group: gateway.envoyproxy.io + kind: Backend + name: external-model ``` +The patch later in this document targets rule index `0` of the `model-serving/model-route` HTTPRoute and modifies every match generated from that rule. It does not modify another rule or another HTTPRoute. The same method applies to an HTTPRoute generated by Envoy AI Gateway; use the generated HTTPRoute namespace and name in the selector. + Envoy connects to backend port `7448`, but the upstream HTTP `Host` header or HTTP/2 `:authority` contains only the hostname: ```text @@ -73,7 +101,9 @@ Envoy Gateway: v1.8.0 Envoy Proxy: v1.38.0 ``` -`EnvoyPatchPolicy` modifies generated xDS configuration and is an unstable advanced API. For Envoy Gateway v1.7.x, v1.9.x, or another version, inspect that version's generated RouteConfiguration and adjust the resource name and route indexes before applying the policy. +`EnvoyPatchPolicy.operation.jsonPath` is supported starting with Envoy Gateway v1.2.0. This document uses `jsonPath` to select a generated Envoy route by name instead of depending on `virtual_hosts` or `routes` array indexes. + +`EnvoyPatchPolicy` modifies generated xDS configuration and is an unstable advanced API. For Envoy Gateway v1.7.x, v1.9.x, or another version, inspect that version's generated RouteConfiguration and adjust the resource name and route-name selector before applying the policy. ## Root Cause @@ -114,7 +144,7 @@ Use this workaround when the upstream must receive exactly: Host: model.example.com:7448 ``` -#### 1. Enable EnvoyPatchPolicy +#### Prerequisite: Enable EnvoyPatchPolicy Merge the following field into the existing `EnvoyGatewayCtl`. Do not replace its other configuration. @@ -140,56 +170,225 @@ spec: Do not directly edit the ConfigMap or Deployment generated by `EnvoyGatewayCtl`. -#### 2. Identify the Values to Replace +#### Step 1: Find the Listener Used by the HTTPRoute -The PatchPolicy targets the Gateway and modifies the generated Envoy RouteConfiguration. Replace the following example values: +Start with the namespace and name of the Gateway and HTTPRoute resources: -| Example value | Meaning | Action | -| --- | --- | --- | -| `model-serving` | Gateway namespace | `CHANGE` | -| `model-gateway` | Gateway name | `CHANGE` | -| `http` | Gateway listener name | `CHANGE` | -| `model-serving/model-gateway/http` | RouteConfiguration name: `//` | `CHANGE` | -| First `0` in the path | Virtual-host array index | `MAY CHANGE` | -| Second `0` in the path | Route array index containing the affected HTTPRoute | `MAY CHANGE` | -| `model.example.com:7448` | Host or authority required by the backend | `CHANGE` | +| Variable | Source | +| --- | --- | +| `VAR_$GATEWAY_NAMESPACE` | Namespace of the parent Gateway | +| `VAR_$GATEWAY_NAME` | Name of the parent Gateway | +| `VAR_$HTTPROUTE_NAMESPACE` | `HTTPRoute.metadata.namespace` | +| `VAR_$HTTPROUTE_NAME` | `HTTPRoute.metadata.name` | + +Replace the HTTPRoute placeholders and inspect the complete resource, including its status: + +```bash +kubectl get httproute VAR_$HTTPROUTE_NAME \ + -n VAR_$HTTPROUTE_NAMESPACE \ + -o yaml +``` + +In `spec.parentRefs`, find the entry whose `name` and `namespace` match `VAR_$GATEWAY_NAME` and `VAR_$GATEWAY_NAMESPACE`. If `namespace` is omitted, it defaults to the HTTPRoute namespace. The matching entry's `sectionName` is `VAR_$LISTENER_NAME`: + +```yaml +spec: + parentRefs: + - group: gateway.networking.k8s.io + kind: Gateway + name: model-gateway + sectionName: http +``` + +Next, find the corresponding entry in `status.parents` and confirm that its `Accepted` condition is `True`. The `parentRef.sectionName` in status should identify the same listener: + +```yaml +status: + parents: + - parentRef: + group: gateway.networking.k8s.io + kind: Gateway + name: model-gateway + namespace: model-serving + sectionName: http + conditions: + - type: Accepted + status: "True" +``` + +For this example, `VAR_$LISTENER_NAME` is `http`. The matching `parentRef` must contain `sectionName`, because that field identifies the exact Gateway listener used by the HTTPRoute. If it is absent, stop and make the listener attachment explicit before creating the PatchPolicy. Do not guess a listener. + +For the xDS naming scheme used by this validated configuration, construct the RouteConfiguration resource name as: + +```text +VAR_$GATEWAY_NAMESPACE/VAR_$GATEWAY_NAME/VAR_$LISTENER_NAME +``` + +For the example resources, the result is: + +```text +model-serving/model-gateway/http +``` + +#### Step 2: Create the EnvoyPatchPolicy + +Treat one rule in one HTTPRoute as the exact patch target. In addition to the four resource inputs from Step 1, choose the zero-based `VAR_$RULE_INDEX` from `HTTPRoute.spec.rules` and set `VAR_$UPSTREAM_AUTHORITY` to the required upstream `host:port`. + +The PatchPolicy example contains documentation placeholders beginning with `VAR_$`. They are not shell variables, and Kubernetes does not replace them. Replace every placeholder before applying the file: + +| Placeholder | Replace with | +| --- | --- | +| `VAR_$GATEWAY_NAMESPACE` | Supplied Gateway namespace | +| `VAR_$GATEWAY_NAME` | Supplied Gateway name | +| `VAR_$LISTENER_NAME` | Listener name returned by Step 1 | +| `VAR_$HTTPROUTE_NAMESPACE` | Supplied HTTPRoute namespace | +| `VAR_$HTTPROUTE_NAME` | Supplied HTTPRoute name | +| `VAR_$RULE_INDEX` | Rule index selected from `HTTPRoute.spec.rules` | +| `VAR_$UPSTREAM_AUTHORITY` | Required upstream authority, for example `model.example.com:7448` | -The `targetRef` must remain a `Gateway`. The HTTPRoute name is not placed directly in `targetRef`; it determines which generated route index must be patched. +Before applying the file, this command must print no output: -If the Gateway contains one virtual host and one route, the indexes are commonly `0` and `0`. For Gateways with multiple routes, use the Envoy configuration dump to locate the route containing `auto_host_rewrite: true` and use its indexes. +```bash +grep -n 'VAR_\$' upstream-authority-with-port.yaml +``` + +`EnvoyPatchPolicy.targetRef` attaches the policy to the Gateway. The `jsonPath` expression then selects every generated Envoy route belonging to the specified HTTPRoute rule. + +Rule index `0` in the example has two matches, so it generates route names similar to: + +```yaml +name: httproute/model-serving/model-route/rule/0/match/0/* +route: + auto_host_rewrite: true +--- +name: httproute/model-serving/model-route/rule/0/match/1/* +route: + auto_host_rewrite: true +``` + +The generated route-name pattern is: + +```text +httproute/VAR_$HTTPROUTE_NAMESPACE/VAR_$HTTPROUTE_NAME/rule/VAR_$RULE_INDEX/match/VAR_$MATCH_INDEX/VAR_$HOSTNAME +``` + +Each component means: + +| Envoy route name component | Source | +| --- | --- | +| `httproute` | Resource type | +| `VAR_$HTTPROUTE_NAMESPACE` | HTTPRoute namespace | +| `VAR_$HTTPROUTE_NAME` | HTTPRoute name | +| `rule/VAR_$RULE_INDEX` | Exact HTTPRoute rule to patch | +| `match/VAR_$MATCH_INDEX` | One match generated from that rule | +| `VAR_$HOSTNAME` | Hostname represented by the generated Envoy route | + +Use this selector to target all matches generated from the specified rule: + +```text +$.virtual_hosts[*].routes[?match(@.name, '^httproute/VAR_$HTTPROUTE_NAMESPACE/VAR_$HTTPROUTE_NAME/rule/VAR_$RULE_INDEX/match/[0-9]+/.*$')] +``` + +The JSONPath syntax is evaluated as follows: -#### 3. Apply the PatchPolicy +| Syntax | Meaning | +| --- | --- | +| `$` | Start at the root of the Envoy RouteConfiguration JSON document | +| `.virtual_hosts` | Read the `virtual_hosts` array | +| `[*]` | Search every virtual host because the target route can appear in any of them | +| `.routes` | Read the Envoy routes in each virtual host | +| `[? ... ]` | Keep only array entries for which the filter expression is true | +| `@` | The current Envoy route being evaluated | +| `@.name` | The generated name of the current Envoy route | +| `match(@.name, '...')` | Keep the route when its name matches the regular expression | +| `^httproute/.../rule/VAR_$RULE_INDEX/` | Anchor the match to the exact HTTPRoute and rule | +| `match/[0-9]+/` | Match every numeric match index in that rule, such as `match/0` and `match/1` | +| `.*$` | Match any generated hostname suffix and require the match to reach the end of the route name | + +The `^` and `$` anchors are important: together they prevent a partial name match from selecting another HTTPRoute or rule. Adding or reordering other HTTPRoutes does not change the selection. Reordering rules within the target HTTPRoute changes `VAR_$RULE_INDEX` and requires updating the policy. + +Because the policy attaches to the Gateway, `EnvoyPatchPolicy.metadata.namespace` must use `VAR_$GATEWAY_NAMESPACE`, even when the HTTPRoute is in another namespace. + +Create `upstream-authority-with-port.yaml` from this template after replacing every `VAR_$...` placeholder: ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 # KEEP kind: EnvoyPatchPolicy # KEEP metadata: name: upstream-authority-with-port # CHANGE: any valid resource name - namespace: model-serving # CHANGE: must be the Gateway namespace + namespace: VAR_$GATEWAY_NAMESPACE spec: targetRef: group: gateway.networking.k8s.io # KEEP kind: Gateway # KEEP - name: model-gateway # CHANGE: Gateway name + name: VAR_$GATEWAY_NAME type: JSONPatch # KEEP jsonPatches: - type: type.googleapis.com/envoy.config.route.v3.RouteConfiguration # KEEP - name: model-serving/model-gateway/http # CHANGE: namespace/Gateway/listener + name: VAR_$GATEWAY_NAMESPACE/VAR_$GATEWAY_NAME/VAR_$LISTENER_NAME operation: op: remove # KEEP - path: /virtual_hosts/0/routes/0/route/auto_host_rewrite # MAY CHANGE: only the two indexes + jsonPath: >- + $.virtual_hosts[*].routes[?match(@.name, '^httproute/VAR_$HTTPROUTE_NAMESPACE/VAR_$HTTPROUTE_NAME/rule/VAR_$RULE_INDEX/match/[0-9]+/.*$')].route.auto_host_rewrite - type: type.googleapis.com/envoy.config.route.v3.RouteConfiguration # KEEP - name: model-serving/model-gateway/http # CHANGE: same value as above + name: VAR_$GATEWAY_NAMESPACE/VAR_$GATEWAY_NAME/VAR_$LISTENER_NAME operation: op: add # KEEP - path: /virtual_hosts/0/routes/0/route/host_rewrite_literal # MAY CHANGE: same two indexes as above - value: model.example.com:7448 # CHANGE: required backend authority + jsonPath: >- + $.virtual_hosts[*].routes[?match(@.name, '^httproute/VAR_$HTTPROUTE_NAMESPACE/VAR_$HTTPROUTE_NAME/rule/VAR_$RULE_INDEX/match/[0-9]+/.*$')] + path: route/host_rewrite_literal # KEEP + value: VAR_$UPSTREAM_AUTHORITY ``` +##### Complete Example After Replacing the Placeholders + +For example, assume the target values are: + +| Value | Example | +| --- | --- | +| Gateway | `model-serving/model-gateway` | +| HTTPRoute | `model-serving/model-route` | +| Listener returned by Step 1 | `http` | +| HTTPRoute rule index | `0` | +| Required upstream Host or authority | `model.example.com:7448` | + +The completed PatchPolicy is: + +```yaml +apiVersion: gateway.envoyproxy.io/v1alpha1 +kind: EnvoyPatchPolicy +metadata: + name: upstream-authority-with-port + namespace: model-serving +spec: + targetRef: + group: gateway.networking.k8s.io + kind: Gateway + name: model-gateway + type: JSONPatch + jsonPatches: + - type: type.googleapis.com/envoy.config.route.v3.RouteConfiguration + name: model-serving/model-gateway/http + operation: + op: remove + jsonPath: >- + $.virtual_hosts[*].routes[?match(@.name, '^httproute/model-serving/model-route/rule/0/match/[0-9]+/.*$')].route.auto_host_rewrite + - type: type.googleapis.com/envoy.config.route.v3.RouteConfiguration + name: model-serving/model-gateway/http + operation: + op: add + jsonPath: >- + $.virtual_hosts[*].routes[?match(@.name, '^httproute/model-serving/model-route/rule/0/match/[0-9]+/.*$')] + path: route/host_rewrite_literal + value: model.example.com:7448 +``` + +This policy attaches to `model-serving/model-gateway`, modifies every generated match route under rule `0` of `model-serving/model-route`, and leaves other HTTPRoutes and other rules unchanged. + Both patch operations are required: -- The first operation removes `auto_host_rewrite`. -- The second operation adds `host_rewrite_literal` with the required port. +- The first operation selects `auto_host_rewrite` on every Envoy route generated from the target rule and removes it. +- The second operation selects those same Envoy route objects and adds `host_rewrite_literal` with the required port. Because the new field does not exist yet, `jsonPath` selects the route object and `path` identifies the field to add. Apply the policy: @@ -227,7 +426,7 @@ This confirms that the PatchPolicy changed the upstream Host or authority to the Delete only the PatchPolicy: ```bash -kubectl delete envoypatchpolicy upstream-authority-with-port -n model-serving +kubectl delete envoypatchpolicy upstream-authority-with-port -n VAR_$GATEWAY_NAMESPACE ``` Envoy Gateway will regenerate the route with its original `auto_host_rewrite` behavior. The Gateway, HTTPRoute, Backend, and Envoy Gateway instance do not need to be deleted. @@ -235,7 +434,9 @@ Envoy Gateway will regenerate the route with its original `auto_host_rewrite` be ## Limitations - The exact patch in this document is validated for Envoy Gateway v1.8.0. -- Route indexes can change after adding, removing, or reordering HTTPRoutes or after upgrading Envoy Gateway. +- `jsonPath` is supported starting with Envoy Gateway v1.2.0. +- Adding or reordering other HTTPRoutes does not affect the route-name selector. Reordering rules within the target HTTPRoute requires updating `VAR_$RULE_INDEX`; reordering matches does not require an update because the selector matches all numeric match indexes. +- An Envoy Gateway upgrade may change RouteConfiguration or generated route naming. Recheck the resource name and route-name selector after an upgrade. - Recheck `Accepted` and `Programmed` after changing Gateway or route resources. - Prefer port-independent upstream virtual-host matching when that configuration is under your control. @@ -243,6 +444,7 @@ Envoy Gateway will regenerate the route with its original `auto_host_rewrite` be - [ACP Envoy Gateway Operator: Advanced Config Via EnvoyGatewayCtl](https://docs-dev.alauda.cn/container_platform/main/networking/operators/envoy_gateway_operator#envoygatewayctl) - [Envoy Gateway v1.8: Envoy Patch Policy](https://gateway.envoyproxy.io/v1.8/tasks/extensibility/envoy-patch-policy/) +- [Envoy Gateway PR #3757: Support JSONPath in EnvoyPatchPolicy](https://github.com/envoyproxy/gateway/pull/3757) - [Envoy issue #26022: auto_host_rewrite port number loss](https://github.com/envoyproxy/envoy/issues/26022) - [Envoy Gateway issue #8823: Support for Port in auto_host_rewrite](https://github.com/envoyproxy/gateway/issues/8823) - [Envoy AI Gateway issue #2500: Generated Backend host rewrite drops non-default port](https://github.com/envoyproxy/ai-gateway/issues/2500)