Skip to content
Edit page
Copy page

Deploy AI Gateway on OpenShift in standalone mode

This guide explains how to run API Platform AI Gateway on Red Hat OpenShift with the gateway Helm chart only, without the Gateway Operator. The AI Gateway runs the same gateway chart and images as API Platform Gateway, so the deployment steps are the same; the AI-specific steps are in Step 8 and AI traffic on OpenShift. It's intended for platform engineers who deploy and operate the gateway.

Gateway artifacts can be deployed on OpenShift using the same Helm charts, Docker images, and configuration model as standard Kubernetes deployments. By default, OpenShift may assign a random UID to the container process within the project's permitted range. Containers typically run with GID 0 (the root group), allowing applications to access files and directories with appropriate group permissions without requiring root privileges. When deploying gateway artifacts, ensure that the pod and container security contexts configured through the Helm chart values are compatible with the Security Context Constraints (SCCs) enforced by your OpenShift cluster. Review the Security context section for configuration guidance.

For more information about OpenShift container security and file permissions, see Red Hat's guide to OpenShift and UIDs and Group ownership and file permission. To manage gateways through custom resources instead, see Deploy on OpenShift with the Gateway Operator. For standard Kubernetes, see Kubernetes standalone mode.

Before you begin

Make sure you have the following before you deploy:

  • WSO2 subscription credentials: the chart and images come from the WSO2 private registry (registry.wso2.com). You use them in Step 5.
  • An at-rest encryption key Secret: the chart refuses to install without it. You create it in Step 5.
  • cert-manager, or your own TLS Secret: the chart uses it for the gateway's HTTPS listener certificate.
  • A StorageClass: the gateway controller stores its state in a PersistentVolumeClaim (PVC). Either the cluster has a default StorageClass, or you set one explicitly.

Deploy the gateway

Step 1: Install the required tools

The following table lists the tools this guide uses:

Tool Purpose Install guide
oc OpenShift CLI for managing cluster resources Install the OpenShift CLI
helm (v3) Package manager for deploying the gateway Helm chart Install Helm
openssl Generates the at-rest encryption key Preinstalled on most systems

Verify that the tools are installed:

oc version
helm version
openssl version

Step 2: Log in to OpenShift

  1. Authenticate with the OpenShift CLI:

    oc login <API_SERVER_URL> -u <USERNAME>

    oc prompts for the password, which keeps it out of your shell history and the process list.

    oc login <API_SERVER_URL> --token=<TOKEN>

  2. Verify your connection and check the selected project:

    # Verify the connection
oc whoami

# Check the selected project
oc project

  3. Create a dedicated project for the gateway:

    oc new-project <PROJECT-NAME>

    Don't use the default project. A dedicated project isolates the gateway's resources and applies the project's security policies to it. Creating a project also creates the Kubernetes namespace with the same name and switches the current context to it. The rest of this guide uses <PROJECT-NAME> for that namespace.

Step 3: Install cert-manager

The gateway chart uses cert-manager to issue the TLS certificate for the gateway's HTTPS listener. Skip this step if cert-manager is already installed on the cluster.

  1. Install cert-manager:

    helm upgrade --install \
  cert-manager oci://quay.io/jetstack/charts/cert-manager \
  --version v1.19.1 \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true \
  --wait --timeout 10m

  2. Verify that the cert-manager pods are running:

    oc get pods -n cert-manager

Note

You can use the cert-manager Operator for Red Hat OpenShift instead of the Helm chart. To skip cert-manager entirely, provide your own TLS Secret by setting gateway.controller.tls.certificateProvider=secret and gateway.controller.tls.secret.name=<SECRET_NAME>.

Step 4: Review the gateway runtime image

The chart's default gateway runtime image runs on OpenShift as is, so you don't need to build an image. Images that the gateway builder creates with custom policies start from the released image and inherit the same permissions. Set such an image in gateway.gatewayRuntime.image in Step 6, as you would on Kubernetes.

The gateway runtime runs three processes in one container: the Router (Envoy), the Policy Engine, and the Python Executor. They talk to each other over Unix domain sockets in /var/run/api-platform. The image makes that directory writable by the root group, so the processes can create the sockets with whichever UID OpenShift runs them as.

Building a gateway runtime image another way

If you build a gateway runtime image another way, such as from a different base image, make /var/run/api-platform writable by the root group. Add these lines to the Dockerfile after any step that creates or changes the directory, and apply the same pattern to any other directory that the image writes to outside a mounted volume:

USER root
RUN chgrp -R 0 /var/run/api-platform && chmod -R g=u /var/run/api-platform
USER 10001

If you can't change the image, mount a writable emptyDir volume over the directory instead:

gateway:
  gatewayRuntime:
    deployment:
      extraVolumes:
        - name: runtime-sockets
          emptyDir:
            medium: Memory
            sizeLimit: 1Mi
      extraVolumeMounts:
        - name: runtime-sockets
          mountPath: /var/run/api-platform

Step 5: Create the encryption key and image pull Secrets

The gateway controller encrypts sensitive data at rest with an AES-256 key. The chart doesn't install unless this Secret exists and the values reference it.

  1. Generate the key and create the Secret:

    openssl rand 32 > default-aesgcm256-v1.bin
oc create secret generic gateway-encryption-keys \
  --from-file=default-aesgcm256-v1.bin=default-aesgcm256-v1.bin \
  -n <PROJECT-NAME> && \
  rm default-aesgcm256-v1.bin   # remove the plaintext key only after the Secret is created

  2. Verify that the Secret exists:

    oc get secret gateway-encryption-keys -n <PROJECT-NAME>

  3. Log Helm in to the WSO2 private registry, and create an image pull secret so the cluster can pull the gateway images:

    helm registry login registry.wso2.com
oc create secret docker-registry wso2-subscription-creds \
  --docker-server=registry.wso2.com \
  --docker-username=<wso2-username> \
  --docker-password=<wso2-password> \
  -n <PROJECT-NAME>

Note

  • Create both Secrets in the same project that you install the gateway into.
  • Name the key entry exactly default-aesgcm256-v1.bin.
  • Keep a secure backup of the key. You can't recover data encrypted with it if you lose the key.

Step 6: Install the gateway chart

  1. Update your values.yaml file with the following sections. To start from the chart's defaults, export them first:

    helm show values oci://registry.wso2.com/wso2-api-platform/sts/helm-charts/gateway --version 2026.10.0 > values.yaml

    Image pull secret: reference the pull secret from Step 5:

    wso2:
  subscription:
    imagePullSecret: wso2-subscription-creds

    Encryption key: reference the Secret from Step 5:

    gateway:
  controller:
    encryptionKeys:
      enabled: true
      secretName: gateway-encryption-keys

    Management API Route (optional): the chart can create a Route for the gateway controller's management REST API. It's off by default because it's an administrative interface:

    openshift:
  enabled: true
  routes:
    managementApi:
      enabled: false
      # Leave empty to let OpenShift generate <route-name>-<project>.<apps-domain>
      hostname: ""

    To turn it on, set enabled: true and replace the default admin password first. The chart refuses to create the Route while the management API still accepts the default password. For example, generate a bcrypt hash with htpasswd -nBC 10 admin | cut -d: -f2, which prompts for the new password, and set it:

    gateway:
  config:
    controller:
      auth:
        basic:
          users:
            - username: admin
              password: "<BCRYPT_HASH>"
              password_hashed: true
              roles: ["admin"]

    Gateway runtime Service type: the Routes that you create in Step 7 send traffic to the Service inside the cluster, so you don't need a LoadBalancer Service, which is the chart default. Keep LoadBalancer only if you expose the gateway runtime without Routes (see Routing API traffic):

    gateway:
  gatewayRuntime:
    service:
      type: ClusterIP

    Storage class: skip this section if your cluster has a default StorageClass. Check with oc get storageclass.

    gateway:
  controller:
    persistence:
      storageClass: "<STORAGE_CLASS>"

    Security context (recommended): add values that suit your SCC. See the samples in Security context.

  2. Install the chart:

    helm install gateway oci://registry.wso2.com/wso2-api-platform/sts/helm-charts/gateway \
  --version 2026.10.0 \
  --namespace <PROJECT-NAME> \
  -f values.yaml

  3. Wait for the pods to be ready:

    oc get pods -n <PROJECT-NAME> -w

    Both the gateway-controller and gateway-runtime pods must show 1/1 Running before you continue.

Step 7: Create a Route for API traffic

The chart doesn't create Routes for API traffic. The hostnames that the gateway serves depend on the virtual hosts configured on the gateway and in each API definition, as described in Routing API traffic. With the default configuration, the gateway's main virtual host accepts any hostname, so one Route with a generated hostname serves every API that doesn't set its own virtual hosts.

  1. Create the Route:

    oc create route passthrough gateway-runtime \
  --service=gateway-gateway-runtime \
  --port=https \
  -n <PROJECT-NAME>

    With passthrough termination, the router forwards the encrypted connection, and the gateway terminates TLS with its own certificate. For other termination types, see Create a Route.

  2. List the Routes:

    oc get route -n <PROJECT-NAME>

  3. Confirm that the output has the Route that you created. If you enabled it, the output also has the management API Route with edge termination:

    NAME                     HOST/PORT                                           SERVICES                  PORT    TERMINATION
gateway-runtime          gateway-runtime-<PROJECT-NAME>.<APPS_DOMAIN>        gateway-gateway-runtime   https   passthrough
gateway-management-api   gateway-management-api-<PROJECT-NAME>.<APPS_DOMAIN> gateway-controller        rest    edge/Redirect

To reach the sandbox environment of an API, or to serve APIs on their own hostnames, create more Routes as described in Routing API traffic.

Step 8: Deploy and invoke a test LLM proxy

A healthy pod isn't proof that LLM traffic works. Deploy one provider and one proxy, then call them through the gateway runtime Route.

  1. Get the gateway runtime Route's hostname, and forward the controller's management API port:

    GATEWAY_HOST=$(oc get route gateway-runtime -n <PROJECT-NAME> -o jsonpath='{.spec.host}')
oc port-forward svc/gateway-controller 9090:9090 -n <PROJECT-NAME> &

    If you enabled the management API Route in Step 6, you can use it instead of the port-forward. Get its hostname, then in the next steps use $MGMT_HOST in place of localhost in the netrc file, and https://$MGMT_HOST in place of http://localhost:9090. If your machine doesn't trust the router's certificate, also add -k to those curl commands:

    MGMT_HOST=$(oc get route gateway-management-api -n <PROJECT-NAME> -o jsonpath='{.spec.host}')

  2. Write the controller admin credentials to a netrc file. A password passed to curl -u is visible in the host's process list, and a netrc file only you can read keeps it out:

    install -m 600 /dev/null controller.netrc
read -rp "Controller admin username: " ADMIN_USERNAME
read -rsp "Controller admin password: " ADMIN_PASSWORD && echo
printf 'machine %s login %s password %s\n' localhost "$ADMIN_USERNAME" "$ADMIN_PASSWORD" > controller.netrc
unset ADMIN_PASSWORD

    admin and admin are the chart's default management API credentials. Change them through gateway.config.controller.auth.basic.users for any deployment beyond evaluation.

  3. Deploy an LLM provider. Read the provider key with read -rs, which keeps it off the screen and out of your shell history:

    read -rsp "OpenAI API key: " OPENAI_API_KEY && echo

curl --fail --show-error -X POST http://localhost:9090/api/management/v1/llm-providers \
  -H "Content-Type: application/yaml" \
  --netrc-file controller.netrc \
  --data-binary @- <<EOF
apiVersion: gateway.api-platform.wso2.com/v1
kind: LlmProvider
metadata:
  name: openai-provider
spec:
  displayName: OpenAI Provider
  version: v1.0
  template: openai
  context: /openai/latest
  upstream:
    url: https://api.openai.com/v1
    auth:
      type: api-key
      header: Authorization
      value: Bearer ${OPENAI_API_KEY}
  accessControl:
    mode: deny_all
    exceptions:
      - path: /chat/completions
        methods: [POST]
EOF

unset OPENAI_API_KEY

  4. Deploy a proxy that consumes it, then remove the credentials file:

    curl --fail --show-error -X POST http://localhost:9090/api/management/v1/llm-proxies \
  -H "Content-Type: application/yaml" \
  --netrc-file controller.netrc \
  --data-binary @- <<'EOF'
apiVersion: gateway.api-platform.wso2.com/v1
kind: LlmProxy
metadata:
  name: openai-assistant
spec:
  displayName: OpenAI Assistant
  version: v1.0
  context: /assistant
  provider:
    id: openai-provider
  policies: []
EOF

shred -u controller.netrc

  5. Route a request through the gateway runtime Route. Generated hostnames under the cluster's apps domain resolve through the cluster's wildcard DNS record. If you created the Route with a custom --hostname, create a DNS record that points it at the cluster's router first:

    curl -k -X POST "https://$GATEWAY_HOST/assistant/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hi"}]
  }'

  6. Confirm streaming works end to end, since streaming is what exposes timeout misconfiguration between the router, the gateway, and the provider:

    curl -k -N -X POST "https://$GATEWAY_HOST/assistant/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "stream": true,
    "messages": [{"role": "user", "content": "Write a short paragraph about APIs."}]
  }'

    Chunks should arrive progressively. Through an edge or reencrypt Route, a stream cut off after about 30 seconds means the Route's timeout is still at the router default. See Raise the Route timeouts for streaming responses.

For the full verification, including identity-provider tokens instead of basic auth, see Deploy and verify.

Advanced configuration

Security context

The chart exposes the pod and container security contexts of each component as free-form values, and renders them exactly as you provide them:

  • gateway.controller.deployment.podSecurityContext and gateway.controller.deployment.securityContext
  • gateway.gatewayRuntime.deployment.podSecurityContext and gateway.gatewayRuntime.deployment.securityContext

Both are empty by default. The following table describes the properties that matter on OpenShift:

Property Description
runAsNonRoot Rejects the pod if it would run as root.
runAsUser, fsGroup Set only under SCCs that don't assign them. runAsUser lets the kubelet verify runAsNonRoot for the named image user, and fsGroup makes mounted volumes writable.
seccompProfile Required by restricted-v2, nonroot-v2, and the restricted Pod Security Standard. Not allowed by anyuid.
allowPrivilegeEscalation The gateway never needs to escalate privileges.
capabilities The gateway listens only on ports above 1024 and needs no Linux capabilities, so all of them can be dropped.

The values that OpenShift admits depend on the SCC that the pods run under. For the strategies that each SCC applies to these properties, see Managing security context constraints in the OpenShift documentation.

The following samples show values for common SCCs. The default SCC for workloads, restricted-v2, assigns the UID, the fsGroup, and the seccomp profile itself. SCCs such as anyuid and nonroot-v2 assign none of them, so the values must set the IDs.

Sample values for both components. Don't set runAsUser, runAsGroup, or fsGroup: restricted-v2 assigns them from the project's range and rejects a pod that requests other values (see Troubleshooting).

gateway:
  controller:
    deployment:
      podSecurityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
  gatewayRuntime:
    deployment:
      podSecurityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]

If you reuse a values file written for standard Kubernetes that sets fixed IDs, override them with null, for example runAsUser: null, runAsGroup: null, and fsGroup: null.

Sample values for both components, with the image's UID (10001) and an fsGroup. The gateway images name their user (wso2) instead of using a number, so the kubelet can verify runAsNonRoot only when runAsUser is set. The fsGroup makes the controller's volume writable.

gateway:
  controller:
    deployment:
      podSecurityContext:
        runAsNonRoot: true
        runAsUser: 10001
        fsGroup: 10001
        seccompProfile:          # leave out under anyuid
          type: RuntimeDefault
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
  gatewayRuntime:
    deployment:
      podSecurityContext:
        runAsNonRoot: true
        runAsUser: 10001
        fsGroup: 10001
        seccompProfile:          # leave out under anyuid
          type: RuntimeDefault
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]

anyuid doesn't allow any seccomp profile, so leave out seccompProfile under it. Grant the SCC to the gateway's service account, for example oc adm policy add-scc-to-user anyuid -z <SERVICE_ACCOUNT> -n <PROJECT-NAME>.

Routing API traffic

The OpenShift router forwards a request only when its hostname belongs to a Route. The gateway then picks the API from the request's Host header and path. A hostname works end to end only when both of the following are true:

  • A Route with that hostname points to the gateway runtime Service, gateway-gateway-runtime.
  • The hostname matches a virtual host that the gateway serves.

The chart doesn't create these Routes, because you choose the hostnames, and they can change with every API that you deploy. Create them with oc, as described in this section.

How the gateway chooses hostnames

Every API has a main and a sandbox environment, each with its own upstream: spec.upstream.main and spec.upstream.sandbox. The gateway serves them on two virtual hosts, whose defaults you set at the gateway level:

Virtual host Default Setting
Main * (any hostname) gateway.config.router.vhosts.main.default
Sandbox sandbox-* (hostnames that start with sandbox-) gateway.config.router.vhosts.sandbox.default

A non-empty domains list, gateway.config.router.vhosts.main.domains or gateway.config.router.vhosts.sandbox.domains, replaces the default pattern with the listed hostnames.

An API can override these values in its definition:

spec:
  vhosts:
    main: orders.example.com              # one or more hostnames, separated by ";"
    sandbox: orders-sandbox.example.com   # one hostname

The gateway serves an API that sets its own virtual host only on that hostname. It doesn't serve APIs that use the gateway-level defaults on that hostname.

Create one Route for each hostname that clients use, whether the hostname comes from the gateway-level settings or from an API definition.

Create a Route

Set the hostname with --hostname. If you leave it out, OpenShift generates <ROUTE-NAME>-<PROJECT-NAME>.<APPS_DOMAIN>, which the default * main virtual host accepts.

The router forwards the encrypted connection to the gateway, which terminates TLS with its own certificate. Clients see the gateway's certificate, so add custom hostnames to gateway.controller.tls.certManager.dnsNames:

oc create route passthrough <ROUTE-NAME> \
  --service=gateway-gateway-runtime \
  --port=https \
  --hostname=<HOSTNAME> \
  -n <PROJECT-NAME>

The router terminates the client's TLS connection with its own certificate and opens a second TLS connection to the gateway. The router uses the cluster's default certificate unless you pass one. The router must trust the gateway's certificate, so pass the CA that issued it. With the default cert-manager setup, extract the CA from the gateway's TLS Secret:

oc get secret gateway-controller-tls -n <PROJECT-NAME> \
  -o jsonpath='{.data.ca\.crt}' | base64 -d > gateway-ca.crt

oc create route reencrypt <ROUTE-NAME> \
  --service=gateway-gateway-runtime \
  --port=https \
  --hostname=<HOSTNAME> \
  --dest-ca-cert=gateway-ca.crt \
  -n <PROJECT-NAME>

For a hostname outside the cluster's apps domain, also pass the certificate that the router serves: --cert=tls.crt --key=tls.key --ca-cert=ca.crt.

The router terminates TLS and forwards plain HTTP to the gateway's http port (8080):

oc create route edge <ROUTE-NAME> \
  --service=gateway-gateway-runtime \
  --port=http \
  --hostname=<HOSTNAME> \
  --insecure-policy=Redirect \
  -n <PROJECT-NAME>

For a hostname outside the cluster's apps domain, also pass --cert, --key, and --ca-cert.

Use passthrough or reencrypt with the https port, and edge with the http port. With any other combination, the router returns 503.

For a hostname outside the cluster's apps domain, create a DNS record that points it at the cluster's router.

Route timeouts

On edge and reencrypt Routes, the OpenShift router closes a connection when the gateway sends nothing for 30 seconds, by default. For long-running requests, such as streaming LLM responses, raise the timeout on those Routes. passthrough Routes use the router's tunnel timeout, one hour by default, so they don't need it:

oc annotate route <ROUTE-NAME> haproxy.router.openshift.io/timeout=120s -n <PROJECT-NAME>

Sandbox traffic

With the default sandbox virtual host, sandbox-*, give the sandbox Route a name that starts with sandbox-, so that the generated hostname matches:

oc create route passthrough sandbox-gateway \
  --service=gateway-gateway-runtime \
  --port=https \
  -n <PROJECT-NAME>

For an API that defines a sandbox upstream, such as the following:

spec:
  upstream:
    main:
      url: https://httpbin.org/anything/main
    sandbox:
      url: https://httpbin.org/anything/sandbox

Invoke each environment through its Route:

MAIN_HOST=$(oc get route gateway-runtime -n <PROJECT-NAME> -o jsonpath='{.spec.host}')
SANDBOX_HOST=$(oc get route sandbox-gateway -n <PROJECT-NAME> -o jsonpath='{.spec.host}')

curl -k "https://$MAIN_HOST/<CONTEXT>/<VERSION>/<PATH>"      # main upstream
curl -k "https://$SANDBOX_HOST/<CONTEXT>/<VERSION>/<PATH>"   # sandbox upstream

A sandbox request to an API without a sandbox upstream returns 404.

Custom sandbox hostname

A sandbox hostname must be a sandbox virtual host. It either starts with sandbox-, such as sandbox-api.example.com, is listed in gateway.config.router.vhosts.sandbox.domains, or is the API's own spec.vhosts.sandbox. With the default main virtual host, *, the gateway serves any other hostname as main traffic. For example, sandbox.api.example.com reaches the main upstream unless you list it in sandbox.domains.

Generated hostname length

The first label of a generated hostname, <ROUTE-NAME>-<PROJECT-NAME>, must be 63 characters or fewer. If the names are long, pass --hostname instead.

Many hostnames

If your APIs use many hostnames, or hostnames that change often, use one of the following options instead of one Route per hostname:

  • A wildcard Route serves every hostname under one domain. The cluster's ingress controller must allow wildcard Routes, which requires cluster-admin access:

    oc patch ingresscontroller/default -n openshift-ingress-operator --type=merge \
  -p '{"spec":{"routeAdmission":{"wildcardPolicy":"WildcardsAllowed"}}}'

# Serves *.apis.example.com
oc create route reencrypt apis-wildcard \
  --service=gateway-gateway-runtime \
  --port=https \
  --hostname=wildcard.apis.example.com \
  --wildcard-policy=Subdomain \
  --dest-ca-cert=gateway-ca.crt \
  -n <PROJECT-NAME>

    The Route serves one level of subdomain, such as orders.apis.example.com. It doesn't serve apis.example.com or v2.orders.apis.example.com. Create a wildcard DNS record, *.apis.example.com, that points at the router, and use a router certificate that covers the wildcard.

  • A LoadBalancer Service sends traffic straight to the gateway without the router, so every hostname works without a Route. Set gateway.gatewayRuntime.service.type: LoadBalancer, which is the chart default, and point the DNS records at the load balancer. This option needs a load balancer for the cluster, such as the cloud provider's load balancer or MetalLB.

AI traffic on OpenShift

LLM and MCP traffic needs a few settings beyond a standard gateway deployment. Tune the gateway for AI traffic covers the gateway side: streaming timeouts, body buffers, guardrail limits, pricing data, and semantic caching. On OpenShift, three things also need attention.

Raise the Route timeouts for streaming responses

The gateway allows long completions and long gaps between stream chunks, but on edge and reencrypt Routes the OpenShift router closes a connection that is idle for 30 seconds by default. A streaming response that pauses longer than that between chunks is cut off at the router, whatever the gateway allows. The client still gets a 200 status with a truncated body.

passthrough Routes, such as the one in Step 7, aren't affected: the router forwards their TLS connections as tunnels, which use the tunnel timeout of one hour by default. Raise haproxy.router.openshift.io/timeout-tunnel on them only if a stream can stay idle for longer than that.

Raise the timeout on every edge or reencrypt Route that carries LLM or MCP traffic, to at least the gateway's route and stream idle timeouts:

oc annotate route <ROUTE-NAME> -n <PROJECT-NAME> \
  haproxy.router.openshift.io/timeout=10m

Then confirm that a streamed response arrives progressively through the Route, as in Step 8.

Supply your own pricing data

To replace the bundled LLM pricing data, as described in Supply pricing data for token cost tracking, create the ConfigMap in the gateway's project:

oc create configmap llm-model-prices \
  --namespace <PROJECT-NAME> \
  --from-file=model_prices.json=./model_prices.json

Then set gateway.gatewayRuntime.policies.llmPricing.configMapName: llm-model-prices in your values, as that page shows.

Connect to AI Workspace

To govern the gateway from AI Workspace, follow Connect to AI Workspace, with these OpenShift details:

  • The control plane address. When AI Workspace runs on OpenShift, its Platform API is exposed through a Route on port 443, created by the ai-workspace chart. Use that Route's host in gateway.controller.controlPlane.host, as <PLATFORM_API_ROUTE_HOST>:443. Find it with:

    oc get route -n <AI_WORKSPACE_PROJECT> \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.host}{"\n"}{end}' | grep platform-api

    A gateway in the same cluster can use the Platform API's Service instead, as <release>-platform-api.<AI_WORKSPACE_PROJECT>.svc:9243, if no NetworkPolicy blocks the traffic between the two projects.

  • Certificate trust. The Platform API Route uses passthrough termination by default, so the gateway sees the Platform API's own certificate. That certificate must chain to a CA the gateway trusts, as Connect to AI Workspace explains. Leave insecure_skip_verify at false.

  • The registration token. Store it in a Secret in the gateway's project, with oc in place of kubectl:

    install -m 600 /dev/null token.txt
read -rsp "Gateway registration token: " CP_TOKEN && echo
printf '%s' "$CP_TOKEN" > token.txt
unset CP_TOKEN

oc create secret generic gateway-cp-token \
  --namespace <PROJECT-NAME> \
  --from-file=token=./token.txt \
  && shred -u token.txt

  • Reachability. Confirm the gateway's project can reach the control plane address before you configure the chart:

    oc run -n <PROJECT-NAME> conn-test --rm -it --restart=Never --image=curlimages/curl -- \
  curl -fsS https://<PLATFORM_API_ROUTE_HOST>/health

Storage

The gateway controller stores its state in SQLite on a PVC mounted at /app/data. The pod's fsGroup makes the volume writable. Depending on the SCC, OpenShift assigns the fsGroup or you set it in the security context values (see Security context).

The following table lists the storage settings:

Setting Default Description
gateway.controller.persistence.enabled true Set to false to use an emptyDir volume instead of a PVC. The data doesn't survive pod restarts, so use this for evaluation only.
gateway.controller.persistence.storageClass "" An empty value uses the cluster's default StorageClass. Set it explicitly if the cluster has no default.
gateway.controller.persistence.size 100Mi Size of the PVC.
gateway.controller.persistence.existingClaim "" Name of a PVC that you created, to use instead of creating one.

Note

  • Use a storage class whose driver supports fsGroup. Most block storage CSI drivers do. For storage that ignores fsGroup, such as NFS, make the exported directory group-writable by a group that you add through podSecurityContext.supplementalGroups.
  • SQLite with a ReadWriteOnce volume supports a single controller replica. On a multi-node cluster, set gateway.controller.deployment.strategy.type: Recreate. This stops an upgrade from starting the replacement pod on a different node while the previous pod still holds the volume.
  • For multiple controller replicas, use an external PostgreSQL or SQL Server database through gateway.config.controller.storage.type. See Database configuration.

Read-only root filesystem

No default SCC requires a read-only root filesystem. You need one only if your cluster enforces it through a custom SCC, or if you turn it on for hardening. For the SCC setting, see Managing security context constraints in the OpenShift documentation.

With readOnlyRootFilesystem: true, a container can't write anywhere inside its image, so every path that a component writes to at runtime must be a mounted volume. Mount a writable emptyDir volume at each such path through the chart's extraVolumes and extraVolumeMounts settings, under gateway.<component>.deployment. An emptyDir volume is writable by the UID that the pod runs as, whatever the SCC. For example:

gateway:
  gatewayRuntime:
    deployment:
      securityContext:
        readOnlyRootFilesystem: true
      extraVolumes:
        - name: <VOLUME_NAME>
          emptyDir: {}
      extraVolumeMounts:
        - name: <VOLUME_NAME>
          mountPath: <WRITABLE_PATH>

Troubleshooting

Gateway runtime fails with "Permission denied" on its socket

Symptom: The gateway-runtime pod is in CrashLoopBackOff, and its logs contain the following lines, or ERROR: Policy Engine exited before creating socket:

[pye] ... Failed to add port to server: No address added out of total 1 resolved for 'unix:/var/run/api-platform/python-executor.sock' (Error in bind for address '/var/run/api-platform/python-executor.sock': Permission denied)
[ent] ... ERROR: Python Executor exited before creating socket

Cause: The gateway runtime can't create its Unix domain sockets in /var/run/api-platform. The image makes this directory writable, so this happens when the root filesystem is read-only and no writable volume is mounted at that path.

Fix:

  1. Check whether the root filesystem is read-only and which volumes are mounted:

    oc get deployment gateway-gateway-runtime -n <PROJECT-NAME> \
  -o jsonpath='{.spec.template.spec.containers[0].securityContext.readOnlyRootFilesystem}{"\n"}{.spec.template.spec.containers[0].volumeMounts}{"\n"}'

  2. Mount a writable emptyDir volume at /var/run/api-platform, as described in Read-only root filesystem, or turn off the read-only root filesystem.

Pods aren't created: "unable to validate against any security context constraint"

Symptom: The Deployment shows 0/1 ready and no pod exists. The ReplicaSet events contain an error such as one of the following:

[... provider restricted-v2: .containers[0].runAsUser: Invalid value: 10001: must be in the ranges: [1000650000, 1000659999] ...]
[... provider anyuid: .spec.securityContext.seccompProfile.type: Forbidden: seccomp may not be set ...]

Cause: The security context requests something that the SCC doesn't allow: a fixed runAsUser, runAsGroup, or fsGroup outside the project's range under restricted-v2, or a seccomp profile under anyuid.

Fix: Use the values for your SCC from Security context. To inspect the events, run:

oc get events -n <PROJECT-NAME> --field-selector reason=FailedCreate

Pods fail with "image has non-numeric user" or "unable to open database file"

Symptom: Under an SCC other than restricted-v2, a pod stays in CreateContainerConfigError with container has runAsNonRoot and image has non-numeric user (wso2), cannot verify user is non-root, or the gateway controller restarts with unable to open database file.

Cause: The SCC doesn't assign a UID or an fsGroup, and the values don't set them.

Fix: Set runAsUser: 10001 and fsGroup: 10001, as shown in Security context.

Gateway controller pod stays pending

Symptom: The gateway-controller pod and its PVC stay Pending.

Fix:

  1. Check the PVC:

    oc get pvc -n <PROJECT-NAME>
oc describe pvc gateway-controller-data -n <PROJECT-NAME>

  2. If no StorageClass is set and the cluster has no default, set gateway.controller.persistence.storageClass as described in Storage.

Gateway controller stays in ContainerCreating with a missing TLS Secret

Symptom: The gateway-controller pod stays in ContainerCreating, and its events contain MountVolume.SetUp failed ... secret "gateway-controller-tls" not found.

Cause: cert-manager hasn't issued the gateway's certificate. It isn't installed, or the Certificate is failing.

Fix:

  1. Check the Certificate and its events:

    oc get certificate -n <PROJECT-NAME>
oc describe certificate -n <PROJECT-NAME>

  2. Make sure that cert-manager is installed and running, as described in Step 3. To use your own certificate instead, set gateway.controller.tls.certificateProvider=secret and gateway.controller.tls.secret.name=<SECRET_NAME>.

Route returns "Application is not available"

Symptom: Requests return the OpenShift router's 503 Application is not available page.

Fix:

  1. Confirm that a Route exists for the hostname that you call. The router returns 503 for hostnames without a Route. Create one, as described in Create a Route.
  2. Confirm that the pods are 1/1 Running. The router sends traffic only to ready pods.

  3. Confirm that the Route points to the gateway-gateway-runtime Service, and that the port matches the termination type: https for passthrough and reencrypt, and http for edge:

    oc get route -n <PROJECT-NAME> -o wide

  4. For reencrypt termination, confirm that --dest-ca-cert is the CA that issued the gateway's current certificate. A wrong or outdated CA results in 503.

Gateway runtime Service shows a pending EXTERNAL-IP

Cause: The chart creates a LoadBalancer Service for the gateway runtime by default. Many on-premises clusters don't provision load balancers.

Fix: Set gateway.gatewayRuntime.service.type: ClusterIP, and create Routes for external access, as described in Routing API traffic.

Next steps