Skip to content

Architecture

Overview

Envoy Gateway is an open-source project that manages Envoy Proxy as a standalone or Kubernetes-based application gateway. It translates Kubernetes Gateway API resources into Envoy xDS configuration and manages the lifecycle of a fleet of Envoy proxies. Unlike full service meshes (Istio, Linkerd), Envoy Gateway focuses on north-south traffic (ingress and egress at the edge).

See also: index, service-mesh/envoy-gateway/explanation, service-mesh/envoy-gateway/how-to-guides

1. Core Components

graph TB
    subgraph "Control Plane"
        GWAPI["Gateway API Resources<br/>(GatewayClass, Gateway, HTTPRoute, GRPCRoute, TLSRoute)"]
        EG["Envoy Gateway Controller"]
        RP["Resource Provider<br/>(gateway-api runner)"]
        XDS["xDS Translator"]
        IM["Infra Manager<br/>(infrastructure runner)"]
        XS["xDS Server"]
        RL["Global RateLimit Translator"]
    end

    subgraph "Data Plane"
        EP1["Envoy Proxy"]
        EP2["Envoy Proxy"]
        EP3["Envoy Proxy"]
    end

    subgraph "External"
        RLS["Rate Limit Service<br/>(envoyproxy/ratelimit)"]
        REDIS["Redis Backend"]
        K8s["Kubernetes API Server"]
    end

    K8s -->|watches Gateway API resources| RP
    RP -->|translates to IR| XDS
    XDS -->|produces xDS config| XS
    XS -->|xDS gRPC stream| EP1
    XS -->|xDS gRPC stream| EP2
    XS -->|xDS gRPC stream| EP3
    IM -->|provisions/deploys| EP1
    IM -->|provisions/deploys| EP2
    IM -->|provisions/deploys| EP3
    RL -->|rate limit xDS| XS
    RLS --> REDIS

Resource Provider (gateway-api runner)

The Resource Provider watches Kubernetes Gateway API resources (GatewayClass, Gateway, HTTPRoute, GRPCRoute, TLSRoute, TCPRoute, UDPRoute, BackendTLSPolicy) and translates them into an internal Intermediate Representation (IR). This decouples the Gateway API watch logic from Envoy-specific translation.

xDS Translator

The xDS Translator consumes the IR produced by the Resource Provider and generates native Envoy xDS configuration (Listener, Cluster, Route, Endpoint, Secret resources). This is the core translation engine that maps declarative Gateway API intent to the Envoy data plane configuration.

Infra Manager (infrastructure runner)

The Infra Manager is responsible for provisioning and managing the Envoy proxy fleet infrastructure. On Kubernetes this means creating and managing Deployments, Services, and ConfigMaps for the managed Envoy proxies. It handles scaling, updates, and teardown of proxy infrastructure in response to Gateway API resource changes.

xDS Server

The xDS Server runs a gRPC server that streams xDS configuration to managed Envoy proxies. It uses the standard Envoy xDS protocol (incremental xDS, SotW, or delta xDS) and supports connection keepalive settings. Envoy proxies connect to this server to receive their configuration updates.

Global RateLimit Translator

When global rate limiting is enabled, a dedicated translator generates xDS configuration for the Envoy rate limit filter. The rate limit service uses the reference implementation from envoyproxy/ratelimit backed by Redis for shared state across proxy instances.

Watching Components Design

All core runners (Resource Provider, xDS Translator, Infra Manager, xDS Server, Global RateLimit Translator) follow a "Watching Components" pattern. Each runner exposes metrics including watchable_depth, watchable_subscribe_duration_seconds, and watchable_publish_total to monitor internal event processing health.

2. xDS Configuration Flow

sequenceDiagram
    participant User as Cluster Operator
    participant K8s as Kubernetes API
    participant RP as Resource Provider
    participant XT as xDS Translator
    participant XS as xDS Server
    participant EP as Envoy Proxy

    User->>K8s: kubectl apply GatewayClass, Gateway, HTTPRoute
    RP->>K8s: Watch Gateway API resources
    K8s-->>RP: Resource change events
    RP->>RP: Translate to Intermediate Representation (IR)
    RP->>XT: Publish IR
    XT->>XT: Translate IR to xDS (LDS, RDS, CDS, EDS, SDS)
    XT->>XS: Publish xDS resources
    XS->>EP: Stream xDS via gRPC (incremental)
    EP->>EP: Hot-reload listeners, routes, clusters
    EP-->>User: Traffic routed per Gateway API rules

Translation Hierarchy

EnvoyProxy configuration follows a priority hierarchy (highest to lowest):

  1. Gateway-level EnvoyProxy -- referenced via Gateway.spec.infrastructure.parametersRef
  2. GatewayClass-level EnvoyProxy -- referenced via GatewayClass.spec.parametersRef
  3. Default EnvoyProxy spec -- defined in the EnvoyGateway configuration

Currently the most specific configuration wins completely (replace semantics). A future release will introduce merge semantics.

3. Envoy Proxy Fleet

Each managed Envoy proxy instance runs as a Kubernetes Deployment. Key characteristics:

  • Bootstrap configuration is generated by Envoy Gateway and injected into each proxy via a ConfigMap
  • Dynamic configuration is delivered via xDS streaming from the control plane
  • Proxy lifecycle (create, update, delete) is managed by the Infra Manager
  • Prometheus metrics are exposed at /stats/prometheus on each proxy for observability
  • Ready endpoint is available for health checking

4. Rate Limiting Architecture

Envoy Gateway supports two rate limiting modes:

Mode Scope Backend Configuration
Local Per-proxy instance In-memory RateLimitFilter in HTTPRoute
Global Across all proxies Redis + envoyproxy/ratelimit EnvoyGateway ConfigMap + RateLimitFilter

For global rate limiting, the rateLimit field in the EnvoyGateway configuration specifies the backend type (Redis) and connection details. The RateLimitFilter CRD defines the actual rate limit rules applied to HTTPRoute traffic.

5. Extension Points

Envoy Gateway provides several extension mechanisms:

  • Extension Manager -- register an external gRPC extension server that can modify xDS configuration after translation. !!! warning "Enabling an Extension Server may lead to complete security compromise of your system. Users that control the Extension Server can inject arbitrary configuration to proxies."
  • EnvoyPatchPolicy -- directly patch generated xDS resources for advanced customization
  • AuthenticationFilter -- configure JWT authentication, OIDC, or external authorization (ExtAuth)
  • Backend CRD -- define custom backend endpoints with TLS settings including mTLS

6. Debugging with egctl

The egctl CLI tool provides several debugging capabilities:

  • egctl x translate --from gateway-api --to xds -- translate Gateway API manifests to xDS for offline inspection
  • egctl config envoy-proxy route -- dump live xDS route configuration from managed proxies
  • egctl config envoy-proxy cluster -- dump live cluster configuration
  • egctl config envoy-proxy listener -- dump live listener configuration

Key Insight

Envoy Gateway separates the concerns of resource watching, xDS translation, infrastructure management, and xDS serving into distinct runners. This modular architecture makes each component independently testable and observable.

How It Works

Gateway API to xDS translation, Envoy fleet lifecycle, IR (Intermediate Representation) pipeline, and policy attachment model.

Request Flow

sequenceDiagram
    participant Client as External Client
    participant LB as Load Balancer
    participant Envoy as Envoy Proxy Pod
    participant EG as Envoy Gateway Controller
    participant K8sAPI as K8s API
    participant Backend as Backend Service

    Note over EG,K8sAPI: Startup / config change
    EG->>K8sAPI: Watch Gateway, HTTPRoute, Policies
    EG->>EG: Translate to xDS config
    EG->>Envoy: Push xDS (gRPC stream)

    Note over Client,Backend: Runtime request
    Client->>LB: HTTPS request
    LB->>Envoy: Forward
    Envoy->>Envoy: TLS termination
    Envoy->>Envoy: Route matching (HTTPRoute rules)
    Envoy->>Envoy: Apply policies (auth, rate limit)
    Envoy->>Backend: Forward to matched backend
    Backend-->>Client: Response

Translation Pipeline

Envoy Gateway translates Kubernetes Gateway API resources into Envoy xDS configuration through a multi-stage pipeline:

flowchart LR
    K8s["K8s Resources\n(Gateway, HTTPRoute,\nSecurityPolicy, ...)"] --> Infra_IR["Infra IR\n(Envoy deployment,\nService, ConfigMap)"]
    K8s --> XDS_IR["xDS IR\n(Listeners, Routes,\nClusters, Secrets)"]
    Infra_IR --> Infra_Mgr["Infrastructure Manager\n(Provision Envoy pods)"]
    XDS_IR --> XDS_Translator["xDS Translator\n(Generate Envoy config)"]
    XDS_Translator --> Envoy_Proxy["Envoy Proxy Fleet"]

    style K8s fill:#7b42bc,color:#fff

IR (Intermediate Representation)

Envoy Gateway uses an Intermediate Representation to decouple Kubernetes-specific resource parsing from Envoy xDS generation:

  • Infra IR: Describes the managed Envoy infrastructure (Deployment replicas, Service type/ports, ConfigMap mounts). The infrastructure manager reconciles this into Kubernetes resources.
  • xDS IR: Describes the listener, route, cluster, and endpoint configuration in a provider-agnostic format. The xDS translator converts this into the protobuf-based xDS resources of Envoy.

This separation allows Envoy Gateway to potentially support non-Kubernetes deployment targets (for example, standalone Envoy) by replacing only the infrastructure manager layer.

Envoy Fleet Lifecycle

The controller manages a fleet of Envoy proxy pods:

  1. Provisioning: Creates a Deployment of Envoy pods based on the GatewayClass configuration. Default: 2 replicas.
  2. Configuration push: Opens a gRPC xDS stream to each Envoy pod. Uses incremental xDS to push only changed resources.
  3. Health checking: Monitors Envoy pod readiness via /ready endpoint on port 19001.
  4. Scaling: The Envoy Deployment can be scaled manually or via HPA. All replicas receive the same xDS configuration.

Policy Attachment Model

flowchart TB
    GC["GatewayClass"] --> GW["Gateway\n(listener config)"]
    GW --> HR["HTTPRoute\n(path/header routing)"]
    HR --> Backend_E["Backend Service"]

    CTP["ClientTrafficPolicy"] -.->|"attach to"| GW
    SP_E["SecurityPolicy\n(JWT, OIDC, mTLS)"] -.->|"attach to"| GW
    BTP_E["BackendTrafficPolicy\n(LB, circuit break)"] -.->|"attach to"| HR
    EEP["EnvoyExtensionPolicy\n(Wasm, Ext Proc)"] -.->|"attach to"| HR

    style GW fill:#7b42bc,color:#fff

Policies are attached via targetRef or targetSelectors fields that reference Gateway API resources. When a policy targets a Gateway, it applies to all routes on that Gateway. When it targets an HTTPRoute, it applies only to the traffic of that route.

xDS Resources Generated

Gateway API Resource xDS Resources Generated
Gateway (listener) Listener, FilterChain, TLS context
HTTPRoute (rules) Route, VirtualHost, Cluster, Endpoint
SecurityPolicy (JWT) HTTP Filter (JWT auth), Route-level per-route config
SecurityPolicy (mTLS) TLS context with client certificate
BackendTrafficPolicy Cluster load balancing policy, circuit breaker, health check
RateLimitFilter Rate limit HTTP filter + global rate limit service config

Sources

Benchmarks

Scope

Performance characteristics, scaling limits, and resource consumption for Envoy Gateway.

Gateway API Performance

Metric Value Conditions
Throughput (HTTP/1.1) 30,000-50,000 RPS Single gateway, small payloads
Throughput (HTTP/2) 40,000-70,000 RPS Multiplexed connections
Latency (P50) 0.5-1ms Added by proxy
Latency (P99) 2-5ms Under moderate load

Scaling

Gateways Routes Controller Memory Controller CPU
1 50 128Mi 100m
5 200 256Mi 200m
20 1,000 512Mi 500m

Envoy Proxy Resources

Traffic Level Proxy CPU Proxy Memory
Low (< 1k RPS) 100m 64Mi
Medium (1-10k RPS) 500m 256Mi
High (10-100k RPS) 2+ 1Gi+

Sourcing Status

Unsourced Performance Data

The performance numbers in this document are estimated from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.

Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.

Sources

Security

Overview

Envoy Gateway provides north-south security at the edge of the cluster. It handles TLS termination for incoming client traffic, TLS origination to backend services, JWT authentication, rate limiting, and CORS. Unlike full service meshes, it does not manage east-west mTLS between in-cluster services.

See also: index, service-mesh/envoy-gateway/explanation, service-mesh/envoy-gateway/how-to-guides

1. TLS Termination

Envoy Gateway terminates TLS at the gateway proxy for incoming client connections. Configuration is done through standard Gateway API resources.

graph LR
    Client["Client<br/>(HTTPS)"] -->|TLS terminated| EG["Envoy Gateway Proxy"]
    EG -->|plaintext or TLS| Backend["Backend Service"]

Configuration

TLS certificates are stored as Kubernetes Secrets and referenced in the Gateway listener configuration:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg
spec:
  gatewayClassName: eg
  listeners:
  - name: https
    protocol: HTTPS
    port: 8443
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: example-cert

Key points:

  • mode: Terminate -- the gateway decrypts client TLS and forwards plaintext to backends
  • mode: Passthrough -- TLS is passed through to the backend. The gateway does not decrypt
  • Supports multiple certificates per listener via SNI (Server Name Indication)

2. Backend TLS (Gateway to Backend)

The BackendTLSPolicy resource configures TLS between the gateway and backend services. This enables end-to-end encryption even when the gateway terminates client TLS.

apiVersion: gateway.networking.k8s.io/v1alpha3
kind: BackendTLSPolicy
metadata:
  name: example-backend-tls
spec:
  targetRefs:
  - group: ""
    kind: Service
    name: backend-service
  validation:
    hostname: backend.example.com
    caCertificateRefs:
    - name: backend-ca
      group: ""
      kind: ConfigMap
    subjectAltNames:
    - san.backend.example.com

Validation fields:

Field Purpose
hostname SNI hostname the gateway sends when connecting to the backend
caCertificateRefs References to CA certificates for validating backend certificates
wellKnownCACertificates Use system CA certificates instead of custom refs
subjectAltNames SANs that the backend certificate must match

3. Backend mTLS

For backends that require mutual TLS (client certificates from the gateway), Envoy Gateway provides the Backend CRD:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: Backend
metadata:
  name: mtls-backend
spec:
  endpoints:
  - fqdn:
      hostname: secure-backend.default.svc.cluster.local
      port: 443
  tls:
    clientCertificateRef:
      kind: Secret
      name: gateway-client-cert
    caCertificateRefs:
    - group: ""
      kind: ConfigMap
      name: backend-ca

This configures the gateway to present a client certificate and validate the certificate of the backend. This achieves full mTLS on the gateway-to-backend leg.

4. JWT Authentication

Envoy Gateway supports JWT validation through the SecurityPolicy CRD. This validates JWT tokens in incoming requests before forwarding them to backends.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: jwt-auth
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: backend
  jwt:
    providers:
    - name: example
      remoteJWKS:
        uri: https://YOUR_DOMAIN/.well-known/jwks.json
      claimToHeaders:
      - claim: sub
        header: x-subject

Features:

  • Remote JWKS verification via HTTPS
  • Inline JWKS for air-gapped environments
  • Claim-to-header extraction for passing identity to backends
  • Multiple provider support for different audiences/issuers
  • recomputeRoute option for claim-based routing after JWT validation

5. External Authorization (ExtAuth)

For custom authentication logic, Envoy Gateway supports delegating authorization decisions to an external service via gRPC or HTTP. This is configured within the SecurityPolicy CRD:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: ext-auth
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: backend
  extAuth:
    grpc:
      backendRefs:
      - name: auth-service
        port: 9000
        namespace: auth
      backendTLSPolicyRef:
        name: ext-auth-tls

External Authorization Security

The external auth service receives all request headers. Make sure that the auth service is trusted and that the communication channel is secured with BackendTLSPolicy.

6. Rate Limiting

Rate limiting provides defense against application-layer abuse and DDoS attacks. Envoy Gateway supports two modes:

Local Rate Limiting

Applied per-proxy instance with in-memory counters. No external backend required. Configured via RateLimitFilter in HTTPRoute rule filters.

Global Rate Limiting

Uses a shared Redis backend and the envoyproxy/ratelimit service for distributed rate limiting across all proxy instances:

# EnvoyGateway configuration
rateLimit:
  backend:
    type: Redis
    redis:
      url: redis.redis-system.svc.cluster.local:6379

7. CORS

CORS (Cross-Origin Resource Sharing) is configured on HTTPRoute resources to control which origins, methods, and headers are permitted for browser-based clients. Envoy Gateway translates CORS settings into Envoy CORS filter configuration.

8. Security Policy Summary

Layer Mechanism Resource
Client-to-Gateway TLS TLS termination Gateway listener with certificateRefs
Gateway-to-Backend TLS Backend TLS origination BackendTLSPolicy
Gateway-to-Backend mTLS Client certificate + CA validation Backend CRD with tls config
JWT validation Token verification SecurityPolicy (jwt section)
Custom auth External authorization SecurityPolicy (extAuth section)
Rate limiting Per-instance or global RateLimitFilter + EnvoyGateway config
CORS Origin filtering HTTPRoute CORS settings

Scope Note

Envoy Gateway handles north-south (ingress) security. For east-west (service-to-service) mTLS within the cluster, use a service mesh like Istio or Linkerd alongside Envoy Gateway. See service-mesh/istio/explanation or service-mesh/linkerd/explanation.