Skip to content

Architecture

Overview

Envoy Gateway is an open-source project that manages Envoy Proxy as a standalone or Kubernetes-based application gateway. It translates Kubernetes Gateway API resources into Envoy xDS configuration and manages the lifecycle of a fleet of Envoy proxies. Unlike full service meshes (Istio, Linkerd), Envoy Gateway focuses on north-south traffic (ingress and egress at the edge).

See also: index, security, operations

1. Core Components

graph TB
    subgraph "Control Plane"
        GWAPI["Gateway API Resources<br/>(GatewayClass, Gateway, HTTPRoute, GRPCRoute, TLSRoute)"]
        EG["Envoy Gateway Controller"]
        RP["Resource Provider<br/>(gateway-api runner)"]
        XDS["xDS Translator"]
        IM["Infra Manager<br/>(infrastructure runner)"]
        XS["xDS Server"]
        RL["Global RateLimit Translator"]
    end

    subgraph "Data Plane"
        EP1["Envoy Proxy"]
        EP2["Envoy Proxy"]
        EP3["Envoy Proxy"]
    end

    subgraph "External"
        RLS["Rate Limit Service<br/>(envoyproxy/ratelimit)"]
        REDIS["Redis Backend"]
        K8s["Kubernetes API Server"]
    end

    K8s -->|watches Gateway API resources| RP
    RP -->|translates to IR| XDS
    XDS -->|produces xDS config| XS
    XS -->|xDS gRPC stream| EP1
    XS -->|xDS gRPC stream| EP2
    XS -->|xDS gRPC stream| EP3
    IM -->|provisions/deploys| EP1
    IM -->|provisions/deploys| EP2
    IM -->|provisions/deploys| EP3
    RL -->|rate limit xDS| XS
    RLS --> REDIS

Resource Provider (gateway-api runner)

The Resource Provider watches Kubernetes Gateway API resources (GatewayClass, Gateway, HTTPRoute, GRPCRoute, TLSRoute, TCPRoute, UDPRoute, BackendTLSPolicy) and translates them into an internal Intermediate Representation (IR). This decouples the Gateway API watch logic from Envoy-specific translation.

xDS Translator

The xDS Translator consumes the IR produced by the Resource Provider and generates native Envoy xDS configuration (Listener, Cluster, Route, Endpoint, Secret resources). This is the core translation engine that maps declarative Gateway API intent to Envoy's data plane configuration.

Infra Manager (infrastructure runner)

The Infra Manager is responsible for provisioning and managing the Envoy proxy fleet infrastructure. On Kubernetes this means creating and managing Deployments, Services, and ConfigMaps for the managed Envoy proxies. It handles scaling, updates, and teardown of proxy infrastructure in response to Gateway API resource changes.

xDS Server

The xDS Server runs a gRPC server that streams xDS configuration to managed Envoy proxies. It uses the standard Envoy xDS protocol (incremental xDS, SotW, or delta xDS) and supports connection keepalive settings. Envoy proxies connect to this server to receive their configuration updates.

Global RateLimit Translator

When global rate limiting is enabled, a dedicated translator generates xDS configuration for the Envoy rate limit filter. The rate limit service uses the reference implementation from envoyproxy/ratelimit backed by Redis for shared state across proxy instances.

Watching Components Design

All core runners (Resource Provider, xDS Translator, Infra Manager, xDS Server, Global RateLimit Translator) follow a "Watching Components" pattern. Each runner exposes metrics including watchable_depth, watchable_subscribe_duration_seconds, and watchable_publish_total to monitor internal event processing health.

2. xDS Configuration Flow

sequenceDiagram
    participant User as Cluster Operator
    participant K8s as Kubernetes API
    participant RP as Resource Provider
    participant XT as xDS Translator
    participant XS as xDS Server
    participant EP as Envoy Proxy

    User->>K8s: kubectl apply GatewayClass, Gateway, HTTPRoute
    RP->>K8s: Watch Gateway API resources
    K8s-->>RP: Resource change events
    RP->>RP: Translate to Intermediate Representation (IR)
    RP->>XT: Publish IR
    XT->>XT: Translate IR to xDS (LDS, RDS, CDS, EDS, SDS)
    XT->>XS: Publish xDS resources
    XS->>EP: Stream xDS via gRPC (incremental)
    EP->>EP: Hot-reload listeners, routes, clusters
    EP-->>User: Traffic routed per Gateway API rules

Translation Hierarchy

EnvoyProxy configuration follows a priority hierarchy (highest to lowest):

  1. Gateway-level EnvoyProxy -- referenced via Gateway.spec.infrastructure.parametersRef
  2. GatewayClass-level EnvoyProxy -- referenced via GatewayClass.spec.parametersRef
  3. Default EnvoyProxy spec -- defined in the EnvoyGateway configuration

Currently the most specific configuration wins completely (replace semantics). A future release will introduce merge semantics.

3. Envoy Proxy Fleet

Each managed Envoy proxy instance runs as a Kubernetes Deployment. Key characteristics:

  • Bootstrap configuration is generated by Envoy Gateway and injected into each proxy via a ConfigMap
  • Dynamic configuration is delivered via xDS streaming from the control plane
  • Proxy lifecycle (create, update, delete) is managed by the Infra Manager
  • Prometheus metrics are exposed at /stats/prometheus on each proxy for observability
  • Ready endpoint is available for health checking

4. Rate Limiting Architecture

Envoy Gateway supports two rate limiting modes:

Mode Scope Backend Configuration
Local Per-proxy instance In-memory RateLimitFilter in HTTPRoute
Global Across all proxies Redis + envoyproxy/ratelimit EnvoyGateway ConfigMap + RateLimitFilter

For global rate limiting, the rateLimit field in the EnvoyGateway configuration specifies the backend type (Redis) and connection details. The RateLimitFilter CRD defines the actual rate limit rules applied to HTTPRoute traffic.

5. Extension Points

Envoy Gateway provides several extension mechanisms:

  • Extension Manager -- register an external gRPC extension server that can modify xDS configuration after translation. !!! warning "Enabling an Extension Server may lead to complete security compromise of your system. Users that control the Extension Server can inject arbitrary configuration to proxies."
  • EnvoyPatchPolicy -- directly patch generated xDS resources for advanced customization
  • AuthenticationFilter -- configure JWT authentication, OIDC, or external authorization (ExtAuth)
  • Backend CRD -- define custom backend endpoints with TLS settings including mTLS

6. Debugging with egctl

The egctl CLI tool provides several debugging capabilities:

  • egctl x translate --from gateway-api --to xds -- translate Gateway API manifests to xDS for offline inspection
  • egctl config envoy-proxy route -- dump live xDS route configuration from managed proxies
  • egctl config envoy-proxy cluster -- dump live cluster configuration
  • egctl config envoy-proxy listener -- dump live listener configuration

Key Insight

Envoy Gateway separates the concerns of resource watching, xDS translation, infrastructure management, and xDS serving into distinct runners. This modular architecture makes each component independently testable and observable.

How It Works

Gateway API to xDS translation, Envoy fleet lifecycle, IR (Intermediate Representation) pipeline, and policy attachment model.

Request Flow

sequenceDiagram
    participant Client as External Client
    participant LB as Load Balancer
    participant Envoy as Envoy Proxy Pod
    participant EG as Envoy Gateway Controller
    participant K8sAPI as K8s API
    participant Backend as Backend Service

    Note over EG,K8sAPI: Startup / config change
    EG->>K8sAPI: Watch Gateway, HTTPRoute, Policies
    EG->>EG: Translate to xDS config
    EG->>Envoy: Push xDS (gRPC stream)

    Note over Client,Backend: Runtime request
    Client->>LB: HTTPS request
    LB->>Envoy: Forward
    Envoy->>Envoy: TLS termination
    Envoy->>Envoy: Route matching (HTTPRoute rules)
    Envoy->>Envoy: Apply policies (auth, rate limit)
    Envoy->>Backend: Forward to matched backend
    Backend-->>Client: Response

Translation Pipeline

Envoy Gateway translates Kubernetes Gateway API resources into Envoy xDS configuration through a multi-stage pipeline:

flowchart LR
    K8s["K8s Resources\n(Gateway, HTTPRoute,\nSecurityPolicy, ...)"] --> Infra_IR["Infra IR\n(Envoy deployment,\nService, ConfigMap)"]
    K8s --> XDS_IR["xDS IR\n(Listeners, Routes,\nClusters, Secrets)"]
    Infra_IR --> Infra_Mgr["Infrastructure Manager\n(Provision Envoy pods)"]
    XDS_IR --> XDS_Translator["xDS Translator\n(Generate Envoy config)"]
    XDS_Translator --> Envoy_Proxy["Envoy Proxy Fleet"]

    style K8s fill:#7b42bc,color:#fff

IR (Intermediate Representation)

Envoy Gateway uses an Intermediate Representation to decouple Kubernetes-specific resource parsing from Envoy xDS generation:

  • Infra IR: Describes the managed Envoy infrastructure (Deployment replicas, Service type/ports, ConfigMap mounts). The infrastructure manager reconciles this into Kubernetes resources.
  • xDS IR: Describes the listener, route, cluster, and endpoint configuration in a provider-agnostic format. The xDS translator converts this into Envoy's protobuf-based xDS resources.

This separation allows Envoy Gateway to potentially support non-Kubernetes deployment targets (e.g., standalone Envoy) by replacing only the infrastructure manager layer.

Envoy Fleet Lifecycle

The controller manages a fleet of Envoy proxy pods:

  1. Provisioning: Creates a Deployment of Envoy pods based on the GatewayClass configuration. Default: 2 replicas.
  2. Configuration push: Opens a gRPC xDS stream to each Envoy pod. Uses incremental xDS to push only changed resources.
  3. Health checking: Monitors Envoy pod readiness via /ready endpoint on port 19001.
  4. Scaling: The Envoy Deployment can be scaled manually or via HPA. All replicas receive the same xDS configuration.

Policy Attachment Model

flowchart TB
    GC["GatewayClass"] --> GW["Gateway\n(listener config)"]
    GW --> HR["HTTPRoute\n(path/header routing)"]
    HR --> Backend_E["Backend Service"]

    CTP["ClientTrafficPolicy"] -.->|"attach to"| GW
    SP_E["SecurityPolicy\n(JWT, OIDC, mTLS)"] -.->|"attach to"| GW
    BTP_E["BackendTrafficPolicy\n(LB, circuit break)"] -.->|"attach to"| HR
    EEP["EnvoyExtensionPolicy\n(Wasm, Ext Proc)"] -.->|"attach to"| HR

    style GW fill:#7b42bc,color:#fff

Policies are attached via targetRef or targetSelectors fields that reference Gateway API resources. When a policy targets a Gateway, it applies to all routes on that Gateway. When it targets an HTTPRoute, it applies only to that route's traffic.

xDS Resources Generated

Gateway API Resource xDS Resources Generated
Gateway (listener) Listener, FilterChain, TLS context
HTTPRoute (rules) Route, VirtualHost, Cluster, Endpoint
SecurityPolicy (JWT) HTTP Filter (JWT auth), Route-level per-route config
SecurityPolicy (mTLS) TLS context with client certificate
BackendTrafficPolicy Cluster load balancing policy, circuit breaker, health check
RateLimitFilter Rate limit HTTP filter + global rate limit service config

Sources

Benchmarks

Scope

Performance characteristics, scaling limits, and resource consumption for Envoy Gateway.

Gateway API Performance

Metric Value Conditions
Throughput (HTTP/1.1) 30,000-50,000 RPS Single gateway, small payloads
Throughput (HTTP/2) 40,000-70,000 RPS Multiplexed connections
Latency (P50) 0.5-1ms Added by proxy
Latency (P99) 2-5ms Under moderate load

Scaling

Gateways Routes Controller Memory Controller CPU
1 50 128Mi 100m
5 200 256Mi 200m
20 1,000 512Mi 500m

Envoy Proxy Resources

Traffic Level Proxy CPU Proxy Memory
Low (< 1k RPS) 100m 64Mi
Medium (1-10k RPS) 500m 256Mi
High (10-100k RPS) 2+ 1Gi+

Sourcing Status

Unsourced Performance Data

The performance numbers in this document are estimated from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.

Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.

Sources