Architecture

Related Notes

secrets/external-secrets-operator/index | secrets/external-secrets-operator/explanation | secrets/external-secrets-operator/how-to-guides |

Overview

External Secrets Operator (ESO) is a Kubernetes operator that synchronizes secrets from external secret management systems into native Kubernetes Secret objects. It extends Kubernetes with Custom Resource Definitions (CRDs) that declare where secrets live and how to synchronize them. The controller fetches secrets from an external API and creates or updates Kubernetes Secrets accordingly.

Component Architecture

graph TD
    subgraph "Kubernetes Cluster"
        MGR["ESO Controller Manager"]
        ESC["ExternalSecret Controller"]
        SSC["SecretStore Controller"]
        WH["Admission Webhooks"]

        ES["ExternalSecret CRD"]
        CSS["ClusterSecretStore CRD"]
        SS["SecretStore CRD"]
        PS["PushSecret CRD"]
        CES["ClusterExternalSecret CRD"]

        K8S["Kubernetes Secret"]

        MGR --> ESC
        MGR --> SSC
        MGR --> WH
        ESC -->|watches| ES
        ESC -->|reads| SS
        ESC -->|reads| CSS
        ESC -->|creates/updates| K8S
        ESC -->|watches| PS
        SSC -->|validates| SS
        SSC -->|validates| CSS
        CES -->|distributes to namespaces| ES
    end

    subgraph "Provider Plugins"
        AWS["AWS Secrets Manager / SSM"]
        VLT["HashiCorp Vault"]
        GCP["GCP Secret Manager"]
        AZ["Azure Key Vault"]
        OP["1Password Connect"]
        DP["Doppler"]
        GL["GitLab"]
        WHK["Webhook (custom HTTP)"]
    end

    ESC -->|fetches secrets| AWS
    ESC -->|fetches secrets| VLT
    ESC -->|fetches secrets| GCP
    ESC -->|fetches secrets| AZ
    ESC -->|fetches secrets| OP
    ESC -->|fetches secrets| DP
    ESC -->|fetches secrets| GL
    ESC -->|fetches secrets| WHK
    PS -->|pushes secrets| AWS
    PS -->|pushes secrets| VLT

Core CRDs

SecretStore

A namespaced resource that defines how to access an external secret provider. It separates the concern of authentication from the secret data itself. A SecretStore contains:

• Provider configuration -- specifies the external API (AWS, Vault, GCP, Azure, and more) and its connection parameters.
• Authentication credentials -- references to Kubernetes Secrets holding provider credentials (API keys, tokens, service account references).
• Retry settings -- optional maxRetries and retryInterval for HTTP connection resilience.
• Controller field -- optional selector to target a specific ESO controller instance when running multiple controllers.

apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: vault-store
  namespace: my-app
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "my-app-role"
          serviceAccountRef:
            name: "my-app-sa"

ClusterSecretStore

A cluster-scoped SecretStore that serves as a centralized gateway to a secret provider. Any ExternalSecret in any namespace can reference it. Cluster operators typically manage ClusterSecretStores to provide shared access to organizational secret backends.

The conditions field on ClusterSecretStore can restrict which namespaces are allowed to reference it. This enables multi-tenancy isolation.

ExternalSecret

A namespaced resource that declares what data to fetch and how to materialize it as a Kubernetes Secret. Key fields:

Field	Purpose
spec.secretStoreRef	References a SecretStore or ClusterSecretStore
spec.refreshInterval	How often to re-sync (default 1 hour)
spec.target	Target Secret name, creation policy, and optional template
spec.data[]	Explicit key-to-remoteRef mappings
spec.dataFrom[]	Bulk extraction of all keys from a remote secret
spec.target.template	Go template for transforming secret data before creation

Creation policies control ownership: - Owner -- ESO creates and manages the Secret lifecycle. Deleting the ExternalSecret deletes the Secret. - Orphan -- ESO creates the Secret but does not delete it when the ExternalSecret is removed. - Merge -- ESO merges values into an existing Secret without overwriting unrelated keys. - None -- ESO does not create the Secret (used with PushSecret or generator patterns).

PushSecret

Reverses the normal flow: pushes data from a Kubernetes Secret out to an external provider. Use cases include: - Syncing certificates generated by cert-manager into Vault or AWS Secrets Manager. - Distributing auto-generated credentials to external systems. - Propagating Secrets across clusters.

PushSecret supports deletionPolicy (delete from provider on removal) and updatePolicy to control how provider-side secrets are managed.

ClusterExternalSecret

Distributes ExternalSecret resources into selected namespaces across the cluster. The controller creates a copy of the ExternalSecret in each matching namespace. This enables bulk distribution of secrets with a single resource.

Secret Sync Flow

sequenceDiagram
    participant User as Developer
    participant K8s as Kubernetes API
    participant Ctrl as ESO Controller
    participant Store as SecretStore
    participant Prov as External Provider

    User->>K8s: Apply ExternalSecret
    K8s->>Ctrl: Watch triggers reconciliation
    Ctrl->>K8s: Read referenced SecretStore
    K8s-->>Ctrl: Return SecretStore config + credentials
    Ctrl->>Ctrl: Instantiate provider client from plugin
    Ctrl->>Prov: Fetch secret data (GetSecret)
    Prov-->>Ctrl: Return secret values
    Ctrl->>Ctrl: Apply template engine (Go templating)
    Ctrl->>K8s: Create or update Kubernetes Secret
    Note over Ctrl: Repeats at spec.refreshInterval

Reconciliation Loop

The controller follows a structured reconciliation process:

1. Validate SecretStore reference -- ESO checks spec.secretStoreRef and verifies the SecretStore exists and its spec.controller field matches (for multi-controller setups).
2. Instantiate provider client -- Using the SecretStore credentials, ESO creates a provider-specific API client from the registered plugin.
3. Fetch secret data -- The controller calls the external API to retrieve the requested secret values, decoding them if required.
4. Apply template engine -- If spec.target.template is defined, the controller processes Go templates against the fetched data. Templates support advanced transformations including base64encode, base64decode, fromJson, toString, and conditional logic.
5. Create or update Secret -- The controller materializes the final Kubernetes Secret, respecting the configured creation policy.
6. Periodic re-sync -- The controller re-queues reconciliation at spec.refreshInterval to detect and apply upstream changes.

Provider Plugin Architecture

ESO uses a Go interface-based plugin system. Each provider implements a common interface that abstracts the external API:

Provider interface:
  - GetSecret(ctx, remoteRef) -> []byte
  - GetSecretMap(ctx, remoteRef) -> map[string][]byte
  - PushSecret(ctx, secret, remoteRef) -> error
  - DeleteSecret(ctx, remoteRef) -> error
  - Validate() -> error

Supported providers (as of 2026):

Category	Providers
AWS	Secrets Manager, SSM Parameter Store
HashiCorp	Vault (KV v1/v2)
Google Cloud	Secret Manager
Microsoft	Azure Key Vault
1Password	1Password Connect
Others	Doppler, GitLab, Pulumi Cloud, Akeyless, CyberArk, Fortanix, Senhasegura, Yandex Lockbox, Oracle Vault, IBM Cloud Secrets Manager, Alibaba Cloud
Custom	Webhook (arbitrary HTTP endpoint)

Template Engine

The template engine processes Go templates defined in spec.target.template. This allows transformation of external secret data before it is written to the Kubernetes Secret.

Supported capabilities: - Variable substitution -- {{ .secretKey }} references fetched values. - Built-in functions -- base64encode, base64decode, fromJson, toJson, toString, toUint, toInt, toFloat, toUpperCase, toLowerCase, trim, regexMatch, regexReplaceAll. - Pipelines and conditionals -- Full Go template control structures (if, range, with). - Data from extract -- dataFrom.extract automatically maps all keys from the remote secret into the template context.

Multi-Controller Deployment

ESO supports running multiple controller instances within a single cluster. Each controller can be scoped to process only SecretStore and ExternalSecret resources with a matching spec.controller field. This enables: - Separation of tenant workloads. - Different provider configurations per controller. - Gradual rollout of controller upgrades.

Multi-Controller Maturity

Running multiple controllers is not widely tested in production. Test thoroughly before relying on this pattern for critical workloads.

Roles and Responsibilities

Role	Responsibility
Cluster Operator	Installs ESO, manages ClusterSecretStores, defines access policies
Application Developer	Defines ExternalSecrets and application configuration

Each role maps roughly to a Kubernetes RBAC role. ESO itself runs with elevated privileges to create and manage Secrets across all namespaces.

Deployment Model

ESO is typically deployed via Helm into a dedicated namespace (external-secrets-system by default):

Namespace: external-secrets-system
  Deployment: external-secrets (single replica by default)
    - Controller manager with leader election for HA
    - Validating and mutating admission webhooks
  ServiceAccount: external-secrets-sa
  CRDs: SecretStore, ClusterSecretStore, ExternalSecret, ClusterExternalSecret, PushSecret

For high availability, increase the replica count. The controller-runtime leader election mechanism makes sure that only one replica actively reconciles at a time.

---

How It Works

Controller reconciliation loop, secret store authentication, provider gRPC interface, and sync lifecycle.

Reconciliation Flow

sequenceDiagram
    participant K8sAPI as K8s API
    participant ESO_H as ESO Controller
    participant Store as SecretStore config
    participant Provider as External Provider (Vault/AWS/GCP)
    participant Secret_H as K8s Secret

    ESO_H->>K8sAPI: Watch ExternalSecret resources
    K8sAPI-->>ESO_H: ExternalSecret created/updated
    ESO_H->>Store: Read SecretStore connection config
    ESO_H->>Provider: Authenticate (SA/IAM/AppRole)
    ESO_H->>Provider: Fetch secret values
    Provider-->>ESO_H: Secret data
    ESO_H->>ESO_H: Apply template (if defined)
    ESO_H->>Secret_H: Create/Update K8s Secret
    Note over ESO_H: Re-sync every refreshInterval (for example, 1h)

SecretStore Authentication Chain

Each SecretStore defines a provider and an auth method. The controller resolves credentials before calling the provider:

Provider	Auth Method	Mechanism
AWS Secrets Manager	JWT (IRSA)	ESO assumes the IAM role of the pod via sts:AssumeRoleWithWebIdentity
AWS Secrets Manager	AccessKey	Static AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY
HashiCorp Vault	Kubernetes	ServiceAccount JWT token sent to Vault's auth/kubernetes/login
HashiCorp Vault	AppRole	RoleID + SecretID sent to auth/approle/login
GCP Secret Manager	Workload Identity	GCP IAM via federated identity
Azure Key Vault	Managed Identity	AAD pod identity or user-assigned managed identity

For ClusterSecretStore, the auth credentials are typically namespace-scoped using auth.targetNamespace references. This enables multi-tenant access from a single store definition.

Provider Interface (Internal)

ESO providers implement a Go interface that abstracts secret operations:

GetSecret(ctx, remoteRef) → (secretValue, error)
GetSecretMap(ctx, remoteRef) → (map[string][]byte, error)
SetSecret(ctx, remoteRef, value) → error

Each provider (Vault, AWS SM, GCP SM, and more) implements this interface. The controller calls the appropriate provider based on the provider field of the SecretStore. New providers can be added without modifying the core reconciliation loop.

Sync Lifecycle

stateDiagram-v2
    [*] --> Pending: ExternalSecret created
    Pending --> SecretSynced: First successful sync
    SecretSynced --> SecretSynced: refreshInterval tick
    SecretSynced --> Error: Provider unreachable / auth failure
    Error --> SecretSynced: Retry succeeds
    Error --> Error: Retry with backoff
    SecretSynced --> Deleted: ExternalSecret deleted
    Deleted --> [*]: Owner policy determines Secret cleanup

Ownership Policies

The target.creationPolicy controls what happens to the underlying K8s Secret:

Policy	Behavior
Owner (default)	ESO creates and owns the Secret. Deleting the ExternalSecret deletes the Secret.
Orphan	ESO creates the Secret but does not set ownerReference. Secret persists after ExternalSecret deletion.
Merge	ESO merges values into an existing Secret it does not own. Only modifies specified keys.
None	ESO does not create or modify Secrets. Used with template to generate data for other controllers.

Template Engine

ESO can transform secret data before writing to the K8s Secret using Go templates:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
spec:
  target:
    template:
      type: kubernetes.io/tls
      data:
        tls.crt: "{{ .tlsCert }}"
        tls.key: "{{ .tlsKey }}"
  data:
    - secretKey: tlsCert
      remoteRef:
        key: prod/cert
        property: certificate
    - secretKey: tlsKey
      remoteRef:
        key: prod/cert
        property: private_key

Template functions include: base64 encode/decode, replace, toYAML, toJson, and access to all fetched secret values.

PushSecret Flow (Reverse Sync)

flowchart LR
    K8sS["K8s Secret"] -->|"PushSecret CRD"| ESO_PS["ESO Controller"]
    ESO_PS -->|"push"| Vault_PS["Vault / AWS SM / GCP SM"]

    style ESO_PS fill:#1565c0,color:#fff

PushSecret enables syncing K8s Secrets to external providers -- the reverse of the normal ExternalSecret flow. Use cases include: - Writing TLS certificates generated by cert-manager to Vault - Pushing service mesh identity credentials to an external store - Synchronizing secrets across clusters via a shared provider

Sources

• ESO Architecture
• ESO Provider Interface
• ESO Templates

---

Benchmarks

Scope

Performance characteristics, scaling limits, and resource consumption for External Secrets Operator.

Sync Performance

Provider	Secret Fetch Time	Refresh Interval	Notes
AWS Secrets Manager	50-200ms	1h default	API rate limits apply
HashiCorp Vault	10-50ms	1h default	Depends on network
Azure Key Vault	50-200ms	1h default	Regional latency
GCP Secret Manager	30-100ms	1h default	Fast in-region

Resource Consumption

ExternalSecrets Count	Controller CPU	Controller Memory
50	50-100m	128Mi
200	100-300m	256Mi
1,000	500m-1	512Mi-1Gi

Scaling Limits

Dimension	Limit	Notes
ExternalSecrets per cluster	10,000+	Controller memory is bottleneck
Providers per cluster	50+	Multiple SecretStores
Refresh rate	Provider API limits	AWS: 10,000/sec, Vault: unlimited

Sourcing Status

Unsourced Performance Data

The performance numbers in this document are estimated from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.

Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.

Sources

• External Secrets Operator Documentation
• External Secrets Operator GitHub

---

Security

Related Notes

secrets/external-secrets-operator/index | secrets/external-secrets-operator/explanation | | secrets/external-secrets-operator/how-to-guides

Threat Model

ESO operates with elevated privileges inside the Kubernetes cluster. It can create, read, update, and delete Secrets in all namespaces and has access to external secret provider credentials. Understanding its threat surface is critical.

Threat Vector	Impact	Mitigation
Compromised ESO pod	Full read access to all synced secrets	Restrict with network policies, pod security standards
Leaked provider credentials	Access to external secret store	Use short-lived tokens, IRSA, Workload Identity
Malicious ExternalSecret	Exfiltrate secrets from provider to attacker-controlled namespace	RBAC on ExternalSecret creation, OPA/Kyverno policies
Malicious SecretStore	Route secret fetching through attacker-controlled endpoint	Validate provider configs, restrict SecretStore creation
PushSecret abuse	Push cluster secrets to attacker-controlled external store	Restrict PushSecret RBAC, audit PushSecret creation
Privilege escalation via ClusterSecretStore	Cross-namespace secret access	Use namespace conditions on ClusterSecretStore

RBAC Model

Cluster Operator Permissions

Cluster operators manage the ESO deployment and ClusterSecretStores. They require broad permissions:

• external-secrets.io/clustersecretstores -- CRUD at cluster scope.
• external-secrets.io/externalsecrets -- Read/list across namespaces for troubleshooting.
• Core Kubernetes secrets -- Read/write across namespaces (delegated to ESO ServiceAccount).

Application Developer Permissions

Application developers define ExternalSecrets within their namespaces. Give them limited scope:

# Example: Developer can create ExternalSecrets in their namespace only
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: external-secret-developer
  namespace: my-app
rules:
  - apiGroups: ["external-secrets.io"]
    resources: ["externalsecrets"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups: ["external-secrets.io"]
    resources: ["secretstores"]
    verbs: ["get", "list", "watch"]

SecretStore Creation

If developers can create SecretStore resources, they can define arbitrary provider connections. In most environments, only cluster operators should create SecretStores, while developers reference pre-approved stores.

ESO Controller ServiceAccount

The ESO controller runs with a ServiceAccount that has broad permissions. The Helm chart creates:

• ClusterRole with permissions to manage Secrets across all namespaces.
• ClusterRole with permissions to manage all ESO CRDs.
• ClusterRoleBinding binding these to the ESO ServiceAccount.

Restrict this by running multiple controllers with different ServiceAccounts, each scoped to specific namespaces or controller names.

Provider Credential Security

AWS (IRSA / Pod Identity)

The recommended approach for AWS is IAM Roles for Service Accounts (IRSA) or EKS Pod Identity, which eliminates static credentials:

apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: aws-secretsmanager
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        jwt:
          serviceAccountRef:
            name: eso-sa

With IRSA, the eso-sa ServiceAccount is annotated with an IAM role ARN. AWS SDK in the ESO pod exchanges the ServiceAccount JWT token for temporary STS credentials. No static accessKeyID or secretAccessKey is stored in Kubernetes.

GCP (Workload Identity)

For GCP Secret Manager, use Workload Identity to bind a Kubernetes ServiceAccount to a GCP Service Account:

apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: gcp-secret-manager
spec:
  provider:
    gcpsm:
      projectID: my-project
      auth:
        workloadIdentity:
          serviceAccountRef:
            name: eso-workload-id-sa
          clusterLocation: us-central1
          clusterName: my-gke-cluster

HashiCorp Vault (Kubernetes Auth)

When using Vault as the provider, the Kubernetes auth method allows ESO pods to authenticate using their ServiceAccount JWT token:

apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: vault-store
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "eso-role"
          serviceAccountRef:
            name: eso-vault-sa

Static Token Risks

Do not use tokenSecretRef with static Vault tokens in production. These tokens never expire unless manually revoked. Prefer the Kubernetes auth method or AppRole with periodic secret IDs.

Namespace Isolation

SecretStore (Namespaced)

A SecretStore is namespaced and can only be referenced by ExternalSecret resources in the same namespace. Credentials referenced by the SecretStore (via secretRef) must also exist in the same namespace. This provides natural namespace isolation.

ClusterSecretStore (Cluster-Scoped)

A ClusterSecretStore can be referenced by ExternalSecret resources in any namespace. To restrict access, use the conditions field:

apiVersion: external-secrets.io/v1
kind: ClusterSecretStore
metadata:
  name: restricted-store
spec:
  conditions:
    - namespaces:
        - "team-a"
        - "team-b"
      secretRef:
        namespace: "eso-system"
  provider:
    vault:
      server: "https://vault.example.com"
      path: "secret"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "eso-role"

This restricts the ClusterSecretStore to only serve ExternalSecret resources in the team-a and team-b namespaces.

Cross-Namespace References

Some SecretStore fields allow referencing Kubernetes Secrets in other namespaces (for example, certSecretRef.namespace). In multi-tenant environments, carefully audit these cross-namespace references to prevent privilege escalation.

PushSecret Security Considerations

PushSecret reverses the normal data flow by pushing Kubernetes Secrets to external providers. This introduces additional security concerns:

1. Data exfiltration risk -- A malicious PushSecret can push secrets to an attacker-controlled external provider. Restrict PushSecret creation with RBAC and admission policies.
2. Provider write permissions -- The SecretStore referenced by a PushSecret needs write access to the external provider. Grant minimum required permissions.
3. Deletion policy -- PushSecretDeletionPolicyDelete removes secrets from the external provider when the PushSecret is deleted. Make sure that this does not cause unintended data loss.
4. Update policy -- Configure PushSecretUpdatePolicy to control whether existing provider secrets are overwritten.

# Restrict PushSecret creation with RBAC
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pushsecret-restricted
  namespace: my-app
rules:
  - apiGroups: ["external-secrets.io"]
    resources: ["pushsecrets"]
    verbs: ["get", "list"]  # No create/update/delete

Admission Control Integration

Use Kubernetes admission controllers (OPA Gatekeeper, Kyverno) to enforce policies on ESO resources:

• Allowed providers -- Restrict which provider types can be configured in SecretStores.
• Allowed namespaces -- Restrict which namespaces can create ExternalSecrets or PushSecrets.
• Template validation -- Make sure that ExternalSecret templates do not inject malicious content.
• Creation policy enforcement -- Prevent creationPolicy: Merge or None in sensitive namespaces.

Network Security

• Deploy ESO in a dedicated namespace with strict network policies.
• Restrict egress traffic from the ESO controller to only the required provider endpoints.
• Use TLS for all provider connections. Verify certificates using caBundle or caProvider in SecretStore configuration.
• For Vault, use mutual TLS (tls.certSecretRef and tls.keySecretRef) when the Vault server requires client certificates.

Audit and Observability

• ESO emits Kubernetes events for reconciliation successes and failures. Monitor these events for anomalous patterns.
• The status field on ExternalSecret and SecretStore resources reflects the current state (Ready, SecretSynced, SecretSyncedError).
• Log all changes to ESO CRDs via Kubernetes audit logging.
• Monitor for unexpected Secret modifications that correlate with ExternalSecret reconciliation events.