Architecture¶
Component breakdown, reconciliation model, HA topology, and multi-cluster architecture.
Component Overview¶
ArgoCD is composed of six core services, each with a distinct responsibility:
| Component | Role |
|---|---|
argocd-server |
REST/gRPC API server. Serves Web UI and CLI. Handles SSO via Dex |
argocd-repo-server |
Fetches Git repos and renders manifests (Helm, Kustomize, Jsonnet, plain YAML) |
argocd-application-controller |
Watches K8s clusters. Computes drift. Drives sync operations |
argocd-redis |
In-memory cache for rendered manifests and live cluster state. Speeds up diffs |
argocd-dex-server |
OpenID Connect provider proxy. Enables SSO (OIDC, SAML, LDAP, GitHub OAuth) |
argocd-applicationset-controller |
Generates Application resources from templates and generator inputs |
argocd-notifications-controller |
Sends alerts to Slack, PagerDuty, email on sync/health events |
System Architecture¶
graph TB
subgraph Users["Users & CI/CD"]
UI["Web UI\n(Browser)"]
CLI["argocd CLI"]
CICD["CI Pipeline\n(API caller)"]
end
subgraph ArgoCD["ArgoCD Control Plane"]
Server["argocd-server\n(API + UI)"]
Dex["argocd-dex-server\n(OIDC/SAML/LDAP)"]
RepoSvr["argocd-repo-server\n(manifest rendering)"]
AppCtrl["argocd-application-controller\n(reconciliation)"]
AppSet["argocd-applicationset-controller"]
Notif["argocd-notifications-controller"]
Redis["argocd-redis\n(cache)"]
end
subgraph Sources["Git Sources"]
Git["Git Repository\n(GitHub/GitLab/Gitea)"]
Helm["Helm Registry\n(OCI or HTTP)"]
end
subgraph Clusters["Target Clusters"]
LocalK8s["In-cluster\nKubernetes API"]
RemoteK8s["Remote Cluster\nKubernetes API"]
end
UI --> Server
CLI --> Server
CICD --> Server
Server --> Dex
Server --> RepoSvr
Server --> Redis
AppCtrl --> RepoSvr
AppCtrl --> Redis
AppCtrl --> LocalK8s
AppCtrl --> RemoteK8s
RepoSvr --> Git
RepoSvr --> Helm
AppSet --> Server
Notif --> Server
Reconciliation Model¶
The Application Controller is the heart of ArgoCD. It runs a control loop every 3 minutes (configurable):
sequenceDiagram
participant AppCtrl as Application Controller
participant RepoSvr as Repo Server
participant Redis as Redis Cache
participant Git as Git Repository
participant Cluster as Target Cluster
loop Every reconciliation interval
AppCtrl->>RepoSvr: Get rendered manifests
RepoSvr->>Redis: Check manifest cache
alt Cache miss
RepoSvr->>Git: git fetch + render (Helm/Kustomize)
RepoSvr->>Redis: Store rendered manifests
end
RepoSvr-->>AppCtrl: Desired state manifests
AppCtrl->>Cluster: List live resources
AppCtrl->>AppCtrl: Diff desired vs live
alt Drift detected
AppCtrl->>AppCtrl: Mark Application: OutOfSync
alt AutoSync enabled
AppCtrl->>Cluster: kubectl apply (server-side)
end
else No drift
AppCtrl->>AppCtrl: Mark Application: Synced ✓
end
end
Sync Architecture: Waves and Hooks¶
ArgoCD applies resources in ordered waves, with lifecycle hooks at each phase:
flowchart LR
PreSync["PreSync Hooks\n(migrations, prep)"] --> Wave0["Wave 0\n(Namespaces, CRDs)"] --> Wave1["Wave 1\n(ConfigMaps, Secrets)"] --> Wave2["Wave 2\n(Deployments, Services)"] --> PostSync["PostSync Hooks\n(smoke tests, notify)"] --> SyncFail["SyncFail Hooks\n(rollback, alert)"]
style PreSync fill:#e65100,color:#fff
style PostSync fill:#2e7d32,color:#fff
style SyncFail fill:#b71c1c,color:#fff
Wave ordering uses the annotation: argocd.argoproj.io/sync-wave: "2"
ApplicationSet Architecture¶
ApplicationSet generates multiple Application resources from a single template:
| Generator | Mechanism | Example Use Case |
|---|---|---|
| Git Directory | One app per subdirectory in a Git path | Monorepo — each team folder becomes an app |
| Git File | JSON/YAML config files drive app parameters | Environment-specific config per file |
| Cluster | One app per registered ArgoCD cluster | Fleet deployment — same app to all clusters |
| Matrix | Cartesian product of two generators | All clusters × all environments |
| Merge | Combine multiple generators with overrides | Base config + per-cluster patches |
| Pull Request | One app per open PR in a repository | Preview environments per PR |
High Availability Topology¶
For production deployments, ArgoCD scales each component independently:
graph TB
LB["Load Balancer / Ingress"] --> Svr1["argocd-server (replica 1)"]
LB --> Svr2["argocd-server (replica 2)"]
Svr1 --> RepoSvr1["argocd-repo-server (replica 1)"]
Svr1 --> RepoSvr2["argocd-repo-server (replica 2)"]
Svr2 --> RepoSvr1
Svr2 --> RepoSvr2
subgraph Controller["Application Controller (Sharded)"]
Shard0["Shard 0\n(clusters 0-49)"]
Shard1["Shard 1\n(clusters 50-99)"]
end
Redis["Redis / Redis Sentinel\n(HA cache)"]
Svr1 --> Redis
Svr2 --> Redis
Shard0 --> Redis
Shard1 --> Redis
Scaling guidelines:
- argocd-server: 2+ replicas. Stateless. Horizontally scalable
- argocd-repo-server: 2+ replicas. CPU-bound by rendering. Scale with repo + app count
- argocd-application-controller: 1 per shard. Enable sharding for 50+ clusters
- argocd-redis: Use Redis Sentinel or Redis Cluster for HA. Without them, Redis is a single point of failure
Multi-Cluster Architecture¶
ArgoCD manages remote clusters via bearer token or kubeconfig credentials stored as K8s Secrets in the argocd namespace:
graph LR
subgraph Mgmt["Management Cluster"]
ArgoCD["ArgoCD Control Plane"]
end
subgraph Prod["Production Clusters"]
ProdUS["prod-us\n(remote cluster)"]
ProdEU["prod-eu\n(remote cluster)"]
end
subgraph Dev["Dev Clusters"]
Dev1["dev-1\n(remote cluster)"]
end
ArgoCD -->|"ServiceAccount\nbearer token"| ProdUS
ArgoCD -->|"ServiceAccount\nbearer token"| ProdEU
ArgoCD -->|"kubeconfig\n(in-cluster)"| Dev1
ArgoCD stores cluster credentials as Secret resources with label argocd.argoproj.io/secret-type: cluster.
Repo Server: Manifest Rendering¶
The repo server isolates manifest rendering in a separate process to prevent privilege escalation:
- Clones repositories into a local cache (
/tmp/repo-server/cache) - Runs rendering (Helm, Kustomize, Jsonnet) in sandboxed subprocesses
- Caches rendered output in Redis with a TTL based on the
revision - Supports Config Management Plugins (CMP) via sidecar containers for custom renderers
Storage Model¶
ArgoCD is largely stateless — all persistent state lives in Kubernetes:
| Data | Storage |
|---|---|
| Application specs | Application CRDs in argocd namespace |
| Cluster credentials | Secret resources in argocd namespace |
| Repository credentials | Secret resources in argocd namespace |
| RBAC policies | ConfigMap (argocd-rbac-cm) |
| Runtime cache | Redis (ephemeral, rebuilt on restart) |
| Audit logs | Kubernetes API server audit log |
Related Notes¶
How It Works¶
Reconciliation engine, sync phases, ApplicationSet generators, and drift detection.
Reconciliation Loop¶
sequenceDiagram
participant Git as Git Repository
participant RepoSvr as Repository Server
participant AppCtrl as Application Controller
participant Cluster as Target Cluster
loop Every 3 minutes (configurable)
AppCtrl->>RepoSvr: Fetch latest manifests
RepoSvr->>Git: git clone/pull (or cached)
RepoSvr->>RepoSvr: Render: Helm template / Kustomize build
RepoSvr-->>AppCtrl: Desired state (manifests)
AppCtrl->>Cluster: Get live state (kubectl get)
AppCtrl->>AppCtrl: Diff: desired vs live
alt Drift detected
AppCtrl->>AppCtrl: Mark: OutOfSync
alt Auto-sync enabled
AppCtrl->>Cluster: Apply manifests (kubectl apply)
end
else No drift
AppCtrl->>AppCtrl: Mark: Synced ✓
end
end
Sync Waves & Hooks¶
flowchart LR
PreSync["PreSync\n(DB migrations, \nconfig setup)"] --> Sync["Sync\n(main resources)"] --> PostSync["PostSync\n(smoke tests,\nnotifications)"]
PreSync -.->|"Wave 0"| NS["Namespace"]
PreSync -.->|"Wave 1"| CM["ConfigMaps"]
Sync -.->|"Wave 2"| Deploy["Deployments"]
Sync -.->|"Wave 3"| SVC["Services"]
PostSync -.->|"Wave 4"| Test["Test Jobs"]
style PreSync fill:#e65100,color:#fff
style Sync fill:#1565c0,color:#fff
style PostSync fill:#2e7d32,color:#fff
ApplicationSet Generators¶
| Generator | Use Case |
|---|---|
| Git Directory | One app per directory in monorepo |
| Git File | Config-driven from JSON/YAML in Git |
| Cluster | Deploy same app to all registered clusters |
| List | Static list of environments |
| Matrix | Cross-product of two generators |
| Merge | Combine multiple generators |
| Pull Request | Preview environments per PR |
| SCM Provider | Auto-discover repos from GitHub/GitLab org |
Drift Detection Mechanism¶
ArgoCD compares two states to detect configuration drift:
- Desired state: Rendered manifests from Git (Helm template, Kustomize build, or raw YAML)
- Live state: Current resources in the target cluster (fetched via kubectl get --show-managed-fields)
The diff is computed using a three-way merge strategy, similar to kubectl apply. ArgoCD ignores:
- Runtime-injected fields (for example, kubectl.kubernetes.io/last-applied-configuration)
- Status subresources
- Fields managed by other controllers (via metadata.managedFields)
Drift status is displayed as: Synced (no diff), OutOfSync (diff detected), or Unknown (comparison error).
Sources¶
Benchmarks¶
Scope
Performance characteristics, scaling limits, and resource consumption profiles.
Sync Performance¶
| Scenario | Apps | Clusters | Avg Sync Time | P95 Sync Time | Notes |
|---|---|---|---|---|---|
| Small deployment | 50 | 3 | 2-5s | 8s | Single controller |
| Medium deployment | 200 | 10 | 5-15s | 30s | Controller sharded x2 |
| Large deployment | 1000 | 50 | 10-30s | 60s | Controller sharded x4 |
| Monorepo (10k files) | 100 | 5 | 30-60s | 120s | Sparse checkout recommended |
Resource Consumption¶
Controller Memory Scaling¶
The memory of the application controller grows linearly with the number of managed resources:
| Managed Resources | Controller Memory | Controller CPU |
|---|---|---|
| 1,000 | ~256Mi | ~200m |
| 5,000 | ~1Gi | ~500m |
| 20,000 | ~4Gi | ~2000m |
| 50,000 | ~8Gi | ~4000m |
Repo Server Performance¶
| Operation | Small Chart | Large Chart (500+ templates) | Monorepo |
|---|---|---|---|
| Helm template | < 1s | 5-15s | N/A |
| Kustomize build | < 1s | 2-5s | 10-30s |
| Git clone (cold) | 1-3s | 1-3s | 30-120s |
| Git fetch (warm) | < 1s | < 1s | 2-5s |
Scaling Limits¶
| Dimension | Tested Limit | Recommended Limit | Bottleneck |
|---|---|---|---|
| Applications per controller | ~5,000 | 2,000 | Memory + reconciliation |
| Clusters per instance | ~200 | 100 | API server connections |
| Concurrent syncs | 50 (default) | 20-30 | Repo server CPU |
| Webhook events/sec | ~100 | 50 | Redis throughput |
| ApplicationSets | ~500 | 200 | Generator evaluation time |
Community Benchmarks¶
- Intuit manages 3,000+ applications across multiple clusters using controller sharding
- The official scalability testing of the Argo Project targets 1,000 apps / 100 clusters per instance
- Red Hat OpenShift GitOps (ArgoCD-based) recommends max 300 apps per non-sharded instance
Sourcing Status¶
Unsourced Performance Data
The performance numbers in this document are estimated from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.
Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.
Sources¶
- ArgoCD Docs — Scaling Clusters — cluster management and scaling
- ArgoCD Docs — High Availability — HA topology and resource sizing
- ArgoCD Repo — Scalability Tests — e2e test suite including scalability
- Red Hat OpenShift GitOps Limits — recommended limits per instance
Security¶
Authentication, authorization, encryption, network security, and hardening for ArgoCD deployments.
Authentication¶
ArgoCD supports four authentication mechanisms, configured primarily through the argocd-cm ConfigMap.
Built-in Accounts¶
- Admin account: Created by default with a bcrypt-hashed password stored in the
argocd-secretSecret. Disable after SSO is configured by settingadmin.enabled: falseinargocd-cm. - Local users: Defined in
argocd-cmunderaccounts.<username>. Suitable for service accounts and automation. - API tokens: Generated per user via
argocd account generate-tokenor per project viaargocd proj role create-token. Tokens are JWTs signed by ArgoCD.
SSO via OIDC¶
ArgoCD integrates with external identity providers through native OIDC or via the bundled Dex server.
Native OIDC (recommended for cloud-native IdPs like Okta, Azure AD, Keycloak):
# argocd-cm
data:
url: https://argocd.YOUR_DOMAIN
oidc.config: |
name: Okta
issuer: https://dev-12345.okta.com/oauth2/default
clientID: aaaabbbbccccddd
clientSecret: $oidc.okta.clientSecret
requestedScopes: ["openid", "profile", "email", "groups"]
requestedIDTokenClaims: {"groups": {"essential": true}}
Dex (required for SAML, LDAP, GitHub OAuth, and other non-OIDC connectors):
# argocd-cm
data:
url: https://argocd.YOUR_DOMAIN
dex.config: |
connectors:
- type: github
id: github
name: GitHub
config:
clientID: $dex.github.clientID
clientSecret: $dex.github.clientSecret
orgs:
- name: my-org
Group Claim Mapping
Always configure requestedIDTokenClaims to pull the groups claim. This makes sure that OIDC groups are available for RBAC policy mapping via g, <oidc-group>, role:<argo-role> entries.
Authorization (RBAC)¶
ArgoCD uses a Casbin-based RBAC engine. Policies are defined in the argocd-rbac-cm ConfigMap.
Policy Format¶
- subject:
role:<name>or a user/group identifier - resource:
applications,projects,repositories,clusters,logs,exec - action:
get,create,update,delete,sync,action/<action-name> - object:
<project>/<app-name>or*/*for all
Built-in Roles¶
| Role | Scope |
|---|---|
role:admin |
Full access to all resources |
role:readonly |
Read access to all resources. No sync or modify |
| Custom roles | Defined in policy.csv with arbitrary permissions |
Project-Level RBAC¶
AppProjects provide namespace-like isolation within ArgoCD. Each project defines:
- Source repos: Which Git repositories apps in this project can pull from
- Destination clusters/namespaces: Where apps can be deployed
- Roles: Fine-grained permissions scoped to the project
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: team-a
namespace: argocd
spec:
sourceRepos:
- "https://github.com/org/team-a-*"
destinations:
- namespace: "team-a-*"
server: "https://kubernetes.default.svc"
roles:
- name: admin
groups:
- org:team-a-admins
policies:
- p, proj:team-a:admin, applications, *, team-a/*, allow
Default Policy¶
Set policy.default in argocd-rbac-cm:
role:readonly-- safest default. Unauthenticated users get read-only access""(empty) -- denies all access for unmapped users. Recommended for enterprise
Anonymous Access
Setting users.anonymous.enabled: true in argocd-cm grants public read access to all applications. Set it to false in production unless the ArgoCD instance is on an internal network.
Encryption and Secret Management¶
At Rest¶
| Asset | Encryption Method |
|---|---|
| Admin password | bcrypt hash in argocd-secret |
| SSO client secrets | Base64-encoded in argocd-secret. Use Sealed Secrets or External Secrets Operator for GitOps management |
| Cluster credentials | Base64-encoded in Secret with label argocd.argoproj.io/secret-type: cluster |
| Repository credentials | Base64-encoded SSH keys, HTTPS tokens, or TLS certs in Secrets with label argocd.argoproj.io/secret-type: repository |
| Redis cache | In-memory only. Ephemeral. Rebuilt on restart |
| Dex storage | ConfigMap-backed or etcd. No persistent database |
In Transit¶
- All inter-component communication uses TLS: server-to-repo-server, server-to-controller, server-to-Redis
- Configure
argocd-serverwith a valid TLS certificate (via Ingress or direct) - The repo server communicates with Git repositories over HTTPS or SSH
Secret Management Integration¶
ArgoCD does not manage secrets natively. Use one of these patterns:
| Approach | Mechanism | Trade-off |
|---|---|---|
| External Secrets Operator (ESO) | ArgoCD deploys ExternalSecret CRDs. ESO syncs from Vault/AWS SM/GCP SM to K8s Secrets |
Best separation of concerns. Secrets stay in Vault |
| ArgoCD Vault Plugin (AVP) | Replaces <path:secret#key> placeholders during manifest rendering in repo-server |
True GitOps but requires plugin sidecar |
| SOPS + Kustomize | Encrypt secrets in Git with SOPS. Decrypt at render time via ksops plugin |
Everything in Git. Key management is critical |
| Sealed Secrets | Encrypt secrets client-side with kubeseal. SealedSecret controller decrypts in-cluster |
Simple, but secrets are visible in Git (encrypted) |
Recommended Pattern
Use External Secrets Operator for production. ArgoCD manages the ExternalSecret and SecretStore manifests, while the actual secret values live in an external vault (HashiCorp Vault, AWS Secrets Manager, and more).
Network Security¶
Network Policies¶
Restrict inter-component and external traffic with Kubernetes NetworkPolicies:
# Restrict ArgoCD server ingress to internal networks
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: argocd-server-ingress
namespace: argocd
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: argocd-server
ingress:
- from:
- ipBlock:
cidr: 10.0.0.0/8
ports:
- port: 8080
protocol: TCP
Key network segmentation rules:
- Restrict argocd-repo-server egress to only required Git hosts and registries
- Restrict Redis ingress to only ArgoCD components
- Deny argocd-repo-server access to the Kubernetes API server (prevents privilege escalation from compromised repos)
- Use a separate argocd-repo-server deployment for untrusted repositories
Cluster Credential Security¶
- Prefer short-lived ServiceAccount tokens over static bearer tokens
- Use cloud-native auth (AWS IAM Roles, GCP Workload Identity, Azure Managed Identity) for managed clusters
- Store cluster secrets as K8s Secrets with restricted RBAC (only
argocd-application-controllerneeds read access)
Hardening Checklist¶
- Disable admin account after SSO is configured (
admin.enabled: false) - Set
policy.defaultto empty string (deny-by-default) - Enable
server.rbac.log.enforce.enable: trueto enforce RBAC on log access - Use Sealed Secrets or ESO instead of plaintext Secrets in Git
- Run ArgoCD components with
SecurityContext(non-root, read-only root filesystem) - Apply Pod Security Standards (restricted profile) to the
argocdnamespace - Configure NetworkPolicies for all ArgoCD components
- Use
resource.customizations.ignoreDifferencesto prevent reconciliation loops on dynamically-mutated resources (for example, webhook configurations modified by operators) - Set
repository.credentialsinargocd-cmto use templated credential matching instead of per-repo secrets - Enable audit logging at the Kubernetes API server level for ArgoCD namespace
Known Pitfalls¶
| Pitfall | Impact | Mitigation |
|---|---|---|
Overly broad policy.default: role:readonly |
Any unauthenticated user can read all application manifests (which can contain sensitive config) | Set policy.default: "" and require authentication |
| Static cluster bearer tokens that never expire | Compromised tokens provide persistent cluster access | Use short-lived ServiceAccount tokens or cloud IAM |
Missing ignoreDifferences for operator-managed webhooks |
ArgoCD enters infinite sync loop against operators like cert-manager, gatekeeper | Add resource.customizations.ignoreDifferences in argocd-cm |
| Unrestricted repo-server network access | Compromised Git repo can exfiltrate data or get to internal services | Apply strict egress NetworkPolicies to argocd-repo-server |