Architecture¶

Component Overview¶

graph TB
    subgraph CLI["CLI Layer"]
        TF["terraform CLI"]
    end

    subgraph CORE["Terraform Core"]
        HCL["HCL Parser<br/>(hcl lang library)"]
        CFG["Config Loader<br/>(module resolver)"]
        GRAPH["Graph Builder<br/>(DAG construction)"]
        EVAL["Evaluator<br/>(expression evaluation)"]
        PLAN["Plan Engine"]
        APPLY["Apply Engine"]
    end

    subgraph PLUGINS["Plugin System"]
        PV["Provider Plugins<br/>(gRPC / protocol v5-v6)"]
        PROV["Provisioner Plugins"]
    end

    subgraph STATE["State Layer"]
        SF["State File<br/>(JSON)"]
        BE["Backend<br/>(S3, GCS, Azure, consul, local)"]
        WS["Workspaces"]
    end

    subgraph REG["Registry"]
        TREG["Terraform Registry<br/>(registry.terraform.io)"]
        PRIV["Private Registry<br/>(HCP Terraform)"]
    end

    subgraph HCPTF["HCP Terraform (optional)"]
        API["Terraform Cloud API"]
        SENT["Sentinel Policy"]
        VCS["VCS Integration"]
    end

    TF --> HCL
    HCL --> CFG
    CFG --> GRAPH
    GRAPH --> EVAL
    EVAL --> PLAN
    PLAN --> APPLY
    APPLY --> PV
    APPLY --> PROV
    PLAN --> SF
    APPLY --> SF
    SF --> BE
    WS --> SF
    CFG -->|resolve modules| TREG
    CFG -->|resolve modules| PRIV
    TF -->|remote operations| HCPTF

Core Components¶

HCL Parser¶

The HCL parser (implemented in the hcl Go library, github.com/hashicorp/hcl/v2) handles:

Parsing .tf files into an abstract syntax tree (AST).
Evaluating HCL expressions including variables, locals, conditionals, for expressions, and template strings.
Validating block types and attribute schemas against provider-defined schemas.
Producing diagnostic messages with source location information.

The parser is language-agnostic -- the same library is used by other HashiCorp tools (Vault, Consul, Nomad).

Config Loader and Module System¶

The config loader resolves the full configuration from multiple sources:

Root module: The .tf files in the working directory.
Child modules: Referenced via module blocks, sourced from:
Local filesystem paths.
Terraform Registry (registry.terraform.io).
Git repositories (GitHub, GitLab, generic).
S3 buckets, GCS buckets, and other storage backends.
HCP Terraform private module registry.

Module versioning is handled through a lock file (.terraform.lock.hcl) that pins provider versions and checksums.

Graph Builder¶

The graph builder constructs a directed acyclic graph (DAG) that determines the order of operations:

Nodes: Root variables, module variables, providers, resources, data sources, outputs, locals.
Edges: Implicit dependencies detected by scanning HCL expression references, plus explicit depends_on declarations.
Transformers: A series of graph transformers process the DAG:
ReferenceTransformer -- connects references to their targets.
ProviderTransformer -- associates resources with their provider instances.
CountBoundaryTransformer -- handles count and for_each expansion.
OrphanResourceTransformer -- detects resources in state but not in config.
TransitiveReductionTransformer -- removes redundant edges.

Separate graph builders for each operation phase: - PlanGraphBuilder -- produces the plan. - ApplyGraphBuilder -- executes planned changes. - DestroyGraphBuilder -- reverse-order destruction. - RefreshGraphBuilder -- updates state from real infrastructure. - ValidateGraphBuilder -- validates configuration without planning.

Evaluator¶

The evaluator walks the graph and resolves all expressions:

Evaluates variable values (from .tfvars files, environment variables, CLI flags).
Resolves locals blocks in dependency order.
Expands count and for_each into individual resource instances.
Handles lifecycle meta-arguments (create_before_destroy, prevent_destroy, ignore_changes).
Produces the complete "desired state" that represents what the configuration declares.

Plugin Protocol¶

Terraform communicates with providers via a gRPC-based plugin protocol (protocol version 5 and 6).

Protocol Evolution¶

Version	Transport	Introduced	Notes
v1-v4	`net/rpc` (Go-specific)	Pre-0.12	Legacy, deprecated
v5	gRPC (`tfplugin5`)	Terraform 0.12	Current standard
v6	gRPC (`tfplugin6`)	Terraform 1.0+	Adds log-level control, moved types

Plugin Handshake¶

sequenceDiagram
    participant Core as Terraform Core
    participant Plugin as Provider Plugin

    Core->>Plugin: Launch process
    Plugin-->>Core: Handshake (stdout)<br/>protocol version + gRPC address + TLS cert
    Core->>Plugin: gRPC connection (mTLS)
    Core->>Plugin: GetSchema
    Plugin-->>Core: Resource & data source schemas
    Core->>Plugin: ConfigureProvider
    Plugin-->>Core: Config validated
    Note over Core,Plugin: Plan phase
    Core->>Plugin: PlanResourceChange (per resource)
    Plugin-->>Core: Planned state + requires replacement?
    Note over Core,Plugin: Apply phase
    Core->>Plugin: ApplyResourceChange (per resource)
    Plugin-->>Core: New state
    Core->>Plugin: Close (graceful shutdown)

Plugin SDKs¶

SDK	Status	Usage
`terraform-plugin-sdk/v2` (SDKv2)	Mature	Most existing providers
`terraform-plugin-framework`	Recommended	New providers, protocol v6 features
`terraform-plugin-mux`	Active	Combine SDKv2 and framework in one provider

Plan / Apply Flow¶

flowchart TD
    A["terraform init"] --> B["Install providers<br/>Download modules<br/>Initialize backend"]
    B --> C["terraform plan"]
    C --> D["Parse HCL config"]
    D --> E["Load prior state from backend"]
    E --> F["Refresh: read real-world state via providers"]
    F --> G["Evaluate config against refreshed state"]
    G --> H["Build plan graph"]
    H --> I["Compute diff per resource instance"]
    I --> J["Save plan file (optional)"]
    J --> K["terraform apply"]
    K --> L["Load saved plan OR re-plan"]
    L --> M["Build apply graph"]
    M --> N["Walk graph: create / update / delete resources"]
    N --> O["Update state incrementally"]
    O --> P["Persist final state to backend"]
    P --> Q["terraform output"]

Deterministic plans

The plan file is a binary artifact that captures the exact set of changes. When applied, Terraform guarantees that only those planned changes are executed. The plan file includes a cryptographic hash of the configuration to detect configuration drift between plan and apply.

State Management¶

State File Structure¶

The state file (terraform.tfstate) is a JSON document containing:

Version: State format version (currently 4).
Terraform version: Version that produced the state.
Serial: Monotonically increasing counter for optimistic locking.
Lineage: Unique UUID identifying the state lineage (prevents corruption from swapping state files).
Outputs: Root module output values.
Resources: Array of resource instances with:
type, name, provider identifier.
instances: Array of individual instances (from count/for_each).
Each instance has attributes (current state), dependencies, schema_version.

Backend Options¶

Backend	State Locking	Encryption	Notes
`local`	No	None	Default, `terraform.tfstate` on disk
`s3`	Yes (DynamoDB)	SSE-S3, SSE-KMS	Most common AWS choice
`gcs`	Yes (native)	Google-managed or CMEK	Google Cloud
`azurerm`	Yes (blob lease)	SSE	Azure
`consul`	Yes (session)	Optional (base64)	HashiCorp Consul
`pg`	Yes (advisory lock)	Optional	PostgreSQL
`remote` (HCP TF)	Yes	Encryption at rest	Terraform Cloud managed
`oss`	Yes	SSE	Alibaba Cloud

Workspaces¶

Workspaces provide isolated state instances within the same configuration:

Each workspace maintains its own state file.
terraform.workspace variable enables workspace-conditional logic.
Default workspace is always present (default).
In HCP Terraform, workspaces map to separate workspaces with independent VCS triggers, variables, and run settings.

HCP Terraform (Terraform Cloud)¶

HCP Terraform extends the open-source CLI with managed infrastructure:

Feature	Description
Remote operations	Plans and applies run on HashiCorp-managed infrastructure
VCS integration	Auto-trigger runs on GitHub, GitLab, Bitbucket pushes
Sentinel policies	Policy-as-code framework for governance
Private registry	Organizational modules and providers
Cost estimation	Estimated cloud spend per plan (Business tier)
RBAC	Team-based access control per workspace
SSO	SAML/SSO with major identity providers
API-driven workflow	Full CRUD API for workspace, run, variable management

Comparison with Alternatives¶

Aspect	Terraform	OpenTofu	Pulumi
Language	HCL	HCL	TS, Python, Go, .NET, Java
License	BSL 1.1	MPL 2.0	Apache 2.0
Provider protocol	gRPC v5/v6	gRPC v5/v6 (compatible)	gRPC (own protocol)
State encryption	Not native	Built-in (AES-GCM + KMS)	Per-secret encryption
Policy-as-code	Sentinel (paid)	OPA (community)	CrossGuard
Managed platform	HCP Terraform	Community/self-hosted	Pulumi Cloud

References¶

How It Works¶

Provider plugin architecture, state management, plan/apply lifecycle, and dependency graph.

Core Lifecycle¶

sequenceDiagram
    participant User as User
    participant CLI as Terraform CLI
    participant State as State File
    participant Provider as Provider Plugin (gRPC)
    participant Cloud as Cloud API

    User->>CLI: terraform plan
    CLI->>State: Read current state
    CLI->>Provider: Refresh resource statuses
    Provider->>Cloud: API calls to check real state
    Provider-->>CLI: Current state
    CLI->>CLI: Diff: desired (HCL) vs current
    CLI-->>User: Execution plan (+ / ~ / -)

    User->>CLI: terraform apply
    CLI->>CLI: Build dependency graph (DAG)
    CLI->>Provider: Create/Update/Delete resources
    Provider->>Cloud: API calls
    Provider-->>CLI: Resource attributes
    CLI->>State: Write updated state

Dependency Graph (DAG)¶

Terraform builds a Directed Acyclic Graph of all resources, ensuring correct ordering:

flowchart TB
    VPC["aws_vpc"] --> Subnet["aws_subnet"]
    Subnet --> SG["aws_security_group"]
    Subnet --> Instance["aws_instance"]
    SG --> Instance
    Instance --> EIP["aws_eip"]

    style VPC fill:#ff6f00,color:#fff

Resources without dependencies are created in parallel.

Provider Plugin Architecture¶

flowchart LR
    CLI_T["Terraform CLI\n(core engine)"] <-->|"gRPC"| AWS["AWS Provider\n(plugin binary)"]
    CLI_T <-->|"gRPC"| GCP["GCP Provider"]
    CLI_T <-->|"gRPC"| K8s_P["K8s Provider"]
    CLI_T <-->|"gRPC"| Custom["Custom Provider"]

    style CLI_T fill:#7b42bc,color:#fff

Each provider is a separate binary communicating via gRPC. Providers are downloaded during terraform init from the Terraform Registry.

State File¶

Aspect	Detail
Purpose	Maps HCL resources to real-world objects
Format	JSON (human-readable but not for editing)
Locking	DynamoDB (AWS), GCS, Consul for remote
Sensitive data	⚠️ Stored in plaintext (use remote backend + encryption)
Remote backends	S3, GCS, Azure Blob, Terraform Cloud, Consul

Sources¶

Benchmarks¶

Scope

Performance characteristics, scaling limits, and resource consumption for Terraform.

Plan/Apply Performance¶

State Size	Plan Time	Apply Time	Memory
50 resources	< 5s	1-3m	100MB
500 resources	15-60s	5-15m	500MB
2,000 resources	2-10m	15-45m	2GB
10,000 resources	10-30m	1-3h	8GB+

Provider API Limits¶

Provider	Rate Limit	Impact on Large Plans
AWS	20-100 req/s (varies by API)	Parallelism > 10 may hit limits
Azure	12,000 req/h per subscription	Large plans need throttling
GCP	10-100 req/s	Moderate limitation

Module Performance¶

Module Count	Init Time	Plan Overhead
5	5-10s	Negligible
20	10-30s	10-20% slower
50+	30-120s	Consider splitting

State File Benchmarks¶

Metric	Small (< 1MB)	Medium (1-10MB)	Large (10-100MB)
Read	< 1s	1-5s	5-30s
Write	< 1s	1-5s	5-30s
Plan (full)	< 30s	30s-5m	5-30m

Sourcing Status¶

Unsourced Performance Data

The performance numbers in this document are estimated from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.

Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.