Architecture¶

Component Overview¶

graph TB
    subgraph CLI["CLI Layer"]
        TF["terraform CLI"]
    end

    subgraph CORE["Terraform Core"]
        HCL["HCL Parser<br/>(hcl lang library)"]
        CFG["Config Loader<br/>(module resolver)"]
        GRAPH["Graph Builder<br/>(DAG construction)"]
        EVAL["Evaluator<br/>(expression evaluation)"]
        PLAN["Plan Engine"]
        APPLY["Apply Engine"]
    end

    subgraph PLUGINS["Plugin System"]
        PV["Provider Plugins<br/>(gRPC / protocol v5-v6)"]
        PROV["Provisioner Plugins"]
    end

    subgraph STATE["State Layer"]
        SF["State File<br/>(JSON)"]
        BE["Backend<br/>(S3, GCS, Azure, consul, local)"]
        WS["Workspaces"]
    end

    subgraph REG["Registry"]
        TREG["Terraform Registry<br/>(registry.terraform.io)"]
        PRIV["Private Registry<br/>(HCP Terraform)"]
    end

    subgraph HCPTF["HCP Terraform (optional)"]
        API["Terraform Cloud API"]
        SENT["Sentinel Policy"]
        VCS["VCS Integration"]
    end

    TF --> HCL
    HCL --> CFG
    CFG --> GRAPH
    GRAPH --> EVAL
    EVAL --> PLAN
    PLAN --> APPLY
    APPLY --> PV
    APPLY --> PROV
    PLAN --> SF
    APPLY --> SF
    SF --> BE
    WS --> SF
    CFG -->|resolve modules| TREG
    CFG -->|resolve modules| PRIV
    TF -->|remote operations| HCPTF

Core Components¶

HCL Parser¶

The HCL parser (implemented in the hcl Go library, github.com/hashicorp/hcl/v2) handles:

Parsing .tf files into an abstract syntax tree (AST).
Evaluating HCL expressions including variables, locals, conditionals, for expressions, and template strings.
Validating block types and attribute schemas against provider-defined schemas.
Producing diagnostic messages with source location information.

The parser is language-agnostic -- the same library is used by other HashiCorp tools (Vault, Consul, Nomad).

Config Loader and Module System¶

The config loader resolves the full configuration from multiple sources:

Root module: The .tf files in the working directory.
Child modules: Referenced via module blocks, sourced from:
Local filesystem paths.
Terraform Registry (registry.terraform.io).
Git repositories (GitHub, GitLab, generic).
S3 buckets, GCS buckets, and other storage backends.
HCP Terraform private module registry.

Module versioning is handled through a lock file (.terraform.lock.hcl) that pins provider versions and checksums.

Graph Builder¶

The graph builder constructs a directed acyclic graph (DAG) that determines the order of operations:

Nodes: Root variables, module variables, providers, resources, data sources, outputs, locals.
Edges: Implicit dependencies detected by scanning HCL expression references, plus explicit depends_on declarations.
Transformers: A series of graph transformers process the DAG:
ReferenceTransformer -- connects references to their targets.
ProviderTransformer -- associates resources with their provider instances.
CountBoundaryTransformer -- handles count and for_each expansion.
OrphanResourceTransformer -- detects resources in state but not in config.
TransitiveReductionTransformer -- removes redundant edges.

Separate graph builders for each operation phase: - PlanGraphBuilder -- produces the plan. - ApplyGraphBuilder -- executes planned changes. - DestroyGraphBuilder -- reverse-order destruction. - RefreshGraphBuilder -- updates state from real infrastructure. - ValidateGraphBuilder -- validates configuration without planning.

Evaluator¶

The evaluator walks the graph and resolves all expressions:

Evaluates variable values (from .tfvars files, environment variables, CLI flags).
Resolves locals blocks in dependency order.
Expands count and for_each into individual resource instances.
Handles lifecycle meta-arguments (create_before_destroy, prevent_destroy, ignore_changes).
Produces the complete "desired state" that represents what the configuration declares.

Plugin Protocol¶

Terraform communicates with providers via a gRPC-based plugin protocol (protocol version 5 and 6).

Protocol Evolution¶

Version	Transport	Introduced	Notes
v1-v4	`net/rpc` (Go-specific)	Pre-0.12	Legacy, deprecated
v5	gRPC (`tfplugin5`)	Terraform 0.12	Current standard
v6	gRPC (`tfplugin6`)	Terraform 1.0+	Adds log-level control, moved types

Plugin Handshake¶

sequenceDiagram
    participant Core as Terraform Core
    participant Plugin as Provider Plugin

    Core->>Plugin: Launch process
    Plugin-->>Core: Handshake (stdout)<br/>protocol version + gRPC address + TLS cert
    Core->>Plugin: gRPC connection (mTLS)
    Core->>Plugin: GetSchema
    Plugin-->>Core: Resource & data source schemas
    Core->>Plugin: ConfigureProvider
    Plugin-->>Core: Config validated
    Note over Core,Plugin: Plan phase
    Core->>Plugin: PlanResourceChange (per resource)
    Plugin-->>Core: Planned state + requires replacement?
    Note over Core,Plugin: Apply phase
    Core->>Plugin: ApplyResourceChange (per resource)
    Plugin-->>Core: New state
    Core->>Plugin: Close (graceful shutdown)

Plugin SDKs¶

SDK	Status	Usage
`terraform-plugin-sdk/v2` (SDKv2)	Mature	Most existing providers
`terraform-plugin-framework`	Recommended	New providers, protocol v6 features
`terraform-plugin-mux`	Active	Combine SDKv2 and framework in one provider

Plan / Apply Flow¶

flowchart TD
    A["terraform init"] --> B["Install providers<br/>Download modules<br/>Initialize backend"]
    B --> C["terraform plan"]
    C --> D["Parse HCL config"]
    D --> E["Load prior state from backend"]
    E --> F["Refresh: read real-world state via providers"]
    F --> G["Evaluate config against refreshed state"]
    G --> H["Build plan graph"]
    H --> I["Compute diff per resource instance"]
    I --> J["Save plan file (optional)"]
    J --> K["terraform apply"]
    K --> L["Load saved plan OR re-plan"]
    L --> M["Build apply graph"]
    M --> N["Walk graph: create / update / delete resources"]
    N --> O["Update state incrementally"]
    O --> P["Persist final state to backend"]
    P --> Q["terraform output"]

Deterministic plans

The plan file is a binary artifact that captures the exact set of changes. When applied, Terraform guarantees that only those planned changes are executed. The plan file includes a cryptographic hash of the configuration to detect configuration drift between plan and apply.

State Management¶

State File Structure¶

The state file (terraform.tfstate) is a JSON document containing:

Version: State format version (currently 4).
Terraform version: Version that produced the state.
Serial: Monotonically increasing counter for optimistic locking.
Lineage: Unique UUID identifying the state lineage (prevents corruption from swapping state files).
Outputs: Root module output values.
Resources: Array of resource instances with:
type, name, provider identifier.
instances: Array of individual instances (from count/for_each).
Each instance has attributes (current state), dependencies, schema_version.

Backend Options¶

Backend	State Locking	Encryption	Notes
`local`	No	None	Default, `terraform.tfstate` on disk
`s3`	Yes (DynamoDB)	SSE-S3, SSE-KMS	Most common AWS choice
`gcs`	Yes (native)	Google-managed or CMEK	Google Cloud
`azurerm`	Yes (blob lease)	SSE	Azure
`consul`	Yes (session)	Optional (base64)	HashiCorp Consul
`pg`	Yes (advisory lock)	Optional	PostgreSQL
`remote` (HCP TF)	Yes	Encryption at rest	Terraform Cloud managed
`oss`	Yes	SSE	Alibaba Cloud

Workspaces¶

Workspaces provide isolated state instances within the same configuration:

Each workspace maintains its own state file.
terraform.workspace variable enables workspace-conditional logic.
Default workspace is always present (default).
In HCP Terraform, workspaces map to separate workspaces with independent VCS triggers, variables, and run settings.

HCP Terraform (Terraform Cloud)¶

HCP Terraform extends the open-source CLI with managed infrastructure:

Feature	Description
Remote operations	Plans and applies run on HashiCorp-managed infrastructure
VCS integration	Auto-trigger runs on GitHub, GitLab, Bitbucket pushes
Sentinel policies	Policy-as-code framework for governance
Private registry	Organizational modules and providers
Cost estimation	Estimated cloud spend per plan (Business tier)
RBAC	Team-based access control per workspace
SSO	SAML/SSO with major identity providers
API-driven workflow	Full CRUD API for workspace, run, variable management

Comparison with Alternatives¶

Aspect	Terraform	OpenTofu	Pulumi
Language	HCL	HCL	TS, Python, Go, .NET, Java
License	BSL 1.1	MPL 2.0	Apache 2.0
Provider protocol	gRPC v5/v6	gRPC v5/v6 (compatible)	gRPC (own protocol)
State encryption	Not native	Built-in (AES-GCM + KMS)	Per-secret encryption
Policy-as-code	Sentinel (paid)	OPA (community)	CrossGuard
Managed platform	HCP Terraform	Community/self-hosted	Pulumi Cloud

References¶

How It Works¶

Provider plugin architecture, state management, plan/apply lifecycle, and dependency graph.

Core Lifecycle¶

sequenceDiagram
    participant User as User
    participant CLI as Terraform CLI
    participant State as State File
    participant Provider as Provider Plugin (gRPC)
    participant Cloud as Cloud API

    User->>CLI: terraform plan
    CLI->>State: Read current state
    CLI->>Provider: Refresh resource statuses
    Provider->>Cloud: API calls to check real state
    Provider-->>CLI: Current state
    CLI->>CLI: Diff: desired (HCL) vs current
    CLI-->>User: Execution plan (+ / ~ / -)

    User->>CLI: terraform apply
    CLI->>CLI: Build dependency graph (DAG)
    CLI->>Provider: Create/Update/Delete resources
    Provider->>Cloud: API calls
    Provider-->>CLI: Resource attributes
    CLI->>State: Write updated state

Dependency Graph (DAG)¶

Terraform builds a Directed Acyclic Graph of all resources. This ensures correct ordering:

flowchart TB
    VPC["aws_vpc"] --> Subnet["aws_subnet"]
    Subnet --> SG["aws_security_group"]
    Subnet --> Instance["aws_instance"]
    SG --> Instance
    Instance --> EIP["aws_eip"]

    style VPC fill:#ff6f00,color:#fff

Resources without dependencies are created in parallel.

Provider Plugin Architecture¶

flowchart LR
    CLI_T["Terraform CLI\n(core engine)"] <-->|"gRPC"| AWS["AWS Provider\n(plugin binary)"]
    CLI_T <-->|"gRPC"| GCP["GCP Provider"]
    CLI_T <-->|"gRPC"| K8s_P["K8s Provider"]
    CLI_T <-->|"gRPC"| Custom["Custom Provider"]

    style CLI_T fill:#7b42bc,color:#fff

Each provider is a separate binary communicating via gRPC. Providers are downloaded during terraform init from the Terraform Registry.

State File¶

Aspect	Detail
Purpose	Maps HCL resources to real-world objects
Format	JSON (human-readable but not for editing)
Locking	DynamoDB (AWS), GCS, Consul for remote
Sensitive data	⚠️ Stored in plaintext (use remote backend + encryption)
Remote backends	S3, GCS, Azure Blob, Terraform Cloud, Consul

Sources¶

Benchmarks¶

Scope

Performance characteristics, scaling limits, and resource consumption for Terraform.

Plan/Apply Performance¶

State Size	Plan Time	Apply Time	Memory
50 resources	< 5s	1-3m	100MB
500 resources	15-60s	5-15m	500MB
2,000 resources	2-10m	15-45m	2GB
10,000 resources	10-30m	1-3h	8GB+

Provider API Limits¶

Provider	Rate Limit	Impact on Large Plans
AWS	20-100 req/s (varies by API)	Parallelism > 10 can hit limits
Azure	12,000 req/h per subscription	Large plans need throttling
GCP	10-100 req/s	Moderate limitation

Module Performance¶

Module Count	Init Time	Plan Overhead
5	5-10s	Negligible
20	10-30s	10-20% slower
50+	30-120s	Consider splitting

State File Benchmarks¶

Metric	Small (< 1MB)	Medium (1-10MB)	Large (10-100MB)
Read	< 1s	1-5s	5-30s
Write	< 1s	1-5s	5-30s
Plan (full)	< 30s	30s-5m	5-30m

Sourcing Status¶

Unsourced Performance Data

Do not plan capacity from these numbers. We estimated them from vendor documentation, community benchmarks, and engineering judgment. They do not represent controlled benchmarks with documented test conditions. Specific hardware configurations, software versions, and test methodologies were not recorded.

Use these figures as rough guidance only. For production capacity planning, run your own benchmarks against your specific workload and infrastructure.

Sources¶

Security¶

Authentication Model¶

Terraform itself does not authenticate users — the state backend and the cloud providers it manages handle authentication:

Component	Authentication Method
Terraform Cloud/Enterprise	User accounts, SSO (SAML/OIDC), API tokens, Team tokens
AWS Provider	Static keys, IAM roles, `AssumeRoleWithWebIdentity` (OIDC)
Azure Provider	Service principals, managed identity, OIDC federated credentials
GCP Provider	Service accounts, Workload Identity Federation
State backends (S3, GCS, Azure)	Backend-specific credentials

graph TB
    subgraph Operators
        Dev["Developer\n(terraform CLI)"]
        VCS["VCS Webhook\n(GitHub/GitLab)"]
        API["API Token\n(CI/CD)"]
    end
    subgraph TFCloud["Terraform Cloud / Enterprise"]
        AuthN["Identity\n(SAML/OIDC/Tokens)"]
        Sentinel["Sentinel\n(Policy Engine)"]
        StateStore["State Storage\n(Encrypted)"]
    end
    subgraph Providers
        AWS["AWS\n(IAM/OIDC)"]
        Azure["Azure\n(MSI/OIDC)"]
        GCP["GCP\n(WIF/OIDC)"]
    end
    Dev --> AuthN
    VCS --> AuthN
    API --> AuthN
    AuthN --> Sentinel
    Sentinel --> StateStore
    Sentinel --> Providers

State File Security¶

The Terraform state file is the most critical security asset — it contains all resource IDs, computed attributes, and potentially plaintext secrets.

State File Risks¶

Risk	Impact
Plaintext secrets	Database passwords, API keys stored in cleartext
Resource manipulation	Modified state can destroy or modify infrastructure
Sensitive outputs	IP addresses, internal DNS names, instance IDs
Drift concealment	Altered state masks unauthorized infrastructure changes

Terraform Cloud State Security¶

State encrypted at rest (AES-256) and in transit (TLS 1.2+)
State access URLs are time-limited (25 hours) with embedded secrets
State never written to disk during Terraform Cloud operations
Full state versioning with rollback capability

Self-Managed Backend Hardening¶

For S3, GCS, Azure Blob backends:

terraform {
  backend "s3" {
    bucket         = "terraform-state-prod"
    key            = "infra/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:..."
    dynamodb_table = "terraform-locks"
  }
}

Enable server-side encryption with customer-managed keys
Enable state locking (DynamoDB for S3, native for GCS/Azure)
Restrict bucket access with IAM policies (least privilege)
Enable versioning to recover from accidental deletion
Enable access logging for audit trail

Sentinel Policy as Code¶

Terraform Enterprise and HCP Terraform enforce organizational policies via Sentinel:

import "tfplan"

main = rule {
    version.new(tfplan.terraform_version).greater_than("1.5.0")
}

Common Sentinel Policies¶

Policy Category	Example
Resource restrictions	Only allow specific instance types
Encryption enforcement	Require KMS encryption on S3, EBS, RDS
Network security	Deny public S3 buckets, require security groups
Tagging compliance	Mandate cost-center and environment tags
Provider version pinning	Enforce minimum provider versions

Sentinel enforcement levels: - Advisory: Log violations, allow apply - Soft-mandatory: Allow override with justification - Hard-mandatory: Block apply entirely

Provider Credential Management¶

Dynamic Credentials (Recommended)¶

Use OIDC federation to generate short-lived credentials instead of static keys:

Provider	OIDC Method
AWS	`AssumeRoleWithWebIdentity` with Terraform Cloud OIDC
Azure	Workload Identity Federation
GCP	Workload Identity Federation

Static Credentials (Legacy)¶

When OIDC is unavailable:

Store credentials in environment variables, never in .tf files
Use HashiCorp Vault to generate short-lived credentials
Rotate access keys on a defined schedule
Use IAM roles with conditional trust policies (IP restrictions, MFA)

Credential Exposure Prevention¶

output "db_password" {
  value     = aws_db_instance.main.password
  sensitive = true
}

Use sensitive = true on all outputs containing secrets
Enable .terraform.lock.hcl to pin provider versions
Verify provider checksums with terraform providers lock

Module Registry Security¶

Module Verification¶

Pin module versions to specific tags or commit SHAs — never use latest
Verify module sources: prefer Terraform Registry or internal Git repos
Use terraform providers lock to create a dependency lock file with SHA256 checksums

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.5.0"
}

Module Security Checklist¶

Module source is from trusted registry or internal repository
Pin the module version to a specific tag
Provider versions are constrained with >= and < bounds
No default credentials in variable definitions
All sensitive variables declared with sensitive = true

Terraform Cloud RBAC¶

Role	Scope	Permissions
Organization Owner	Org	Manage users, teams, policies, billing
Organization Member	Org	Read org settings, create workspaces
Team Lead	Team	Manage team membership
Write	Workspace	Plan and apply
Read	Workspace	View state and outputs only
Plan	Workspace	Run plans but not applies
Admin	Workspace	Manage workspace settings and permissions

Workspace Isolation¶

Each workspace in Terraform Cloud has isolated state storage, variable sets, run history, and access controls. Use separate workspaces for each environment (dev, staging, prod) with escalating RBAC restrictions.

Hardening Checklist¶

Known Pitfalls¶

Pitfall	Risk	Mitigation
Plaintext secrets in state	Credential exposure	Use `sensitive = true`, encrypt state
`terraform state pull` on local disk	Unencrypted state file	Use remote backends exclusively
Unpinned module versions	Supply chain attacks	Pin versions, use lock file
Broad workspace write access	Unauthorized infrastructure changes	Least-privilege RBAC per workspace
Missing state locking	Concurrent apply corruption	Enable DynamoDB/native locking
Stale provider credentials	Compromised long-lived keys	Use OIDC dynamic credentials

Architecture¶

Component Overview¶

Core Components¶

HCL Parser¶

Config Loader and Module System¶

Graph Builder¶

Evaluator¶

Plugin Protocol¶

Protocol Evolution¶

Plugin Handshake¶

Plugin SDKs¶

Plan / Apply Flow¶

State Management¶

State File Structure¶

Backend Options¶

Workspaces¶

HCP Terraform (Terraform Cloud)¶

Comparison with Alternatives¶

References¶

How It Works¶

Core Lifecycle¶

Dependency Graph (DAG)¶

Provider Plugin Architecture¶

State File¶

Sources¶

Benchmarks¶

Plan/Apply Performance¶

Provider API Limits¶

Module Performance¶

State File Benchmarks¶

Sourcing Status¶

Sources¶

Security¶

Authentication Model¶

State File Security¶

State File Risks¶

Terraform Cloud State Security¶

Self-Managed Backend Hardening¶

Sentinel Policy as Code¶

Common Sentinel Policies¶

Provider Credential Management¶

Dynamic Credentials (Recommended)¶

Static Credentials (Legacy)¶

Credential Exposure Prevention¶

Module Registry Security¶

Module Verification¶

Module Security Checklist¶

Terraform Cloud RBAC¶

Workspace Isolation¶

Hardening Checklist¶

Known Pitfalls¶

Related Notes¶