What is the Terraform state bootstrap problem?

Terraform needs an S3 bucket to store state, but you need Terraform to create the bucket. Solve it by first running with local state to create the bucket, then migrating to the S3 backend with "tofu init" and answering yes to migrate.

Why use DynamoDB for Terraform state locking?

DynamoDB provides atomic locking to prevent concurrent Terraform runs from corrupting state. When one apply is running, others wait or fail. This is essential for CI/CD where multiple pipelines might run simultaneously.

How do multiple repositories share the same Terraform state backend?

Each repository uses the same S3 bucket but a unique "key" (path within the bucket). The DynamoDB lock table is shared - locks are per-key, so different repositories don't block each other.

Solving the State Problem - Terraform Backend Bootstrap

Terraform (and OpenTofu) need somewhere to store state. The state file tracks what resources exist and their current configuration. Without it, Terraform doesn't know what's already deployed.

The obvious choice is an S3 bucket with DynamoDB for locking. But there's a problem: how do you create the S3 bucket with Terraform if Terraform needs the S3 bucket to store its state?

This is the bootstrap problem, and solving it requires a two-phase deployment.

Why Remote State Matters

Local state files (the default) have serious problems:

No collaboration: Only one person has the state file
No locking: Concurrent runs can corrupt state
No versioning: Mistakes are permanent
Security risk: State contains sensitive data in plaintext

Remote state in S3 solves all of these:

The S3 Bucket

Here's the OpenTofu configuration for the state bucket:

hcl

module "terraform_state_bucket" {
  source  = "terraform-aws-modules/s3-bucket/aws"
  version = "~> 4.1"

  bucket = "${var.organization_name}-terraform-state-${data.aws_caller_identity.current.account_id}"

  # Versioning protects against accidental deletion
  versioning = {
    enabled = true
  }

  # Encryption at rest
  server_side_encryption_configuration = {
    rule = {
      apply_server_side_encryption_by_default = {
        sse_algorithm = "AES256"
      }
    }
  }

  # Block all public access
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true

  # Require HTTPS
  attach_deny_insecure_transport_policy = true
}

Naming convention: We include the account ID in the bucket name to ensure uniqueness across AWS. S3 bucket names are globally unique.

Versioning: Critical. If state gets corrupted, you can recover a previous version.

Encryption: AES256 is sufficient. KMS adds cost and complexity without significant benefit for state files.

HTTPS-only: The attach_deny_insecure_transport_policy adds a bucket policy denying non-HTTPS requests.

The DynamoDB Lock Table

DynamoDB provides state locking - preventing concurrent Terraform runs from corrupting state:

hcl

module "terraform_state_lock" {
  source  = "terraform-aws-modules/dynamodb-table/aws"
  version = "~> 4.0"

  name         = "terraform-state-lock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attributes = [
    {
      name = "LockID"
      type = "S"
    }
  ]

  deletion_protection_enabled = true
  point_in_time_recovery_enabled = true

  server_side_encryption_enabled = true
}

billing_mode = "PAY_PER_REQUEST": On-demand pricing. State locking operations are infrequent, so pay-per-request is cheaper than provisioned capacity.

deletion_protection_enabled: Prevents accidental deletion of the lock table.

point_in_time_recovery_enabled: Allows recovery if the table is corrupted or accidentally modified.

The Bootstrap Process

The backend configuration lives in the code from day one - no commenting or uncommenting required. OpenTofu's -backend=false flag handles the chicken-and-egg problem cleanly.

The providers.tf contains the final backend configuration:

hcl

terraform {
  required_version = ">= 1.6"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket         = "myorg-terraform-state-123456789012"
    key            = "aws-bootstrap/terraform.tfstate"
    region         = "eu-west-2"
    encrypt        = true
    dynamodb_table = "terraform-state-lock"
  }
}

Phase 1: Create the Backend Infrastructure

Tell OpenTofu to skip backend initialisation. It will use local state temporarily:

bash

cd tf
tofu init -backend=false   # Skips S3 backend, uses local state
tofu plan
tofu apply                 # Creates S3 bucket and DynamoDB table

The -backend=false flag tells OpenTofu to ignore the backend block entirely for this run. The S3 bucket and DynamoDB table now exist, but state is still local.

Phase 2: Migrate to Remote State

Run init again, this time without the flag. OpenTofu detects the backend configuration and offers to migrate:

bash

tofu init -migrate-state

# OpenTofu will ask:
# "Do you want to copy existing state to the new backend?"
# Answer: yes

The local state is now migrated to S3. Delete the local terraform.tfstate - it's no longer needed.

The payoff: other repositories use the same backend. Each repository just needs a unique key:

hcl

# In aws-identity-management
terraform {
  backend "s3" {
    bucket         = "myorg-terraform-state-123456789012"
    key            = "aws-identity-management/terraform.tfstate"  # Different key
    region         = "eu-west-2"
    encrypt        = true
    dynamodb_table = "terraform-state-lock"  # Same lock table
  }
}

# In aws-dns-management
terraform {
  backend "s3" {
    bucket         = "myorg-terraform-state-123456789012"
    key            = "aws-dns-management/terraform.tfstate"  # Different key
    region         = "eu-west-2"
    encrypt        = true
    dynamodb_table = "terraform-state-lock"  # Same lock table
  }
}

The lock table is shared - DynamoDB locks are per-key, so different repositories don't block each other.

Backend Configuration for CI/CD

For GitHub Actions, we pass backend configuration via a partial config file:

hcl

# backend.conf (not committed to git)
bucket         = "myorg-terraform-state-123456789012"
region         = "eu-west-2"
encrypt        = true
dynamodb_table = "terraform-state-lock"

bash

tofu init -backend-config=backend.conf

This keeps the bucket name out of the code, allowing the same repository to target different environments.

State Locking in Action

When Terraform runs, it acquires a lock:

Acquiring state lock. This may take a few moments...

If another process is already running, you'll see:

Error: Error acquiring the state lock

Error message: ConditionalCheckFailedException: The conditional request failed
Lock Info:
  ID:        a1b2c3d4-...
  Path:      myorg-terraform-state-123456789012/aws-bootstrap/terraform.tfstate
  Operation: OperationTypeApply
  Who:       user@hostname
  Created:   2026-01-15 10:30:00 UTC

This prevents two people (or two CI jobs) from modifying infrastructure simultaneously.

Force Unlocking

If a Terraform run crashes without releasing the lock:

bash

tofu force-unlock LOCK_ID

Use this carefully - only when you're certain no other process is running.

Cost

This entire setup costs almost nothing:

S3: A few megabytes of state files, versioned. Pennies per month.
DynamoDB: Pay-per-request with minimal reads/writes. Typically under $1/month.

The state bucket for aws-bootstrap, aws-identity-management, and aws-dns-management combined uses less than 1MB of storage.

What Can Go Wrong

Forgetting -backend=false: If you run tofu init before the bucket exists, it fails. Use -backend=false for the first run, then -migrate-state after the bucket is created.

Different bucket names in different repos: Keep them consistent. The bootstrap repository outputs the bucket name and lock table name - other repositories reference these values in their backend configuration (as shown in the "Sharing Across Repositories" section above).

State file corruption: That's why versioning is enabled. Recover a previous version from S3 if needed.

Lock table deleted: With deletion_protection_enabled, this requires deliberate action. If it happens, recreate the table with the same name.

Summary

The bootstrap problem is solved with a simple pattern:

Deploy with local state to create the backend infrastructure
Configure the backend and migrate state
Never think about it again

All subsequent changes flow through CI/CD using the remote backend. The state is versioned, encrypted, locked, and shared across repositories.

In practice, we found a peer-reviewed init/migrate sequence avoids the common pitfalls: inconsistent backend configuration between repos, forgotten migration steps, and emergency manual state edits.