Terraform Fundamentals: Why Clicking Is Technical Debt
Table of Contents
You click “Create bucket” in the AWS console. Fifteen minutes later you click it again with slightly different settings. By Friday, you have four buckets, two with public access you did not intend, and no record of which settings you chose for which. Your agent creates resources even faster: a VPC, three subnets, a security group with port 22 open to 0.0.0.0/0, all in a region you did not ask for. Nothing is documented because the console does not document.
Time: About 40 minutes.
Outcome: A Terraform project with remote S3 state, DynamoDB locking, a reusable module structure, and your first deployed resource. Plus a .gitignore and state management strategy that prevents the two most common Terraform disasters.
What We’re Building
- Remote state backend with S3 and DynamoDB locking
- Terraform project structure with environment separation
- Budget alerts codified as infrastructure
.gitignorewith a clear secrets strategy- First
terraform applywith real AWS resources
IaC Philosophy
Terraform is declarative. You describe the end state you want, and Terraform figures out how to get there.
# You write this (what you want):
resource "aws_s3_bucket" "logs" {
bucket = "mycompany-logs"
}
# Terraform figures out:
# - Does this bucket exist?
# - If not, create it
# - If yes, is it configured correctly?
# - If not, update it
Plan before apply. The terraform plan command shows you exactly what will change before you commit to it. No surprises.
┌─────────────────────────────────────────────────────────────────┐
│ TERRAFORM WORKFLOW │
├─────────────────────────────────────────────────────────────────┤
│ │
│ terraform plan │
│ │ │
│ ▼ │
│ Review changes ◄──── "2 to add, 1 to change, 0 to destroy" │
│ │ │
│ │ Looks good? │
│ ▼ │
│ terraform apply │
│ │ │
│ ▼ │
│ Infrastructure updated │
│ │ │
│ ▼ │
│ State file updated │
│ │
└─────────────────────────────────────────────────────────────────┘
| ❌ Under | All infrastructure created manually in the console. No record of what exists or why. State stored locally in terraform.tfstate on one developer’s laptop. Team members cannot collaborate. |
| ✅ Right | Remote state in S3 with DynamoDB locking. Environment separation via workspaces or directory structure. Non-sensitive config in committed terraform.tfvars, secrets in gitignored secret.auto.tfvars. Modules for reusable patterns. |
| ❌ Over | Custom Terraform wrapper scripts. Atlantis or Spacelift before you have a second developer. Terragrunt for a single-environment project. Policy-as-code (Sentinel/OPA) for a 3-person team. |
| 🤖 Agent Trap | Agent generates a 200-line Terraform file with every resource inline, no modules, no variables, and hardcoded values for region, account ID, and bucket names. It works on terraform apply but is unmaintainable, unreusable, and breaks the moment you need a second environment. |
Installing Terraform
Terraform requires version 1.10+ for native S3 state locking (no DynamoDB needed).
Option 1: Homebrew
brew tap hashicorp/tap
brew install hashicorp/tap/terraformOption 2: tfenv (version manager, recommended)
brew install tfenv
tfenv install 1.10.0
tfenv use 1.10.0 Option 1: Package manager (Ubuntu/Debian)
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraformOption 2: tfenv (version manager, recommended)
git clone https://github.com/tfutils/tfenv.git ~/.tfenv
echo 'export PATH="$HOME/.tfenv/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
tfenv install 1.10.0
tfenv use 1.10.0 Option 1: Chocolatey
choco install terraformOption 2: winget
winget install HashiCorp.Terraform Verify Installation
terraform version
Terraform v1.10.0
on darwin_arm64 Editor Setup
Install the HashiCorp Terraform extension for VS Code. It provides:
- Syntax highlighting
- Auto-completion
- Format on save
- Go to definition
HCL Basics
HashiCorp Configuration Language (HCL) is Terraform’s syntax. Here’s the essentials:
HCL Syntax Reference
# Provider: How Terraform talks to AWS
provider "aws" {
region = "us-east-1"
profile = "dev-admin"
}
# Resource: Something you're creating
resource "aws_s3_bucket" "logs" {
bucket = "mycompany-logs"
tags = {
Environment = "production"
}
}
# Data Source: Read existing infrastructure
data "aws_caller_identity" "current" {}
# Variable: Parameterize your config
variable "environment" {
description = "Environment name"
type = string
default = "dev"
}
# Local: Computed value within the module
locals {
bucket_prefix = "${var.project_name}-${var.environment}"
}
# Output: Expose values
output "bucket_arn" {
description = "ARN of the logs bucket"
value = aws_s3_bucket.logs.arn
}References:
# Reference a resource attribute
aws_s3_bucket.logs.id
aws_s3_bucket.logs.arn
# Reference a variable
var.environment
# Reference a local
local.bucket_prefix
# Reference a data source
data.aws_caller_identity.current.account_idWe’ll learn more syntax as we use it. Let’s set up your project.
Project Setup: Directory Structure
Create this structure for your Terraform project:
terraform/ bootstrap/ main.tf outputs.tf modules/ s3-bucket/ main.tf variables.tf outputs.tf environments/ dev/ main.tf backend.tf providers.tf variables.tf outputs.tf terraform.tfvars cloudtrail.tf budgets.tf iam.tf prod/ (same structure) .gitignore
Complete Project Structure Template
# Create the structure
mkdir -p terraform/{bootstrap,modules/s3-bucket,environments/{dev,prod}}
# Create placeholder files
touch terraform/bootstrap/{main.tf,outputs.tf}
touch terraform/modules/s3-bucket/{main.tf,variables.tf,outputs.tf}
touch terraform/environments/dev/{main.tf,backend.tf,providers.tf,variables.tf,outputs.tf,terraform.tfvars,cloudtrail.tf,budgets.tf,iam.tf}
touch terraform/environments/prod/{main.tf,backend.tf,providers.tf,variables.tf,outputs.tf,terraform.tfvars}Why this structure?
bootstrap/: One-time setup (state bucket). Run once, rarely touched.modules/: Reusable components. Write once, use everywhere.environments/: Environment-specific configs. Each environment is isolated.
Terraform .gitignore Template
# terraform/.gitignore
# Local .terraform directories
**/.terraform/*
# .tfstate files
*.tfstate
*.tfstate.*
# Crash log files
crash.log
crash.*.log
# Exclude secret tfvars files
secret.auto.tfvars
# But keep example tfvars
!*.tfvars.example
# Ignore override files
override.tf
override.tf.json
*_override.tf
*_override.tf.json
# Ignore CLI configuration files
.terraformrc
terraform.rc
# Keep the lock file (dependency versions)
!.terraform.lock.hcltfvars strategy: We commit terraform.tfvars for non-sensitive environment config (instance sizes, feature flags, region settings). Only secret.auto.tfvars is gitignored, for values like database passwords. This way your environment configuration is version-controlled and reviewable in PRs, while secrets stay out of the repo.
Step 1: Bootstrap the State Bucket
Terraform tracks what it has created in a state file. By default, this is stored locally (terraform.tfstate), but that’s problematic:
- Lose your laptop? Lose your state.
- Team members can’t collaborate.
- No locking, so two people can corrupt state simultaneously.
We’ll store state in S3 with native locking (Terraform 1.10+).
The chicken-and-egg problem: Terraform state needs S3, but we need Terraform to create S3. Solution: bootstrap with local state first, then migrate.
Bootstrap Template (terraform/bootstrap/main.tf)
# terraform/bootstrap/main.tf
# Run once to create the state bucket. Uses local state.
terraform {
required_version = ">= 1.10.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
provider "aws" {
region = "us-east-1"
profile = "prod-admin" # Use your SSO profile from Part 3
}
variable "project_name" {
description = "Project name (used for bucket naming)"
type = string
default = "mycompany" # Change this
}
# S3 bucket for Terraform state
resource "aws_s3_bucket" "terraform_state" {
bucket = "${var.project_name}-terraform-state"
lifecycle {
prevent_destroy = true
}
tags = {
Name = "Terraform State"
ManagedBy = "terraform-bootstrap"
}
}
# Enable versioning (recover from bad state)
resource "aws_s3_bucket_versioning" "terraform_state" {
bucket = aws_s3_bucket.terraform_state.id
versioning_configuration {
status = "Enabled"
}
}
# Encrypt state at rest
resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
bucket = aws_s3_bucket.terraform_state.id
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "aws:kms"
}
}
}
# Block all public access
resource "aws_s3_bucket_public_access_block" "terraform_state" {
bucket = aws_s3_bucket.terraform_state.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}# terraform/bootstrap/outputs.tf
output "state_bucket_name" {
description = "Name of the S3 bucket for Terraform state"
value = aws_s3_bucket.terraform_state.id
}
output "state_bucket_arn" {
description = "ARN of the S3 bucket for Terraform state"
value = aws_s3_bucket.terraform_state.arn
}
output "state_bucket_region" {
description = "Region of the S3 bucket"
value = aws_s3_bucket.terraform_state.region
}Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
state_bucket_arn = "arn:aws:s3:::mycompany-terraform-state"
state_bucket_name = "mycompany-terraform-state"
state_bucket_region = "us-east-1" Save these outputs. You’ll need the bucket name for the backend configuration.
The bootstrap directory uses local state (stored in terraform/bootstrap/terraform.tfstate). This is intentional: we can’t store bootstrap state in a bucket that doesn’t exist yet. Keep this file safe or commit it to a private repo.
Step 2: Configure Remote Backend
Now configure your environment to use the S3 bucket for state.
Backend Configuration Template
# terraform/environments/dev/backend.tf
terraform {
backend "s3" {
bucket = "mycompany-terraform-state" # Your bucket name
key = "environments/dev/terraform.tfstate"
region = "us-east-1"
encrypt = true
use_lockfile = true # Native S3 locking (Terraform 1.10+)
}
}Key points:
key: Path within the bucket. Use a clear naming convention.use_lockfile = true: Enables native S3 locking. No DynamoDB needed!- Each environment gets its own state file (
dev/,prod/, etc.)
Step 3: Provider Configuration
Configure the AWS provider with sensible defaults.
Provider Configuration Template
# terraform/environments/dev/providers.tf
terraform {
required_version = ">= 1.10.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
provider "aws" {
region = var.aws_region
profile = var.aws_profile # Your SSO profile from Part 3
default_tags {
tags = {
Environment = var.environment
Project = var.project_name
Owner = var.owner
ManagedBy = "terraform"
}
}
}Default tags are applied to every resource. No more forgetting to tag things.
Variables Template
# terraform/environments/dev/variables.tf
variable "aws_region" {
description = "AWS region"
type = string
default = "us-east-1"
}
variable "aws_profile" {
description = "AWS CLI profile to use"
type = string
}
variable "environment" {
description = "Environment name (dev, staging, prod)"
type = string
}
variable "project_name" {
description = "Project name for resource naming and tagging"
type = string
}
variable "owner" {
description = "Owner for tagging"
type = string
}
variable "alert_email" {
description = "Email address for alerts and notifications"
type = string
}# terraform/environments/dev/terraform.tfvars
aws_region = "us-east-1"
aws_profile = "dev-admin" # Your SSO profile from Part 3
environment = "dev"
project_name = "mycompany"
owner = "platform-team"
alert_email = "alerts@mycompany.com"Outputs Template
# terraform/environments/dev/outputs.tf
output "account_id" {
description = "AWS Account ID"
value = data.aws_caller_identity.current.account_id
}
output "region" {
description = "AWS Region"
value = var.aws_region
}
output "environment" {
description = "Environment name"
value = var.environment
}Main Configuration Template
# terraform/environments/dev/main.tf
# Get current AWS account info
data "aws_caller_identity" "current" {}
# Additional resources will be added in the following files:
# - cloudtrail.tf
# - budgets.tf
# - iam.tfInitializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.82.0...
- Installed hashicorp/aws v5.82.0 (signed by HashiCorp)
Terraform has been successfully initialized! Step 4: Codify Part 1, CloudTrail and Budgets
Now let’s recreate what we did manually in Part 1.
CloudTrail Template
# terraform/environments/dev/cloudtrail.tf
# S3 bucket for CloudTrail logs
resource "aws_s3_bucket" "cloudtrail_logs" {
bucket = "${var.project_name}-cloudtrail-logs-${var.environment}"
tags = {
Name = "CloudTrail Logs"
}
}
resource "aws_s3_bucket_versioning" "cloudtrail_logs" {
bucket = aws_s3_bucket.cloudtrail_logs.id
versioning_configuration {
status = "Enabled"
}
}
resource "aws_s3_bucket_server_side_encryption_configuration" "cloudtrail_logs" {
bucket = aws_s3_bucket.cloudtrail_logs.id
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "aws:kms"
}
}
}
resource "aws_s3_bucket_public_access_block" "cloudtrail_logs" {
bucket = aws_s3_bucket.cloudtrail_logs.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
# Bucket policy for CloudTrail
resource "aws_s3_bucket_policy" "cloudtrail_logs" {
bucket = aws_s3_bucket.cloudtrail_logs.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "AWSCloudTrailAclCheck"
Effect = "Allow"
Principal = {
Service = "cloudtrail.amazonaws.com"
}
Action = "s3:GetBucketAcl"
Resource = aws_s3_bucket.cloudtrail_logs.arn
Condition = {
StringEquals = {
"AWS:SourceArn" = "arn:aws:cloudtrail:${var.aws_region}:${data.aws_caller_identity.current.account_id}:trail/${var.project_name}-trail"
}
}
},
{
Sid = "AWSCloudTrailWrite"
Effect = "Allow"
Principal = {
Service = "cloudtrail.amazonaws.com"
}
Action = "s3:PutObject"
Resource = "${aws_s3_bucket.cloudtrail_logs.arn}/AWSLogs/${data.aws_caller_identity.current.account_id}/*"
Condition = {
StringEquals = {
"s3:x-amz-acl" = "bucket-owner-full-control"
"AWS:SourceArn" = "arn:aws:cloudtrail:${var.aws_region}:${data.aws_caller_identity.current.account_id}:trail/${var.project_name}-trail"
}
}
}
]
})
}
# CloudTrail
resource "aws_cloudtrail" "main" {
name = "${var.project_name}-trail"
s3_bucket_name = aws_s3_bucket.cloudtrail_logs.id
include_global_service_events = true
is_multi_region_trail = true
enable_log_file_validation = true
event_selector {
read_write_type = "All"
include_management_events = true
}
tags = {
Name = "Main CloudTrail"
}
depends_on = [aws_s3_bucket_policy.cloudtrail_logs]
}
# Outputs
output "cloudtrail_name" {
description = "Name of the CloudTrail"
value = aws_cloudtrail.main.name
}
output "cloudtrail_logs_bucket" {
description = "S3 bucket for CloudTrail logs"
value = aws_s3_bucket.cloudtrail_logs.id
}Budget Alerts Template
# terraform/environments/dev/budgets.tf
resource "aws_budgets_budget" "monthly" {
name = "${var.project_name}-monthly-budget"
budget_type = "COST"
limit_amount = "50" # Adjust based on your expected spend
limit_unit = "USD"
time_unit = "MONTHLY"
time_period_start = "2024-01-01_00:00"
# Alert at 50% (forecasted)
notification {
comparison_operator = "GREATER_THAN"
threshold = 50
threshold_type = "PERCENTAGE"
notification_type = "FORECASTED"
subscriber_email_addresses = [var.alert_email]
}
# Alert at 80% (actual)
notification {
comparison_operator = "GREATER_THAN"
threshold = 80
threshold_type = "PERCENTAGE"
notification_type = "ACTUAL"
subscriber_email_addresses = [var.alert_email]
}
# Alert at 100% (actual)
notification {
comparison_operator = "GREATER_THAN"
threshold = 100
threshold_type = "PERCENTAGE"
notification_type = "ACTUAL"
subscriber_email_addresses = [var.alert_email]
}
# Alert at 150% (actual) - something is wrong
notification {
comparison_operator = "GREATER_THAN"
threshold = 150
threshold_type = "PERCENTAGE"
notification_type = "ACTUAL"
subscriber_email_addresses = [var.alert_email]
}
tags = {
Name = "Monthly Cost Budget"
}
}
output "budget_name" {
description = "Name of the budget"
value = aws_budgets_budget.monthly.name
}Terraform will perform the following actions:
# aws_budgets_budget.monthly will be created
# aws_cloudtrail.main will be created
# aws_s3_bucket.cloudtrail_logs will be created
# aws_s3_bucket_policy.cloudtrail_logs will be created
# aws_s3_bucket_public_access_block.cloudtrail_logs will be created
# aws_s3_bucket_server_side_encryption_configuration.cloudtrail_logs will be created
# aws_s3_bucket_versioning.cloudtrail_logs will be created
Plan: 7 to add, 0 to change, 0 to destroy. Step 5: Codify Part 2, IAM
Now let’s codify the IAM setup from Part 2.
IAM Developer Group Template
# terraform/environments/dev/iam.tf
# =============================================================================
# DEVELOPER GROUP
# =============================================================================
resource "aws_iam_group" "developers" {
name = "Developers"
}
# Attach PowerUserAccess (everything except IAM and Organizations)
resource "aws_iam_group_policy_attachment" "developers_power_user" {
group = aws_iam_group.developers.name
policy_arn = "arn:aws:iam::aws:policy/PowerUserAccess"
}
# Deny dangerous actions even for power users
resource "aws_iam_group_policy" "developers_deny_dangerous" {
name = "DenyDangerousActions"
group = aws_iam_group.developers.name
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "DenyAccountLevelActions"
Effect = "Deny"
Action = [
"organizations:*",
"account:*",
"iam:CreateUser",
"iam:DeleteUser",
"iam:UpdateUser",
"iam:CreateGroup",
"iam:DeleteGroup",
"iam:UpdateGroup",
"iam:AttachUserPolicy",
"iam:DetachUserPolicy",
"iam:AttachGroupPolicy",
"iam:DetachGroupPolicy",
"iam:PutUserPolicy",
"iam:DeleteUserPolicy",
"iam:PutGroupPolicy",
"iam:DeleteGroupPolicy",
]
Resource = "*"
},
{
Sid = "DenyCloudTrailModification"
Effect = "Deny"
Action = [
"cloudtrail:DeleteTrail",
"cloudtrail:StopLogging",
"cloudtrail:UpdateTrail"
]
Resource = "*"
}
]
})
}
# Allow developers to manage their own MFA
resource "aws_iam_group_policy" "developers_self_service_mfa" {
name = "SelfServiceMFA"
group = aws_iam_group.developers.name
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "AllowManageOwnMFA"
Effect = "Allow"
Action = [
"iam:CreateVirtualMFADevice",
"iam:DeleteVirtualMFADevice",
"iam:EnableMFADevice",
"iam:ResyncMFADevice",
"iam:ListMFADevices"
]
Resource = [
"arn:aws:iam::*:mfa/$${aws:username}",
"arn:aws:iam::*:user/$${aws:username}"
]
},
{
Sid = "AllowViewAccountInfo"
Effect = "Allow"
Action = [
"iam:GetAccountPasswordPolicy",
"iam:ListVirtualMFADevices"
]
Resource = "*"
}
]
})
}
output "developers_group_name" {
description = "Name of the developers IAM group"
value = aws_iam_group.developers.name
}
output "developers_group_arn" {
description = "ARN of the developers IAM group"
value = aws_iam_group.developers.arn
}CI/CD Role Template (GitHub Actions OIDC)
# terraform/environments/dev/iam-cicd.tf
# =============================================================================
# GITHUB ACTIONS OIDC
# =============================================================================
# OIDC Provider for GitHub Actions
resource "aws_iam_openid_connect_provider" "github" {
url = "https://token.actions.githubusercontent.com"
client_id_list = ["sts.amazonaws.com"]
thumbprint_list = ["6938fd4d98bab03faadb97b34396831e3780aea1", "1c58a3a8518e8759bf075b76b750d4f2df264fcd"]
tags = {
Name = "GitHub Actions OIDC Provider"
}
}
# Role for GitHub Actions
variable "github_org" {
description = "GitHub organization name"
type = string
default = "your-org" # Change this
}
variable "github_repo" {
description = "GitHub repository name"
type = string
default = "your-repo" # Change this
}
resource "aws_iam_role" "github_actions" {
name = "GitHubActionsRole-${var.environment}"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Principal = {
Federated = aws_iam_openid_connect_provider.github.arn
}
Action = "sts:AssumeRoleWithWebIdentity"
Condition = {
StringEquals = {
"token.actions.githubusercontent.com:aud" = "sts.amazonaws.com"
}
StringLike = {
# Allow from any branch in your repo
"token.actions.githubusercontent.com:sub" = "repo:${var.github_org}/${var.github_repo}:*"
}
}
}
]
})
tags = {
Name = "GitHub Actions Role"
}
}
# Attach deployment permissions
# Scope this down based on what your CI/CD actually needs
resource "aws_iam_role_policy_attachment" "github_actions_deploy" {
role = aws_iam_role.github_actions.name
policy_arn = "arn:aws:iam::aws:policy/PowerUserAccess"
}
output "github_actions_role_arn" {
description = "ARN of the GitHub Actions role"
value = aws_iam_role.github_actions.arn
}Lambda Execution Role Template
# terraform/environments/dev/iam-lambda.tf
# =============================================================================
# LAMBDA EXECUTION ROLE (Reusable pattern)
# =============================================================================
resource "aws_iam_role" "lambda_execution" {
name = "${var.project_name}-lambda-execution-${var.environment}"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Principal = {
Service = "lambda.amazonaws.com"
}
Action = "sts:AssumeRole"
}
]
})
tags = {
Name = "Lambda Execution Role"
}
}
# Basic execution (CloudWatch Logs)
resource "aws_iam_role_policy_attachment" "lambda_basic" {
role = aws_iam_role.lambda_execution.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
# Custom permissions - add as needed
resource "aws_iam_role_policy" "lambda_custom" {
name = "CustomPermissions"
role = aws_iam_role.lambda_execution.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "S3Access"
Effect = "Allow"
Action = [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
]
Resource = "arn:aws:s3:::${var.project_name}-*/*"
},
{
Sid = "DynamoDBAccess"
Effect = "Allow"
Action = [
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:UpdateItem",
"dynamodb:DeleteItem",
"dynamodb:Query",
"dynamodb:Scan"
]
Resource = "arn:aws:dynamodb:${var.aws_region}:${data.aws_caller_identity.current.account_id}:table/${var.project_name}-*"
}
]
})
}
output "lambda_execution_role_arn" {
description = "ARN of the Lambda execution role"
value = aws_iam_role.lambda_execution.arn
}
output "lambda_execution_role_name" {
description = "Name of the Lambda execution role"
value = aws_iam_role.lambda_execution.name
}Step 6: Create Your First Module
Modules are reusable Terraform components. When you find yourself copying similar resource blocks, it’s time for a module.
Rule of three: If you’ve written the same pattern three times, make it a module.
S3 Bucket Module Template
# terraform/modules/s3-bucket/variables.tf
variable "bucket_name" {
description = "Name of the S3 bucket"
type = string
}
variable "versioning" {
description = "Enable versioning"
type = bool
default = true
}
variable "force_destroy" {
description = "Allow bucket to be destroyed even with objects"
type = bool
default = false
}
variable "tags" {
description = "Additional tags for the bucket"
type = map(string)
default = {}
}# terraform/modules/s3-bucket/main.tf
resource "aws_s3_bucket" "this" {
bucket = var.bucket_name
force_destroy = var.force_destroy
tags = merge(
{ Name = var.bucket_name },
var.tags
)
}
resource "aws_s3_bucket_versioning" "this" {
bucket = aws_s3_bucket.this.id
versioning_configuration {
status = var.versioning ? "Enabled" : "Disabled"
}
}
resource "aws_s3_bucket_server_side_encryption_configuration" "this" {
bucket = aws_s3_bucket.this.id
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "aws:kms"
}
}
}
resource "aws_s3_bucket_public_access_block" "this" {
bucket = aws_s3_bucket.this.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}# terraform/modules/s3-bucket/outputs.tf
output "bucket_id" {
description = "ID of the bucket"
value = aws_s3_bucket.this.id
}
output "bucket_arn" {
description = "ARN of the bucket"
value = aws_s3_bucket.this.arn
}
output "bucket_domain_name" {
description = "Domain name of the bucket"
value = aws_s3_bucket.this.bucket_domain_name
}
output "bucket_regional_domain_name" {
description = "Regional domain name of the bucket"
value = aws_s3_bucket.this.bucket_regional_domain_name
}Using the module:
# In any environment file
module "logs_bucket" {
source = "../../modules/s3-bucket"
bucket_name = "${var.project_name}-logs-${var.environment}"
versioning = true
tags = {
Purpose = "Application Logs"
}
}
module "uploads_bucket" {
source = "../../modules/s3-bucket"
bucket_name = "${var.project_name}-uploads-${var.environment}"
versioning = false
force_destroy = var.environment == "dev" # Only in dev
tags = {
Purpose = "User Uploads"
}
}
# Use outputs
output "logs_bucket_arn" {
value = module.logs_bucket.bucket_arn
}Essential Commands Reference
Terraform Commands Cheatsheet
| Command | Description |
|---|---|
terraform init | Initialize working directory, download providers |
terraform plan | Preview changes without applying |
terraform apply | Apply changes (creates/updates/deletes resources) |
terraform destroy | Destroy all managed resources |
terraform fmt | Format code to canonical style |
terraform fmt -check | Check if code is formatted (CI use) |
terraform validate | Validate configuration syntax |
terraform state list | List all resources in state |
terraform state show <resource> | Show details of a specific resource |
terraform output | Show all outputs |
terraform output <name> | Show specific output |
terraform refresh | Sync state with actual infrastructure |
terraform import <resource> <id> | Import existing resource into state |
terraform graph | Generate dependency graph (DOT format) |
Common flags:
| Flag | Description |
|---|---|
-auto-approve | Skip confirmation prompt (use in CI) |
-var="key=value" | Set a variable |
-var-file=file.tfvars | Use a specific variables file |
-target=resource | Apply to specific resource only |
-parallelism=n | Limit concurrent operations |
Working with Existing Resources
If you created resources manually in Part 1-3, you can bring them under Terraform management.
Import Block Template (Terraform 1.5+)
Modern Terraform uses declarative imports in your config files:
# Import an existing S3 bucket
import {
to = aws_s3_bucket.existing_logs
id = "mycompany-existing-logs-bucket"
}
# Write the resource configuration
resource "aws_s3_bucket" "existing_logs" {
bucket = "mycompany-existing-logs-bucket"
}Then run:
# Generate the full configuration
terraform plan -generate-config-out=imported.tf
# Review imported.tf and merge into your config
# Then apply
terraform applyClassic import (still works):
# Import an IAM user
terraform import aws_iam_user.admin your-admin-username
# Import an S3 bucket
terraform import aws_s3_bucket.logs my-existing-bucketWhen to import vs recreate:
- Import when the resource has data you can’t lose (S3 buckets with objects, databases)
- Recreate when it’s easier to destroy and recreate (empty buckets, IAM roles)
Quick Reference: AWS CLI Profiles (from Part 3)
AWS CLI SSO Profile Template
# ~/.aws/config
[sso-session my-sso]
sso_start_url = https://d-xxxxxxxxxx.awsapps.com/start
sso_region = us-east-1
sso_registration_scopes = sso:account:access
[profile dev-admin]
sso_session = my-sso
sso_account_id = 111111111111
sso_role_name = AdministratorAccess
region = us-east-1
output = json
[profile prod-admin]
sso_session = my-sso
sso_account_id = 222222222222
sso_role_name = AdministratorAccess
region = us-east-1
output = json
[profile prod-readonly]
sso_session = my-sso
sso_account_id = 222222222222
sso_role_name = ViewOnlyAccess
region = us-east-1
output = jsonUsage with Terraform:
provider "aws" {
region = "us-east-1"
profile = "dev-admin" # Uses SSO profile
}# Login before running Terraform
aws sso login --profile dev-adminQuick Reference: IAM Policy Patterns (from Part 2)
Common IAM Policy Patterns
# Pattern 1: Service-specific access
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::bucket-name/*"
}
# Pattern 2: Resource tag-based conditions
{
"Effect": "Allow",
"Action": "ec2:*",
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/Environment": "dev"
}
}
}
# Pattern 3: Deny with exceptions
{
"Effect": "Deny",
"Action": "ec2:TerminateInstances",
"Resource": "*",
"Condition": {
"StringNotEquals": {
"aws:ResourceTag/Environment": "dev"
}
}
}
# Pattern 4: Require MFA
{
"Effect": "Deny",
"Action": "*",
"Resource": "*",
"Condition": {
"BoolIfExists": {
"aws:MultiFactorAuthPresent": "false"
}
}
}IAM Trust Policy Patterns
# AWS Service (Lambda, EC2, ECS)
{
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
# Cross-account access
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::OTHER_ACCOUNT_ID:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "your-external-id"
}
}
}
# GitHub Actions OIDC
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/token.actions.githubusercontent.com"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
},
"StringLike": {
"token.actions.githubusercontent.com:sub": "repo:org/repo:*"
}
}
}Quick Reference: Terraform Patterns
Common Terraform Patterns
# Pattern: Conditional resource creation
resource "aws_instance" "bastion" {
count = var.create_bastion ? 1 : 0
# ...
}
# Access conditional resource
output "bastion_ip" {
value = var.create_bastion ? aws_instance.bastion[0].public_ip : null
}
# Pattern: For each from list
variable "user_names" {
default = ["alice", "bob", "carol"]
}
resource "aws_iam_user" "users" {
for_each = toset(var.user_names)
name = each.value
}
# Pattern: For each from map
variable "buckets" {
default = {
logs = { versioning = true }
uploads = { versioning = false }
}
}
resource "aws_s3_bucket" "buckets" {
for_each = var.buckets
bucket = "${var.project_name}-${each.key}"
}
# Pattern: Dynamic blocks
variable "ingress_rules" {
default = [
{ port = 80, cidr = ["0.0.0.0/0"] },
{ port = 443, cidr = ["0.0.0.0/0"] },
]
}
resource "aws_security_group" "web" {
name = "web"
dynamic "ingress" {
for_each = var.ingress_rules
content {
from_port = ingress.value.port
to_port = ingress.value.port
protocol = "tcp"
cidr_blocks = ingress.value.cidr
}
}
}
# Pattern: Data source lookup
data "aws_ami" "amazon_linux" {
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["al2023-ami-*-x86_64"]
}
}
# Pattern: Moved block (refactoring without recreate)
moved {
from = aws_s3_bucket.old_name
to = aws_s3_bucket.new_name
}
# Pattern: Lifecycle rules
resource "aws_instance" "critical" {
# ...
lifecycle {
prevent_destroy = true # Can't be destroyed
ignore_changes = [tags] # Ignore external tag changes
}
}Common Mistakes & Troubleshooting
Common Errors and Fixes
| Error | Cause | Fix |
|---|---|---|
Error acquiring state lock | Someone else is running Terraform | Wait, or use terraform force-unlock LOCK_ID (careful!) |
Resource already exists | Resource created outside Terraform | Import it or delete manually first |
Invalid provider configuration | Wrong profile or region | Check aws_profile variable, run aws sso login |
Cycle detected | Resources depend on each other | Refactor to break the cycle, use depends_on carefully |
Provider produced inconsistent result | Provider bug or API issue | Run terraform apply again, or upgrade provider |
Debugging tips:
# Enable verbose logging
export TF_LOG=DEBUG
terraform plan
# Show state for a resource
terraform state show aws_s3_bucket.logs
# Check who's holding the lock
aws s3 ls s3://your-state-bucket/environments/dev/ --recursiveValidation Checklist
Verify your Terraform setup is complete:
INSTALLATION
PROJECT SETUP
REMOTE STATE
RESOURCES CODIFIED
MODULES
0 of 0 completed
Key Takeaways
- Remote state in S3 with DynamoDB locking is non-negotiable from day one, because recovering from a corrupted local state file means reimporting every resource by hand.
- Commit
terraform.tfvarsfor non-sensitive environment config and gitignoresecret.auto.tfvarsfor secrets, so your environment settings are reviewable in PRs. - Run
terraform planbefore everyapplyand read the output, because the plan is the only moment you can catch an agent’s mistake before it becomes infrastructure. importblocks bring existing console-created resources under Terraform management without recreating them, which is how you migrate from clicking to coding.- Start with flat files and extract modules only when you copy-paste the same resource block a third time.
What’s Coming
Next in Part 5: OpenTelemetry Setup, we deploy observability infrastructure with Terraform. The patterns you learned today (modules, remote state, environment config) carry forward into every Terraform file for the rest of the series.
Have questions? Found an error? Open an issue or reach out on LinkedIn.
Related Posts
Your First 60 Minutes in AWS: What Everyone Skips and Regrets Later
What everyone skips and regrets later. Set up your AWS account the right way: secure enough to sleep at night, without the enterprise overhead that kills velocity.
IAM: The Key to Everything
Least privilege without losing your mind. Build a working IAM setup for your startup, not enterprise complexity, not 'just use AdministratorAccess.' The practical middle ground.
Your Local AWS Setup: Stop Hardcoding Access Keys Forever
Stop hardcoding access keys forever. Configure your local development environment the right way: AWS CLI, IAM Identity Center, and credential security that scales with your team.
OpenTelemetry Setup: Observability That Grows With You
Set up the OTel Collector once, choose your backend. SigNoz or Grafana stack. Your observability foundation for the entire series.
Git & GitHub: The Foundation Your Whole Team Builds On
The foundation your whole team, humans and agents, builds on. Git conventions, agent commit hygiene, and attribution that tracks who wrote what.
Branch Protection & PR Workflow: Quality Gates for Humans and Agents
Quality gates that catch both human and agent mistakes. Branch protection, PR templates with agent-review sections, and CODEOWNERS for your growing team.
Pre-Commit Hooks & Code Quality: Your Agents' First Guardrail
Your agents' first guardrail, and your Scorecard's first data source. Pre-commit hooks that catch what human review misses, especially from AI-generated code.