Modules should do one thing

A module that creates a VPC, a database, a cache, and a monitoring stack is not a module. It’s a novel. Split it.

modules/
  vpc/
    main.tf
    variables.tf
    outputs.tf
  database/
    main.tf
    variables.tf
    outputs.tf
  monitoring/
    main.tf
    variables.tf
    outputs.tf

Each module should be understandable in one read. If you need to scroll to understand what a module creates, it’s doing too much.

Variables need contracts

Every variable should have a type, a description, and a default when a sensible one exists.

variable "instance_type" {
  type        = string
  description = "EC2 instance class for the app servers"
  default     = "t3.medium"

  validation {
    condition     = can(regex("^t3\\.", var.instance_type))
    error_message = "Only t3 instance types are supported for this workload."
  }
}

variable "enable_monitoring" {
  type        = bool
  description = "Whether to attach CloudWatch agent and alarms"
  default     = true
}

The validation block is not optional. It’s the difference between a typo caught at plan time and a production outage at apply time.

Outputs are the API

A module’s outputs are its public interface. Export what consumers need, nothing more.

output "vpc_id" {
  description = "The ID of the created VPC"
  value       = aws_vpc.main.id
}

output "private_subnet_ids" {
  description = "IDs of the private subnets, for use by app modules"
  value       = aws_subnet.private[*].id
}

Don’t export every attribute of every resource. That couples consumers to implementation details you might want to change later.

State is sacred

Remote state, state locking, and a clear separation between environments. Non-negotiable.

terraform {
  backend "s3" {
    bucket         = "terraform-state-prod"
    key            = "app/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"
    encrypt        = true
  }
}

One state file per environment per component. Never share state between dev and prod. Never run terraform apply from a laptop against production state.

The test that matters

The best Terraform test is terraform plan in CI on every pull request. If the plan output is clean, the change is probably safe. If it’s a wall of red, something is wrong.

Readable infrastructure is a team sport. Write it for the person who has to change it at 2am, because eventually that person is you.

Output is the interface

Your CLI’s output is its UI. Treat it that way.

use std::io::{self, Write};

fn main() {
    // Wrong: dumping raw data
    println!("{:?}", results);

    // Right: structured, scannable output
    for item in &results {
        println!("  {}  {:<24}  {}", item.status_icon(), item.name, item.detail);
    }

    // Always flush before exit on interactive output
    io::stdout().flush().unwrap();
}

Align columns. Use color to communicate state, not to decorate. Red means failure, green means success, yellow means attention. Never use color as the only signal, and always respect NO_COLOR.

Errors should teach

A good error message tells the user what went wrong and what to do next.

// Wrong
eprintln!("Error: connection failed");

// Right
eprintln!("error: could not reach registry.example.com:443");
eprintln!("  hint: check your network connection or VPN status");
eprintln!("  hint: run with --offline to use the local cache");

Exit codes matter too. Zero for success, non-zero for failure, and different codes for different failure classes so scripts can branch on them.

Flags should feel inevitable

Follow the conventions users already know:

• -h / --help for help
• -v / --verbose for more output
• -q / --quiet for less output
• --version for version info
• --dry-run when the tool changes state

Use clap with derive macros. The help output it generates is better than anything you’ll write by hand.

use clap::Parser;

#[derive(Parser)]
#[command(name = "wiretap", about = "Inspect HTTP traffic from the terminal")]
struct Args {
    /// Port to listen on
    #[arg(short, long, default_value_t = 8080)]
    port: u16,

    /// Filter by route pattern
    #[arg(short, long)]
    filter: Option<String>,

    /// Output raw JSON instead of formatted text
    #[arg(long)]
    json: bool,
}

Speed is a feature

Users notice startup time. If your tool takes 400ms to print help, it feels broken. Keep the binary small, avoid lazy network calls at startup, and profile the cold path.

The best CLI tools feel instant. That’s not an accident. It’s a design decision.

The failure mode that matters

Most pipelines fail in the same way: a change passes CI, lands in production, and breaks something that CI never tested. The fix is not more tests. The fix is a pipeline that assumes failure and routes around it.

Three rules

1. Every deploy is reversible in one step. If rolling back requires a meeting, your pipeline is broken. Rollback should be a button, not a procedure.

2. Health checks gate the rollout. A deploy that reports success while the app is crash-looping is a liar. The pipeline should verify the service is actually healthy before it calls the job done.

3. Small batches beat big bangs. Ten small deploys a day is safer than one large deploy a week. The blast radius of each change stays small, and the cause of any failure is obvious.

A minimal GitLab pipeline

stages:
  - test
  - build
  - deploy
  - verify

test:
  stage: test
  script:
    - bundle exec rspec

build:
  stage: build
  script:
    - docker build -t app:$CI_COMMIT_SHORT_SHA .
    - docker push registry.example.com/app:$CI_COMMIT_SHORT_SHA

deploy:
  stage: deploy
  script:
    - ./scripts/deploy.sh $CI_COMMIT_SHORT_SHA
  environment:
    name: production

verify:
  stage: verify
  script:
    - ./scripts/healthcheck.sh || ./scripts/rollback.sh

The verify stage is the part most teams skip. It’s also the part that matters most. If the health check fails, the pipeline rolls back automatically and alerts the team. No human required.

What “healthy” means

A health check is not a 200 response from /. It’s a check that the service can actually do its job: connect to the database, reach its dependencies, and respond within a latency budget.

#!/usr/bin/env bash
# healthcheck.sh
set -euo pipefail

for i in {1..12}; do
  if curl -sf --max-time 5 https://app.example.com/health/deep > /dev/null; then
    echo "healthy"
    exit 0
  fi
  sleep 10
done

echo "unhealthy after 2 minutes"
exit 1

The /health/deep endpoint checks the database connection, the cache, and any critical downstream services. A shallow ping tells you the process is alive. A deep check tells you the service is working.

The real lesson

Pipelines are not about automation. They’re about trust. When the team trusts the pipeline, they deploy often, they take smaller risks, and they sleep through the night. When they don’t, they batch changes, delay releases, and get woken up anyway.

Build the pipeline you’d trust at 3am. Then never get woken up by it.