Terraform Environments: A Guide to Managing Different Deployments

• Introduction
• Step-by-Step Guide
• Code Example
• Additional Notes
• Summary
• Conclusion
• References

While Terraform workspaces might seem like an easy way to handle different environments like development, staging, and production, they're not the most effective method. This article presents a superior approach using a combination of directories, modules, variables, and other best practices to manage your infrastructure code for multiple environments in a more organized, scalable, and reliable manner.

Terraform workspaces can be tempting for managing multiple environments (like dev, staging, prod), but they're not the ideal solution. Here's a better approach:

1. Organize with directories: Create separate directories for each environment (e.g., dev/, staging/, prod/). This keeps environment-specific code cleanly separated.

2. Leverage modules: Design reusable Terraform modules for common infrastructure components. This promotes consistency and reduces code duplication.

3. Variables for flexibility: Define variables within each environment's directory to customize deployments. Use .tfvars files or input variables during Terraform runs.

4. Conditional logic: When necessary, use Terraform's conditional logic (count, for_each, if statements) within your modules to create environment-specific variations.

5. Separate state files: Maintain separate state files for each environment. This prevents accidental modifications across environments and ensures independent deployments.

6. CI/CD pipelines: Integrate Terraform with your CI/CD system. This automates deployments, enforces code reviews, and provides a clear deployment history.

Why this approach is better than workspaces:

• Clarity and maintainability: Separate directories and modules make it easier to understand and manage the infrastructure for each environment.
• Reduced blast radius: Isolating state files prevents accidental changes in one environment from affecting others.
• Flexibility: Conditional logic allows you to tailor deployments to specific environment needs without code duplication.
• Version control: Version control systems work best with directories, making it easier to track changes and roll back if needed.

Example:

# modules/network/main.tf
resource "aws_vpc" "main" {
  cidr_block = var.vpc_cidr
}

# environments/dev/main.tf
module "network" {
  source = "../../modules/network"
  vpc_cidr = "10.0.0.0/16"
}

# environments/prod/main.tf
module "network" {
  source = "../../modules/network"
  vpc_cidr = "10.1.0.0/16"
}

This way, you have a clear structure, reusable modules, and environment-specific configurations.

The code represents a Terraform project structured to manage multiple environments (dev, staging, prod) separately. It uses a modular approach, defining a reusable "network" module that creates a VPC and subnets. Each environment directory contains its own configuration files (main.tf, variables.tf) to customize the network module with environment-specific values. This structure promotes code reusability, consistency across environments, and isolation of resources, preventing accidental modifications in one environment from affecting others.

├── modules
│   └── network
│       ├── variables.tf
│       └── main.tf
├── environments
│   ├── dev
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── staging
│   │   ├── main.tf
│   │   └── variables.tf
│   └── prod
│       ├── main.tf
│       └── variables.tf
└── variables.tf

modules/network/variables.tf:

variable "vpc_cidr" {
  type = string
}

variable "create_subnet_public" {
  type = bool
  default = false
}

variable "public_subnet_cidr" {
  type = string
  default = null
}

modules/network/main.tf:

resource "aws_vpc" "main" {
  cidr_block = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support = true

  tags = {
    Name = "vpc-${var.vpc_cidr}"
  }
}

resource "aws_subnet" "public" {
  count = var.create_subnet_public ? 1 : 0
  vpc_id     = aws_vpc.main.id
  cidr_block = var.public_subnet_cidr
  availability_zone = "eu-west-1a"

  tags = {
    Name = "public-subnet-${var.vpc_cidr}"
  }
}

environments/dev/main.tf:

module "network" {
  source = "../../modules/network"
  vpc_cidr = "10.0.0.0/16"
  create_subnet_public = true
  public_subnet_cidr = "10.0.1.0/24"
}

environments/dev/variables.tf:

# Dev environment specific variables

environments/prod/main.tf:

module "network" {
  source = "../../modules/network"
  vpc_cidr = "10.1.0.0/16"
}

environments/prod/variables.tf:

# Prod environment specific variables

How to use:

1. Initialize Terraform: Navigate to the desired environment directory (e.g., environments/dev) and run terraform init.
2. Deploy: Run terraform apply to create or modify infrastructure for that environment.
3. Destroy: Run terraform destroy to delete the resources for that environment.

Benefits:

• Clear separation: Each environment has its own directory, making it easy to understand and manage.
• Reusability: The network module is reused across environments, promoting consistency and reducing code duplication.
• Flexibility: Variables and conditional logic allow for environment-specific customizations.
• Safety: Separate state files prevent accidental modifications across environments.

This is a basic example, and you can expand it further by adding more modules, variables, and conditional logic to suit your specific needs. Remember to integrate this structure with your CI/CD pipeline for automated deployments and a robust workflow.

• Scalability: This approach scales well for a large number of environments and complex infrastructure setups. You can easily add new environments or modules as your needs grow.
• Team Collaboration: Clear separation of environments and modules makes it easier for teams to work on different parts of the infrastructure without conflicts.
• State File Management: Consider using a remote backend for storing state files, especially in a team environment. This ensures state consistency and enables collaboration.
• Environment Variables: Use environment variables to manage sensitive data like API keys and secrets. Never hardcode sensitive information in your Terraform code.
• Terraform Cloud/Enterprise: For larger organizations or more complex workflows, consider using Terraform Cloud or Terraform Enterprise. These platforms offer advanced features like remote state management, collaboration tools, and policy enforcement.
• Documentation: Document your modules and environment configurations clearly. This helps with onboarding new team members and understanding the infrastructure setup.
• Testing: Implement automated testing for your Terraform code to catch errors early and ensure the reliability of your infrastructure deployments.

Remember: This is a general guideline, and the best approach may vary depending on your specific needs and project requirements. Always prioritize clarity, maintainability, and security when designing your Terraform workflow.

While tempting, using Terraform workspaces for managing multiple environments (dev, staging, prod) isn't ideal. This table summarizes a superior approach:

Advantages over Workspaces:

• Enhanced Clarity and Maintainability: Clearer code organization simplifies understanding and management.
• Reduced Blast Radius: Isolated state files prevent unintended consequences across environments.
• Increased Flexibility: Conditional logic allows for tailored deployments without code duplication.
• Improved Version Control: Directory-based structure integrates seamlessly with version control systems for easier tracking and rollback.

Example:

Reusable network module (modules/network/main.tf):

resource "aws_vpc" "main" {
  cidr_block = var.vpc_cidr
}

Environment-specific configurations:

• environments/dev/main.tf:

  module "network" {
    source = "../../modules/network"
    vpc_cidr = "10.0.0.0/16"
  }

• environments/prod/main.tf:

  module "network" {
    source = "../../modules/network"
    vpc_cidr = "10.1.0.0/16"
  }

This approach provides a structured, reusable, and maintainable way to manage multiple environments in Terraform.

In conclusion, while Terraform workspaces might appear as a straightforward solution for managing different environments, a more robust and scalable approach involves leveraging directories, modules, variables, and conditional logic. This method provides a clearer structure, promotes code reusability, and ensures better isolation of environments, ultimately leading to a more maintainable and reliable infrastructure management workflow. By adopting these best practices and integrating them with a robust CI/CD pipeline, organizations can streamline their infrastructure deployments and enhance the overall efficiency of their development process.

• Why does Hashicorp advise against using workspaces to manage ... | Posted by u/Minute_Ad_5524 - 25 votes and 49 comments
• What's the preferred strategy to deploying multiple cloud ... | Hi. There’s not much/anything on the documentation about strategies to deploying to different cloud environments such as dev, staging, prod, etc. Since these envs are mostly the same, it makes sense to share all the code by default and only do conditions when you have differences. However, I don’t want to deploy changes I have made to every env at the same time – I want to be able to deploy changes to dev only or staging to verify things work before I deploy to prod. So, there needs to be a w...
• Refactor monolithic Terraform configuration | Terraform | HashiCorp ... | Separate configuration. Defining multiple environments in the same main.tf file may become hard to manage as you add more resources. The HashiCorp Configuration ...
• GitHub Actions workflow w/multiple environments - Terraform ... | I’m trying to figure out a reasonable branching strategy for GitHub Actions using Terraform with multiple environments. In my previous company, we used Atlantis and had a more traditional setup where you have a directory per environment (i.e. ./dev and ./prod). I’ve been using Terraform Workspaces in my new repository and really like that approach better as it leads to a lot less duplication and ensures consistency between environments. My current GH action setup is basically the same as Automa...
• Different environments for Terraform (Hashicorp) | by Harold Finch ... | In Terraform, managing different environments (such as development, staging, and production) is a common practice to ensure that changes…
• Seperate automatically included .tfvars for different environments ... | Currently, if a terraform.tfvars file is present in the current directory, Terraform automatically loads it to populate variables. With respect to the new environments feature, we have two environm...
• How to manage multiple environments with Terraform using ... | This is part 1 of the How to manage multiple environments with Terraform blog post series. In this post, I’ll show you how to manage…
• Different environments for Terraform (Hashicorp) - Stack Overflow | May 3, 2016 ... I've created a template for maintaining a multiple environment infrastructure with Terraform. This template includes a CI/CD process, that applies the ...
• A Practitioner's Guide to Using HashiCorp Terraform Cloud with ... | Aug 12, 2020 ... ... multiple workspaces in order to handle multiple environments or other cases where similar infrastructure is used in a different context.