Split Terraform main.tf into Multiple Files (No Modules)

• Introduction
• Step-by-Step Guide
• Code Example
• Additional Notes
• Summary
• Conclusion
• References

In Terraform, you have the flexibility to organize your code into multiple .tf files for better management. Although Terraform treats all .tf files in a directory as a single unit, splitting your code into separate files can enhance readability and collaboration. You can group code by functionality, resource types, or any other logical criteria. For instance, you could have separate files for variables, outputs, networking resources, and storage resources. While Terraform doesn't inherently recognize subdirectories as modules, you can create reusable modules by following a specific structure. Keep in mind that Terraform processes all .tf files in a directory as one, so be aware of dependencies between resources defined in different files. When naming your files, use descriptive names and maintain consistency in your organization structure. The ultimate goal is to enhance code clarity and maintainability, so choose a structure that aligns with your project's needs and team's preferences.

Terraform doesn't impose strict rules on how you name or organize your .tf files. It treats all .tf files within a directory as a single collection of code. This means you can split your code into multiple files based on your preference, making it easier to manage and read, especially for large projects.

Let's say you have a large main.tf file. You can break it down into smaller files like variables.tf for variables, outputs.tf for outputs, and separate files for different resource types, like networking.tf for networking resources and storage.tf for storage resources.

While you can organize your code into subdirectories, Terraform doesn't inherently treat them as modules. If you want to create reusable modules, you need to follow the module structure, which involves placing your module code in a separate directory and calling it using the module block.

Splitting your code into multiple files offers several benefits. It improves readability by separating different aspects of your infrastructure code. It also simplifies collaboration by allowing multiple team members to work on different files simultaneously.

However, it's important to remember that Terraform will still process all .tf files in a directory as one. This means you need to be mindful of dependencies between resources defined in different files.

While there are no strict naming conventions, it's good practice to use descriptive names for your files and organize them logically. For instance, you can group files by resource type, environment, or any other relevant criteria.

Remember, the goal is to improve code organization and maintainability. Choose a structure that makes sense for your project and team, and be consistent in its application.

The code demonstrates how to split a single Terraform file into multiple files for better organization. It starts with a monolithic Terraform configuration defining variables, networking, compute resources, and outputs. It then refactors the code by moving variables to variables.tf, networking resources to networking.tf, compute resources to compute.tf, and outputs to outputs.tf. The main main.tf file is left with only the Terraform block and provider configuration. This results in a cleaner and more maintainable project structure.

Let's imagine you're building a web application infrastructure on AWS. Your original main.tf might look like this:

# main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

variable "region" {
  default = "us-east-1"
}

# Networking
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "public" {
  vpc_id     = aws_vpc.main.id
  cidr_block = "10.0.1.0/24"
}

# EC2 Instance
resource "aws_instance" "web_server" {
  ami           = "ami-0c55b159c9e670230"
  instance_type = "t2.micro"
  subnet_id     = aws_subnet.public.id
}

# Output
output "public_ip" {
  value = aws_instance.web_server.public_ip
}

Here's how you can split this into multiple files:

1. Variables (variables.tf)

# variables.tf
variable "region" {
  default = "us-east-1"
}

2. Networking (networking.tf)

# networking.tf
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "public" {
  vpc_id     = aws_vpc.main.id
  cidr_block = "10.0.1.0/24"
}

3. Compute (compute.tf)

# compute.tf
resource "aws_instance" "web_server" {
  ami           = "ami-0c55b159c9e670230"
  instance_type = "t2.micro"
  subnet_id     = aws_subnet.public.id
}

4. Outputs (outputs.tf)

# outputs.tf
output "public_ip" {
  value = aws_instance.web_server.public_ip
}

5. Main file (main.tf)

# main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

Now you have a cleaner project structure:

├── compute.tf
├── main.tf
├── networking.tf
├── outputs.tf
└── variables.tf

This example demonstrates a simple way to split your code. You can further organize it based on environments (e.g., dev, prod), resource types, or any other logical grouping that suits your project. Remember to maintain consistency and use descriptive names for better readability and maintainability.

General Considerations:

• Consistency is Key: The most important aspect is choosing a logical structure and sticking to it throughout your project. This makes it easier for everyone on the team to understand and maintain the codebase.
• Balance Granularity: While splitting into many files can improve focus, having too many overly small files can hinder navigation. Aim for a balance that makes sense for the complexity of your resources.
• Documentation: Use comments effectively within each file to explain the purpose of resources, variables, and outputs. This is crucial for maintainability, especially as your project grows.

Beyond Basic Splitting:

• Environments: For projects spanning multiple environments (dev, staging, production), consider separate directories for each environment, containing their specific .tf files and potentially using different variable values.
• Shared Resources: If you have resources used across multiple parts of your infrastructure, consider creating a separate directory for them and referencing them using Terraform's data sources.
• Terraform Workspaces: Explore using Terraform workspaces to manage different environments or configurations within the same codebase. This can be a powerful way to avoid code duplication.

Tooling and Best Practices:

• Linters: Utilize Terraform linters like tflint to enforce code style consistency, identify potential errors, and ensure best practices are followed.
• Code Formatting: Use a code formatter like terraform fmt to automatically format your code and maintain a consistent style across all files.
• Version Control: Always use a version control system (like Git) to track changes, collaborate effectively, and revert to previous versions if needed.

Remember: The way you organize your Terraform code can significantly impact the maintainability and scalability of your infrastructure. Invest the time to find a structure that works well for your project and team, and don't be afraid to adapt it as your needs evolve.

This article summarizes how Terraform manages code organization within a directory:

Key Points:

• Flexibility: Terraform treats all .tf files in a directory as a single unit of code, allowing flexible organization.
• Splitting Code: Encourage breaking down large files into smaller, manageable ones based on function (e.g., variables.tf, networking.tf).
• Modules for Reusability: Subdirectories don't automatically become modules. Use the module block to define and call reusable code blocks.
• Benefits of Splitting:
  • Improved readability and comprehension.
  • Easier collaboration among team members.
• Considerations:
  • Terraform processes all .tf files in a directory as one, so manage dependencies between files carefully.
  • Adopt descriptive file names and logical organization (by resource type, environment, etc.) for clarity.

Overall:

While Terraform offers flexibility, prioritize a well-structured and consistent approach to file and directory organization for better code maintainability and collaboration.

In conclusion, organizing your Terraform code effectively is crucial for managing infrastructure-as-code, especially in large projects. Splitting your code into multiple .tf files based on functionality, resource types, or environments enhances readability, collaboration, and maintainability. While Terraform treats all .tf files in a directory as a single unit, employing a consistent and descriptive naming convention, along with logical grouping, significantly improves code clarity. Remember to consider dependencies between resources defined in different files. For reusable code blocks, leverage Terraform modules. Ultimately, the goal is to create a well-structured and maintainable codebase that aligns with your project's needs and team's preferences.

• Best practice for splitting a large main.tf without modules : r/Terraform | Posted by u/Thelmaden - 5 votes and 18 comments
• Splitting out main.tf into separate folders - Terraform - HashiCorp ... | Our team generates a lot of resources. So many that our main.tf has just exceeded 10,000 lines long. This isn’t great for our developer experience and we were intending to try to split out the files to fix this. Our current structure is similar to the suggested Terraform of: modules/ modulea/ main.tf ... moduleb/ ... main.tf // import modulea, moduleb in here ... Our intention is to split up ‘main.tf’ by product, so as to get something like: modules/ modulea/ main.tf ...
• How to Split and Organize Terraform Code Into Modules · Coderbook | Terraform is one of my favorite tools that I picked up last year and part of why I like it is the ability to organize your infrastructure as code into readable, logical chunks of digestible code that any developer can lookup and easily understand within a quick glance.
• How to split Your Terraform main.tf File into Multiple Files | Jhooq | May 4, 2023 ... There are several good reasons to split a Terraform main.tf file into different files. Some of the main reasons why you might want to break ...
• allow terraform to choose which .tf files to include / ignore · Issue ... | By default, TF includes all *.tf files in the current directory. This is a bit rigid and makes it harder to add / remove parts of an environment dynamically. Please allow the user to more easily se...
• Running CI/CD Pipeline Across Multiple Directories in one project ... | Hello, So I’ve recently created a little test environment, just to mess around with creating a pipeline. I have my .gitlab-ci.yml file in the root of my directory, and if I add a new file, s3.tf for example, with the necessary code to create an S3 Bucket in AWS, I can run my pipeline and it will deploy the infrastructure with no problem at all. However, I’d like to expand this project at some point, so I would prefer to put all of my ‘infrastructure’ files into a directory called ‘infra’. The...
• Terraform — how I split my monolithic state | by Adrian Arba | Medium | into smaller, more granular state files that cater to smaller infrastructure components and help with providing reusability and sharing…
• How I Organize Terraform Modules: OFF the Beaten Path | A few times now in my "devops" career, I have, upon arrival to a new codebase, run into somewhat complex terraform modules that an author (or a small team) didn't have too much trouble getting initially written and stood up (once), but about which there is now some
• Terraform Files - How to Structure Terraform Project | Learn how to structure your Terraform files in an efficient and standardized way that makes it easier to scale your projects in the future.