🐶
Terraform

Cross-Subscription Terraform State Management in Azure

By Filip on 10/05/2024

Learn how to manage your Terraform Azure infrastructure efficiently by storing your Terraform state file in a separate subscription for improved security, organization, and collaboration.

Cross-Subscription Terraform State Management in Azure

Table of Contents

Introduction

Managing resources across multiple Azure subscriptions with Terraform requires a good understanding of authentication, authorization, and best practices for configuration and security. This article provides a comprehensive guide to effectively manage resources in such scenarios using Terraform.

Step-by-Step Guide

To manage resources in different Azure subscriptions with Terraform, you need to handle authentication and authorization properly. Here's a breakdown:

1. Authentication:

  • Azure CLI: The simplest way is to log in to Azure CLI (az login) with an account having access to all relevant subscriptions. Terraform will inherit these credentials.
  • Service Principal: For automation, create a Service Principal (SP) in Azure Active Directory. Assign it contributor role to the target subscriptions. Store its credentials securely (environment variables, Azure Key Vault).
  • Managed Identities: When running Terraform in Azure (e.g., Azure DevOps), use Managed Identities for automatic authentication without storing secrets.

2. Terraform Configuration:

  • Provider Configuration: In your Terraform provider block (provider "azurerm"), specify the subscription_id for each target subscription. If using a Service Principal, provide its credentials here.
  • Multiple Providers: You can define multiple azurerm providers, each targeting a different subscription. Use aliases to distinguish them (e.g., provider "azurerm" { alias = "dev" ... }).
  • Data Sources: Use data sources like azurerm_subscription to fetch subscription details dynamically.

3. State File Management:

  • Separate State Files: Best practice is to have separate state files for each environment (dev, prod) and subscription. This improves isolation and reduces blast radius.
  • Centralized Storage: Store state files in a central location like Azure Storage Account. Ensure proper access control (RBAC) to restrict modifications.
  • Backend Configuration: Configure the Terraform backend (terraform { backend "azurerm" { ... } }) to point to your chosen storage.

4. Resource Deployment:

  • Explicit Provider: When defining resources, explicitly specify the provider alias if using multiple providers (e.g., resource "azurerm_virtual_network" "example" { provider = azurerm.dev ... }).
  • Cross-Subscription Resources: Some resources (e.g., VNet peering) require careful configuration when spanning subscriptions. Ensure both subscriptions have necessary permissions.

5. Security Considerations:

  • Principle of Least Privilege: Grant only necessary permissions to Service Principals and Managed Identities.
  • Secret Management: Never hardcode secrets in your Terraform code. Use environment variables, Azure Key Vault, or similar mechanisms.
  • State File Encryption: Enable encryption at rest for your state files in Azure Storage Account.

Example (using Service Principal):

# Configure Azure Provider for "dev" subscription
provider "azurerm" {
  alias           = "dev"
  features {}
  subscription_id = "your-dev-subscription-id"
  client_id       = var.client_id
  client_secret   = var.client_secret
  tenant_id       = var.tenant_id
}

# Configure Azure Provider for "prod" subscription
provider "azurerm" {
  alias           = "prod"
  features {}
  subscription_id = "your-prod-subscription-id"
  client_id       = var.client_id
  client_secret   = var.client_secret
  tenant_id       = var.tenant_id
}

# Define a resource in the "dev" subscription
resource "azurerm_resource_group" "dev_rg" {
  provider = azurerm.dev
  name     = "my-dev-rg"
  location = "eastus"
}

Remember to adapt this example to your specific needs and always prioritize security best practices.

Code Example

This Terraform code demonstrates managing resources across two Azure subscriptions, "dev" and "prod", using a single Service Principal for authentication. It outlines creating a Service Principal, assigning it the "Contributor" role in both subscriptions, and securely storing its credentials using environment variables or Azure Key Vault. The code then configures two Azure providers in Terraform, each targeting a different subscription with the shared credentials. It showcases creating a resource group in each subscription, highlighting how to deploy to multiple subscriptions within one Terraform configuration. The example emphasizes security considerations like avoiding hardcoded credentials, adhering to least privilege, encrypting state files, and regularly rotating credentials. It provides a foundation for managing multi-subscription Azure environments with Terraform, adaptable to specific needs and infrastructure.

This example demonstrates how to manage resources in two different Azure subscriptions ("dev" and "prod") using Terraform and a Service Principal for authentication.

1. Create a Service Principal and Assign Roles:

2. Store Service Principal Credentials Securely:

  • Option 1: Environment Variables: 
    export ARM_CLIENT_ID="your-client-id"
export ARM_CLIENT_SECRET="your-client-secret"
export ARM_TENANT_ID="your-tenant-id"
  • Option 2: Azure Key Vault:
    • Store the credentials in Azure Key Vault and configure Terraform to access them. Refer to the Azure Key Vault documentation for details.

3. Terraform Configuration (main.tf):

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0"
    }
  }
}

# Configure Azure Provider for "dev" subscription
provider "azurerm" {
  alias           = "dev"
  features {}
  subscription_id = "your-dev-subscription-id"
  client_id       = var.client_id
  client_secret   = var.client_secret
  tenant_id       = var.tenant_id
}

# Configure Azure Provider for "prod" subscription
provider "azurerm" {
  alias           = "prod"
  features {}
  subscription_id = "your-prod-subscription-id"
  client_id       = var.client_id
  client_secret   = var.client_secret
  tenant_id       = var.tenant_id
}

# Define variables for resource group names
variable "dev_rg_name" {
  default = "my-dev-rg"
}

variable "prod_rg_name" {
  default = "my-prod-rg"
}

# Create a resource group in the "dev" subscription
resource "azurerm_resource_group" "dev_rg" {
  provider = azurerm.dev
  name     = var.dev_rg_name
  location = "eastus"
}

# Create a resource group in the "prod" subscription
resource "azurerm_resource_group" "prod_rg" {
  provider = azurerm.prod
  name     = var.prod_rg_name
  location = "westus2"
}

4. Variables (variables.tf):

variable "client_id" {
  type = string
}

variable "client_secret" {
  type      = string
  sensitive = true
}

variable "tenant_id" {
  type = string
}

5. Configure Terraform Backend (optional):

  • Uncomment and configure the backend block in your main.tf file to store the state file in Azure Storage Account.
# terraform {
#   backend "azurerm" {
#     resource_group_name  = "your-tfstate-rg"
#     storage_account_name = "your-tfstate-storage"
#     container_name       = "tfstate"
#     key                  = "multi-subscription-example.tfstate"
#   }
# }

6. Deploy the Infrastructure:

terraform init
terraform plan
terraform apply

Explanation:

  • This code defines two Azure providers, each targeting a different subscription and using the same Service Principal credentials.
  • It creates a resource group in each subscription, demonstrating how to deploy resources to different subscriptions within the same Terraform configuration.
  • Remember to replace the placeholder values with your actual subscription IDs, Service Principal credentials, and desired resource group names.

Security Considerations:

  • Never hardcode sensitive information like client secrets directly in your Terraform code. Use environment variables, Azure Key Vault, or other secure methods.
  • Follow the principle of least privilege. Grant the Service Principal only the necessary permissions to perform its tasks.
  • Enable encryption at rest for your state files if you are using a remote backend like Azure Storage Account.
  • Regularly review and rotate your Service Principal credentials.

This example provides a starting point for managing resources in multiple Azure subscriptions with Terraform. You can adapt and extend it to fit your specific needs and infrastructure requirements.

Additional Notes

Authentication:

  • Azure CLI: While convenient, relying solely on Azure CLI authentication can be problematic in CI/CD pipelines. It's better to define a dedicated Service Principal for non-interactive environments.
  • Service Principal:
    • Carefully manage the scope of roles assigned to the Service Principal. Overly permissive roles pose security risks.
    • Consider using Azure role-based access control (RBAC) to fine-tune permissions on specific resource groups or resources.
  • Managed Identities:
    • Ensure the Managed Identity assigned to your Azure service (e.g., Azure VM, Azure DevOps pipeline) has the necessary permissions on the target subscriptions.

Terraform Configuration:

  • Provider Versions: Use consistent AzureRM provider versions across your environments to avoid unexpected behavior due to API changes.
  • Terraform Workspaces: Consider using Terraform workspaces to manage different environments (dev, prod) within the same configuration directory. This can simplify state file management but requires careful planning.

State File Management:

  • State Locking: For collaborative environments, enable state locking to prevent concurrent modifications and potential corruption of the state file.
  • Remote State Backends: Explore other remote state backends like HashiCorp Consul, AWS S3, or Google Cloud Storage, depending on your infrastructure and security requirements.

Resource Deployment:

  • Resource Dependencies: Pay close attention to resource dependencies across subscriptions. Terraform's implicit dependency management might not always cover cross-subscription scenarios, requiring explicit depends_on statements.
  • Terraform Modules: Leverage Terraform modules to encapsulate reusable infrastructure components and promote code reusability across subscriptions.

Security Considerations:

  • Regular Audits: Periodically audit your Service Principal permissions and Terraform configurations to ensure they adhere to security best practices and comply with your organization's policies.
  • Network Security: If applicable, configure network security groups (NSGs) and firewalls to restrict access to your Azure resources and state file storage.

Additional Tips:

  • Use Terraform Output Values: Utilize output values to share information (e.g., resource IDs) across subscriptions and modules.
  • Implement Infrastructure as Code (IaC) Best Practices: Follow general IaC best practices like modularity, code reusability, testing, and documentation to ensure maintainability and scalability of your Terraform codebase.

By incorporating these additional notes and best practices, you can effectively manage resources across multiple Azure subscriptions with Terraform while maintaining a secure and well-organized infrastructure.

Summary

This table summarizes key aspects of managing resources across multiple Azure subscriptions using Terraform:

| Aspect | Options | Notes

Conclusion

By following the guidelines and examples presented, you can leverage Terraform's capabilities to manage multi-subscription environments effectively while adhering to security best practices. Remember to adapt the provided code snippets to your specific requirements and always prioritize the security of your infrastructure. As you gain more experience, explore advanced Terraform features and Azure services to optimize your multi-subscription management workflows further.

References

Were You Able to Follow the Instructions?

😍Love it!
😊Yes
😐Meh-gical
😞No
🤮Clickbait

Related posts

Terraform Modules: Passing Variables via CLI & tfvars

Learn how to efficiently pass variables to your Terraform modules using both the command line interface (CLI) and tfvars files for flexible and reusable infrastructure deployments.

Terraform Route 53 Record to ALB Configuration

Learn how to use Terraform to effortlessly create a Route 53 record that directs traffic to your Application Load Balancer (ALB) in AWS.

Terraform Nested Loops: A Comprehensive Guide

Learn how to efficiently iterate and manage complex infrastructure deployments using Terraform's powerful nested for loop functionality.