Terraform Secrets Manager: Reuse Existing Secrets Safely

• Introduction
• Step-by-Step Guide
• Code Example
• Additional Notes
• Summary
• Conclusion
• References

When managing sensitive information in your AWS infrastructure with Terraform, understanding how to safely handle the deletion and recreation of secrets stored in AWS Secrets Manager is crucial. This guide provides a comprehensive walkthrough of the process, potential challenges, and best practices to ensure smooth operations and prevent unexpected issues. We'll explore the intricacies of secret deletion in AWS Secrets Manager, including the soft delete mechanism and its implications for Terraform-managed resources. You'll learn how to address conflicts that may arise during recreation and gain insights into managing Terraform's state effectively when dealing with deleted secrets. We'll cover recommended practices for both Terraform-managed and manual deletions, along with strategies for handling name conflicts during recreation. Additionally, we'll touch upon important considerations such as secret versions, dependencies, and automation possibilities. Practical examples using Terraform code snippets will illustrate the concepts, empowering you to confidently manage your AWS secrets. Remember to prioritize security by backing up your secrets and thoroughly testing your changes in a controlled environment before deploying to production.

When working with AWS Secrets Manager and Terraform, you might encounter situations where you need to delete and recreate secrets. Here's a breakdown of the process and considerations:

Understanding Secret Deletion in AWS Secrets Manager

• Soft Delete with Recovery Period: When you delete a secret in Secrets Manager, it's initially marked for deletion but remains recoverable for a defined period (recovery window). This protects against accidental deletions.
• Permanent Deletion: After the recovery window, the secret is permanently deleted.
• Cost: You are not charged for secrets marked for deletion.

Challenges with Terraform and Secret Deletion

• Recreation Conflicts: If you delete a secret and immediately try to create a new one with the same name within the recovery window, you'll encounter an error. Secrets Manager enforces unique names even during the recovery period.
• Terraform State: Terraform might encounter issues if it tries to manage a secret that's been manually deleted outside of its control.

Best Practices for Deleting and Recreating Secrets

1. Terraform Managed Deletion (Recommended):

   • Use Terraform to delete the secret using the aws_secretsmanager_secret resource with the force_delete argument set to true. This ensures Terraform is aware of the deletion.

2. Manual Deletion and Terraform Import:

   • If you manually delete a secret, you need to update Terraform's state to reflect this change.
   • Use the terraform import command to import the deleted secret into your Terraform state. This will mark the secret as "tainted" in Terraform.
   • Run terraform apply to allow Terraform to recreate the secret.

3. Handling Conflicts During Recreation:

   • If you need to recreate a secret with the same name immediately, you have two options:
     • Wait for Recovery Window: Allow the recovery window to expire before creating the new secret.
     • Use a Different Name: Temporarily create the new secret with a different name and update any resources referencing it. Once the original secret is permanently deleted, you can rename the new secret.

Additional Considerations

• Secret Versions: Be mindful of secret versions. Deleting a secret deletes all its versions.
• Dependencies: Ensure no other resources are actively using the secret before deleting it.
• Automation: If you frequently need to delete and recreate secrets, consider automating the process using scripts or CI/CD pipelines.

Example: Deleting a Secret with Terraform

resource "aws_secretsmanager_secret" "example" {
  name = "my-secret"
  # ... other configurations ...
}

# To delete the secret:
resource "aws_secretsmanager_secret" "example" {
  name       = "my-secret"
  force_delete = true
}

Remember to adapt these steps and examples to your specific use case and infrastructure setup. Always back up your secrets and test your changes in a non-production environment before deploying to production.

This code provides three methods for deleting and recreating AWS Secrets Manager secrets using Terraform. The first method uses Terraform's force_delete option to delete and recreate the secret with the same name. The second method involves manually deleting the secret and then importing it into Terraform state before recreating it. The third method demonstrates how to handle potential conflicts by using a temporary secret name during the recreation process. Remember to replace placeholders with your actual secret names and configurations, back up your secrets, and test changes in a non-production environment.

1. Terraform Managed Deletion (Recommended):

# Create a secret
resource "aws_secretsmanager_secret" "example" {
  name = "my-secret"
}

# Delete the secret
resource "aws_secretsmanager_secret" "example" {
  name        = "my-secret"
  force_delete = true
}

# Recreate the secret with the same name
resource "aws_secretsmanager_secret" "example" {
  name = "my-secret"
  # ... new secret configurations ...
}

2. Manual Deletion and Terraform Import:

# Manually delete the secret using the AWS Console or CLI

# Import the deleted secret into Terraform state
terraform import aws_secretsmanager_secret.example my-secret

# The imported resource will be marked as "tainted"

# Apply changes to recreate the secret
terraform apply

# Configure the recreated secret
resource "aws_secretsmanager_secret" "example" {
  name = "my-secret"
  # ... new secret configurations ...
}

3. Handling Conflicts During Recreation (Using a Different Name):

# Delete the secret using any method

# Create a new secret with a temporary name
resource "aws_secretsmanager_secret" "temp_secret" {
  name = "my-temp-secret"
  # ... new secret configurations ...
}

# Update resources to reference the temporary secret

# Wait for the original secret's recovery window to expire

# Delete the temporary secret
resource "aws_secretsmanager_secret" "temp_secret" {
  name        = "my-temp-secret"
  force_delete = true
}

# Recreate the secret with the original name
resource "aws_secretsmanager_secret" "example" {
  name = "my-secret"
  # ... same configurations as the temporary secret ...
}

# Update resources to reference the recreated secret

Important Notes:

• Replace "my-secret" with your actual secret name.
• Adapt the code snippets to your specific needs and configurations.
• Always back up your secrets before making any changes.
• Test your changes in a non-production environment before deploying to production.

• force_delete Caveat: While force_delete streamlines deletion in most cases, be aware of potential issues with its implementation in the aws_secretsmanager_secret resource. Refer to the linked GitHub issue for details and workarounds if you encounter problems.
• Secret Reuse: If your goal is to reuse an existing secret's name after modification, directly updating the secret might be preferable to deleting and recreating. Consider whether deletion is truly necessary for your use case.
• Manual Deletion Impacts: Manually deleting a secret outside of Terraform necessitates importing the deleted resource into your state. This ensures Terraform recognizes the change and avoids inconsistencies.
• Versioning Implications: Deleting a secret removes all its versions. If you need to retain previous versions, explore rotating secrets instead of outright deletion.
• Dependency Management: Before deleting a secret, thoroughly check for any dependencies. Ensure no other resources, applications, or services rely on the secret to prevent disruptions.
• State Management: Terraform's state file plays a crucial role in managing your infrastructure. Keep your state file consistent with your actual AWS environment to avoid unexpected behavior.
• Error "You can't create this secret because a secret with this name is already scheduled for deletion": This error indicates that you are trying to create a secret with the same name as one that is currently in the deletion recovery window. You can either wait for the recovery window to expire or choose a different name for the new secret.
• Terraform Best Practices: Adhering to Terraform best practices, such as using modules and organizing your code effectively, can simplify secret management and improve your infrastructure's maintainability.

Remember that managing secrets is a critical aspect of security. Always prioritize security best practices, such as following the principle of least privilege and implementing robust access control mechanisms.

Key Takeaway: Carefully manage secret deletion and recreation to avoid conflicts and maintain Terraform state consistency. Prioritize Terraform-managed deletion for smoother operations.

By understanding the nuances of AWS Secrets Manager's deletion behavior and its interaction with Terraform, you can establish a robust and secure workflow for managing your sensitive information. Implementing the best practices outlined in this guide, such as leveraging Terraform-managed deletion and understanding the implications of manual interventions, will help you avoid common pitfalls and ensure the integrity of your infrastructure as code. Remember to prioritize security by incorporating backups, access control mechanisms, and thorough testing in non-production environments. As you continue to work with AWS Secrets Manager and Terraform, consider exploring advanced topics like secret rotation and automation to further enhance your secret management practices.

• amazon web services | Reuse of existing secrets without deleting | Aug 9, 2019 ... I am creating Secrets in AWS using Terraform code. My Jenkins pipeline will create the infrastructure every 2 hours and destroys it.
• Delete an AWS Secrets Manager secret | Until the recovery window ends, you can recover a secret you previously deleted. There is no charge for secrets that you have marked for deletion. You can't ...
• amazon web services - Terraform AWS Provider: SecretsManager ... | Feb 13, 2020 ... Terraform | Secrets Manager | Reuse of existing secrets without deleting · 44 · How to retrieve a secret in terraform from aws secret manager.
• Delete a Secrets Manager secret and create a new secret with the ... | I deleted an AWS Secrets Manager secret. When I try to create a new secret with the same name, I get the following error: "You can't create this secret because a secret with this name is already sc...
• Updated secrets_manager secrets values manually , now terraform ... | Hey Team , I have updated the secret values manually in console , later when I run apply in terraform. Getting error while applying. Not able to delete the current version ID also which is conflicting the terraform new ID IN PLAN : #module.opensearch.aws_secretsmanager_secret_version.users[“readonly”] will be updated in-place ~ resource “aws_secretsmanager_secret_version” “users” { id = “arn:aws:secretsmanager:ap-southeast-2:123456789㊙abc|7873FBF4-BB8B-42CC-8E89-7FFDAA8CD...
• AWS Secrets Manager and Terraform State Delete Issue | by Kevin ... | I recently ran into an issue with pushing up some Terraform changes and seeing the following error:
•  Locking secrets | To lock a secret, you attach one or more locks to its current or previous version. When you try to modify or delete a secret while it is locked, Secrets Manager ...
• Lifecycle: ownership & deletion - External Secrets Operator | ... not allowed as they would delete existing secrets: - deletionPolicy=Delete & creationPolicy=Merge - deletionPolicy=Delete & creationPolicy=None - ...
• AWS Secrets Manager - Delete and Recreate Secrets | Aug 9, 2018 ... Easily automate secrets management with AWS Secrets Manager by deleting secrets without a recovery window ... AWS Secrets Manager is available ...