Following our previous tutorial about using AWS KMS for secrets management, this second part explores how to leverage AWS Secrets Manager with Terraform/OpenTofu for more advanced secrets management capabilities. AWS Secrets Manager provides additional features like automatic rotation, fine-grained access control, and centralized secrets management.

Prerequisites

1 — Setting Up AWS Secrets Manager

First, let’s create the necessary resources to store and manage our secrets:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

    # Create a Secrets Manager secret
    resource "aws_secretsmanager_secret" "application_secret" {
      name                    = "awsmorocco/application/secrets"
      description            = "Secrets for the AWS Morocco application"
      recovery_window_in_days = 7

      # Optional: Use our KMS key from Part 1
      # ARN or Id of the AWS KMS key to be used to encrypt the secret
      # values in the versions stored in this secret.
      kms_key_id = aws_kms_key.secrets_key.arn
    }

    resource "random_password" "password" {
      length           = 24
      special          = true
      override_special = "!#$%&*()-_=+[]{}<>:?"
    }

    # Create a secret version with initial secret values
    resource "aws_secretsmanager_secret_version" "application_secret_version" {
      secret_id = aws_secretsmanager_secret.application_secret.id

      secret_string = jsonencode({
        db_username = "awsmorocco-user1"
        db_password = random_password.password.result
        api_key     = data.aws_kms_secrets.application_secrets.plaintext["api_key"]
      })
    }

    # Create an IAM policy for accessing the secret
    resource "aws_iam_policy" "secret_access_policy" {
      name        = "awsmorocco-secret-access-policy"
      description = "Policy for accessing application secrets"

      policy = jsonencode({
        Version = "2012-10-17"
        Statement = [
          {
            Effect = "Allow"
            Action = [
              "secretsmanager:GetSecretValue",
              "secretsmanager:DescribeSecret"
            ]
            Resource = aws_secretsmanager_secret.application_secret.arn
          }
        ]
      })
    }

As AWS Secrets Manager is fully managed by AWS, it integrates seamlessly with IAM to support granular access rules. Here’s how to implement fine-grained access control:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

    # Example of a more detailed IAM policy with granular permissions
    resource "aws_iam_policy" "detailed_secret_access_policy" {
      name        = "awsmorocco-granular-secret-access"
      description = "Granular access policy for application secrets"

      policy = jsonencode({
        Version = "2012-10-17"
        Statement = [
          {
            Sid    = "AllowSecretsRead"
            Effect = "Allow"
            Action = [
              "secretsmanager:GetSecretValue",
              "secretsmanager:DescribeSecret"
            ]
            Resource = [
              aws_secretsmanager_secret.application_secret.arn
            ]
            Condition = {
              StringEquals = {
                "aws:PrincipalTag/Environment": ["development", "production"]
              }
            }
          },
          {
            Sid    = "AllowSecretsWrite"
            Effect = "Allow"
            Action = [
              "secretsmanager:PutSecretValue",
              "secretsmanager:UpdateSecret"
            ]
            Resource = [
              aws_secretsmanager_secret.application_secret.arn
            ]
            Condition = {
              StringEquals = {
                "aws:PrincipalTag/Role": "administrator"
              }
            }
          }
        ]
      })
    }

    # Resource-based policy for cross-account access
    resource "aws_secretsmanager_secret_policy" "secret_resource_policy" {
      secret_arn = aws_secretsmanager_secret.application_secret.arn

      policy = jsonencode({
        Version = "2012-10-17"
        Statement = [
          {
            Sid    = "EnableCrossAccountAccess"
            Effect = "Allow"
            Principal = {
              AWS = "arn:aws:iam::<aws-account-id>:root"  # Replace with actual account ID
            }
            Action = [
              "secretsmanager:GetSecretValue"
            ]
            Resource = "*"
          }
        ]
      })
    }

2 — Secrets Rotation

AWS Secrets Manager supports automatic secret rotation. Here’s how to set it up:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

    # Lambda execution role for the rotation function
    resource "aws_iam_role" "rotation_lambda_role" {
      name = "awsmorocco-secret-rotation-role"

      assume_role_policy = jsonencode({
        Version = "2012-10-17"
        Statement = [
          {
            Action = "sts:AssumeRole"
            Effect = "Allow"
            Principal = {
              Service = "lambda.amazonaws.com"
            }
          }
        ]
      })
    }

    # Enable rotation for the secret
    resource "aws_secretsmanager_secret_rotation" "secret_rotation" {
      secret_id           = aws_secretsmanager_secret.application_secret.id
      rotation_lambda_arn = "arn:aws:lambda:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:function:awsmorocco-rotation-function"

      rotation_rules {
        automatically_after_days = 30
      }
    }

3 — Accessing Secrets in Terraform

You can access secrets from AWS Secrets Manager in your Terraform using a Data Source:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

    # Fetch the entire secret
    data "aws_secretsmanager_secret_version" "application_secrets" {
      secret_id = aws_secretsmanager_secret.application_secret.id
      version_stage = "AWSCURRENT"
    }

    locals {
      secrets = jsondecode(data.aws_secretsmanager_secret_version.application_secrets.secret_string)
    }

    # Use in resources
    resource "aws_db_instance" "application_db" {
      identifier          = "awsmorocco-db-2"
      engine              = "mysql"
      instance_class      = "db.t3.micro"
      username            = local.secrets.db_username
      password            = local.secrets.db_password
      skip_final_snapshot = true
    }

4 — Examples — Integration with EC2 and ECS

  • EC2 Integration
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

    # IAM role for EC2 instances
    resource "aws_iam_role" "ec2_role" {
      name = "awsmorocco-ec2-secrets-role"

      assume_role_policy = jsonencode({
        Version = "2012-10-17"
        Statement = [
          {
            Action = "sts:AssumeRole"
            Effect = "Allow"
            Principal = {
              Service = "ec2.amazonaws.com"
            }
          }
        ]
      })
    }

    # Attach secrets access policy to EC2 role
    resource "aws_iam_role_policy_attachment" "ec2_secret_policy" {
      policy_arn = aws_iam_policy.secret_access_policy.arn
      role       = aws_iam_role.ec2_role.name
    }
  • ECS Integration
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

    # Task definition with secrets
    resource "aws_ecs_task_definition" "application" {
      family                   = "awsmorocco-application"
      requires_compatibilities = ["FARGATE"]
      network_mode            = "awsvpc"
      cpu                     = 256
      memory                  = 512

      container_definitions = jsonencode([
        {
          name  = "application"
          image = "application:latest"
          secrets = [
            {
              name      = "DB_PASSWORD"
              valueFrom = "${aws_secretsmanager_secret.application_secret.arn}:db_password::"
            },
            {
              name      = "API_KEY"
              valueFrom = "${aws_secretsmanager_secret.application_secret.arn}:api_key::"
            }
          ]
        }
      ])
    }

5 — AWS Services Integration with Secrets Manager and KMS

AWS Secrets Manager and KMS integrate natively with many AWS services to provide automated password management capabilities. Let’s explore some of these integrations:

5.1 — RDS Password Management Integration

Amazon RDS provides built-in integration with Secrets Manager for managing master user passwords. This integration simplifies password management and enhances security:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

    resource "aws_db_instance" "default" {
      allocated_storage           = 10
      db_name                    = "awsmorocco-db"
      engine                     = "mysql"
      engine_version             = "8.0"
      instance_class             = "db.t3.micro"
      manage_master_user_password = true  # Enable Secrets Manager integration
      username                   = "admin"
      parameter_group_name       = "default.mysql8.0"
    }

5.2 — RDS with Custom KMS Key

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

    resource "aws_kms_key" "database_key" {
      description = "KMS key for RDS master password encryption"
      enable_key_rotation = true
    }

    resource "aws_db_instance" "default" {
      allocated_storage             = 10
      db_name                       = "mydb"
      engine                        = "mysql"
      engine_version                = "8.0"
      instance_class                = "db.t3.micro"
      manage_master_user_password   = true
      master_user_secret_kms_key_id = aws_kms_key.database_key.key_id
      username                      = "admin"
      parameter_group_name          = "default.mysql8.0"
    }

5.3 — AWS Managed Services Integration

Many AWS services automatically create and manage secrets in Secrets Manager (see more here) :

  • Amazon RDS (rds)
  • Amazon Redshift (redshift)
  • Amazon ECS (ecs-sc)
  • AWS DataSync (datasync)
  • Amazon AppFlow (appflow)
  • AWS Glue DataBrew (databrew)
  • AWS Direct Connect (directconnect)
  • Amazon EventBridge (events)
  • AWS OpsWorks (opsworks-cm)

5.4 — Managed Secrets Characteristics

Managed secrets provided by AWS services include:

  • Automated creation and lifecycle management
  • Built-in rotation capabilities
  • Protection against accidental modifications
  • Required recovery period for deletions
  • Service-specific access controls

6 — Best Practices

1. Secret Organization — Use hierarchical naming for secrets (e.g., /env/service/secret) — Tag secrets for better organization and cost allocation — Use separate secrets for different environments

2. Access Control — Implement least-privilege access using IAM policies — Use resource-based policies for cross-account access — Regularly audit secret access using CloudTrail

3. Rotation — Enable automatic rotation for supported secret types — Implement custom rotation functions for application-specific secrets — Test rotation procedures in non-production environments

4. Security — Enable encryption at rest using KMS keys — Use VPC endpoints for secure access — Implement proper backup and recovery procedures

6 — Monitoring and Logging

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

    # CloudWatch Log Group for Secrets Manager
    resource "aws_cloudwatch_log_group" "secrets_logs" {
      name              = "/aws/secretsmanager/awsmorocco"
      retention_in_days = 14
    }

    # CloudWatch Metric Alarm for failed secret rotations
    resource "aws_cloudwatch_metric_alarm" "rotation_failures" {
      alarm_name          = "awsmorocco-secret-rotation-failures"
      comparison_operator = "GreaterThanThreshold"
      evaluation_periods  = "1"
      metric_name         = "RotationFailed"
      namespace           = "AWS/SecretsManager"
      period              = "300"
      statistic           = "Sum"
      threshold           = "0"
      alarm_description   = "This metric monitors failed secret rotations"

      dimensions = {
        SecretId = aws_secretsmanager_secret.application_secret.id
      }
    }

Conclusion

AWS Secrets Manager, when properly integrated with Terraform, provides a robust solution for managing application secrets. By following these practices and implementing proper security controls, you can ensure your sensitive information remains secure while being easily accessible to your applications.

The combination of AWS KMS (covered in Part 1) and AWS Secrets Manager gives you a comprehensive secrets management solution that can scale with your infrastructure needs.

ℹ️ Complete Terraform code is available in our GitHub repository :

aws-morocco-samples/secrets-management-in-terraform/terraform at main ·Z4ck404/aws-morocco-samples

Secure Secrets Management in Terraform — Part2: AWS Secret Manager was originally published in AWS Morocco on Medium, where people are continuing the conversation by highlighting and responding to this story.