feat: restructure and add new datadog integration policies

[RoseSecurity]

Why

• Expanding permission sets as Datadog and AWS evolve

What

• Style update: change all join("", resource.kind.*.name) and resource.kind[0].name to one(resource.kind[*].name)
• Deprecate var.integrations and add var.policies in its place
  1. Update the description to indicate it is deprecated
  2. Make its default value null
• Add var.policies
• Create a local.policies which is the list of policies specified via var.integrations and var.policies combined with mappings and then de-duplicated

Note

For compatibility, map var.integrations "core" -> "core_integration" and "all" -> "full_integration" when adding to local.policies.

• Rename the "all" policy "full-integration" and update it
• Rename iam_policy_all.tf -> iam-policy-full-integration.tf and rename all the resources etc. named "all" to "full_integration", and trigger it with policy name "full-integration"
• Update the policy reference
• Update the permissions (statement.actions) from those sources:

full-integration permissions

  actions = [
    "apigateway:GET",
    "autoscaling:Describe*",
    "backup:List*",
    "budgets:ViewBudget",
    "cloudfront:GetDistributionConfig",
    "cloudfront:ListDistributions",
    "cloudtrail:DescribeTrails",
    "cloudtrail:GetTrailStatus",
    "cloudtrail:LookupEvents",
    "cloudwatch:Describe*",
    "cloudwatch:Get*",
    "cloudwatch:List*",
    "codedeploy:List*",
    "codedeploy:BatchGet*",
    "directconnect:Describe*",
    "dynamodb:List*",
    "dynamodb:Describe*",
    "ec2:Describe*",
    "ec2:GetTransitGatewayPrefixListReferences",
    "ec2:SearchTransitGatewayRoutes",
    "ecs:Describe*",
    "ecs:List*",
    "elasticache:Describe*",
    "elasticache:List*",
    "elasticfilesystem:DescribeFileSystems",
    "elasticfilesystem:DescribeTags",
    "elasticfilesystem:DescribeAccessPoints",
    "elasticloadbalancing:Describe*",
    "elasticmapreduce:List*",
    "elasticmapreduce:Describe*",
    "es:ListTags",
    "es:ListDomainNames",
    "es:DescribeElasticsearchDomains",
    "events:CreateEventBus",
    "fsx:DescribeFileSystems",
    "fsx:ListTagsForResource",
    "health:DescribeEvents",
    "health:DescribeEventDetails",
    "health:DescribeAffectedEntities",
    "kinesis:List*",
    "kinesis:Describe*",
    "lambda:GetPolicy",
    "lambda:List*",
    "logs:DeleteSubscriptionFilter",
    "logs:DescribeLogGroups",
    "logs:DescribeLogStreams",
    "logs:DescribeSubscriptionFilters",
    "logs:FilterLogEvents",
    "logs:PutSubscriptionFilter",
    "logs:TestMetricFilter",
    "oam:ListSinks",
    "oam:ListAttachedLinks",
    "organizations:Describe*",
    "organizations:List*",
    "rds:Describe*",
    "rds:List*",
    "redshift:DescribeClusters",
    "redshift:DescribeLoggingStatus",
    "route53:List*",
    "s3:GetBucketLogging",
    "s3:GetBucketLocation",
    "s3:GetBucketNotification",
    "s3:GetBucketTagging",
    "s3:ListAllMyBuckets",
    "s3:PutBucketNotification",
    "ses:Get*",
    "sns:List*",
    "sns:Publish",
    "sns:GetSubscriptionAttributes",
    "sqs:ListQueues",
    "states:ListStateMachines",
    "states:DescribeStateMachine",
    "support:DescribeTrustedAdvisor*",
    "support:RefreshTrustedAdvisorCheck",
    "tag:GetResources",
    "tag:GetTagKeys",
    "tag:GetTagValues",
    "wafv2:ListLoggingConfigurations",
    "wafv2:GetLoggingConfiguration",
    "xray:BatchGetTraces",
    "xray:GetTraceSummaries"
  ],

• Rename iam_policy_core.tf -> iam-policy-core-integration.tf and rename all the resources etc. named "core" to "core_integration", and trigger it with policy name "core-integration"
• Update the policy reference
• Update the permissions (statement.actions) by removing 'support:*'
• Create iam-policy-resource-collection.tf
• Follow the pattern of iam-policy-full-integration.tf and create iam-policy-resource-collection.tf to implement the resource-collection option, referencing and using the policy from https://docs.datadoghq.com/integrations/amazon_web_services/?tab=roledelegation#aws-resource-collection-iam-policy-1

resource-collection permissions

  actions = [
    "backup:ListRecoveryPointsByBackupVault",
    "bcm-data-exports:GetExport",
    "bcm-data-exports:ListExports",
    "cassandra:Select",
    "cur:DescribeReportDefinitions",
    "ec2:GetSnapshotBlockPublicAccessState",
    "glacier:GetVaultNotifications",
    "glue:ListRegistries",
    "lightsail:GetInstancePortStates",
    "savingsplans:DescribeSavingsPlanRates",
    "savingsplans:DescribeSavingsPlans",
    "timestream:DescribeEndpoints",
    "waf-regional:ListRuleGroups",
    "waf-regional:ListRules",
    "waf:ListRuleGroups",
    "waf:ListRules",
    "wafv2:GetIPSet",
    "wafv2:GetRegexPatternSet",
    "wafv2:GetRuleGroup"
  ],

• Create iam-policy-security-audit.tf
• Updated documentation and examples

[RoseSecurity]

/terratest