Terraform Module for Amazon Redshift Copy UDF

This terraform module provides complimentary capabilities to COPY command by enabling data copy from S3 API compliant storage solutions such as Cloudian, MinIO, and Weka into Amazon Redshift with AWS Lambda UDF (User Defined Function).

Architecture Diagram

Usage

module "udf" {
  source  = "aws-ia/redshift-copy-udf/aws"
  version = "~> 1.0"

  name        = "redshift-copy-udf"
  memory_size = 128
  timeout     = 5

  vpc_subnet_ids     = null # replace with comma separated values
  security_group_ids = null # replace with comma separated values

  storage_url        = null # replace with storage url
  storage_user       = null # replace with storage user
  storage_pass       = null # replace with storage pass
  storage_token      = null # replace with storage token (optional)
  storage_secret_arn = null # replace with secrets manager arn (optional)
}

Requirements

Name	Version
terraform	>= 1.0.0
aws	>= 4.0.0
random	>= 3.0.0

Providers

Name	Version
aws	>= 4.0.0
random	>= 3.0.0

Modules

Name	Source	Version
lambda	terraform-aws-modules/lambda/aws	~> 7.0

Resources

Name	Type
aws_iam_role.redshift	resource
aws_iam_role_policy.redshift	resource
random_id.this	resource
aws_partition.this	data source
aws_secretsmanager_secret.this	data source
aws_secretsmanager_secret_version.this	data source

Inputs

Name	Description	Type	Default	Required
memory_size	Lambda UDF memory size	number	128	no
name	Lambda UDF function name	string	"redshift-copy-udf"	no
security_group_ids	Security Group IDs (comma separated values)	string	null	no
storage_pass	Storage Password to Access S3 API Compliant Storage	string	null	no
storage_secret_arn	Secrets Manager ARN Holding Credentials to Access S3 API Compliant Storage (Optional)	string	null	no
storage_token	Storage Token to Access S3 API Compliant Storage (Optional)	string	null	no
storage_url	Storage URL to Access S3 API Compliant Storage	string	null	no
storage_user	Storage Username to Access S3 API Compliant Storage	string	null	no
timeout	Lambda UDF timeout	number	300	no
vpc_subnet_ids	VPC Subnet IDs (comma separated values)	string	null	no

Outputs

Name	Description
iam_role_arn	IAM Role ARN for Redshift Permissions
iam_role_id	IAM Role ID for Redshift Permissions
iam_role_name	IAM Role Name for Redshift Permissions
lambda_function_arn	Lambda Function ARN for Redshift UDF
lambda_function_name	Lambda Function Name for Redshift UDF