Skip to content

Latest commit

 

History

History
144 lines (117 loc) · 11.8 KB

README.md

File metadata and controls

144 lines (117 loc) · 11.8 KB
Raw

Prerequisites

Build AMI

Set necessary variables by creating a file ds-ami.pkrvars.hcl and adding the following variables according to your own usage.

cat <<EOF > ds-ami.pkrvars.hcl
aws_access_key = ""
aws_secret_key = ""
aws_region     = "cn-north-1"


ds_ami_name = "my-test-ds-2"

# If you want to use the official distribution tar, just set the `ds_version` to the one you want.
ds_version  = "3.1.1"

# If you want to use a locally built distribution tar, set the `ds_tar` to the tar file location.
ds_tar      = "~/workspace/dolphinscheduler/dolphinscheduler-dist/target/apache-dolphinscheduler-3.1.3-SNAPSHOT-bin.tar.gz"
EOF

Then run the following command to initialize and build a custom AMI.

  • If you want to use the official distribution tar.
packer init --var-file=ds-ami.pkrvars.hcl packer/ds-ami-official.pkr.hcl
packer build --var-file=ds-ami.pkrvars.hcl packer/ds-ami-official.pkr.hcl
  • If you want to use the locally built distribution tar.
packer init --var-file=ds-ami.pkrvars.hcl packer/ds-ami-local.pkr.hcl
packer build --var-file=ds-ami.pkrvars.hcl packer/ds-ami-local.pkr.hcl

Create resources

Set necessary variables by creating a file terraform.tfvars and adding the following variables according to your own usage.

Make sure ds_ami_name is the same as the one in ds-ami.pkrvars.hcl above.

cat <<EOF > terraform.tfvars
aws_access_key = ""
aws_secret_key = ""
aws_region     = ""

name_prefix = "test-ds-terraform"
ds_ami_name = "my-test-ds"

ds_component_replicas = {
  master            = 1
  worker            = 1
  alert             = 1
  api               = 1
  standalone_server = 0
}
EOF

Then run the following commands to apply necessary resources.

terraform init -var-file=terraform.tfvars
terraform apply -var-file=terraform.tfvars -auto-approve

Open DolphinScheduler UI

open http://$(terraform output -json api_server_instance_public_dns | jq -r '.[0]'):12345/dolphinscheduler/ui

Inputs

Name Description Type Default Required
aws_access_key AWS access key string n/a yes
aws_region AWS region string "cn-north-1" no
aws_secret_key AWS secret key string n/a yes
db_instance_class Database instance class string "db.t3.micro" no
db_password Database password string n/a yes
db_username Database username string "dolphinscheduler" no
ds_ami_name Name of DolphinScheduler AMI string "dolphinscheduler-ami" no
ds_component_replicas Replicas of the DolphinScheduler Components map(number) 
{
  "alert": 1,
  "api": 1,
  "master": 1,
  "standalone_server": 0,
  "worker": 1
}
 no
ds_version DolphinScheduler Version string "3.1.1" no
name_prefix Name prefix for all resources string "dolphinscheduler" no
private_subnet_cidr_blocks Available CIDR blocks for private subnets list(string) 
[
  "10.0.101.0/24",
  "10.0.102.0/24",
  "10.0.103.0/24",
  "10.0.104.0/24"
]
 no
public_subnet_cidr_blocks CIDR blocks for the public subnets list(string) 
[
  "10.0.1.0/24",
  "10.0.2.0/24",
  "10.0.3.0/24",
  "10.0.4.0/24"
]
 no
s3_bucket_prefix n/a string "dolphinscheduler-test-" no
subnet_count Number of subnets map(number) 
{
  "private": 2,
  "public": 1
}
 no
tags Tags to apply to all resources map(string) 
{
  "Deployment": "Test"
}
 no
vm_associate_public_ip_address Associate a public IP address to the EC2 instance map(bool) 
{
  "alert": true,
  "api": true,
  "master": true,
  "standalone_server": true,
  "worker": true
}
 no
vm_data_volume_size Data volume size of the EC2 Instance map(number) 
{
  "alert": 10,
  "api": 10,
  "master": 10,
  "standalone_server": 10,
  "worker": 10
}
 no
vm_data_volume_type Data volume type of the EC2 Instance map(string) 
{
  "alert": "gp2",
  "api": "gp2",
  "master": "gp2",
  "standalone_server": "gp2",
  "worker": "gp2"
}
 no
vm_instance_type EC2 instance type map(string) 
{
  "alert": "t2.micro",
  "api": "t2.small",
  "master": "t2.medium",
  "standalone_server": "t2.small",
  "worker": "t2.medium"
}
 no
vm_root_volume_size Root Volume size of the EC2 Instance map(number) 
{
  "alert": 30,
  "api": 30,
  "master": 30,
  "standalone_server": 30,
  "worker": 30
}
 no
vm_root_volume_type Root volume type of the EC2 Instance map(string) 
{
  "alert": "gp2",
  "api": "gp2",
  "master": "gp2",
  "standalone_server": "gp2",
  "worker": "gp2"
}
 no
vpc_cidr CIDR for the VPC string "10.0.0.0/16" no
zookeeper_connect_string Zookeeper connect string, if empty, will create a single-node zookeeper for demonstration, don't use this in production string "" no

Outputs

Name Description
alert_server_instance_id Instance IDs of alert instances
alert_server_instance_private_ip Private IPs of alert instances
alert_server_instance_public_dns Public domain names of alert instances
alert_server_instance_public_ip Public IPs of alert instances
api_server_instance_id Instance IDs of api instances
api_server_instance_private_ip Private IPs of api instances
api_server_instance_public_dns Public domain names of api instances
api_server_instance_public_ip Public IPs of api instances
db_address Database address
db_name Database name
db_port Database port
master_server_instance_id Instance IDs of master instances
master_server_instance_private_ip Private IPs of master instances
master_server_instance_public_dns Public domain names of master instances
master_server_instance_public_ip Public IPs of master instances
s3_access_key S3 access key
s3_address S3 address
s3_bucket S3 bucket name
s3_regional_domain_name S3 regional domain name
s3_secret S3 access secret
vm_server_instance_id Instance IDs of standalone instances
vm_server_instance_private_ip Private IPs of standalone instances
vm_server_instance_public_dns Public domain names of standalone instances
vm_server_instance_public_ip Public IPs of standalone instances
worker_server_instance_id Instance IDs of worker instances
worker_server_instance_private_ip Private IPs of worker instances
worker_server_instance_public_dns Public domain names of worker instances
worker_server_instance_public_ip Public IPs of worker instances
zookeeper_server_instance_id Instance IDs of zookeeper instances
zookeeper_server_instance_private_ip Private IPs of zookeeper instances
zookeeper_server_instance_public_dns Public domain names of zookeeper instances
zookeeper_server_instance_public_ip Public IPs of zookeeper instances