Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Self hosting! #67

Merged
merged 16 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ Host ipa
User ec2-user
IdentityFile ~/.ssh/<ssh_key.pem>
```
4. Update the `draft/ansible/inventory.ini` file to only include a single host. (Unless you are running all 4 servers.)
5. Provision your machine: `ansible-playbook -i ansible/inventory.ini ansible/provision.yaml`
4. Update the `draft/sidecar/ansible/inventory.ini` file to only include a single host. (Unless you are running all 4 servers.)
5. Provision your machine: `ansible-playbook -i sidecar/ansible/inventory.ini sidecar/ansible/provision.yaml`

To deploy new changes in draft, run: `ansible-playbook -i ansible/inventory.ini ansible/deploy.yaml`
To deploy new changes in draft, run: `ansible-playbook -i sidecar/ansible/inventory.ini sidecar/ansible/deploy.yaml`

### Generating TLS certs with Let's Encrypt

Expand Down Expand Up @@ -68,10 +68,10 @@ Once you know these:
### Run draft

```
draft start-helper-sidecar --identity <identity> --root_domain example.com --config_path config
draft start-helper-sidecar --identity <identity> --helper_domain helper.example.com --sidecar_domain sidecar.example.com --config_path config
```

This will start the sidecar in the background. To confirm, visit `example.com/status`.
This will start the sidecar in the background. To confirm, visit `sidecar.example.com/status`.


## Local Dev
Expand Down
13 changes: 6 additions & 7 deletions etc/start_helper_sidecar.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,10 @@ fi

config_path=$1
root_path=$2
root_domain=$3
helper_domain=$4
sidecar_domain=$5
helper_port=$6
sidecar_port=$7
identity=$8
helper_domain=$3
sidecar_domain=$4
helper_port=$5
sidecar_port=$6
identity=$7

nohup draft run-helper-sidecar --config_path "$config_path" --root_path "$root_path" --root_domain "$root_domain" --helper_domain "$helper_domain" --sidecar_domain "$sidecar_domain" --helper_port "$helper_port" --sidecar_port "$sidecar_port" --identity "$identity" > .draft/logs/helper_sidecar.log 2>&1 & echo $! > $pid_file
nohup draft run-helper-sidecar --config_path "$config_path" --root_path "$root_path" --helper_domain "$helper_domain" --sidecar_domain "$sidecar_domain" --helper_port "$helper_port" --sidecar_port "$sidecar_port" --identity "$identity" > .draft/logs/helper_sidecar.log 2>&1 & echo $! > $pid_file
3 changes: 2 additions & 1 deletion server/.prettierignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ coverage/*
next.config.js
tsconfig.json
babel.config.js

traefik/*
ansible/*

# Not JS
README.md
44 changes: 44 additions & 0 deletions server/ansible/deploy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
- name: Deploy updates to Draft
hosts: all
tasks:

- name: Pull new commits from GitHub
git:
repo: 'https://github.com/private-attribution/draft.git'
dest: '{{ ansible_env.HOME }}/draft'
update: yes

- name: Install packages based on package-lock.json via npm
npm:
path: '{{ ansible_env.HOME}}/draft/server'
state: present
ci: true

- name: Load .env file
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please help me understand what is getting stored in .env file here? According to cli doc

The decrypted secret value, if the secret value was originally provided as a string or through the Secrets Manager console.

If this secret was created by using the console, then Secrets Manager stores the information as a JSON structure of key/value pairs.

If that's true, I am a little nervous about keeping the secret on durable storage. Is it possible to configure env variable or (probably better) assume role on this instance?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's variables needed server side like API keys, db passwords, etc. We could certainly set these as environmental variables instead of using the .env file, and it seems reasonable to avoid writing them to a file.

When you say "assume role on this instance, this does assume a specific AMI role, which is what allows the aws secretmanager` cli to actually access those secrets. Is there something else you mean by this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, fixed. I added a shell script "load_secrets.sh" which is sourced and adds them into the env. the TLS keys are still written to disk, because traefik expects them there. it's not immediately obvious of a better way to handle that.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

When you say "assume role on this instance, this does assume a specific AMI role, which is what allows the aws secretmanager` cli to actually access those secrets. Is there something else you mean by this?

Yea I was thinking about using https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sts/assume-role.html to get temporary credentials, but this serves different purpose as far as I can tell.

If we store more than one password, API key, etc, we could consider partitioning this and store one secret with one key

shell: >
aws secretsmanager get-secret-value
--secret-id {{ env_secret_id }}
--region {{ aws_region }}
--query SecretString
--output text |
akoshelev marked this conversation as resolved.
Show resolved Hide resolved
jq -r 'to_entries|map("\(.key)=\"\(.value|tostring)\"")|.[]' >
.env
args:
chdir: '{{ ansible_env.HOME }}/draft/server'
executable: /bin/bash


- name: Rebuild draft website
shell: >
npm run build
args:
chdir: '{{ ansible_env.HOME }}/draft/server'
executable: /bin/bash


- name: Restart draft website
shell: >
npm run pm2-restart
args:
chdir: '{{ ansible_env.HOME }}/draft/server'
executable: /bin/bash
5 changes: 5 additions & 0 deletions server/ansible/inventory.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[myhosts]
draft-ipa draft_domain=draft.ipa-helper.dev draft_port=3000 env_secret_id=prod-draft-env aws_region=us-west-2

[myhosts:vars]
ansible_python_interpreter=/usr/bin/python3
132 changes: 132 additions & 0 deletions server/ansible/provision.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
- name: Setup Draft frontend
hosts: all
tasks:
- name: Store HOME directory
debug:
var: ansible_env.HOME

- name: Check if Node.js is installed
command: node --version
register: node_installed
failed_when: false
changed_when: false

- name: Install Node.js Package Manager
yum:
name: nodejs
state: latest
become: yes
when: node_installed.rc != 0

- name: Check if npm is installed
command: npm --version
register: npm_installed
failed_when: false
changed_when: false

- name: Install Node.js Package Manager
yum:
name: npm
state: latest
become: yes
when: npm_installed.rc != 0

- name: Check if Git is installed
command: git --version
register: git_installed
failed_when: false
changed_when: false

- name: Install Git
yum:
name: git
state: latest
become: yes
when: git_installed.rc != 0

- name: Clone repository if it doesn't exist
git:
repo: 'https://github.com/private-attribution/draft.git'
dest: '{{ ansible_env.HOME }}/draft'

- name: Install packages based on package-lock.json via npm
npm:
path: '{{ ansible_env.HOME}}/draft/server'
state: present
ci: true

- name: Check if Traefik is installed
command: '{{ ansible_env.HOME }}/draft/traefik version'
register: traefik_installed
failed_when: false
changed_when: false

- name: Download Traefik
get_url:
url: 'https://github.com/traefik/traefik/releases/download/v2.11.0/traefik_v2.11.0_linux_amd64.tar.gz'
dest: '{{ ansible_env.HOME }}/traefik_v2.11.0_linux_amd64.tar.gz'
checksum: 'sha256:7f31f1cc566bd094f038579fc36e354fd545cf899523eb507c3cfcbbdb8b9552'
when: traefik_installed.rc != 0

- name: Ensure extraction directory exists
file:
path: '{{ ansible_env.HOME }}/traefix_extract/'
state: directory

- name: Extract Traefik
unarchive:
src: '{{ ansible_env.HOME }}/traefik_v2.11.0_linux_amd64.tar.gz'
dest: '{{ ansible_env.HOME }}/traefix_extract/'
remote_src: yes
when: traefik_installed.rc != 0

- name: Copy Traefik binary into draft directory
copy:
src: '{{ ansible_env.HOME }}/traefix_extract/traefik'
dest: '{{ ansible_env.HOME }}/draft'
mode: '0775'
remote_src: yes

- name: Grant CAP_NET_BIND_SERVICE capability to traefik binary
command: 'setcap cap_net_bind_service=+ep {{ ansible_env.HOME }}/draft/traefik'
become: yes

- name: Create cert directory
file:
path: '{{ ansible_env.HOME }}/cert'
state: directory

- name: Load cert.pem file
shell: >
aws secretsmanager get-secret-value
--secret-id cert.pem
--region {{ aws_region }}
--query SecretString
--output text > cert.pem
args:
chdir: '{{ ansible_env.HOME }}/cert'
executable: /bin/bash


- name: Load key.pem file
shell: >
aws secretsmanager get-secret-value
--secret-id key.pem
--region {{ aws_region }}
--query SecretString
--output text > key.pem
args:
chdir: '{{ ansible_env.HOME }}/cert'
executable: /bin/bash

- name: start traefik and nextjs
shell: >
npm run pm2-start
environment:
CERT_DIR: '{{ ansible_env.HOME }}/cert/'
DRAFT_DOMAIN: '{{ draft_domain }}'
DRAFT_PORT: '{{ draft_port }}'

args:
chdir: '{{ ansible_env.HOME }}/draft/server'
executable: /bin/bash
Loading
Loading