Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create Docker builds and add docker-compose config for self-hosting #280

Open
wants to merge 48 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
047cbe5
create Docker builds and add support for runtime env config
jshimko Sep 18, 2024
1efcd23
Merge branch 'dev' into docker-builds
jshimko Sep 20, 2024
f7a2e24
Merge branch 'dev' into docker-builds
jshimko Sep 23, 2024
114ff41
[email protected]
jshimko Sep 23, 2024
083dfa3
remove logged default adin details from seed script
jshimko Sep 23, 2024
12efde1
build for both cpu platforms
jshimko Sep 23, 2024
04bae10
disable client team creation by default in seed
jshimko Sep 23, 2024
315e47e
Merge branch 'dev' into docker-builds
jshimko Sep 24, 2024
2ea9788
fix self host seed script
jshimko Sep 24, 2024
ccc2594
Merge branch 'dev' into docker-builds
jshimko Sep 24, 2024
896088d
prisma 5.20.0
jshimko Sep 24, 2024
1128361
disable arm builds for now
jshimko Sep 24, 2024
9b39752
use prisma directly in docker entrypoint
jshimko Sep 24, 2024
3e0538a
fix wrong dir
jshimko Sep 24, 2024
7f224a2
Merge branch 'dev' into docker-builds
jshimko Sep 25, 2024
d8a0426
env cleanup
jshimko Sep 25, 2024
12b4f2c
revert to previous prisma command in entrypoint script
jshimko Sep 25, 2024
3019f97
Merge branch 'dev' into docker-builds
jshimko Sep 26, 2024
8ebb72d
remove unused multi platform config
jshimko Sep 26, 2024
210afd3
move self-host seed script config to Docker env file
jshimko Sep 26, 2024
9e639b3
self host seed script cleanup
jshimko Sep 26, 2024
e27365b
seed script cleanup
jshimko Sep 26, 2024
656fd4f
db rename
jshimko Sep 26, 2024
7e3c062
only run docker build on dev and main
jshimko Sep 26, 2024
9d2251f
refactor the backend docker build to support NextJS standalone mode
jshimko Sep 26, 2024
2a12fb1
Merge branch 'dev' into docker-builds
jshimko Sep 26, 2024
f6b7de8
add more comments to Dockerfiles
jshimko Sep 26, 2024
44f3917
remove unused script
jshimko Sep 26, 2024
8c5e0e9
delete empty line
jshimko Sep 26, 2024
c6f3d49
remove unused pnpm in final docker build
jshimko Sep 26, 2024
9c46cbd
improve backend startup healthcheck in docker-compose
jshimko Sep 26, 2024
1baec26
lint cleanup
jshimko Sep 26, 2024
10c4ca7
remove unused script
jshimko Sep 26, 2024
016676f
Merge branch 'dev' into docker-builds
jshimko Sep 30, 2024
18b0675
Merge branch 'dev' into docker-builds
jshimko Oct 1, 2024
f85efa5
fix self host seed script after authMethod schema updates
jshimko Oct 1, 2024
5bde713
skip docker buildx check for SecretsUsedInArgOrEnv
jshimko Oct 1, 2024
80c2460
Merge branch 'dev' into docker-builds
jshimko Oct 2, 2024
fe3c631
remove exposed db ports
jshimko Oct 2, 2024
ca7954c
remove unnecessary EXPOSE directives
jshimko Oct 2, 2024
3fea967
Merge branch 'dev' into docker-builds
jshimko Oct 4, 2024
0551457
Merge branch 'dev' into docker-builds
jshimko Oct 7, 2024
2eceee7
[email protected]
jshimko Oct 7, 2024
cc2cee2
fix authMethodConfigs schema change in seed script
jshimko Oct 7, 2024
a69d738
ensure seed script admin user isn’t created because of code comments
jshimko Oct 7, 2024
78dbf23
fix self host seed script contactChannel auth option
jshimko Oct 7, 2024
acf5b94
Merge branch 'dev' into docker-builds
jshimko Oct 8, 2024
b1072aa
Merge branch 'dev' into docker-builds
jshimko Oct 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
.changeset
.git
.github
.turbo
**/.turbo
.vscode

.env
.env.*
**/.env
**/.env.*
**/.next

**/dist

examples

node_modules
**/node_modules

deploy
!deploy/docker/**/entrypoint.sh
docker-compose.yaml
87 changes: 87 additions & 0 deletions .github/workflows/docker-build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
name: Docker Build and Push

on:
push:
branches:
- dev
- main
- docker-build # remove me before PR merge
tags:
- "*.*.*"

jobs:
build-dashboard:
name: Docker Build and Push Dashboard
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ secrets.DOCKER_REPO }}/stack-dashboard
tags: |
type=ref,event=branch
type=sha,prefix=
type=match,pattern=\d.\d.\d

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./deploy/docker/dashboard/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

build-backend:
name: Docker Build and Push Backend
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ secrets.DOCKER_REPO }}/stack-backend
tags: |
type=ref,event=branch
type=sha,prefix=
type=match,pattern=\d.\d.\d

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./deploy/docker/backend/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
34 changes: 34 additions & 0 deletions apps/backend/.env.docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@

STACK_BASE_URL=http://host.docker.internal:8102
STACK_SERVER_SECRET=23-wuNpik0gIW4mruTz25rbIvhuuvZFrLOLtL7J4tyo

# Postgres connection strings
STACK_DATABASE_CONNECTION_STRING=postgres://stack:stack123@postgres:5432/stack
STACK_DIRECT_DATABASE_CONNECTION_STRING=postgres://stack:stack123@postgres:5432/stack

# optionally skip migrations on container startup
STACK_SKIP_MIGRATIONS="false"

# run Prisma seed script on container startup
STACK_RUN_SEED_SCRIPT="true"

# Self-host seed script options
# Optionally create a default admin user and disable signups to the platform when the seed script is run.
# This can be useful for self hosted deployments where you want to restrict access to the Stack "internal" project.
# If not specified, no user will be created and you will have to create the first user manually by signing up.
# [email protected]
# STACK_DEFAULT_ADMIN_PASSWORD=admin123
# STACK_DEFAULT_ADMIN_DISPLAY_NAME=Admin

# Sign up
STACK_SIGN_UP_DISABLED= # set to "true" to disable sign up on the platform ("internal" project). If set to "true", the default admin user above will need to be created to log in

STACK_EMAIL_HOST=inbucket
STACK_EMAIL_PORT=2500
STACK_EMAIL_SECURE=false
STACK_EMAIL_USERNAME=does not matter, ignored by Inbucket
STACK_EMAIL_PASSWORD=does not matter, ignored by Inbucket
[email protected]

STACK_SVIX_SERVER_URL=http://host.docker.internal:8113
STACK_SVIX_API_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2NTUxNDA2MzksImV4cCI6MTk3MDUwMDYzOSwibmJmIjoxNjU1MTQwNjM5LCJpc3MiOiJzdml4LXNlcnZlciIsInN1YiI6Im9yZ18yM3JiOFlkR3FNVDBxSXpwZ0d3ZFhmSGlyTXUifQ.En8w77ZJWbd0qrMlHHupHUB-4cx17RfzFykseg95SUk
4 changes: 4 additions & 0 deletions apps/backend/next.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ const withConfiguredSentryConfig = (nextConfig) =>

/** @type {import('next').NextConfig} */
const nextConfig = {
// optionally set output to "standalone" for Docker builds
// https://nextjs.org/docs/pages/api-reference/next-config-js/output
output: process.env.NEXT_CONFIG_OUTPUT,

// we're open-source, so we can provide source maps
productionBrowserSourceMaps: true,
poweredByHeader: false,
Expand Down
9 changes: 6 additions & 3 deletions apps/backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
"with-env:prod": "dotenv -c --",
"dev": "concurrently -n \"dev,codegen,prisma-studio\" -k \"next dev --port 8102\" \"pnpm run codegen:watch\" \"pnpm run prisma-studio\"",
"build": "pnpm run codegen && next build",
"build:self-host-seed-script": "tsup --config prisma/tsup.config.ts",
"analyze-bundle": "ANALYZE_BUNDLE=1 pnpm run build",
"start": "next start --port 8102",
"codegen-prisma": "pnpm run prisma generate",
Expand Down Expand Up @@ -41,8 +42,8 @@
"@opentelemetry/sdk-trace-base": "^1.26.0",
"@opentelemetry/sdk-trace-node": "^1.26.0",
"@opentelemetry/semantic-conventions": "^1.27.0",
"@prisma/client": "^5.9.1",
"@prisma/instrumentation": "^5.19.1",
"@prisma/client": "^5.20.0",
"@prisma/instrumentation": "^5.20.0",
"@sentry/nextjs": "^7.105.0",
"@stackframe/stack-emails": "workspace:*",
"@stackframe/stack-shared": "workspace:*",
Expand All @@ -53,6 +54,7 @@
"dotenv-cli": "^7.3.0",
"jose": "^5.2.2",
"next": "^14.2.5",
"next-runtime-env": "^3.2.2",
"nodemailer": "^6.9.10",
"openid-client": "^5.6.4",
"oslo": "^1.2.1",
Expand All @@ -73,8 +75,9 @@
"@types/semver": "^7.5.8",
"concurrently": "^8.2.2",
"glob": "^10.4.1",
"prisma": "^5.9.1",
"prisma": "^5.20.0",
"rimraf": "^5.0.5",
"tsup": "^8.3.0",
"tsx": "^4.7.2"
}
}
164 changes: 164 additions & 0 deletions apps/backend/prisma/seed-self-host.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
/* eslint-disable no-restricted-syntax */
import { BooleanTrue, PrismaClient } from '@prisma/client';
import { hashPassword } from '@stackframe/stack-shared/dist/utils/password';

const prisma = new PrismaClient();

async function seed() {
console.log('Seeding database...');

// Optional default admin user
const adminDisplayName = process.env.STACK_DEFAULT_ADMIN_DISPLAY_NAME || 'Admin';
const adminEmail = process.env.STACK_DEFAULT_ADMIN_EMAIL;
const adminPassword = process.env.STACK_DEFAULT_ADMIN_PASSWORD;

// Optionally disable sign up for "internal" project
const signUpEnabled = process.env.STACK_SIGN_UP_DISABLED !== 'true';

const existingProject = await prisma.project.findUnique({
where: {
id: 'internal',
},
});

if (existingProject) {
console.log('Internal project already exists, skipping seed script');
return;
}

await prisma.$transaction(async (tx) => {
const createdProject = await tx.project.create({
data: {
id: 'internal',
displayName: 'Stack Dashboard',
description: 'Stack\'s admin dashboard',
isProductionMode: false,
apiKeySets: {
create: [{
description: "Internal API key set",
// These keys must match the values used in the Stack dashboard env to be able to login via the UI.
publishableClientKey: process.env.NEXT_PUBLIC_STACK_PUBLISHABLE_CLIENT_KEY,
secretServerKey: process.env.STACK_SECRET_SERVER_KEY,
superSecretAdminKey: process.env.STACK_SUPER_SECRET_ADMIN_KEY,
expiresAt: new Date('2099-12-31T23:59:59Z'),
}],
},
config: {
create: {
allowLocalhost: true,
signUpEnabled, // see STACK_SIGN_UP_DISABLED var above
emailServiceConfig: {
create: {
proxiedEmailServiceConfig: {
create: {}
}
}
},
createTeamOnSignUp: false,
clientTeamCreationEnabled: false,
authMethodConfigs: {
create: [
{
otpConfig: {
create: {
contactChannelType: 'EMAIL',
}
}
},
{
passwordConfig: {
create: {},
}
},
],
}
}
}
},
});

console.log('Internal project created');

// Create optional default admin user if credentials are provided.
// This user will be able to login to the dashboard with both email/password and magic link.
if (adminEmail && adminPassword) {
const newUser = await tx.projectUser.create({
data: {
projectId: 'internal',
displayName: adminDisplayName,
serverMetadata: { managedProjectIds: ['internal'] }
}
});

await tx.contactChannel.create({
data: {
projectUserId: newUser.projectUserId,
projectId: 'internal',
type: 'EMAIL' as const,
value: adminEmail as string,
isVerified: true,
isPrimary: 'TRUE',
usedForAuth: BooleanTrue.TRUE,
}
});

const otpConfig = await tx.otpAuthMethodConfig.findFirstOrThrow({
where: {
projectConfigId: createdProject.configId
},
include: {
authMethodConfig: true,
}
});

await tx.authMethod.create({
data: {
projectId: 'internal',
projectUserId: newUser.projectUserId,
projectConfigId: createdProject.configId,
authMethodConfigId: otpConfig.authMethodConfigId,
otpAuthMethod: {
create: {
projectUserId: newUser.projectUserId,
}
}
}
});

const passwordConfig = await tx.passwordAuthMethodConfig.findFirstOrThrow({
where: {
projectConfigId: createdProject.configId
},
include: {
authMethodConfig: true,
}
});

await tx.authMethod.create({
data: {
projectId: 'internal',
projectConfigId: createdProject.configId,
projectUserId: newUser.projectUserId,
authMethodConfigId: passwordConfig.authMethodConfigId,
passwordAuthMethod: {
create: {
passwordHash: await hashPassword(adminPassword),
projectUserId: newUser.projectUserId,
}
}
}
});

console.log('Initial admin user created: ', adminEmail);
}
});

console.log('Seeding complete!');
}

seed().catch(async (e) => {
console.error(e);
await prisma.$disconnect();
process.exit(1);
// eslint-disable-next-line @typescript-eslint/no-misused-promises
}).finally(async () => await prisma.$disconnect());
13 changes: 13 additions & 0 deletions apps/backend/prisma/tsup.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import { defineConfig } from 'tsup';

// tsup config to build the self-hosting seed script so it can be
// run in the Docker container with no extra dependencies.
export default defineConfig({
entry: ['prisma/seed-self-host.ts'],
format: ['cjs'],
outDir: 'dist',
target: 'node22',
platform: 'node',
noExternal: ['@stackframe/stack-shared', '@prisma/client'],
clean: true
});
4 changes: 2 additions & 2 deletions apps/backend/src/analytics.tsx
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import { getEnvVariable } from '@stackframe/stack-shared/dist/utils/env';
import { env } from "next-runtime-env";
import { PostHog } from 'posthog-node';

export default async function withPostHog<T>(callback: (posthog: PostHog) => Promise<T>) {
const postHogKey = getEnvVariable("NEXT_PUBLIC_POSTHOG_KEY", "phc_vIUFi0HzHo7oV26OsaZbUASqxvs8qOmap1UBYAutU4k");
const postHogKey = env("NEXT_PUBLIC_POSTHOG_KEY") || "phc_vIUFi0HzHo7oV26OsaZbUASqxvs8qOmap1UBYAutU4k";
const posthogClient = new PostHog(postHogKey, {
host: "https://eu.i.posthog.com",
flushAt: 1,
Expand Down
10 changes: 10 additions & 0 deletions apps/dashboard/.env.docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
NEXT_PUBLIC_STACK_URL=http://host.docker.internal:8102

NEXT_PUBLIC_STACK_PROJECT_ID=internal
NEXT_PUBLIC_STACK_PUBLISHABLE_CLIENT_KEY=this-publishable-client-key-is-for-local-development-only
STACK_SECRET_SERVER_KEY=this-secret-server-key-is-for-local-development-only
STACK_SUPER_SECRET_ADMIN_KEY=this-super-secret-admin-key-is-for-local-development-only

NEXT_PUBLIC_INSECURE_COOKIE="true" # required for local docker testing on localhost

NEXT_PUBLIC_STACK_SVIX_SERVER_URL=http://host.docker.internal:8113
Loading