Skip to content

Commit

Permalink
remove data-plane-gateway implementation and make it a simple proxy
Browse files Browse the repository at this point in the history
Since data-plane-gateway was written:

* The connector networking frontend was moved into reactors and
  significantly improved.

* gRPC Web REST handlers were implemented in reactors and gazette.

* reactors and gazette implemented first-class fine grain
  authorizations.

Remove the historical implementations of these features, instead
updating to the implementations used by reactors and brokers today.

Remove authorization checks, and just verify and pass-through an
authorization header.

Issue #estuary/flow/issues/1627
  • Loading branch information
jgraettinger committed Dec 9, 2024
1 parent a906ada commit cc4a961
Show file tree
Hide file tree
Showing 31 changed files with 363 additions and 5,045 deletions.
2 changes: 0 additions & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,2 @@
docs/
client/
test/
*.sh
40 changes: 2 additions & 38 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,10 @@ on:

jobs:
build:
runs-on: ubuntu-20.04
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Prepare
id: prep
Expand All @@ -26,45 +24,11 @@ jobs:
echo "${{ secrets.GITHUB_TOKEN }}" | \
docker login --username ${{ github.actor }} --password-stdin ghcr.io
- uses: actions/setup-go@v3
with:
# Installs the go version specified in go.mod
go-version-file: 'go.mod'
cache: true

- name: Install protobuf-compiler
run: sudo apt install -y libprotobuf-dev protobuf-compiler

- uses: denoland/setup-deno@v1
with:
deno-version: v1.32.1

- name: Fetch Flow
run: |
mkdir $HOME/bin \
&& curl -L --proto '=https' --tlsv1.2 -sSf "https://github.com/estuary/flow/releases/download/dev/flow-x86-linux.tar.gz" \
| tar -zx -C $HOME/bin
- name: Setup Protobuf Tools
run: make protobuf_tools

- name: Add GOBIN to PATH
run: echo "$HOME/go/bin" >> $GITHUB_PATH

- name: Install Go deps
run: go mod download

- name: Build
run: make && go build -o $HOME/bin/data-plane-gateway .

- name: Ensure that generated files are unchanged.
run: |
git status \
&& git diff \
&& [[ -z "$(git status --porcelain)" ]] || exit 1
- name: Run Tests
run: ./test.sh run $HOME/bin/data-plane-gateway $HOME/bin/flowctl-go
run: go build -o data-plane-gateway

- name: Build Docker Image
uses: docker/build-push-action@v2
Expand Down
7 changes: 1 addition & 6 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1 @@
# This certificate and key are generated by start-flow.sh for local development
local-tls-cert.pem
local-tls-private-key.pem

data-plane-gateway
test/tmp/*
data-plane-gateway
44 changes: 4 additions & 40 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,51 +1,15 @@
# Build Stage
################################################################################
FROM golang as builder

WORKDIR /builder

RUN apt-get update && apt-get install -y openssl

# Download & compile dependencies early. Doing this separately allows for layer
# caching opportunities when no dependencies are updated.
COPY go.* ./
RUN go mod download

# Build the gateway.
COPY *.go ./
COPY gen ./gen
COPY auth ./auth
COPY proxy ./proxy
RUN go build .

# Generate a self-signed certificate to allow the server to use TLS
RUN openssl req -x509 -nodes -days 1095 \
-subj "/C=CA/ST=QC/O=Estuary/CN=not-a-real-hostname.test" \
-newkey rsa:2048 -keyout tls-private-key.pem \
-out tls-cert.pem

# We'll copy the sh executable out of this, since distroless doesn't have a package manager with
# which to install one
FROM busybox:1.34-musl as busybox

# Runtime Stage
################################################################################
FROM gcr.io/distroless/base-debian12

COPY --from=busybox /bin/sh /bin/sh
FROM ubuntu:24.04

WORKDIR /app
ENV PATH="/app:$PATH"

# Bring in the compiled artifact from the builder.
COPY --from=builder /builder/data-plane-gateway ./
COPY --from=builder --chown=nonroot /builder/tls-private-key.pem ./
COPY --from=builder --chown=nonroot /builder/tls-cert.pem ./
COPY data-plane-gateway ./

# Avoid running the data-plane-gateway as root.
USER nonroot:nonroot
USER 65534:65534

# Ensure data-plane-gateway can run on this runtime image.
RUN ./data-plane-gateway --help
RUN /app/data-plane-gateway print-config

ENTRYPOINT ["/app/data-plane-gateway"]
42 changes: 0 additions & 42 deletions Makefile

This file was deleted.

184 changes: 0 additions & 184 deletions auth/auth.go

This file was deleted.

11 changes: 0 additions & 11 deletions broker_service.yaml

This file was deleted.

11 changes: 0 additions & 11 deletions consumer_service.yaml

This file was deleted.

Loading

0 comments on commit cc4a961

Please sign in to comment.