feat: prebuilt binaries with Ledger support (backport #2872) (#2893)

This is an automatic backport of pull request #2872 done by [Mergify](https://mergify.com). After this merges, I plan on cutting 1.5.0-rc0. --------- Co-authored-by: Rootul P <[email protected]>
celestiaorg · Dec 4, 2023 · 01e080b · 01e080b
1 parent 2dbfabf
commit 01e080b
Show file tree

Hide file tree

Showing 6 changed files with 100 additions and 97 deletions.
diff --git a/.github/workflows/ci-release.yml b/.github/workflows/ci-release.yml
@@ -70,7 +70,7 @@ jobs:
   # Generate the release with goreleaser to include pre-built binaries
   goreleaser:
     needs: version_bump
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     if: |
       github.event_name == 'workflow_dispatch' ||
       (github.event_name == 'push' && contains(github.ref, 'refs/tags/'))
@@ -81,19 +81,8 @@ jobs:
       - uses: actions/setup-go@v4
         with:
           go-version: 1.21.1
-      # Import the GPG key from Github secrets to sign the binaries
-      - name: Import GPG key
-        id: import_gpg
-        uses: crazy-max/ghaction-import-gpg@v4
-        with:
-          gpg_private_key: ${{ secrets.GPG_SIGNING_KEY }}
-          passphrase: ${{ secrets.GPG_PASSPHRASE }}
-      # Generate the binaries, release, and sign the checksum
-      - uses: goreleaser/goreleaser-action@v4
-        with:
-          distribution: goreleaser
-          version: latest
-          args: release --clean
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          GPG_FINGERPRINT: ${{ steps.import_gpg.outputs.fingerprint }}
+      - name: Create .release-env file
+        run: |-
+          echo 'GITHUB_TOKEN=${{secrets.GORELEASER_ACCESS_TOKEN}}' >> .release-env
+      - name: Create prebuilt binaries and attach them to the GitHub release
+        run: make prebuilt-binary
diff --git a/.gitignore b/.gitignore
@@ -14,3 +14,5 @@ run.sh
 testing/e2e/networks/*/
 square/testdata
 **/*.html
+.release-env
+**/*.DS_Store
diff --git a/.goreleaser.yaml b/.goreleaser.yaml
@@ -1,36 +1,93 @@
-# This is an example .goreleaser.yml file with some sensible defaults.
-# Make sure to check the documentation at https://goreleaser.com
-
-# NOTE: CGO is required for ledger support to work, however goreleaser
-# technically doesn't support CGO. But it seems to work so that's cool. This
-# only seems to work because we are building for a single binary. Cross
-# compiling binaries for multiple distributions doesn't work and a proper
-# workaround will be needed.
-#
-# REF: https://goreleaser.com/limitations/cgo/
-
+# Ledger support is only available if the binary is built with CGO enabled.
+# Since GoReleaser doesn't support CGO natively, our GoReleaser process builds
+# binaries in a Docker image maintained by goreleaser-cross that has CGO support
+# for multiple platforms. See https://github.com/goreleaser/goreleaser-cross
 before:
   hooks:
     - go mod tidy
 builds:
-  - main: ./cmd/celestia-appd
+  - id: darwin-amd64
+    main: ./cmd/celestia-appd
     binary: celestia-appd
     env:
       - SDKPath={{ "github.com/cosmos/cosmos-sdk/version" }}
+      - CGO_ENABLED=1
+      - CC=o64-clang
+      - CXX=o64-clang++
     goarch:
       - amd64
+    goos:
+      - darwin
+    tags:
+      - ledger
+    ldflags:
+      # Ref: https://goreleaser.com/customization/templates/#common-fields
+      # .Version is the version being released
+      # .FullCommit is git commit hash goreleaser is using for the release
+      - -X "{{ .Env.SDKPath }}.Name=celestia-app"
+      - -X "{{ .Env.SDKPath }}.AppName=celestia-appd"
+      - -X "{{ .Env.SDKPath }}.Version={{ .Version }}"
+      - -X "{{ .Env.SDKPath }}.Commit={{ .FullCommit }}"
+  - id: darwin-arm64
+    main: ./cmd/celestia-appd
+    binary: celestia-appd
+    env:
+      - SDKPath={{ "github.com/cosmos/cosmos-sdk/version" }}
+      - CGO_ENABLED=1
+      - CC=oa64-clang
+      - CXX=oa64-clang++
+    goarch:
       - arm64
     goos:
       - darwin
+    tags:
+      - ledger
+    ldflags:
+      # Ref: https://goreleaser.com/customization/templates/#common-fields
+      # .Version is the version being released
+      # .FullCommit is git commit hash goreleaser is using for the release
+      - -X "{{ .Env.SDKPath }}.Name=celestia-app"
+      - -X "{{ .Env.SDKPath }}.AppName=celestia-appd"
+      - -X "{{ .Env.SDKPath }}.Version={{ .Version }}"
+      - -X "{{ .Env.SDKPath }}.Commit={{ .FullCommit }}"
+  - id: linux-amd64
+    main: ./cmd/celestia-appd
+    binary: celestia-appd
+    env:
+      - SDKPath={{ "github.com/cosmos/cosmos-sdk/version" }}
+      - CC=x86_64-linux-gnu-gcc
+      - CXX=x86_64-linux-gnu-g++
+    goarch:
+      - amd64
+    goos:
       - linux
     tags:
       - ledger
     ldflags:
       # Ref: https://goreleaser.com/customization/templates/#common-fields
-      #
+      # .Version is the version being released
       # .FullCommit is git commit hash goreleaser is using for the release
-      #
+      - -X "{{ .Env.SDKPath }}.Name=celestia-app"
+      - -X "{{ .Env.SDKPath }}.AppName=celestia-appd"
+      - -X "{{ .Env.SDKPath }}.Version={{ .Version }}"
+      - -X "{{ .Env.SDKPath }}.Commit={{ .FullCommit }}"
+  - id: linux-arm64
+    main: ./cmd/celestia-appd
+    binary: celestia-appd
+    env:
+      - SDKPath={{ "github.com/cosmos/cosmos-sdk/version" }}
+      - CC=aarch64-linux-gnu-gcc
+      - CXX=aarch64-linux-gnu-g++
+    goarch:
+      - arm64
+    goos:
+      - linux
+    tags:
+      - ledger
+    ldflags:
+      # Ref: https://goreleaser.com/customization/templates/#common-fields
       # .Version is the version being released
+      # .FullCommit is git commit hash goreleaser is using for the release
       - -X "{{ .Env.SDKPath }}.Name=celestia-app"
       - -X "{{ .Env.SDKPath }}.AppName=celestia-appd"
       - -X "{{ .Env.SDKPath }}.Version={{ .Version }}"
@@ -49,18 +106,6 @@ archives:
       {{- if .Arm }}v{{ .Arm }}{{ end }}
 checksum:
   name_template: "checksums.txt"
-signs:
-  - artifacts: checksum
-    args:
-      [
-        "--batch",
-        "-u",
-        "{{ .Env.GPG_FINGERPRINT }}",
-        "--output",
-        "${signature}",
-        "--detach-sign",
-        "${artifact}",
-      ]
 snapshot:
   name_template: "{{ incpatch .Version }}-next"
 changelog:

diff --git a/Makefile b/Makefile
@@ -6,6 +6,8 @@ IMAGE := ghcr.io/tendermint/docker-build-proto:latest
 DOCKER_PROTO_BUILDER := docker run -v $(shell pwd):/workspace --workdir /workspace $(IMAGE)
 PROJECTNAME=$(shell basename "$(PWD)")
 HTTPS_GIT := https://github.com/celestiaorg/celestia-app.git
+PACKAGE_NAME          := github.com/celestiaorg/celestia-app
+GOLANG_CROSS_VERSION  ?= v1.21.4
 
 # process linker flags
 ldflags = -X github.com/cosmos/cosmos-sdk/version.Name=celestia-app \
@@ -139,19 +141,19 @@ adr-gen:
 	@curl -sSL https://raw.githubusercontent.com/celestiaorg/.github/main/adr-template.md > docs/architecture/adr-template.md
 .PHONY: adr-gen
 
-## goreleaser: List Goreleaser commands and checks if GoReleaser is installed.
-goreleaser: Makefile
-	@echo " Choose a goreleaser command to run:"
-	@sed -n 's/^## goreleaser/goreleaser/p' $< | column -t -s ':' |  sed -e 's/^/ /'
-	@goreleaser --version
-.PHONY: goreleaser
-
-## goreleaser-build: Builds the celestia-appd binary using GoReleaser for your local OS.
-goreleaser-build:
-	goreleaser build --snapshot --clean --single-target
-.PHONY: goreleaser-build
-
-## goreleaser-release: Builds the release celestia-appd binary as defined in .goreleaser.yaml. This requires there be a git tag for the release in the local git history.
-goreleaser-release:
-	goreleaser release --clean --fail-fast --skip-publish
-.PHONY: goreleaser-release
+## prebuilt-binary: Create prebuilt binaries and attach them to GitHub release. Requires Docker.
+prebuilt-binary:
+	@if [ ! -f ".release-env" ]; then \
+		echo "A .release-env file was not found but is required to create prebuilt binaries. This command is expected to be run in CI where a .release-env file exists. If you need to run this command locally to attach binaries to a release, you need to create a .release-env file with a Github token (classic) that has repo:public_repo scope."; \
+		exit 1;\
+	fi
+	docker run \
+		--rm \
+		-e CGO_ENABLED=1 \
+		--env-file .release-env \
+		-v /var/run/docker.sock:/var/run/docker.sock \
+		-v `pwd`:/go/src/$(PACKAGE_NAME) \
+		-w /go/src/$(PACKAGE_NAME) \
+		ghcr.io/goreleaser/goreleaser-cross:${GOLANG_CROSS_VERSION} \
+		release --clean
+.PHONY: prebuilt-binary
diff --git a/README.md b/README.md
@@ -46,9 +46,9 @@ node            |  |                               |  |
     make install
     ```
 
-### Pre-built binary
+### Prebuilt binary
 
-If you'd rather not install from source, you can download a pre-built binary from the [releases](https://github.com/celestiaorg/celestia-app/releases) page.
+If you'd rather not install from source, you can download a prebuilt binary from the [releases](https://github.com/celestiaorg/celestia-app/releases) page.
 
 1. Navigate to the latest release on <https://github.com/celestiaorg/celestia-app/releases>.
 1. Download the binary for your platform (e.g. `celestia-app_Linux_x86_64.tar.gz`) from the **Assets** section. Tip: if you're not sure what platform you're on, you can run `uname -a` and look for the operating system (e.g. `Linux`, `Darwin`) and architecture (e.g. `x86_64`, `arm64`).
@@ -64,13 +64,7 @@ If you'd rather not install from source, you can download a pre-built binary fro
     ./celestia-appd --help
     ```
 
-#### Optional: Verify the pre-built binary checksums and signatures
-
-If you use a pre-built binary, you may also want to verify the checksums and signatures.
-
-1. Navigate to the latest release on <https://github.com/celestiaorg/celestia-app/releases>.
-1. Download `checksums.txt`, `checksums.txt.sig`, and the binary for your platform (e.g. `celestia-app_Linux_x86_64.tar.gz`) from the **Assets** section. Tip: if you're not sure what platform you're on, you can run `uname -a` and look for the operating system (e.g. `Linux`, `Darwin`) and architecture (e.g. `x86_64`, `arm64`).
-1. Verify the checksums
+1. [Optional] verify the prebuilt binary checksum. Download `checksums.txt` and then verify the checksum:
 
     ```shell
     sha256sum --ignore-missing --check checksums.txt
@@ -82,21 +76,6 @@ If you use a pre-built binary, you may also want to verify the checksums and sig
     celestia-app_Linux_x86_64.tar.gz: OK
     ```
 
-1. Download the [verify-signature.sh](./scripts/signing/verify-signature.sh) script.
-1. Verify the signature via the [verify-signature.sh](./scripts/signing/verify-signature.sh) script
-
-    ```shell
-    ./verify-signature.sh checksums.txt.sig checksums.txt
-    ```
-
-    You should see output like this:
-
-    ```shell
-    gpg: Signature made Thu Sep 21 14:39:26 2023 EDT
-    gpg:                using EDDSA key BF02F32CC36864560B90B764D469F859693DC3FA
-    gpg: Good signature from "celestia-app-maintainers <[email protected]>" [ultimate]
-    ```
-
 ### Ledger Support
 
 Ledger is not supported on Windows and OpenBSD.

diff --git a/scripts/signing/celestia-app-maintainers.asc b/scripts/signing/celestia-app-maintainers.asc