Skip to content

Commit

Permalink
Merge #615: Build for arm64 on amd64
Browse files Browse the repository at this point in the history
3d3c02d Improve (Kiminuo)
ee3cf5f Add release instructions (Kiminuo)
d93c8f7 Add support for arm64 (Kiminuo)

Pull request description:

  Related to #345
  Related to WalletWasabi/WalletWasabi#4051 (comment)
  Discussion: Some discussion here: kiminuo#1

  This PR just shows how to do a deterministic build for `arm64` architecture. The hope is HWI's support for `arm64` will improve a bit.

  ### Setup

  To set up your environment, install:

  ```s
  sudo apt install qemu-user-static
  ```

  ### Build for arm64

  Use [docker buildx](https://docs.docker.com/buildx/working-with-buildx/) to replicate [deterministic build instructions](https://github.com/bitcoin-core/HWI/blob/master/docs/development/release-process.rst#deterministic-builds-with-docker) for the arm64 build:

  ```s
  docker buildx build --no-cache --platform linux/arm64 -t hwi-builder -f contrib/build.Dockerfile .

  # Note the use of "--without-gui".
  docker run --platform linux/arm64 -it --rm --name hwi-builder -v $PWD:/opt/hwi  --workdir /opt/hwi hwi-builder /bin/bash -c "contrib/build_bin.sh --without-gui && contrib/build_dist.sh --without-gui"
  ```

  i.e. the change is:

  ```diff
  -docker build --no-cache -t hwi-builder -f contrib/build.Dockerfile .
  -docker run -it --name hwi-builder -v $PWD:/opt/hwi --rm  --workdir /opt/hwi hwi-builder /bin/bash -c "contrib/build_bin.sh && contrib/build_dist.sh && contrib/build_wine.sh"
  +docker buildx build --no-cache --platform linux/arm64 -t hwi-builder -f contrib/build.Dockerfile .
  +docker run --platform linux/arm64 -it --rm --name hwi-builder -v $PWD:/opt/hwi  --workdir /opt/hwi hwi-builder /bin/bash -c "contrib/build_bin.sh --without-gui && contrib/build_dist.sh --without-gui"
  ```

  ### Resources

  * https://docs.docker.com/build/building/multi-platform/#building-multi-platform-images
  * https://medium.com/@artur.klauser/building-multi-architecture-docker-images-with-buildx-27d80f7e2408

  ## Limitations

  ### No GUI support

  So far I had no luck with building `hwi-qt` because I get the following error

  ```
  Installing dependencies from lock file

  Package operations: 14 installs, 0 updates, 0 removals

    • Installing shiboken2 (5.15.2): Failed

    RuntimeError

    Unable to find installation candidates for shiboken2 (5.15.2)

    at ~/.pyenv/versions/3.9.7/lib/python3.9/site-packages/poetry/installation/chooser.py:72 in choose_for
         68│
         69│             links.append(link)
         70│
         71│         if not links:
      →  72│             raise RuntimeError(
         73│                 "Unable to find installation candidates for {}".format(package)
         74│             )
         75│
         76│         # Get the best link
  ```

  during execution of [`poetry install -E qt`](https://github.com/kiminuo/HWI/blob/a025af3ee3fdf0bac8dbdbc51b9e82ab946660b4/contrib/build_bin.sh#L12)

  That's why I use `build_bin.sh --without-gui` (see #655)

ACKs for top commit:
  achow101:
    ACK 3d3c02d

Tree-SHA512: 3df877ec3da07997c8281722ea0ec4687d850f1fec33cb5664e3f03cd8854cb02a90ce799da2123a88058b681fbb2bad31bdacbb485474d19fa88b60a4b73d83
  • Loading branch information
achow101 committed Jan 30, 2024
2 parents 5a1beaf + 3d3c02d commit ac1a323
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 5 deletions.
10 changes: 6 additions & 4 deletions contrib/build_bin.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

set -ex

ARCH=$(uname -m | tr '[:upper:]' '[:lower:]')

eval "$(pyenv init --path)"
eval "$(pyenv virtualenv-init -)"
pip install -U pip
Expand All @@ -12,7 +14,7 @@ pip install poetry
gui_support="${1:---with-gui}";

# Setup poetry and install the dependencies
if [[ $gui_support == "--with-gui" ]]; then
if [[ $gui_support == "--with-gui" && $ARCH == "x86_64" ]]; then
poetry install -E qt
else
poetry install
Expand All @@ -26,7 +28,7 @@ TZ=UTC find ${lib_dir} -name '*.py' -type f -execdir touch -t "201901010000.00"
export PYTHONHASHSEED=42
poetry run pyinstaller hwi.spec

if [[ $gui_support == "--with-gui" ]]; then
if [[ $gui_support == "--with-gui" && $ARCH == "x86_64" ]]; then
poetry run contrib/generate-ui.sh
poetry run pyinstaller hwi-qt.spec
fi
Expand All @@ -40,7 +42,7 @@ OS=`uname | tr '[:upper:]' '[:lower:]'`
if [[ $OS == "darwin" ]]; then
OS="mac"
fi
ARCH=$(uname -m | tr '[:upper:]' '[:lower:]')

target_tarfile="hwi-${VERSION}-${OS}-${ARCH}.tar.gz"

if [[ $gui_support == "--with-gui" ]]; then
Expand All @@ -54,7 +56,7 @@ target_dir="$target_tarfile.dir"
mkdir $target_dir
mv hwi $target_dir

if [[ $gui_support == "--with-gui" ]]; then
if [[ $gui_support == "--with-gui" && $arch == "x86_64" ]]; then
mv hwi-qt $target_dir
fi

Expand Down
7 changes: 7 additions & 0 deletions docs/development/release-process.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,17 @@ Create the docker images::
docker build --no-cache -t hwi-builder -f contrib/build.Dockerfile .
docker build --no-cache -t hwi-wine-builder -f contrib/build-wine.Dockerfile .

# arm64
sudo apt-get install qemu-user-static
docker buildx build --no-cache --platform linux/arm64 -t hwi-builder-arm64 -f contrib/build.Dockerfile .

Build everything::

docker run -it --name hwi-builder -v $PWD:/opt/hwi --rm --workdir /opt/hwi hwi-builder /bin/bash -c "contrib/build_bin.sh && contrib/build_dist.sh"
docker run -it --name hwi-wine-builder -v $PWD:/opt/hwi --rm --workdir /opt/hwi hwi-wine-builder /bin/bash -c "contrib/build_wine.sh"
docker run --platform linux/arm64 -it --rm --name hwi-builder-arm64 -v $PWD:/opt/hwi --workdir /opt/hwi hwi-builder-arm64 /bin/bash -c "contrib/build_bin.sh --without-gui && contrib/build_dist.sh --without-gui"

i.e.

Building macOS binary
=====================
Expand Down
2 changes: 1 addition & 1 deletion hwi.spec
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ def get_libusb_path():
prefix = proc.communicate()[0].rstrip().decode()
return os.path.join(prefix, "lib", "libusb-1.0.dylib")
if platform.system() == "Linux":
for lib_dir in ["/lib/x86_64-linux-gnu", "/usr/lib64", "/lib64" "/usr/lib", "/lib"]:
for lib_dir in ["/lib/x86_64-linux-gnu", "/lib/aarch64-linux-gnu", "/usr/lib64", "/lib64" "/usr/lib", "/lib"]:
libusb_path = os.path.join(lib_dir, "libusb-1.0.so.0")
if os.path.exists(libusb_path):
return libusb_path
Expand Down

0 comments on commit ac1a323

Please sign in to comment.