Skip to content

Commit

Permalink
Bootstrap with snapshot of sharp/packaging
Browse files Browse the repository at this point in the history
  • Loading branch information
lovell committed Sep 22, 2017
0 parents commit 9ccb759
Show file tree
Hide file tree
Showing 15 changed files with 540 additions and 0 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
libvips*
*.log
57 changes: 57 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Packaging scripts

libvips and its dependencies are provided as pre-compiled shared libraries
for the most common operating systems and CPU architectures.

During `npm install`, these binaries are fetched as tarballs from
[Bintray](https://dl.bintray.com/lovell/sharp/) via HTTPS
and stored locally within `node_modules/sharp`.

## Using a custom tarball

A custom tarball stored on the local filesystem can be used instead.
Place it in the following location, where `x.y.z` is the libvips version,
`platform` is the value of `process.platform` and
`arch` is the value of `process.arch` (plus the version number for ARM).

`node_modules/sharp/packaging/libvips-x.y.z-platform-arch.tar.gz`

For example, for libvips v8.3.3 on an ARMv6 Linux machine, use:

`node_modules/sharp/packaging/libvips-8.3.3-linux-armv6.tar.gz`

Remove any `sharp/lib` and `sharp/include` directories
before running `npm install` again.

## Creating a tarball

Most people will not need to do this; proceed with caution.

The `packaging` directory contains the top-level [build script](build.sh).

### Linux

One [build script](build/lin.sh) is used to (cross-)compile
the same shared libraries within multiple containers.

* [x64](linux-x64/Dockerfile)
* [ARMv6](linux-armv6/Dockerfile)
* [ARMv7-A](linux-armv7/Dockerfile)
* [ARMv8-A](linux-armv8/Dockerfile)

The QEMU user mode emulation binaries are required to build for
the ARMv6 platform as the Debian armhf cross-compiler erroneously
generates unsupported Thumb 2 instructions.

```sh
sudo apt-get install qemu-user-static
```

### Windows

The output of libvips' [build-win64](https://github.com/jcupitt/build-win64)
"web" target is [post-processed](build/win.sh) within a [container](win32-x64/Dockerfile).

### OS X

See [package-libvips-darwin](https://github.com/lovell/package-libvips-darwin).
48 changes: 48 additions & 0 deletions build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/bin/sh
set -e

if [ $# -lt 1 ]; then
echo
echo "Usage: $0 VERSION [PLATFORM]"
echo "Build shared libraries for libvips and its dependencies via containers"
echo
echo "Please specify the libvips VERSION, e.g. 8.3.3"
echo
echo "Optionally build for only one PLATFORM, defaults to building for all"
echo "Possible values for PLATFORM are: win32-x64, linux-x64, linux-armv6,"
echo "linux-armv7, linux-armv8"
echo
exit 1
fi
VERSION_VIPS="$1"
PLATFORM="${2:-all}"

# Is docker available?
if ! type docker >/dev/null; then
echo "Please install docker"
exit 1
fi

# Update base images
for baseimage in debian:wheezy debian:jessie debian:stretch socialdefect/raspbian-jessie-core; do
docker pull $baseimage
done

# Windows (x64)
if [ $PLATFORM = "all" ] || [ $PLATFORM = "win32-x64" ]; then
echo "Building win32-x64..."
docker build -t vips-dev-win32-x64 win32-x64
docker run --rm -e "VERSION_VIPS=${VERSION_VIPS}" -v $PWD:/packaging vips-dev-win32-x64 sh -c "/packaging/build/win.sh"
fi

# Linux (x64, ARMv6, ARMv7, ARMv8)
for flavour in linux-x64 linux-armv6 linux-armv7 linux-armv8; do
if [ $PLATFORM = "all" ] || [ $PLATFORM = $flavour ]; then
echo "Building $flavour..."
docker build -t vips-dev-$flavour $flavour
docker run --rm -e "VERSION_VIPS=${VERSION_VIPS}" -v $PWD:/packaging vips-dev-$flavour sh -c "/packaging/build/lin.sh"
fi
done

# Display checksums
sha256sum *.tar.gz
255 changes: 255 additions & 0 deletions build/lin.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
#!/bin/sh
set -e

# Working directories
DEPS=/deps
TARGET=/target
mkdir ${DEPS}
mkdir ${TARGET}

# Common build paths and flags
export PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:${TARGET}/lib/pkgconfig"
export PATH="${PATH}:${TARGET}/bin"
export CPPFLAGS="-I${TARGET}/include"
export LDFLAGS="-L${TARGET}/lib -Wl,-rpath,'\$\$ORIGIN'"
export CFLAGS="${FLAGS}"
export CXXFLAGS="${FLAGS}"

# Dependency version numbers
VERSION_ZLIB=1.2.11
VERSION_FFI=3.2.1
VERSION_GLIB=2.53.5
VERSION_XML2=2.9.4
VERSION_GSF=1.14.41
VERSION_EXIF=0.6.21
VERSION_LCMS2=2.8
VERSION_JPEG=1.5.2
VERSION_PNG16=1.6.32
VERSION_WEBP=0.6.0
VERSION_TIFF=4.0.8
VERSION_ORC=0.4.27
VERSION_GDKPIXBUF=2.36.8
VERSION_FREETYPE=2.8
VERSION_EXPAT=2.2.3
VERSION_FONTCONFIG=2.12.4
VERSION_HARFBUZZ=1.4.8
VERSION_PIXMAN=0.34.0
VERSION_CAIRO=1.14.10
VERSION_PANGO=1.40.10
VERSION_CROCO=0.6.12
VERSION_SVG=2.40.18
VERSION_GIF=5.1.4

# Least out-of-sync Sourceforge mirror
SOURCEFORGE_MIRROR=netix

mkdir ${DEPS}/zlib
curl -Ls http://zlib.net/zlib-${VERSION_ZLIB}.tar.xz | tar xJC ${DEPS}/zlib --strip-components=1
cd ${DEPS}/zlib
./configure --prefix=${TARGET} --uname=linux
make install
rm ${TARGET}/lib/libz.a

mkdir ${DEPS}/ffi
curl -Ls ftp://sourceware.org/pub/libffi/libffi-${VERSION_FFI}.tar.gz | tar xzC ${DEPS}/ffi --strip-components=1
cd ${DEPS}/ffi
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking --disable-builddir
make install-strip

mkdir ${DEPS}/glib
curl -Ls https://download.gnome.org/sources/glib/2.53/glib-${VERSION_GLIB}.tar.xz | tar xJC ${DEPS}/glib --strip-components=1
cd ${DEPS}/glib
echo glib_cv_stack_grows=no >>glib.cache
echo glib_cv_uscore=no >>glib.cache
./configure --cache-file=glib.cache --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking \
--with-pcre=internal --disable-libmount
make install-strip

mkdir ${DEPS}/xml2
curl -Ls http://xmlsoft.org/sources/libxml2-${VERSION_XML2}.tar.gz | tar xzC ${DEPS}/xml2 --strip-components=1
cd ${DEPS}/xml2
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking \
--without-python --without-debug --without-docbook --without-ftp --without-html --without-legacy \
--without-pattern --without-push --without-regexps --without-schemas --without-schematron --with-zlib=${TARGET}
make install-strip

mkdir ${DEPS}/gsf
curl -Ls https://download.gnome.org/sources/libgsf/1.14/libgsf-${VERSION_GSF}.tar.xz | tar xJC ${DEPS}/gsf --strip-components=1
cd ${DEPS}/gsf
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip

mkdir ${DEPS}/exif
curl -Ls http://${SOURCEFORGE_MIRROR}.dl.sourceforge.net/project/libexif/libexif/${VERSION_EXIF}/libexif-${VERSION_EXIF}.tar.bz2 | tar xjC ${DEPS}/exif --strip-components=1
cd ${DEPS}/exif
autoreconf -fiv
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip

mkdir ${DEPS}/lcms2
curl -Ls http://${SOURCEFORGE_MIRROR}.dl.sourceforge.net/project/lcms/lcms/${VERSION_LCMS2}/lcms2-${VERSION_LCMS2}.tar.gz | tar xzC ${DEPS}/lcms2 --strip-components=1
cd ${DEPS}/lcms2
# Apply patches for lcms2 vulnerabilities reported since v2.8
VERSION_LCMS2_GIT_MASTER_SHA=$(curl -Ls https://api.github.com/repos/mm2/Little-CMS/git/refs/heads/master | jq -r '.object.sha' | head -c7)
curl -Ls https://github.com/mm2/Little-CMS/compare/lcms2.8...master.patch | patch -p1 -t || true
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip

mkdir ${DEPS}/jpeg
curl -Ls https://github.com/libjpeg-turbo/libjpeg-turbo/archive/${VERSION_JPEG}.tar.gz | tar xzC ${DEPS}/jpeg --strip-components=1
cd ${DEPS}/jpeg
autoreconf -fiv
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking --with-jpeg8 --without-turbojpeg
make install-strip

mkdir ${DEPS}/png16
curl -Ls https://ftp-osl.osuosl.org/pub/libpng/src/libpng16/libpng-${VERSION_PNG16}.tar.xz | tar xJC ${DEPS}/png16 --strip-components=1
cd ${DEPS}/png16
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip

mkdir ${DEPS}/webp
curl -Ls http://downloads.webmproject.org/releases/webp/libwebp-${VERSION_WEBP}.tar.gz | tar xzC ${DEPS}/webp --strip-components=1
cd ${DEPS}/webp
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking \
--disable-neon --enable-libwebpmux
make install-strip

mkdir ${DEPS}/tiff
VERSION_TIFF_GIT_MASTER_SHA=$(curl -Ls https://api.github.com/repos/vadz/libtiff/git/refs/heads/master | jq -r '.object.sha' | head -c7)
curl -Ls https://github.com/vadz/libtiff/archive/${VERSION_TIFF_GIT_MASTER_SHA}.tar.gz | tar xzC ${DEPS}/tiff --strip-components=1
cd ${DEPS}/tiff
if [ -n "${CHOST}" ]; then autoreconf -fiv; fi
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking --disable-mdi --disable-pixarlog --disable-cxx
make install-strip

mkdir ${DEPS}/orc
curl -Ls http://gstreamer.freedesktop.org/data/src/orc/orc-${VERSION_ORC}.tar.xz | tar xJC ${DEPS}/orc --strip-components=1
cd ${DEPS}/orc
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip
cd ${TARGET}/lib
rm -rf liborc-test-*

mkdir ${DEPS}/gdkpixbuf
curl -Ls https://download.gnome.org/sources/gdk-pixbuf/2.36/gdk-pixbuf-${VERSION_GDKPIXBUF}.tar.xz | tar xJC ${DEPS}/gdkpixbuf --strip-components=1
cd ${DEPS}/gdkpixbuf
touch gdk-pixbuf/loaders.cache
LD_LIBRARY_PATH=${TARGET}/lib \
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking \
--disable-introspection --disable-modules --disable-gio-sniffing \
--without-libtiff --without-gdiplus --with-included-loaders=png,jpeg
make install-strip

mkdir ${DEPS}/freetype
curl -Ls http://${SOURCEFORGE_MIRROR}.dl.sourceforge.net/project/freetype/freetype2/${VERSION_FREETYPE}/freetype-${VERSION_FREETYPE}.tar.gz | tar xzC ${DEPS}/freetype --strip-components=1
cd ${DEPS}/freetype
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static
make install

mkdir ${DEPS}/expat
curl -Ls http://${SOURCEFORGE_MIRROR}.dl.sourceforge.net/project/expat/expat/${VERSION_EXPAT}/expat-${VERSION_EXPAT}.tar.bz2 | tar xjC ${DEPS}/expat --strip-components=1
cd ${DEPS}/expat
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static
make install

mkdir ${DEPS}/fontconfig
curl -Ls https://www.freedesktop.org/software/fontconfig/release/fontconfig-${VERSION_FONTCONFIG}.tar.bz2 | tar xjC ${DEPS}/fontconfig --strip-components=1
cd ${DEPS}/fontconfig
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking \
--with-expat-includes=${TARGET}/include --with-expat-lib=${TARGET}/lib --sysconfdir=/etc
make install-strip

mkdir ${DEPS}/harfbuzz
curl -Ls https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-${VERSION_HARFBUZZ}.tar.bz2 | tar xjC ${DEPS}/harfbuzz --strip-components=1
cd ${DEPS}/harfbuzz
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip

mkdir ${DEPS}/pixman
curl -Ls http://cairographics.org/releases/pixman-${VERSION_PIXMAN}.tar.gz | tar xzC ${DEPS}/pixman --strip-components=1
cd ${DEPS}/pixman
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking --disable-libpng --disable-arm-iwmmxt
make install-strip

mkdir ${DEPS}/cairo
curl -Ls http://cairographics.org/releases/cairo-${VERSION_CAIRO}.tar.xz | tar xJC ${DEPS}/cairo --strip-components=1
cd ${DEPS}/cairo
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking \
--disable-xlib --disable-xcb --disable-quartz --disable-win32 --disable-egl --disable-glx --disable-wgl \
--disable-script --disable-ps --disable-gobject --disable-trace --disable-interpreter
make install-strip

mkdir ${DEPS}/pango
curl -Ls https://download.gnome.org/sources/pango/1.40/pango-${VERSION_PANGO}.tar.xz | tar xJC ${DEPS}/pango --strip-components=1
cd ${DEPS}/pango
./autogen.sh
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip

mkdir ${DEPS}/croco
curl -Ls https://download.gnome.org/sources/libcroco/0.6/libcroco-${VERSION_CROCO}.tar.xz | tar xJC ${DEPS}/croco --strip-components=1
cd ${DEPS}/croco
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip

mkdir ${DEPS}/svg
curl -Ls https://download.gnome.org/sources/librsvg/2.40/librsvg-${VERSION_SVG}.tar.xz | tar xJC ${DEPS}/svg --strip-components=1
cd ${DEPS}/svg
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking --disable-introspection --disable-tools --disable-pixbuf-loader
make install-strip

mkdir ${DEPS}/gif
curl -Ls http://${SOURCEFORGE_MIRROR}.dl.sourceforge.net/project/giflib/giflib-${VERSION_GIF}.tar.gz | tar xzC ${DEPS}/gif --strip-components=1
cd ${DEPS}/gif
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking
make install-strip

mkdir ${DEPS}/vips
curl -Ls https://github.com/jcupitt/libvips/releases/download/v${VERSION_VIPS}-alpha4/vips-${VERSION_VIPS}-3.tar.gz | tar xzC ${DEPS}/vips --strip-components=1
cd ${DEPS}/vips
./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking \
--disable-debug --disable-introspection --without-python --without-fftw \
--without-magick --without-pangoft2 --without-ppm --without-analyze --without-radiance \
--with-zip-includes=${TARGET}/include --with-zip-libraries=${TARGET}/lib \
--with-jpeg-includes=${TARGET}/include --with-jpeg-libraries=${TARGET}/lib
make install-strip

# Remove the old C++ bindings
cd ${TARGET}/include
rm -rf vips/vipsc++.h vips/vipscpp.h
cd ${TARGET}/lib
rm -rf pkgconfig .libs *.la libvipsCC*

# Create JSON file of version numbers
cd ${TARGET}
echo "{\n\
\"cairo\": \"${VERSION_CAIRO}\",\n\
\"croco\": \"${VERSION_CROCO}\",\n\
\"exif\": \"${VERSION_EXIF}\",\n\
\"expat\": \"${VERSION_EXPAT}\",\n\
\"ffi\": \"${VERSION_FFI}\",\n\
\"fontconfig\": \"${VERSION_FONTCONFIG}\",\n\
\"freetype\": \"${VERSION_FREETYPE}\",\n\
\"gdkpixbuf\": \"${VERSION_GDKPIXBUF}\",\n\
\"gif\": \"${VERSION_GIF}\",\n\
\"glib\": \"${VERSION_GLIB}\",\n\
\"gsf\": \"${VERSION_GSF}\",\n\
\"harfbuzz\": \"${VERSION_HARFBUZZ}\",\n\
\"jpeg\": \"${VERSION_JPEG}\",\n\
\"lcms\": \"${VERSION_LCMS2}-${VERSION_LCMS2_GIT_MASTER_SHA}\",\n\
\"orc\": \"${VERSION_ORC}\",\n\
\"pango\": \"${VERSION_PANGO}\",\n\
\"pixman\": \"${VERSION_PIXMAN}\",\n\
\"png\": \"${VERSION_PNG16}\",\n\
\"svg\": \"${VERSION_SVG}\",\n\
\"tiff\": \"${VERSION_TIFF}-${VERSION_TIFF_GIT_MASTER_SHA}\",\n\
\"vips\": \"${VERSION_VIPS}\",\n\
\"webp\": \"${VERSION_WEBP}\",\n\
\"xml\": \"${VERSION_XML2}\",\n\
\"zlib\": \"${VERSION_ZLIB}\"\n\
}" >lib/versions.json

# Create .tar.gz
tar czf /packaging/libvips-${VERSION_VIPS}-${PLATFORM}.tar.gz include lib
advdef --recompress --shrink-insane /packaging/libvips-${VERSION_VIPS}-${PLATFORM}.tar.gz
22 changes: 22 additions & 0 deletions build/win.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/sh
set -e

VERSION_VIPS_MAJOR=$(echo $VERSION_VIPS | cut -d. -f1)
VERSION_VIPS_MINOR=$(echo $VERSION_VIPS | cut -d. -f2)

# Fetch and unzip
mkdir /vips
cd /vips
curl -L -O https://github.com/lovell/build-win64/releases/download/v${VERSION_VIPS}/vips-dev-w64-web-${VERSION_VIPS}.zip
unzip vips-dev-w64-web-${VERSION_VIPS}.zip

# Clean and zip
cd /vips/vips-dev-${VERSION_VIPS_MAJOR}.${VERSION_VIPS_MINOR}
rm bin/libvipsCC-42.dll bin/libvips-cpp-42.dll bin/libgsf-win32-1-114.dll bin/libgettext*.dll
cp bin/*.dll lib/
cp -r lib64/* lib/

echo "Creating tarball"
tar czf /packaging/libvips-${VERSION_VIPS}-${PLATFORM}.tar.gz include lib/glib-2.0 lib/libvips.lib lib/libglib-2.0.lib lib/libgobject-2.0.lib lib/*.dll
echo "Shrinking tarball"
advdef --recompress --shrink-insane /packaging/libvips-${VERSION_VIPS}-${PLATFORM}.tar.gz
Loading

0 comments on commit 9ccb759

Please sign in to comment.