Skip to content

Commit

Permalink
Compile zstdec and xzdec as es6 modules (#998)
Browse files Browse the repository at this point in the history
Required for #554
  • Loading branch information
Jaifroid authored May 4, 2023
1 parent 88057a6 commit 2b4ff6b
Show file tree
Hide file tree
Showing 14 changed files with 124 additions and 136 deletions.
9 changes: 9 additions & 0 deletions emscripten/xzdec/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Source code and compilations of zstandard decompression library

This folder shows how to compile the public-domain XZ decoder to make a webassembly binary and an asm JavaScript version.

The source code can be compiled with Emscripten by setting up the Emscripten SDK and running the Linux script `./compile.sh` in this directory. This will produce `xzddec.js`. Further optimization and minification of this file may be possible by adjusting the commandline options in `compile.sh`.

You may be able to compile easily using docker. Look in the `/scripts` directory for helper scripts for your platform.

If you need these builds to support ES6 modules (i.e. `import` and `export` module syntax), you can add `-s EXPORT_ES6=1` to the compile commands in `compile.sh`.
4 changes: 2 additions & 2 deletions emscripten/xzdec/compile.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
echo "Compiling ASM version xzdec-asm.js"
emcc --memory-init-file 0 -O3 --closure 1 -s ENVIRONMENT="web" -s WASM=0 -s MALLOC="emmalloc" -s TOTAL_MEMORY=83886080 -s FILESYSTEM=0 -s DOUBLE_MODE=0 -s DYNAMIC_EXECUTION=0 -s MIN_IE_VERSION=11 -s EXPORT_NAME="XZ" -s MODULARIZE=1 -s AGGRESSIVE_VARIABLE_ELIMINATION=1 -s EXPORTED_FUNCTIONS="['_init', '_init_decompression', '_input_empty', '_get_in_buffer', '_set_new_input', '_decompress', '_get_out_pos', '_get_out_buffer', '_out_buffer_cleared', '_release']" -DXZ_USE_CRC64=1 -DXZ_INTERNAL_CRC64=1 ./*.c -o xzdec-asm.js
emcc --memory-init-file 0 -O3 --closure 1 -s ENVIRONMENT="web" -s WASM=0 -s MALLOC="emmalloc" -s TOTAL_MEMORY=83886080 -s FILESYSTEM=0 -s DOUBLE_MODE=0 -s DYNAMIC_EXECUTION=0 -s MIN_IE_VERSION=11 -s EXPORT_NAME="XZ" -s MODULARIZE=1 -s EXPORT_ES6=1 -s AGGRESSIVE_VARIABLE_ELIMINATION=1 -s EXPORTED_FUNCTIONS="['_init', '_init_decompression', '_input_empty', '_get_in_buffer', '_set_new_input', '_decompress', '_get_out_pos', '_get_out_buffer', '_out_buffer_cleared', '_release']" -DXZ_USE_CRC64=1 -DXZ_INTERNAL_CRC64=1 ./*.c -o xzdec-asm.js
echo "Compiling WASM version xzdec-wasm.js"
emcc --memory-init-file 0 -O3 --closure 1 -s ENVIRONMENT="web" -s WASM=1 -s MALLOC="emmalloc" -s TOTAL_MEMORY=83886080 -s FILESYSTEM=0 -s DOUBLE_MODE=0 -s DYNAMIC_EXECUTION=0 -s EXPORT_NAME="XZ" -s MODULARIZE=1 -s AGGRESSIVE_VARIABLE_ELIMINATION=1 -s EXPORTED_FUNCTIONS="['_init', '_init_decompression', '_input_empty', '_get_in_buffer', '_set_new_input', '_decompress', '_get_out_pos', '_get_out_buffer', '_out_buffer_cleared', '_release']" -DXZ_USE_CRC64=1 -DXZ_INTERNAL_CRC64=1 ./*.c -o xzdec-wasm.js
emcc --memory-init-file 0 -O3 --closure 1 -s ENVIRONMENT="web" -s WASM=1 -s MALLOC="emmalloc" -s TOTAL_MEMORY=83886080 -s FILESYSTEM=0 -s DOUBLE_MODE=0 -s DYNAMIC_EXECUTION=0 -s EXPORT_NAME="XZ" -s MODULARIZE=1 -s EXPORT_ES6=1 -s AGGRESSIVE_VARIABLE_ELIMINATION=1 -s EXPORTED_FUNCTIONS="['_init', '_init_decompression', '_input_empty', '_get_in_buffer', '_set_new_input', '_decompress', '_get_out_pos', '_get_out_buffer', '_out_buffer_cleared', '_release']" -DXZ_USE_CRC64=1 -DXZ_INTERNAL_CRC64=1 ./*.c -o xzdec-wasm.js
echo "Finished."
72 changes: 33 additions & 39 deletions emscripten/xzdec/xzdec-asm.js

Large diffs are not rendered by default.

38 changes: 16 additions & 22 deletions emscripten/xzdec/xzdec-wasm.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file modified emscripten/xzdec/xzdec-wasm.wasm
Binary file not shown.
4 changes: 2 additions & 2 deletions emscripten/zstandard/compile.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
echo "Compiling ASM version zstddec-asm.js"
emcc --memory-init-file 0 -O3 --closure 1 -s ENVIRONMENT="web" -s WASM=0 -s MALLOC="emmalloc" -s TOTAL_MEMORY=150994944 -s FILESYSTEM=0 -s DOUBLE_MODE=0 -s DYNAMIC_EXECUTION=0 -s MIN_IE_VERSION=11 -s EXPORT_NAME="ZD" -s MODULARIZE=1 -s EXPORTED_FUNCTIONS="['_malloc', '_free', '_ZSTD_createDStream', '_ZSTD_initDStream', '_ZSTD_decompressStream', '_ZSTD_isError', '_ZSTD_getErrorName', '_ZSTD_freeDStream', '_ZSTD_DStreamInSize', '_ZSTD_DStreamOutSize']" -s EXPORTED_RUNTIME_METHODS="['cwrap']" ./*.c -o zstddec-asm.js
emcc --memory-init-file 0 -O3 --closure 1 -s ENVIRONMENT="web" -s WASM=0 -s MALLOC="emmalloc" -s TOTAL_MEMORY=150994944 -s FILESYSTEM=0 -s DOUBLE_MODE=0 -s DYNAMIC_EXECUTION=0 -s MIN_IE_VERSION=11 -s EXPORT_NAME="ZD" -s MODULARIZE=1 -s EXPORT_ES6=1 -s EXPORTED_FUNCTIONS="['_malloc', '_free', '_ZSTD_createDStream', '_ZSTD_initDStream', '_ZSTD_decompressStream', '_ZSTD_isError', '_ZSTD_getErrorName', '_ZSTD_freeDStream', '_ZSTD_DStreamInSize', '_ZSTD_DStreamOutSize']" -s EXPORTED_RUNTIME_METHODS="['cwrap']" ./*.c -o zstddec-asm.js
echo "Compiling WASM version zstddec-wasm.js"
emcc --memory-init-file 0 -O3 --closure 1 -s ENVIRONMENT="web" -s WASM=1 -s MALLOC="emmalloc" -s TOTAL_MEMORY=150994944 -s FILESYSTEM=0 -s DOUBLE_MODE=0 -s DYNAMIC_EXECUTION=0 -s EXPORT_NAME="ZD" -s MODULARIZE=1 -s EXPORTED_FUNCTIONS="['_malloc', '_free', '_ZSTD_createDStream', '_ZSTD_initDStream', '_ZSTD_decompressStream', '_ZSTD_isError', '_ZSTD_getErrorName', '_ZSTD_freeDStream', '_ZSTD_DStreamInSize', '_ZSTD_DStreamOutSize']" -s EXPORTED_RUNTIME_METHODS="['cwrap']" ./*.c -o zstddec-wasm.js
emcc --memory-init-file 0 -O3 --closure 1 -s ENVIRONMENT="web" -s WASM=1 -s MALLOC="emmalloc" -s TOTAL_MEMORY=150994944 -s FILESYSTEM=0 -s DOUBLE_MODE=0 -s DYNAMIC_EXECUTION=0 -s EXPORT_NAME="ZD" -s MODULARIZE=1 -s EXPORT_ES6=1 -s EXPORTED_FUNCTIONS="['_malloc', '_free', '_ZSTD_createDStream', '_ZSTD_initDStream', '_ZSTD_decompressStream', '_ZSTD_isError', '_ZSTD_getErrorName', '_ZSTD_freeDStream', '_ZSTD_DStreamInSize', '_ZSTD_DStreamOutSize']" -s EXPORTED_RUNTIME_METHODS="['cwrap']" ./*.c -o zstddec-wasm.js
echo "Finished."
2 changes: 2 additions & 0 deletions emscripten/zstandard/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ It has been combined by following the instructions at https://github.com/faceboo

The source code can be compiled with Emscripten by setting up the Emscripten SDK and running the Linux script `./compile.sh` in this directory. This will produce `zstddec.js`. Further optimization and minification of this file may be possible by adjusting the commandline options in `compile.sh`.

If you need these builds to support ES6 modules (i.e. `import` and `export` module syntax), you can add `-s EXPORT_ES6=1` to the compile commands in `compile.sh`.

You may be able to compile easily using docker. Look in the `/scripts` directory for helper scripts for your platform.

For a fuller Emscripten distribution of the full zstandard library for `node.js`, see https://github.com/yoshihitoh/zstd-codec/tree/develop/js/example.
Loading

0 comments on commit 2b4ff6b

Please sign in to comment.