diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 838d1ecf..df66146c 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.11.1","generation_timestamp":"2024-10-17T23:32:27","documenter_version":"1.7.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.11.1","generation_timestamp":"2024-10-20T05:16:46","documenter_version":"1.7.0"}} \ No newline at end of file diff --git a/dev/apps.html b/dev/apps.html index 609aed66..889723b3 100644 --- a/dev/apps.html +++ b/dev/apps.html @@ -81,4 +81,4 @@ │ %8 = Base.PROGRAM_FILE │ value@_16 = %8 │ %10 = Base.repr(%8) -...

Preferences in <app_dir>/share/julia

As described above, a TOML file with all preferences active during the compilation process will be stored with the app bundle. If your preferences may contain confidential information, you can either delete the <app_dir>/share/julia/LocalPreferences.toml file before distributing the app bundle, or suppress the preference file generation by passing include_preferences=false to create_app. Note, however, that if the preference file is not present, any preference loaded in your app at runtime will use their default value (or crash, if no default is provided).

+...

Preferences in <app_dir>/share/julia

As described above, a TOML file with all preferences active during the compilation process will be stored with the app bundle. If your preferences may contain confidential information, you can either delete the <app_dir>/share/julia/LocalPreferences.toml file before distributing the app bundle, or suppress the preference file generation by passing include_preferences=false to create_app. Note, however, that if the preference file is not present, any preference loaded in your app at runtime will use their default value (or crash, if no default is provided).

diff --git a/dev/devdocs/binaries_part_2.html b/dev/devdocs/binaries_part_2.html index d04460a4..6c053296 100644 --- a/dev/devdocs/binaries_part_2.html +++ b/dev/devdocs/binaries_part_2.html @@ -116,4 +116,4 @@ Stacktrace: [1] error(::String) at ./error.jl:33 [2] real_main() at /home/kc/MyApp/MyApp.jl:21 - [3] julia_main() at /home/kc/MyApp/MyApp.jl:7

macOS considerations

On macOS, instead of $ORIGIN for the rpath, use @executable_path.

Windows considerations

On Windows, it is recommended to increase the size of the stack from the default 1 MB to 8MB which can be done by passing the -Wl,--stack,8388608 flag. Windows doesn't have (at least in an as simple way as Linux and macOS) the concept of rpath. The goto solution is to either set the PATH environment variable to the Julia bin folder or alternatively copy paste all the libraries in the Julia bin folder so they sit next to the executable.

+ [3] julia_main() at /home/kc/MyApp/MyApp.jl:7

macOS considerations

On macOS, instead of $ORIGIN for the rpath, use @executable_path.

Windows considerations

On Windows, it is recommended to increase the size of the stack from the default 1 MB to 8MB which can be done by passing the -Wl,--stack,8388608 flag. Windows doesn't have (at least in an as simple way as Linux and macOS) the concept of rpath. The goto solution is to either set the PATH environment variable to the Julia bin folder or alternatively copy paste all the libraries in the Julia bin folder so they sit next to the executable.

diff --git a/dev/devdocs/intro.html b/dev/devdocs/intro.html index 78d05e7d..fda96783 100644 --- a/dev/devdocs/intro.html +++ b/dev/devdocs/intro.html @@ -1,2 +1,2 @@ -Introduction · PackageCompiler
GitHub

Introduction

Info

This section is for people who want to understand PackageCompiler.jl under the hood. It is not required reading to use the package.

This part of the documentation contains a set of tutorials aimed to teach how PackageCompiler works internally. This is done by going through some examples of manually creating sysimages and apps, mostly from the command line. By knowing the internals of PackageCompiler you can more easily figure out root causes of problems and help others. The inner functionality of PackageCompiler is actually quite simple. There are a few julia commands and compiler invocations that everything is built around, the rest is mostly scaffolding.

Part 1 focuses on how to build a local system image to reduce package load times and reduce the latency that can occur when calling a function for the first time. Part 2 targets how to build an executable based on the custom sysimage so that it can be run without having to explicitly start a Julia session. Part 3 details how to bundle that executable together with the Julia libraries and other files needed so that the bundle can be sent to and run on a different system where Julia might not be installed. These functionalities are exposed from PackageCompiler as create_sysimage and create_app.

It should be noted that there is some usage of non-documented Julia functions and flags. They have not been changed for quite a long time (and are unlikely to change too much in the future), but some care should be taken.

+Introduction · PackageCompiler
GitHub

Introduction

Info

This section is for people who want to understand PackageCompiler.jl under the hood. It is not required reading to use the package.

This part of the documentation contains a set of tutorials aimed to teach how PackageCompiler works internally. This is done by going through some examples of manually creating sysimages and apps, mostly from the command line. By knowing the internals of PackageCompiler you can more easily figure out root causes of problems and help others. The inner functionality of PackageCompiler is actually quite simple. There are a few julia commands and compiler invocations that everything is built around, the rest is mostly scaffolding.

Part 1 focuses on how to build a local system image to reduce package load times and reduce the latency that can occur when calling a function for the first time. Part 2 targets how to build an executable based on the custom sysimage so that it can be run without having to explicitly start a Julia session. Part 3 details how to bundle that executable together with the Julia libraries and other files needed so that the bundle can be sent to and run on a different system where Julia might not be installed. These functionalities are exposed from PackageCompiler as create_sysimage and create_app.

It should be noted that there is some usage of non-documented Julia functions and flags. They have not been changed for quite a long time (and are unlikely to change too much in the future), but some care should be taken.

diff --git a/dev/devdocs/relocatable_part_3.html b/dev/devdocs/relocatable_part_3.html index 8d40fe01..ee4682a3 100644 --- a/dev/devdocs/relocatable_part_3.html +++ b/dev/devdocs/relocatable_part_3.html @@ -119,4 +119,4 @@ function __init__() libfoo = Libdl.dlopen(LIBFOO_PATH) -end

The problem here is that deps.jl contains an absolute path to the library and this gets encoded into the source code of the package. If we would store the package in the sysimage and try use it on another system, it would error when initialized since the LIBFOO_PATH variable is not valid on the other system. However, sometimes we need to bundle libraries and data files since the package uses them. Fortunately, there is a plan for that which can be seen in the blog post about artifacts.

The idea is that with the new artifact system a file (Artifacts.toml), a package can declaratively list external libraries and files that it needs. In addition, the artifact system provides a way to find these files at runtime in a deterministic way. It is then possible to make sure that all artifacts needed for the package is bundled in the app and can also be found by the package during runtime.

The details are left out here since they become a bit technical but it should give some incentive to switch to the artifact system.

+end

The problem here is that deps.jl contains an absolute path to the library and this gets encoded into the source code of the package. If we would store the package in the sysimage and try use it on another system, it would error when initialized since the LIBFOO_PATH variable is not valid on the other system. However, sometimes we need to bundle libraries and data files since the package uses them. Fortunately, there is a plan for that which can be seen in the blog post about artifacts.

The idea is that with the new artifact system a file (Artifacts.toml), a package can declaratively list external libraries and files that it needs. In addition, the artifact system provides a way to find these files at runtime in a deterministic way. It is then possible to make sure that all artifacts needed for the package is bundled in the app and can also be found by the package during runtime.

The details are left out here since they become a bit technical but it should give some incentive to switch to the artifact system.

diff --git a/dev/devdocs/sysimages_part_1.html b/dev/devdocs/sysimages_part_1.html index 92399eb3..9536a6aa 100644 --- a/dev/devdocs/sysimages_part_1.html +++ b/dev/devdocs/sysimages_part_1.html @@ -92,4 +92,4 @@ 0.031504 seconds (441 allocations: 37.383 KiB) julia> @time CSV.read("FL_insurance_sample.csv"); - 0.021355 seconds (423 allocations: 34.695 KiB)

And finally, our first time for parsing the CSV-file is close to the second time.

+ 0.021355 seconds (423 allocations: 34.695 KiB)

And finally, our first time for parsing the CSV-file is close to the second time.

diff --git a/dev/examples/ohmyrepl.html b/dev/examples/ohmyrepl.html index 8cd3178f..e5c49d56 100644 --- a/dev/examples/ohmyrepl.html +++ b/dev/examples/ohmyrepl.html @@ -1,4 +1,4 @@ Creating a sysimage with OhMyREPL · PackageCompiler
GitHub

Creating a sysimage with OhMyREPL

OhMyREPL.jl is a package that enhances the REPL with, for example, syntax highlighting. It does, however, come with a bit of a startup time increase, so compiling a new system image with OhMyREPL included is useful. Importing the OhMyREPL package is not the only factor that contributes to the extra load time from using OhMyREPL. In addition, the time of compiling functions that OhMyREPL uses is also a factor. Therefore, we also want to do "Profile Guided Compilation" (PGC), where we record what functions get compiled when using OhMyREPL, so they can be cached into the system image. OhMyREPL is a bit different from most other packages in that it is used interactively. Normally to do PGC with PackageCompiler we pass a script to execute as the precompile_execution_file which is used to collect compilation data, but in this case, we will use Julia to manually collect this data.

First, install OhMyREPL in the global environment using import Pkg; Pkg.add("OhMyREPL"). Run using OhMyREPL and write something (like 1+1). It should be syntax highlighted, but you might have noticed that there was a bit of a delay before the characters appeared. This is the extra latency from using the package that we want to get rid of.

OhMyREPL installation

The first goal is to have Julia emit the functions it compiles when running OhMyREPL. To this end, start Julia with the --trace-compile=ohmyrepl_precompile.jl flag. This will start a standard Julia session but all functions that get compiled are output to the file ohmyrepl_precompile.jl. In the Julia session, load OhMyREPL, use the REPL a bit so that the functionality of OhMyREPL is exercised. Quit Julia and look into the file ohmyrepl_precompile.jl. It should be filled with lines like:

precompile(Tuple{typeof(OhMyREPL.Prompt.insert_keybindings), Any})
 precompile(Tuple{typeof(OhMyREPL.__init__)})

These are functions that Julia compiled. We now just tell create_sysimage to use these precompile statements when creating the system image:

PackageCompiler.create_sysimage(["OhMyREPL"]; sysimage_path="OMR-sysimage.so", 
-                                precompile_statements_file="ohmyrepl_precompile.jl")

Exit julia and restart it with the --sysimage=OMR-sysimage.so command-line argument and start typing something. If everything went well, you should see the typed text become highlighted with a significantly smaller delay than with the default system image.

+ precompile_statements_file="ohmyrepl_precompile.jl")

Exit julia and restart it with the --sysimage=OMR-sysimage.so command-line argument and start typing something. If everything went well, you should see the typed text become highlighted with a significantly smaller delay than with the default system image.

diff --git a/dev/examples/plots.html b/dev/examples/plots.html index d1282f5f..3bca0598 100644 --- a/dev/examples/plots.html +++ b/dev/examples/plots.html @@ -22,4 +22,4 @@ # Custom sysimage ➜ time julia --sysimage sys_plots.so -e '' -julia --sysimage sys_plots.so -e '' 0.43s user 0.30s system 347% cpu 0.211 total +julia --sysimage sys_plots.so -e '' 0.43s user 0.30s system 347% cpu 0.211 total diff --git a/dev/index.html b/dev/index.html index 9638fe72..5fd07890 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,3 +1,3 @@ Home · PackageCompiler
GitHub

PackageCompiler

Julia is, in general, a "just-barely-ahead-of-time" compiled language. When you call a function for the first time, Julia compiles it for precisely the types of the arguments given. This can take some time. All subsequent calls within that same session use this fast compiled function, but if you restart Julia you lose all the compiled work. PackageCompiler allows you to do this work upfront — further ahead of time — and store the results for a lower latency startup.

There are three main workflows:

  1. You can save loaded packages and compiled functions into a file (called a sysimage) that you pass to julia upon startup. Typically the goal is to reduce latency on your machine; for example, you could load the packages and compile the functions used in common plotting workflows use that saved image by default. In general, sysimages are not relocatable to other machines; they'll only work on the machine they were created on.

  2. You can further compile an entire project into a relocatable "app". This generates a bundle of files (including the executable) that can be sent and run on other machines that might not even have Julia installed. Not only does this aim to do as much compilation up-front, but it also bundles together all dependencies (including potentially cross-platform binary libraries) and Julia itself with a single executable as its entry point.

  3. Alternatively, you can create a C library. In this case, your package should define C-callable functions to be included in the sysimage. As with apps, generating a library bundles together Julia and all dependencies in a (hopefully) redistributable directory structure that can be moved to other machines (of the same architecture).

The most challenging part in all cases is in determining which methods need to be compiled ahead of time. For example, your project will certainly require addition between integers (+(::Int, ::Int); thankfully that's already compiled into Julia itself), but does it require addition of dates (like +(::Date, ::Day) or +(::DateTime, ::Hour))? What about high-precision complex numbers (+(::Complex{BigFloat}, ::Complex{BigFloat}))? It's completely intractable to compile all possible combinations, so instead PackageCompiler relies upon "tracing" an exemplar session and recording which methods were used. Any methods that were missed will still be compiled as they are needed (by default).

Note that to use PackageCompiler.jl effectively some knowledge on how packages and "environments" work is required. If you are just starting with Julia, it is unlikely that you would want to use PackageCompiler.jl

The manual contains some uses of Linux commands like ls (dir in Windows) and cat but hopefully these commands are common enough that the points still come across.

Installation instructions

The package is installed using the standard way with the package manager:

using Pkg
-Pkg.add("PackageCompiler")
Note

It is strongly recommended to use the official binaries that are downloaded from https://julialang.org/downloads/. Distribution-provided Julia installations are unlikely to work properly with this package.

To use PackageCompiler a C-compiler needs to be available:

macOS, Linux

Having a decently modern gcc or clang available should be enough to use PackageCompiler on Linux or macOS. For macOS, this can be the built-in Xcode command line tools or homebrew and for Linux, using the system package manager to get a compiler should work fine.

Windows

A suitable compiler will be automatically installed the first time it is needed.

Upgrading from PackageCompiler 1.0.

PackageCompiler 2.0 comes with a few breaking changes.

  • The functionality for replacing the default sysimage (replace_default=true) has been removed. Instead, you can e.g. create an alias or shortcut that starts Julia with a custom sysimage by specifying the --sysimage=<PATH/TO/SYSIMAGE> command line option.
  • Lazy artifacts (those not downloaded until used) are not included in apps by default anymore. Use include_lazy_artifacts=true to re-enable this.
  • Passing no packages to create_sysimage will now include all packages in the given project instead of a sysimage with no packages. Use String[] as a first argument if you want the old behavior.
  • The audit_app function has been removed. It caught too few problems to be useful in practice.
  • The keyword app_name in create_app has been removed and replaced with a more flexible version. If you used app_name="Foo", replace it with executables=["Foo"=>"julia_main"].
  • The @ccallable in front of the entry point functions of apps should be removed. Failure to do so might lead to strange errors during creation of the app.
+Pkg.add("PackageCompiler")
Note

It is strongly recommended to use the official binaries that are downloaded from https://julialang.org/downloads/. Distribution-provided Julia installations are unlikely to work properly with this package.

To use PackageCompiler a C-compiler needs to be available:

macOS, Linux

Having a decently modern gcc or clang available should be enough to use PackageCompiler on Linux or macOS. For macOS, this can be the built-in Xcode command line tools or homebrew and for Linux, using the system package manager to get a compiler should work fine.

Windows

A suitable compiler will be automatically installed the first time it is needed.

Upgrading from PackageCompiler 1.0.

PackageCompiler 2.0 comes with a few breaking changes.

diff --git a/dev/libs.html b/dev/libs.html index 65bc4001..d0f0d739 100644 --- a/dev/libs.html +++ b/dev/libs.html @@ -31,4 +31,4 @@ libjulia.1.6.dylib libjulia.1.dylib libjulia.dylib -...

(These will have a .so extension on Linux, and a .dll extension on Windows. There may also be other files in the same directory, depending on your operating system and version of Julia.)

In addition to most of the same keyword arguments as create_app, create_library has additional keyword arguments related to library naming, versioning, and including C header files in the output library bundle. See the function documentation for details.

Presumably, you're creating the library to use some functionality that is available in Julia but not (easily) implementable in some other language, like C or C++. To use this functionality from, e.g., C, you'll need to link against the library, and also make it accessible at run time (because it's a dynamic library, not a static one).

Here you have different options depending on your operating system and needs.

  1. Install the libraries in a non-standard location, and update an appropriate environment variable to point to the library location.

  2. (Linux/Unix/Mac) Install the library files in a standard library location. /usr/local/ is one possible location:

    Note that on Linux, installing under /usr/local/lib or another standard location requires that you run ldconfig as root after install.

  3. (Mac) Include the full library bundle in an application bundle and set the rpath on the application bundle to the relative path of the library from the executable.

  4. (Windows) Include all libraries in the same directory as an executable.

In all cases, you also need to link to the library while building your executable. For C/C++ compilers, the link step looks something like this:

cc -o my_application my_application.o -L/path/to/my_library -lmylib

Note that on Unix-like operating systems (including Mac), your library must have a lib prefix (e.g., libmylib.so (linux/unix) or libmylib.dylib (Mac)). create_library() ensures this. (On windows, the lib prefix is optional.)

See here for a more complete example of how this might look.

Preferences

Compile-time preferences used by any of the packages included in the library will be stored in the sysimage. To support runtime preferences, all preferences that the library "sees" during the compilation process are stored in the library bundle under <dest_dir>/share/julia/LocalPreferences.toml. Note that preferences loaded at compile time are not affected by the values in the LocalPreferences.toml, but modifying the file will change the value of preferences loaded at runtime.

To learn more about compile time preferences and runtime preferences, please refer to the Preferences.jl docs.

+...

(These will have a .so extension on Linux, and a .dll extension on Windows. There may also be other files in the same directory, depending on your operating system and version of Julia.)

In addition to most of the same keyword arguments as create_app, create_library has additional keyword arguments related to library naming, versioning, and including C header files in the output library bundle. See the function documentation for details.

Presumably, you're creating the library to use some functionality that is available in Julia but not (easily) implementable in some other language, like C or C++. To use this functionality from, e.g., C, you'll need to link against the library, and also make it accessible at run time (because it's a dynamic library, not a static one).

Here you have different options depending on your operating system and needs.

  1. Install the libraries in a non-standard location, and update an appropriate environment variable to point to the library location.

  2. (Linux/Unix/Mac) Install the library files in a standard library location. /usr/local/ is one possible location:

    Note that on Linux, installing under /usr/local/lib or another standard location requires that you run ldconfig as root after install.

  3. (Mac) Include the full library bundle in an application bundle and set the rpath on the application bundle to the relative path of the library from the executable.

  4. (Windows) Include all libraries in the same directory as an executable.

In all cases, you also need to link to the library while building your executable. For C/C++ compilers, the link step looks something like this:

cc -o my_application my_application.o -L/path/to/my_library -lmylib

Note that on Unix-like operating systems (including Mac), your library must have a lib prefix (e.g., libmylib.so (linux/unix) or libmylib.dylib (Mac)). create_library() ensures this. (On windows, the lib prefix is optional.)

See here for a more complete example of how this might look.

Preferences

Compile-time preferences used by any of the packages included in the library will be stored in the sysimage. To support runtime preferences, all preferences that the library "sees" during the compilation process are stored in the library bundle under <dest_dir>/share/julia/LocalPreferences.toml. Note that preferences loaded at compile time are not affected by the values in the LocalPreferences.toml, but modifying the file will change the value of preferences loaded at runtime.

To learn more about compile time preferences and runtime preferences, please refer to the Preferences.jl docs.

diff --git a/dev/refs.html b/dev/refs.html index c38e0d76..ddea7864 100644 --- a/dev/refs.html +++ b/dev/refs.html @@ -1,8 +1,8 @@ -References · PackageCompiler
GitHub

References

PackageCompiler.create_sysimage — Function
create_sysimage(packages::Vector{String}; kwargs...)

Create a system image that includes the package(s) in packages (given as a string or vector). If the packages argument is not passed, all packages in the project will be put into the sysimage.

An attempt to automatically find a compiler will be done but can also be given explicitly by setting the environment variable JULIA_CC to a path to a compiler (can also include extra arguments to the compiler, like -g).

Keyword arguments:

  • sysimage_path::String: The path to where the resulting sysimage should be saved.

  • project::String: The project directory that should be active when the sysimage is created, defaults to the currently active project.

  • precompile_execution_file::Union{String, Vector{String}}: A file or list of files that contain code from which precompilation statements should be recorded.

  • precompile_statements_file::Union{String, Vector{String}}: A file or list of files that contain precompilation statements that should be included in the sysimage.

  • incremental::Bool: If true, build the new sysimage on top of the sysimage of the current process otherwise build a new sysimage from scratch. Defaults to true.

  • filter_stdlibs::Bool: If true, only include stdlibs that are in the project file. Defaults to false, only set to true if you know the potential pitfalls.

  • include_transitive_dependencies::Bool: If true, explicitly put all transitive dependencies into the sysimage. This only makes a difference if some packages do not load all their dependencies when themselves are loaded. Defaults to true.

  • import_into_main::Bool: If true, import all packages from packages into Main. This allows calling using .Package without the Project.toml the sysimage was built with.

Advanced keyword arguments

  • base_sysimage::Union{Nothing, String}: If a String, names an existing sysimage upon which to build the new sysimage incrementally, instead of the sysimage of the current process. Defaults to nothing. Keyword argument incremental must be true if base_sysimage is not nothing.

  • cpu_target::String: The value to use for JULIA_CPU_TARGET when building the system image. Defaults to native.

  • script::String: Path to a file that gets executed in the --output-o process.

  • sysimage_build_args::Cmd: A set of command line options that is used in the Julia process building the sysimage, for example -O1 --check-bounds=yes.

source
PackageCompiler.create_app — Function
create_app(package_dir::String, compiled_app::String; kwargs...)

Compile an app with the source in package_dir to the folder compiled_app. The folder package_dir needs to contain a package where the package includes a function with the signature

julia_main()::Cint
+References · PackageCompiler
GitHub
References
PackageCompiler.create_sysimage — Function
create_sysimage(packages::Vector{String}; kwargs...)
Create a system image that includes the package(s) in packages (given as a string or vector). If the packages argument is not passed, all packages in the project will be put into the sysimage.
An attempt to automatically find a compiler will be done but can also be given explicitly by setting the environment variable JULIA_CC to a path to a compiler (can also include extra arguments to the compiler, like -g).
Keyword arguments:
  • sysimage_path::String: The path to where the resulting sysimage should be saved.
  • project::String: The project directory that should be active when the sysimage is created, defaults to the currently active project.
  • precompile_execution_file::Union{String, Vector{String}}: A file or list of files that contain code from which precompilation statements should be recorded.
  • precompile_statements_file::Union{String, Vector{String}}: A file or list of files that contain precompilation statements that should be included in the sysimage.
  • incremental::Bool: If true, build the new sysimage on top of the sysimage of the current process otherwise build a new sysimage from scratch. Defaults to true.
  • filter_stdlibs::Bool: If true, only include stdlibs that are in the project file. Defaults to false, only set to true if you know the potential pitfalls.
  • include_transitive_dependencies::Bool: If true, explicitly put all  transitive dependencies into the sysimage. This only makes a difference if some  packages do not load all their dependencies when themselves are loaded. Defaults to true.
  • import_into_main::Bool: If true, import all packages from packages into Main. This allows calling using .Package without the Project.toml the sysimage was built with.
Advanced keyword arguments
  • base_sysimage::Union{Nothing, String}: If a String, names an existing sysimage upon which to build  the new sysimage incrementally, instead of the sysimage of the current process. Defaults to nothing.  Keyword argument incremental must be true if base_sysimage is not nothing.
  • cpu_target::String: The value to use for JULIA_CPU_TARGET when building the system image. Defaults to native.
  • script::String: Path to a file that gets executed in the --output-o process.
  • sysimage_build_args::Cmd: A set of command line options that is used in the Julia process building the sysimage, for example -O1 --check-bounds=yes.
source
PackageCompiler.create_app — Function
create_app(package_dir::String, compiled_app::String; kwargs...)
Compile an app with the source in package_dir to the folder compiled_app. The folder package_dir needs to contain a package where the package includes a function with the signature
julia_main()::Cint
     # Perhaps do something based on ARGS
     ...
-end
The executable will be placed in a folder called bin in compiled_app and when the executable run the julia_main function is called. Note that since an app-specific Project.toml is placed in the share/julia folder in compiled_app, it is generally not possible to install multiple unrelated apps to the same location.
Standard Julia arguments are set by passing them after a --julia-args argument, for example:
$ ./MyApp input.csv --julia-args -O3 -t8
An attempt to automatically find a compiler will be done but can also be given explicitly by setting the environment variable JULIA_CC to a path to a compiler (can also include extra arguments to the compiler, like -g).
Keyword arguments:
  • executables::Vector{Pair{String, String}}: A list of executables to produce, given as pairs of executable_name => julia_main where executable_name is the name of the produced executable with the julia function julia_main. If not provided, the name of the package (as specified in Project.toml) is used and the main function in julia is taken as julia_main.
  • precompile_execution_file::Union{String, Vector{String}}: A file or list of files that contain code from which precompilation statements should be recorded.
  • precompile_statements_file::Union{String, Vector{String}}: A file or list of files that contain precompilation statements that should be included in the sysimage for the app.
  • incremental::Bool: If true, build the new sysimage on top of the sysimage of the current process otherwise build a new sysimage from scratch. Defaults to false.
  • filter_stdlibs::Bool: If true, only include stdlibs that are in the project file. Defaults to false, only set to true if you know the potential pitfalls.
  • force::Bool: Remove the folder compiled_app if it exists before creating the app.
  • include_lazy_artifacts::Bool: if lazy artifacts should be included in the bundled artifacts, defaults to false.
  • include_transitive_dependencies::Bool: If true, explicitly put all transitive dependencies into the sysimage. This only makes a difference if some packages do not load all their dependencies when themselves are loaded. Defaults to true.
  • include_preferences::Bool: If true, store all preferences visible by the project in package_dir in the app bundle. Defaults to true.
Advanced keyword arguments
  • cpu_target::String: The value to use for JULIA_CPU_TARGET when building the system image.
  • sysimage_build_args::Cmd: A set of command line options that is used in the Julia process building the sysimage, for example -O1 --check-bounds=yes.
  • script::String: Path to a file that gets executed in the --output-o process.
source
PackageCompiler.create_library — Function
create_library(package_or_project::String, dest_dir::String; kwargs...)
Compile a library with the source in package_or_project to the folder dest_dir. The folder package_or_project should contain a package with C-callable functions, e.g.
Base.@ccallable function julia_cg(fptr::Ptr{Cvoid}, cx::Ptr{Cdouble}, cb::Ptr{Cdouble}, len::Csize_t)::Cint
+end
The executable will be placed in a folder called bin in compiled_app and when the executable run the julia_main function is called. Note that since an app-specific Project.toml is placed in the share/julia folder in compiled_app, it is generally not possible to install multiple unrelated apps to the same location.
Standard Julia arguments are set by passing them after a --julia-args argument, for example:
$ ./MyApp input.csv --julia-args -O3 -t8
An attempt to automatically find a compiler will be done but can also be given explicitly by setting the environment variable JULIA_CC to a path to a compiler (can also include extra arguments to the compiler, like -g).
Keyword arguments:
  • executables::Vector{Pair{String, String}}: A list of executables to produce, given as pairs of executable_name => julia_main where executable_name is the name of the produced executable with the julia function julia_main. If not provided, the name of the package (as specified in Project.toml) is used and the main function in julia is taken as julia_main.
  • precompile_execution_file::Union{String, Vector{String}}: A file or list of files that contain code from which precompilation statements should be recorded.
  • precompile_statements_file::Union{String, Vector{String}}: A file or list of files that contain precompilation statements that should be included in the sysimage for the app.
  • incremental::Bool: If true, build the new sysimage on top of the sysimage of the current process otherwise build a new sysimage from scratch. Defaults to false.
  • filter_stdlibs::Bool: If true, only include stdlibs that are in the project file. Defaults to false, only set to true if you know the potential pitfalls.
  • force::Bool: Remove the folder compiled_app if it exists before creating the app.
  • include_lazy_artifacts::Bool: if lazy artifacts should be included in the bundled artifacts, defaults to false.
  • include_transitive_dependencies::Bool: If true, explicitly put all transitive dependencies into the sysimage. This only makes a difference if some packages do not load all their dependencies when themselves are loaded. Defaults to true.
  • include_preferences::Bool: If true, store all preferences visible by the project in package_dir in the app bundle. Defaults to true.
Advanced keyword arguments
  • cpu_target::String: The value to use for JULIA_CPU_TARGET when building the system image.
  • sysimage_build_args::Cmd: A set of command line options that is used in the Julia process building the sysimage, for example -O1 --check-bounds=yes.
  • script::String: Path to a file that gets executed in the --output-o process.
source
PackageCompiler.create_library — Function
create_library(package_or_project::String, dest_dir::String; kwargs...)
Compile a library with the source in package_or_project to the folder dest_dir. The folder package_or_project should contain a package with C-callable functions, e.g.
Base.@ccallable function julia_cg(fptr::Ptr{Cvoid}, cx::Ptr{Cdouble}, cb::Ptr{Cdouble}, len::Csize_t)::Cint
     try
         x = unsafe_wrap(Array, cx, (len,))
         b = unsafe_wrap(Array, cb, (len,))
@@ -13,4 +13,4 @@
         return 1
     end
     return 0
-end
Alternatively, it can contain a project with dependencies that have C-callable functions.
The library will be placed in the lib folder in dest_dir (or bin on Windows), and can be linked to and called into from C/C++ or other languages that can use C libraries. Note that since a library-specific Project.toml is placed in the share/julia folder in dest_dir, it is generally not possible to install multiple libraries to the same location.
Note that any applications/programs linking to this library may need help finding it at run time. Options include
  • Installing all libraries somewhere in the library search path.
  • Adding /path/to/libname to an appropriate library search path environment variable (DYLD_LIBRARY_PATH on OSX, PATH on Windows, or LD_LIBRARY_PATH on Linux/BSD/Unix).
  • Running install_name_tool -change libname /path/to/libname (OSX)
To use any Julia exported functions, you must first call init_julia(argc, argv), where argc and argv are parameters that would normally be passed to julia on the command line (e.g., to set up the number of threads or processes).
When your program is exiting, it is also suggested to call shutdown_julia(retcode), to allow Julia to cleanly clean up resources and call any finalizers. (This function simply calls jl_atexit_hook(retcode).)
An attempt to automatically find a compiler will be done but can also be given explicitly by setting the environment variable JULIA_CC to a path to a compiler (can also include extra arguments to the compiler, like -g).
Keyword arguments:
  • lib_name::String: an alternative name for the compiled library. If not provided, the name of the package (as specified in Project.toml) is used. lib will be prepended to the name if it is not already present.
  • precompile_execution_file::Union{String, Vector{String}}: A file or list of files that contain code from which precompilation statements should be recorded.
  • precompile_statements_file::Union{String, Vector{String}}: A file or list of files that contain precompilation statements that should be included in the sysimage for the library.
  • incremental::Bool: If true, build the new sysimage on top of the sysimage of the current process otherwise build a new sysimage from scratch. Defaults to false.
  • filter_stdlibs::Bool: If true, only include stdlibs that are in the project file. Defaults to false, only set to true if you know the potential pitfalls.
  • force::Bool: Remove the folder compiled_lib if it exists before creating the library.
  • header_files::Vector{String}: A list of header files to include in the library bundle.
  • julia_init_c_file::::Union{String, Vector{String}}: A file or list of files to include in the system image with functions for initializing Julia from external code (default: PackageCompiler.default_julia_init()).
  • julia_init_h_file::::Union{String, Vector{String}}: A file or list of files to include in the library bundle, with declarations for the functions defined in the file(s) provided via julia_init_c_file (default: PackageCompiler.default_julia_init_header()).
  • version::VersionNumber: Library version number. Added to the sysimg .so name on Linux, and the .dylib name on Apple platforms, and with compat_level, used to determine and set the current_version, compatibility_version (on Apple) and soname (on Linux/UNIX)
  • compat_level::String: compatibility level for library. One of "major", "minor". Used to determine and set the compatibility_version (on Apple) and soname (on Linux/UNIX).
  • include_lazy_artifacts::Bool: if lazy artifacts should be included in the bundled artifacts, defaults to false.
  • include_transitive_dependencies::Bool: If true, explicitly put all transitive dependencies into the sysimage. This only makes a difference if some packages do not load all their dependencies when themselves are loaded. Defaults to true.
  • include_preferences::Bool: If true, store all preferences visible by the project in project_or_package in the library bundle. Defaults to true.
  • script::String: Path to a file that gets executed in the --output-o process.
Advanced keyword arguments
  • cpu_target::String: The value to use for JULIA_CPU_TARGET when building the system image.
  • sysimage_build_args::Cmd: A set of command line options that is used in the Julia process building the sysimage, for example -O1 --check-bounds=yes.
source

+end

Alternatively, it can contain a project with dependencies that have C-callable functions.

The library will be placed in the lib folder in dest_dir (or bin on Windows), and can be linked to and called into from C/C++ or other languages that can use C libraries. Note that since a library-specific Project.toml is placed in the share/julia folder in dest_dir, it is generally not possible to install multiple libraries to the same location.

Note that any applications/programs linking to this library may need help finding it at run time. Options include

  • Installing all libraries somewhere in the library search path.
  • Adding /path/to/libname to an appropriate library search path environment variable (DYLD_LIBRARY_PATH on OSX, PATH on Windows, or LD_LIBRARY_PATH on Linux/BSD/Unix).
  • Running install_name_tool -change libname /path/to/libname (OSX)

To use any Julia exported functions, you must first call init_julia(argc, argv), where argc and argv are parameters that would normally be passed to julia on the command line (e.g., to set up the number of threads or processes).

When your program is exiting, it is also suggested to call shutdown_julia(retcode), to allow Julia to cleanly clean up resources and call any finalizers. (This function simply calls jl_atexit_hook(retcode).)

An attempt to automatically find a compiler will be done but can also be given explicitly by setting the environment variable JULIA_CC to a path to a compiler (can also include extra arguments to the compiler, like -g).

Keyword arguments:

  • lib_name::String: an alternative name for the compiled library. If not provided, the name of the package (as specified in Project.toml) is used. lib will be prepended to the name if it is not already present.

  • precompile_execution_file::Union{String, Vector{String}}: A file or list of files that contain code from which precompilation statements should be recorded.

  • precompile_statements_file::Union{String, Vector{String}}: A file or list of files that contain precompilation statements that should be included in the sysimage for the library.

  • incremental::Bool: If true, build the new sysimage on top of the sysimage of the current process otherwise build a new sysimage from scratch. Defaults to false.

  • filter_stdlibs::Bool: If true, only include stdlibs that are in the project file. Defaults to false, only set to true if you know the potential pitfalls.

  • force::Bool: Remove the folder compiled_lib if it exists before creating the library.

  • header_files::Vector{String}: A list of header files to include in the library bundle.

  • julia_init_c_file::::Union{String, Vector{String}}: A file or list of files to include in the system image with functions for initializing Julia from external code (default: PackageCompiler.default_julia_init()).

  • julia_init_h_file::::Union{String, Vector{String}}: A file or list of files to include in the library bundle, with declarations for the functions defined in the file(s) provided via julia_init_c_file (default: PackageCompiler.default_julia_init_header()).

  • version::VersionNumber: Library version number. Added to the sysimg .so name on Linux, and the .dylib name on Apple platforms, and with compat_level, used to determine and set the current_version, compatibility_version (on Apple) and soname (on Linux/UNIX)

  • compat_level::String: compatibility level for library. One of "major", "minor". Used to determine and set the compatibility_version (on Apple) and soname (on Linux/UNIX).

  • include_lazy_artifacts::Bool: if lazy artifacts should be included in the bundled artifacts, defaults to false.

  • include_transitive_dependencies::Bool: If true, explicitly put all transitive dependencies into the sysimage. This only makes a difference if some packages do not load all their dependencies when themselves are loaded. Defaults to true.

  • include_preferences::Bool: If true, store all preferences visible by the project in project_or_package in the library bundle. Defaults to true.

  • script::String: Path to a file that gets executed in the --output-o process.

Advanced keyword arguments

  • cpu_target::String: The value to use for JULIA_CPU_TARGET when building the system image.

  • sysimage_build_args::Cmd: A set of command line options that is used in the Julia process building the sysimage, for example -O1 --check-bounds=yes.

source
diff --git a/dev/sysimages.html b/dev/sysimages.html index 275f014f..5e0fac91 100644 --- a/dev/sysimages.html +++ b/dev/sysimages.html @@ -61,4 +61,4 @@ ~/NewSysImageEnv ❯

Using a manually generated list of precompile statements

Starting Julia with --trace-compile=file.jl will emit precompilation statements to file.jl for the duration of the started Julia process. This can be useful in cases where it is difficult to give a script that executes the code (like with interactive use). A file with a list of such precompile statements can be used when creating a sysimage by passing the keyword argument precompile_statements_file. See the OhMyREPL.jl example in the docs for more details on how to use --trace-compile with PackageCompiler.

It is also possible to use SnoopCompile.jl to create files with precompilation statements.

Using a package's test suite to generate precompile statements

It is also possible to use a package's test suite to generate a list of precompile statements by including the content:

import Example
-include(joinpath(pkgdir(Example), "test", "runtests.jl"))

in the precompile file. Note that you need to have any test dependencies installed in your current project.

Advanced options to create_sysimage

+include(joinpath(pkgdir(Example), "test", "runtests.jl"))

in the precompile file. Note that you need to have any test dependencies installed in your current project.

Advanced options to create_sysimage