Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release additional static binaries to use with cargo binstall #402

Open
echoix opened this issue Feb 22, 2023 · 2 comments
Open

Release additional static binaries to use with cargo binstall #402

echoix opened this issue Feb 22, 2023 · 2 comments

Comments

@echoix
Copy link

echoix commented Feb 22, 2023

I'd like to request adding more binary releases to the tools published here, especially static binaries (ie: with x86_64-unknown-linux-musl and aarch64-unknown-linux-musl), in order to be able to quickly use the tools without having to compile from source if the released binaries aren't compatible with the target.

More context:

The tools shellcheck-sarif and sarif-fmt are used in MegaLinter to support outputing the results of the linter shellcheck and to convert sarif output into the console. Currently, our docker images build times are long in CI, and these two small tools make up most of the 5-6 minutes for the rust tools installation (all but the first 19 seconds), which represents 20-25% of the 20-30 min build times for the full image, which includes hundreds of tools. And it needs to run for all PRs, and most flavours, and some individual linters need it, plus other CI tasks daily. So it is worth it. In my issue here (I'm part of MegaLinter's team) oxsecurity/megalinter#2382, we are planning on switching to cargo binstall, that this repo is configured to use, but since the docker images are based on alpine, the target triplet is x86_64-unknown-linux-musl, so everything must be recompiled.

I could try and help on working with the CI to accomplish that in the next weekends, but I'm not experienced yet with rust building, so I might have some trouble.

Let me know if you're open to that.

@psastras
Copy link
Owner

psastras commented Feb 26, 2023

@echoix hi, definitely open to this.

unfortunately the way the CI is written right now is that the target binary is same as the host it's running on (ie. no cross compiling), and it just runs on two separate hosts right now which correspond to each target it releases to (https://github.com/psastras/sarif-rs/blob/main/.github/workflows/main.yml#L138).

in principle rust supports cross compilation with built in flags, so it should be easy (even if you're less familiar with rust) -- i'd take a look here: https://rust-lang.github.io/rustup/cross-compilation.html

so most of the work IMO is just in the CI pipeline.

if you have time and want to dig into this great!

otherwise if / when i find the time i can take a look.

@echoix
Copy link
Author

echoix commented Feb 27, 2023

From what I learned, if there isn't any OpenSSL needed in one of the dependencies, the solution might be as simple as expanding the CI config for the matrix to include the added build cases, add the apt package musl-tools (and maybe pkg-config if it's not enough), and add the target with rustup target add x86_64-unknown-linux-musl and rustup target add aarch64-unknown-linux-musl, then build normally.

Of course that means to adapt the workflow in order to have the file names for the outputs to work.

If it doesn't work like this (that was the case that would keep most of your CI config the same), then there are other completely different options that exists, either with some GitHub actions in the marketplace to help out these tasks, or the cargo cross project, that will build the files in a container from an image maintained by that project (before it was from the rust tools team).

The cargo binstall uses a very different and unique building pattern that was changed recently, so it's not really an example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants