Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Third-party library repositories on Windows CI #14802

Open
HertzDevil opened this issue Jul 9, 2024 · 5 comments
Open

Third-party library repositories on Windows CI #14802

HertzDevil opened this issue Jul 9, 2024 · 5 comments
Labels
platform:windows Windows support based on the MSVC toolchain / Win32 API status:discussion topic:infrastructure/ci

Comments

@HertzDevil
Copy link
Contributor

Our Windows CI workflow tries to use the original library repositories to build Crystal's dependencies whenever possible, but we are still left with a few third-party repos in use:

These all add MSBuild support since their upstreams do not support the MSVC toolchain out of the box. However, they might become unmaintained, tempered, or even deleted, potentially compromising our Windows builds. So I think we need to act:

  • Clone the upstreams in crystal-lang and maintain the MSVC projects ourselves directly;
  • Figure out whether building with MSYS(2) on CI is possible;
  • Somehow get the upstreams to provide MSVC toolchain support.
@HertzDevil HertzDevil added status:discussion platform:windows Windows support based on the MSVC toolchain / Win32 API topic:infrastructure/ci labels Jul 9, 2024
@ysbaddaden
Copy link
Contributor

Figure out whether building with MSYS(2) on CI is possible;

You mean installing the libraries as dependencies from MSYS(2) repos (mingw/ucrt?) to retain compatibility with MSVC? Or to build the libraries in a MSYS(2) environment?

@HertzDevil
Copy link
Contributor Author

The latter, as we have always done

@HertzDevil
Copy link
Contributor Author

Slightly related: these are precisely the repositories that don't offer CMake support, and our PowerShell build scripts for them hardcode the use of x86-64, so we might have to add ARM64 ourselves (since it turns out ARM64 support isn't that far-off)

@HertzDevil
Copy link
Contributor Author

HertzDevil commented Sep 16, 2024

GMP itself actually builds directly under MSYS2's UCRT64 environment using mingw-w64-ucrt-x86_64-gcc. Cygwin also works if MinGW is installed there, except the resulting binaries link against the old MSVC runtime rather than the UCRT. (GNU libiconv works on Cygwin because it provides its own auxiliary scripts to use the host MSVC compilers. We could probably get it building under MSYS2 if we want to.)

The main problem is GMP doesn't have the equivalent of MPIR's SI and UI types; with the exception of LibGMP::MpLimb continuing to be 64-bit, a good chunk of functions work with a much reduced range, and LibGMP.mpf_get_str in particular crashes on values above 2.to_big_f ** 0x35269E1C0, or around 3.869 * 10.to_big_f ** (4294967296 + 43). So moving from MPIR to GMP would technically be a breaking change if someone is dealing with those very large floats.

A second problem is some of those crash dumps look really odd:

Invalid memory access (C0000005) at address 0x3f0e6d2cd8
[0x7ff933e6d42d] _gmpn_divrem_1 +381 in C:\Users\nicet\AppData\Local\crystal\cache\libgmp-10.dll
[0x3ea41ff0a0] ???
[0x3ea41fefb0] ???
[0x7ff933e6d5fb] _gmpn_divrem_2 +283 in C:\Users\nicet\AppData\Local\crystal\cache\libgmp-10.dll
[0xd8e7dff660] ???
[0xd8e7dff570] ???
[0x30] ???
[0x6a4d3c48] ???
[0x2be6d6f0010] ???
[0xd49a789] ???
[0xd8e7dff600] ???

These are either a natural consequence of GMP's heavy use of assembly, or a warning that we should indeed not mix libraries built with MinGW and MSVC.

I haven't checked whether MPIR builds under Cygwin or MSYS2.

@HertzDevil
Copy link
Contributor Author

GMP-specific fixes are now available in #15070.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform:windows Windows support based on the MSVC toolchain / Win32 API status:discussion topic:infrastructure/ci
Projects
Status: In Progress
Development

No branches or pull requests

2 participants