Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python312Packages.mlx: upgrade and fix build #367011

Merged
merged 4 commits into from
Dec 23, 2024

Conversation

Gabriella439
Copy link
Contributor

@Gabriella439 Gabriella439 commented Dec 21, 2024

I polished up the mlx package, both upgrading it and also fixing the build

Things done

  • Built on platform(s)
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@Gabriella439 Gabriella439 force-pushed the gabriella/fix_mlx_build branch from 52d409a to 2e06f26 Compare December 21, 2024 05:52
@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Dec 21, 2024
@ofborg ofborg bot requested review from doronbehar and markuskowa December 21, 2024 17:19
@ofborg ofborg bot added 10.rebuild-linux: 101-500 and removed 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Dec 21, 2024
@Gabriella439 Gabriella439 changed the title pythonPackages.mlx: upgrade and fix build python3Packages.mlx: upgrade and fix build Dec 21, 2024
Copy link
Contributor

@doronbehar doronbehar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the nature of all the changes, but they don't look bad. Only the separation to different commits is very wrong.

pkgs/by-name/op/openmpi/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/pr/prrte/package.nix Show resolved Hide resolved
pkgs/by-name/op/openmpi/package.nix Outdated Show resolved Hide resolved
@Gabriella439 Gabriella439 force-pushed the gabriella/fix_mlx_build branch from ea72fc1 to ca06082 Compare December 21, 2024 18:37
@Gabriella439 Gabriella439 changed the title python3Packages.mlx: upgrade and fix build python312Packages.mlx: upgrade and fix build Dec 21, 2024
@Gabriella439
Copy link
Contributor Author

Also, just to clarify: do you want me to merge these as separate commits or is that just for review purposes? Typically I use squash merging, which is why I'm asking

@github-actions github-actions bot added 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux and removed 10.rebuild-linux: 101-500 labels Dec 21, 2024
@ofborg ofborg bot requested a review from markuskowa December 22, 2024 06:10
Copy link
Contributor

@doronbehar doronbehar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK now that the commit order is better (but still not ideal), I understand the purpose of the PR better. As for the Git commit log, I'd do it like this:

prrte: enable on Darwin
openmpi: Build with prrte by default, for all platforms
python312Packages.mlx: add abriella439 as maintainer
python312Packages.mlx: 0.18.0 -> 0.21.1; unbreak

Note how:

  • The packages changed start from the bottom of the dependency tree upwards.
  • Each commit prefix starts with a package attribute that is directly handled by Hydra and therefor trigger a build by ofborg thanks to the behavior defined here. Specifically, note the difference between python3Packages and python312Packages. The former is an alias to the later, and Hydra and ofborg really build only the later.
  • A package version update like this (to mlx), by nature requires many changes to the expression, and hence there is no point in separating to different commits changes to the mlx expression other then the meta.maintainers change. Especially when the package was broken before this PR.

As for the openmpi debate around the prrte change: I'm glad you have discovered that prrte builds fine on Darwin and that openmpi builds fine with it too. I'm also not sure why mlx won't build for you with openmpi compiled with the internal prrte - it'd be interesting to investigate but I don't think it is important enough, because either way, we don't necessarily have to settle this debate, if we'd simply use in openmpi/package.nix:

lib.withFeatureAs true "prrte" (lib.getBin prrte)

Also, I expect the --without-prrte flag to be unsupported by openmpi's ./configure script, just like --without-ofi-libdir which is mentined there as well.

In general, Nixpkgs favors using our builds of dependencies instead of upstream vendored versions of such dependencies, as it can potentially save disk usage. That's another reason to enable prrte unconditionally / use our version of prrte unconditionally.

@Gabriella439 Gabriella439 force-pushed the gabriella/fix_mlx_build branch from ca06082 to 7456f74 Compare December 22, 2024 20:46
The package builds just fine on darwin and wasn't broken
@Gabriella439 Gabriella439 force-pushed the gabriella/fix_mlx_build branch from 7456f74 to 45f34f3 Compare December 22, 2024 20:48
`openmpi` can build against either an external `prrte` or its own
vendored `prrte`.  This change prefers using the external `prrte`
built by Nixpkgs on all platform (including Darwin, now that the
`prrte` build is enabled on all platforms)
@Gabriella439 Gabriella439 force-pushed the gabriella/fix_mlx_build branch from 45f34f3 to 32ff266 Compare December 22, 2024 21:02
@github-actions github-actions bot added 10.rebuild-linux: 101-500 and removed 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Dec 22, 2024
Copy link
Contributor

@doronbehar doronbehar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look great, including the commit messages are very well undestood. Thanks for the contribution! I'll give @markuskowa a day or two to approve / merge.

@@ -75,6 +75,6 @@ stdenv.mkDerivation rec {
homepage = "https://docs.prrte.org/";
license = lib.licenses.bsd3;
maintainers = with lib.maintainers; [ markuskowa ];
platforms = lib.platforms.linux;
platforms = lib.platforms.unix;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I once heard someone saying that platforms.unix includes BSD, which is not necessarily a supported platform :). However, for most practical purposes, this is the same, as Hydra doesn't build for BSD ;). Just noting it here for you so you'd know for next time. Don't worry this is not a blocker for the PR.

@ofborg ofborg bot requested a review from doronbehar December 23, 2024 10:46
Copy link
Member

@markuskowa markuskowa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM.

Side note: When I package new packages (such as PRRTE), I only enable them for Linux usually. Darwin support is difficult to handle if you do not have access to a Darwin machine. I thus leave it to Darwin user/devs to enable Darwin support on packages.

@doronbehar doronbehar merged commit 52fc4c0 into NixOS:master Dec 23, 2024
40 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants