Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random parallel build failures #1072

Open
Apteryks opened this issue Mar 7, 2024 · 10 comments
Open

Random parallel build failures #1072

Apteryks opened this issue Mar 7, 2024 · 10 comments

Comments

@Apteryks
Copy link

Apteryks commented Mar 7, 2024

Hi,

The build of nx-libs appears to suffer from non-deterministic build failures when parallelized (make -jN). Here's an example:

making all in programs/Xserver...
make[6]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs'
make[6]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/include...
making all in programs/Xserver/dix...
making all in programs/Xserver/os...
making all in programs/Xserver/mi...
making all in programs/Xserver/Xext...
making all in programs/Xserver/Xi...
making all in programs/Xserver/xkb...
making all in programs/Xserver/GL...
making all in programs/Xserver/dbe...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/record...
make[7]: Nothing to be done for 'all'.
make[7]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver/include
'
making all in programs/Xserver/randr...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/render...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/xfixes...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
making all in programs/Xserver/damageext...
make[7]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[7]: warning: -j24 forced in submake: resetting jobserver mode.
make[7]: *** No rule to make target 'all'.  Stop.
make[7]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver/os'
making all in programs/Xserver/miext/cw...
make[6]: *** [Makefile:682: os] Error 2
make[6]: *** Waiting for unfinished jobs....

It'd be nice if it worked normally. I also not that parallel build is disabled in the Debian rules file (--no-parallel).

@uli42
Copy link
Member

uli42 commented Apr 8, 2024

You ouput is unclear to me. What subdir exactly fails?

Is it damageext?

@Apteryks
Copy link
Author

It's not clear to me either, but that's what I see! It does look like the error is triggered from the damageext directory.

@uli42
Copy link
Member

uli42 commented May 9, 2024

Just stumbled over this in nx-X11/programs/Xserver/Imakefile:

#if HasParallelMake
MakeMutex($(NXAGENTDIRS) $(NXCOMPEXT) $(NXAGENTOBJS) $(NXAGENTLIBS) $(NXCOMPEXTSYSLIBS) $(NXAGENTSYSLIBS))
#endif

So you probably have to run the topdir makefile with IMAKE_DEFINES=-DHasParallelMake=1

@Apteryks
Copy link
Author

Hm, I tried that, but it didn't seem to make any difference:

make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/include/GL'
make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/include'
including in ./config...
make[4]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config'
including in config/cf...
make[5]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config/cf'
make[5]: Nothing to be done for 'includes'.
make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config/cf'
make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/config'
including in ./programs...
make[4]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs'
including in programs/Xserver...
make[5]: Entering directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
Makefile:539: *** missing separator.  Stop.
make[5]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs/Xserver'
make[4]: *** [Makefile:615: includes] Error 2
make[4]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11/programs'
make[3]: *** [xmakefile:674: includes] Error 2
make[3]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11'
make[2]: *** [Makefile:102: includes] Error 2
make[2]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source/nx-X11'
make[1]: *** [Makefile:148: build-env] Error 2
make[1]: Leaving directory '/tmp/guix-build-nx-libs-3.5.99.27.drv-0/source'
make: *** [Makefile:220: build] Error 2
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-j" "180" "PREFIX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "ETCDIR_NX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/etc" "LOCAL_LDFLAGS=-Wl,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib/nx/X11" "IMAKE_DEFINES=-DUseTIRPC=1 -DHasParallelMake=1 -DBourneShell=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh -DProjectRoot=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27 -DDefaultUserPath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/bin" "CONFIG_SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "CONFIGURE=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh ./configure --prefix=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "VERBOSE=1") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `build' failed after 5.9 seconds
command "make" "-j" "180" "PREFIX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "ETCDIR_NX=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/etc" "LOCAL_LDFLAGS=-Wl,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib,-rpath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/lib/nx/X11" "IMAKE_DEFINES=-DUseTIRPC=1 -DHasParallelMake=1 -DBourneShell=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh -DProjectRoot=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27 -DDefaultUserPath=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27/bin" "CONFIG_SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "SHELL=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh" "CONFIGURE=/gnu/store/9vw5slrffp27rzy2i2plnw7xfqjyk7m4-bash-minimal-5.1.16/bin/sh ./configure --prefix=/gnu/store/xz96mncfbz47spmdllxyki4hv6vaal4a-nx-libs-3.5.99.27" "VERBOSE=1" failed with status 2
builder for `/gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv' failed with exit code 1
build of /gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv failed
View build log at '/var/log/guix/drvs/zi/k74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv.gz'.
guix build: error: build of `/gnu/store/zik74m5vcclr9bcs44sfqh46i2rh0rv3-nx-libs-3.5.99.27.drv' failed

real    0m42.514s
user    0m6.178s
sys     0m0.594s

@uli42
Copy link
Member

uli42 commented May 19, 2024 via email

@Apteryks
Copy link
Author

If you are wiling to help there's an automake branch that needs some polishing. It should be findable in the pull-requests. Let me know if you don't find it.

That's tempting but am I bit swamped with things to do at the moment. I'll try to remember revisiting this in the future!

@Apteryks
Copy link
Author

Apteryks commented May 21, 2024

Xorg uses meson nowadays so the optimum was to restructure everything to use just that.

"Was" makes it sound like the migration of the build system to Meson already happened? Is that so?

@uli42
Copy link
Member

uli42 commented May 22, 2024 via email

@Apteryks
Copy link
Author

Apteryks commented May 22, 2024

Ah, I see :-) I guess I'll look at that Autotools branch at some point; it'd be a nice improvement over a build system whose default assumption is that make doesn't support parallel builds ^^'.

@uli42
Copy link
Member

uli42 commented May 22, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants