PackageCompiler fails nightly PkgEval (but nightly CI passes in this repo's GitHub Actions)

[DilumAluthge]

I don't know if this is a PkgEval bug or a PackageCompiler bug (or a Julia bug).

In PkgEval on julia#master, PackageCompiler times out, even if the timeout is really high. For example, this PkgEval job times out at 8181 seconds:

PkgEval terminated after 8181.26s: test duration exceeded the time limit

8181 seconds = 2 hours 16 minutes. Here's the versioninfo for the aforementioned PkgEval job:

Version Info

Primary Build

Julia Version 1.12.0-DEV.1731
Commit 6cb9f04c1e* (2024-12-07 18:56 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  uname: Linux 6.8.0-49-generic #49-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov  4 02:06:24 UTC 2024 x86_64 unknown
  CPU: AMD EPYC 7502 32-Core Processor: 
                  speed         user         nice          sys         idle          irq
       #1-128  1500 MHz  243963906 s      11828 s   21339642 s  275226894 s          0 s
  Memory: 32.0 GB (32601.44140625 MB free)
  Uptime: 423311.14 sec
  Load Avg:  7.07  9.13  5.97
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, znver2)
Threads: 1 default, 0 interactive, 1 GC (on 128 virtual cores)

However, when I run PackageCompiler on GitHub Actions CI on Julia nightly, the job passes in 1 hour 27 minutes. See for example this recent nightly CI job on this repo:

succeeded 3 days ago in 1h 27m 30s

Here is the versioninfo for the aforementioned GitHub Actions job:

Julia Version 1.12.0-DEV.1731
Commit 6cb9f04c1ec (2024-12-07 18:56 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

It is unfortunate that PackageCompiler fails in nightly PkgEval, because it means we can't detect if/when something in upstream Julia breaks PkgEval.

@maleadt @KristofferC Any ideas?

[maleadt]

Looking at the server now that the job has been running for close to 90 minutes, it doesn't seem stuck, just chugging along. Maybe it's slow because of enabling assertions in Julia? In any case, I don't want to extend the PkgEval time limits any further, even if it's only a single package exceeding them, the "tail latency" is large enough already (so testing PackageCompiler is likely to block the entire process while it's the only package being tested).

[DilumAluthge]

Could we use a regular (non-assert) build of Julia when testing PackageCompiler in PkgEval?

[maleadt]

No, that's the kind of hacking around I want to avoid adding to PkgEval.

[KristofferC]

We can make

PackageCompiler.jl/test/runtests.jl

Line 28 in 1c35331

const is_slow_ci = is_ci && Sys.ARCH == :aarch64 && !Sys.isapple()

be true on PkgEval?

[maleadt]

Doesn't that trigger already? We do set CI: https://github.com/JuliaCI/PkgEval.jl/blob/c2d46a6e8f5551fff6b8e3e0bcbf847be59c2370/src/sandbox.jl#L339-L343

[KristofferC]

But that is not the only condition?

[maleadt]

Oh, sorry, I'm blind. Yeah, it's probably worth extending that to cover PkgEval.