Increased number of builds timing out

[nathanchance]

Recently, I have been noticing that in the past few days, there are a few TuxSuite jobs that have been timing out. For example:

https://github.com/ClangBuiltLinux/continuous-integration2/runs/6163015742?check_suite_focus=true
https://github.com/ClangBuiltLinux/continuous-integration2/runs/6162999661?check_suite_focus=true
https://github.com/ClangBuiltLinux/continuous-integration2/runs/6157414481?check_suite_focus=true
https://github.com/ClangBuiltLinux/continuous-integration2/runs/6143546501?check_suite_focus=true
https://github.com/ClangBuiltLinux/continuous-integration2/runs/6143546098?check_suite_focus=true

There does not really seem to be a pattern here, is something amiss on the TuxSuite side?

cc @vishalbhoj

[nickdesaulniers]

Bump, looks like at least two more tuxsuite runs failed to complete today because of this.
https://github.com/ClangBuiltLinux/continuous-integration2/runs/6282954996?check_suite_focus=true
https://github.com/ClangBuiltLinux/continuous-integration2/runs/6306340804?check_suite_focus=true

[nathanchance]

It sounds like this is now being investigated: https://gitlab.com/Linaro/tuxsuite/-/issues/159

[nickdesaulniers]

https://github.com/ClangBuiltLinux/continuous-integration2/runs/6854118344?check_suite_focus=true#step:4:22

🔧 Infrastructure Error, Please retry - retrying (attempt 1): 23e79f6c0760 ("Merge branch into tip/master: 'x86/mm'") x86_64 (allmodconfig+1) with clang-nightly @ https://builds.tuxbuild.com/2AVEsQhjqPDKzIWRstY0SkNnbeD/

[nathanchance]

That could be fallout from an LLVM regression that we got bit by over the weekend (see all the other red clang-15 builds).

Otherwise, not really sure what we can do about that. If one of the (virtual) machines that Linaro bids on for AWS in their spot market (I think that is what they said they use) gets preempted towards the end of an allmodconfig or allyesconfig build, the build will restart then we'll run into the GitHub Actions job limit of 360 minutes.

[nickdesaulniers]

We need to remember to re-enable x86_64 allyesconfig (#482) once this is resolved.