Randomly failing tests on MacOS

[Isty001]

Taking a look at the actions results it seems some tests fail on Mac, but not always.

Running clib(1) tests

./test/gh-35-exit-codes.sh: line 8: 21111 Segmentation fault: 11  clib install stephenmathieson/rot132.c > /dev/null 2>&1
  (✓) test/gh-35-exit-codes.sh
./test/help.sh: line 3: 21113 Segmentation fault: 11  clib help 2> /dev/null
Expected `clib help` to fail
  (✖) test/help.sh
./test/install-binary-dependencies.sh: line 6: 21115 Segmentation fault: 11  clib install -c stephenmathieson/[email protected] -P tmp > /dev/null
Failed to install stephenmathieson/tabs-to-spaces
  (✖) test/install-binary-dependencies.sh
./test/install-brace-expansion.sh: line 9: 21117 Segmentation fault: 11  clib install -c -o tmp stephenmathieson/trim.c stephenmathieson/case.c > /dev/null
expecting successful exit code
  (✖) test/install-brace-expansion.sh
./test/install-deps-from-package-json.sh: line 23: 21122 Segmentation fault: 11  clib install > /dev/null 2>&1
expecting exit code of 1
  (✖) test/install-deps-from-package-json.sh
./test/install-multiple-clibs-libs.sh: line 9: 21124 Segmentation fault: 11  clib install -c -o tmp ms file hash > /dev/null
expecting successful exit code
  (✖) test/install-multiple-clibs-libs.sh
./test/install-multiple-libs.sh: line 10: 21126 Segmentation fault: 11  clib install -c -o tmp stephenmathieson/case.c stephenmathieson/trim.c > /dev/null
expecting successful exit code
  (✖) test/install-multiple-libs.sh

Edit: I can't reproduce any of it on Linux now.

[andy5995]

This log file might give some hints. It's from when I built with gcc 11.1.0 on Debian Bullseye and added the -fanalyzer flag.

clib2.log

[andy5995]

I had some problems recently with tests randomly failing on MacOS using Travis CI. Turned out to be a cache issue.

Btw, I've found Coverity to be a useful tool for testing. Although it sometimes gives false positives, it's pretty good at spotting potential resource leaks or buffer overflows.

[diasbruno]

Haven't seen this in a long time. Should we close this, @Isty001?

[jwerle]

@Isty001 what do you think?

[Isty001]

Yeah, I think we can close it