Use doctest instead of Catch2 for tests #1064
Conversation
|
I'll note that while the point of this is mostly to improve test build times, the test builds seem slower on CI. Subjectively, they're faster on my machine now, so maybe this is just an artifact of doctest not yet being cached? Not sure. A larger sample size is probably needed. |
cesium-native aggressively uses ccache on CI, so it won't even compile source files at all if they don't change. It makes our CI builds really fast, but also makes it impossible to compare build performance on CI. So I wouldn't worry about that, but if you have any doubts about the local performance actually improving, it might be worth doing a quick measurement just to make sure. |
| @@ -15,6 +13,24 @@ | |||
| #include <string> | |||
| #include <vector> | |||
|
|
|||
| using namespace doctest; | |||
|
|
|||
| bool compareVectors( | |||
There was a problem hiding this comment.
This should be in the anonymous namespace. Or maybe moved to CesiumNativeTests?
|
|
||
| #include <bitset> | ||
| #include <climits> | ||
| #include <cstddef> | ||
| #include <cstring> | ||
| #include <ostream> |
| @@ -21,13 +21,14 @@ | |||
| #include <glm/fwd.hpp> | |||
|
|
|||
| #include <cstdint> | |||
| #include <ostream> | |||
There was a problem hiding this comment.
Not sure why ostream is needed here, either.
There was a problem hiding this comment.
Don't think I could really tell you either, but it is apparently needed: https://github.com/CesiumGS/cesium-native/actions/runs/12797730812/job/35680280884
|
Ran though this script on this branch and the current rm benchmark.log
for i in $(seq 1 10);
do
cmake -B build_benchmark -S .
/usr/bin/time -f "%e" -o benchmark.log -a cmake --build build_benchmark -j22
rm -rf build_benchmark
done
awk '{s+=$1}END{print "average time:", s/NR}' RS="\n" benchmark.logAverage time per build on this branch (doctest): 210.464 seconds So doctest does speed up the build, but only by around 20 seconds. That probably indicates our main slowdowns while building lie outside of the tests. I discovered |
|
That's a nice improvement from just changing the test framework! Thanks for doing the test. Merging! |
Closes #69. We've been intending for quite a while to switch to doctest for tests instead of our current test framework, Catch2. Not only is doctest the test framework that other Cesium projects use, but the compile times are also quite a bit faster. It would be good to get this out of the way while we have the chance - as @kring mentioned, it will only get more difficult as time goes on.