Skip to content

Latest commit

 

History

History
190 lines (116 loc) · 10.5 KB

assertions.md

File metadata and controls

190 lines (116 loc) · 10.5 KB
Raw

Assertion macros

Most test frameworks have a large collection of assertion macros to capture all possible conditional forms (_EQUALS, _NOTEQUALS, _GREATER_THAN etc).

doctest is different (but it's like Catch in this regard). Because it decomposes comparison expressions most of these forms are reduced to one or two that you will use all the time. That said, there is a rich set of auxiliary macros as well.

There are 3 levels of assert severity for all assertion macros:

  • REQUIRE - this level will immediately quit the test case if the assert fails and will mark the test case as failed.
  • CHECK - this level will mark the test case as failed if the assert fails but will continue with the test case.
  • WARN - this level will only print a message if the assert fails but will not mark the test case as failed.

The CHECK level is mostly useful if you have a series of essentially orthogonal assertions and it is useful to see all the results rather than stopping at the first failure.

All asserts evaluate the expressions only once and if they fail - the values are stringified properly.

Since doctest is thread-safe all asserts and logging macros can be used in threads spawned from test cases.

Note that the REQUIRE level of asserts uses exceptions to end the current test case. It might be dangerous to use this level of asserts inside destructors of user-defined classes - if a destructor is called during stack unwinding due to an exception and a REQUIRE assert fails then the program will terminate. Also since C++11 all destructors are by default noexcept(true) unless specified otherwise so such an assert will lead to std::terminate() being called.

Expression decomposing asserts

These are of the form CHECK(expression) (Same for REQUIRE and WARN).

expression can be a binary comparison like a == b or just a single thing like vec.isEmpty().

If an exception is thrown it is caught, reported, and counted as a failure (unless the assert is of level WARN).

Examples:

CHECK(flags == state::alive | state::moving);
CHECK(thisReturnsTrue());
REQUIRE(i < 42);
  • Negating asserts - <LEVEL>_FALSE(expression) - evaluates the expression and records the logical NOT of the result.

These forms exist as a workaround for the fact that ! prefixed expressions cannot be decomposed properly.

Example:

REQUIRE_FALSE(thisReturnsFalse());
  • Using the DOCTEST_CONFIG_SUPER_FAST_ASSERTS config option can make compilation of asserts up to 31-63% faster!
  • These asserts also have a _MESSAGE form - like CHECK_MESSAGE(expression, message) which is basically a code block {} with a scoped INFO() logging macro together with the CHECK macro - that way the message will be relevant only to that assert. The binary/unary asserts don't have this variation yet.

Examples:

INFO("this is relevant to all asserts, and here is some var: ", local);

CHECK_MESSAGE(a < b, "relevant only to this assert ", other_local, " more text!");

CHECK(b < c); // here only the first INFO() will be relevant

For more information about the INFO() macro visit the logging page.

Binary and unary asserts

These asserts don't use templates to decompose the comparison expressions for the left and right parts.

These have the same guarantees as the expression decomposing ones but 57-68% faster for compilation.

<LEVEL> is one of 3 possible: REQUIRE/CHECK/WARN.

  • <LEVEL>_EQ(left, right) - same as <LEVEL>(left == right)
  • <LEVEL>_NE(left, right) - same as <LEVEL>(left != right)
  • <LEVEL>_GT(left, right) - same as <LEVEL>(left > right)
  • <LEVEL>_LT(left, right) - same as <LEVEL>(left < right)
  • <LEVEL>_GE(left, right) - same as <LEVEL>(left >= right)
  • <LEVEL>_LE(left, right) - same as <LEVEL>(left <= right)
  • <LEVEL>_UNARY(expr) - same as <LEVEL>(expr)
  • <LEVEL>_UNARY_FALSE(expr) - same as <LEVEL>_FALSE(expr)

Using the DOCTEST_CONFIG_SUPER_FAST_ASSERTS config option can make the binary asserts to compile up to 84-91% faster!

Exceptions

<LEVEL> is one of 3 possible: REQUIRE/CHECK/WARN.

  • <LEVEL>_THROWS(expression)

Expects that an exception (of any type) is thrown during evaluation of the expression.

  • <LEVEL>_THROWS_AS(expression, exception_type)

Expects that an exception of the specified type is thrown during evaluation of the expression.

Note that const and & are added to the exception type if missing (users shouldn't care) - the standard practice for exceptions in C++ is Throw by value, catch by (const) reference.

CHECK_THROWS_AS(func(), const std::exception&);
CHECK_THROWS_AS(func(), std::exception); // same as above
  • <LEVEL>_THROWS_WITH(expression, c_string)

Expects that an exception is thrown during evaluation of the expression and is successfully translated to the specified c string (see translating exceptions).

CHECK_THROWS_WITH(func(), "invalid operation!");
  • <LEVEL>_THROWS_WITH_AS(expression, c_string, exception_type)

This is a combination of <LEVEL>_THROWS_WITH and <LEVEL>_THROWS_AS.

CHECK_THROWS_WITH_AS(func(), "invalid operation!", std::runtime_error);
  • <LEVEL>_NOTHROW(expression)

Expects that no exception is thrown during evaluation of the expression.

Note that these asserts also have a _MESSAGE form - like CHECK_THROWS_MESSAGE(expression, message) - these work identically to the _MESSAGE form of the normal macros (CHECK_MESSAGE(a < b, "this shouldn't fail")) described earlier.

One may use the DOCTEST_CONFIG_VOID_CAST_EXPRESSIONS config identifier to cast the expression in these asserts to void to avoid warnings or other issues - for example nodiscard statements whose result isn't checked. This will however limit the ability to write entire {} blocks of code as the expression (or multiple statements) but in that case a simple lambda can be used. This should have been the default behavior from day 1 of the framework...

Using asserts out of a testing context

Asserts can be used outside of a testing context (in code not called from a TEST_CASE()) instead of assert().

A doctest::Context object still has to be created somewhere and set as the default one using the setAsDefaultForAssertsOutOfTestCases() method - and then asserts will work. A handler can be registered by calling the setAssertHandler() method on the context object. If no handler is set then std::abort() is called on failure.

The results would be best when using the DOCTEST_CONFIG_SUPER_FAST_ASSERTS config identifier.

Checkout the example showcasing how that is done. For more information see the issue for the feature request.

Currently logging macros cannot be used for extra context for asserts outside of a test run. That means that the _MESSAGE variants of asserts are also not usable - since they are just a packed INFO() with an assert right after it.

String containment

doctest::Contains can be used to check whether the string passed to its constructor is contained within the string it is compared with. Here's a simple example:

REQUIRE("foobar" == doctest::Contains("foo"));

It can also be used with the THROWS_WITH family of assertion macros to check whether the thrown exception translated to a string contains the provided string. Here's another example:

REQUIRE_THROWS_WITH(func(), doctest::Contains("Oopsie"));

Floating point comparisons

When comparing floating point numbers - especially if at least one of them has been computed - great care must be taken to allow for rounding errors and inexact representations.

doctest provides a way to perform tolerant comparisons of floating point values through the use of a wrapper class called doctest::Approx. doctest::Approx can be used on either side of a comparison expression. It overloads the comparisons operators to take a relative tolerance into account. Here's a simple example:

REQUIRE(performComputation() == doctest::Approx(2.1));

By default a small epsilon value (relative - in percentages) is used that covers many simple cases of rounding errors. When this is insufficient the epsilon value (the amount within which a difference either way is ignored) can be specified by calling the epsilon() method on the doctest::Approx instance. e.g.:

REQUIRE(22.0/7 == doctest::Approx(3.141).epsilon(0.01)); // allow for a 1% error

When dealing with very large or very small numbers it can be useful to specify a scale, which can be achieved by calling the scale() method on the doctest::Approx instance.

NaN checking

Two NaN floating point numbers do not compare equal to each other. This makes it quite inconvenient to check for NaN while capturing the value.

CHECK(std::isnan(performComputation()); // does not capture the result of the call

doctest provides doctest::IsNaN which can be used in assertions to check if a float (or any other floating point fundamental type) is indeed NaN, outputting the actual value if it is not.

CHECK(doctest::IsNaN(performComputation()); // captures the result!

IsNaN is able to capture the value, even if negated via !.

  • Check out the example which shows many of these macros
  • Do not wrap assertion macros in try/catch - the REQUIRE macros throw exceptions to end the test case execution!

Home