Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify the generated files, move testthat functionnality in external script #162

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

vogr
Copy link
Collaborator

@vogr vogr commented Aug 13, 2021

After this PR, the generated files are much simpler. Here is for instance yaml/as.yaml/test-36.R:

genthat_extracted_call <- function() {
    yaml:::as.yaml(x = 0)
}

The return value and the seed are recorded in the .ext file (with the function record_test_exts) after the generation of the file ; this way the files can be run in isolation (using callr).

This makes it possible to still use the generated files as unit tests using testthat, but also to use them for other uses like benchmarks. These functionalities are taken care of by external scripts. This approach has the advantage that it easier to add new functionalities without having to re-generate all the files.

For instance, the files benchmark.R and test_file.R (in tools/harnesses/) implement the previously mentioned functionalities:

# with the tests generated in ./tests

# Test one file
$ "/path/to/harnesses/test_files.R" "tests/yaml/as.yaml/test-36.R"
Test passed 🎉

# Test all the files in "tests"
$ GENERATED_TESTS_DIR="./tests"  "/path/to/harnesses/test_files.R"
Test passed 🎉
Test passed 😀
Test passed 🥳

# From R with test_file (pass environment variable to R, or use withr::with_envvar)
$ GENERATED_TESTS_DIR="/path/to/tests" R
> testthat::test_file("/path/to/harnesses/test_files.R")
══ Testing test_files.R ═════════════════════════════════════════════════════════
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 3 ] Done!

# Benchmark the test (the output format is compatible with [ReBench](https://rebench.readthedocs.io))
$ benchmark.R tests/yaml/as.yaml/test-36 5 100
tests/yaml/as.yaml/test-36: iterations=1 runtime: 3876us
tests/yaml/as.yaml/test-36: iterations=1 runtime: 1234us
tests/yaml/as.yaml/test-36: iterations=1 runtime: 1222us
tests/yaml/as.yaml/test-36: iterations=1 runtime: 1226us
tests/yaml/as.yaml/test-36: iterations=1 runtime: 1189us
tests/yaml/as.yaml/test-36: iterations=5; average: 1749 us; total: 8747us

vogr added 3 commits August 12, 2021 15:32
- test_files.R using testthat, restoring the unit test functionnality
- harness.R for benchmarking
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant