Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ποΈ Description
First iteration at implementation of idea in #591
Implements the
fuzz
command that allows testers to mark a test as fuzzable, and then listing parameters that can be randomized:In this simple example, a test function is marked as fuzzable by adding the
@pytest.mark.fuzzable
decorator and as its parameters it lists two parameter names: "key" and "value".When the
fuzz
command is executed and pointed to this file, the test is prepared by first removing its existingpytest.mark.parametrize
that already contains "key" and "value" parameters, and then the test is re-parametrized to fuzzed values for both parameters.The fuzzer reads the annotation of the test function signature and then proceeds to attempt to generate a fuzzing function for each one.
In the case of "value", the annotation is simply
int
so the default fuzzer for integers is used (random.randint(0, 2**256)
).In the case of "key", the type is annotated and the second parameter of the annotation is a callable lambda function which specifies how the value should be randomized, in this case, a random integer from 0 to 2**64.
We can implement many fuzzers for each type, including e.g., random opcodes, random code, random valid or invalid EOF code.
π Related Issues
β Checklist
mkdocs serve
locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.