Feat/fuzzing

[FloydZ]

Description

• This PR adds fuzzing tests

DO NOT MERGE, only for discussion.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Dioprz]

(I changed this to a draft to avoid accidental merging)

After some reads about Atheris and fuzzy testing, this sounds very interesting. Happy to learn about its existence.

Let me know if I can help with something here, @FloydZ.

[FloydZ]

Hi @Dioprz ,
currently I'm not really 100% happy with the solution. There are two main issues:

• atheris only allows for a single function to be fuzzed at each time. Thats why I have code like:

#atheris.Setup(sys.argv, SDFuzz)
...
atheris.Setup(sys.argv, RankSDFuzz)
...
#atheris.Setup(sys.argv, MAYOFuzz)

And you need enable/disable the estimator by hand. If you know how to circumvent that, without adding a bash script, which calls the python code multiple times. let me know.
The second problem is the exception handling. At first I thought catching ValueError is enough to separate between "real problematic" errors in the code, and errors which are already caught by the class constructor. But turns out errors like log2(0) are also ValueErrors, hence I have this ugly:

if type(e) != ValueError or str(e) == "math domain error":

check to difference between those two. If you have a better solution for this, I would really appreciate it. Plus Im not 100% that this is actually really correct.