Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Distinguish warnings and errors in verification #4

Open
ghost opened this issue Jun 7, 2021 · 0 comments
Open

Distinguish warnings and errors in verification #4

ghost opened this issue Jun 7, 2021 · 0 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@ghost
Copy link

ghost commented Jun 7, 2021

Currently the verifier runs a suite of unit tests against the set of WebMs and reports success and failures based on assertion exceptions being raised.

However, there is an intended variance of severity level that is missed with this design. For some tests, we want to fail the test. For example, if the video codec is not VP9, this is an outright violation of the encoding standards. However, there are some tests that are intended as warnings, flagging a file property or format meant to be scrutinized further.

I believe we should adopt a new design that does not make use of unit tests and instead collects a list of warnings and errors using the same test criteria. This list is then outputted to the user upon completion.

@ghost ghost added enhancement New feature or request help wanted Extra attention is needed labels Jun 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

0 participants