RFC 8259 Validation & Compliance

[CrazyKidJack]

Is this library 100% compliant with the JSON specification (RFC 8259)?

Can this library be used as a document validator for that specification?

Is the JSON produced by this library 100% compliant with that specification?

I was planning to use simdjson before I realized I can't create JSON documents with it (at least I didn't see how). One big reason I wanted to use simdjson was that it is 100% compliant with RFC 8259 and can validate documents for compliance (i.e. it will reject documents that are not compliant). Just wondering if this library can be used for that purpose as well as creating compliant JSON documents.

[stephenberry]

We are aiming for 100% compliance, but we aren't quite there yet. For example, we currently support comments (/**/ and //) when reading. We can easily add a compile time option to turn this off, but it doesn't exist yet.

Currently the library uses exceptions for error handling, which isn't ideal for validation. However, we are actively working on updating the codebase to not throw exceptions and instead return error codes and provide error context. This will be a better approach to validation, since exception handling can be slow.

We'll keep this issue open until we achieve this goal. I wouldn't recommend using it for validation until the error handling has been updated.

Thanks for your question and giving even more reasons for our planned changes.

[mwalcott3]

Is this library 100% compliant with the JSON specification (RFC 8259)?
We would want to add some compliance tests like the ones in JSON_checker test suite before we can state something like 100% compliance.

Is the JSON produced by this library 100% compliant with that specification?
It should be with the exception that nan and infinity are currently written out by the floating point conversion routine. We should probably write null since neither of those are valid JSON. I think it was done this way to match charconv but id prefer to not output invalid JSON @stephenberry thoughts?

Can this library be used as a document validator for that specification?
While glaze should not be emitting invalid json (except whats talked about above which will probably be changed) I believe it accepts some invalid JSON and would probably not work well as a validator. I think there are some areas in the code like comments where the thought was "be conservative in what you do, be liberal in what you accept from others". As @stephenberry said we should probably add an option to force compliance when reading.

[stephenberry]

@mwalcott3, I think dumping null instead of infinity or nan is probably best. We could add a compile time option to dump infinity or nan, but I think null should be the default.

[mwalcott3]

Validation and conformance updates were made in #156.

Writing should be 100% compliant and reading should be 100% compliant when using the force_conformance option. We cant guarantee this but we have added a bunch of new conformance tests and any conformance issue reported will be treated as a bug.

A validate_json convivence function was added

std::string fail10 = R"({"Extra value after close": true} "misplaced quoted value")";
assert(glz::validate_json(fail10) != glz::error_code::none);

This essentially just calls glz::read<glz::opts{.force_conformance = true}> on the input buffer with a glz::skip

My personal opinions on spec conformance:

• Writing out invalid JSON is not acceptable unless requested for explicitly.
• Reading in invalid JSON by default is fine for common stuff like "NaN" or unescaped tab characters in strings.
  • This is provided supporting these things doesn't add overhead when parsing conformant JSON.
  • This prevents conflicts with many implementations like pythons json.dumps.
• People should have the option to force conformance when reading.

Edit:
As of right now we assume but do not validate that the input buffer is valid utf8. We will likely look into this but you could use something like https://github.com/simdutf/simdutf