Incremental parsing

[FromHub]

ArduinoJson provides the filter feature to limit extracted data while parsing. This is a nice feature to control the parsing behavior and the required memory. However, it lacks the ability to parse incrementally the JSON string and to process independently a set of keys.

As of today, while using deserializeJson function with a modifiable string (char*), the original JSON string is rearranged and cannot be parsed any more. This is a side effect of the design which is acceptable regarding the principle (zero-copy, minimal memory).
To balance the limitation, the proposed idea is to be able to extract a JSON element (key/value) as a c string (whatever its meaning in JSON: string/array/object) which then could be parsed in a next step. This feature could provide the ability to parse part of the original JSON string in some nested steps maintaining all features of the parser.

Here is a naïve dummy proposal which use the filter definition to enable filter in a capture mode instead of parsing mode. The preferred approach is probably to extend filter definition with a custom rule (such as allowStringExtractionOnly). But it would be convenient to get this feature out of the box from the filter definition.

char json[] = "{"sensor":"gps","time":1351824120,"data":{"lat":48.756080, "long":2.302038, "height":101.3}}";

StaticJsonDocument<128> filter1;
filter1["sensor"] = true;
filter1["time"] = true;
filter1["data"] = "capture"; // activate the filter for extraction only (i.e. without JSON parsing) --> proper feature activation shall be defined

StaticJsonDocument<128> doc1;
const DeserializationError parsingErr1 = deserializeJson(doc1, json, DeserializationOption::Filter(filter1));

StaticJsonDocument<128> filter2;
filter2["lat"] = true;
filter2["long"] = true;
filter2["height"] = true;

StaticJsonDocument<128> doc2;
const DeserializationError parsingErr2 = deserializeJson(doc2, doc1["data"].as<const char*>(), DeserializationOption::Filter(filter2));

const float lat = doc2["lat"];
const float long = doc2["long"];
const float height = doc2["height"];

[bblanchon]

Hi @FromHub,

I will implement an incremental parser at some point, but not in the foreseeable future.
However, I will not implement it as you suggested because it's too convoluted and requires too many changes to the existing code.
Instead, I will most likely create a JsonParser where you can push incoming data, as requested in #1690.

BTW, remember that you can disable the zero-copy mode by casting the pointer to const char*, and you can also deserialize in chunks.

Best regards,
Benoit