Validate examples within the schema

[cc-stjm]

Is there any way (in the main library or check-jsonschema) to allow you to validate the examples within the JSON schema? I want to ensure the examples provided in the schema are actually valid

[Julian]

Hi there!

I assume you mean "I want to ensure that things in the examples keyword are valid under the schema they live inside"?
That's behavior not part of the specification (which basically says to do nothing with that keyword), though of course it's reasonable to want it.

There's not a public API built in at the moment for walking schemas (to e.g. find instances of the keyword) if you want to do this dynamically without knowledge of the specific schema -- such an API will come some day.

In the interim of course though you could either write the (fairly simple) function which walks a schema looking for the keyword, and just call validate.

[cc-stjm]

Thanks, I'll give that a try.

[Julian]

(I moved this to a discussion though maybe I shouldn't have as I'm not sure there's an existing issue covering the schema walking API, so if you have any trouble feel free to either follow up and/or re-open one there for that!)

[cc-stjm]

In case it helps anyone else, I've lashed together a jq based script that'll extract top level examples & test them.

check-jsonschema -v --color always --check-metaschema *.json 
rm -r examples
mkdir -p examples
for schemafile in *.json; do
    number_of_examples=$(jq 'if has("examples") then .examples | length else 0 end' "$schemafile")
    if [[ $number_of_examples -eq 0 ]]; then
        continue
    fi
    for i in $(seq $number_of_examples); do
        index=$( expr $i - 1);
        filename="examples/${schemafile%.*}_${index}.json"
        jq ".examples[$index]" "$schemafile" > $filename
    done;
    check-jsonschema -v --color always --schemafile "$schemafile" examples/${schemafile%.*}_*.json
done;

It needs a little more work to count as a CI error if there are invalid examples, but it's hopefully a useful start.