Find references to other files for download and validation (#96)

* Basic support for processing referenced files in table of contents

This is the most basic version of support for this feature to provide
proof of concept. When a local file is valid against the table of
contents schema, open it and collect the URLs of referenced files.
Ask the user if they want to validate those files. If they say yes,
download and validate the files against the appropriate schemas. If
an output file is specified, the outputs for the referenced files are
appended to that file.

* Get referenced files while validating

The validator executable creates a JSON file with lists of referenced
JSON files. The types of files referenced depends on the type of
file being validated.

Note that this is only a partial implementation, applicable to
validation of local files. Additionally, files found with references
will not have their references checked.

* Reorganize functions for working with discovered files

A table of contents file can have in-network and allowed amounts files.
An in-network file can have provider reference files. As long as we
start with only one file (either local or downloaded), we can try
exploring discovered files.

SchemaManager now keeps track of whether it is in strict validation
mode, rather than requiring the strict parameter to be passed to its
various functions.

DockerManager is currently just a container for two docker-related
functions, but will eventually also be responsible for storing some
values to reduce the amount of parameters that have to be sent to
each function call.

Eventually, there will be some kind of implementation for supporting
discovered files when working with a ZIP archive. But, this will likely
require a more complex UI for choosing between files in the ZIP archive
and discovered files.

* Fix object path tracking with object at end of array

Clear out the last known object key when reaching the end of an object.
This avoids problems when processing an array that contains objects.

* Fix validator path tracking bug

The validator previously would track its traversal path incorrectly
when an array contained objects. This fix allows the validator to
correctly find provider references.

Add a debug flag that provides output of the startup command and
docker container run commands.

SchemaManager returns null when asked for a schema that doesn't exist.

Add tests for SchemaManager and DockerManager. Remove functions and
tests for utils for the features that are implemented by those two
classes. Update tests for commands to use SchemaManager and
DockerManager instead of removed utils functions.

* Update output from help command in readme

* Add tests for SchemaManager.ensureRepo

These tests require mocking the exec function, since we don't want
these tests to try to actually clone anything. This makes things a
little bit complicated, because other tests specifically do not want
to mock this function. Mock the module, but add specific mock
implementations in cases where we want to call the actual version
of the exec function.

* Provide mock implementation in ensureRepo test

Doesn't change local test output, but might help with the problem
occurring with the test workflow.

* Use test repo directory in constructor

Dockerfile

@@ -6,7 +6,7 @@ RUN apt-get install -y g++ cmake doxygen valgrind wget
 COPY ./schemavalidator.cpp /
 COPY ./rapidjson /rapidjson
 COPY ./tclap /tclap
-RUN g++ -O3 -I /rapidjson/include -I /tclap/include/ schemavalidator.cpp -o validator
+RUN g++ -O3 --std=c++17 -I /rapidjson/include -I /tclap/include/ schemavalidator.cpp -o validator -lstdc++fs
 
 FROM ubuntu
 COPY --from=build /validator /validator

README.md

@@ -65,10 +65,12 @@ cms-mrf-validator help
 Tool for validating health coverage machine-readable files.
 
 Options:
+  -d, --debug                                      show debug output
   -h, --help                                       display help for command
 
 Commands:
   validate [options] <data-file> <schema-version>  Validate a file against a specific published version of a CMS schema.
+  from-url [options] <data-url> <schema-version>   Validate the file retrieved from a URL against a specific published version of a CMS schema.
   update                                           Update the available schemas from the CMS repository.
   help [command]                                   display help for command
 ```

package.json

@@ -13,7 +13,7 @@
     "build": "del-cli out && tsc",
     "prettier": "prettier --check \"src/*.ts\"",
     "prepare": "npm run build",
-    "test": "jest --maxWorkers=4 --coverage"
+    "test": "jest --coverage"
   },
   "repository": {
     "type": "git",
@@ -28,6 +28,7 @@
   "devDependencies": {
     "@types/fs-extra": "^9.0.13",
     "@types/jest": "^29.2.5",
+    "@types/node": "^20.4.8",
     "@types/readline-sync": "^1.4.4",
     "@types/temp": "^0.9.1",
     "@types/yauzl": "^2.10.0",