Migrating From RefResolver, unable to resolve $ref

[aryabharat]

Subject: Issue with $ref resolution after updating to the latest jsonschema version

Hi there 👋,

I hope this finds you well. I recently updated the code to the latest jsonschema version, and as part of the process, I refactored the following code:

    resolver = RefResolver(file_prefix + schema_path.replace("\\", "/"), schema, store=SCHEMA_STORE)
    schema_id = schema.get('$id', '')
    if schema_id:
        resolver.store[schema_id] = schema
    # RefResolver creates a new store internally; persist it so we can use the same mappings every time
    SCHEMA_STORE = resolver.store
    validator = STIXValidator(schema, resolver=resolver, format_checker=Draft202012Validator.FORMAT_CHECKER)

The updated code looks like:

    registry = Registry().with_resource(file_prefix + schema_path.replace("\\", "/"), schema)
    validator = STIXValidator(schema,  registry=registry, format_checker=Draft202012Validator.FORMAT_CHECKER)
    return validator

However, during testing, I encountered the following issue:
jsonschema.exceptions._RefResolutionError: <urlopen error [Errno 2] No such file or directory: '../common/core.json'>

The JSON schema is using $ref as follows:

json
Copy code

{
  "$ref": "../common/core.json"
}

After reviewing the documentation, I couldn't find any information on how to handle the file URI in this context.

Am I missing something or doing something incorrectly? Your insights and guidance would be highly appreciated.

Thank you for your time and assistance! 🙏

[Julian]

Hey! It's hard to say exactly what bit is missing.

Have you seen this page which has some info on migrating and/or files specifically?

In general the shortest thing that may be helpful is "avoid relative references for file paths", they're ambiguous and/or nonportable (i.e. your error is saying there's no file at ../common/core.json, which depends on your working directory). They do work if you correctly have base URIs set, but you essentially do have to do that correctly for it to work.

Beyond that I'm not sure which bit to help with -- what is file_prefix + schema_path.replace("\\", "/")? In your referencing code you're saying you want whatever URI that resolves to (which definitely needs to be a URI) to refer to your schema. Is that what you're intending to do? I suspect you're intending to instead or in addition register whatever schema is at ../common/core.json, which doesn't seem like it's happening there. The page I linked has a reference for how to do so, or essentially to call your_schemas.walk() and add each of them to a Registry, or ... depending what you want to do / what URIs you want to be able to refer to your schemas.

[trivialkettle]

I have a similar issue.
I also use refs with relative paths like

# main.schema.json
{
    "$ref": "node/any.schema.json"
}

...
# node/any.schema.json
{
    "$ref": "foo.schema.json"
}

I create this registry:

def retrieve_from_filesystem(uri: str):
    print(f"\nuri: {uri}")
    path = (schema_root_path + "/v1") / pathlib.Path(uri)
    print(path)
    contents = json.loads(path.read_text())
    return Resource.from_contents(contents)

Registry(retrieve=retrieve_from_filesystem).with_resource("https://my-uri/schema/v1/", resource=Resource.from_contents(schema)), schema

I can resolve node/any.schema.json because it contains the subdirectory node in the uri. Then during parsing this schema only foo.schema.json is passed to retrieve_from_filesystem and I don't know the subdirectory.

The old implementation somehow passed the correct file e.g. https://ny-uri/schema/v1/node/foo.schema.json)

They do work if you correctly have base URIs set, but you essentially do have to do that correctly for it to work.

I did not find any docs about this. Could you give me a hint

[trivialkettle]

I did a bit more research:

The old implementation (jsonschema v3.2) resolved like this:

# jsonschema/validators.py:761
    def resolve(self, ref):
        """
        Resolve the given reference.
        """
        url = self._urljoin_cache(self.resolution_scope, ref)
        return url, self._remote_cache(url)

wtih

# call	ref	self.resolution_scope	url
1	node/any.schema.json	https://my-uri/schema/v1/main.schema.json	https://my-uri/schema/v1/node/any.schema.json
2	foo.schema.json	https://my-uri/schema/v1/node/any.schema.json	https://my-uri/schema/v1/node/foo.schema.json
3	../common.schema.json#/definitions/node	'https://my-uri/schema/v1/node/foo.schema.json	https://my-uri/schema/v1/common.schema.json#definitions/node

Also ajv understands relative $ref

(i.e. your error is saying there's no file at ../common/core.json, which depends on your working directory)

It does not depend on the working directory, it should resolve relative to the baseuri / current loaded file. As it did before the API change.