Unhandled Exception on Request Failure to Resolve $ref

[firyalff]

Background

No try catch on request call causing unhandled exception which can cause application error. See image below for server log on remote resolve error occurrence. In this case, one of ref API returns HTTP 502 on JSON Schema $ref resolve process

Proposal to Resolve

Looking at piece of code from this library on https://github.com/python-jsonschema/jsonschema/blob/main/jsonschema/validators.py#L975

try:
    import requests
except ImportError:
    requests = None

scheme = urlsplit(uri).scheme

if scheme in self.handlers:
    result = self.handlers[scheme](uri)
elif scheme in ["http", "https"] and requests:
    # Requests has support for detecting the correct encoding of
    # json over http
    result = requests.get(uri).json()
else:
    # Otherwise, pass off to urllib and assume utf-8
    with urlopen(uri) as url:
        result = json.loads(url.read().decode("utf-8"))

if self.cache_remote:
    self.store[uri] = result
return result

Try/Catch should be wrapping this part of code

result = requests.get(uri).json()

to ensure that error on request (e.g. 5xx HTTP response) can be handled gracefully.

If this is OK, I'd be glad to open PR for fix

[Julian]

Hello there! Thanks for the ticket.

There's a PR which is about to be released which will essentially be replacing the entire reference resolution implementation (deprecating RefResolver and replacing it). It's #1049 with the introduction of this new referencing library which is fully compliant and has APIs which I hope are a lot easier to understand and customize. (In case you didn't notice, at some point I transferred this issue between repositories).

The next release of jsonschema (v4.18.0) will contain a merged version of that PR, and should be released shortly in beta, and followed quickly by a regular release, assuming no critical issues are reported.

referencing will indeed allow you to make HTTP requests however you'd like to do so, including by catching requests and doing whatever you'd like if they error. I'd love it if you tried out the beta once it is released (likely in the next 2 weeks), or certainly it'd be hugely helpful to immediately install the branch containing this work (https://github.com/python-jsonschema/jsonschema/tree/referencing) and confirm. You can in the interim find documentation for the change in a preview page here -- a section specifically for hooking in HTTP clients is here and feedback is definitely welcome.

I'm going to close this given it will change in #1049 (wherein jsonschema will no longer do the retrieval at all, so yeah, you can do it how you'd like), but feel free to follow up with any comments. Let me know what you think!

[firyalff]

Hello @Julian , thanks for the response. I've check the referencing library, that looks awesome since overriding reference resolve can be overridden with specific object. But, when will the referencing merged to master? As of now, is it safe to use it in production?

[Julian]

Great! Glad to hear. It'll likely be merged in the next week or so, and released in a beta at that moment, and finally in a full release shortly thereafter if a short time passes without critical issues.