Validate JSON strings during parsing

[pschichtel]

Hi everyone!

I have applications that have a lot JSON-based REST endpoints that receive quite a bit of traffic. I have a use-case to generically reject any JSON payloads, that contain strings with certain patterns in side (think of it like automated intrusion detection by looking for e.g. SQL injection patterns). The applications primarily use jackson-databind through ObjectMappers.

I'm looking for a way to provide a special ObjectMapper to the applications, that automatically validates strings during parsing and, if something illegal is detected, throws a JsonParseException.

I have considered subclassing JsonFactoryBuilder, JsonFactory and JsonParser, such that I can build an ObjectMapper that uses my JsonParser subclass which validates strings produced by getText, nextTextValue, ....

While that does work and seems to have the performance characteristics that I'm looking for, the implementation is rather clunky. I have to implement 5 different methods (if that's even all) in JsonParser and 8 methods plus 2 constructors in JsonFactory. If I now update to a newer version of jackson, there might be new methods, that need to be overridden, but aren't, leading to gaps in the validation.

It would be nice to be able to supply some form of visitor that gets called for primitives and completed arrays/objects during the parsing process.

Is something like this already possible?

[pjfanning]

Subclassing can be hard - definitely needs good unit tests of the subclass and its usage to make sure that when you do upgrade Jackson libs that issues are caught.

Could you provide some or all of your subclassing code so that it can be reviewed?

[pschichtel]

Subclassing can be hard

yep, that's exactly why I'd really like to avoid it.

definitely needs good unit tests of the subclass and its usage to make sure that when you do upgrade Jackson libs that issues are caught.

The issue is: How do I test for things I don't know about? I do have tests that cover the validation in various string positions in JSON, but who knows what weird streaming performance special case the new method would be for?

Here is my code (kotlin): https://gist.github.com/pschichtel/31bd3099ce4d4b149fec45740e5df848/9ff2d4507bf77fbace76c4e4ca8a097b9e1463b1

[pschichtel]

An updated version using FilteringParserDelegate: https://gist.github.com/pschichtel/31bd3099ce4d4b149fec45740e5df848/5812233e81629948dc5f4c325d7341d560ac9e5c

[cowtowncoder]

I would strongly advise against sub-classing; it will be fragile, even beyond being difficult/lots of work to implement initially (both of which you are aware of, just reiterating).
Of existing things, use of JsonParserDelegate might be helpful, although has some of same challeneges.

But I am thinking this:

#355

might be something that -- if implemented -- could work here: instead of changing String value, registered handler could throw an exception

[cowtowncoder]

Oh, also, existing

src/main/java/com/fasterxml/jackson/core/filter/FilteringParserDelegate.java

might give ideas as well (wrt use of JsonParserDelegate)

[pschichtel]

Of existing things, use of JsonParserDelegate might be helpful, although has some of same challenges.

yeah, I'm already using that, it helps, but not to the point where it isn't fragile.

#355 does indeed sound like something that would work.

I did have a look at FilteringParserDelegate, but discarded it for some reason, but looking at it again I'm wondering why. Here is a version using that, which at least resolved some of the complexity: https://gist.github.com/pschichtel/31bd3099ce4d4b149fec45740e5df848/5812233e81629948dc5f4c325d7341d560ac9e5c

If you have another trick up your sleeve to simplify my JsonFactory wrapper I would be almost happy with this tbh.

[pschichtel]

ok that version breaks some things unrelated to the validation: Internal error: failed to locate expected buffered tokens

[cowtowncoder]

If there's a bug in state handling, a bug report would be useful: over time we have fixed quite a few cases but it is complicated thing. Although in your case there shouldn't be too many complications as you are not actually filtering anything. :-/