support pydantic (de)serialization on MetadataFilter

[pmeier]

The is a prerequisite for using MetadataFilter as part of FastAPI. I've scrapped the actual JSON serialization, because that would create JSON inside JSON with pydantic and pydantic knows how to serialize dicts anyway.

[nenb]

I was working through this, and I was a little bit dissatisfied that dict_schema was only able to do a very basic check on input schemas eg Model(mf={"IN": {}}) passes the input schema check, but then throws an obscure validation error.

It then occurred to me, do we really need the children key?

For example, instead of

{'OR': {'children': [{'RAW': {'filter': 'raw'}}, {'GE': {'key': 1}}]}}

could we have

{'OR': [{'RAW': {'filter': 'raw'}}, {'GE': {'key': 1}}]}

If so, perhaps this could simplify some of the pydantic logic here?

[nenb]

Also, I haven't looked into it, but a lot of tests seem to fail. Not sure what is going on...

[pmeier]

It then occurred to me, do we really need the children key?

Nope

ragna/ragna/core/_metadata_filter.py

Lines 23 to 26 in ad9c130

	class MetadataFilter:
	# These are just to be consistent. The actual values have no effect.
	_RAW_KEY = "filter"
	_CHILDREN_KEY = "children"

Per comment, the same for the raw filter. So you can break down your example even more to

{'OR': [{'RAW': 'raw'}, {'GE': {'key': 1}}]}

That is as concise as it gets. However we also need to consider the use case of translating it. As discussed offline, I dislike the bits you highlighted as well. I invite everyone to come up with a better scheme and I'm happy to adopt it.

Also, I haven't looked into it, but a lot of tests seem to fail. Not sure what is going on...

Just minor stuff that should be fixed by #461.

[pmeier]

Thinking about this more, I realized that there is no requirement for the primitive version to have a 1-to-1 relationship to the Python object. Meaning, we can go for the simplifications proposed in #476 (review) and #476 (comment) and still keep one Python object that always has the operator, key, and value fields.

I implemented that in 70ac0b5. Here is the example from above:

import json

from ragna.core import MetadataFilter

metadata_filter = MetadataFilter.or_(
    [
        MetadataFilter.raw("raw"),
        MetadataFilter.ge("key", 1),
    ]
)

print(metadata_filter)
print(json.dumps(metadata_filter.to_primitive()))

OR(
  RAW('raw'),
  GE('key', 1),
)
{"OR": [{"RAW": "raw"}, {"GE": {"key": 1}}]}