Schema classes use parseDict and other_elems from constructor to handle FromDict behavior

[LswaN58]

Have so far had a hell of a time coming up with a good way to do "FromDict" approach to parsing and instantiating Schemas. Finally have what seems to be a solution that accomplishes the following two important things:

• Schema subclasses cleanly utilize parent class logic for parsing parent's required items from dictionaries, without redundant implementation
• External callers of the Schema class constructors have a set of parameters to pass, rather than needing to know a dict format

Old approach

Currently, each Schema class is supposed to just implement a FromDict that parses everything needed from a dict, and return an instance of the class.
However, suppose we have a hierarchy where SchemaB inherits from SchemaB:

classDiagram

SchemaA <|-- SchemaB

Loading

Then SchemaB would need to redundantly parse out everything needed by SchemaA to call the super constructor, e.g.

class SchemaA(Schema):
   def __init__(self, name:str, a, other_elements:Dict[str,Any]):
       self._a = a
       super().__init__(name=name, other_elements=other_elements)
    
   @classmethod
   def FromDict(cls, name:str, all_elements:Dict[str, Any]):
      _a = all_elements['a']
      _leftovers = # all_elements minus element 'a'
      return SchemaA(name=name, a=_a, other_elements=_leftovers)

class SchemaB(SchemaA):
   def __init__(self, name:str, a, b, other_elements:Dict[str,Any]):
       self._b = b
       super().__init__(name=name, a=a, other_elements=other_elements)
    
   @classmethod
   def FromDict(cls, name:str, all_elements:Dict[str, Any]):
      _a = all_elements['a']
      _b = all_elements['b']
      _leftovers = # all_elements minus elements 'a' and 'b'
      return SchemaB(name=name, a=_a, b=_b, other_elements=_leftovers)

Redundancy isn't good, so we try to separate this out by adding a second layer, where there is a _fromDict that accepts pre-parsed params to pass directly to constructor, after parsing out subclass elements:

class SchemaA(Schema):
   def __init__(self, name:str, a, other_elements:Dict[str,Any]):
       self._a = a
       super().__init__(name=name, other_elements=other_elements)
    
   @classmethod
   def FromDict(cls, name:str, all_elements:Dict[str, Any]):
      _a = all_elements['a']
      _leftovers = # all_elements minus element 'a'
      return cls._fromDict(name=name, a=_a, other_elements=_leftovers)

   @classmethod
   @abstractmethod
   def _fromDict(cls, name:str, a, other_elements:Dict[str,Any]):
      pass

class SchemaB(SchemaA):
   def __init__(self, name:str a, b, other_elements:Dict[str,Any]):
       self._b = b
       super().__init__(name=name, a=a, other_elements=other_elements)
    
   @classmethod
   def _fromDict(cls, name:str, a, other_elements:Dict[str, Any]):
      _b = other_elements['b']
      _leftovers = # other_elements minus element 'b'
      return SchemaB(name=name, a=a, b=_b, other_elements=_leftovers)

However, this would require an additional level of "from dict" sub-function for each level of the hierarchy, and that's not clean at all.

An even older approach was to just have the __init__ function act as a de facto FromDict, where all required elements are parsed from a dictionary.
However, this makes for really, really awkward functions that need to call the constructor but don't have a dictionary ready to go.
Writing such functions requires checking and double-checking exactly what is parsed by the init, to understand what keys to use in a custom-built dictionary.
That's not any fun, and it is much, much nicer to just use actual parameters, where linting and autocomplete help ensure the right data is passed in.

New Approach

The best solution I've come up with is as follows:

The __init__ functions should always assume class-specific params are optional and default to None, and fall back on a search of other_elements for a missing value.
The assumption is that all elements passed via other_elements and intended for the class have been parsed into the correct data type - no redundant parsing.
Then the thing currently called FromDict will get renamed to ParseDict, and return a Dict of parsed elements. Then FromDict can call cls.ParseDict(...), merge with super().ParseDict(...), and put that all into the call to the init function.

Sample:

from typing import Any, Dict
class SchemaA(Schema):
   def __init__(self, a=None, other:Dict[str,Any]={}): 
      self._a = a or other.get('a') # or class default

      _leftovers = {key:val for key,val in other.items() if key != 'a'}
      self._other = _leftovers

   @classmethod
   def FromDict(cls, d):
      parsed = cls.ParseDict(d=d)
      return A(other=parsed)

   @classmethod
   def ParseDict(cls, d) -> Dict[str, Any]:
      _a = cls.ParseElement(d.get('a'), ...)
      return d | { 'a' : _a }

class SchemaB(SchemaA):
   def __init__(self, a=None, b=None, other:Dict[str,Any]):
      self._b = b or other.get('b') # or class default

      _leftovers = {key:val for key,val in other.items() if key not in {'a', 'b'}}
      super().__init__(a= a, other=_leftovers)

   @classmethod
   def FromDict(cls, d):
      parsed = super().ParseDict(d=d) | cls.ParseDict(d=d)
      return B(other=parsed)

   @classmethod         
   def _fromDict(cls, d):
      _b = d.get('b')
      return super()._fromDict(d) | { 'b' : _b }

This isn't perfect, as we don't really want the __init__ parameters to be optional with null defaults, but this gives a clean way to have parsing separated by class, but also allow us to have clearly-defined parameters for constructing objects.

[LswaN58]

On further reflection, this doesn't play nice with type linting. We could do something with TypedDict and/or dataclasses, but after playing around with the immediate obvious idea of using a TypedDict subclass as a way to exchange data between ParseDict, FromDict, and __init__ just feels too clunky.

May revisit this at some point from perspective of using TypedDict or dataclasses as super-class to Schema, but short of that this is going on hold. We don't have deep enough inheritance trees yet for Schemas to to need a solution, so putting this back onto the back-burner for now.

[LswaN58]

New idea struck; relevant portions of this discussion have been copied to discussion on reworking the parsing logic to better handle our FromDict approach.