The implementation of the preprocessing is also somewhat rudimentary. As I said in one of the notes in the code I think it's a more winning strategy to have something like a RawIndex which we then transform into SerializableIndex which has everything expanded, checked etc. this helps the type system enforce that we only write things that have been checked, expanded etc. It might also be a nicer interface for the checking etc.
The implementation of the preprocessing is also somewhat rudimentary. As I said in one of the notes in the code I think it's a more winning strategy to have something like a
RawIndexwhich we then transform intoSerializableIndexwhich has everything expanded, checked etc. this helps the type system enforce that we only write things that have been checked, expanded etc. It might also be a nicer interface for the checking etc.