Sparkdantic should also support pydantic v1 (>=1.10.17) which is still actively maintained and supports migration nicely.
Some projects may not yet have migrated to pydantic v2 (potentially due to other project dependencies). This would also benefit by providing projects with a migration pathway to v2.
Background: We recently added support for Spark-compatible JSON schema generation in v2.0.0 (see #583) which doesn't require PySpark to be installed. This means that any non-Spark repository (e.g. a schema repository or a data producer) could use sparkdantic.
I'm happy to have a look into implementing this in the near future if we want to proceed.
Cheers!
Sparkdantic should also support pydantic v1 (
>=1.10.17) which is still actively maintained and supports migration nicely.Some projects may not yet have migrated to pydantic v2 (potentially due to other project dependencies). This would also benefit by providing projects with a migration pathway to v2.
Background: We recently added support for Spark-compatible JSON schema generation in
v2.0.0(see #583) which doesn't require PySpark to be installed. This means that any non-Spark repository (e.g. a schema repository or a data producer) could use sparkdantic.I'm happy to have a look into implementing this in the near future if we want to proceed.
Cheers!