You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The MetadataComponent, as I understand it, speeds up querying metadata in production by having a pre-computed metadata index.
But when developing doc classes for new datasets, this component often intercepts calls to docs_count etc, making it difficult to trace down issues in the development code (because the MetadataComponent sort of monkey-patches the corresponding functions).
Describe the solution you'd like
I'd be happy to have some opt-out environment variable to effectively switch off the monkey patching in this line:
obj=Dataset(MetadataComponent(name, obj), obj) # add metadata from default provider
Describe alternatives you've considered
Alternatively, the metadata component should at least have clear error messages that tell developers that some metadata is missing and how to add those.
Additional context
For developing integrations for large datasets, e.g., #213, having to generate metadata while developing the parsers is also not that great as one would often test the parsers on a smaller sample before computing metadata on the whole database.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
The
MetadataComponent
, as I understand it, speeds up querying metadata in production by having a pre-computed metadata index.But when developing doc classes for new datasets, this component often intercepts calls to
docs_count
etc, making it difficult to trace down issues in the development code (because theMetadataComponent
sort of monkey-patches the corresponding functions).Describe the solution you'd like
I'd be happy to have some opt-out environment variable to effectively switch off the monkey patching in this line:
ir_datasets/ir_datasets/util/registry.py
Lines 41 to 42 in 546baf8
Describe alternatives you've considered
Alternatively, the metadata component should at least have clear error messages that tell developers that some metadata is missing and how to add those.
Additional context
For developing integrations for large datasets, e.g., #213, having to generate metadata while developing the parsers is also not that great as one would often test the parsers on a smaller sample before computing metadata on the whole database.
The text was updated successfully, but these errors were encountered: