Extra arguments for hf dataset loading

[anmoisio]

Hi,

I would need to pass extra keyword arguments for the load_dataset function of the datasets library, like this:

from datasets import load_dataset
load_dataset("europarl_bilingual", lang1="en", lang2="fi")

But it seems this is not supported by the API. Is there a way to load this dataset using the API?

[kazemnejad]

Thanks for bringing up this feature. At this point, we plan to add in the final version. In the meantime, you can override Task._load_data_source() method in your Task implementation to add this.

https://github.com/GenBench/genbench_cbt/blob/caa63fbf37733c76b218db5fa08bb2e67ac07b6c/src/genbench/task.py#L466

[anmoisio]

Okay thanks for the help!