Synchronize parameters for various datasets

[grofte]

Describe the bug

IterableDatasetDict map function is missing the desc parameter. You can see the equivalent map function for Dataset here.

There might be other parameters missing - I haven't checked.

Steps to reproduce the bug

from datasets import Dataset, IterableDataset, IterableDatasetDict

ds = IterableDatasetDict({"train": Dataset.from_dict({"a": range(6)}).to_iterable_dataset(num_shards=3),
"validate": Dataset.from_dict({"a": range(6)}).to_iterable_dataset(num_shards=3)})

for d in ds["train"]:
print(d)

ds = ds.map(lambda x: {k: v+1 for k, v in x.items()}, desc="increment")

for d in ds["train"]:
print(d)

Expected behavior

The description parameter should be available for all datasets (or none).

Environment info

• datasets version: 3.2.0
• Platform: Linux-6.1.85+-x86_64-with-glibc2.35
• Python version: 3.11.11
• huggingface_hub version: 0.28.1
• PyArrow version: 17.0.0
• Pandas version: 2.2.2
• fsspec version: 2024.9.0

[lhoestq]

Hi ! the desc parameter is only available for Dataset / DatasetDict for the progress bar of `map()``

Since IterableDataset only runs the map functions when you iterate over the dataset, there is no progress bar and desc is useless. We could still add the argument for parity but it wouldn't be used for anything