Minor fixes to mason + caching

[hamishivi]

1. Add the ability to manually specify extra secrets for mason.py
2. respect the split argument for datasets. The old code split if you used a non-train split for the eval dataset. This fixes it by instead constructing a dataset dict following the specified splits in the dataset config.

[vwxyzjn]

Nice addition on the secret. The dataset_transformation.py might have some issues.

[vwxyzjn]

Here is a use case, say I want to mix "train, test, train, test" splits of 4 datasets. Would this implementation only load "train, train" of the first and thrid dataset?

What do you think about making a default split like

open-instruct/open_instruct/dataset_transformation.py

Lines 801 to 808 in 7d1c5d9

	# NOTE: the cached dataset is always train split
	DEFAULT_SPLIT_FOR_CACHED_DATASET = "train"
	# Check if the revision exists
	if revision_exists(repo_name, config_hash, repo_type="dataset"):
	print(f"✅ Found cached dataset at https://huggingface.co/datasets/{repo_name}/tree/{config_hash}")
	# Use the split from the first dataset config as default
	return load_dataset(repo_name, split=DEFAULT_SPLIT_FOR_CACHED_DATASET, revision=config_hash)

and

open-instruct/open_instruct/dataset_transformation.py

Lines 858 to 859 in 7d1c5d9

	print(f"✅ Found cached dataset at https://huggingface.co/datasets/{repo_name}/tree/{config_hash}")
	return load_dataset(repo_name, split=DEFAULT_SPLIT_FOR_CACHED_DATASET, revision=config_hash)

[hamishivi]

Ah, I guess i was thinking of this differently, this setup makes more sense.

[hamishivi]

Okay changed to this, and tested for my use case and it works!

[vwxyzjn]

Did you push the commit?

[hamishivi]

lol im dumb