Is the source parameter for synthetic datasets still supported, or is prompt generation now always random?

[valebes]

I'd like to ask the current and intended behavior regarding the source parameter for synthetic datasets:

Question
The documentation accepts a source parameter, meaning that synthetic prompts can be generated based on custom source text (file or URL). However, in guidellm 0.5.x, the prompt generation for synthetic datasets only uses the Faker library, and the source parameter appears to be ignored.

• Is this a regression from previous versions, where prompt text was sampled/generated from the specified source?
• Or is reading from a custom source/text no longer a supported feature, and all synthetic prompts are now always generated via Faker/random text?

Additional context

• In the code, the _create_prompt method (src/guidellm/data/deserializers/synthetic.py) does not use config.source at all.
• The utils file src/guidellm/utils/text.py contains an EndlessTextCreator class that appears designed for this purpose.
• In TestSyntheticDatasetConfig (tests/unit/data/deserializers/test_synthetic.py), the source param is present.
• There is a drift (normal, as docs are currently WIP) between documentation and code regarding the synthetic dataset source.

Would appreciate clarification on:

• Is support for the source parameter as a text source for synthetic prompts intended? If not a feature anymore, would a PR to restore/implement this be welcome?
• Otherwise, should the documentation be updated to match the current state (and if so, what is the intended design)?

Thanks for your input 😊

[sjmonson]

source is no longer a supported parameter. Some of our docs still need updating. Pre-v0.4.0 synthetic dataset were generated with a finite number of rows in advance of the benchmark. This was less then ideal for benchmarks with large prompt token counts or longer run benchmarks that exhausted the finite number of rows.

If you want something nearly identical to the previous synthetic dataset behavior, I suggest taking a look at guidellm preprocess. It currently assumes your padding/trimming existing dataset rows so a PR may be necessary to make it work for your exact use-case.