From ea8985d2eb27ffc7abfe81ecdd5472d1fc0392a0 Mon Sep 17 00:00:00 2001 From: Jenny Yang Yi Date: Fri, 9 May 2025 16:32:51 -0400 Subject: [PATCH 1/2] Update datasets.md Added recommended dataset profiles for four usecases --- docs/datasets.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/datasets.md b/docs/datasets.md index 86aa3af2..07b951cf 100644 --- a/docs/datasets.md +++ b/docs/datasets.md @@ -39,6 +39,12 @@ GuideLLM supports several types of datasets, each with its own advantages and us Synthetic datasets allow you to generate data on the fly with customizable parameters. This is useful for controlled experiments, stress testing, and simulating specific scenarios. For example, you might want to evaluate how a model handles long prompts or generates outputs with specific characteristics. +For different use cases, here are the recommendeded dataset profiles that can be passed in as arguments in GuideLLM: +- Chat: --data "prompt_tokens=512,output_tokens=256" +- RAG: --data "prompt_tokens=4096,output_tokens=512" +- Summarization: --data "prompt_tokens=1024,output_tokens=256" +- Code Generation: --data "prompt_tokens=512,output_tokens=512" + #### Example Commands ```bash From f4705a41ab3ef6adeeea64065998e6e805062bd2 Mon Sep 17 00:00:00 2001 From: Jenny Yang Yi Date: Tue, 13 May 2025 12:44:26 -0400 Subject: [PATCH 2/2] Update datasets.md --- docs/datasets.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/datasets.md b/docs/datasets.md index 07b951cf..3ba05e0d 100644 --- a/docs/datasets.md +++ b/docs/datasets.md @@ -39,11 +39,13 @@ GuideLLM supports several types of datasets, each with its own advantages and us Synthetic datasets allow you to generate data on the fly with customizable parameters. This is useful for controlled experiments, stress testing, and simulating specific scenarios. For example, you might want to evaluate how a model handles long prompts or generates outputs with specific characteristics. -For different use cases, here are the recommendeded dataset profiles that can be passed in as arguments in GuideLLM: -- Chat: --data "prompt_tokens=512,output_tokens=256" -- RAG: --data "prompt_tokens=4096,output_tokens=512" -- Summarization: --data "prompt_tokens=1024,output_tokens=256" -- Code Generation: --data "prompt_tokens=512,output_tokens=512" +For different use cases, here are the recommended dataset profiles to pass as arguments in GuideLLM: + +- **Chat**: `--data "prompt_tokens=512,output_tokens=256"` +- **RAG**: `--data "prompt_tokens=4096,output_tokens=512"` +- **Summarization**: `--data "prompt_tokens=1024,output_tokens=256"` +- **Code Generation**: `--data "prompt_tokens=512,output_tokens=512"` + #### Example Commands