Skip to content

Replace various key=value string parsers with a single utility#569

Merged
sjmonson merged 8 commits intomainfrom
feat/arg_parser
May 6, 2026
Merged

Replace various key=value string parsers with a single utility#569
sjmonson merged 8 commits intomainfrom
feat/arg_parser

Conversation

@sjmonson
Copy link
Copy Markdown
Collaborator

@sjmonson sjmonson commented Jan 30, 2026

Summary

Replaces various config string handlers with a new ArgString format.

Details

Replaces the dataset config parser with a more advance "arg_string" parser with support for nested and list config structures. For example:

prefix_buckets[0].bucket_weight=10,prefix_buckets[0].prefix_count=10,prefix_buckets[1].bucket_weight=20,prefix_buckets[1].prefix_count=4

Renders to

{
    "prefix_buckets": [
        {"bucket_weight": 10, "prefix_count": 10},
        {"bucket_weight": 20, "prefix_count": 4},
     ]
}

Also added some glue logic to allow this format anywhere on the CLI that supports JSON. This code should be considered temporary as it will mostly be replaced through the rest of the CLI refactor.

Also open to suggestions for a better name then "arg_string"; a 2-4 letter acronym like JSON or YAML would be good.

Test Plan

The following benchmark should run without errors and have ~384 ISL per request (make sure to check startup logs confirm backend-kwargs and warmup are configured):

guidellm benchmark run \
    --profile concurrent \
    --data "prompt_tokens=256,output_tokens=256,prefix_buckets[0].bucket_weight=10,prefix_buckets[0].prefix_tokens=128,prefix_buckets[0].prefix_count=10" \
    --request-format /v1/completions \
    --backend-kwargs '{"timeout":120}' \
    --warmup percent=0.1,value=20 \
    --max-seconds 30 \
    --rate 30

  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

@sjmonson sjmonson force-pushed the feat/arg_parser branch 2 times, most recently from 2cb80bd to 5d724ce Compare April 15, 2026 17:00
@sjmonson sjmonson force-pushed the feat/arg_parser branch 2 times, most recently from f21b3f0 to 55745f2 Compare April 29, 2026 18:20
sjmonson added 4 commits May 4, 2026 15:52
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
@sjmonson sjmonson force-pushed the feat/arg_parser branch from 55745f2 to 5392826 Compare May 4, 2026 20:03
Copy link
Copy Markdown
Collaborator

@dbutenhof dbutenhof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments from a first pass without getting deep into the algorithms ... but it looks good.

Comment thread src/guidellm/data/config.py
Comment thread src/guidellm/utils/arg_string.py Outdated
Comment thread src/guidellm/utils/arg_string.py Outdated
Comment thread src/guidellm/utils/arg_string.py
@sjmonson sjmonson force-pushed the feat/arg_parser branch from a6fe13e to 0889559 Compare May 5, 2026 18:21
Signed-off-by: Samuel Monson <smonson@redhat.com>
@sjmonson sjmonson force-pushed the feat/arg_parser branch from 0889559 to 5402c02 Compare May 5, 2026 18:36
@sjmonson sjmonson marked this pull request as ready for review May 5, 2026 18:48
Signed-off-by: Samuel Monson <smonson@redhat.com>
dbutenhof
dbutenhof previously approved these changes May 5, 2026
@sjmonson sjmonson self-assigned this May 5, 2026
Signed-off-by: Samuel Monson <smonson@redhat.com>
jaredoconnell
jaredoconnell previously approved these changes May 6, 2026
Copy link
Copy Markdown
Collaborator

@jaredoconnell jaredoconnell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. And basic data args are working for me.

@jaredoconnell
Copy link
Copy Markdown
Collaborator

A test is failing. It looks like maybe you changed the error message under that condition.

Signed-off-by: Samuel Monson <smonson@redhat.com>
Copy link
Copy Markdown
Collaborator

@dbutenhof dbutenhof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. Minor issue: you added the external get/set methods in the arg string parser class, which aren't used, and aren't tested. If we're going to have them, they should be tested.

Comment thread src/guidellm/utils/arg_string.py
@sjmonson sjmonson force-pushed the feat/arg_parser branch from 169b9da to ce8ab03 Compare May 6, 2026 15:40
@sjmonson sjmonson merged commit b6749eb into main May 6, 2026
11 checks passed
@sjmonson sjmonson deleted the feat/arg_parser branch May 6, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants