Skip to content

Report cumulative vLLM generation progress across batches#179

Open
finbarrtimbers wants to merge 1 commit into
mainfrom
finbarr/vllm-cumulative-progress
Open

Report cumulative vLLM generation progress across batches#179
finbarrtimbers wants to merge 1 commit into
mainfrom
finbarr/vllm-cumulative-progress

Conversation

@finbarrtimbers

Copy link
Copy Markdown
Contributor

Summary

  • Add BeakerStatusReporter.start_accumulator / accumulating_callback so progress can be reported as a single cumulative counter across batches.
  • Wire VLLMServerProvider to use the accumulator when the runner exposes OLMO_VLLM_GEN_TOTAL, falling back to per-batch reporting otherwise.
  • Runner exports OLMO_VLLM_GEN_TOTAL (sum of total_instances * num_samples) and OLMO_EVAL_BENCHMARK so the provider can label progress with the experiment name.

Test plan

  • Small vLLM run; confirm the progress bar reflects cumulative generation count instead of resetting per batch.

🤖 Generated with Claude Code

…-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant