Revised Examples For Conversation Visualizer #1091
Conversation
… tracking - Show how to build stateful visualizers with incremental metrics display - Demonstrate tracking LLM costs and token usage from conversation stats - Teach handling multiple event types (ActionEvent, MessageEvent, AgentErrorEvent) - Example extracting command and path details from tool action events
…ples Split the custom visualizer example into two files: - 26_custom_visualizer.py: Simple visualizer showing only latency - 27_detailed_llm_metrics.py: Advanced visualizer with latency, cost, and token tracking Changes: - Simplified 26 to only display latency (removed cost/token tracking) - Created 27 with MetricsCache class for efficient metrics lookup - Improved formatting with right-aligned step numbers and event type labels - Renamed class in 27 from SimpleVisualizer to TokenCountingVisualizer - Added documentation explaining SDK metrics structure and cost matching The MetricsCache class demonstrates how to match costs to response_ids using ResponseLatency index, since Cost objects don't have response_id fields.
|
Looks like there are a few issues preventing this PR from being merged!
If you'd like me to help, just leave a comment, like Feel free to include any additional details that might help me get this PR into a better state. You can manage your notification settings |
There was a problem hiding this comment.
Can we combine both examples? Or we should mention both examples in the docs
There was a problem hiding this comment.
We could use just the 27 example if you don't think it is too complex. I split them only so that there would be a simpler example first but I did that largely because I assumed you'd want that : ) If we do have two examples we could have doc for each one that focusses on the specific stuff we showcased in each. For the second it's largely about working with metrics and including them in visualization.
| func._handles_event_type = event_type | ||
| return func | ||
|
|
||
| return decorator |
There was a problem hiding this comment.
I actually like this pattern, do you want to directly integrate this syntax into the default visualizer and/or even base class?
There was a problem hiding this comment.
Yes, seems very nice! Could we name it "@visualize" ?
Our existing example for Conversation Visualizer demonstrates swapping in a new Visualizer but also doesn't orient the reader to the aspects of the SDK they are likely to need in order to produce a useful visualization. This PR updates the existing visualization with a more realistic if simple example and adds another which offers a glimpse into a more realistic scenario for working with metrics. (A reviewer of the example we previously merged had requested that we showcase that.)
Agent Server images for this PR
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdknikolaik/python-nodejs:python3.12-nodejs22golang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:b140b1d-pythonRun
All tags pushed for this build
About Multi-Architecture Support
b140b1d-python) is a multi-arch manifest supporting both amd64 and arm64b140b1d-python-amd64) are also available if needed