Skip to content

docs: refactor telemetry docs into dedicated tracing, metrics, and logging pages#662

Open
ajbozarth wants to merge 1 commit intogenerative-computing:mainfrom
ajbozarth:docs/refactor-telemetry
Open

docs: refactor telemetry docs into dedicated tracing, metrics, and logging pages#662
ajbozarth wants to merge 1 commit intogenerative-computing:mainfrom
ajbozarth:docs/refactor-telemetry

Conversation

@ajbozarth
Copy link
Contributor

@ajbozarth ajbozarth commented Mar 16, 2026

Misc PR

Type of PR

  • Bug Fix
  • New Feature
  • Documentation
  • Other

Description

Refactor the telemetry documentation from two pages into a four-page structure with dedicated pages for each telemetry pillar. Absorbs all content from the internal dev doc (docs/dev/telemetry.md) into public docs, then deletes it.

New structure:

  • Telemetry (Overview) — env var reference, quick start, balanced summaries
  • Tracing — span attributes, exporter setup (Jaeger, Tempo), debugging guidance
  • Metrics — token usage, backend support matrix, Console/OTLP/Prometheus exporters, custom metrics API, troubleshooting
  • Logging — FancyLogger console logging, OTLP log export, get_otlp_log_handler() API

Other changes:

  • Move "Handling Exceptions" from Evaluation and Observability to How-To section
  • Reorder sidebar: LLM-as-a-Judge first, then Telemetry subgroup
  • Fix mint devmintlify dev in CONTRIBUTING.md and README.md
  • Update all cross-references (10+ files)

Note: The metrics and token usage documentation reflects the hooks/plugins architecture introduced in #653. Merge #653 first.

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code as added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

…gging pages

Absorb all content from docs/dev/telemetry.md into the public docs and
restructure the Evaluation and Observability section:

- Replace metrics-and-telemetry.md with telemetry.md (balanced overview hub)
- Replace opentelemetry-tracing.md with tracing.md (add Mellea-specific
  backend span attributes, gen_ai.response.model)
- Create metrics.md (token usage, backend support matrix, three exporters,
  custom metrics API, programmatic access, troubleshooting)
- Create logging.md (FancyLogger console logging, OTLP log export,
  get_otlp_log_handler() API, collector setup)
- Move handling-exceptions.md from eval-observability to how-to section
- Reorder sidebar: LLM-as-a-Judge first, then Telemetry subgroup
- Delete docs/dev/telemetry.md (all content migrated)
- Update all cross-references across 10+ files
- Fix mintlify dev command in CONTRIBUTING.md and README.md

Signed-off-by: Alex Bozarth <ajbozart@us.ibm.com>
@ajbozarth ajbozarth self-assigned this Mar 16, 2026
@ajbozarth ajbozarth requested a review from a team as a code owner March 16, 2026 23:04
@github-actions
Copy link
Contributor

The PR description has been updated. Please fill out the template for your PR to be reviewed.

@mergify
Copy link

mergify bot commented Mar 16, 2026

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

@ajbozarth ajbozarth changed the title docs: refactor telemetry docs into dedicated tracing, metrics, and lo… docs: refactor telemetry docs into dedicated tracing, metrics, and logging pages Mar 16, 2026
@ajbozarth ajbozarth requested a review from planetf1 March 16, 2026 23:12
Copy link
Contributor

@planetf1 planetf1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great addition!

  • just spotted a broken link
  • obv need the PR merged for this to be accurate
  • consider if you want to add any glossary entries. Nothing particularly stood out to me
  • there's a broken link in an orphan index.md (used for a static index before the mintlify pipeline but I'm removing in #645


For structured telemetry across all calls, see
[Metrics and Telemetry](./metrics-and-telemetry).
[Telemetry](./telemetry).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[Telemetry](./telemetry).
[Telemetry](../evaluation-and-observability/telemetry).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed as this doc was moved into howtos, so now the telemetry is back in the other directory

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: Migrate comprehensive telemetry documentation to published docs

2 participants