feat(deepagents): support DeepAgents instrumentation by sipercai · Pull Request #228 · alibaba/loongsuite-python

sipercai · 2026-06-24T09:08:44Z

Description

Adds DeepAgents instrumentation coverage and LangChain tracer support for DeepAgents skill loading. DeepAgents exposes skill loading as a builtin filesystem tool call, read_file(file_path="/skills/<skill>/SKILL.md"), so this change annotates matching tool spans with skill metadata captured from SkillsMiddleware.

This also maps DeepAgents model nodes to ReAct STEP spans, keeps LangGraph/DeepAgents ReAct step handling separated when both frameworks are installed, and captures OpenAI-style DeepAgents/LangGraph root input messages on the root AGENT span.

Fixes # (none)

Type of change

New feature (non-breaking change which adds functionality)
This change requires a documentation update

How Has This Been Tested?

Does This PR Require a Core Repo Change?

No.

Checklist:

Followed the style guidelines of this project
Changelogs have been updated
Unit tests have been added
Documentation has been updated

ralf0131

Summary

Adds DeepAgents instrumentation: a new loongsuite-instrumentation-deepagents package that patches create_deep_agent to inject ReAct metadata, plus LangChain tracer enhancements for DeepAgents model-node STEP spans and read_file skill loading. Overall code quality is high — follows existing LangGraph patterns, comprehensive test coverage (470-line span tests + instrumentor lifecycle tests), CLA signed.

Findings

[Warning] pyproject.toml:26 — requires-python = ">=3.10" should be >=3.11; deepagents 0.6.x needs Python 3.11+, causing CI failure on the 3.10 test job. Remove the 3.10 classifier too.
[Info] patch.py:42 — Duplicated metadata key constants vs langchain/internal/_utils.py; a silent break if one side changes.
[Info] _tracer.py:621 — _extract_tool_call_arguments behavioral change: structured dict inputs now preserved as-is instead of JSON-serialized. Improvement, but changes span data shape for existing tool calls.

Suggestions

Fix requires-python to >=3.11 and drop the 3.10 classifier to resolve the CI failure.
Consider importing metadata key constants from _utils.py (or a shared module) instead of duplicating them, to prevent silent drift.
The _extract_tool_call_arguments refactor is well-motivated and tested — no action needed, just ensure downstream span consumers handle dict-typed tool_call_arguments.

Automated review by github-manager-bot

ralf0131

Summary

Re-review following commit 0ae1399a ("fix(deepagents): align python version matrix"). The previous [Warning] about requires-python >= 3.10 is fully resolved — updated to >=3.11 in pyproject.toml, the 3.10 classifier removed, tox-loongsuite.ini envlist updated to py3{11,12,13}, and README compatibility updated to "Python 3.11+". No source code changes in this commit — only build/CI config alignment.

The two [Info] findings from the previous review remain non-blocking:

Metadata key constant duplication in patch.py vs _utils.py — DRY suggestion, no functional impact.
_extract_tool_call_arguments behavioral change (structured dict preserved as-is) — improvement, well-tested, just a downstream-consumer awareness note.

CLA signed. LGTM — ready to merge.

Automated review by github-manager-bot

Copilot

Pull request overview

Adds first-class DeepAgents instrumentation and extends the LangChain tracer to recognize DeepAgents ReAct runs, STEP spans, and skill-loading tool calls (annotating read_file tool spans with gen_ai.skill.* attributes). This fits into the existing instrumentation-loongsuite suite by enabling consistent AGENT/STEP/TOOL/LLM telemetry for DeepAgents workflows, while keeping LangGraph and DeepAgents ReAct handling distinct.

Changes:

Introduces a new loongsuite-instrumentation-deepagents package (patching create_deep_agent, instrumentor lifecycle, docs, and tests).
Extends loongsuite-instrumentation-langchain tracer to: (1) preserve structured tool call arguments and (2) detect DeepAgents skills metadata and apply skill attributes to read_file tool spans.
Wires DeepAgents into tox + GitHub Actions job matrices and adds LangChain changelog coverage.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tox-loongsuite.ini	Adds DeepAgents tox test + lint environments and dependency wiring.
loongsuite-distro/src/loongsuite/distro/bootstrap_gen.py	Adds DeepAgents to distro bootstrap instrumentation mapping.
instrumentation-loongsuite/loongsuite-instrumentation-langchain/tests/test_data_extraction.py	Adds unit tests for new extraction helpers.
instrumentation-loongsuite/loongsuite-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/internal/_utils.py	Adds DeepAgents metadata key + STEP node constant + detection helper.
instrumentation-loongsuite/loongsuite-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/internal/_tracer.py	Implements DeepAgents ReAct routing + skill metadata caching + tool span skill attributes + structured tool args.
instrumentation-loongsuite/loongsuite-instrumentation-langchain/CHANGELOG.md	Documents DeepAgents skill-load detection + structured tool args preservation.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/tests/test_instrumentor.py	Validates DeepAgents instrumentor dependency (un)instrument behavior.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/tests/test_deepagents_spans.py	Integration tests for AGENT/STEP spans and skill-load tool attribution.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/tests/requirements.latest.txt	Adds DeepAgents test dependencies.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/tests/conftest.py	Test fixtures for in-memory exporters + DeepAgents instrumentation.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/src/opentelemetry/instrumentation/deepagents/version.py	Introduces DeepAgents package version module.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/src/opentelemetry/instrumentation/deepagents/package.py	Declares instrumented library constraints.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/src/opentelemetry/instrumentation/deepagents/internal/patch.py	Patches `create_deep_agent` and injects metadata into run configs.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/src/opentelemetry/instrumentation/deepagents/internal/init.py	Adds internal package marker.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/src/opentelemetry/instrumentation/deepagents/init.py	Implements `DeepAgentsInstrumentor` and dependency instrumentation.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/README.md	Documents installation/usage and telemetry behavior.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/pyproject.toml	Adds packaging metadata, deps, and OTEL entrypoint.
instrumentation-loongsuite/loongsuite-instrumentation-deepagents/CHANGELOG.md	Adds initial DeepAgents instrumentation changelog.
instrumentation-genai/opentelemetry-instrumentation-vertexai/src/opentelemetry/instrumentation/vertexai/patch.py	Removes a couple of Pyright `type: ignore` suppressions.
.github/workflows/loongsuite_test_0.yml	Adds DeepAgents test jobs to generated workflow matrix.
.github/workflows/loongsuite_lint_0.yml	Adds DeepAgents lint job to generated workflow matrix.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ralf0131

Summary

Re-review after new commits. Adds a new loongsuite-instrumentation-deepagents package and integrates DeepAgents skill-loading and ReAct-step tracing into the LangChain tracer. The graph-marker/with_config re-marking approach mirrors LangGraph's metadata handoff cleanly, the ReAct STEP rules are correctly separated per framework, and test coverage is thorough. Findings below are all minor hardening/maintainability notes; nothing blocking. CI (license/cla) is passing; some claw-eval jobs were still pending at review time.

Suggestions

Cap or document the _strong_wrapped_graphs fallback list to avoid unbounded growth in long-running processes.
Verify downstream OTLP serialization handles the now-structured tool_call_arguments (dict/list) without double-encoding.
Extract the SkillsMiddleware.before_agent node name to a constant so a rename is easy to track.

Cross-repo Note

The vertexai changes here are pure type: ignore cleanup — no behavioral change. No shared protocol surface with loongsuite-pilot.

Automated review by github-manager-bot

ralf0131 · 2026-06-25T04:42:11Z

+    try:
+        _wrapped_graphs.add(graph)
+    except TypeError:
+        _strong_wrapped_graphs.append(graph)


[Info] _strong_wrapped_graphs is a module-level list that holds strong references to graphs that cannot be weak-referenced. In long-running processes that create many such graphs this grows without bound. Most LangChain graphs are weak-referenceable so this is rarely hit, but consider capping the list size or documenting the expectation. Non-blocking.

ralf0131 · 2026-06-25T04:42:11Z

        return False


+def _extract_tool_call_arguments(inputs: Any) -> Any:


[Info] _extract_tool_call_arguments now returns the raw dict/list value instead of JSON-serializing to a string (the intentional improvement noted in the diff). Worth confirming that downstream consumers — OTLP attribute serialization and any gen_ai.tool.call.arguments redaction — consistently handle non-string values so structured inputs are not double-encoded or dropped.

ralf0131 · 2026-06-25T04:42:11Z

+        """Cache SkillsMiddleware metadata on the parent DeepAgents run."""
+        if not _has_deepagents_metadata(run):
+            return
+        if getattr(run, "name", "") != "SkillsMiddleware.before_agent":


[Info] The hardcoded node name "SkillsMiddleware.before_agent" couples this instrumentation to a specific DeepAgents internal name. If DeepAgents renames the node, skill-metadata capture degrades silently. Consider extracting it to a module-level constant for easier maintenance and discoverability.

sipercai · 2026-06-25T06:39:59Z

Thanks for the re-review. I agree these are useful hardening notes, and I’m keeping them non-blocking for this PR to avoid expanding the scope after CI is green.

_strong_wrapped_graphs: this fallback is only for graph objects that cannot be weak-referenced; the common LangChain graph path uses weakrefs. A cap or documentation note can be handled as a follow-up hardening item if we see this in practice.
Structured tool_call_arguments: preserving dict/list values is intentional for DeepAgents read_file skill loading, and the PR validation already covers the serialization path with focused tests and real OTLP export evidence.
SkillsMiddleware.before_agent: agreed that a constant would improve maintainability. The current literal matches the DeepAgents node name covered by the integration tests, and I would keep that cleanup as a follow-up rather than changing this PR again.

No additional code changes planned from these non-blocking comments.

github-actions Bot assigned 123liuziming, Cirilla-zmh and ralf0131 Jun 24, 2026

sipercai and others added 12 commits June 24, 2026 18:15

feat(deepagents): add root agent instrumentation

effaf5a

docs(deepagents): clarify manual instrumentation order

4af6f06

chore(deepagents): use loongsuite util genai for direct installs

6b5ad37

feat(deepagents): map model nodes to react steps

f9e7bdc

feat(deepagents): capture skill load attributes

f799020

chore(deepagents): satisfy PR readiness checks

238ab3d

style(deepagents): apply precommit formatting

385f7c5

fix(deepagents): uninstrument managed dependencies

875f970

test(deepagents): cover structured tool arguments

a456f57

perf(langchain): route react step entry by framework

2d51b6c

fix(deepagents): restore util package source metadata

30f0a4c

fix(deepagents): include core deps for lint env

974b460

sipercai force-pushed the liuyu/simple-deepagents-instrumentation branch from 2a30d21 to 974b460 Compare June 24, 2026 10:21

sipercai changed the title ~~feat(deepagents): add skill load telemetry~~ feat(deepagents): support DeepAgents instrumentation Jun 24, 2026

sipercai marked this pull request as ready for review June 24, 2026 10:22

github-actions Bot requested review from 123liuziming, Cirilla-zmh and ralf0131 June 24, 2026 10:22

fix(vertexai): remove stale pyright ignores

d270136

ralf0131 reviewed Jun 24, 2026

View reviewed changes

fix(deepagents): align python version matrix

0ae1399

ralf0131 approved these changes Jun 24, 2026

View reviewed changes

ralf0131 requested a review from Copilot June 24, 2026 15:27

Copilot started reviewing on behalf of ralf0131 June 24, 2026 15:28 View session

Copilot AI reviewed Jun 24, 2026

View reviewed changes

Comment thread loongsuite-distro/src/loongsuite/distro/bootstrap_gen.py

sipercai added 2 commits June 25, 2026 09:59

fix(deepagents): align bootstrap pin version

8c70103

fix(deepagents): capture root agent input messages

8aa32ec

style(deepagents): apply ruff formatting

66ac6cd

ralf0131 reviewed Jun 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(deepagents): support DeepAgents instrumentation#228

feat(deepagents): support DeepAgents instrumentation#228
sipercai wants to merge 17 commits into
mainfrom
liuyu/simple-deepagents-instrumentation

sipercai commented Jun 24, 2026 •

edited

Loading

Uh oh!

ralf0131 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ralf0131 left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

ralf0131 left a comment

Uh oh!

ralf0131 Jun 25, 2026

Uh oh!

ralf0131 Jun 25, 2026

Uh oh!

ralf0131 Jun 25, 2026

Uh oh!

sipercai commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		return False


		def _extract_tool_call_arguments(inputs: Any) -> Any:

Uh oh!

Conversation

sipercai commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

How Has This Been Tested?

Does This PR Require a Core Repo Change?

Checklist:

Uh oh!

ralf0131 left a comment

Choose a reason for hiding this comment

Summary

Findings

Suggestions

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ralf0131 left a comment

Choose a reason for hiding this comment

Summary

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

ralf0131 left a comment

Choose a reason for hiding this comment

Summary

Suggestions

Cross-repo Note

Uh oh!

ralf0131 Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

ralf0131 Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

ralf0131 Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

sipercai commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sipercai commented Jun 24, 2026 •

edited

Loading