Skip to content

fix: constrain Hermes Relay compatibility - #206

Merged
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
zhongxuanwang-nv:fix/hermes-relay-version-compatibility
Aug 11, 2026
Merged

fix: constrain Hermes Relay compatibility#206
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
zhongxuanwang-nv:fix/hermes-relay-version-compatibility

Conversation

@zhongxuanwang-nv

@zhongxuanwang-nv zhongxuanwang-nv commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Overview

Constrain the Hermes adapter's relay and full extras to NeMo Relay 0.6.x.

A live Hermes Agent 0.19.0 + NeMo Relay 0.7.2 run returned the correct model response, but generated no Relay artifacts. Relay 0.7 rejected Hermes' generated v3 plugin configuration; Hermes failed open, so the integration looked successful while exporting no telemetry. Hermes 0.19.0 is the only published compatible Hermes Agent release, and the supported Relay 0.6.0 combination produces both ATOF and ATIF artifacts in a fresh live run.

This aligns the Hermes leaf adapter with the root project's existing nemo-relay>=0.6.0,<0.7 constraint. It is intentionally limited to dependency metadata, the regenerated lock, and its metadata expectation; no runtime code changes.

Details

  • Change nemo-relay>=0.6.0,<0.8 to nemo-relay>=0.6.0,<0.7 for both relay and full extras.
  • Regenerate the Hermes adapter lockfile, resolving NeMo Relay 0.6.0.
  • Update the adapter metadata assertion.
  • Relay 0.7 support can be restored when a released Hermes Agent version supports its configuration contract and passes live end-to-end export validation.
Exact Relay 0.7.2 configuration and responses

This is the exact plugins.toml emitted by the Hermes adapter in a clean Relay 0.7.2 reproduction of the ATOF+ATIF configuration (including its emitted local paths):

version = 1

[[components]]
kind = "observability"
enabled = true

[components.config]
version = 3

[components.config.atif]
enabled = true
output_directory = "/private/tmp/nemo-fabric-hermes-relay07-pr.EoDqG8/artifacts/relay/relay07-pr-repro"
filename_template = "trajectory-{session_id}.atif.json"
agent_name = "code-review-agent"
agent_version = "fabric-sdk-example"
model_name = "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning"

[components.config.atof]
enabled = true

[[components.config.atof.sinks]]
type = "file"
output_directory = "/private/tmp/nemo-fabric-hermes-relay07-pr.EoDqG8/artifacts/relay/relay07-pr-repro"
filename = "events.atof.jsonl"
mode = "append"

Relay's plugin.validate() response for that parsed TOML was exactly:

{"diagnostics":[]}

Relay's plugin.initialize() response for the same parsed TOML was exactly:

RuntimeError: invalid config: OpenTelemetry mark_projection moved into each typed endpoint in observability config version 3; OpenTelemetry transport moved into each typed endpoint in observability config version 3; OpenTelemetry endpoint moved into each typed endpoint in observability config version 3; OpenTelemetry headers moved into each typed endpoint in observability config version 3; OpenTelemetry resource_attributes moved into each typed endpoint in observability config version 3; OpenTelemetry service_name moved into each typed endpoint in observability config version 3; OpenTelemetry service_namespace moved into each typed endpoint in observability config version 3; OpenTelemetry instrumentation_scope moved into each typed endpoint in observability config version 3; OpenTelemetry timeout_millis moved into each typed endpoint in observability config version 3; enabled OpenTelemetry section requires at least one endpoint

The original live Hermes Agent 0.19.0 + Relay 0.7.2 model run returned:

input: Reply with exactly: RELAY_ZERO_SEVEN_OK
status: succeeded
response: RELAY_ZERO_SEVEN_OK
relay_artifacts: []

Validation

  • .venv/bin/python -m pytest -q tests/adapters/test_adapter_package_metadata.py — 18 passed
  • uv lock --check --project adapters/hermes
  • Built the Hermes adapter wheel and verified its relay and full METADATA each require nemo-relay<0.7,>=0.6.0
  • uv run --no-project python scripts/licensing/license_diff.py --base-ref upstream/main — no license changes
  • pre-commit run --all-files attributions-python and attributions-rust
  • Fresh live NVIDIA API smoke with Hermes Agent 0.19.0 + NeMo Relay 0.6.0 — succeeded with 7 ATOF records and both ATOF/ATIF artifact kinds
  • Clean isolated NeMo Relay 0.7.2 reproduction — captured the emitted plugin TOML and full plugin.validate() / plugin.initialize() responses above

Where should the reviewer start?

Start with adapters/hermes/pyproject.toml; adapters/hermes/uv.lock is the corresponding resolution update.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • Relates to feat(hermes): use upstream Relay integration #192

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.

  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
@zhongxuanwang-nv
zhongxuanwang-nv requested a review from a team as a code owner August 11, 2026 19:29
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 49f6b572-f6b3-4fd8-89b1-fcb788798154

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

The Hermes adapter now restricts nemo-relay to versions from 0.6.0 up to, but not including, 0.7. The package metadata test now checks the updated constraint.

Changes

Hermes relay dependency

Layer / File(s) Summary
Dependency bound and metadata validation
adapters/hermes/pyproject.toml, tests/adapters/test_adapter_package_metadata.py
The relay and full optional dependency groups now require nemo-relay>=0.6.0,<0.7. The metadata test expects the same constraint.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

  • NVIDIA/NeMo-Fabric#192: Directly related changes to the Hermes Relay dependency range and its metadata test.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title uses valid Conventional Commits format and clearly describes the dependency compatibility change.
Description check ✅ Passed The description includes the required overview, reviewer starting point, related issue, confirmations, and detailed validation information.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@zhongxuanwang-nv zhongxuanwang-nv self-assigned this Aug 11, 2026
@zhongxuanwang-nv zhongxuanwang-nv added this to the 0.2 milestone Aug 11, 2026
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
@github-actions

Copy link
Copy Markdown

@zhongxuanwang-nv

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit df151a8 into NVIDIA:main Aug 11, 2026
32 checks passed
rapids-bot Bot pushed a commit that referenced this pull request Aug 13, 2026
#### Overview

Upgrade NeMo Fabric to NeMo Relay `>=0.7.2,<0.8` and move the public Relay observability configuration to schema v3.

This is a hard migration: explicit schema-v2 configurations and removed flat OTLP/OpenInference fields are rejected with actionable errors. Configurations without an explicit observability version follow Relay 0.7's implicit-v3 behavior.

Relay 0.7.2 is the narrowest supported dependency range for this contract. Retaining Relay 0.6 would preserve two configuration models across adapters and retain the Hermes compatibility constraint introduced in #206. The reviewed license delta is limited to `nemo-relay` changing from 0.6.0 to 0.7.2 and the addition of `nemo-relay-cli-bin` 0.7.2; both packages are Apache-2.0, with no new copyleft or unresolved licenses.

Claude and Codex harness installations now depend directly on the supported `nemo-relay-cli-bin` distribution, so Platform Evaluator environments receive the version-matched gateway executable without the unused Relay Python package or separate Cargo/shell provisioning. Bare adapter installations remain unchanged.

The Python SDK now requires Pydantic 2.12 or newer because the Relay v3 models use `Field(exclude_if=...)`; older Pydantic releases accept that metadata but do not apply it during serialization.

This branch now incorporates merged #197's Claude v1alpha2 adapter contract. The conflict resolution preserves typed `AgentConfig` input while retaining Relay 0.7 validation and schema-v3 configuration.

This PR supersedes #215. The branch moved from a fork to `NVIDIA/NeMo-Fabric` so the required NVSkills validation can run.

#### Details

- Replace the Relay observability v2 model with Relay 0.7's native v3 endpoint model across the Rust schema, Python SDK, and generated references.
- Remove v2 rendering and version-negotiation code from the shared Relay adapter path.
- Apply the v3 contract consistently across Claude, Codex, Deep Agents, and Hermes.
- Reject explicit v2, malformed OpenTelemetry containers/endpoints, and legacy flat exporter fields before adapter setup or TOML materialization.
- Update dependency metadata, lockfiles, attributions, examples, and integration guidance.
- Package the standalone Relay PyPI CLI binary with Claude and Codex harness/full installs and remove separate Cargo CLI provisioning from CI.
- Preserve the top-level Relay plugin configuration at version 1; only the observability component uses schema version 3.

#### Validation

- `just test-python` with the packaged NeMo Relay 0.7.2 CLI - 1,067 passed, 16 skipped
- Focused cross-layer regression matrix before the packaging-only follow-up - 298 passed
- `cargo fmt --all -- --check`
- `just test-rust` - 117 passed
- `cargo check -p fabric-python --locked`
- `just test-typescript`
- `just docs`
- Full pre-commit - all 9 hooks passed, including workflow lint, lockfiles, attributions, and dependency-license checks
- Clean, exact Claude and Codex leaf `harness` installs both exposed `nemo-relay 0.7.2` while excluding the `nemo-relay` Python distribution
- Final-head `Runtime.invoke_stream()` checks with mocked inference: Claude and Codex streamed through the packaged CLI - 2 passed
- Final-head `Runtime.invoke_stream()` checks with `nemo-relay` absent from `PATH`: Deep Agents and Hermes streamed through in-process Relay - 2 passed
- Built both leaf wheels and verified `harness` and `full` metadata require `nemo-relay-cli-bin>=0.7.2,<0.8`
- SDK-level AgentEvaluator matrix before the packaging-only follow-up with mocked inference and the actual NeMo Relay 0.7.2 CLI - 30/30: Codex 10/10, Claude 10/10, Deep Agents 5/5, and Hermes 5/5
- Verified 30 unique ATIF directories, paths, and session IDs; all trajectories use `ATIF-v1.7`
- Verified schema v3 in all authored Relay JSON configurations and materialized `plugins.toml` files
- Reviewed the lockfile license delta: `nemo-relay` 0.6.0 to 0.7.2 plus `nemo-relay-cli-bin` 0.7.2; all Apache-2.0

#### Where should the reviewer start?

Start with `crates/fabric-core/src/config.rs` for the public v3 configuration contract and `adapters/common/src/nemo_fabric_adapters/common/relay_gateway.py` for the shared adapter validation boundary. The corresponding compatibility coverage is in `tests/adapters/test_adapters_common_relay_gateway.py` and `tests/python/test_sdk_contract.py`.

For the merged Claude contract, review `adapters/claude/src/nemo_fabric_adapters/claude/adapter.py` and `tests/adapters/test_claude_adapter.py`.

#### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

- Relates to [FABRIC-189: Update NeMo Relay dependency to 0.7](https://linear.app/nvidia/issue/FABRIC-189/update-nemo-relay-dependency-to-07)
- Relates to #197
- Relates to #206
- Relates to #215

- [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license.
- [x] I searched existing issues and open pull requests, and this does not duplicate existing work.


## Summary by CodeRabbit

- **New Features**
  - Added Relay observability schema v3 with typed OpenTelemetry endpoints, projections, mappings, headers, timeouts, and optional full-payload retention.
  - Added support for multiple telemetry endpoints and clearer exporter validation.
  - Relay gateways now use isolated user-scoped configuration.

- **Bug Fixes**
  - Prevented unintended ambient or inherited Relay configuration from affecting runs.
  - Rejected unsupported legacy observability configurations with clearer errors.

- **Documentation**
  - Updated installation, integration, API, and example guidance for Relay CLI versions 0.7.2–0.7.x and schema v3.
  - Clarified which adapter installation options include the Relay CLI.

Authors:
  - Ajay Thorve (https://github.com/AjayThorve)

Approvers:
  - Zhongxuan (Daniel) Wang (https://github.com/zhongxuanwang-nv)
  - David Gardner (https://github.com/dagardner-nv)

URL: #216
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants