Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 62 additions & 5 deletions Sensor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,12 @@ its `server_name` populated.
│ Ingest → Filter → Display → Export │
└─────────────────────────────┬───────────────────────────────────┘
┌───────┴───────┐
▼ ▼
JSON/JSONL Your Detection
Export Pipeline / SIEM
┌────┴────┐
▼ ▼
JSON/JSONL OTLP Logs
Files │
Collector / SIEM
```

## Quick Start
Expand All @@ -109,6 +111,12 @@ Tagged releases are installed from [PyPI](https://pypi.org/project/adr-sensor/):
pip install adr-sensor
```

Install the optional OpenTelemetry dependencies when OTLP log export is needed:

```bash
pip install "adr-sensor[otel]"
```

Or install from source:

```bash
Expand Down Expand Up @@ -141,6 +149,12 @@ adr-sensor --all-history

# Custom output directory
adr-sensor --output-dir ./my-output

# Export the same records to an OTLP/HTTP logs endpoint
adr-sensor --otel-config ./opentelemetry-config.json

# Export to OTLP without also writing JSON files
adr-sensor --no-save --otel-config ./opentelemetry-config.json
```

Sources whose agent only runs on some operating systems are skipped automatically
Expand Down Expand Up @@ -178,6 +192,45 @@ for event in events:
print(f" Args: {tool.arguments}")
```

### OpenTelemetry Logs Export

OpenTelemetry export is disabled by default. The Sensor only initializes an
OTLP exporter when `--otel-config` points to a JSON configuration file. Without
that argument, CLI and file-export behavior are unchanged and no OpenTelemetry
logs are sent.

Start from [`examples/opentelemetry-config.json`](examples/opentelemetry-config.json):

```json
{
"endpoint": "http://localhost:4318/v1/logs",
"service_name": "adr-sensor",
"headers": {},
"timeout_seconds": 10,
"flush_timeout_seconds": 30
}
```

`endpoint` must be the complete OTLP/HTTP logs URL, including `/v1/logs` when
required by the receiver. `headers` can contain authentication headers. An
optional `certificate_file` names a PEM certificate bundle; relative paths are
resolved from the configuration file's directory.

Each `AgentEvent` is sent as an `adr.agent.session` OpenTelemetry LogRecord. Its
body is the complete dictionary returned by `AgentEvent.get_non_null_fields()`,
the same content written to JSON/JSONL today. The OpenTelemetry exporter applies
no redaction or field projection, so prompts, responses, tool arguments, tool
results, usernames, hostnames, and local paths can be transmitted. Any
normalization already performed by a source parser still applies.

System-configuration records are sent as `adr.system.configuration` logs. Runs
are not checkpointed specifically for OTLP: repeated runs can resend the same
records, and consumers can use `adr.event.uuid` to deduplicate them.

The one-shot Sensor process flushes and shuts down the exporter before exiting.
Use an OpenTelemetry Collector when vendor-specific routing, transformation,
retry, or persistent queuing is needed.

## Output Schema

### AgentEvent
Expand Down Expand Up @@ -337,7 +390,7 @@ builds its `--source` choices from `SOURCES`, so it picks the new agent up for f
| --------------- | ------------------------------------------- |
| Python | 3.9, 3.10, 3.11, 3.12, 3.13 |
| Operating system| macOS, Linux, Windows |
| Dependencies | `tabulate` (runtime only — no native deps) |
| Dependencies | `tabulate`; OpenTelemetry is an optional `otel` extra |

Which sources yield data depends on the host OS and on which agents are installed;
see the platform column in [Supported AI Agents](#supported-ai-agents). Sources that
Expand Down Expand Up @@ -392,6 +445,9 @@ adr-sensor/
│ ├── __init__.py # Package exports
│ ├── cli.py # CLI entry point
│ ├── observer.py # AgentObserver orchestrator
│ ├── exporters/
│ │ ├── config.py # OTLP/HTTP JSON configuration
│ │ └── opentelemetry.py # OpenTelemetry Logs exporter
│ ├── parsers/
│ │ ├── base_parser.py # Abstract base class
│ │ ├── claude_parser.py
Expand All @@ -409,6 +465,7 @@ adr-sensor/
│ └── timestamp_utils.py
├── tests/
├── examples/
│ └── opentelemetry-config.json
├── CONTRIBUTING.md
├── LICENSE
├── pyproject.toml
Expand Down
34 changes: 31 additions & 3 deletions Sensor/adr_sensor/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import json
import os
import platform
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
Expand All @@ -23,6 +24,8 @@
resource_mod = None

from . import __version__
from .exporters import OpenTelemetryConfigError, load_opentelemetry_config
from .exporters.opentelemetry import OpenTelemetryExportError, OpenTelemetryLogExporter
from .observer import AgentObserver


Expand Down Expand Up @@ -53,6 +56,7 @@ def main():
adr-sensor --source opencode Ingest opencode logs only
adr-sensor --save-sessions Save individual session files
adr-sensor --output-format jsonl Export as JSONL
adr-sensor --otel-config ./otel.json Export logs to an OTLP/HTTP endpoint
adr-sensor --all-history Include all logs (not just last 2 weeks)
""",
)
Expand Down Expand Up @@ -92,9 +96,22 @@ def main():
action="store_true",
help="Include all event logs regardless of age (default: last 2 weeks)",
)
parser.add_argument(
"--otel-config",
type=Path,
default=None,
help="JSON configuration for OTLP/HTTP log export (disabled when omitted)",
)

args = parser.parse_args()

otel_config = None
if args.otel_config is not None:
try:
otel_config = load_opentelemetry_config(args.otel_config)
except OpenTelemetryConfigError as exc:
parser.error(str(exc))

host_os = platform.system()
capture_resource = bool(args.resource and host_os != "Windows" and resource_mod is not None)

Expand Down Expand Up @@ -139,9 +156,7 @@ def main():
if args.save_sessions:
if entries:
saved_files = observer.save_sessions_to_individual_files(entries, output_dir=args.output_dir)
print(
f"\nSession files saved to: {saved_files[0].parent if saved_files else 'No files saved'}"
)
print(f"\nSession files saved to: {saved_files[0].parent if saved_files else 'No files saved'}")
else:
if args.output_dir is None:
project_output_dir = Path.cwd() / "output"
Expand All @@ -153,8 +168,21 @@ def main():
entries, system_config_data, output_format=args.output_format, output_dir=project_output_dir
)

if otel_config is not None:
otel_exporter = OpenTelemetryLogExporter(otel_config, service_version=get_version())
try:
exported_count = otel_exporter.export(entries, system_config_data)
finally:
otel_exporter.shutdown()
print(f"\nOpenTelemetry logs sent: {exported_count}")

print("\nADR Sensor complete!\n")

except OpenTelemetryExportError as exc:
success = False
print(f"OpenTelemetry export failed: {exc}", file=sys.stderr)
raise SystemExit(1)

except BaseException:
success = False
raise
Expand Down
9 changes: 9 additions & 0 deletions Sensor/adr_sensor/exporters/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""Outbound telemetry exporters for ADR Sensor."""

from .config import OpenTelemetryConfig, OpenTelemetryConfigError, load_opentelemetry_config

__all__ = [
"OpenTelemetryConfig",
"OpenTelemetryConfigError",
"load_opentelemetry_config",
]
114 changes: 114 additions & 0 deletions Sensor/adr_sensor/exporters/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
"""Configuration loading for the optional OpenTelemetry exporter."""

import json
import math
from dataclasses import dataclass, field
from pathlib import Path
from typing import Dict, Optional
from urllib.parse import urlparse

MAX_CONFIG_SIZE = 1024 * 1024
_CONFIG_KEYS = {
"endpoint",
"service_name",
"headers",
"timeout_seconds",
"flush_timeout_seconds",
"certificate_file",
}


class OpenTelemetryConfigError(ValueError):
"""Raised when an OpenTelemetry configuration file is invalid."""


@dataclass(frozen=True)
class OpenTelemetryConfig:
"""Settings used to send ADR records to an OTLP/HTTP logs endpoint."""

endpoint: str
service_name: str = "adr-sensor"
headers: Dict[str, str] = field(default_factory=dict)
timeout_seconds: float = 10.0
flush_timeout_seconds: float = 30.0
certificate_file: Optional[str] = None


def load_opentelemetry_config(path: Path) -> OpenTelemetryConfig:
"""Load and validate an OTLP/HTTP exporter configuration from JSON."""
config_path = Path(path)
try:
with open(config_path, "rb") as config_file:
raw = config_file.read(MAX_CONFIG_SIZE + 1)
except OSError as exc:
raise OpenTelemetryConfigError(f"cannot read OpenTelemetry config {config_path}: {exc}") from exc

if len(raw) > MAX_CONFIG_SIZE:
raise OpenTelemetryConfigError(f"OpenTelemetry config exceeds {MAX_CONFIG_SIZE} bytes")

try:
document = json.loads(raw)
except (json.JSONDecodeError, UnicodeDecodeError) as exc:
raise OpenTelemetryConfigError(f"OpenTelemetry config is not valid JSON: {exc}") from exc

if not isinstance(document, dict):
raise OpenTelemetryConfigError("OpenTelemetry config root must be an object")

unexpected = sorted(set(document) - _CONFIG_KEYS)
if unexpected:
raise OpenTelemetryConfigError(f"unknown OpenTelemetry config field(s): {', '.join(unexpected)}")

endpoint = _required_string(document, "endpoint")
parsed_endpoint = urlparse(endpoint)
if parsed_endpoint.scheme not in ("http", "https") or not parsed_endpoint.netloc:
raise OpenTelemetryConfigError("OpenTelemetry endpoint must be an absolute HTTP(S) URL")

service_name = _optional_string(document, "service_name", "adr-sensor")
timeout_seconds = _positive_number(document, "timeout_seconds", 10.0)
flush_timeout_seconds = _positive_number(document, "flush_timeout_seconds", 30.0)

headers_value = document.get("headers", {})
if not isinstance(headers_value, dict) or not all(
isinstance(key, str) and key and isinstance(value, str) for key, value in headers_value.items()
):
raise OpenTelemetryConfigError("OpenTelemetry headers must be an object with non-empty string keys and values")

certificate_file = document.get("certificate_file")
if certificate_file is not None:
if not isinstance(certificate_file, str) or not certificate_file.strip():
raise OpenTelemetryConfigError("OpenTelemetry certificate_file must be a non-empty string")
certificate_path = Path(certificate_file)
if not certificate_path.is_absolute():
certificate_path = config_path.parent / certificate_path
certificate_file = str(certificate_path.resolve())
if not Path(certificate_file).is_file():
raise OpenTelemetryConfigError(f"OpenTelemetry certificate file does not exist: {certificate_file}")

return OpenTelemetryConfig(
endpoint=endpoint,
service_name=service_name,
headers=dict(headers_value),
timeout_seconds=timeout_seconds,
flush_timeout_seconds=flush_timeout_seconds,
certificate_file=certificate_file,
)


def _required_string(document: dict, key: str) -> str:
value = document.get(key)
if not isinstance(value, str) or not value.strip():
raise OpenTelemetryConfigError(f"OpenTelemetry {key} must be a non-empty string")
return value.strip()


def _optional_string(document: dict, key: str, default: str) -> str:
if key not in document:
return default
return _required_string(document, key)


def _positive_number(document: dict, key: str, default: float) -> float:
value = document.get(key, default)
if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value) or value <= 0:
raise OpenTelemetryConfigError(f"OpenTelemetry {key} must be a positive number")
return float(value)
Loading