Affected area
- Rust core runtime
- Documentation or examples
Problem or opportunity
NeMo Relay operational file sinks currently append to the configured file without rotation or retention. The asynchronous queue bounds pending records in memory, but it does not bound the size of the persisted log file.
Long-running Relay processes can therefore create continuously growing files that require external cleanup and may eventually consume available disk space.
Proposed enhancement
Add optional size-based rotation and retention settings to operational file sinks.
Users should be able to configure:
- The maximum size of the active log file.
- The maximum number of rotated files retained.
When the active file reaches the configured size, Relay should rotate it and remove files beyond the retention limit. Existing asynchronous delivery, periodic flushing, stderr logging, formatting, filtering, and root Relay ID behavior should remain unchanged.
When rotation is not configured, existing append-only behavior should remain the default for backward compatibility.
This enhancement applies only to Relay operational file sinks. Remote logging destinations and agent telemetry exporters are separate concerns.
Runtime contract and binding impact
Rotation should be implemented by the core logging runtime so every frontend using the same LoggingConfig and LoggingRuntime receives consistent behavior.
No Python, Node.js, Go, or C FFI API changes are required. Existing processes continue using append-only files unless rotation is explicitly configured.
A rotation failure should use the existing sink-error reporting path to stderr and should not crash an otherwise healthy Relay process.
Alternatives considered
External tools such as logrotate, container logging drivers, or service managers can manage log retention. However, they are not always available for embedded applications, local development, or directly configured Relay file sinks. They may also require coordination with the process's open file handle.
Truncating the file at startup would limit history between runs but would not prevent unbounded growth during a long-running process.
Acceptance criteria
- A file sink can optionally configure a maximum active-file size and retained-file count.
- Reaching the configured size rotates the active file without blocking Relay's operational path.
- Files exceeding the configured retention count are removed.
- Omitting rotation settings preserves the existing append-only behavior.
- Invalid values, including zero or unsafe limits, return clear configuration errors.
- Rotation failures are reported through the existing stderr sink-error path without crashing Relay.
- Focused tests cover rotation, retention, append-only compatibility, asynchronous flushing, and shutdown.
- The operational logging documentation explains the new configuration and behavior.
This extends the append-only sink introduced by #413 without overlapping the operational event instrumentation completed in #455.
Affected area
Problem or opportunity
NeMo Relay operational file sinks currently append to the configured file without rotation or retention. The asynchronous queue bounds pending records in memory, but it does not bound the size of the persisted log file.
Long-running Relay processes can therefore create continuously growing files that require external cleanup and may eventually consume available disk space.
Proposed enhancement
Add optional size-based rotation and retention settings to operational file sinks.
Users should be able to configure:
When the active file reaches the configured size, Relay should rotate it and remove files beyond the retention limit. Existing asynchronous delivery, periodic flushing, stderr logging, formatting, filtering, and root Relay ID behavior should remain unchanged.
When rotation is not configured, existing append-only behavior should remain the default for backward compatibility.
This enhancement applies only to Relay operational file sinks. Remote logging destinations and agent telemetry exporters are separate concerns.
Runtime contract and binding impact
Rotation should be implemented by the core logging runtime so every frontend using the same
LoggingConfigandLoggingRuntimereceives consistent behavior.No Python, Node.js, Go, or C FFI API changes are required. Existing processes continue using append-only files unless rotation is explicitly configured.
A rotation failure should use the existing sink-error reporting path to stderr and should not crash an otherwise healthy Relay process.
Alternatives considered
External tools such as
logrotate, container logging drivers, or service managers can manage log retention. However, they are not always available for embedded applications, local development, or directly configured Relay file sinks. They may also require coordination with the process's open file handle.Truncating the file at startup would limit history between runs but would not prevent unbounded growth during a long-running process.
Acceptance criteria
This extends the append-only sink introduced by #413 without overlapping the operational event instrumentation completed in #455.