Download miner logs via fleet nodes

[ankitgoswami]

Reviewable diff: +425/-155 across 9 files (excludes generated, test, and story files).

Summary

Operators can now download miner logs from miners paired through fleet nodes using the existing ProtoFleet DownloadLogs and GetCommandBatchLogBundle flow. Fleet nodes collect the logs from their paired miners, upload the CSV as a command artifact, and the server materializes that artifact into the same batch log directory used by direct miners. The browser-facing output remains the existing single CSV or ZIP bundle, with no new client UX or public minercommand.v1 API surface.

How it works

The operator starts the existing download-logs command. For remote-node miners, the server sends a fleet-node DownloadLogsAction containing the batch UUID and registers one expected MINER_LOGS upload for the target device identifier.

The fleet node receives the command, downloads logs from the SDK device, formats them with the same CSV formatter used by direct plugin miners, uploads the CSV through UploadCommandArtifact, and only then acknowledges success. The gateway-side control registry returns completed upload refs before removing the in-flight command, so the remote miner adapter can require the uploaded ref and copy the validated artifact into logs/<batchUUID>/ using the same miner-log filename convention as direct miners.

The existing batch finalizer then bundles whatever files are in the batch directory. Direct-only, fleet-node-only, and mixed batches all keep the current CSV-for-single-device and ZIP-for-multiple-devices behavior.

Diagrams

flowchart LR
  Operator["Operator: DownloadLogs"] --> CommandSvc["Miner command service"]
  CommandSvc --> RemoteMiner["remote-node miner adapter"]
  RemoteMiner --> Control["fleet-node control registry"]
  Control --> FleetNode["server/cmd/fleetnode"]
  FleetNode --> SDK["SDK device DownloadLogs"]
  FleetNode --> Upload["UploadCommandArtifact MINER_LOGS"]
  Upload --> Artifacts["validated command artifact storage"]
  Control --> RemoteMiner
  RemoteMiner --> LogsDir["logs/<batchUUID>/ miner CSV"]
  LogsDir --> Bundle["existing CSV/ZIP bundle finalizer"]
  Bundle --> Browser["GetCommandBatchLogBundle response"]

Loading

sequenceDiagram
  participant Queue as Command worker
  participant Remote as Remote miner
  participant Control as Control registry
  participant Node as Fleet node
  participant Files as Files service

  Queue->>Remote: DownloadLogs(batchUUID)
  Remote->>Control: SendCommandWithArtifactResults(expect MINER_LOGS)
  Control->>Node: DownloadLogsAction(batchUUID)
  Node->>Node: SDK DownloadLogs + CSV format
  Node->>Control: UploadCommandArtifact(command_id, purpose, device_id)
  Node->>Control: Ack OK after upload succeeds
  Control-->>Remote: Ack OK + artifact ref
  Remote->>Files: SaveCommandArtifactLog(batchUUID, mac, artifactID)
  Files-->>Queue: CSV ready for existing bundle finalizer

Loading

Areas of the code involved

Area / package / file	What changed	Why it matters for review
proto/fleetnodegateway/v1	Adds DownloadLogsAction to the internal fleet-node command envelope	Defines the control-plane payload fleet nodes execute; generated Go/TS output is included and can be skipped
server/internal/domain/fleetnode/control	Adds an artifact-result send path that snapshots completed upload refs and rejects OK acks that arrive before expected uploads	This is the ordering boundary that makes log-download success depend on the artifact being available
server/internal/domain/miner/remotenode	Sends DownloadLogsAction, expects one MINER_LOGS upload scoped to the device, and materializes the returned artifact	Bridges the existing command worker to fleet-node-paired miners without changing the public command API
server/cmd/fleetnode	Handles DownloadLogsAction, downloads device logs, formats CSV, uploads command artifacts, and acks only after upload success	This is the node-side execution path and upload contract
server/internal/domain/miner/logformat and plugins	Centralizes miner-log CSV formatting for direct and fleet-node paths	Keeps direct-miner and fleet-node log output consistent
server/internal/infrastructure/files	Copies validated command artifacts into the existing batch log directory with the existing filename/sanitization convention	Lets the current bundle finalizer work for direct, fleet-node, and mixed batches
Tests	Adds focused coverage for control artifact refs, remote-node materialization, fleet-node upload handling, shared formatting, and files bundling	Reviewers can check behavior at each boundary without needing hardware

Key technical decisions & trade-offs

• Keep minercommand.v1 unchanged; fleet-node log download support rides on the internal fleetnodegateway.v1 command payload instead of adding a new operator-facing API.
• Treat upload completion as part of command success; an OK ack without the expected uploaded artifact is surfaced as an error rather than producing an empty or missing bundle.
• Materialize command artifacts into the existing batch log directory instead of creating a second browser download path; this preserves current CSV/ZIP behavior and the existing bundle size cap.
• Share CSV formatting between direct and fleet-node downloads instead of letting the node and server drift in output shape.
• Keep artifact upload scoped by command id, purpose, and device identifier; retries can create a new artifact after a post-upload failure, but stale or mismatched uploads remain rejected by the in-flight command expectations.

Testing & validation

• buf lint
• (cd server && golangci-lint run -c .golangci.yaml)
• (cd server && go test ./cmd/fleetnode ./internal/infrastructure/files ./internal/domain/miner/logformat ./internal/domain/plugins ./internal/domain/fleetnode/control ./internal/domain/miner/remotenode)
• git diff --check

Not covered locally: full just lint still needs client dependencies installed (eslint was missing from client/node_modules), and full DB-backed server tests were not rerun because local Postgres auth for user fleet failed in earlier attempts.

[github-actions]

🔐 Codex Security Review

Note: This is an automated security-focused code review generated by Codex.
It should be used as a supplementary check alongside human review.
False positives are possible - use your judgment.

Scope summary

• Reviewed pull request diff only (fcb671c0547136494ff2c95bf54a23685d1053ff...bdc7600a0fc947300190eb401d2b647fe818d798, exact PR three-dot diff)
• Model: gpt-5.5

💡 Click "edited" above to see previous reviews for this PR.

---

Review Summary

Overall Risk: NONE

Findings

No security, correctness, or reliability findings in the scoped diff.

Notes

Reviewed .git/codex-review.diff only, for commit bdc7600a0fc947300190eb401d2b647fe818d798.

The new miner log artifact path includes the expected controls: authenticated fleet-node subject lookup, in-flight command/artifact expectation binding, per-node upload limits, size and SHA-256 validation before storing artifacts, max-size enforcement for miner logs, and CSV formula neutralization before batch log materialization.

I did not run tests; this was a static review of the supplied PR diff.

---

_{Generated by Codex Security Review |
Triggered by: @ankitgoswami |
Review workflow run}

[Copilot]

Pull request overview

Enables the existing ProtoFleet “DownloadLogs / GetCommandBatchLogBundle” workflow to work for miners paired through fleet nodes by having fleet nodes upload a MINER_LOGS command artifact, and having the server materialize that artifact into the same batch logs directory used for directly-connected miners.

Changes:

• Adds a fleet-node DownloadLogsAction and implements node-side log download → CSV formatting → artifact upload, with size limiting.
• Extends the fleet-node control registry to return completed upload refs with the terminal ack and to fail “OK” acks that arrive before expected uploads complete.
• Adds server-side artifact materialization into the existing batch log directory (plus tests) and centralizes CSV formatting in miner/logformat (including CSV formula neutralization).

Reviewed changes

Copilot reviewed 20 out of 22 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
server/internal/infrastructure/files/service.go	Adds SaveCommandArtifactLog to copy/verify a validated command artifact into the batch logs directory.
server/internal/infrastructure/files/service_test.go	Adds coverage for artifact log materialization and bundling behavior (single + mixed + error cases).
server/internal/handlers/fleetnode/gateway/handler.go	Plumbs SizeBytes into artifact expectations and maps “too large” admission failures to ResourceExhausted.
server/internal/handlers/fleetnode/gateway/handler_artifact_test.go	Tests rejecting oversized miner log uploads and allowing retry with bounded size.
server/internal/domain/plugins/plugin_miner.go	Switches direct-miner log CSV formatting to shared logformat.
server/internal/domain/plugins/plugin_miner_test.go	Updates tests to use logformat.FormatLineToCSVRow.
server/internal/domain/miner/service.go	Injects files service into remote-node miner config for log artifact materialization.
server/internal/domain/miner/remotenode/miner.go	Implements DownloadLogs for remote-node miners using artifact-result command send + artifact materialization.
server/internal/domain/miner/remotenode/miner_test.go	Adds tests for remote-node DownloadLogs action dispatch + artifact requirement behavior.
server/internal/domain/miner/logformat/logformat.go	New shared CSV formatter (streaming + row-based) with CSV formula neutralization and a max-bytes constant.
server/internal/domain/miner/logformat/logformat_test.go	Tests streaming formatter equivalence and formula neutralization behavior.
server/internal/domain/fleetnode/control/stream.go	Enforces per-expectation max-size on admitted artifact transfers.
server/internal/domain/fleetnode/control/registry.go	Adds size fields to artifact expectations and introduces ErrArtifactTooLarge.
server/internal/domain/fleetnode/control/registry_test.go	Tests for returning completed upload refs and rejecting OK acks before uploads complete.
server/internal/domain/fleetnode/control/command.go	Adds SendCommandWithArtifactResults and snapshots upload refs before removing in-flight command state.
server/cmd/fleetnode/minercommand.go	Implements node-side DownloadLogs action to format CSV, upload artifact, and only then ack success.
server/cmd/fleetnode/minercommand_test.go	Adds tests covering successful uploads, upload failure, partial-data rejection, and size limiting.
server/cmd/fleetnode/fake_gateway_test.go	Extends fake gateway to capture artifact upload streams and return refs/errors.
server/cmd/fleetnode/control.go	Passes gateway client into miner-command handler so DownloadLogs can upload artifacts.
proto/fleetnodegateway/v1/fleetnodegateway.proto	Adds DownloadLogsAction to the internal fleet-node miner command envelope.
client/src/protoFleet/api/generated/fleetnodegateway/v1/fleetnodegateway_pb.ts	Regenerated TS output for the proto change (generated).