Streaming/watch-mode commands still buffer or suppress stdout outside proxy mode

[nkootstra]

Summary

rtk proxy streaming was fixed by the src/core/stream.rs work, but the fix is not global. Several filtered or nominally passthrough RTK command routes still do not emit stdout for long-running/watch/dev commands until the child exits, or until the parent is killed.

This is a different failure mode than the already-closed proxy-only issue: rtk proxy <cmd> streams, while rtk <recognized-filter-route> <watch/dev command> can still hang silently.

Verified behavior on current master

I tested current master (rtk 0.37.2 source tree) with fake long-running executables placed first in PATH. Each fake tool prints one line immediately and then sleeps forever. I measured whether RTK emitted the first stdout line within 2 seconds.

Streams correctly

STREAMS  rtk proxy npm run dev
STREAMS  rtk pnpm run dev
STREAMS  rtk pnpm dev
STREAMS  rtk docker logs -f web
STREAMS  rtk docker compose logs -f web
STREAMS  rtk kubectl logs -f web

Still broken before this fix

NO_STDOUT_WITHIN_2S  rtk npm run dev
NO_STDOUT_WITHIN_2S  rtk npm start
NO_STDOUT_WITHIN_2S  rtk prettier --watch .
NO_STDOUT_WITHIN_2S  rtk playwright codegen
NO_STDOUT_WITHIN_2S  rtk gh pr checks 123 --watch
NO_STDOUT_WITHIN_2S  rtk dotnet watch run
NO_STDOUT_WITHIN_2S  rtk go run .

The user-visible effect is that an agent or human sees no progress and may kill/retry the command even though the child process is alive and already wrote output.

Minimal real-world repro

mkdir /tmp/rtk-stream-repro
cd /tmp/rtk-stream-repro
cat > package.json <<'JSON'
{
  "scripts": {
    "dev": "node -e \"console.log('dev-start'); setInterval(() => {}, 1000);\""
  }
}
JSON

npm run dev          # prints immediately
rtk proxy npm run dev # prints immediately
rtk npm run dev       # no stdout before forced termination

Root cause

The merged streaming infrastructure is present, but not all command paths use it.

Important paths:

• src/core/runner.rs: RunMode::Passthrough uses FilterMode::Passthrough, which inherits stdout/stderr and streams.
• src/core/runner.rs: RunMode::Filtered calls stream::run_streaming(..., FilterMode::CaptureOnly), which captures stdout/stderr until EOF before applying a filter.
• src/core/stream.rs: exec_capture() still uses cmd.output(), which waits for the child process to exit.
• Some functions named run_passthrough were not actually streaming passthrough. For example, unsupported go and dotnet routes captured output and printed it only after completion.

So #222 was fixed for rtk proxy, but filtered routes and some passthrough-named routes remained EOF-buffered.

Related issues and PRs

• bug: rtk proxy does not stream long-running command output progressively #222: fixed the narrow rtk proxy long-running streaming bug. This issue is not the same because rtk proxy npm run dev streams while rtk npm run dev does not.
• 6 hook bugs on master — routing, streaming, extensibility, hook collision (fixed in PR #156) #361: broad hook/streaming issue. It correctly identified .output() buffering, but the merged stream infrastructure did not eliminate all buffered filter paths.
• feat(refacto-core): binary hook w/ native cmd exec + streaming #956: merged src/core/stream.rs, FilterMode, and rtk proxy streaming. This introduced the needed primitives, but many routes still use CaptureOnly or exec_capture().
• docker logs summarization hides progress bars and streaming output #1267: related Docker logs/progress problem. Container follow paths now tend to stream via passthrough, but filtered log summarization can still hide progress.
• When using the rtk git push command, it sometimes times out #963 and PR fix(git): stream push output to avoid spurious 30s timeout (#963) #1531: git push timeout has the same root cause pattern (Command::output() hiding progress/prompts), but PR fix(git): stream push output to avoid spurious 30s timeout (#963) #1531 only fixes git push.
• enhancement: add --raw / --no-filter global flag for unfiltered command output #1156 and PR feat(cli): add --raw flag for unfiltered command output #1171: --raw / --no-filter is related, but users should not need to know to opt out manually for commands that are inherently streaming.

Suggested fix

RTK should detect inherently streaming invocations before choosing a buffered filtering path and run them with inherited stdout/stderr instead.

Practical detection heuristics:

• --watch, --watchAll, -w
• --follow, -f for log/follow tools such as Docker, kubectl, tail, journalctl
• streaming/dev subcommands such as watch, dev, serve, start
• known interactive/long-running commands such as playwright codegen, playwright show-report, go run, dotnet watch
• inherently streaming binaries such as tail, nodemon, watchman

Also, routes called run_passthrough should use actual passthrough execution (Stdio::inherit() / FilterMode::Passthrough), not Command::output().

Acceptance criteria

• rtk npm run dev emits output immediately and does not wait for EOF.
• rtk npm start emits output immediately and does not wait for EOF.
• rtk prettier --watch . emits output immediately.
• rtk playwright codegen emits output immediately.
• rtk gh pr checks <pr> --watch emits output immediately.
• rtk dotnet watch run emits output immediately.
• rtk go run . emits output immediately.
• Finite commands such as rtk npm run build and rtk gh pr checks <pr> remain filterable.

[pszymkowiak]

wshm · Automated triage by AI

🔍 Automated Triage

🐛 Category	bug
🟠 Priority	high
🎯 Confidence	93%

Summary

Streaming/watch-mode commands buffer stdout until EOF on filtered routes; fix in progress via PR #1547 — review and merge.

📁 Relevant files

• src/core/stream.rs
• src/core/runner.rs

---

Triaged automatically by wshm · This is an automated analysis, not a human review.

[nkootstra]

@aeppling Should I just close this issue and PR?