Skip to content

[Enhancement] Self-hosted reliability for long-running/local model requests#1291

Closed
joonsoome wants to merge 19 commits intokwaroran:mainfrom
joonsoome:local-network
Closed

[Enhancement] Self-hosted reliability for long-running/local model requests#1291
joonsoome wants to merge 19 commits intokwaroran:mainfrom
joonsoome:local-network

Conversation

@joonsoome
Copy link
Contributor

@joonsoome joonsoome commented Feb 26, 2026

Summary

This PR improves self-host reliability for long-running local-model requests in Node self-host deployments, while preserving default behavior when Local Network Mode is disabled.

What changed

1) Local Network routing for OpenAI-compatible requests

  • Added local/private network detection utilities.
  • Added DB/preset controls:
    • localNetworkMode
    • localNetworkTimeoutSec
  • Extended fetch args:
    • networkRoute?: 'auto' | 'local_network'
    • requestTimeoutMs?: number
  • Applied routing only for OpenAI-compatible request paths:
    • Node self-host:
      • non-streaming => /proxy2
      • streaming => Job + WebSocket relay (/proxy-stream-jobs + /proxy-stream-jobs/:jobId/ws), fallback to /proxy2
    • Tauri => native/direct path
    • Public web runtime => local/private direct calls are blocked when Local Network Mode is enabled
  • Added timeout propagation and abort handling.

2) Proxy Job + WebSocket relay (Node self-host, streaming)

  • Added server-side job registry and lifecycle endpoints:
    • POST /proxy-stream-jobs
    • DELETE /proxy-stream-jobs/:jobId
    • WS /proxy-stream-jobs/:jobId/ws
  • Added event streaming protocol (job_accepted, upstream_headers, chunk, error, done, ping).
  • Added auth checks, local-target validation, rate limiting, job caps, timeout/cleanup hardening.
  • Kept existing /proxy2 path for fallback/regression safety.

3) Trigger v2 LLM streaming fix

  • Fixed v2RunLLM to honor streaming.
  • Added streaming text collection so output variables are populated correctly.
  • Added Trigger v2 UI toggle/default for streaming mode.

4) UX/help update

  • Added help tooltip for Local Network Mode (Experimental).
  • Added English help text for:
    • purpose
    • runtime behavior
    • constraints

Architecture

flowchart LR

  subgraph A["1. Local ON, Node self-host"]
    direction TB
    A1["Browser"] --> A2["Cloudflare Tunnel or reverse proxy"] --> A3["Risu Node self-host"]
    A3 --> A4["Streaming OpenAI: POST /proxy-stream-jobs + WS relay, fallback /proxy2"]
    A3 --> A5["Non-stream OpenAI: /proxy2"]
    A4 --> A6["LAN model, 192.168.x.x or localhost"]
    A5 --> A6
  end

  subgraph B["2. Local OFF, Node self-host"]
    direction TB
    B1["Browser"] --> B2["Risu Node self-host frontend runtime"]
    B2 --> B3["Streaming OpenAI: browser fetch to target URL"]
    B2 --> B4["Non-stream OpenAI: /proxy2 default path"]
    B3 --> B5["Local URL may fail by browser private-network policy"]
  end

  subgraph C["3. Public web, risuai.xyz"]
    direction TB
    C1["Browser"] --> C2["hubURL/proxy2 path"] --> C3["Target URL"]
    C3 --> C4["Local Network ON: blocked immediately by policy"]
    C3 --> C5["Local Network OFF: may fail or timeout, origin cannot reach user LAN"]
  end

  A ~~~ B
  B ~~~ C
Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant