Feature: Native Gpu Terminal by lmvdz · Pull Request #99 · collabs-inc/collab-public

lmvdz · 2026-04-02T16:15:20Z

Native GPU Terminal Rendering

Replace per-terminal Chromium webview processes with in-process terminal rendering, then layer on a WebGL2 GPU-accelerated renderer for high-density terminal workloads. Target scenario: 120Hz 4K HiDPI display with 20+ simultaneous terminals.

What changed

In-process terminals (terminal-embed.js, tile-manager.js, preload/shell.ts)
Mount xterm.js directly in the shell renderer DOM instead of spawning a separate Chromium process per terminal tile. Reduces memory by ~60MB per terminal on Windows. Gated behind inProcessTerminals config flag (defaults true on Windows).

WebGL2 GPU rendering engine (3 new modules)

font-atlas.js — Rasterizes monospace glyphs to a power-of-2 texture atlas with CJK/fullwidth support, sub-pixel coverage for font-weight 300/500
shared-gl-context.js — Single WebGL2 context shared across all tiles via per-terminal FBOs, bypassing Chromium's ~16 context limit
gpu-terminal-renderer.js — 3-pass instanced quad rendering (backgrounds, glyphs, cursor) implementing the xterm.js CanvasRenderer interface

PTY async & debouncing (pty.ts, tmux.ts)
Migrate synchronous tmuxExecForTarget calls to tmuxExecAsync throughout session creation, reconnection, and resize. Debounce tmux resize-window calls (150ms) during rapid tile drags. Extend tmux exec timeout from 5s to 30s for WSL cold starts.

Performance overlay (perf-overlay.js)
Game-style HUD toggled with F3: FPS counter, rolling frame time graph (120 samples), CPU render timing (via term.onRender), GPU timing (via EXT_disjoint_timer_query_webgl2), memory usage, terminal count. Canvas-rendered, zero overhead when hidden. Tracks right panel position via ResizeObserver.

Shell rendering improvements (canvas-viewport.js, tile-renderer.js)

CSS radial-gradient dot grid replaces Canvas2D grid (GPU-composited, zero JS cost during pan)
positionTile uses translate3d + scale instead of left/top (avoids layout recalc)

Settings — Performance pane (App.tsx)
Renderer mode selector (GPU WebGL / standard DOM / legacy webview), uncap frame rate toggle for 120Hz+, app restart button, F3 overlay hint.

Config & IPC (config.ts, index.ts, ipc-workspace.ts, preloads)
New prefs: gpuRenderer (default: true), uncapFrameRate (default: false). disable-frame-rate-limit Chromium switch applied at startup. app:restart IPC handler.

Stats

26 files changed, +5,322 / -241 (3 new modules, 2 new test files)

Test plan

config.test.ts — 22 tests: getGpuRenderer, getUncapFrameRate, getInProcessTerminals, isTerminalTarget, getPref/setPref round-trips
font-atlas.test.ts — 28 tests: nextPow2, fontString, glyph cache key encoding/uniqueness, charWidth (ASCII, CJK, Hangul, fullwidth, boundaries), atlas sizing math, UV re-normalisation on growth
tile-renderer.test.ts — 23 tests: positionTile updated for translate3d, width/height caching test added
Manual: launch app, open 5+ terminals, verify GPU renderer path renders text correctly
Manual: Settings > Performance > switch between renderer modes, restart, verify each mode
Manual: F3 overlay — verify FPS, CPU timing, frame graph, panel tracking
Manual: rapid tile resize drag — verify no main-process blocking from tmux calls

Resolve tmux.conf from the module tree in dev/test mode and make the tmux test suite target-aware on Windows so the WSL tmux path is exercised reliably.

Mounts xterm.js directly in the shell renderer DOM instead of spawning a separate Chromium process per terminal tile. Reduces memory by ~60MB per terminal on Windows. Gated behind `inProcessTerminals` config flag (defaults true on Windows, false on macOS). - Add terminal-embed.js: xterm.js lifecycle, input, clipboard, resize - Route PTY data to shell window via getEffectiveSender() across all backends (direct, tmux, sidecar) - Expose PTY IPC methods on shellApi in shell preload - Add spawnTerminalDiv/spawnTerminal gate in tile-manager - Add getInProcessTerminals config flag and IPC handler

Swap the VT parser and renderer from xterm.js to ghostty-web (libghostty-vt WASM). ghostty-web provides a drop-in xterm.js-compatible Terminal class backed by Ghostty's VT parser with Canvas2D rendering and built-in Unicode support. Removes dependency on @xterm/addon-webgl and @xterm/addon-unicode11 for in-process terminals. - WASM (~413KB) is inlined as base64 in the bundle, no file serving needed - Lazy init: WASM loads on first terminal creation via ensureGhosttyInit() - Same API surface: write, onData, onResize, focus, blur, dispose, getSelection - FitAddon from ghostty-web replaces @xterm/addon-fit for in-process path

Research, adversarial design (draft + red team + arbiter), and phased implementation plan for replacing per-terminal webviews with in-process ghostty-web rendering.

Migrate synchronous tmuxExecForTarget calls to tmuxExecAsync throughout session creation, reconnection and resize paths. Debounce tmux resize-window calls (150ms) to avoid spawning dozens of wsl.exe processes during rapid tile drags. Extend tmux exec timeout from 5s to 30s to handle WSL cold-start latency. Add targeted debug logging for PTY data routing diagnostics.

Implement WebGL2 instanced terminal renderer as a drop-in replacement for Canvas2D rendering. Introduces SharedGLContext (single WebGL2 context shared across all tiles to bypass Chromium's ~16 context limit), FontAtlas (glyph rasterization to GPU texture atlas), and GpuTerminalRenderer (3-pass instanced quad rendering for backgrounds, glyphs, and cursor). Adds gpuRenderer and uncapFrameRate config prefs with IPC plumbing, CSS-based dot grid (GPU-composited, zero JS cost during pan), app restart handler, and WebGL-safe terminal reset to preserve texture atlas. Includes debug watchdog for PTY data diagnostics.

Add Performance settings pane with three renderer modes (GPU-accelerated WebGL, standard DOM, legacy per-terminal webview), uncap frame rate toggle for 120Hz+ displays, and a restart button to apply changes.

…tlas math Cover getBoolPref-backed accessors (gpuRenderer, uncapFrameRate, inProcessTerminals), isTerminalTarget validation, getPref/setPref round-trips, and font-atlas pure logic (nextPow2, fontString, glyph cache keys, charWidth, atlas sizing and UV re-normalisation).

Remove DEBUG IPC test messages injected in createSession (tmux and direct paths), PTY data/exit console.log calls, rate-limited sendToSender warnings, getEffectiveSender route logging, and the 3-second PTY data watchdog timer in TerminalTab.

positionTile now uses a single translate3d+scale transform instead of separate left/top/transform/transformOrigin properties. Update all assertions to match and add a test verifying width/height caching skips redundant style writes.

ghostty-web is no longer imported — the GPU renderer uses a custom WebGL2 pipeline with FontAtlas instead.

Self-contained HUD overlay showing FPS counter, rolling frame time graph (120 samples), CPU/GPU timing, memory usage, and terminal count. Uses EXT_disjoint_timer_query_webgl2 for GPU timing when available. Canvas-rendered, zero overhead when hidden — RAF loop only runs while visible. Toggle with F3 or see Settings > Performance for details.

Increase overlay height from 160 to 180px so the graph legend is not clipped. Observe #panel-terminal via ResizeObserver + MutationObserver to offset the overlay's right position when the terminal panel is expanded or collapsed, with a CSS transition for smooth movement.

Instrument terminal data flush with markCpuStart/markCpuEnd so the overlay shows actual JS-side parse+render cost. Attach the WebglAddon's internal GL context to the perf overlay for GPU timer queries.

perf-overlay.js: - Store MutationObserver in module scope so it can be disconnected - Guard collectGpuTime against null timerExt - Clear pending GPU query on context loss instead of stalling forever - Skip first RAF frame to avoid scheduler jitter in initial dt - Seed FPS estimate after 100ms so it doesn't read 0 for a full second - Handle DPR changes when window moves between monitors - Extract GRAPH_MAX_MS constant (was magic number 50 in 3 places) - Use ctx.save/restore in drawGraphLine instead of implicit lineWidth reset - Defer CSS transition to avoid animating initial position - Guard initKeyboardShortcut against double-registration terminal-embed.js: - Move markCpuEnd to term.onRender callback so CPU time measures actual parse+render cost, not just the near-instant term.write() enqueue - Add comments explaining rAF deferral and attachGL idempotency - Simplify _renderer._gl access chain

yiliush · 2026-04-02T16:15:33Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

lmvdz · 2026-04-02T16:18:51Z

I have read the Contributor License Agreement (CLA) and hereby sign the CLA.

- Add pref key allowlist to setPref with prototype pollution guard - Gate app:restart IPC to main/settings window senders only - Clamp PTY cols/rows to safe bounds (2-500 / 1-300) in create, reconnect, and resize paths to prevent conpty crashes - Cap pty:write and pty:send-raw-keys payloads at 1MB with type check - Cap earlyDataBuffers at 512KB per session with 30s auto-cleanup - Fix __terminalRegistry dev-mode guard for Electron renderer context

Overlapping rename() calls to the same destination file race on Windows and produce EPERM errors when spam-clicking terminal creation. Serialize saves with a flight-tracking promise and fall back to direct writeFile if rename still fails.

Remove duplicate ptyKillSession preload export (identical to ptyKill), remove unused isVisible export from perf-overlay. Add renderer mode awareness to perf overlay: shows "CPU: n/a (webview)" and "Legacy (webview)" label when in legacy mode, dynamic renderer label when in-process mode is active.

Convert remaining synchronous tmuxExec calls in the legacy tmux branch of createSession to tmuxExecAsync (was blocking main for up to 30s). Add per-session ownership tracking: createSession records the effective sender webContentsId, and pty:write, pty:send-raw-keys, pty:resize, and pty:kill IPC handlers verify the caller owns the session before proceeding. Legacy sessions without ownership are allowed through.

Call gpuTimerBegin before term.write() and gpuTimerEnd in the term.onRender callback so GPU timer queries bracket the actual WebGL render pass. Pass setInProcessMode from tile-manager so the overlay knows which renderer is active.

When the atlas reaches MAX_ATLAS_SIZE and cannot grow, evict all non-ASCII glyphs (preserving pre-warmed ASCII) and reset the slot pointer to reuse freed space. Frequently-used non-ASCII glyphs are re-rasterized on demand. Logs a warning when eviction fires.

config.test.ts: add tests for getTerminalMode (platform-specific defaults, pref override) and getTerminalBackend (win32 default, invalid pref fallback). perf-overlay.test.ts: new test file covering fpsColor/msColor thresholds, circular buffer wrap-around and traversal, FPS accumulation at 60/30fps with running estimate, and graph Y coordinate mapping with clamping.

Critical: - Restore `const shell = process.env.SHELL` in legacy tmux createSession branch (was deleted during refactor, causing ReferenceError on macOS) Medium: - Add sessionOwners.set in all reconnectSession paths (direct, sidecar, tmux) so reconnected sessions have proper sender verification - Clean up sessionOwners in all exit paths: attachClient onExit, direct session onExit, killAll, killAllAndWait - Convert sync tmuxExecForTarget to async in killSession (kill-session) and attachClient onExit (has-session) to avoid 30s main-process block - Fix clampDims type: pass cols ?? 80 to satisfy strict types Low: - Cap pref key length at 64 chars to prevent unbounded key growth via panel-width-* prefix - Rate-limit font atlas eviction console.warn to first 3 occurrences - Remove duplicate JSDoc blocks on FontAtlas constructor

- Add sessionOwners.delete in killSession sidecar early-return path - Add sessionOwners.delete + sidecarSessionIds.delete in session.exited notification handler - Add ownership check to pty:capture, pty:foreground-process, and pty:read-meta IPC handlers - Cap pty:capture lines parameter to 10,000 - Initialize _asciiSlotCount and _evictionCount to 0 in FontAtlas constructor for defensive hygiene - Add session-owner.test.ts: 9 tests covering isSessionOwner logic (legacy fallback, owner match, shell window elevation, independent sessions, owner ID 0 edge case)

Split createTerminal into 3 animation-frame phases so the main thread is never blocked for more than one lightweight step. The handle is returned immediately with data buffering, so PTY data arriving during init is not lost. Phase 1 creates the xterm instance and mounts the DOM, Phase 2 wires addons/input/resize, and Phase 3 loads the WebGL addon. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

With multiple terminals rendering per frame, gpuTimerBegin/End could nest incorrectly. Add a refcount so the actual GL query spans from the first begin to the last end, measuring aggregate GPU time. Also add resetGpuTimer() for clean recovery from context-loss mid-query. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…init Replace the static "Loading..." splash text with a typewriter effect that cycles through whimsical status words. Reorder app.whenReady so the window appears before heavy init (watcher, IPC, sidecar), and defer CLI installation by 3s. Remove the now-redundant shell loading overlay (HTML, CSS, JS) since the splash screen covers the wait. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

BrowserWindow.fromWebContents returns null for <webview> guest contents, so the old allowlist check rejected restarts from the settings webview. Accept null senders (only reachable via our own preload scripts) and always relaunch+quit instead of branching on isPackaged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Log lap times for ptyReconnect, ptyCreate, and createTerminal within spawnTerminalDiv to help profile terminal spawn latency. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Replace vitest imports with bun:test and vi.fn with mock(). Add happy-dom for DOM globals. Update files.test image extension mock to match expanded set. Fix tmux.test cleanup race on WSL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

These tests require Node's libuv for node-pty and must run via npx tsx --test. Rename convention makes this explicit and lets bunfig.toml exclude them from bun test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Make target and cwdGuestPath types explicit with undefined union. Fix nested ternary in listKnownTmuxTargets for null meta. Wrap sendToMainWindow in try/catch so tests without Electron don't throw. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

… guard Add plans/ to gitignore. Suppress TS 6.0 deprecation warnings in tsconfig.node.json. Guard canvas-viewport isMac check for SSR/test environments where window is undefined. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The .node-test.ts suffix already prevents bun test discovery, so the explicit exclude is unnecessary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

yiliush · 2026-04-02T19:24:17Z

Incredible 🙏🏼 reviewing

lmvdz · 2026-04-02T19:26:24Z

@yiliushenburke running more full reviews will be a few more commits coming, also want to look at what Bear is doing in #100 might be easier to rebase ontop of what he is doing there, these two PRs are orthogonal to performance.

Split WebGL2TerminalRenderer into SharedGPUResources (compiled once) and TerminalDrawState (per terminal). Add XtermAdapter for zero-alloc Float32Array packing from xterm.js buffer API with 256-color palette. Add GPUCompositor for single-canvas compositing pass. Refactor SharedGLContext into ref-counted singleton with presentTerminal() bitmap transfer. Wire into terminal-embed.js Phase 3 replacing the WebglAddon path. Includes GPU-rendered selection (custom mouse overlay bypassing xterm.js CSS transform issues), cursor with DECTCEM visibility and blink, theme color resolution, atlas dirty re-upload for on-demand Unicode glyphs, and Ctrl+wheel font zoom. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…dening Add assertSessionId (hex-16 regex) to all PTY IPC handlers. Add path containment checks on workspace:read-tree and workspace:update-frontmatter to prevent directory traversal. Change isSessionOwner to default-deny for unknown sessions. Add cwd stat validation on pty:create. Add sender verification on pty:discover and pty:clean-detached. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Rapid concurrent saves (e.g. during resize with many terminals) could starve under the old while-loop pattern. New approach: pendingState holds the latest, a single drainer loop writes at most one at a time, intermediate states are skipped. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

yiliush · 2026-04-03T17:21:37Z

fyi #100 has been merged into main

lmvdz · 2026-04-14T02:36:07Z

have to put a pause on this, entering a hackathon

BearHuddleston and others added 18 commits April 2, 2026 09:58

Add WSL tmux restore support for Windows terminals

c5de7e6

Harden WSL tmux test coverage

136a5e1

Resolve tmux.conf from the module tree in dev/test mode and make the tmux test suite target-aware on Windows so the WSL tmux path is exercised reliably.

Add native GPU terminal rendering plan and design docs

c6a49ca

Research, adversarial design (draft + red team + arbiter), and phased implementation plan for replacing per-terminal webviews with in-process ghostty-web rendering.

remove plans

3146212

feat(settings): add Performance pane with renderer selection

c06dfc6

Add Performance settings pane with three renderer modes (GPU-accelerated WebGL, standard DOM, legacy per-terminal webview), uncap frame rate toggle for 120Hz+ displays, and a restart button to apply changes.

chore(security): enforce min release age of 7d supply chain risk

ef34d6c

chore(pty): remove debug instrumentation

f914702

Remove DEBUG IPC test messages injected in createSession (tmux and direct paths), PTY data/exit console.log calls, rate-limited sendToSender warnings, getEffectiveSender route logging, and the 3-second PTY data watchdog timer in TerminalTab.

chore(deps): remove unused ghostty-web dependency

584d1fc

ghostty-web is no longer imported — the GPU renderer uses a custom WebGL2 pipeline with FontAtlas instead.

fix(perf-overlay): wire up CPU timing and GPU context

f35aaf9

Instrument terminal data flush with markCpuStart/markCpuEnd so the overlay shows actual JS-side parse+render cost. Attach the WebglAddon's internal GL context to the perf overlay for GPU timer queries.

lmvdz added 7 commits April 2, 2026 11:31

lmvdz force-pushed the feature/native-gpu-terminal branch from a9c2eae to 1e4519a Compare April 2, 2026 16:50

lmvdz added 2 commits April 2, 2026 12:00

lmvdz and others added 6 commits April 2, 2026 13:24

style: fix indentation and trailing whitespace

06d222b

chore: add startup timing instrumentation to tile-manager

0b91325

Log lap times for ptyReconnect, ptyCreate, and createTerminal within spawnTerminalDiv to help profile terminal spawn latency. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

lmvdz force-pushed the feature/native-gpu-terminal branch from ab15ae8 to 0b91325 Compare April 2, 2026 18:39

lmvdz and others added 5 commits April 2, 2026 14:15

chore(bunfig): drop redundant sidecar test exclude

ff9eb39

The .node-test.ts suffix already prevents bun test discovery, so the explicit exclude is unnecessary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

lmvdz and others added 4 commits April 2, 2026 17:13

chore(types): add ImportMeta.env type augmentation

b3712a4

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feature: Native Gpu Terminal - #99

Feature: Native Gpu Terminal#99
lmvdz wants to merge 42 commits into
collabs-inc:mainfrom
lmvdz:feature/native-gpu-terminal

lmvdz commented Apr 2, 2026

Uh oh!

yiliush commented Apr 2, 2026 •

edited

Loading

Uh oh!

lmvdz commented Apr 2, 2026

Uh oh!

yiliush commented Apr 2, 2026

Uh oh!

lmvdz commented Apr 2, 2026

Uh oh!

yiliush commented Apr 3, 2026

Uh oh!

lmvdz commented Apr 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

lmvdz commented Apr 2, 2026

Native GPU Terminal Rendering

What changed

Stats

Test plan

Uh oh!

yiliush commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lmvdz commented Apr 2, 2026

Uh oh!

yiliush commented Apr 2, 2026

Uh oh!

lmvdz commented Apr 2, 2026

Uh oh!

yiliush commented Apr 3, 2026

Uh oh!

lmvdz commented Apr 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yiliush commented Apr 2, 2026 •

edited

Loading