Skip to content

feat(streaming): support iOS simulator with screen curved, framing - #62

Merged
saikrishna321 merged 2 commits into
appclawhq:mainfrom
Delta456:feat/device-stream-frame
Sep 3, 2026
Merged

saikrishna321 merged 2 commits into
appclawhq:mainfrom
Delta456:feat/device-stream-frame

Conversation

@Delta456

@Delta456 Delta456 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

This PR introduces streaming on iOS simulator through xcrun simctl io … screenshot command like how adb is used.

PR also adds curved screen with frame display to mimic the real phone which also makes it aesthetic.

Prompt history across dialogs and screen changes is also kept properly now. Earlier MainScreen held history in a useState, and TuiApp swaps the whole screen component out for every dialog and screen change — so the state was unmounted and re-initialised constantly.

Screenshots

iOS simulator

image

Android

image

Delta456 and others added 2 commits August 31, 2026 17:41
TuiApp renders dialogs and other screens *instead of* the active screen,
so their useInput handlers stop competing for keystrokes — which unmounts
MainScreen and destroyed the `useState` the history lived in. Every
/list, /yaml, /preview, /doctor, /memory, /session, /settings and
/history silently wiped it, and the prompt came back remembering only
what had been typed since.

The list moves to module scope in input-history.ts, where it outlives the
screen that offers it. The recall index and the stashed draft stay
component-local, since those should reset on remount.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0163wRmYBwSsV8xUDmuiEZio
…rame

`/stream` was Android-only. iOS simulators go through `xcrun simctl io …
screenshot`, which shapes the whole path: `screenshot -` is documented
for stdout but writes zero bytes, so every capture lands in a file the
caller owns; and there is no raw framebuffer, so the half-block backend
takes `--type=bmp` — a 32-bit BMP is a fixed header plus uncompressed
pixels, keeping the "no image decoder, no new dependency" property.
That BMP is BGRA and top-down, both read from the header rather than
assumed, because getting either wrong yields a plausible-looking picture
instead of an error. `RawFrame.order` carries the channel order so the
renderer resolves its offsets once instead of swapping ~12MB per frame.

Frames are now composited into a device body (`device-frame.ts`) on the
kitty path, which is why that path sends raw RGBA rather than a PNG: the
rounded outline needs an alpha channel neither adb nor simctl supplies.
iOS gets its measured corner radius plus the Dynamic Island drawn on top
— the framebuffer does not contain it, since iOS renders app content
underneath. Android's bare screen stays square (`corner-mask.ts`: a
guessed radius would crop real content), but its framed screen is
rounded, because a rounded body around a square screen pinches at the
corners: the body's arc is centred on (R,R) while the screen's corner
sits at (bezel,bezel), so keeping the full bezel there needs R <= bezel.

`scale.ts` box-filters the canvas down to roughly what the cell box will
show before writing it. Raw frames are ~11MB and the terminal rereads and
rescales all of them every frame; this costs ~19ms for a 5.5x smaller
payload. Measured on an arm64 emulator, capture dominates everything
else at ~870ms (~480ms of it on-device), and PNG is worse at ~1360ms, so
frame rate there is device-bound rather than format-bound.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0163wRmYBwSsV8xUDmuiEZio
@saikrishna321
saikrishna321 merged commit a79e4b1 into appclawhq:main Sep 3, 2026
7 checks passed
github-actions Bot pushed a commit that referenced this pull request Sep 3, 2026
## [2.7.0](v2.6.0...v2.7.0) (2026-09-03)

### Features

* **streaming:** support iOS simulator with screen curved, framing ([#62](#62)) ([a79e4b1](a79e4b1))
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 2.7.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@Delta456
Delta456 deleted the feat/device-stream-frame branch September 3, 2026 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants