Implement the Live Interactivity (Lumens) extension — omadia-canvas-protocol/1.1

[iret77]

Summary

This issue asks the maintainers to implement the Live Interactivity (Lumens) extension — the Omadia, on-canvas answer to sandbox-style "live artifacts", deliberately re-aimed to be both more capable and structurally safer.

The concept and a full normative definition have been written and are ready for implementation (PR #31). This issue tracks turning those specs into code.

Thesis: most of what arbitrary-code sandboxes block is blocked by missing capabilities, not missing compute. So Lumens constrain computation (a declarative, bounded, deterministic, interpreted behaviour model — no arbitrary code; the whitelist parser/CSP default-src 'self' model extends to it) and open capabilities (real data, write-back, allowlisted network, generated assets) mediated through the existing Tier-2/3 orchestration and effect classification.

This unblocks the target use cases: small games (build/play/share), interactive data workflows, unusual visualisations (defrag-style), live maps, kiosk/iPad surfaces — all generated by the agent, run Tier-1-fast (60 fps, no server round-trip per frame), and safe to share with selected users or save as a preset.

Specs to implement (all on claude/omadia-ui-interactivity-concept-wu7isw, PR #31)

• Concept / rationale: docs/interactivity-concept.md
• Normative definition / spec: docs/lumens-spec.md — omadia-canvas-protocol/1.1 (additive minor bump)
• Lume visual treatment: docs/visual-spec.md §4.13
• Implementation outline: docs/implementation-plan.md §10
• Reference mockup (kiosk Lumen, Lume/Lagoon): docs/mockups/kiosk-lumen-aura.html
• CONCEPT.md §"Extension — Live Interactivity (Lumens)"

What a Lumen is

Declarative data, not code: state + transitions + view + events + capabilities, plus a new scene primitive (declarative draw surface), declarative ports/wires for cross-element interaction, per-region render cadence, and a preset library so the agent authors once and reuses constantly.

Key guarantees the implementation must preserve:

• No arbitrary code — LX is a validated JSON-AST walked by a shipped interpreter (no eval, CSP unchanged).
• Bounded & total — gas + frame ceiling + bounded iteration + state cap → a Lumen can never hang the host (halts with surface_error).
• Deterministic — host-seeded random/now/tick → replay, undo, safe sharing, v2 multiplayer.
• Default-deny, brokered capabilities — reuse local/internal/external-effect + confirmation modal; nothing reaches the outside directly.
• Content-addressed assets — id = kind-sha256(content) ⇒ never-stale cache; generated material comes from omadia-core LLM connectors, the Lumen only requests + renders.
• Additive — rides existing surface_* grammar, DataRef/HMAC scoping, authority split, shared-canvas hooks; 1.0-only clients ignore unknown types.

Proposed workstream (smallest mergeable units first — see implementation-plan §10)

• L0 — JSON Schemas: Lumen behavior section, LX-AST, scene, ports/wires, capability manifest + accept/reject fixtures (docs/protocol/schema/) → protocol 1.1
• L1 — Tier-1 LX interpreter: deterministic AST evaluator, gas + frame ceiling, bounded iteration, seeded random/now; extended whitelist validator
• L2 — scene primitive: draw-list rasteriser (canvas2d → WebGL), token-only styling, buffer-native hit-testing → TargetRef
• L3 — cadence & animation: per-region static/reactive/{tick} dirty-tracking + rAF scheduling; declarative animate on the Lume effect vocabulary; reduced-motion
• L4 — events & touch: tap/longPress/drag/pinch/swipe/key/tick; 44 pt hit-targets; host gesture arbitration; input-modality handshake fields
• L5 — capabilities broker (Tier 2): persist/loadData/writeData/tiles/fetch/generateAsset/clipboard; effect-classified brokering + confirmation gate; surface_capability_* events; asset transport + content-addressed cache (generateAsset ↔ omadia-core connectors)
• L6 — ports & wires: typed ports, Tier-1 wire resolution, shared viewState.selection cross-element
• L7 — lifecycle & presets: author-once/patch; lumen-presets/** + lumen-state/** stores; resolve-then-generate lookup; fork+patch; behaviour-idiom library in the UI Skill
• L8 — sharing: canvasOwnership group extension + channel fan-out + import consent
• L9 — reference Lumens: an arcade game · interactive workflow · defrag-viz · map, traced end-to-end + conformance fixtures

Riskiest items (mirror concept §13)

LLM reliability emitting valid LX (likely a strong-model authoring job + fast-model patching); gas/scene-perf calibration on the reference Lumens; capability-consent UX.

---

Specs are ready for review on PR #31. Requesting maintainer review of the approach and prioritisation of the workstream above.

[Weegy]

Implementation status — L0–L9 cores landed (PRs up)

The headlessly-verifiable scope of the whole workstream is implemented, tested, and hardened, split across two stacked PRs:

• feat: Lumens (Live Interactivity) 1.1 — canvas-core + Tier-2 producer (server) omadia#315 — canvas-core protocol 1.1 (schemas, LX interpreter, Tier-2 capability policy). Merge first.
• feat: Lumens (Live Interactivity) renderer — scene, runtime, data-bound (impl of #34) #36 — host-app renderer (scene, live Lumen runtime, motion). Stacked on this branch; merge after #315 + Concept + definition: Live Interactivity (Lumens) — Tier-1-fast, agent-generated, safe #31.

376 tests passing across the two repos; all typechecks + lint clean; nothing partially renders.

Workstream

• L0 — schemas: lx-ast, scene, ports-wires, capability-manifest, lumen + accept/reject fixtures → omadia-canvas-protocol/1.1. (Added two LX nodes the spec needed: var, get.)
• L1 — LX interpreter (Tier 1): deterministic, gas + depth + value-size bounded, no-eval, seeded random/now; static semantic validator. Prototype-pollution hardened — a 10-finding adversarial review is fully addressed.
• L2 — scene primitive: canvas2d rasteriser, token-only styling, buffer-native hit-testing → TargetRef.
• L3 — cadence & animation: per-region dirty-tracking + rAF; declarative animate layer; reduced-motion.
• L4 — events & touch: tap/longPress/drag/pinch/swipe/key/tick; gesture arbitration via the host; input-modality handshake fields.
• [~] L5 — capabilities broker: policy core done & tested (effect classification, egress rate/quota/in-flight/idempotency/backpressure, content-addressed asset cache, import consent). Remaining = live wiring (see below).
• L6 — ports & wires: typed wire resolution + least-privilege expose.
• [~] L7 — lifecycle & presets: core done (content-addressed ids, resolve-then-generate, fork/patch lineage). Remaining = the lumen-presets/** / lumen-state/** memoryStore wiring + UI Skill idiom library.
• [~] L8 — sharing: core done (token-strip, recipient-scoped re-mint, inert unauthorised assets, ownership group). Remaining = channel fan-out + import-consent UI.
• L9 — reference Lumens: arcade game · interactive workflow · defrag-viz · map, traced end-to-end + conformance fixtures.

Remaining — live-middleware integration (not headlessly buildable; consumes the policy in #315)

• omadia-ui-orchestrator + omadia-ui-channel plugin packages (Tier-2 dispatch, surface_capability_request/_result transport).
• omadia-core LLM connectors for generateAsset + real HMAC token minting.
• Renderer DOM glue: capability-consent modal, inert-sprite rendering, a canvas-level wiring provider threading resolveWires into live LumenNodes.

These all sit on top of the shipped pure cores; they need a live server + real I/O to verify.