Skip to content

feat(mind): ask the wallet in plain words, Explain, and proactive insights - #118

Merged
bitwalt merged 17 commits into
mainfrom
feat/mind-intents
Oct 9, 2026
Merged

bitwalt merged 17 commits into
mainfrom
feat/mind-intents

Conversation

@bitwalt

@bitwalt bitwalt commented Oct 9, 2026

Copy link
Copy Markdown
Member

Summary

KaleidoMind moves from a separate chat into the wallet itself:

  • Intent bar on the Dashboard and in Mind. Type or say "send 10€ to Mario", "swap half my BTC to USDT", "receive 50k sats on Lightning" or "how much did I spend this week". The answer is a card with the amount, recipient, route and fee. Review opens Send, Swap or Receive prefilled.
  • Explain next to activity items, fees, RGB transfers and errors: a short plain-language explanation of that exact item.
  • Insights on the Dashboard: small cards you can dismiss, e.g. RGB backup missing or failed, low inbound room on an RGB channel, a swap or payment that needs attention, a large payment received, low mainnet fees.

Everything runs on the phone, and money still moves only through the existing Send, Swap and Receive confirm flows.

Builds on #117 (Prismo). This branch merges it, so until #117 lands this diff also shows its commits.

How it works

  • Intents (services/mindIntents/):
    • Deterministic rules (English and Italian: €/$/£, k/mila, decimal commas, half/metà/all/%, contacts, pasted invoices and addresses) run first. Then the Funnel's existing extractors. The model is asked only when neither is confident.
    • The schema prompt isn't tuned to any model, and validateIntent drops amounts or recipients that aren't in the user's own words.
  • Numbers come from the wallet, never the model (card.ts):
    • fiat from the price feed, fractions from balances;
    • send fees from the kaleidoPay quotes, swap output from a new read-only previewSwapQuote;
    • contacts via matchContacts, which shows a warning on zero or several matches instead of guessing.
  • Review navigates with prefill params to Send, Receive and Swap, whose confirm steps are unchanged.
  • Explain (services/mindExplain.ts): the model sees only the item's structured facts (no ids or preimages). Its answer is kept only if it is 2–4 sentences and every number appears in the facts; otherwise a built-in explanation is shown.
  • Insights (services/insights.ts, useInsights): rule-based, no model.
    • at most 2 cards at a time;
    • fees checked at most every 30 minutes, activity every 5;
    • dismissals saved per wallet, each with a cooldown.

Details

  • Lite mode hides route and account details. Receive applies a network only in Advanced.
  • Prismo shows up on insight cards and on the Explain sheet.
  • Regexes avoid lookbehind and \p{…} so they cannot fail to parse on Hermes; a test guards this.
  • Previewing a send to a Lightning-address contact fetches an invoice from the contact's server, the same thing Send's Continue does.

Testing

  • tsc is clean, with about 120 new tests across parsing, schema, cards, the runner, Explain, insights and the sheets.
  • The full jest suite is green except services/kaleidoPay/barkOffer.test.ts, which already fails on main and is unrelated.

Needs a device:

  • dictation on long-press;
  • the Explain sheet on top of the activity and swap-failure sheets;
  • Review prefill on all three screens;
  • real fee and swap quotes;
  • low-fee cards only on mainnet.

bitwalt added 17 commits October 9, 2026 13:07
The on-device assistant now presents itself as Prismo, KaleidoMind's
assistant: friendly, concise and a little playful, in plain language,
never joking about money safety and always showing the amount before
spending. Tool contracts are unchanged.
A small faceted prism with dot eyes, filled with the 135° green-to-violet
brand gradient. Seven moods (idle, listening, thinking, speaking, happy,
concerned, sleeping) each get their own pose and motion: breathing,
blinks, facet shimmer, hue drift with orbiting dots, a level-driven
mouth, a sparkle burst and an amber tint.

Motion runs on the UI thread as transforms and opacity over static SVG
layers. Loops stop under Reduce Motion, while paused, in the background
and on unmount. MindCharacterBadge is a static variant for inline and
tab-bar use, with a monochrome mode that follows the tab tint.

Pure helpers map chat and voice state to a mood.
The empty chat greets you with a large Prismo, and a small one sits
beside assistant replies and the pending-reply row. Its mood follows
real state: asleep while the assistant is off or not loaded, thinking
while the model loads or reasons, speaking while the reply streams,
happy after an approved payment succeeds and concerned on errors.

Only the newest reply animates; older avatars are static, and loops
pause while the tab is not focused.
… data

Rules for English and Italian phrasings (amount, unit, contact, network,
period) run first and work without a model. The on-device model only fills
gaps, and its JSON is accepted through a schema validator that drops amounts
and recipients the user never wrote. Cards take sats, fiat, fees and swap
returns from the wallet's price, balances and quotes; questions are answered
from balances and Activity.
Prismo replaces the push-to-talk orb: it listens with its facets
pulsing to the live mic level, thinks while the reply is prepared and
speaks while the answer plays, with a small mic or speaker chip for the
tap affordance. It looks happy after an approved payment succeeds and
concerned when voice or the model fails.

VoiceInput gains an optional onLevel callback that enables metering
only when a caller asks for it. The ring pulse behind Prismo now stays
still under Reduce Motion.
The tab and the chat header show a static Prismo badge that follows
the tab tint: outlined when inactive, filled when selected. The old
brain mark is no longer used anywhere and is removed.
… params

Send takes an amount in sats regardless of the display unit, Receive an
amount, network (Advanced only) and asset, and Swap a pair and amount. Each
screen still quotes and confirms on its own.
Type or dictate a request and get a card with the amount, fiat value,
recipient, route and estimated fee, or a small answer for balance and
spending questions. Review opens the prefilled Send, Receive or Swap screen,
so their confirm steps remain the only way to spend. Fees and swap returns
come from the wallet's own Send and swap quotes.
…lain words

An Explain action on the activity detail sheet (the item, its fee and an RGB
transfer's step), the swap failure screen and Send errors opens a small sheet
with Prismo. The on-device model gets only the item's structured facts and
its answer is kept only if it stays within them; otherwise, or with the model
off, a built-in explanation is shown.
Dismissible cards with Prismo and one action each: RGB backup failed or never
done, channel funds a recovery phrase can't restore (Advanced), little room to
receive on an RGB channel, a stuck swap or unconfirmed payment, a large
payment received and low mainnet fees. No model is involved. At most two show
at once, fee and activity checks are rate-limited, and dismissals are kept
per wallet with a cooldown per insight.
Fraction words use explicit letter classes instead of lookbehind and Unicode
property escapes, which also fixes matching an accented word like "metà" at
a word end. A test fails if those constructs appear in the KaleidoMind
sources.
# Conflicts:
#	screens/AIAssistantScreen.tsx
@bitwalt
bitwalt merged commit 92d43bb into main Oct 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant