Skip to content

UX Persona Audit — 2026-03-24 17:02 #1312

@Dexploarer

Description

@Dexploarer

Audit timestamp: 2026-03-24 17:02
Git state: develop @ d2b329d3 — pulled from origin/develop before this run
Latest commits:

  • d2b329d milady: salvage onboarding bugfixes from pr 1308
  • 6d4ef77 fix: correct config paths, plugin names, and CLI docs across repo (fix: correct config paths, plugin names, and CLI docs across repo #1309)
  • ad5d049 fix: make pre-commit hook executable
    Audit method: Fresh dev environment startup + API live-testing (localhost:31337) + UI audit (localhost:2138) + static code analysis
    App state: API :31337 ✅ Running — Dashboard UI :2138 ✅ Running
    Environment startup: PID 30112 — API ready in ~15s — UI ready in ~15s

Runtime Environment

Service Port Status Startup Time
API Server 31337 ✅ Running ~15s
Dashboard UI (Vite) 2138 ✅ Running ~15s
Agent Runtime ✅ Ready (Satoshi) ~15s total
Database (PGLite) ✅ ok within 15s

bun install: 2 packages installed in 2.49s. Postinstall: imagesnap installed. No errors.


Issue Index

ID Severity Title
I-01 P1 cloud.apiKey not redacted in GET /api/config — regex misses camelCase
I-02 P2 Duplicate env var reference in isAuthorized()MILADY_API_TOKEN silently ignored upstream
I-03 P2 /api/health reports "plugins": {"loaded": 0, "failed": 86} with no context
I-04 P2 fetchAgentName() calls /api/agents (404) — desktop title always falls back to "Milady"
I-05 P3 Settings sidebar nav buttons have no accessible name at the button element level
I-06 P3 CLAUDE.md documents wrong health-check URL (/api/agents instead of /api/health)
I-07 P3 Persistent "LOADING" / "Initializing entity" ARIA elements remain in DOM post-boot
I-08 P3 /api/permissions exposes _shellEnabled: true without auth when no token configured

Per-Persona Findings

1. Alex (Solo Dev) ⭐⭐⭐⭐ 4/5

Likes:

  • CLI boots cleanly via bun run dev; API ready in ~15s with clear log output
  • Plugin system loads dynamically with NODE_PATH correctly set in all 3 locations (verified)
  • bun install postinstall pipeline is well-guarded with a lock file and clear error messages

Dislikes:

  • /api/health shows "plugins": {"loaded": 0, "failed": 86} — alarming for someone debugging startup
  • No obvious way to distinguish "plugin not configured" vs "plugin crashed" in health output

Issues found:

  • I-03 (P2): GET /api/health{"plugins":{"loaded":0,"failed":86}} with ready:true. No breakdown of which plugins failed or why. A dev checking health will see 86 failures and assume the system is broken. packages/agent/src/api/server.ts (health route, confirmed live).

2. Maria (Non-Technical Creator) ⭐⭐⭐⭐ 4/5

Likes:

  • Onboarding wizard is polished: split left-nav + right-panel layout, VRM avatar, animated gold dot progress
  • Language picker visible top-right on the onboarding screen
  • OnboardingPanel now animates on step change (entry + child stagger) — smooth UX

Dislikes:

  • First-time visitors who already have a ~/.milady/milady.json skip onboarding entirely — no way to "restart" the flow without digging into Settings → Advanced → Reset Everything
  • Onboarding VRM reveal has a 3.5s timeout fallback but no visible loading indicator during that wait

Issues found:

  • No code-level issue; the 3.5s fallback is intentional (OnboardingWizard.tsx:81-85). P3 UX only — no spinner or skeleton during VRM load.

3. Jordan (DevOps Engineer) ⭐⭐⭐ 3/5

Likes:

  • Port configuration is fully env-driven (MILADY_API_PORT, MILADY_PORT, etc.)
  • Dev server orchestrator pre-picks free loopback ports when defaults are busy

Dislikes:

  • /api/health reporting 86 plugin failures while ready: true breaks standard health-check tooling (Kubernetes liveness probes, uptime monitors, etc.)
  • No structured endpoint to enumerate which connectors are actually running vs merely configured

Issues found:

  • I-03 (P2): Health endpoint reports "failed": 86 unconditionally. A standard k8s liveness probe checking for ready: true passes, but any operator inspecting the payload sees 86 failures with no context. This is misleading and will generate false alerts. packages/agent/src/api/server.ts (health route, confirmed live: {"plugins":{"loaded":0,"failed":86}}).

4. Sam (Crypto-Native Power User) ⭐⭐⭐ 3/5

Likes:

  • Wallet & RPC settings page is well-organized — Eliza Cloud vs Custom RPC toggle is clear
  • Cloud service toggles (inference, RPC, media, TTS, embeddings) are individually controllable

Dislikes:

  • GET /api/wallet and POST /api/wallet/export return 404 — wallet API endpoints appear to have been removed or renamed
  • No live wallet balance shown in the UI sidebar beyond the cloud credits button ($18.9)

Issues found:

  • GET /api/wallet → HTTP 404 (confirmed live). Route removed or not yet re-registered. Wallet import/export functions in packages/agent/src/api/wallet.ts still exist (getWalletAddresses imported in packages/app-core/src/api/server.ts:97) but route handler may not be wired up. P2 for Sam's workflow.

5. Priya (Enterprise Evaluator) ⭐⭐⭐ 3/5

Likes:

  • ensureCompatSensitiveRouteAuthorized() correctly blocks sensitive routes without a token in non-dev environments (packages/app-core/src/api/auth.ts:98-112)
  • redactDeep() recursive walk is architecturally sound — automatically covers future config fields
  • Prototype pollution prevention via isBlockedObjectKey() on PUT /api/config is solid

Dislikes:

  • GET /api/config exposes cloud.apiKey in plaintext because the redaction regex misses camelCase
  • Auth is completely open when no ELIZA_API_TOKEN / MILADY_API_TOKEN env var is set — any process on the same machine can read the full config including the cloud API key

Issues found:

  • I-01 (P1): SENSITIVE_KEY_RE = /api.?key/i does NOT match apiKey (camelCase). api.?key requires a separator character between api and key; apiKey has K then ey.? matches K but then key is not present. Live test confirmed: GET http://localhost:31337/api/config returns "cloud":{"apiKey":"eliza_baaf6c..."} in plaintext. File: packages/agent/src/api/server.ts:3945-3946.
  • I-02 (P2): isAuthorized() reads process.env.ELIZA_API_TOKEN ?? process.env.ELIZA_API_TOKEN (same var twice). Intent was process.env.MILADY_API_TOKEN ?? process.env.ELIZA_API_TOKEN. MILADY_API_TOKEN is silently ignored by the upstream auth gate even when set. File: packages/agent/src/api/server.ts:5961-5963.

6. Chen (Plugin Developer) ⭐⭐⭐⭐ 4/5

Likes:

  • NODE_PATH is set in all 3 required locations (verified in packages/agent/src/runtime/eliza.ts, scripts/run-node.mjs, apps/app/electrobun/src/native/agent.ts)
  • scripts/patch-deps.mjs postinstall pipeline is well-documented and runs correctly
  • bun run repair available for recovering broken plugin state

Dislikes:

  • 86 plugin load failures at startup — confusing for a developer trying to understand which plugins are actually available
  • No per-plugin failure reason surfaced in /api/health — just a count

Issues found:

  • I-03 (P2): Same health endpoint issue — but especially painful for plugin developers who need to distinguish between "plugin missing" and "plugin errored". packages/agent/src/api/server.ts (health route).

7. Riley (Mobile-First User) ⭐⭐⭐⭐ 4/5

Likes:

  • OnboardingPanel has max-md: responsive breakpoints — stacks vertically on narrow viewports (OnboardingPanel.tsx:33)
  • OnboardingStepNav collapses to horizontal pill row on mobile (max-md:flex-row in OnboardingStepNav.tsx:38)
  • Chat input (textbox "Chat message") and action buttons remain accessible at 375px

Dislikes:

  • Browser window resize to 375×812 did not change the Vite dev server viewport (remained 1274×768) — this is a dev-server limitation but means mobile testing requires a real device or emulator
  • Sidebar (conversation list) remains present in accessibility tree at mobile width — unclear if it collapses visually

Issues found:

  • No blocking code-level issues confirmed for mobile. Responsive CSS is present. P3 observation: sidebar collapse behavior at mobile breakpoints should be verified on real hardware.

8. Taylor (Accessibility User) ⭐⭐⭐ 3/5

Likes:

  • OnboardingStepNav now has aria-label on the <ul> (onboarding.stepNavigation) — added in latest commit d2b329d
  • Active step has aria-current="step" — correct ARIA pattern (OnboardingStepNav.tsx:109)
  • Completed step buttons have full aria-label with step number and "(completed)" suffix (OnboardingStepNav.tsx:91)
  • Chat textbox has placeholder="Type a message..." and button "Voice input" has a visible label

Dislikes:

  • Settings sidebar nav buttons (Cloud, AI Model, Coding Agents, Wallet & RPC, etc.) have no accessible name on the button element — only a generic child text node. Screen reader users hear "button" with no context.
  • Two persistent generic ARIA elements ("LOADING", "Initializing entity") remain in the DOM tree throughout the session, creating screen reader noise after the app is fully loaded
  • File attachment button[type="file"] in chat has no accessible label (observed as unlabeled button ref_36)

Issues found:

  • I-05 (P3): Settings sidebar buttons (ref_14–21) lack aria-label. Observed live via read_page on /settings. The button elements contain only a generic child <span> with text, not an accessible name at the <button> level. File: packages/app-core/src/components/ (settings sidebar component).
  • I-07 (P3): "LOADING" and "Initializing entity" generic elements persist in the DOM post-boot. These create screen reader noise. Observed live at both /companion and /settings routes.

9. Jamie (Multi-Platform User) ⭐⭐⭐⭐ 4/5

Likes:

  • Platform connector settings page (/settings) clearly lists configured vs unconfigured connectors
  • GET /api/connectors returns {"connectors":{}} cleanly — predictable shape for tooling

Dislikes:

  • No active platform connectors configured in this test run (empty connectors object)
  • Telegram/Discord/WeChat pairing UIs couldn't be tested without live credentials

Issues found:

  • No blocking issues confirmed for multi-platform use. Connection architecture (channel → plugin mapping) is correctly implemented per packages/agent/src/runtime/core-plugins.ts.

10. Casey (First-Time Visitor) ⭐⭐⭐⭐ 4/5

Likes:

  • App loads at HTTP 200 within 15 seconds
  • Post-onboarding companion view is immediately functional — chat input ready, cloud credits visible
  • Settings → Advanced → "Reset Everything" provides a clear path back to onboarding

Dislikes:

  • App skips onboarding entirely when ~/.milady/milady.json already exists — new contributors cloning the repo are dropped straight into the main UI with no orientation
  • The "LOADING" / "Initializing entity" banners at the top of every page suggest the app is still starting even when fully ready

Issues found:

  • I-07 (P3): "LOADING" and "Initializing entity" elements present on a fully-loaded session. First-time visitors may believe the app hasn't finished booting. Observed live via read_page.

Confirmed Non-Issues

Item Checked Verdict
/api/config auth gate open in dev Verified — isAuthorized() returns true when no token set By design for loopback dev; auto-generates temp token for non-loopback hosts (server.ts:5944-5958)
redactDeep() architecture Verified — recursive walk is correct Bug is in regex pattern only, not the walk logic
Electrobun health-check uses /api/agents Verified in code — actual code uses /api/health (agent.ts:506) /api/health works correctly and returns ready:true
OnboardingStepNav ARIA Verified — latest commit added aria-label on <ul> and aria-current="step" Fixed in d2b329d
NODE_PATH in 3 required locations Verified in eliza.ts, run-node.mjs, agent.ts All present, not broken
Prototype pollution guard on PUT /api/config Verified — isBlockedObjectKey() at every level of safeMerge() Working correctly

Prioritized Fix Checklist

P1 — Fix before next release

  • I-01 Fix SENSITIVE_KEY_RE to match camelCase apiKey. Current: /api.?key/i. Fix: add apiKey as an explicit alternation or change to /api[-_]?key|apiKey/i. File: packages/agent/src/api/server.ts:3945.

P2 — Fix in current sprint

  • I-02 Fix duplicate env var in isAuthorized(). Change process.env.ELIZA_API_TOKEN ?? process.env.ELIZA_API_TOKEN to process.env.MILADY_API_TOKEN ?? process.env.ELIZA_API_TOKEN. File: packages/agent/src/api/server.ts:5962.
  • I-03 Add plugin failure breakdown to /api/health. Either include per-plugin failure reasons, or separate "unconfigured optional plugins" from "crashed plugins" so operators can distinguish expected vs unexpected failures.
  • I-04 Fix fetchAgentName() to use the correct endpoint (e.g. /api/character which returns {"character":{"name":"Satoshi",...}}) instead of /api/agents (404). File: apps/app/electrobun/src/native/agent.ts:1301.

P3 — Backlog

  • I-05 Add aria-label to settings sidebar nav buttons (Cloud, AI Model, etc.) to match the visible text. File: settings sidebar component in packages/app-core/src/components/.
  • I-06 Update CLAUDE.md desktop agent section: replace "Health-polls /api/agents" with "Health-polls /api/health". File: CLAUDE.md (desktop agent lifecycle section).
  • I-07 Remove or conditionally render the "LOADING" / "Initializing entity" ARIA elements after the app is fully booted. File: apps/app/src/main.tsx or the top-level app component.
  • I-08 (Optional) Consider auth-gating /api/permissions for production deployments. Currently acceptable for local dev.

Positive Security Findings — Do Not Regress

  • ensureCompatSensitiveRouteAuthorized() correctly blocks wallet export without a token in non-dev environments (packages/app-core/src/api/auth.ts:98-112)
  • PUT /api/config uses safeMerge() with isBlockedObjectKey() at every recursive level — prototype pollution prevented (packages/agent/src/api/server.ts:2150-2174)
  • Timing-safe token comparison via crypto.timingSafeEqual in tokenMatches() (packages/app-core/src/api/auth.ts:34-39)
  • redactDeep() is recursive and covers future config fields automatically — no manual enumeration needed (packages/agent/src/api/server.ts:4015-4030)
  • Electrobun desktop health check uses /api/health with structured ready/agentState fields — robust to partial failures
  • $include directive blocked in config mutations — prevents arbitrary local file reads via config patches (packages/agent/src/api/server.ts:3956)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions