Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,11 @@
</a>
</p>

> **Status:** v0.1.0 · macOS only · early alpha
> **Status:** v0.1.0 · macOS + Windows 11 · early alpha

OpenChronicle gives AI agents a local, inspectable memory built from real screen and app context.

It runs on your Mac, captures structured context from what you're doing, and turns it into persistent Markdown memory: what you're working on, what you've decided, which tools you use, and which people or projects matter.
It runs on your Mac or Windows PC, captures structured context from what you're doing, and turns it into persistent Markdown memory: what you're working on, what you've decided, which tools you use, and which people or projects matter.

Any agent that can call tools can use it. MCP clients work especially well today, but OpenChronicle is meant to be a general memory layer for tool-using agents - not something tied to one protocol, one model provider, or one app.

Expand Down Expand Up @@ -120,7 +120,7 @@ The core idea is simple:

## What you get

* **Event-driven capture** from macOS AX events
* **Cross-platform capture** from macOS AX events or Windows UI Automation polling
* **Session-aware memory writing** instead of noisy per-snapshot logs
* **Human-readable Markdown memory**
* **Local SQLite indexing**
Expand All @@ -133,14 +133,22 @@ The core idea is simple:

## Install

Requires **macOS 13+** and **Xcode Command Line Tools** (`xcode-select --install`).
Requires either **macOS 13+** with Xcode Command Line Tools (`xcode-select --install`) or **Windows 11**.

macOS:
```bash
git clone https://github.com/Einsia/OpenChronicle.git
cd openchronicle
bash install.sh
```

Windows PowerShell:
```powershell
git clone https://github.com/Einsia/OpenChronicle.git
cd OpenChronicle
.\install.ps1
```

---

## Run
Expand Down
191 changes: 191 additions & 0 deletions README_ADB_AGENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
# OpenChronicle ADB Agent

This document explains the minimal Android ADB control layer added on top of
OpenChronicle.

Architecture:

```text
Codex / Claude Code / Hermes Agent
-> MCP tool call
-> OpenChronicle ADB Control MCP Server
-> adb
-> Android device
-> OpenChronicle event memory
```

The original OpenChronicle memory MCP server is unchanged. The ADB server is a
separate stdio MCP server started with:

```powershell
uv run openchronicle adb-mcp
```

## Tools

The ADB MCP server exposes these tools:

- `adb_list_devices`
- `adb_screenshot`
- `adb_dump_ui`
- `adb_tap`
- `adb_swipe`
- `adb_input_text`
- `adb_keyevent`
- `adb_current_app`
- `adb_open_app`
- `adb_read_logcat`

Every tool call appends an entry to OpenChronicle's daily
`event-YYYY-MM-DD.md` memory file. Screenshots are written under:

```text
<OPENCHRONICLE_ROOT>/adb/screenshots/
```

## Safety Policy

The ADB command runner denies high-risk operations before subprocess execution.

Blocked by default:

- `adb uninstall`
- `adb reboot`
- `adb root`, `adb unroot`, `adb remount`
- `adb disable-verity`, `adb enable-verity`
- `adb shell rm`, `adb shell rmdir`
- `adb shell pm clear`
- `adb shell pm uninstall`
- `adb shell cmd package clear`
- `adb shell cmd package uninstall`
- `adb shell settings put/delete/reset`
- `adb shell content delete`
- `adb shell su`
- `adb shell input keyevent POWER`
- `adb shell input keyevent 26`

`adb_input_text` rejects shell metacharacters and records only text length in
memory, not the raw text.

Agent operating rules:

1. Call `adb_screenshot` or `adb_dump_ui` before `adb_tap`, `adb_swipe`, or
`adb_input_text`.
2. Stop and ask the user before payments, login passwords, SMS codes, privacy
grants, account changes, or destructive workflows.
3. Do not ask for a generic adb shell. Use the fixed tools only.

## Connect a Phone

1. On the Android phone, enable Developer options.
2. Enable USB debugging.
3. Connect the phone by USB.
4. Accept the RSA debugging prompt on the phone.
5. Verify from PowerShell:

```powershell
E:\ai\product\.tooling\platform-tools\adb.exe devices -l
```

Expected output:

```text
List of devices attached
<serial> device ...
```

If the device is `unauthorized`, unlock the phone and accept the USB debugging
prompt. If it is `offline`, unplug/replug USB or restart the adb server.

## ADB Path Resolution

The server looks for adb in this order:

1. `OPENCHRONICLE_ADB_PATH`
2. `ADB_PATH`
3. `ANDROID_HOME/platform-tools`
4. `ANDROID_SDK_ROOT/platform-tools`
5. `PATH`
6. local `.tooling/platform-tools` or `.tool/platform-tools` directories in the
current working directory or one of its parents

Windows 11 example:

```powershell
$env:OPENCHRONICLE_ADB_PATH = "E:\ai\product\.tooling\platform-tools\adb.exe"
uv run openchronicle adb-mcp
```

WSL2 example using the Windows adb.exe:

```bash
export OPENCHRONICLE_ADB_PATH=/mnt/e/ai/product/.tooling/platform-tools/adb.exe
uv run openchronicle adb-mcp
```

Native Linux adb inside WSL2 can also work, but USB needs to be attached to WSL
with `usbipd-win`; using Windows `adb.exe` is simpler for this MVP.

## Codex / Claude / Hermes MCP Config

Generic stdio MCP config:

```json
{
"mcpServers": {
"openchronicle-adb": {
"command": "uv",
"args": ["run", "openchronicle", "adb-mcp"]
}
}
}
```

Codex CLI can also register a stdio server from the repo:

```powershell
codex mcp add openchronicle-adb -- uv run openchronicle adb-mcp
```

Claude Code can register the same stdio command:

```powershell
claude mcp add openchronicle-adb -- uv run openchronicle adb-mcp
```

Keep the existing OpenChronicle memory MCP configured separately if you also
want memory search tools. Use `openchronicle adb-mcp` only for phone control.

## Quick Self-Test

Run these from the repository root:

```powershell
E:\ai\product\.tooling\platform-tools\adb.exe devices -l
uv run pytest tests/test_adb_control.py
uv run openchronicle adb-mcp
```

MCP client smoke test:

```python
import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

async def main():
params = StdioServerParameters(
command="uv",
args=["run", "openchronicle", "adb-mcp"],
)
async with stdio_client(params) as (read, write):
async with ClientSession(read, write) as session:
await session.initialize()
tools = await session.list_tools()
print([tool.name for tool in tools.tools])

asyncio.run(main())
```

To call the phone through MCP, use `adb_list_devices` first. That call should
return the visible device list and create an `event-YYYY-MM-DD.md` memory entry.
14 changes: 9 additions & 5 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ OpenChronicle is a single daemon that ingests capture events, compresses them th

```mermaid
flowchart LR
W[mac-ax-watcher<br/>Swift binary]
W[platform capture source<br/>mac-ax-watcher · windows-uia-poller]

subgraph capture [Capture Layer]
direction TB
Expand Down Expand Up @@ -60,7 +60,7 @@ A typical 5-minute flush window, showing how one AX event propagates through to

```mermaid
sequenceDiagram
participant W as mac-ax-watcher
participant W as capture source
participant S0 as S0 dispatcher
participant S1 as S1 parser
participant BUF as capture-buffer
Expand Down Expand Up @@ -109,7 +109,7 @@ Defined in `src/openchronicle/daemon.py`.

| Task | Purpose |
|---|---|
| `capture` | Consumes `mac-ax-watcher` events, debounces, writes enriched JSON captures (incl. S1 fields) to `~/.openchronicle/capture-buffer/`. Heartbeat catches quiet periods. Also calls `SessionManager.on_event` on every capture so the session cutter sees the same signal. |
| `capture` | Consumes macOS `mac-ax-watcher` events or Windows UI Automation poll events, debounces, writes enriched JSON captures (incl. S1 fields) to the capture buffer. Heartbeat catches quiet periods. Also calls `SessionManager.on_event` on every capture so the session cutter sees the same signal. |
| `timeline` | Every 60s scans for closed wall-clock windows (default 1 min) and runs the `timeline` LLM stage to normalize each window while preserving authored text verbatim. Cleans buffer files older than the newest block. |
| `session` | Every `session.tick_seconds` (default 30), calls `SessionManager.check_cuts()` so idle-gap and timeout cuts fire even when the dispatcher is quiet. |
| `flush` | Every `session.flush_minutes` (default 5, clamped to 5-min floor), runs the reducer incrementally over the active session's newly closed timeline blocks (~5 of them at defaults) and appends `[flush]`-tagged partial entries to today's event-daily. |
Expand All @@ -133,8 +133,11 @@ Force-end is also called on daemon shutdown and on the 23:55 safety net, so a se

## On-disk state

macOS/Linux: `~/.openchronicle/`
Windows: `%LOCALAPPDATA%\OpenChronicle\`

```
~/.openchronicle/
<root>/
├── config.toml # single source of truth for runtime config
├── .pid # daemon PID; absence ⇒ stopped
├── .paused # sentinel — capture skips while present
Expand Down Expand Up @@ -163,10 +166,11 @@ src/openchronicle/
├── cli.py # Typer entry point
├── daemon.py # Async task orchestration
├── config.py # TOML loader, per-stage ModelConfig inheritance
├── paths.py # ~/.openchronicle/* paths
├── paths.py # platform data-root paths
├── logger.py # Rotating file sinks per component
├── capture/
│ ├── watcher.py # Spawns mac-ax-watcher, parses JSONL
│ ├── windows_uia.py # Windows UI Automation provider + poller
│ ├── event_dispatcher.py # Debounce / dedup / min-gap
│ ├── ax_capture.py # One-shot mac-ax-helper invocation
│ ├── ax_models.py # ax_tree_to_markdown, prune helpers
Expand Down
18 changes: 10 additions & 8 deletions docs/capture.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# Capture

Capture is the only layer that touches the outside world. It produces one JSON file per observation into `~/.openchronicle/capture-buffer/`; nothing above it ever talks to macOS directly.
Capture is the only layer that touches the outside world. It produces one JSON file per observation into the platform capture buffer (`~/.openchronicle/capture-buffer/` on macOS, `%LOCALAPPDATA%\OpenChronicle\capture-buffer\` on Windows); nothing above it talks to OS APIs directly.

## Two signal sources
## Platform signal sources

**`mac-ax-watcher`** (primary, event-driven). A vendored Swift binary that subscribes to AX notifications across all running apps: window focus, value changes (typing), title changes, app activation. It emits one JSON object per event on stdout. The Python side reads that stream line-by-line in `capture/watcher.py` → `capture/event_dispatcher.py`.

**Windows UI Automation poller** (Windows 11). A Python polling source reads the foreground app/window through Win32 APIs, emits macOS-style event names (`AXApplicationActivated`, `AXFocusedWindowChanged`, `AXValueChanged`), and lets the existing dispatcher apply the same debounce, dedup, and min-gap rules. It does not install low-level hooks.

**Heartbeat timer** (fallback). Every `heartbeat_minutes` (default 10), the scheduler fires a capture even if no event arrived — so long idle periods leave a trail. Set `heartbeat_minutes = 0` to disable entirely (watcher-only); values `>0` are clamped to a 60-second floor.

Both funnel into `capture_once` in `capture/scheduler.py`, which runs:

1. `ax_capture.capture_frontmost(focused_window_only=True)` — one-shot invocation of `mac-ax-helper` for the current window, pruned to `ax_depth` layers.
1. `ax_capture.capture_frontmost(focused_window_only=True)` — one-shot invocation of `mac-ax-helper` on macOS or Windows UI Automation on Windows, pruned to `ax_depth` layers.
2. `s1_parser.enrich()` — extracts `focused_element`, `visible_text`, and `url` from the AX tree (see [S1 fields](#s1-fields) below).
3. `screenshot.grab()` — unless `include_screenshot = false`.
4. `window_meta.active_window()` — app name, title, bundle_id via `NSRunningApplication`.
Expand Down Expand Up @@ -39,9 +41,9 @@ On top of the time-based knobs, the scheduler compares each built capture agains

This catches the case the time knobs can't: a screen that doesn't change (lock screen overnight, a paused video, an idle IDE) keeps generating AX events with the same content indefinitely. Without content-dedup those would both fill the buffer and keep the current session from ever idling out. Timestamps, triggers, and screenshots are excluded from the fingerprint so only meaningful changes count.

## AX depth — the #1 footgun
## AX / UIA depth — the #1 footgun

AX Trees for native Cocoa apps are shallow (5–15 layers). Electron apps (Claude Desktop, VS Code, Slack, Notion) nest user content 20–60 layers deep under chrome.
AX Trees for native Cocoa apps are shallow (5–15 layers). Electron apps (Claude Desktop, VS Code, Slack, Notion) and browser UIA trees can nest user content 20–60 layers deep under chrome.

**Default `ax_depth = 100`** was chosen after diagnosing silent capture misses: a 90-second Claude Desktop conversation about an interview at 18:00 was producing captures where "18:00" appeared at character 5639 of the tree — past any reasonable prune limit. At depth 8, the tree contained only window chrome and sidebar headers; at depth 100, the full conversation was there.

Expand Down Expand Up @@ -91,15 +93,15 @@ A 10×+ ratio means there's content past depth 30 you'd miss.

`trigger` is `{"event_type": "heartbeat"}` for timer captures and `{"event_type": "manual"}` for `capture-once`. Screenshot is omitted entirely when `include_screenshot = false`.

Secure fields (password inputs) are replaced with `"[REDACTED]"` at the helper levelthe Python side never sees them.
Secure fields (password inputs) are replaced with `"[REDACTED]"` at the macOS helper level. Windows UIA capture reads only the accessibility values exposed by the foreground app.

## S1 fields

Ported from Einsia-Partner's `s1_collector`. These are what downstream LLM stages consume — the raw `ax_tree` is kept only for future vision-model support and debugging.

- **`focused_element`** — `{role, title, value, is_editable, value_length}` for the currently focused AX element. This is the user's cursor context: what they're typing into, which sidebar row is selected, etc.
- **`visible_text`** — a length-capped markdown rendering of the AX tree (up to ~10 k chars). What the user is currently reading on screen.
- **`url`** — regex-extracted from `visible_text` when present; `null` otherwise.
- **`url`** — regex-extracted from browser address fields when present; `null` otherwise. Chrome and Edge are supported on Windows first.

Screenshots live in the capture JSON but are **not** passed to the timeline / reducer / classifier prompts. They exist for future vision-model paths and for debugging.

Expand Down Expand Up @@ -151,7 +153,7 @@ openchronicle rebuild-captures-index
openchronicle pause
```

Drops a `~/.openchronicle/.paused` sentinel. The watcher keeps streaming but `capture_once` short-circuits on sentinel presence. `resume` removes the sentinel.
Drops a `.paused` sentinel in the platform data root. The watcher/poller keeps running but `capture_once` short-circuits on sentinel presence. `resume` removes the sentinel.

## Smoke test

Expand Down
2 changes: 1 addition & 1 deletion docs/config.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Configuration

Runtime config lives at `~/.openchronicle/config.toml` (or `$OPENCHRONICLE_ROOT/config.toml`). It's created with sensible defaults the first time you run `openchronicle status`.
Runtime config lives at `~/.openchronicle/config.toml` on macOS/Linux, `%LOCALAPPDATA%\OpenChronicle\config.toml` on Windows, or `$OPENCHRONICLE_ROOT/config.toml` when overridden. It's created with sensible defaults the first time you run `openchronicle status`.

View the resolved config any time with:

Expand Down
Loading