Local-first: store data in a SQLite vault instead of Supabase - #7
Open
myaschmitz wants to merge 2 commits into
Open
Local-first: store data in a SQLite vault instead of Supabase#7myaschmitz wants to merge 2 commits into
myaschmitz wants to merge 2 commits into
Conversation
Phase 0 of the local-first migration described in docs/local-first-design.md. Pages and components no longer talk to Supabase directly. They go through a repository interface in src/lib/db, with the existing Supabase client behind a single adapter. Behavior is unchanged; this exists so a local SQLite/vault backend can be swapped in as one implementation rather than a rewrite. Notable points: - The repository scopes every query to the current user internally, so the 19 auth.getUser() calls that existed only to build a user_id filter are gone. - updated_at is now set by the adapter, not by each call site. - Settings reads always resolve to a full object, removing the default-settings literal that was duplicated across three pages. - Deck count aggregation moved into a shared helper. - upload-image.ts is replaced by the blob repository; resolveImageUrl moves to lib/image-url so it stays dependency-free for render paths. - The offline queue and its sync hook now share one applyOp helper and write through the repository, keeping the stale-write guard as an explicit ifUnmodifiedSince option. - Deleted image-upload.tsx and image-grid.tsx, which were unreferenced. Fixes a latent bug: undo looked up the newest review log by created_at, a column review_logs does not have. It now orders by reviewed_at. Tests move to mocking the repository seam instead of the Supabase client. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Phase 1 of the local-first migration. Koku now runs with no account, no
Supabase project, and no network: clone, npm install, npm run dev.
Data lives in a vault directory, KOKU_VAULT_DIR or ~/.koku/vault by default:
vault/
koku.sqlite decks, cards, review history, settings
blobs/ card images, addressed by content hash
Point KOKU_VAULT_DIR at a Drive/Dropbox/Syncthing folder to move a collection
between machines. Concurrent multi-device editing is not safe yet; the vault is
still a single SQLite file. The append-only op log that fixes that is phase 2.
How it fits together:
- schema.ts ports the Postgres migrations: no user_id or RLS, TIMESTAMPTZ as
ISO text, booleans as INTEGER, user_settings pinned to a single row. It is a
TS module rather than a .sql file so it survives bundling.
- server-repo.ts implements the repository over node:sqlite (built in, so no
native dependency). Booleans are coerced back on read, since SQLite returns
0/1 and the UI relies on truthiness.
- A single /api/local/rpc endpoint dispatches repository calls, and the client
adapter is a Proxy over it, so the method list is not restated per backend.
Only own methods on allow-listed repositories are reachable.
- Images are content-addressed under blobs/, so duplicating a card reuses the
same bytes and copy() is a no-op. Reads validate the key shape before
touching the filesystem.
- end_vacation_mode moves from a Postgres RPC to a local transaction.
Supabase is still available via NEXT_PUBLIC_KOKU_BACKEND=supabase.
Auth is gone: login, the auth callback/confirm routes, the account-delete
endpoint, the middleware, and the server client are all deleted, along with
user_id on every domain type. Landing page now redirects to /decks, and the
privacy/terms copy is updated to match local storage.
Requires Node 22+ for node:sqlite; engines and @types/node bumped to match.
Adds 19 tests covering boolean coercion, due-card filtering, the vacation-mode
shift, LIKE-wildcard escaping in duplicate detection, and the stale-write
guard. Also fixes a pre-existing type error in the import/export test surfaced
by the @types/node bump.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes Koku run entirely on the user's own machine: clone,
npm install,npm run dev. No account, no Supabase project, no network.This is phases 0 and 1 of
docs/local-first-design.md, which is added in the first commit and is the best place to start reviewing. Phase 2 (conflict-free multi-device sync) is deliberately not here — see the caveat at the bottom.Two commits, reviewable independently
1.
refactor: introduce storage-agnostic repository layer— no behavior change. Pages and components stop talking to Supabase directly and go through a repository interface insrc/lib/db, with the existing Supabase client behind a single adapter. This is the de-risking step: it makes the backend swap an adapter change rather than a rewrite.2.
feat: store data in a local SQLite vault instead of Supabase— adds the local backend, makes it the default, and removes auth.How it fits together
Data lives in a vault directory,
KOKU_VAULT_DIRor~/.koku/vaultby default:Decisions worth a look
node:sqlite(built into Node 22+) rather thanbetter-sqlite3, so there's no native module to compile. Cost:engines.nodemoves to>=22.Proxyclient adapter, so the repository's method list lives only in the interface instead of being restated per backend. Only own methods on allow-listed repositories are reachable; prototype keys likeconstructorand__proto__are rejected..sqlfile read at runtime, so it survives bundling.duplicateCardImagesbecomes a no-op. Blob reads validate the key shape before touching the filesystem.end_vacation_modemoves from a Postgres RPC to a local transaction.Removed
Auth in full: login, the auth callback/confirm routes, the account-delete endpoint, the middleware, and the Supabase server client — plus
user_idon every domain type and the 19auth.getUser()calls that existed only to build auser_idfilter. Also deletedimage-upload.tsxandimage-grid.tsx, which were unreferenced.Privacy and terms copy is updated to describe local storage. That's user-facing legal text and deserves a careful read.
Incidental fixes
created_at, a columnreview_logsdoesn't have. Now orders byreviewed_at.@types/nodebump.Verification
npm test(95 passing, 19 new),npm run lint,npx tsc --noEmit, andnpm run buildare all clean. The build no longer needs any Supabase env vars.Beyond that I ran the app against a live vault and exercised deck/card CRUD, due-card filtering and ordering, duplicate detection, review logging, settings round-trips, vacation-mode date shifting, image upload/serve/dedup, and the RPC and path-traversal guards.
Known limitation
Multi-device sync via Google Drive or Dropbox is not safe yet. The vault is still a single SQLite file, which is exactly the hazard section 3 of the design doc warns about: file-sync services replicate whole files plus
-wal/-shmsidecars, and two devices editing offline resolve as last-writer-wins. The README says this plainly. Phase 2 replaces the single file with per-device append-only op logs, which is what actually makes syncing safe.Supabase remains available via
NEXT_PUBLIC_KOKU_BACKEND=supabase— whether to keep that adapter long-term is still an open question in the design doc.