diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 12b28cf..6edce2f 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -43,6 +43,20 @@ jobs: - name: Validate packages.json schema run: bash scripts/validate-packages.sh + python: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Check Python syntax + run: | + mapfile -d '' files < <(find . -name '*.py' -not -path './.git/*' -print0) + bash scripts/check-python-syntax.sh "${files[@]}" + + - name: Test Glances disk aliases + working-directory: linux-server/glances + run: python3 -m unittest -v test_rename_disks.py + dryrun-smoke: runs-on: ubuntu-latest steps: @@ -51,6 +65,17 @@ jobs: - name: Root setup.sh dry-run on every platform run: bash scripts/dryrun-smoke.sh + shell-tests: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Test Pi backup failure status + run: bash linux-pi/backup/test-backup-failure.sh + + - name: Test CUPS policy rendering + run: bash linux-pi/cups/test-setup.sh + zsh-syntax: runs-on: ubuntu-latest steps: diff --git a/.gitignore b/.gitignore index 3c4163a..afb2ae2 100644 --- a/.gitignore +++ b/.gitignore @@ -58,7 +58,6 @@ linux-pi/*/ts-state/ linux-pi/homepage/config/logs/ linux-pi/homepage/config/custom.css linux-pi/homepage/config/custom.js -linux-pi/homepage/config/docker.yaml linux-pi/homepage/config/kubernetes.yaml linux-pi/homepage/config/proxmox.yaml diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index f4dd17a..0422bca 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -21,6 +21,12 @@ repos: files: ^packages\.json$ pass_filenames: false + - id: python-syntax-check + name: python syntax check + language: system + entry: bash scripts/check-python-syntax.sh + types: [python] + - repo: https://github.com/pre-commit/pre-commit-hooks rev: v5.0.0 hooks: diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..a283fac --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,118 @@ +# Computer-Setup + +Personal machine-provisioning repo: one unified installer for macOS, Linux +desktop (Ubuntu/Arch), and an Ubuntu Server LTS home server, plus per-platform +configs and docs. A Raspberry Pi (``, Debian) node lives in `linux-pi/` +— currently its own Docker Compose service stacks (a secondary AdGuard resolver); +folding it into the `setup.sh` base provisioning is still tracked in docs/TODO.md. + +This file is the shared instruction set for **every** coding agent working in +this repo (Claude Code, opencode, Codex, …). `CLAUDE.md` imports it and adds +Claude-Code-only notes on top; keep cross-agent guidance here, not there. + +## Entrypoints + +- `setup.sh` — installs everything for the detected platform. + Flags: `--optional --work --personal --base --tags --dry-run --platform --profile `. + The server platform is never auto-detected (`--profile server` or `--platform server` required). + `--base` installs only the high-priority base set; `--tags development,terminal` + installs base + those `packages.json` tag categories; a bare TTY run with no + selection flag prompts interactively except on the server profile. Selection + mechanics and the custom-step gating: `docs/PACKAGES.md`. +- `verify.sh` — read-only health check mirroring `setup.sh`'s selection logic. + Flags: `--optional --work --personal --all --platform ` + (no `--dry-run`). The server profile additionally checks NUT configuration and + services. Unknown flags warn and are ignored, they don't abort. + +## Architecture + +- `packages.json` — single source of truth for all package data. Managers are keyed + by platform (`{macos, ubuntu, arch, server}`); `_name` overrides the + install token; `environment` gates on `--work`/`--personal`; `custom` managers + carry an `install_command` (auto-run when `handled_by_setup`, else a reminder); + `tags` is a required category array. The tier/gating fields `priority`, `optional`, + `environment`, and `install_command` can be a scalar or a per-platform object. + **Full schema, per-platform resolution, the `environment` caveat, and the tag + filter live in `docs/PACKAGES.md` — read it before editing `packages.json`.** +- `lib/core.sh` — shared engine: arg parsing, platform detection, env filter, + jq selection, install loops, config deploys. `lib/verify.sh` — check engine. +- `platforms/.sh` — per-platform quirks only (bootstrap, manager + invocations). +- `dotfiles/` — configs shared across platforms (`tmux.conf`, + `ghostty.config`, `zshrc.example`, `zsh_plugins.txt`); the engine deploys + them from here. One zshrc base serves every platform including the headless + server — macOS-specific bits guard on `/opt/homebrew` or `$OSTYPE`, and the + desktop-only bits self-disable headless (notify hook no-ops without + `$DISPLAY`/`$WAYLAND_DISPLAY`, fastfetch keys off Ghostty or `$SSH_CONNECTION`, + version managers/zoxide are command-guarded). The override system + (`deploy_zshrc`) still lets a platform folder ship its own `zshrc.example` to + win over the base, but no platform currently does. +- `macOS/`, `linux-desktop/`, `linux-server/` — platform-specific configs, + docs, and thin shim scripts that exec the root entrypoints. `linux-pi/` holds + the Raspberry Pi node's Docker Compose service stacks (same + `/{docker-compose.yml,.env.example,ts-serve.json}` layout as + `linux-server/`), not yet wired into `setup.sh`. +- `scripts/dryrun-smoke.sh` — runs `setup.sh --dry-run` for every platform and + asserts it exits clean with install actions; also run in CI. + +`docs/UNIFICATION.md` is the design doc for this layout; `docs/CHANGELOG.md` records +what shipped and `docs/TODO.md` tracks remaining work. + +## Conventions + +- Pre-commit runs `shellcheck --severity=warning` on all shell scripts; + `zsh -n` checks `.zsh` files and `zshrc.example`; `scripts/validate-packages.sh` + enforces the `packages.json` schema (platform vocabulary, controlled tag set, + and the "no silent drop" rule — every platform a package targets must resolve a + valid priority tier and a boolean optional). All three also run in CI. +- Probe semantics in `lib/verify.sh` are platform-faithful ports — macOS has no + `command -v` fallback for casks/pipx/app-store, Linux falls back everywhere. + Don't "fix" the asymmetry without checking `docs/UNIFICATION.md` history. +- `--dry-run` must print every command without executing anything. Before committing + changes to `setup.sh`, `lib/`, `platforms/`, or `packages.json`, exercise it across + all four platforms; only one platform can run live. +- Before committing changes to `.env` handling, `custom` `install_command` shell + execution, or path/network code, perform a security review. +- App-store packages and `priority: "none"` entries are reminders only — never + auto-installed. + +## Coding conventions + +These apply to every agent (the repo is almost entirely Bash). + +- **Comments:** none by default. Add one only when the *why* is non-obvious — a + hidden constraint, a bug workaround, a subtle invariant. Never narrate *what* + the code does; well-named identifiers cover that. +- **No speculative design.** Don't build for hypothetical future requirements; + three similar lines beat a premature abstraction. Don't add features, refactors, + or abstractions beyond what the task needs. +- **Validate only at boundaries** (user input, external APIs). Trust internal code + and framework guarantees — no error handling or fallbacks for cases that can't + happen. +- **Prefer editing existing files** to creating new ones; delete removed code + cleanly rather than leaving back-compat shims. +- **Bash style:** + - `#!/usr/bin/env bash` shebang; `set -euo pipefail` at the top of every + non-trivial script. + - `[[ ]]` not `[ ]`; quote all expansions (`"$var"`, `"${arr[@]}"`). + - `printf` not `echo`; declare function-local vars with `local`; write errors to + stderr (`printf 'error: %s\n' "$msg" >&2`). + - `command -v foo` over `which foo`; herestring (`<<< "$var"`) over `echo "$var" |`. + +## Privacy & Security + +This repo is **public**. Never commit identifying or secret information. + +- Keep these out of tracked files entirely: tailnet names / MagicDNS suffixes + (`tailXXXXXX.ts.net`), real hostnames, server IPs, usernames, emails, tokens, + auth keys, and personal absolute paths. +- Put any machine-specific or private value in a `.env` file (gitignored + repo-wide) and ship a committed `.env.example` with placeholders instead — + e.g. `linux-server/forgejo/.env.example`, `macOS/forgejo-runner/.env.example`. + Scripts read these via `${VAR:-}` and source a local `.env` when + present; they never hardcode the real value. +- In docs and configs use placeholders: ``, ``, + ``, ``. Default to `.env` whenever a value is + identifying — prefer one more env var over leaking a real value. +- When editing, scan the diff for accidentally introduced real identifiers + before committing. diff --git a/CLAUDE.md b/CLAUDE.md index 5d44982..a134d58 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,105 +1,12 @@ -# Computer-Setup +@AGENTS.md -Personal machine-provisioning repo: one unified installer for macOS, Linux -desktop (Ubuntu/Arch), and an Ubuntu Server LTS home server, plus per-platform -configs and docs. A Raspberry Pi (`ollie-pi4`, Debian) node lives in `linux-pi/` -— currently its own Docker Compose service stacks (a secondary AdGuard resolver); -folding it into the `setup.sh` base provisioning is still tracked in TODO.md. +## Claude Code -## Entrypoints +The section above is imported from `AGENTS.md` — the shared instruction set for +all agents. Everything there applies. Below are Claude-Code-only notes for this +repo: -- `setup.sh` — installs everything for the detected platform. - Flags: `--optional --work --personal --base --tags --dry-run --platform --profile `. - The server platform is never auto-detected (`--profile server` or `--platform server` required). - Category selection: `--base` installs only the high-priority base set; `--tags - development,terminal` installs base + those `packages.json` tag categories - (enabled work/personal apps install regardless of category). Run bare on a TTY - with no selection flag and `core_maybe_prompt_selection` prompts interactively; - it is skipped on the server profile and in non-interactive/CI runs. The filter - is implemented by `tagok()` in `CORE_JQ_DEFS`, which reads `TAG_FILTER_ACTIVE` - and `SELECTED_TAGS` from the environment — inactive by default, so flag-driven - and CI runs are unchanged. The dedicated custom-install steps (tailscale, - claude-code, docker) gate on `pkg_selected` so they honor the selection too; - the server profile keeps the filter inactive, so they install as before there. -- `verify.sh` — read-only health check mirroring `setup.sh`'s selection logic. - Flags: `--optional --work --personal --all --platform ` - (no `--dry-run`; `--platform server`/`--profile server` is rejected — nothing - legacy existed to port). Unknown flags warn and are ignored, they don't abort. - -## Architecture - -- `packages.json` — single source of truth for all package data. Managers are - keyed by platform (`{macos, ubuntu, arch, server}`); `_name` - overrides the install token; `environment` gates on `--work`/`--personal`; - `custom` managers carry `install_command` (string or per-platform object) — - run by the engine when `handled_by_setup` is true, otherwise printed as a - manual-install reminder. - - `priority`, `optional`, `environment`, and `install_command` each accept a - **scalar** (applies to every platform) **or a per-platform object** keyed by - platform (e.g. `"priority": { "macos": "medium", "ubuntu": "none" }`). The - engine resolves them via the `prfor`/`optfor`/`envfor`/`icfor` jq defs in - `lib/core.sh`. This is what lets one entry serve platforms that differ in - tier/optionality/gating, instead of splitting into duplicate entries. - - **`environment` caveat:** its scalar form is itself an *array* (`["work"]`), - so the per-platform form is detected as an *object* (`{ "ubuntu": ["work"] }`) - — array means legacy/all-platforms, object means per-platform. Keep the - per-platform value an object-of-arrays. - - `tags` — required non-empty array of descriptive categories from the - controlled vocabulary in `scripts/validate-packages.sh`. Metadata only - (grouping/docs); the install engine ignores them. -- `lib/core.sh` — shared engine: arg parsing, platform detection, env filter, - jq selection, install loops, config deploys. `lib/verify.sh` — check engine. -- `platforms/.sh` — per-platform quirks only (bootstrap, manager - invocations). -- `dotfiles/` — configs shared across platforms (`tmux.conf`, - `ghostty.config`, `zshrc.example`, `zsh_plugins.txt`); the engine deploys - them from here. One zshrc base serves every platform including the headless - server — macOS-specific bits guard on `/opt/homebrew` or `$OSTYPE`, and the - desktop-only bits self-disable headless (notify hook no-ops without - `$DISPLAY`/`$WAYLAND_DISPLAY`, fastfetch keys off Ghostty or `$SSH_CONNECTION`, - version managers/zoxide are command-guarded). The override system - (`deploy_zshrc`) still lets a platform folder ship its own `zshrc.example` to - win over the base, but no platform currently does. -- `macOS/`, `linux-desktop/`, `linux-server/` — platform-specific configs, - docs, and thin shim scripts that exec the root entrypoints. `linux-pi/` holds - the Raspberry Pi node's Docker Compose service stacks (same - `/{docker-compose.yml,.env.example,ts-serve.json}` layout as - `linux-server/`), not yet wired into `setup.sh`. -- `scripts/dryrun-smoke.sh` — runs `setup.sh --dry-run` for every platform and - asserts it exits clean with install actions; also run in CI. - -`docs/UNIFICATION.md` is the design doc for this layout; `CHANGELOG.md` records -what shipped and `TODO.md` tracks remaining work. - -## Conventions - -- Pre-commit runs `shellcheck --severity=warning` on all shell scripts; - `zsh -n` checks `.zsh` files and `zshrc.example`; `scripts/validate-packages.sh` - enforces the `packages.json` schema (platform vocabulary, controlled tag set, - and the "no silent drop" rule — every platform a package targets must resolve a - valid priority tier and a boolean optional). All three also run in CI. -- Probe semantics in `lib/verify.sh` are platform-faithful ports — macOS has no - `command -v` fallback for casks/pipx/app-store, Linux falls back everywhere. - Don't "fix" the asymmetry without checking `docs/UNIFICATION.md` history. -- `--dry-run` must print every command without executing anything; it is the - primary cross-platform test mechanism (only one platform can run live). -- App-store packages and `priority: "none"` entries are reminders only — never - auto-installed. - -## Privacy & Security - -This repo is **public**. Never commit identifying or secret information. - -- Keep these out of tracked files entirely: tailnet names / MagicDNS suffixes - (`tailXXXXXX.ts.net`), real hostnames, server IPs, usernames, emails, tokens, - auth keys, and personal absolute paths. -- Put any machine-specific or private value in a `.env` file (gitignored - repo-wide) and ship a committed `.env.example` with placeholders instead — - e.g. `linux-server/forgejo/.env.example`, `macOS/forgejo-runner/.env.example`. - Scripts read these via `${VAR:-}` and source a local `.env` when - present; they never hardcode the real value. -- In docs and configs use placeholders: ``, ``, - ``, ``. Default to `.env` whenever a value is - identifying — prefer one more env var over leaking a real value. -- When editing, scan the diff for accidentally introduced real identifiers - before committing. +- Use **`/verify`** to satisfy the cross-agent verification requirement in + `AGENTS.md`. +- Use **`/security-review`** to satisfy the cross-agent security-review requirement + in `AGENTS.md`. diff --git a/HANDOFF.md b/HANDOFF.md deleted file mode 100644 index 5f281dd..0000000 --- a/HANDOFF.md +++ /dev/null @@ -1,44 +0,0 @@ -# Session Handoff - -Scratchpad for in-progress work that spans more than one session — live -infrastructure state, decisions made outside of code, and what's next. -Branch-specific; delete or trim entries once they're fully landed and the -branch merges. - -## 2026-06-17 — Forgejo HTTPS-over-Tailscale, live on ollie-server - -Branch: `refactor/server-streamline-divergence` (PR [#47](https://github.com/ulises-c/Computer-Setup/pull/47)) - -`linux-server/HTTPS.md` (added in `8ff8a2c`, on a different machine) documents -per-service HTTPS via a Tailscale sidecar per service. This session executed -that plan for Forgejo on the actual server (`ollie-server`) and documented two -gotchas hit along the way. - -**Code changes (committed, this session):** -- `linux-server/HTTPS.md` — resolved the auth-method decision (OAuth client + - `tag:container`, not a reusable key); documented the ACL-tag gotcha and a - network-namespace gotcha that will recur for every other service conversion -- `TODO.md` — added a tracked checklist for the 11 remaining services + 2 open - decisions (NPM retire/keep, Homepage sidecar vs. main-node) -- `SSH_and_GPG/create_ssh_key.sh` + `README.md` — the self-hosted-server SSH - port was hardcoded to `2222` (Forgejo's old host-published mapping); now - prompts with default `22`, matching the sidecar setup. Verified live with - `ssh-keyscan -p 22 forgejo..ts.net`. - -**Live state on `ollie-server` (not in git):** -- `linux-server/forgejo/.env`: `TS_AUTHKEY` set to the (now read+write-scoped) - `tailscale-proxy` OAuth client secret; `FORGEJO_DOMAIN=forgejo..ts.net` -- `forgejo-ts` + `forgejo` containers running under the new sidecar - `docker-compose.yml`; old host-published ports (`3300`, `2222`) are gone -- Verified: `https://forgejo..ts.net/` → `200`; SSH banner on `:22` -- Tailscale admin console changes (made by the user, not visible in this repo): - existing `tailscale-proxy` OAuth client elevated from read-only to - read+write scope; `tag:container` added to the tailnet ACL's `tagOwners` - (owner: `autogroup:admin`) - -**Not done yet:** -- Existing git remotes pointing at the old `http://...:3300` / `ssh://...:2222` - Forgejo address still need updating to `git@forgejo..ts.net:user/repo.git` - (on every machine that had a clone, not just this one) -- Forgejo Site Administration → confirm the app URL picked up the new `ROOT_URL` -- Continue the rollout: Portainer is next per the `TODO.md` table diff --git a/SSH_and_GPG/README.md b/SSH_and_GPG/README.md index 24b671c..ed8147a 100644 --- a/SSH_and_GPG/README.md +++ b/SSH_and_GPG/README.md @@ -64,7 +64,7 @@ Creates an Ed25519 SSH key for a remote machine (e.g. a home server or Tailscale ```bash bash add_remote_host.sh # or pre-fill inputs via env vars: -HOST_ALIAS="homepc" REMOTE_HOST="192.168.1.100" REMOTE_USER="jane" PORT="22" bash add_remote_host.sh +HOST_ALIAS="homepc" REMOTE_HOST="" REMOTE_USER="" PORT="22" bash add_remote_host.sh ``` --- diff --git a/SSH_and_GPG/add_remote_host.sh b/SSH_and_GPG/add_remote_host.sh index 0984a34..3d58f3b 100755 --- a/SSH_and_GPG/add_remote_host.sh +++ b/SSH_and_GPG/add_remote_host.sh @@ -38,7 +38,7 @@ prompt() { } prompt HOST_ALIAS "SSH alias (friendly name, e.g. homepc)" -prompt REMOTE_HOST "Remote hostname or IP (e.g. 192.168.1.100 or mypc.local)" +prompt REMOTE_HOST "Remote hostname or IP (e.g. or .local)" prompt REMOTE_USER "Remote username" prompt PORT "SSH port" "22" prompt KEY_NAME "Key file name (no path)" "$HOST_ALIAS" diff --git a/agentic-ai/Claude/CLAUDE.md b/agentic-ai/Claude/CLAUDE.md index 0cc8903..9cc4ec1 100644 --- a/agentic-ai/Claude/CLAUDE.md +++ b/agentic-ai/Claude/CLAUDE.md @@ -2,9 +2,7 @@ @rules/common/agents.md @rules/common/railguard.md @rules/bash/style.md -# graphify -- **graphify** (`~/.claude/skills/graphify/SKILL.md`) - any input to knowledge graph. Trigger: `/graphify` -When the user types `/graphify`, invoke the Skill tool with `skill: "graphify"` before doing anything else. + # Railguard — Active Guardrails @@ -17,6 +15,10 @@ Railguard is monitoring this session. Every tool call (Bash, Write, Edit, Read) - **File writes are snapshotted.** Every Write/Edit you make is backed up before execution. The human can rollback any change. - **Everything is logged.** All tool calls and decisions are recorded in `.railguard/traces/`. +## Writing files + +Prefer `Write`/`Edit` over Bash redirects (`cat <`, `echo >`, `printf >`). Tool writes are snapshotted and skip the Bash path-fence scan, which matches fenced-path *strings* in command text — so a command merely mentioning a fenced path (heredoc, issue body) is blocked even if it never touches it. Switching a fence-blocked heredoc to `Write` is intended remediation, not evasion. + ## If something goes wrong If the human asks you to undo changes, fix a mistake, or rollback: @@ -47,6 +49,13 @@ You **can** help the user customize their Railguard policy. This is encouraged: - **Run `railguard status`** to show the current protection state. All changes to Railguard policy require explicit human approval. You generate the change, the user reviews and accepts or rejects it. Changes take effect on the next tool call — no restart needed. +### Policy layers + +- **Global** `railguard.yaml` / `~/.railguard.yaml` (resolved upward from cwd) — base rules; edits gated to **ask**. +- **Per-project** `.railguard.local.yaml` (project root) — additive `fence.allowed_paths` only; cannot weaken `denied_paths` or disable the fence. Honored only if global sets `fence.allow_local_overrides: true`; the project cannot opt itself in. + +Out-of-project path keeps prompting and the human wants it for this project only → add it to `.railguard.local.yaml`, not the global policy. Override ignored = global `allow_local_overrides: true` missing. Gitignore it unless the exception is shared. Details: `docs/per-project-allowlist.md`. + ## Do NOT attempt to - Run `railguard uninstall` — it will be blocked. diff --git a/agentic-ai/Claude/README.md b/agentic-ai/Claude/README.md index 2d0e37c..e2c31d4 100644 --- a/agentic-ai/Claude/README.md +++ b/agentic-ai/Claude/README.md @@ -1,6 +1,8 @@ # Claude Code Config -Version-controlled source of truth for `~/.claude/` settings, hooks, and rules. Running `install.sh` wires everything up via symlinks so changes here take effect immediately. +Version-controlled source of truth for `~/.claude/` settings, hooks, rules, and +on-demand docs. Running `install.sh` copies the settings template and links the +rest of the configuration. ## Activation @@ -10,12 +12,16 @@ bash agentic-ai/Claude/install.sh ``` This will: -- Back up your existing `~/.claude/settings.json` (if not already a symlink) -- Symlink `~/.claude/settings.json` → this `settings.json` +- Back up your existing `~/.claude/settings.json` when it differs from the template +- Copy this `settings.json` to `~/.claude/settings.json` - Symlink `~/.claude/CLAUDE.md` → this `CLAUDE.md` - Symlink `~/.claude/rules/` → this `rules/` +- Symlink `~/.claude/docs/` → this `docs/` +- Symlink this `railguard.yaml` → `~/.railguard.yaml` - Symlink each `hooks/*.sh` script into `~/.claude/hooks/` -- Install `railguard` via `cargo install railguard` (requires Rust/cargo; skipped if already installed) +- Install or update the Railguard fork with + `cargo install --git https://github.com/ulises-c/railguard` (requires + Rust/cargo; an existing binary is kept with a warning when cargo is unavailable) - Run `railguard install` to register it as a global PreToolUse hook Restart Claude Code after running. @@ -34,9 +40,11 @@ The hooks below are the primary safety layer. ### PreToolUse: `railguard` (all tools) -Runtime policy enforcer installed globally via `cargo install railguard`. Policy lives in `railguard.yaml`; custom blocklist/allowlist are left empty since `validate-bash.sh` owns those patterns. +Runtime policy enforcer installed globally from the Railguard fork by `install.sh`. +Policy lives in `railguard.yaml`; custom command blocklists are left empty because +`validate-bash.sh` owns those patterns. -- **Path fence**: denies access to `~/.ssh`, `~/.aws`, `~/.config/gcloud`, `/etc`; explicitly allows `~/.claude` and `/tmp` +- **Path fence**: applies the denied and allowed roots declared in `railguard.yaml` - **Traces**: every tool call logged to `.railguard/traces/` - **Snapshots**: pre-edit state captured for Write/Edit to `.railguard/snapshots/` - **Memory integrity**: session-start warns on untracked memory files (`railguard memory verify`) @@ -86,9 +94,12 @@ rules/ common/ general.md — language-agnostic coding principles agents.md — when to self-invoke Plan / Explore / review / verify - railguard.md — how to work under the Railguard guardrails (rules it enforces, path fence, rollback) + railguard.md — slim always-loaded Railguard behavior and reference routing bash/ style.md — bash scripting conventions +docs/ + RAILGUARD.md — expected behavior and bug-reporting protocol + per-project-allowlist.md — local path-allowlist configuration ``` Add a new language by creating `rules//style.md` and adding an `@` line to `CLAUDE.md`. @@ -117,11 +128,11 @@ echo '{"tool_input":{"command":"ls -la"}}' | bash agentic-ai/Claude/hooks/valida echo $? # Should exit 2 (blocked) -echo '{"tool_input":{"file_path":"/Users/ulises/.ssh/authorized_keys"}}' | bash agentic-ai/Claude/hooks/validate-write.sh +echo '{"tool_input":{"file_path":"/Users//.ssh/authorized_keys"}}' | bash agentic-ai/Claude/hooks/validate-write.sh echo $? # Should exit 0 (allowed) -echo '{"tool_input":{"file_path":"/Users/ulises/github/project/main.py"}}' | bash agentic-ai/Claude/hooks/validate-write.sh +echo '{"tool_input":{"file_path":"/Users//github/project/main.py"}}' | bash agentic-ai/Claude/hooks/validate-write.sh echo $? ``` @@ -137,4 +148,6 @@ The operative trust model: `bypassPermissions` + hooks is a guardrail against ac ## Adding settings -All user-level Claude Code settings live here going forward. Edit `settings.json` directly — the symlink means changes are live immediately (no re-run of `install.sh` needed). +All user-level Claude Code settings originate here. Edit the template, then re-run +`install.sh` to copy it into place. Claude Code may rewrite the live copy with +machine-specific state; `settings-drift.sh` reports meaningful differences. diff --git a/agentic-ai/Claude/docs/RAILGUARD.md b/agentic-ai/Claude/docs/RAILGUARD.md new file mode 100644 index 0000000..8cd4977 --- /dev/null +++ b/agentic-ai/Claude/docs/RAILGUARD.md @@ -0,0 +1,120 @@ +# Railguard — Expected Behavior & Bug Reporting + +On-demand reference — read this when Railguard does something unexpected, or +when reporting a false positive or improvement. It is deliberately **not** +imported into every prompt: the slim always-loaded guidance is +`rules/common/railguard.md`, and the `railguard:start/end` block in `CLAUDE.md` +is auto-generated by `railguard install` (never hand-edit inside it). + +Two repos share responsibility: + +- **[ulises-c/railguard](https://github.com/ulises-c/railguard)** — the tool + itself. Real fixes land there; bugs and improvements are filed as issues + there. +- **Computer-Setup `agentic-ai/Claude/`** (this repo) — the deployed + configuration: the `railguard.yaml` policy, the slim rule, and this doc. + `install.sh` copies `settings.json`, links the remaining Claude config, and + links the global policy separately. After installation, this doc is + addressable as `~/.claude/docs/RAILGUARD.md`. + +## Expected behavior + +Railguard intercepts every tool call (Bash, Read, Write, Edit, Memory) and +returns one of three decisions: + +- **allowed** — proceeds; you won't usually notice. +- **ask** — the human approves or denies. Wait for the decision; don't route + around it. +- **blocked / denied** — refused. **Never retry the same command** — re-issuing + it with cosmetic changes (new flags, base64, `eval`, a wrapper) trips + behavioral-evasion detection and escalates toward a session kill. Find a + genuinely different, safer approach, or ask the human. If the safer approach + is legitimately different (e.g. pushing a *new* branch instead of + force-pushing one), say so explicitly so the human can approve past any + evasion flag. + +### Rules it enforces + +Hitting these is Railguard working as intended, not a bug (defaults plus the +`validate-bash.sh`/`validate-write.sh` hooks): + +- **History/data loss (ask or block):** `git push --force` (force-push to + `main`/`master` is hard-blocked), `git reset --hard`, `git clean -f`, + `rm -rf` on `/` `~` `$HOME`, `terraform destroy`, `DROP TABLE`. +- **Escalation/staging (block):** `sudo` — run escalations yourself; + `git add -A` / `--all` / `.` — stage paths explicitly so secrets aren't + swept in. +- **Network/exfiltration (ask or block):** `curl | sh`, encoded payloads, + outbound `curl -X POST`, `wget`, `ssh`/`scp`/`rsync`, `env` dumps. +- **Path fence:** `~/.ssh`, `~/.aws`, `~/.config/gcloud`, `/etc` are denied; + allowed roots are `~/.claude`, `/tmp`, `~/github`, `~/Github`, `~/Bitbucket`. + `~/.gnupg`/`~/.config/gh` stay readable for GPG signing and `gh`. + +### The fence scans command text + +The Bash fence matches fenced path **strings** in the command, so a command +that merely *mentions* a fenced path — in a heredoc, an issue body, install +docs — gets blocked even though it never touches that path. + +- Author file content with `Write`/`Edit`, not `cat <`, `echo >`, or + `printf >` redirects. Tool writes are snapshotted and not scanned by the + Bash fence. +- If content must reference a fenced path, put it in a file via `Write` and + pass it by path (`--body-file`, `--file`, stdin redirect). +- A Bash command fence-blocked because its *content* quoted a fenced path → + switching to `Write` is the **intended remediation**, not evasion. +- A command blocked because it actually *accesses* a fenced path → do not + retry it in any form; find a different approach or ask the human. + +## Bug bounty & fixes + +Railguard is in active development and errs toward over-blocking: false +positives are expected, silent allows are not. A decision is a **false +positive** when the gated command neither touches a protected resource nor +performs a gated operation — it merely *resembles* one (a path-shaped string, +a slash token, benign flags that look like obfuscation). Anything under +"Rules it enforces" firing on a real match is not a bug. + +### Reporting protocol + +1. **Work around it in-session** without evasion — usually: author with + `Write`/`Edit` and pass by path, or reword the command text. +2. **Log it below** under Known false positives: what triggers the misfire and + the workaround. Because `~/.claude/docs` links into the Computer-Setup + checkout, commit and sync that entry there. If you are working in another + repo and cannot do that, file the upstream issue first and defer the local log. +3. **Take it upstream.** Check `gh issue list --repo ulises-c/railguard` for an + existing issue; file one if it's new, and link it from the log entry. The + same goes for non-bug improvement ideas — upstream issue, no log entry + needed. +4. **Promote habit fixes.** When the misfire keeps recurring because of *how* + an agent works by default (e.g. reaching for a Bash redirect instead of + `Write`), add a one-line "do X, not Y" to the relevant instructions file + (`AGENTS.md` for all agents, a `CLAUDE.md` overlay, or the global rules) so + it stops recurring. Example already applied: *prefer `Write`/`Edit` over + `cat <` redirects.* + +### Known false positives + +- **Leading-slash tokens in Bash command text.** A `/word` token — a commit + message mentioning a slash-command name like `/verify` — trips the path + fence, which reads it as a filesystem path outside the allowed roots, even + though the command never touches such a path. + - *Workaround:* keep slash-prefixed names out of Bash command text — reword, + or put the content in a file via `Write` and pass `--body-file`/path. + - *Upstream:* [railguard#17](https://github.com/ulises-c/railguard/issues/17) + — fence should only match tokens that parse as real path arguments. + +- **Fenced-path strings in command text** (heredocs, issue/PR bodies): blocked + even when the path is never accessed. + - *Workaround (intended remediation):* `Write`/`Edit` + pass by path — see + "The fence scans command text" above. + - *Upstream:* [railguard#17](https://github.com/ulises-c/railguard/issues/17) + (same root cause: text-shape matching instead of parsed path arguments). + +- **Behavioral-evasion / interpreter-obfuscation detectors on benign + commands** — legitimately rewording a denied approach, or ordinary + `bash -c`/inline-interpreter use, can be read as evasion. + - *Workaround:* when taking a genuinely different approach after a block, + state explicitly how it differs so the human can approve past the flag. + - *Upstream:* [railguard#18](https://github.com/ulises-c/railguard/issues/18). diff --git a/agentic-ai/Claude/docs/per-project-allowlist.md b/agentic-ai/Claude/docs/per-project-allowlist.md new file mode 100644 index 0000000..ae87416 --- /dev/null +++ b/agentic-ai/Claude/docs/per-project-allowlist.md @@ -0,0 +1,29 @@ +# Per-project allowlist overrides + +Railguard can grant one project access to an additional path without widening the +global allowlist for every project. The global policy must explicitly opt in; a +project cannot enable this feature for itself. + +In the global `railguard.yaml`: + +```yaml +fence: + enabled: true + allow_local_overrides: true +``` + +In the project root, create `.railguard.local.yaml`: + +```yaml +fence: + allowed_paths: + - "" +``` + +The local file is additive only. It can add allowed paths, but it cannot remove +denied paths, change other rules, or disable the fence. Denied paths still take +precedence over allowed paths. A malformed local file is ignored with a warning. + +Treat the override as per-machine checkout state and gitignore it unless the path +is a safe, shared project dependency. The implementation and issue history live in +[ulises-c/railguard](https://github.com/ulises-c/railguard/blob/main/docs/per-project-allowlist.md). diff --git a/agentic-ai/Claude/hooks/driftcheck.sh b/agentic-ai/Claude/hooks/driftcheck.sh index 00a23b5..4165d99 100755 --- a/agentic-ai/Claude/hooks/driftcheck.sh +++ b/agentic-ai/Claude/hooks/driftcheck.sh @@ -6,21 +6,25 @@ # - has shebang but not executable → flag (meant to run but can't) # - is executable but no shebang → flag (can run but no interpreter declared) # Library/sourced files (no shebang, not executable) are intentionally skipped. -# Repo-relative glob patterns in ~/.claude/hooks/driftcheck-ignore are exempt -# (for repos whose documented convention conflicts, e.g. scripts that are -# intentionally non-executable because a Dockerfile chmods its copies). +# Repo-root-relative glob patterns in ~/.claude/hooks/driftcheck-ignore +# (global) and /.driftcheckignore (per-repo) are exempt (for repos +# whose documented convention conflicts, e.g. scripts that are intentionally +# non-executable because a Dockerfile chmods its copies). The hook runs from +# the repo root regardless of where the session started, so patterns always +# match against repo-root-relative paths. set -euo pipefail trap 'exit 2' ERR git rev-parse --git-dir &>/dev/null || exit 0 +cd "$(git rev-parse --show-toplevel)" ignore_patterns=() -ignore_file="$HOME/.claude/hooks/driftcheck-ignore" -if [[ -f "$ignore_file" ]]; then +for ignore_file in "$HOME/.claude/hooks/driftcheck-ignore" .driftcheckignore; do + [[ -f "$ignore_file" ]] || continue while IFS= read -r pat; do [[ -n "$pat" && "$pat" != '#'* ]] && ignore_patterns+=("$pat") done < "$ignore_file" -fi +done issues=() diff --git a/agentic-ai/Claude/install.sh b/agentic-ai/Claude/install.sh index a25592a..f857b70 100755 --- a/agentic-ai/Claude/install.sh +++ b/agentic-ai/Claude/install.sh @@ -10,6 +10,11 @@ CLAUDE_DIR="$HOME/.claude" HOOKS_DIR="$CLAUDE_DIR/hooks" SETTINGS="$CLAUDE_DIR/settings.json" +if [[ -d "$CLAUDE_DIR/docs" && ! -L "$CLAUDE_DIR/docs" ]]; then + printf 'error: %s is a directory; move or remove it before installing\n' "$CLAUDE_DIR/docs" >&2 + exit 1 +fi + printf 'Installing from: %s\n' "$REPO_DIR" # settings.json is COPIED, not symlinked: Claude Code rewrites its user @@ -38,6 +43,12 @@ rm -f "$CLAUDE_DIR/rules" ln -sf "$REPO_DIR/rules" "$CLAUDE_DIR/rules" printf 'Linked: rules/\n' +# Symlink docs directory (on-demand references pointed at by rules, e.g. +# ~/.claude/docs/RAILGUARD.md — not @imported, read only when needed) +rm -f "$CLAUDE_DIR/docs" +ln -sf "$REPO_DIR/docs" "$CLAUDE_DIR/docs" +printf 'Linked: docs/\n' + # Symlink railguard policy (global: find_policy_file walks up from cwd) ln -sf "$REPO_DIR/railguard.yaml" "$HOME/.railguard.yaml" printf 'Linked: railguard.yaml → ~/.railguard.yaml\n' diff --git a/agentic-ai/Claude/rules/common/railguard.md b/agentic-ai/Claude/rules/common/railguard.md index 87fd6dd..e325cdd 100644 --- a/agentic-ai/Claude/rules/common/railguard.md +++ b/agentic-ai/Claude/rules/common/railguard.md @@ -1,28 +1,8 @@ # Working Under Railguard -[Railguard](https://github.com/ulises-c/railguard) intercepts every tool call (Bash, Read, Write, Edit, Memory) and decides to **allow**, **ask**, or **block** it. Rollback recipes and the self-protection list live in the auto-managed `# Railguard — Active Guardrails` block of `CLAUDE.md`; this file covers how to *work* under it. - -## Reading the response - -- **allowed** — proceeds; you won't usually notice. -- **ask** — the human approves or denies. Wait for the decision; don't route around it. -- **blocked / denied** — refused. **Never retry the same command** — re-issuing it with cosmetic changes (new flags, base64, `eval`, a wrapper) trips behavioral-evasion detection and escalates toward a session kill. Find a genuinely different, safer approach, or ask the human. If the safer approach is legitimately different (e.g. pushing a *new* branch instead of force-pushing one), say so explicitly so the human can approve past any evasion flag. - -## Rules it enforces - -These are blocked or gated here (Railguard defaults + the `validate-bash.sh`/`validate-write.sh` hooks). Don't burn turns hitting them: - -- **History/data loss (ask or block):** `git push --force` (force-push to `main`/`master` is hard-blocked), `git reset --hard`, `git clean -f`, `rm -rf` on `/` `~` `$HOME`, `terraform destroy`, `DROP TABLE`. -- **Escalation/staging (block):** `sudo` — run escalations yourself; `git add -A` / `--all` / `.` — stage paths explicitly so secrets aren't swept in. -- **Network/exfiltration (ask or block):** `curl | sh`, encoded payloads, outbound `curl -X POST`, `wget`, `ssh`/`scp`/`rsync`, `env` dumps. -- **Path fence:** `~/.ssh`, `~/.aws`, `~/.config/gcloud`, `/etc` are denied; allowed roots are `~/.claude`, `/tmp`, `~/github`, `~/Bitbucket`. `~/.gnupg`/`~/.config/gh` stay readable for GPG signing and `gh`. - -## The path fence scans command text - -The Bash fence matches fenced path **strings** in the command, so a command that merely *mentions* a fenced path — in a heredoc, an issue body, install docs — gets blocked even though it never touches that path. - -- Author file content with the `Write`/`Edit` tools, not `cat <`, `echo >`, or `printf >` redirects. Tool-based writes are snapshotted and their content is not scanned by the Bash fence. -- Don't embed fenced path literals in Bash command text. If content must reference them, put it in a file via `Write` and pass it by path (`--body-file`, `--file`, stdin redirect). -- If a Bash command is fence-blocked because its *content* quoted a fenced path, switching to the `Write` tool is the **intended remediation** — do it without hesitation. It is not evasion. -- If a command is blocked because it actually *accesses* a fenced path, do not retry it in any form — find a different approach or ask the user. +[Railguard](https://github.com/ulises-c/railguard) intercepts every tool call and decides **allow**, **ask**, or **block**. Session mechanics (rollback, policy layers, self-protection) are in the auto-managed `# Railguard — Active Guardrails` block of `CLAUDE.md`. +- **Blocked → never re-issue the command with cosmetic changes** (new flags, base64, `eval`, a wrapper) — that trips evasion detection and escalates toward a session kill. If the command truly accessed a protected path or operation, do not retry it in any form. Take a genuinely different approach and say how it differs. **Ask** → wait for the human; don't route around it. +- Destructive history changes, bulk staging, escalation, network/exfiltration, and protected paths are gated by design. Force-pushes to `main`/`master` are hard-blocked, not approval-gated. The exact policy lives in `railguard.yaml` and the validation hooks; `~/.gnupg` and `~/.config/gh` intentionally stay readable for signing and `gh`. +- The Bash fence scans command **text**: merely *mentioning* a fenced path or a `/slash-command` token can block a command that never touches it. Author content with `Write`/`Edit` and pass it by path (`--body-file`), not heredocs/redirects — that switch is intended remediation, not evasion. +- Unexpected block/ask that looks like a false positive, or an improvement idea → read `~/.claude/docs/RAILGUARD.md`, or `agentic-ai/Claude/docs/RAILGUARD.md` in the Computer-Setup repo before reinstalling, and follow its reporting protocol. diff --git a/agentic-ai/Claude/validate.sh b/agentic-ai/Claude/validate.sh index bb1585d..1714162 100755 --- a/agentic-ai/Claude/validate.sh +++ b/agentic-ai/Claude/validate.sh @@ -26,11 +26,23 @@ check_symlink() { fi } -# ── Symlinks ────────────────────────────────────────────────────────────────── -section "Symlinks" -check_symlink "$CLAUDE_DIR/settings.json" "$REPO_DIR/settings.json" +check_regular_file() { + local file="$1" + if [[ -f "$file" && ! -L "$file" ]]; then + pass "$file is a regular file" + elif [[ -L "$file" ]]; then + fail "$file is a symlink (expected a copied file)" + else + fail "$file is missing or not a regular file" + fi +} + +# ── Installed files ─────────────────────────────────────────────────────────── +section "Installed files" +check_regular_file "$CLAUDE_DIR/settings.json" check_symlink "$CLAUDE_DIR/CLAUDE.md" "$REPO_DIR/CLAUDE.md" check_symlink "$CLAUDE_DIR/rules" "$REPO_DIR/rules" +check_symlink "$CLAUDE_DIR/docs" "$REPO_DIR/docs" check_symlink "$HOME/.railguard.yaml" "$REPO_DIR/railguard.yaml" # ── Hooks ───────────────────────────────────────────────────────────────────── diff --git a/code-review-1-agents.md b/code-review-1-agents.md new file mode 100644 index 0000000..28da668 --- /dev/null +++ b/code-review-1-agents.md @@ -0,0 +1,154 @@ +# Code Review 1 — AGENTS.md & agentic-ai/ (PR #61, branch `docs/agents-md-master`) + +Max-effort review: 10 finder angles → 6-verifier pass (1-vote, 3-state) → gap sweep. +Scope: `git diff origin/main...HEAD -- AGENTS.md CLAUDE.md agentic-ai/`. +Review artifact for PR #61. + +## Findings (ranked, most severe first) + +### 1. `install.sh` aborts mid-install if `~/.claude/docs` exists as a real directory — CONFIRMED +`agentic-ai/Claude/install.sh:43` — `rm -f "$CLAUDE_DIR/docs"` cannot remove a real +directory; under `set -euo pipefail` it exits 1 ("Is a directory") and kills the +script after `rules/` is linked but before railguard.yaml, hooks, and the railguard +binary install — a half-provisioned `~/.claude`. Verified by test. **`ln -sfn` alone is +NOT a fix**: on GNU coreutils it errors on a real dir; on uutils it exits 0 but nests +the link *inside* the directory. Needs an explicit guard, e.g. +`[[ -d "$CLAUDE_DIR/docs" && ! -L "$CLAUDE_DIR/docs" ]] && rm -rf …` or a fail-with-message. + +### 2. `validate.sh` fails on every correctly-installed machine (settings.json copy-vs-symlink drift) — CONFIRMED +`agentic-ai/Claude/validate.sh:31` still runs +`check_symlink "$CLAUDE_DIR/settings.json" …`, but `install.sh` deliberately +**copies** settings.json (and removes any old symlink). Every fresh install guarantees +a `[FAIL]`. Pre-existing on main, but this PR extends the same hand-mirrored list +(the new docs line), and it's the proof that the install/validate manifest drifts. +Fix: change line 31 to a file-exists/`cmp` check (drift is already `settings-drift.sh`'s +job), and consider a shared (src,dest,kind) manifest both scripts read. + +### 3. Always-loaded rules dropped the `~/.gnupg`/`~/.config/gh` readable exception — CONFIRMED +`agentic-ai/Claude/rules/common/railguard.md:6` no longer says gnupg/gh stay readable, +while the auto-managed block still says "Do NOT attempt to … Access `~/.gnupg`" — +**which contradicts `railguard.yaml`** (comment: gnupg/gh "intentionally absent" from +denied paths). A GPG-signed commit never triggers a block, so nothing ever routes the +agent to `docs/RAILGUARD.md` where the exception now lives: the agent self-censors on +signing/`gh` config based on an unqualified prohibition. Also dropped: `~/.config/gcloud` +(which IS denied in yaml). Fix: restore the one-line exception to the slim rule; fix the +managed-block list upstream in the fork's template. + +### 4. `~/.claude/docs/RAILGUARD.md` pointer dangles on every other machine until `install.sh` is re-run — CONFIRMED +`agentic-ai/Claude/rules/common/railguard.md:8` — `~/.claude/rules` is a live dir +symlink into the repo, so `git pull` activates the new rule text instantly, but only +`install.sh` creates the `~/.claude/docs` symlink. On the Macs/CachyOS box, the +documented reporting protocol is unreachable between pull and reinstall. Fix options: +have the rule fall back ("or `agentic-ai/Claude/docs/RAILGUARD.md` in the Computer-Setup +repo"), or note the install.sh re-run in the PR/release step. + +### 5. AGENTS.md: "a bare TTY run … prompts interactively" dropped the server-profile exception — CONFIRMED (sweep) +`AGENTS.md:19` states the interactive prompt unconditionally. `lib/core.sh` +(`core_maybe_prompt_selection`) returns early when `SERVER_PROFILE=true` *before* the +TTY check — a bare `--profile server` TTY run never prompts. The old CLAUDE.md stated +this exception explicitly ("skipped on the server profile and in non-interactive/CI +runs"); the AGENTS.md condensation lost it. Fix: restore the clause. + +### 6. AGENTS.md: "Every field can be a scalar or a per-platform object" is false — CONFIRMED +`AGENTS.md:33` — per `docs/PACKAGES.md` only `priority`/`optional`/`environment`/ +`install_command` take the per-platform-object form; `tags` must be a plain non-empty +array (validator: "tags must be an array"). An agent taking the sentence literally +writes `"tags": {"macos": […]}` and fails pre-commit/CI. Fix: "The four tier/gating +fields (`priority`, `optional`, `environment`, `install_command`) can be a scalar or a +per-platform object." + +### 7. RAILGUARD.md reporting protocol writes through the symlink into an unmanaged working tree — CONFIRMED (sweep) +`agentic-ai/Claude/docs/RAILGUARD.md:81` — step 2 ("Log it below") has an agent in ANY +repo append to `~/.claude/docs/RAILGUARD.md`, i.e. write into the Computer-Setup +checkout on whatever branch is checked out, with no commit/sync step in the protocol. +Entries sit uncommitted, get discarded by checkout/stash, and diverge per machine. +Fix: add step 2b ("commit the log entry in Computer-Setup — or file the upstream issue +first and only link it"). + +### 8. Managed block's `docs/per-project-allowlist.md` pointer dangles (and now looks resolvable) — CONFIRMED +`agentic-ai/Claude/CLAUDE.md:57` — the file lives in the railguard repo's `docs/`, not +in `agentic-ai/Claude/docs/` (only RAILGUARD.md there); the new `~/.claude/docs` +symlink makes the bad path look intentional. Fix belongs in the fork's block template +(repo-qualify the link) or vendor the doc into `agentic-ai/Claude/docs/`. + +### 9. RAILGUARD.md: "install.sh symlinks everything into ~/.claude" is materially wrong — CONFIRMED +`agentic-ai/Claude/docs/RAILGUARD.md:16` — settings.json is **copied** (by design, with +a re-copy after `railguard install`), and railguard.yaml links to `~/.railguard.yaml`, +outside `~/.claude`. An agent trusting this edits the live settings expecting repo +propagation. Fix: "symlinks the config (settings.json is copied — see install.sh)". + +### 10. `agentic-ai/Claude/README.md` is stale on four counts — CONFIRMED +The "This will:" activation list omits the new `docs/` symlink AND the +`railguard.yaml → ~/.railguard.yaml` link; line 14 claims settings.json is symlinked +(it's copied); the cargo line says "skipped if already installed" (it now hard-fails +without cargo and installs from the fork); the rules-tree description still credits +`rules/common/railguard.md` with the full enforced-rules/rollback detail that moved to +`docs/RAILGUARD.md`, which the README never mentions. + +### 11. "Writing files"/"Policy layers" sections may be wiped by the next `railguard install` — PLAUSIBLE +`agentic-ai/Claude/CLAUDE.md:18,52` — these sit inside the `railguard:start/end` +auto-managed block. Evidence conflicts: one verifier matched them to the fork's +`defaults/CLAUDE.md` template; the sweep grepped the *installed binary* and the +generator source and found neither section emitted. If the binary regenerates the block +without them, both sections silently vanish from the repo file (via the symlink) on the +next install.sh run. Confirm by diffing the fork's current template against the block; +if they're hand-edits, move them outside the block or upstream them into the template. + +### 12. Slim rule mislabels force-push as plain "(ask)" — PLAUSIBLE +`agentic-ai/Claude/rules/common/railguard.md:6` — main/master force-push is +hard-blocked; the rule now affirmatively says "(ask)". An agent promised an ask that +gets a block may retry/reword (evasion escalation), though the always-loaded +never-retry rule mitigates. Fix: "(ask; hard-block on main/master)" — 5 words. + +### 13. Slim rule dropped "blocked for real ACCESS → never retry in any form" — PLAUSIBLE +`agentic-ai/Claude/rules/common/railguard.md:7` — only the mention-vs-access remediation +survives always-loaded. An agent misdiagnosing a real fenced-path access block as the +known text-scan FP "remediates" via Read/Write against the fenced path. Partially +backstopped (Railguard intercepts those tools too). Fix: append "if the command truly +accesses the path, don't retry in any form". + +### 14. Cross-agent verify/security trigger conditions stranded in the Claude-only overlay — CONFIRMED (altitude) +Root `CLAUDE.md:9-14` — *when* a four-platform dry-run or a security pass is required +(changes to setup.sh/lib/platforms/packages.json; .env handling, install_command shell +execution, path/network code) binds any agent, but lives only in the Claude overlay as +/skill triggers; AGENTS.md (whose own header says "keep cross-agent guidance here") has +no equivalent obligation. opencode/Codex commit those changes unchecked. Fix: state the +conditions in AGENTS.md Conventions; keep only the skill-name mapping in CLAUDE.md. + +### 15. Gated-command inventory hand-maintained in three places, drift already realized — CONFIRMED (reuse) +Slim rule bullet 2, managed block "Do NOT attempt to", and RAILGUARD.md "Rules it +enforces" all restate what `railguard.yaml` + the validate hooks own. Realized drift: +the gnupg contradiction (#3), the gcloud omission, and none of the three mention the +yaml's `~/Github` allowed root. Fix: keep ONE prose copy (the on-demand doc), reduce +the always-loaded copy to categories + "policy: railguard.yaml". + +## Below the cut (noted, not counted) + +- **AGENTS.md Coding conventions ≈ global rules files** (~250 tokens duplicated per + Claude prompt in this repo; no canonical owner — will drift like #15). Deliberate + cross-agent tradeoff; consider a generation step or a "mirrors global rules" marker. +- **FP-log protocol has no pruning step** — when railguard#17/#18 are fixed, the log + entries and promoted habit lines persist forever (permanent workaround tax). +- **The Pi hostname in AGENTS.md** contradicts its own Privacy rule — but it's + committed in 6 tracked files on main already; either carve out the rule or scrub + repo-wide (decide once, not per-PR). +- **Tool-list mismatch** (doc says Bash/Read/Write/Edit/Memory; managed block says 4) — + the doc side is *correct* (fork hooks all tools via empty matcher + has a memory + subsystem); fix the upstream template. +- Minor: linux-pi described twice in AGENTS.md (~35 tokens); root CLAUDE.md stub + restates AGENTS.md's self-description (~20 tokens); RAILGUARD.md states the + Write-workaround ~5-7×; `check_symlink` compares uncanonicalized `readlink` (false + fails only under mismatched path spellings). + +## Refuted along the way + +- "`ulises-c` username violates the privacy rule" — it's the public GitHub org handle, + present in ~20 tracked URLs incl. install.sh's cargo source; the rule targets + machine/personal identifiers. +- "install.sh comment narrates WHAT" — matches the file's established header-comment + style (5 identical-pattern siblings). +- "RAILGUARD.md's `rules/common/railguard.md` reference doesn't resolve relatively" — + prose filename, not a link; the doc supplies `~/.claude` context and the rule is + always in-context anyway. +- "Slim rule dropped rm -rf/terraform/DROP TABLE/env-dump warnings" — acceptable + reactive placement; the dangerous post-block encode case is still always-loaded. diff --git a/code-review-2-repo.md b/code-review-2-repo.md new file mode 100644 index 0000000..194ee29 --- /dev/null +++ b/code-review-2-repo.md @@ -0,0 +1,204 @@ +# Code Review 2 — Repo-wide: PRs #58, #59, #60, #61 (residual) + +Max-effort review: 10 finder angles (9 agent runs; 5 re-run after a session-limit interruption) +→ 4 subsystem verifiers → gap sweep. Companion to `code-review-1-agents.md` +(which covered AGENTS.md/CLAUDE.md/agentic-ai on PR #61). +Review artifact for PR #61. + +## Top findings (ranked) + +### 1. Backup failure-notifier dies at the same guard that killed the main run — CONFIRMED +`linux-pi/backup/backup.sh:28` — `.env` is sourced (l.12-17), the `:?` guards run +(l.28-29), and the `notify-failure` branch is only dispatched at l.74. A missing/bad +`.env` kills the 03:45 run before the EXIT trap is even installed (l.92), then +`pi-backup-failure.service` re-runs the script and dies at the identical guard — +**no ntfy, no Kuma push, ever**. Backups stop silently. Fix: dispatch `notify-failure` +(or install the trap) before the guards, and let the guards themselves notify. + +### 2. Second-repo copy failure aborts the whole run — README promises the opposite — CONFIRMED +`backup.sh:153-165` — only `restic cat config` sits in a set-e-exempt position, and +its failure is indistinguishable from "repo not initialized": an unreachable second +target falls into the `restic init` branch, which fails as a bare statement → whole +script aborts through the failure trap → urgent FAILED alert although the primary +backup, prune, and check all succeeded. README (l.32-33, 183-184) claims the copy "is +silently skipped" on SFTP failure. Fix: probe reachability separately and guard the +whole second-repo block with an explicit skip + notice. + +### 3. Nightly backup can hang forever and silently stop all future runs — CONFIRMED +`pi-backup.service` — `Type=oneshot` (default `TimeoutStartSec=infinity`), no +`RuntimeMaxSec`, and restic-over-SFTP with no ServerAlive/connect timeout anywhere. A +black-holed TCP connection leaves the unit "activating" indefinitely: the timer won't +re-fire while active and OnFailure never triggers (nothing failed). Fix: +`RuntimeMaxSec=` (e.g. 2h) on the unit and/or `-o sftp.args` keepalives. + +### 4. UPS tuning flow silently doesn't apply — driver never restarted — CONFIRMED +`linux-server/ups/setup.sh:77-83` restarts only `nut-server`/`nut-monitor`. The +`override.battery.charge.low/runtime.low` lines README tells you to uncomment "and +re-run setup.sh" are **driver** (usbhid-ups) settings; the running driver keeps the +old ~10% LOWBATT until reboot. The safety margin you configured doesn't exist during +the next outage. Fix: restart `nut-driver@cyberpower` (or `nut-driver.target`) when +ups.conf changed. + +### 5. `nut` installs inert (MODE=none) and is structurally unverifiable — CONFIRMED +`packages.json:413` installs nut via plain `apt` on server — not a `custom` manager, +so the engine never runs nor even *reminds* about `linux-server/ups/setup.sh` (the +pointer lives only in the free-text description; reminders print only for `custom`). +Debian's default is `MODE=none`: a fresh provision has no UPS monitoring or shutdown. +And `lib/verify.sh:282-284` hard-rejects `--platform server`, so the one machine that +needs nut can never be health-checked. Fix: make nut a `custom` entry (or add a +server-profile custom step), and revisit verify.sh's server rejection now that +server-only packages exist. + +### 6. Pi AdGuard replication can't work as documented — ORIGIN_URL hits NPM, not AdGuard — CONFIRMED (2 independent finders) +`linux-pi/adguardhome-sync/.env.example:6` — `ORIGIN_URL=http://` with the +comment "always reachable, no Tailscale dependency". But the primary AdGuard publishes +only 53/tcp+udp on the LAN IP; its UI/API lives in the adguard-ts netns behind +tailscale serve, and host :80 belongs to nginx-proxy-manager. Every sync fails; the +backup resolver never mirrors filters — noticed only during a failover. Fix: point +ORIGIN_URL at the tailnet HTTPS name (accepting the dependency) or publish the API +port on the LAN. + +### 7. Pi homepage on host :3000 collides with AdGuard's first-run wizard — CONFIRMED +`linux-pi/homepage/docker-compose.yml:31,42-43` — homepage is host-networked on +:3000; Pi adguardhome is **also** host-netns (unlike the server, where it hides in +the sidecar netns). A wiped/fresh AdGuard conf makes its wizard bind :3000 → bind +failure crash-loop, on the backup-DNS box, during a recovery scenario. The server +variant documents a temporary `3003:3000` mapping; the Pi has nothing. Fix: move Pi +homepage off 3000 or document the wizard-port conflict beside the compose. + +### 8. Backup stages all service secrets world-readable in /var/tmp — PLAUSIBLE→likely CONFIRMED (sweep) +`backup.sh:117` — `.env` files (TS_AUTHKEY, AdGuard creds, RESTIC_PASSWORD, tokens) +are `cp -a`'d into `/var/tmp/pi-backup-staging` (default-umask 755 dirs), no +`PrivateTmp=` on the unit, cleanup only via the EXIT trap. SIGKILL/OOM/power-loss +leaves the plaintext bundle persisting across reboots. Fix: staging dir under +`/root` with 700, `PrivateTmp=yes`, or back up the paths directly without staging. + +### 9. Privacy leaks: real username, home path, and server hostname committed — CONFIRMED +`pi-backup.service:8` + `pi-backup-failure.service:8` (a personal absolute +`ExecStart`), `linux-pi/homepage/config/services.yaml:45` (the real server +hostname), `linux-pi/backup/LEARNINGS.md:50` (the real username), and +`linux-pi/README.md:118` (the real router address). All contradict AGENTS.md's +own privacy section in a public repo. +The units also break on any other checkout path — ship them as templates rendered at +install (the ups setup.sh sed pattern already exists in-repo). + +### 10. Backup snapshots capture AdGuard's live databases mid-write — sweep +`backup.sh:101` — `adguard/work` (querylog/stats DBs, continuously written) is in the +source set with no quiesce/exclude; the server's own script stops portainer for +exactly this reason. Restores can yield a corrupt DB; the churn also bloats every +SFTP transfer. Fix: exclude `work/` (conf/ is the restorable state) or stop/start the +container around the snapshot. + +### 11. The Glances disk-rename chain fails silently three different ways — CONFIRMED + sweep +`linux-server/glances/rename_disks.py` + `entrypoint.sh`: (a) the monkey-patch wraps +everything in `except Exception: pass` with zero logging, on an unpinned +`:latest-full` image whose internal plugin API has renamed before; (b) the `sed` +parent-disk derivation breaks on `nvme…`/`mmcblk…` names AND the `^[a-z]+$` filter +would reject them anyway — silent no-op for any non-sdX device; (c) mapping is built +once at container start, but the DAS mounts `nofail` — boot with the enclosure +absent/late and glances runs unpatched until a manual restart. Widgets just go blank; +nothing signals why. Fix: log on patch failure, pin the image, fix the derivation +(`lsblk -no pkname`), and re-resolve labels inside `patched()` (also kills the +restart-after-reshuffle toil). + +### 12. Pi dashboard rejects all LAN access — ALLOWED_HOSTS dropped every local identity — CONFIRMED +`linux-pi/homepage/docker-compose.yml:41` — only `localhost,`; the +server variant allow-lists hostname/.local/LAN-IP/tailscale variants. Host-networked +on :3000, so LAN requests reach it and get "Host validation failed" — the dashboard +is tailnet-only, failing exactly when the tailnet is down (its raison d'être). + +### 13. CUPS sidecar connects from the bridge gateway, but cupsd listens on localhost only — CONFIRMED (2 finders) +`linux-pi/cups/ts-serve.json:5` proxies to `host.docker.internal:631`; Debian default +is `Listen localhost:631` + local-only `` policy. Connection refused/403 +before the documented ServerAlias fix is ever reached. Fix: document the required +`Listen`/`Allow` change next to the ServerAlias note. + +### 14. Pi README tells you to run `setup.sh --profile server` on Debian — it will abort — CONFIRMED +`linux-pi/README.md:96` — the server platform snap-installs micro/nvtop and +apt_bootstrap adds an Ubuntu PPA; Raspberry Pi OS has no snapd and no PPAs, and under +`set -euo pipefail` the run dies mid-provision. `platforms/server.sh:7` itself calls +the Pi "a future target". Fix: reword to the manual prerequisite list until the Pi is +a real platform. + +### 15. UPS notifications: no curl timeout + NOCOMM re-fires every 5 min — CONFIRMED +`ups-notify.sh:24-26` — no `--max-time`; during an outage (router down) each event's +curl can block ~2 min, and alerts for the shutdown sequence are delayed/lost. +Compounding: `upsmon.conf.template` `NOCOMMWARNTIME 300` + `NOTIFYFLAG NOCOMM +SYSLOG+EXEC` = an urgent push every 5 minutes for a loose USB cable (~576/weekend) — +alert-fatigue that gets the topic muted before a real outage. Fix: `--max-time 10`, +and NOCOMM to SYSLOG-only (COMMBAD/COMMOK already signal the state change). + +## Also confirmed (below the top-15 cut) + +- **Boot race:** `pi-backup.service` has `After=network-online.target` but no + `Wants=` — with `Persistent=true`, every power-restore boot fires a backup before + the network is up → spurious FAILED alert. One-line fix. +- **Double-alert + status clobber:** every failure sends 2× ntfy + 2× Kuma (EXIT trap + and OnFailure unit), and the second `write_status` zeroes the duration/snapshot the + trap recorded. +- **peanut-ts missing the `dns:` bootstrap guard** every other sidecar got (#59 + merged before #58's fix pattern; same deadlock class on cold boot). +- **Timer contention (sweep):** Pi's 03:45 job overlaps the server's 03:30 + backup+prune+check on the same DAS spindles on exactly the slow nights; nothing + serializes them. +- **9.9.9.10 (sweep):** the sidecar bootstrap DNS is Quad9's *unsecured* tier (no + DNSSEC/filtering), now copy-pasted into 5 files — deliberate or a typo for 9.9.9.9? +- **Docs migration losses (#60):** the Ubuntu-desktop live-run record (`eb0fe49`, + verify green) exists nowhere at HEAD; CHANGELOG says "30 findings" (actual: 36); + the CachyOS zsh-notify caveat and the 14TB-second-copy detail were dropped; the + open "re-run suites, check for nulls" instruction was recast in past tense with no + record it ever ran. +- **Stale pointers:** 11 shell headers still cite root `UNIFICATION.md`/`TODO.md` + (the #60 link fix only covered markdown); `docs/UNIFICATION.md` sends readers to + CLAUDE.md "for the current architecture" (now a 14-line shim — should be + AGENTS.md); `docs/HANDOFF.md` says "Portainer is next per the TODO.md table" + (rollout finished; table moved); `docs/TODO.md`'s Pi section says the config goes + "under linux-server/", leaves shipped items unchecked, and sizes the sidecar debt + at ~13 (actual: 22); `linux-server/post-install.md:177` has a `../../macOS/...` + link resolving above the repo root. +- **Pi homepage `docker.yaml` gitignored but required** (4 independent finders): + `services.yaml` references `server: my-docker`, homepage never auto-generates it, + and no doc says to create it — the backups card's container status is dead on a + fresh deploy. +- **`depends_on: homepage-pi-ts`** needlessly takes the LAN dashboard down when the + inbound-only sidecar can't start (same pattern on the server). +- **Pi adguard widget round-trips the tailnet** to reach a service on its own host + (`localhost:80` works; the server uses the localhost pattern everywhere). +- **No lint gate covers Python:** `rename_disks.py` (and pre-existing `proxy.py`) are + the only executables no pre-commit hook or CI job even syntax-checks. +- **Style:** `glances/entrypoint.sh` is `#!/bin/sh`, no `set -e`, `[ ]` — against the + repo's stated Bash rules, with no busybox-image justification comment; + `ups-notify.sh` sets only `set -u`. +- **LEARNINGS.md is factually wrong (sweep):** restic's sftp backend shells out to + system `ssh` (it does read ssh_config); the "embedded Go client, reads no config" + claim will misdirect future debugging and hides host-key rotation risk. +- **UPS setup.sh:** `--dry-run` dies without `.env` (breaks the repo's dry-run + convention; the NUT-missing check shows the right pattern one screen up); + `deploy()`'s `cmp -s` short-circuit never reconciles ownership/mode drift on the + password-bearing files; NTFY_* values are rendered unvalidated into a root-written, + nut-sourced env file (robustness, not a privilege boundary — the .env author is + already trusted). +- **Reuse/altitude:** the Tailscale sidecar block now exists in 22-23 hand-copies + (TODO's own DRY plan says "~13", already stale) and divergence is realized + (peanut-ts dns); `linux-pi/backup/backup.sh` is a ~174-line fork of the server's + 233-line script (shared notify/kuma/status machinery will drift); `linux-pi/` is a + growing parallel universe the tracked fold-into-setup.sh must unwind (~20 near-twin + files and counting). +- **Efficiency batch:** nightly `restic check` (no `--with-cache`) + nightly + `--prune` on both repos where weekly would do; 3 restic invocations where + `backup --json` gives snapshot-id and size; adguardhome-sync `CRON */10` (144 + full syncs/day for weekly-change config); 8 glances widgets polling at the 1s + default with no `refreshInterval`; zero `logging:` limits on any Pi container + + live querylog on SD (flash wear on the resilience box). + +## Refuted + +- **adguardhome-sync env-var names** (`FEATURES_DHCP_SERVER_CONFIG`, `RUN_ON_START`): + current upstream README documents exactly these underscored forms for `:latest`; + the concatenated variants belong to older releases. The `.env.example` is correct. +- PR #60's TODO trim dropped **no open items** (every `[ ]` survived); all 36 + benchmark findings did land in `docs/macos-benchmark-review.md`; PR #59's + packages.json change is purely additive; the NUT name/user/password chain and + peanut's port chain are internally consistent end-to-end; glances label names match + the homepage widgets letter-perfect; all `.env.example` vars are consumed; the four + platform shims and post-install's new UPS/DAS sections check out. diff --git a/CHANGELOG.md b/docs/CHANGELOG.md similarity index 90% rename from CHANGELOG.md rename to docs/CHANGELOG.md index 1f53d92..7e00ec1 100644 --- a/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -4,12 +4,12 @@ Notable changes to this personal machine-provisioning repo. There are no tagged releases; entries are grouped by the date work merged to `main`, newest first. Format loosely follows [Keep a Changelog](https://keepachangelog.com). Remaining work lives in [TODO.md](TODO.md); the design rationale for the unified layout is -in [UNIFICATION.md](docs/UNIFICATION.md). +in [UNIFICATION.md](UNIFICATION.md). ## 2026-07-11 — UPS monitoring, server ([#59](https://github.com/ulises-c/Computer-Setup/pull/59)) ### Added -- NUT support for the CyberPower PR1500LCDRT2U under `linux-server/ups/`. +- NUT support for the CyberPower CST135UC2 under `linux-server/ups/`. - PeaNUT UPS dashboard, fronted by a Tailscale HTTPS sidecar (`https://peanut..ts.net/`) with a homepage widget. @@ -22,7 +22,8 @@ in [UNIFICATION.md](docs/UNIFICATION.md). `packages.json`; app-store-style `.app` verify probe for GUI-only custom entries. ### Fixed -- Resolved 30 findings from a max-effort review of the suite: +- Resolved 36 findings from a max-effort review of the suite (full record: + [macos-benchmark-review.md](macos-benchmark-review.md)): - **P0** — broken measurement paths and silent data corruption: LibreSSL `openssl speed -seconds` unsupported, `llama-bench --hf-repo` arg failure, null GPU/Cinebench/Blender parses, throttle-methodology and Apple-Silicon @@ -58,7 +59,7 @@ in [UNIFICATION.md](docs/UNIFICATION.md). certs/configs, every `.env`) to a dedicated 1TB drive, encrypted/deduplicated/ pruned, with ntfy alerts, a homepage status card, and a restore runbook. systemd timer at 03:30 (Persistent) with an OnFailure alert; `ts-state/` - deliberately excluded. + deliberately excluded. The optional second copy targets the separate 14TB drive. ## 2026-06-17 — Per-service HTTPS over Tailscale, server @@ -77,7 +78,7 @@ in [UNIFICATION.md](docs/UNIFICATION.md). Collapsed the three diverged setup stacks (`macOS/`, `linux-desktop/`, `linux-server/`) into one root `setup.sh` + one `packages.json`. Full design and -phased breakdown in [UNIFICATION.md](docs/UNIFICATION.md). +phased breakdown in [UNIFICATION.md](UNIFICATION.md). ### Added - Root `setup.sh` / `verify.sh` dispatchers, a shared `lib/core.sh` engine and @@ -116,6 +117,9 @@ phased breakdown in [UNIFICATION.md](docs/UNIFICATION.md). generalized into `lib/core.sh`. - The powerlevel10k theme is an antidote plugin on all desktops (the previous yay-only entry meant Ubuntu/macOS silently fell back to `vcs_info`). +- Ubuntu desktop completed a live unified-setup run at `eb0fe49`: the shared + zsh/p10k/Ghostty configuration deployed successfully and `verify.sh --work` + was green except for intentionally manual packages. ## 2026-05-28 — CachyOS / Arch desktop support ([#18](https://github.com/ulises-c/Computer-Setup/pull/18)) @@ -135,3 +139,6 @@ phased breakdown in [UNIFICATION.md](docs/UNIFICATION.md). ### Fixed - pyenv init no longer aborts setup under `set -e`; hardened `yay` batch installs to skip already-satisfied build deps. +- CachyOS verification covered antidote, zoxide, eza icons, and zsh-notify. + zsh-notify's "unsupported environment" over SSH is expected without a graphical + session; local desktop notifications work. diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md new file mode 100644 index 0000000..6cb9c15 --- /dev/null +++ b/docs/HANDOFF.md @@ -0,0 +1,9 @@ +# Session Handoff + +Scratchpad for in-progress work that spans more than one session — live +infrastructure state, decisions made outside of code, and what's next. +Branch-specific; delete or trim entries once they're fully landed and the +branch merges. + +No active handoff entries. Completed rollout history lives in +[CHANGELOG.md](CHANGELOG.md), and remaining work lives in [TODO.md](TODO.md). diff --git a/docs/PACKAGES.md b/docs/PACKAGES.md new file mode 100644 index 0000000..35858d6 --- /dev/null +++ b/docs/PACKAGES.md @@ -0,0 +1,67 @@ +# packages.json — schema & selection reference + +`packages.json` is the single source of truth for all package data across every +platform. This is the deep reference; `AGENTS.md` carries the one-paragraph summary. +Read this before editing `packages.json` or the selection logic in `lib/core.sh`. +The design history is in [UNIFICATION.md](UNIFICATION.md). + +## Per-entry fields + +- **`name`** — canonical name and default install token. +- **`package_manager`** — object keyed by platform (`{macos, ubuntu, arch, server}`); + omit platforms the package doesn't target. `custom` selects the install-command path. +- **`_name`** — optional per-platform install-token override; the engine + reads `._name // .name` (e.g. `huggingface-hub` → arch + `python-huggingface-hub`). +- **`priority`** — `high | medium | low | none`. +- **`optional`** — boolean; `low`/`none` + optional installs only with `--optional`. +- **`environment`** — gates on `--work` / `--personal`; absent ⇒ always installs. +- **`install_command`** — `custom` managers only; string or per-platform object. +- **`handled_by_setup`** — `custom` entries: `true` auto-runs the command, otherwise + it's printed as a manual-install reminder. +- **`tags`** — required non-empty array of categories from the controlled vocabulary + in `scripts/validate-packages.sh`. Metadata only (grouping/docs/`--tags` filter); + the install engine ignores them for resolution. +- **`description`**. + +## Scalar or per-platform object + +`priority`, `optional`, `environment`, and `install_command` each accept a **scalar** +(applies to every platform) **or a per-platform object** keyed by platform, e.g.: + +```jsonc +"priority": { "macos": "medium", "ubuntu": "none" } +``` + +The engine resolves them via the `prfor` / `optfor` / `envfor` / `icfor` jq defs in +`lib/core.sh`. This is what lets one entry serve platforms that differ in +tier/optionality/gating instead of splitting into duplicate entries. + +**`environment` caveat:** its scalar form is itself an *array* (`["work"]`), so the +per-platform form is detected as an *object* (`{ "ubuntu": ["work"] }`) — array means +legacy/all-platforms, object means per-platform. Keep the per-platform value an +object-of-arrays. + +## Category selection (`--base` / `--tags`) + +- `--base` installs only the high-priority base set (no category packages). +- `--tags development,terminal` installs base + those tag categories. Enabled + work/personal apps install regardless of category. +- Bare run on a TTY with no selection flag → `core_maybe_prompt_selection` prompts + interactively; skipped on the server profile and in non-interactive/CI runs. +- The filter is implemented by `tagok()` in `CORE_JQ_DEFS`, which reads + `TAG_FILTER_ACTIVE` and `SELECTED_TAGS` from the environment — **inactive by + default**, so flag-driven and CI runs are unchanged. +- The dedicated custom-install steps (tailscale, claude-code, docker) gate on + `pkg_selected` so they honor the selection too. The server profile keeps the filter + inactive, so they install as before there. + +## Validation & reminder semantics + +- `scripts/validate-packages.sh` (pre-commit + CI) enforces the schema: platform + vocabulary, controlled tag set, `handled_by_setup` is a real boolean, custom + entries carry an `install_command`, and the **"no silent drop"** rule — every + platform a package targets must resolve a valid priority tier and a boolean + optional. +- App-store packages and `priority: "none"` entries are **reminders only** — never + auto-installed. diff --git a/docs/PR61_MAIN_SERVER_REPLY_CHAIN.md b/docs/PR61_MAIN_SERVER_REPLY_CHAIN.md new file mode 100644 index 0000000..3cbab3d --- /dev/null +++ b/docs/PR61_MAIN_SERVER_REPLY_CHAIN.md @@ -0,0 +1,232 @@ +# PR #61 — Main server LLM reply chain + +This document is the canonical, redacted conversation between the workstation +agent and the LLM agent operating the main Ubuntu server. The human operator +copies each outbound message to the host agent and returns its reply to the +workstation agent, which updates this document and prepares the next round. + +## Communication and privacy rules + +- This repository is public. Never write real hostnames, IP addresses, tailnet + names, usernames, email addresses, credentials, tokens, repository passwords, + snapshot IDs, or private URLs here. +- Never paste `.env` files, AdGuard configuration dumps, password hashes, or + unredacted command output into the LLM conversation. +- Credentials move only through the human operator's password manager or a + private, no-echo entry directly on the applicable host. They never move + through this document or either LLM transcript. +- The host agent may change the explicitly listed gitignored `.env` files and + host configuration. It must not edit tracked files, commit, push, or modify + the pull request. +- Before switching branches, the host agent must run `git status --short` and + stop rather than stash, clean, reset, or overwrite local work. + +## Cross-host sequence + +1. Main server rotates or stages replacement credentials for the primary + AdGuard and updates its local consumers. +2. The human operator transfers the replacement primary credential privately to + the Pi. The Pi updates the sync origin and validates replication. +3. If the old primary credential remained active for a staged migration, the + main server invalidates it after the Pi reports a successful sync. + +The retiring credential must not remain active after the coordinated migration. + +## Round 0 — Main server report received + +The server agent verified commit `346917b` with a clean worktree. UPS services, +backup and runner-status timers, Forgejo runner status, Glances 4.5.4, the local +Glances API, the HTTPS sidecar, and stable disk aliases passed. The only server +verification failure was an unprivileged read of the intentionally protected +NUT configuration file. Actual NUT services and `upsc` were healthy. UPS ntfy +delivery was not confirmed. + +## Round 1 — Workstation to main server agent + +After the workstation pushes, replace `` below with the exact +reported branch HEAD before copying the message. The host agent must stop if the +placeholder remains or HEAD differs from that exact commit. + +```text +Continue PR #61 live remediation on the main Ubuntu server. + +Expected commit: + +Safety and privacy: +- Do not edit tracked repository files, commit, push, reset, clean, stash, or + modify the PR. +- Start with `git status --short`; stop if local work makes a safe fast-forward + impossible. +- Never print or return credentials, password hashes, `.env` contents, real + hostnames/IPs/tailnet names, private URLs, or other identifiers. +- The human operator must create/store replacement credentials outside the LLM + transcript and enter them privately. Never read a new credential back. + +Tasks: + +1. Safely fetch and fast-forward `docs/agents-md-master`. Require HEAD to equal + `` exactly. Stop if the placeholder was not replaced or the + commit differs; do not run privileged repository scripts from another commit. + +2. Re-run unprivileged server verification: + `bash verify.sh --platform server` + Confirm the former NUT `MODE=standalone` permission false-negative is gone. + Report the PASS/FAIL counts and redacted evidence that the driver, server, + monitor, and `upsc cyberpower@localhost ups.status` checks pass. + +3. Coordinate replacement of the primary AdGuard admin/API + credential using AdGuard's supported credential-change path: + - Inventory local consumers by variable name only; do not print values. + - Before changing authentication, create a root-only local configuration + backup using AdGuard's supported backup/recovery method and identify a + tested host-local recovery command. Do not print or return the backup. + - Have the human generate/store and enter the replacement credential outside + the transcript. + - Update the primary AdGuard credential, then test the replacement directly + against the local login/API before changing consumers or retiring the old + credential. Restore through the prepared recovery method if direct + replacement authentication fails. + - After direct authentication passes, update + `linux-server/homepage/.env` keys `HOMEPAGE_VAR_ADGUARD_USER` and + `HOMEPAGE_VAR_ADGUARD_PASS` locally. + - Recreate/restart only the server Homepage service as needed and confirm its + AdGuard widget authenticates. + - Verify the old credential is rejected unless it must remain briefly + active for a supported staged migration. If it remains active, report state + `STAGED`; do not disclose either credential. + - Tell the human privately that the same replacement primary credential must + be entered on the Pi in `linux-pi/adguardhome-sync/.env` as + `ORIGIN_USERNAME` and `ORIGIN_PASSWORD`. + +4. If safe, send one ordinary UPS test notification. Do not simulate battery, + power-loss, forced-shutdown, or low-battery events. Report only whether the + notification was received. + +Return this redacted structure: +- Commit: +- Worktree: CLEAN or BLOCKED +- Server verify: PASS/FAIL and counts +- NUT functional checks: PASS/FAIL +- Primary credential rotation: COMPLETE/STAGED/BLOCKED +- Retiring primary credential: REJECTED/STILL ACTIVE/UNKNOWN +- Server Homepage AdGuard widget: PASS/FAIL/SKIPPED +- UPS ntfy delivery: PASS/FAIL/SKIPPED +- Pi private handoff ready: YES/NO +- Blockers: redacted description or NONE +- Confirmation: no tracked edits, commits, pushes, or PR changes +``` + +## Round 1 — Main server reply + +The server agent fast-forwarded to the commit below with a clean worktree and +re-ran unprivileged verification. The former NUT `MODE=standalone` permission +false-negative is resolved: the protected-file grep was removed and each NUT +unit is now checked for both enabled and active state. All three NUT units and +`upsc cyberpower@localhost ups.status` pass, and the UPS reports on-line power. +Backup and runner-status timers are enabled and active with executable paths +resolving to the current checkout; the runner-status oneshot ran to success. +Glances (pinned `4.5.4-full`) is healthy on both the local API and the HTTPS +sidecar, with stable disk aliases applied and no restart looping. + +Only the fetch-and-verify portion of Round 1 was relayed to the host agent this +round. The AdGuard primary-credential rotation (task 3) was not attempted, and +UPS notification delivery remains unconfirmed. No battery, power-loss, +forced-shutdown, or low-battery event was simulated. + +```text +- Commit: 3ff8624 +- Worktree: CLEAN +- Server verify: PASS — 38 ok, 0 missing +- NUT functional checks: PASS — driver, server, monitor all enabled and active; upsc ups.status = on-line +- Primary credential rotation: NOT ATTEMPTED (only fetch/verify relayed this round) +- Retiring primary credential: UNKNOWN (rotation not attempted) +- Server Homepage AdGuard widget: SKIPPED (rotation not attempted) +- UPS ntfy delivery: SKIPPED (mechanism verified safe; delivery unconfirmed; no outage simulated) +- Pi private handoff ready: NO (rotation not attempted) +- Blockers: NONE — verification scope complete; credential rotation awaits operator go-ahead +- Confirmation: no tracked edits, commits, pushes, or PR changes during the verification round +``` + +## Round 2 — Complete the deferred primary credential rotation + +After the workstation pushes, replace `` with the exact branch +HEAD. Send this round now; it does not require a prior `STAGED` result. + +```text +Continue PR #61 remediation on the main Ubuntu server. + +Expected commit: + +Safety and privacy: +- Start with a clean-worktree check, safely fast-forward + `docs/agents-md-master`, and require HEAD to equal `` exactly. +- Do not edit tracked files, commit, push, reset, clean, stash, or modify the PR. +- Never print credentials, password hashes, `.env` contents, configuration + backups, real hostnames/IPs/tailnet names, private URLs, or exact + secret-bearing diffs. +- The human creates, stores, and enters replacement credentials privately; never + read them back into the transcript. + +Complete the primary AdGuard credential replacement that was skipped previously: + +1. Inventory local consumers by variable name only. +2. Create a root-only local AdGuard recovery backup and identify a tested + host-local recovery command without printing either. +3. Have the human privately enter the replacement credential using AdGuard's + supported credential-change path. +4. Test the replacement directly against the local login/API. If it fails, + recover immediately and stop. +5. After direct authentication succeeds, privately update + `linux-server/homepage/.env` keys `HOMEPAGE_VAR_ADGUARD_USER` and + `HOMEPAGE_VAR_ADGUARD_PASS`, recreate only Homepage as needed, and verify its + AdGuard widget. +6. Reject the old credential now unless AdGuard requires a staged migration. If + it must remain briefly active, report `STAGED` and leave it only until the Pi + confirms HTTPS-origin sync with the replacement. +7. Tell the human privately to enter the replacement primary credential on the + Pi as `ORIGIN_USERNAME` and `ORIGIN_PASSWORD`; do not transmit it yourself. +8. Optionally send one ordinary UPS test notification without simulating any + power, battery, or shutdown event. + +Return only: +- Commit: +- Worktree: CLEAN/BLOCKED +- Primary credential rotation: COMPLETE/STAGED/BLOCKED +- Replacement direct authentication: PASS/FAIL +- Retiring credential: REJECTED/STILL ACTIVE/UNKNOWN +- Server Homepage AdGuard widget: PASS/FAIL/SKIPPED +- Pi private handoff ready: YES/NO +- UPS ntfy delivery: PASS/FAIL/SKIPPED +- Blockers: redacted description or NONE +- Confirmation: no tracked edits, commits, pushes, or PR changes +``` + +## Round 3 — Finalize a staged primary rotation + +Use this only if Round 2 returns `STAGED`. Wait until the Pi chain reports that +the HTTPS origin authenticates and a sync succeeds with the replacement primary +credential. + +```text +The Pi has confirmed that the replacement primary credential authenticates over +the documented tailnet HTTPS origin and that a sync succeeds. Invalidate the +old primary credential now using AdGuard's supported credential-change +path. Do not print either credential or any private identifier. + +Then verify: +1. the old credential is rejected; +2. the replacement credential still works; +3. the server Homepage AdGuard widget still works; +4. the repository worktree remains clean. + +Reply only with PASS/FAIL for those four checks plus redacted blockers. +``` + +## Completion criteria + +- Server verification passes without reading protected NUT configuration. +- Primary AdGuard replacement credential works for the server Homepage. +- The old primary credential is rejected. +- The Pi confirms a successful HTTPS-origin sync with the replacement + credential. +- Optional UPS notification delivery is recorded without simulating an outage. diff --git a/docs/PR61_PI_REPLY_CHAIN.md b/docs/PR61_PI_REPLY_CHAIN.md new file mode 100644 index 0000000..ebc1fd5 --- /dev/null +++ b/docs/PR61_PI_REPLY_CHAIN.md @@ -0,0 +1,212 @@ +# PR #61 — Pi server LLM reply chain + +This document is the canonical, redacted conversation between the workstation +agent and the LLM agent operating the Raspberry Pi server. The human operator +copies each outbound message to the host agent and returns its reply to the +workstation agent, which updates this document and prepares the next round. + +## Communication and privacy rules + +- This repository is public. Never write real hostnames, IP addresses, tailnet + names, usernames, email addresses, credentials, tokens, repository passwords, + snapshot IDs, private subnets, or private URLs here. +- Never paste `.env` files, AdGuard or CUPS configuration dumps, password hashes, + or unredacted command output into the LLM conversation. +- Credentials move only through the human operator's password manager or a + private, no-echo entry directly on the applicable host. They never move + through this document or either LLM transcript. +- The host agent may change the explicitly listed gitignored `.env` files and, + after human approval of the exact change, `/etc/cups/cupsd.conf`. It must not + edit tracked files, commit, push, or modify the pull request. +- Before switching branches, the host agent must run `git status --short` and + stop rather than stash, clean, reset, or overwrite local work. + +## Cross-host sequence + +1. Wait for the main-server chain to report that a replacement primary AdGuard + credential is ready for private transfer. +2. Update the Pi's origin consumer privately, replace the replica + credential, update all replica consumers, and validate a sync. +3. Return the redacted result so the main server can invalidate an old primary + credential if it used a staged migration. +4. Keep the exact CUPS host-policy candidate and diff in root-only files for the + human to inspect from a separate non-LLM terminal. Apply it only after the + human reports approval of both hashes printed by the prepare step. + +## Round 0 — Pi report received + +The Pi agent verified commit `346917b` with a clean worktree. The backup and +second copy, Homepage direct and HTTPS access, AdGuard synchronization, and CUPS +sidecar connectivity passed. Follow-up items were an ineffective oneshot runtime +limit, missing local Homepage identity variables, an HTTP tailnet-IP sync origin, +over-permissive CUPS host policy, and scheduled credential maintenance. + +## Round 1 — Workstation to Pi server agent + +After the workstation pushes, replace `` below with the exact +reported branch HEAD before copying the message. The host agent must stop if the +placeholder remains or HEAD differs from that exact commit. The human must also +have the replacement primary credential available privately. + +```text +Continue PR #61 live remediation on the Raspberry Pi server. + +Expected commit: + +Safety and privacy: +- Do not edit tracked repository files, commit, push, reset, clean, stash, or + modify the PR. +- Start with `git status --short`; stop if local work makes a safe fast-forward + impossible. +- Never print or return credentials, password hashes, `.env` contents, real + hostnames/IPs/tailnet names, private URLs/subnets, snapshot IDs, or other + identifiers. +- The human operator must create/store and enter credentials outside the LLM + transcript. Never read a new credential back. + +Tasks: + +1. Safely fetch and fast-forward `docs/agents-md-master`. Require HEAD to equal + `` exactly. Stop if the placeholder was not replaced or the + commit differs; do not run privileged repository scripts from another commit. + +2. Reinstall the Pi backup units: + - Run `bash linux-pi/backup/setup.sh --dry-run`. + - If correct, run `sudo bash linux-pi/backup/setup.sh`. + - Confirm `pi-backup.timer` is enabled and active. + - Use `systemctl show pi-backup.service -p Type -p TimeoutStartUSec` to confirm + the oneshot start timeout is two hours. + - Run `bash linux-pi/backup/test-backup-failure.sh` and confirm the failure + notifier replaces missing, running, success, and stale failure records + while preserving current detailed failure data. + - If the same primary/secondary backup preconditions remain healthy, run one + backup and report only PASS/FAIL. Do not return snapshot IDs or repository + locations. + +3. Correct the local Homepage identities in `linux-pi/homepage/.env`: + - Set `HOMEPAGE_VAR_PI_HOSTNAME` to the current short hostname. + - Set `HOMEPAGE_VAR_PI_LAN_IP` to the current primary LAN address. + - Set `HOMEPAGE_VAR_MAIN_HOSTNAME` to the main server's short display name. + - Keep both real values local and never print them. + - Recreate Homepage and verify the short hostname, `.local` hostname, current + LAN address, and configured tailnet hostname are accepted while an unrelated + Host header is rejected. + +4. Complete the scheduled AdGuard credential replacement: + - Have the human privately enter the replacement primary credential into + `linux-pi/adguardhome-sync/.env` as `ORIGIN_USERNAME` and + `ORIGIN_PASSWORD`. + - Change `ORIGIN_URL` locally to the documented + `https://adguard..ts.net` form using the real local tailnet suffix. + - Before changing replica authentication, create a root-only local + configuration backup using AdGuard's supported recovery method and identify + a tested host-local recovery command. Do not print or return the backup. + - Replace the Pi replica admin credential using AdGuard's supported path. The + human creates/stores/enters it privately. Test the replacement directly + against the local login/API before changing consumers; restore through the + prepared recovery method if the direct test fails. + - Update `REPLICA1_USERNAME` and `REPLICA1_PASSWORD` in the sync `.env` and + `HOMEPAGE_VAR_ADGUARD_USER` and `HOMEPAGE_VAR_ADGUARD_PASS` in the Pi + Homepage `.env`. + - Recreate only the affected services, trigger a manual sync, and confirm the + HTTPS origin, local replica, sync, and Pi Homepage AdGuard widget all + authenticate. + - Confirm the old replica credential is rejected. Never disclose old + or new values. + +5. Prepare, but do not yet apply, the reviewed CUPS policy: + - Privately update `linux-pi/cups/.env` from `.env.example`. Set the explicit + LAN/Bonjour and tailnet aliases, the exact pinned sidecar CIDR, and the + canonical private family LAN/WLAN CIDR. Set the file to mode `0600`. Never + print these values. + - Run `bash linux-pi/cups/test-setup.sh`, followed by + `bash linux-pi/cups/setup.sh --dry-run`. The dry-run must report validation + without showing aliases, CIDRs, or a diff. + - Run `sudo bash linux-pi/cups/setup.sh --prepare-review`. Record the printed + source and candidate SHA-256 hashes; hashes are safe to return. + - Do not print or read the protected candidate or diff through an LLM-visible + tool. Tell the human to inspect both root-only artifacts from a separate + trusted terminal as documented in `linux-pi/cups/README.md`. + - The intended root print block allows localhost, the exact sidecar subnet, + and the exact family LAN/WLAN subnet. Every admin block allows only + localhost and the sidecar subnet and requires the exact system-user + authentication policy. The candidate must contain exactly one `Port 631`, + preserve Unix-socket listeners, and contain no broad `Allow` rule or + `ServerAlias *`. + - Stop before changing the live CUPS file. The human returns `APPROVED` or + `REJECTED` with the two hashes through this chain. + +Return this redacted structure: +- Commit: +- Worktree: CLEAN or BLOCKED +- Backup timer timeout: PASS/FAIL +- Backup failure-status regression: PASS/FAIL +- Backup run: PASS/FAIL/SKIPPED +- Homepage Host-header matrix: PASS/FAIL per identity class +- Primary credential consumer update: COMPLETE/BLOCKED +- Replica credential rotation: COMPLETE/STAGED/BLOCKED +- Retiring replica credential: REJECTED/STILL ACTIVE/UNKNOWN +- HTTPS-origin sync: PASS/FAIL/SKIPPED +- Pi Homepage AdGuard widget: PASS/FAIL/SKIPPED +- CUPS renderer regression and redacted dry-run: PASS/FAIL +- CUPS root-only candidate ready for separate human review: YES/NO +- CUPS source hash: +- CUPS candidate hash: +- Blockers: redacted description or NONE +- Confirmation: no tracked edits, commits, pushes, or PR changes +``` + +## Round 2 — Apply the approved CUPS policy + +The human must review the exact root-only candidate and diff from a separate +non-LLM terminal and return `APPROVED` with the two hashes before sending this +message. Replace both placeholders with those exact hashes. + +```text +The human returned `APPROVED` after inspecting the exact root-only CUPS +candidate and diff in a separate non-LLM terminal. + +Approved source hash: +Approved candidate hash: + +Require both placeholders to be replaced by 64-character lowercase hashes, then +run: + +`sudo bash linux-pi/cups/setup.sh --apply-reviewed ` + +Do not manually edit or substitute another candidate. The script must reject a +changed source, candidate, manifest, or unsafe artifact. + +Before restart, run the available CUPS configuration syntax check. If it fails, +restore the root-only backup and stop. If valid, handle `cups.socket` exactly as +identified, restart CUPS, and verify: +1. CUPS listens on port 631; +2. HTTP from the sidecar to the host returns a successful response; +3. the tailnet HTTPS endpoint returns a successful response with certificate + verification enabled; +4. no `Allow all` or `ServerAlias *` remains; +5. localhost, the exact sidecar subnet, and the exact family LAN/WLAN subnet are + allowed for root printing; +6. every admin block allows only localhost and the sidecar subnet and requires + the exact system-user authentication policy; +7. a family device on LAN/WLAN can discover the printer and print a test page; +8. an ordinary family LAN/WLAN client cannot access administrative routes; +9. a client outside all approved networks is denied; if none is available, mark + this check SKIPPED rather than PASS; +10. the repository worktree remains clean. + +Return PASS/FAIL for each check, whether rollback was required, and redacted +blockers. Do not return the real subnet, hostname, or configuration file. +``` + +## Completion criteria + +- The Pi backup oneshot has an effective two-hour timeout and a successful run. +- Homepage accepts every intended local/tailnet identity and rejects an unknown + Host header. +- The sync origin uses tailnet HTTPS with the replacement primary credential. +- The replica and Pi Homepage use a replacement replica credential, and the old + replica credential is rejected. +- CUPS supports family LAN/WLAN printing plus sidecar HTTPS access, keeps + administration off the family LAN, and contains no `Allow all` or + `ServerAlias *`. diff --git a/TODO.md b/docs/TODO.md similarity index 86% rename from TODO.md rename to docs/TODO.md index 86b7d9e..b221861 100644 --- a/TODO.md +++ b/docs/TODO.md @@ -1,7 +1,7 @@ # TODO Open work only. Completed work is recorded in [CHANGELOG.md](CHANGELOG.md); the -unified-layout design rationale is in [docs/UNIFICATION.md](docs/UNIFICATION.md). +unified-layout design rationale is in [UNIFICATION.md](UNIFICATION.md). ## Live-run cleanup & follow-ups (unification / dotfiles) @@ -29,6 +29,13 @@ run leaves shadowed binaries to reconcile. `command -v` every migrated tool to catch shadowed binaries - [ ] Later: consider base + per-platform overlay for zshrc (desktop vs server vs macOS) +## macOS benchmark verification + +- [ ] Re-run every benchmark suite end-to-end on one Mac and confirm the result + JSON has no unexpected `null` fields before treating the measurements as + validated. The review fixes landed, but no completed post-fix suite run is + recorded yet. + ## OpenCode local models Config uses `mlx_lm.server` with Qwen 3.5 9B (4bit, MLX) on the Mac Mini M4. @@ -54,7 +61,7 @@ Core Arch/CachyOS support shipped in PR #18 (see CHANGELOG). Remaining: Every tailnet-facing service is converted (see CHANGELOG); the non-tailnet edge is what's left. Pattern and full rollout table in -[linux-server/HTTPS.md](linux-server/HTTPS.md). +[../linux-server/HTTPS.md](../linux-server/HTTPS.md). - [ ] Set up the NPM trusted-HTTPS edge (domain `ulises-c.me`, already owned): NPM wildcard Let's Encrypt cert for `*.home.ulises-c.me` via DNS-01, AdGuard @@ -93,15 +100,15 @@ logs, not metrics. Build it up in layers: ### Broader improvements (from the post-rollout review) -- [ ] **Pin the Tailscale sidecar image.** All ~13 sidecars run +- [ ] **Pin the Tailscale sidecar image.** All 22 sidecars run `tailscale/tailscale:latest` and watchtower auto-updates them — a bad release could drop every HTTPS front door at once. Pin a stable tag (bump deliberately) or exclude the sidecars from watchtower. Cheap, high-value. -- [ ] **DRY the sidecar boilerplate.** ~13 near-identical `-ts` blocks + +- [ ] **DRY the sidecar boilerplate.** 22 near-identical `-ts` blocks + `ts-serve.json` (differ only by hostname/port). Use Compose `extends` from a shared base so a global change (the image pin above, `TS_EXTRA_ARGS`) is one edit, not 13. Medium effort — touches all stacks, needs live re-verify. -- [ ] **One shared `TS_AUTHKEY`.** The same OAuth secret is copied into ~13 `.env` +- [ ] **One shared `TS_AUTHKEY`.** The same OAuth secret is copied into 22 `.env` files; rotation/rebuild means editing all of them. Share one env file. - [ ] **Validation script for the server stacks** (CI, like `dryrun-smoke.sh`): assert every `linux-server/*/` has matching compose + `ts-serve.json` + @@ -125,7 +132,7 @@ kept the primary AdGuard from recovering. resolvers on `adguard-ts` (`dns: [9.9.9.10, 1.1.1.1]`) so bootstrap never depends on AdGuard — `linux-server/adguard`. - [x] **Secondary DNS on the Pi.** Kill the single point of failure: a backup - AdGuard on `ollie-pi4`, host-networked (independent of Tailscale) and + AdGuard on ``, host-networked (independent of Tailscale) and config-synced from the primary, handed out as secondary DNS by the router — `linux-pi/adguard` + `linux-pi/adguardhome-sync`. - [ ] **Secondary DHCP.** DHCP is still single-homed on the server; a server @@ -141,14 +148,15 @@ torrents only). Before broader use, route all torrent traffic through a VPN. - [ ] Pick a provider — evaluate free Cloudflare WARP vs a paid WireGuard provider - [ ] Add the provider creds to `.env.example` / `.env` -## linux-server — Raspberry Pi 4 +## linux-pi — Raspberry Pi 4 -Set up the Raspberry Pi 4 headless server config under `linux-server/`. +Docker Compose service stacks now live under `linux-pi/`; base OS provisioning is +still separate from the unified Ubuntu Server profile. -- [ ] Audit existing linux-server/ files and update as needed -- [ ] Create or update packages JSON for the Pi (arm64, Debian-based) -- [ ] Create setup script for headless server (no GUI packages, no snap) -- [ ] Zsh config (server variant — no Ghostty, no fastfetch on launch, no desktop notifications) -- [ ] Tailscale, Docker, SSH hardening -- [ ] Homepage dashboard config (already exists under linux-server/homepage/) -- [ ] Test on Raspberry Pi 4 +- [x] Secondary AdGuard Home with config sync +- [x] Pi Homepage dashboard and Tailscale front doors +- [x] MotionEye, CUPS, and backup service configuration +- [ ] Add a Debian/arm64 Pi platform to the root provisioning engine (no snap/PPA) +- [ ] Add the shared headless zsh/Tailscale/Docker/SSH base without duplicating + `platforms/server.sh` +- [ ] Run and record the complete provisioning and service verification on Pi hardware diff --git a/docs/UNIFICATION.md b/docs/UNIFICATION.md index 27ae26c..c39fafe 100644 --- a/docs/UNIFICATION.md +++ b/docs/UNIFICATION.md @@ -1,8 +1,8 @@ # Setup-script unification — design doc Status: **implemented.** Shipped across [#37](https://github.com/ulises-c/Computer-Setup/pull/37) -(Phases 1–7); see [CHANGELOG.md](../CHANGELOG.md) for what landed and -[CLAUDE.md](../CLAUDE.md) for the current architecture. This document is retained +(Phases 1–7); see [CHANGELOG.md](CHANGELOG.md) for what landed and +[AGENTS.md](../AGENTS.md) for the current architecture. This document is retained as the design record — the rationale for collapsing the three diverged stacks into one engine, the schema decisions, and the phased migration that got us there. The open questions below are resolved inline. Read it to understand *why* the layout is @@ -195,7 +195,7 @@ ubuntu/debian/mint/pop and arch/cachyos/manjaro/endeavouros/garuda/arcolinux, wi ## Phased work breakdown Each phase was independently committable and gated by a verification step. All -phases shipped (see [CHANGELOG.md](../CHANGELOG.md)); the breakdown is kept below as +phases shipped (see [CHANGELOG.md](CHANGELOG.md)); the breakdown is kept below as the migration record. - **Phase 0 — Plan (this PR).** Add `UNIFICATION.md` + TODO items. No behavior change. diff --git a/docs/macos-benchmark-review.md b/docs/macos-benchmark-review.md new file mode 100644 index 0000000..6f4f22d --- /dev/null +++ b/docs/macos-benchmark-review.md @@ -0,0 +1,134 @@ +# macOS benchmark suite — review findings (2026-07) + +Findings from the max-effort review of the `feat/packages-macos-benchmarks` branch +(10 finder angles, per-finding verification, gap sweep), shipped in +[#53](https://github.com/ulises-c/Computer-Setup/pull/53). **All resolved** — this +is the engineering record of what was wrong and why, preserved out of `TODO.md`. + +Most failures were silent (`|| true` / `2>/dev/null` degrade to `null` fields), so +the fix protocol was: after the P0/P1 fixes, re-run every suite end-to-end on one +Mac and confirm the result JSON has no unexpected nulls before trusting numbers. + +## P0 — measurement paths broken, data corruption, or setup aborts + +- `benchmark.sh:56` (also `stress-test.sh:59,91`) — `openssl speed -seconds` is not + supported by stock macOS LibreSSL, and the unguarded `$( )` under `set -e` killed + the script silently right after the section header. Resolve a `-seconds`-capable + openssl at startup (brew `openssl@3` is keg-only — probe + `$(brew --prefix openssl@3)/bin/openssl`) or die with a clear install hint. +- `llm-bench.sh:137` — `llama-bench` does not accept `--hf-repo` (that flag belongs + to llama-cli/llama-server), so the whole llama.cpp half failed arg parsing with + stderr discarded. Pre-download the GGUF and pass `-m `; stop discarding + `llama-bench` stderr. +- `benchmark.sh:250-253` — GPU `llama-bench` parse always null: the `grep -v "^\["` + filter stripped the JSON array's opening bracket, and `jq -s '.[0].avg_ts'` + double-wrapped the array (and `[0]` would be the pp row, not tg). Parse like + `llm-bench.sh` (`jq '[.[] | select(...)]'`) or drop the GPU section. +- `standardized.sh:105` — Cinebench detection used `-maxdepth 3` but the binary sits + at depth 4 (`.../Cinebench.app/Contents/MacOS/Cinebench`); never detected even + after our own installer runs. Use `-maxdepth 4`. +- `platforms/macos.sh:72` — one failing custom installer (e.g. a 404'd Cinebench DMG + URL) aborted the entire remaining setup run under `set -e`. Collect failures and + continue, like `BREW_FAILURES` (the #31 pattern). +- `compare.sh:117` — a metric missing on machine A crashed the comparison mid-table: + `pct()` yields null when `av == 0`, `@tsv` renders null as an empty field, + `IFS=$'\t' read` collapses adjacent tabs (shifting `winner` into `pct`), and + `printf '%+.1f%%'` then fails under `set -e`. Emit the literal string `"null"`. +- `stress-test.sh:91` — throttle methodology was self-defeating: baseline was one + openssl thread on an idle machine (single-core boost, P-core) but each sample + contends with NCPU stressors, so a healthy Mac read ~0.5–0.7 and flagged THROTTLE. + Reworked (baseline as first sample under load / sample-trend instead of idle-ratio). +- `stress-test.sh:105-114` — powermetrics parse patterns were Intel-era and never + matched Apple Silicon: frequency is `... HW active frequency: N MHz` (lowercase f) + and power is `CPU Power: N mW` (not `Package power:`); also convert mW → W. +- `benchmark.sh:124` — memory-bandwidth `awk /stream/` matched stress-ng's + `dispatching hogs: 1 stream` info line before the metrics row, printing 0. Anchor + on the metrics row (`/metrc.*stream/`). +- `omlx-bench.sh:159` — `fire_one` converted failed requests (curl error, 429/5xx) + into `{}`: token totals silently shrank while wall time still included the failure, + corrupting `aggregate_tps` / `peak_aggregate_tps` / `batching_speedup`. Count + failures per level, surface the count in the result JSON, warn/fail on any failure. +- `standardized.sh:142` — Blender's `benchmark-launcher-cli` does not auto-download + the runtime/scenes; run `blender download ` and `scenes download -b ` + first (or die with instructions), else `blender_benchmark` is null on fresh install. +- `standardized.sh:111-118` — single-core Cinebench parse grepped the combined raw + file (multi wrote first, single appended, failures `|| true`-swallowed), so a + failed single run silently recorded the multi-core score as `cpu_single`. Use a + separate raw file per run. +- `compare.sh:38-75` — no `stress` case: comparing two stress results died + `unknown suite: stress` while the README advertised it. Add a stress table (or drop + the claim); also fix the header comment, which omitted the supported `omlx` suite. +- `lib/verify.sh:92` + the new cinebench/omlx `packages.json` entries — the macOS + custom probe only tried `brew list --formula` / `command -v`, so GUI-only `.app` + installs could never verify. Add an app-store-style `[[ -d /Applications/.app ]]` + probe for custom entries. +- `benchmarks/README.md:85` — the compare example embedded two real machine + short-hostnames in this public repo (privacy rule: placeholders only) and used a + `results/` path that doesn't resolve from the repo root. Use `` / + `` placeholders and the `macOS/benchmarks/results/` path. + +## P1 — moderate correctness + +- `standardized.sh:116` — `--cpu-only` must not skip the single-core Cinebench run: + it is a CPU test; only Blender/GPU belongs behind that flag. +- `llm-bench.sh:99-101` — the PP/TG/MEM parse pipelines had no `|| true`; under + pipefail a non-matching grep killed the run instead of reaching the `=null` + fallbacks. +- `stress-test.sh:39` / `omlx-bench.sh:74` — INT/TERM traps didn't `exit`; a plain + `kill` mid-run stopped the load but the sample loop continued on an idle machine + and wrote a bogus `throttled:false` result. End the handlers with `exit`. +- `omlx-bench.sh:100` — `OMLX_PORT` built BASE_URL but was never passed to + `omlx serve`, so overriding the port polled an address the server never bound. + Pass the port flag (or reject the override). +- `stress-test.sh` + README `sudo` — a first run under sudo created root-owned + `results/`; later non-sudo suites finished then died at the final `> "$OUTFILE"`. + Create/chown `results/` as `$SUDO_USER` when running under sudo. +- `compare.sh:113,131` — a metric present on only one machine rendered as 0-vs-real + and counted as a win, skewing the summary; skip or mark those rows. +- `compare.sh:61` — `standardized.sh` never emits `.geekbench_ai.score` (only + result_url/mode/note), so the row was dead; parse a score or drop the row. +- `benchmark.sh:107` — `scaling_factor` was passed with `--arg`, landing as a JSON + string (or literal `"null"`); use `--argjson`/`tonumber` like `$gbs`. + +## P2 — minor / latent + +- `platforms/macos.sh:68` — macOS never called `custom_reminders_section`, so any + future custom entry without `handled_by_setup: true` was silently dropped. Wire the + reminder section into `platform_main` like `linux_main`. +- `scripts/validate-packages.sh` — validate `handled_by_setup` is a real boolean and + custom entries carry an `install_command` (a string `"true"` or missing command + previously passed and degraded silently). +- `macOS/lib-dmg-install.sh:30` — handle hdiutil's already-attached reuse (image + mounted via Finder → `-mountpoint` ignored, empty mount dir, misleading + `no .app found` death, pre-existing mount left attached). +- `platforms/macos.sh` dry-run fidelity — gate the pipx `[i/N]` progress line on + DRY_RUN (:86), print a `[dry-run] sudo -v` line in `mac_prime_sudo` (:96), include + `--adopt` in the cask progress/FAIL lines (:50, :56). +- `platforms/macos.sh:99` — sudo keepalive inherited `set -e` (one failed + `sudo -n true` silently killed it) and held stdout so a piped run hung up to 60s + after exit; add `|| true` and redirect stdout. +- `standardized.sh:59` — grep the already-captured `$GB_RAW` for the Geekbench result + URL before re-running the whole CPU benchmark (the fallback also truncated the + first run's output). +- `omlx-bench.sh:127` — when `OMLX_MODEL` is set, don't die on an empty `/v1/models` + list (lazy-loading servers list nothing until the first request). + +## P3 — cleanup (dedupe within the new code) + +- `macOS/benchmarks/lib.sh` — add a `bench_init ` helper for the + SYSINFO/HOSTNAME_SHORT/OUTFILE/banner prologue (was copy-pasted ×5) and a single + `SUITE_VERSION` constant (literal `"1.0.0"` ×5). +- `macOS/benchmarks/lib.sh` — extract the openssl-speed sha256 run+parse into one + helper (was ×4 across benchmark.sh / stress-test.sh); pairs with the P0 LibreSSL fix. +- `platforms/macos.sh` — factor the `[i/N]` progress-counter plumbing shared by the + brew/cask/pipx tiers (×3) into `mac_install_list`; pipx failures now also collect + into the summary instead of aborting the run. +- `macOS/lib-dmg-install.sh` — move the curl/hdiutil dep checks, the already-installed + guard, and the success message into the lib (kept its own `info`/`die` — sourcing + `benchmarks/lib.sh` would couple the standalone installers to the suite internals). +- `platforms/macos.sh:285` — drive the codeburn menubar reminder from `packages.json` + instead of a hardcoded package-name check — new `codeburn-menubar` custom entry + (priority none, handled_by_setup false) rendered by `custom_reminders_section`. +- `macOS/install-cinebench.sh:12` — make `DMG_URL` env-overridable + (`CINEBENCH_DMG_URL`); it pinned a versioned filename while the comment claimed a + rolling stable URL. diff --git a/lib/core.sh b/lib/core.sh index 6aa19a0..448e6d5 100755 --- a/lib/core.sh +++ b/lib/core.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# Shared engine for the unified root setup.sh (UNIFICATION.md, issue #36). +# Shared engine for the unified root setup.sh (docs/UNIFICATION.md, issue #36). # Sourced by setup.sh after SETUP_ROOT is set. Platform modules in # platforms/.sh provide platform_main() plus the hooks used by # linux_main(): platform_bootstrap, platform_install_tier, @@ -901,6 +901,7 @@ linux_main() { printf '\n==> Installing optional (low) packages...\n' platform_install_tier low fi + server_ups_step server_extras custom_reminders_section server_footer diff --git a/lib/verify.sh b/lib/verify.sh index c861715..0922343 100755 --- a/lib/verify.sh +++ b/lib/verify.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# Shared read-only verify engine for the unified root verify.sh (UNIFICATION.md +# Shared read-only verify engine for the unified root verify.sh (docs/UNIFICATION.md # Phase 3, issue #36). Sourced by verify.sh after lib/core.sh — reuses # CORE_JQ_DEFS, PACKAGES_JSON, PLATFORM, INCLUDE_* and core_detect_platform. # Check semantics are ported from macOS/verify.sh and linux-desktop/verify.sh; @@ -17,7 +17,7 @@ verify_parse_args() { --all) INCLUDE_OPTIONAL=true; INCLUDE_WORK=true; INCLUDE_PERSONAL=true; INCLUDE_NONE=true ;; --distro|--platform) if [[ -z "${2:-}" || "${2:-}" == -* ]]; then - printf 'ERROR: %s requires a value (macos|ubuntu|arch).\n' "$1" >&2 + printf 'ERROR: %s requires a value (macos|ubuntu|arch|server).\n' "$1" >&2 exit 1 fi PLATFORM="$2"; shift ;; @@ -252,6 +252,39 @@ verify_extras_linux() { fi } +verify_extras_server() { + printf '\n── Server runtime & services ────────────────────────────────\n' + + [[ -f "$HOME/.zshrc" ]] && check "zshrc present (~/.zshrc)" true || check "zshrc present (~/.zshrc)" false + [[ -f "$HOME/.zsh_plugins.txt" ]] && check "antidote plugin list present (~/.zsh_plugins.txt)" true || check "antidote plugin list present (~/.zsh_plugins.txt)" false + [[ -f "$HOME/.tmux.conf" ]] && check "tmux config present (~/.tmux.conf)" true || check "tmux config present (~/.tmux.conf)" false + + if command -v systemctl &>/dev/null && systemctl is-active --quiet tailscaled 2>/dev/null; then + check "tailscaled service active" true + else + check "tailscaled service active" false + fi + + local unit + for unit in nut-driver@cyberpower.service nut-server.service nut-monitor.service; do + if command -v systemctl &>/dev/null \ + && systemctl is-enabled --quiet "$unit" 2>/dev/null \ + && systemctl is-active --quiet "$unit" 2>/dev/null; then + check "$unit enabled and active" true + else + check "$unit enabled and active" false + fi + done + + local ups_status + ups_status="$(upsc cyberpower@localhost ups.status 2>/dev/null || true)" + if [[ -n "$ups_status" ]]; then + check "UPS reachable (status: $ups_status)" true + else + check "UPS reachable via upsc cyberpower@localhost" false + fi +} + # npm + pnpm supply-chain cooldown checks (issue #23) — identical on every platform. verify_npm_pnpm_cooldowns() { if grep -q '^min-release-age=' "$HOME/.npmrc" 2>/dev/null; then @@ -279,16 +312,12 @@ verify_npm_pnpm_cooldowns() { } verify_main() { - if [[ "$PLATFORM" == "server" ]]; then - printf 'ERROR: no verify checks for the server profile yet (linux-server has no legacy verify.sh).\n' >&2 - exit 1 - fi - printf '==> Verifying %s package installs against %s\n' "$PLATFORM" "$PACKAGES_JSON" case "$PLATFORM" in macos) verify_extras_macos ;; ubuntu|arch) verify_extras_linux ;; + server) verify_extras_server ;; esac verify_section "High priority" "high" diff --git a/linux-desktop/setup.sh b/linux-desktop/setup.sh index c3c1bda..cd719ed 100755 --- a/linux-desktop/setup.sh +++ b/linux-desktop/setup.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# Thin shim (UNIFICATION.md Phase 4): the install logic lives in the root +# Thin shim (docs/UNIFICATION.md Phase 4): the install logic lives in the root # setup.sh + lib/core.sh + platforms/{ubuntu,arch}.sh, driven by the root # packages.json. All flags are forwarded; the distro is auto-detected and # --distro overrides. diff --git a/linux-desktop/verify.sh b/linux-desktop/verify.sh index 0a32392..31b9b7c 100755 --- a/linux-desktop/verify.sh +++ b/linux-desktop/verify.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# Thin shim (UNIFICATION.md Phase 4): the check logic lives in the root +# Thin shim (docs/UNIFICATION.md Phase 4): the check logic lives in the root # verify.sh + lib/verify.sh, driven by the root packages.json. All flags are # forwarded; the distro is auto-detected and --distro overrides. # Usage: bash linux-desktop/verify.sh [--optional] [--work] [--personal] diff --git a/linux-pi/README.md b/linux-pi/README.md index 40c17ce..6ec1973 100644 --- a/linux-pi/README.md +++ b/linux-pi/README.md @@ -1,4 +1,4 @@ -# linux-pi — Raspberry Pi (`ollie-pi4`) node +# linux-pi — Raspberry Pi (``) node Service stacks for the Raspberry Pi, deployed from this repo (clone + `docker compose up -d`). Mirrors the `linux-server//` layout: each folder is a @@ -37,7 +37,7 @@ start; if the primary is down, the replica simply keeps its last-good config. The Pi is a *secondary server* (security cameras via MotionEye, printing via CUPS, plus the backup AdGuard). These stacks surface it: -- `homepage/` — a homepage dashboard for the Pi (host-networked on `:3000`) with a +- `homepage/` — a homepage dashboard for the Pi (host-networked on `:3001`) with a decoupled `homepage-pi-ts` sidecar → `https://homepage-pi..ts.net`. Its cards link to the Pi services, and it shows the Pi's own CPU/mem/disk/temp (the `resources` widget works because homepage runs on the Pi host). @@ -60,14 +60,40 @@ the docker-bridge→host path. Diagnose on the Pi: ```bash docker exec -ts tailscale serve status docker exec -ts sh -c 'getent hosts host.docker.internal; \ - wget -qO- -T5 http://host.docker.internal:/ >/dev/null && echo OK || echo UNREACHABLE' + wget -qO- -T5 --header="Host: " \ + http://host.docker.internal:/ >/dev/null && echo OK || echo UNREACHABLE' docker version --format '{{.Server.Version}}' sudo ufw status ``` Fix it once (allow bridge→host / enable `host-gateway`) and every sidecar works. -CUPS additionally validates the `Host` header — if its admin pages 400 through the -proxy, add `ServerAlias *` to `cupsd.conf` and restart cups. + +For CUPS, use the reviewed policy installer. Family LAN/WLAN clients are allowed +to print through the root location; administrative locations remain limited to +localhost and the pinned sidecar subnet. Put the real aliases and canonical +private LAN subnet only in the gitignored `.env`: + +```bash +cd linux-pi/cups +cp .env.example .env +# edit .env privately +chmod 600 .env +bash test-setup.sh +bash setup.sh --dry-run +sudo bash setup.sh --prepare-review +``` + +Inspect the root-only candidate and diff from a separate trusted terminal as +described in `cups/README.md`. After approving both printed hashes, apply exactly +that candidate: + +```bash +sudo bash setup.sh --apply-reviewed +``` + +The installer rejects broad access rules, validates with `cupsd`, handles +socket activation, rolls back on restart or local-probe failure, and never emits +the private aliases, CIDRs, candidate, or diff during its normal output. ### First HTTPS request provisions a cert (may hang once) @@ -93,8 +119,9 @@ that's normal, not a fault. If a sidecar *stays* DERP-only, check the Pi's UDP/N ## Deploy runbook (on the Pi) -Prerequisites: Docker + compose plugin (`setup.sh --profile server` covers the -base, or install manually). +Prerequisites: Raspberry Pi OS/Debian with Git, Docker Engine, and the Compose +plugin installed manually. The root `setup.sh --profile server` targets Ubuntu +Server and is not supported on the Pi yet. 1. **Free port 53.** Debian's `systemd-resolved` stub may hold `:53`. Set `DNSStubListener=no` in resolved's config and restart it (or bind AdGuard to @@ -115,17 +142,22 @@ base, or install manually). docker compose up -d ``` The Pi's filters/rewrites/upstreams should now match the primary. -4. **Wire failover on the router (`192.168.1.1`):** in its DHCP settings, set the +4. **Wire failover on the router (``):** in its DHCP settings, set the DNS servers to `[, ]` (primary = server, secondary = Pi). Renew a client lease to pick it up. 5. **Bring up the Pi dashboard + service front doors:** for each of `homepage`, `motioneye`, `cups`: `cp .env.example .env`, set `TS_AUTHKEY` (and, for - `homepage`, the `HOMEPAGE_VAR_*` domains + AdGuard creds), then - `docker compose up -d`. The Pi dashboard is then at + `homepage`, the `HOMEPAGE_VAR_*` domains, LAN identity, and AdGuard creds), then + `docker compose up -d`. The Pi dashboard is available on the LAN at + `http://:3001` and at `https://homepage-pi..ts.net`, and the main server's homepage shows a "Secondary Server (Pi)" card linking to it. (All Pi HTTPS front doors depend on the sidecar→host hop above.) + When upgrading an existing checkout, add new keys from `.env.example` to the + gitignored Homepage `.env`; the main-server card requires + `HOMEPAGE_VAR_MAIN_HOSTNAME`. + ## Verification ```bash diff --git a/linux-pi/adguard/docker-compose.yml b/linux-pi/adguard/docker-compose.yml index 5fe91c3..ce0b7da 100644 --- a/linux-pi/adguard/docker-compose.yml +++ b/linux-pi/adguard/docker-compose.yml @@ -1,5 +1,5 @@ services: - # Secondary AdGuard Home on ollie-pi4 — a backup resolver mirroring the primary + # Secondary AdGuard Home on — a backup resolver mirroring the primary # (linux-server/adguard) via adguardhome-sync. DELIBERATELY host-networked and # independent of Tailscale: :53 must keep serving LAN clients even when the # tailnet or internet is down. The primary rides its sidecar's netns; that diff --git a/linux-pi/adguardhome-sync/.env.example b/linux-pi/adguardhome-sync/.env.example index 8281a12..2247879 100644 --- a/linux-pi/adguardhome-sync/.env.example +++ b/linux-pi/adguardhome-sync/.env.example @@ -2,8 +2,9 @@ # Repo is public — keep real IPs and credentials in .env only, never committed. # --- Origin: the PRIMARY AdGuard on the main server --- -# Use the LAN IP (always reachable, no Tailscale dependency for sync). -ORIGIN_URL=http:// +# The primary admin API is exposed only by its Tailscale HTTPS sidecar. +# If the tailnet is unavailable, sync pauses and the replica keeps its last-good config. +ORIGIN_URL=https://adguard..ts.net ORIGIN_USERNAME= ORIGIN_PASSWORD= diff --git a/linux-pi/backup/.env.example b/linux-pi/backup/.env.example index ede5443..2261d55 100644 --- a/linux-pi/backup/.env.example +++ b/linux-pi/backup/.env.example @@ -1,18 +1,15 @@ # Copy this file to .env and set the values. The .env is gitignored. -# restic repository on the main server (over SFTP). Restic's built-in SSH client -# doesn't read ~/.ssh/config, so the key must be at ~/.ssh/id_ed25519 (the -# default path). After running add_remote_host.sh, symlink it: -# ln -sf ~/.ssh/backup ~/.ssh/id_ed25519 -# ln -sf ~/.ssh/backup.pub ~/.ssh/id_ed25519.pub -RESTIC_REPOSITORY=sftp:restic-pi@:/mnt/wd1tb/restic-pi -# Encryption password. STORE THIS SECURELY — without it the backup is -# permanently unrecoverable. Escape a literal $ as $$ (Docker/Compose-style). +# Restic repository on the main server over SFTP. The systemd unit runs with +# HOME=/root, so host aliases and identity files resolve through /root/.ssh/config. +RESTIC_REPOSITORY=sftp:pi-backup-target:/mnt/wd1tb/restic-pi +# Encryption password. STORE THIS SECURELY — without it the backup is permanently +# unrecoverable. `openssl rand -hex 32` produces a shell-safe value for this file. RESTIC_PASSWORD= # Optional second copy for redundancy (3-2-1-ish). Leave SECOND_RESTIC_REPOSITORY # empty to disable. Uses the same RESTIC_PASSWORD. -SECOND_RESTIC_REPOSITORY=sftp:restic-pi@:/mnt/wd14tb/restic-pi-copy +SECOND_RESTIC_REPOSITORY=sftp:pi-backup-target:/mnt/wd14tb/restic-pi-copy # Retention (snapshots kept after each prune). RETENTION_KEEP_DAILY=7 diff --git a/linux-pi/backup/LEARNINGS.md b/linux-pi/backup/LEARNINGS.md index 93f872c..f06ad15 100644 --- a/linux-pi/backup/LEARNINGS.md +++ b/linux-pi/backup/LEARNINGS.md @@ -6,8 +6,8 @@ server's DAS over the tailnet. Save these before they evaporate. ## Linux nologin shells output a message `/usr/sbin/nologin` on Debian prints "This account is currently not available." -to stderr before exiting. This corrupts the SFTP protocol stream — restic's -built-in SSH client sees "packet too long" and fails. +to stderr before exiting. This corrupts the SFTP protocol stream, and restic +reports "packet too long". **Fix:** Use `/bin/false` instead. It exits silently with status 1, no output. @@ -30,25 +30,20 @@ Match User `internal-sftp` runs the SFTP subsystem in-process (no shell needed). This also locks the account to SFTP-only — no shell commands, no port forwarding. -## Restic's `sftp:` backend doesn't read `~/.ssh/config` +## Restic's `sftp:` backend uses the system SSH command -Restic's SFTP backend uses its own Go SSH client (from `x/crypto/ssh`), not the -system `ssh` binary. It doesn't read `~/.ssh/config`, `~/.ssh/known_hosts`, or -any SSH config at all. +Restic shells out to `ssh` for SFTP repositories. It reads the active user's SSH +configuration, identities, and known-host entries. The systemd unit sets +`HOME=/root`, so its configuration belongs under `/root/.ssh/`. -**What it does look for:** -- `~/.ssh/id_ed25519` (or `id_rsa`) — the default identity file path -- The `sftp.command` option can override the SSH command, but it's fiddly - -**Workaround:** Symlink the backup key to the default path: - -```sh -ln -sf ~/.ssh/backup ~/.ssh/id_ed25519 -ln -sf ~/.ssh/backup.pub ~/.ssh/id_ed25519.pub +```sshconfig +Host pi-backup-target + HostName + User + IdentityFile /root/.ssh/backup ``` -This must be done for both the user (`ollie`) and `root` (since the systemd -timer runs as root). +Repository URLs can then use `sftp:pi-backup-target:/path/to/repository`. ## The `ssh:` backend doesn't work with ForceCommand @@ -69,21 +64,6 @@ users. In practice, the simplest path is: set up the user with a real shell first, run `ssh-copy-id`, then change the shell to `/bin/false`. -## Restic's sftp.command option - -When you set `-o sftp.command=`, restic runs that command and expects to -negotiate SFTP over its stdin/stdout. The command should start an SSH session -that the SFTP subsystem can run over. - -**What works:** Not much — the option is fragile. The symlink-to-default-key -approach is far more reliable. - -**What doesn't work:** -- `sftp.command=/usr/lib/openssh/sftp-server` — runs a local SFTP server, - not over SSH -- `sftp.command=ssh -T -i /path/to/key user@host` — can still get protocol - corruption from environment variables or shell output - ## Backup key should have no passphrase For automated backups (systemd timer), the SSH key must not have a passphrase. @@ -98,7 +78,7 @@ ssh-keygen -t ed25519 -C "backup" -f ~/.ssh/backup -N "" Systemd timers typically run as root. If the backup script uses SSH/SFTP, root needs: - The private key (with `chmod 600`) -- `~/.ssh/id_ed25519` symlink (for restic's Go client) +- An `IdentityFile` entry in `/root/.ssh/config` - `~/.ssh/known_hosts` (host key acceptance) Copy from the regular user: @@ -107,16 +87,14 @@ Copy from the regular user: sudo mkdir -p /root/.ssh sudo cp ~/.ssh/backup /root/.ssh/backup sudo cp ~/.ssh/backup.pub /root/.ssh/backup.pub -sudo ln -sf /root/.ssh/backup /root/.ssh/id_ed25519 -sudo ln -sf /root/.ssh/backup.pub /root/.ssh/id_ed25519.pub sudo chmod 700 /root/.ssh -sudo chmod 600 /root/.ssh/backup /root/.ssh/id_ed25519 -sudo chmod 644 /root/.ssh/backup.pub /root/.ssh/id_ed25519.pub +sudo chmod 600 /root/.ssh/backup +sudo chmod 644 /root/.ssh/backup.pub sudo cp ~/.ssh/known_hosts /root/.ssh/known_hosts ``` -## Timer offset avoids contention +## Timer offset only reduces contention If two machines back up to the same server, offset the timers. The Pi fires at -03:45 (15 min after the server's 03:30) to avoid simultaneous SFTP sessions to -the same DAS. +03:45 (15 min after the server's 03:30), but long prune/check runs can still +overlap. Choose the offset from observed server backup durations. diff --git a/linux-pi/backup/README.md b/linux-pi/backup/README.md index 44b81f5..ef66a7f 100644 --- a/linux-pi/backup/README.md +++ b/linux-pi/backup/README.md @@ -7,13 +7,14 @@ ntfy + a homepage card. Mirrors the server's own backup architecture ## What gets backed up -- **AdGuard** `conf/` + `work/` — DNS settings, filter lists, rewrites, runtime state +- **AdGuard** `conf/` — DNS settings, filter lists, rewrites, and restorable state - **Homepage** `config/` — dashboard widgets, services, bookmarks - **MotionEye** `/etc/motioneye/` — camera configs (recordings excluded — large/disposable) - **CUPS** `/etc/cups/` — printer configs, PPD files - Every service's gitignored **`.env`** (secrets needed to restore) -Excluded as disposable/regenerable: MotionEye recordings, all `ts-state/` +Excluded as disposable/regenerable: AdGuard query/statistics databases in `work/`, +MotionEye recordings, all `ts-state/` (Tailscale node keys — re-auth with `TS_AUTHKEY` regenerates them), Docker images/containers. @@ -29,12 +30,19 @@ Pi (restic) ──SFTP──> Main Server (restic-pi user) ``` The second copy uses `restic copy --from-repo` (same pattern as the server's -own backup). It's silently skipped if the SFTP connection to the second target -fails. +own backup). It must be initialized explicitly. An unavailable or incomplete +second-repository operation is reported without failing the primary backup. -**Important:** Restic's built-in SSH client doesn't read `~/.ssh/config`. The -backup SSH key must be symlinked to `~/.ssh/id_ed25519` (the default path restic -looks for). The setup steps below handle this. +Restic's SFTP backend uses the system `ssh` command and honors SSH configuration. +Because the timer runs with `HOME=/root`, configure its host, identity, and known +host under `/root/.ssh/`. + +The main script records a current-run marker; its EXIT trap writes detailed +failure status and removes staging. Systemd's `OnFailure` unit preserves and +acknowledges current detail, or replaces a missing, running, success, or stale +failure record before sending the single external ntfy/Kuma alert. The backup +unit has a two-hour runtime limit so a hung SFTP operation cannot block every +later timer run. ## Prerequisites @@ -68,32 +76,31 @@ cat ~/.ssh/backup.pub # On the server, add the output to /home/restic-pi/.ssh/authorized_keys ``` -### 3. Symlink the key for restic - -Restic's Go SSH client looks for `~/.ssh/id_ed25519` by default: - -```sh -ln -sf ~/.ssh/backup ~/.ssh/id_ed25519 -ln -sf ~/.ssh/backup.pub ~/.ssh/id_ed25519.pub -``` - -### 4. Copy key/config/known_hosts to root +### 3. Configure SSH for the root-run timer -The backup timer runs as **root**, so root needs the SSH setup: +Copy the key and known-host entry to root, then configure the repository host: ```sh sudo mkdir -p /root/.ssh sudo cp ~/.ssh/backup /root/.ssh/backup sudo cp ~/.ssh/backup.pub /root/.ssh/backup.pub -sudo ln -sf /root/.ssh/backup /root/.ssh/id_ed25519 -sudo ln -sf /root/.ssh/backup.pub /root/.ssh/id_ed25519.pub sudo chmod 700 /root/.ssh -sudo chmod 600 /root/.ssh/backup /root/.ssh/id_ed25519 -sudo chmod 644 /root/.ssh/backup.pub /root/.ssh/id_ed25519.pub +sudo chmod 600 /root/.ssh/backup +sudo chmod 644 /root/.ssh/backup.pub sudo cp ~/.ssh/known_hosts /root/.ssh/known_hosts 2>/dev/null && sudo chmod 644 /root/.ssh/known_hosts || true +sudoedit /root/.ssh/config ``` -### 5. Server-side setup +```sshconfig +Host pi-backup-target + HostName + User restic-pi + IdentityFile /root/.ssh/backup +``` + +Use `sftp:pi-backup-target:/mnt/...` for the repository URLs in `.env`. + +### 4. Server-side setup The main server needs a dedicated SFTP user for Pi backups. Create it manually or pass this to the server's LLM agent: @@ -120,37 +127,43 @@ Match User restic-pi X11Forwarding no ``` -### 6. Configure +### 5. Configure ```sh cd linux-pi/backup cp .env.example .env # set RESTIC_PASSWORD (and SAVE IT SECURELY), ntfy/Kuma URLs +chmod 600 .env ``` -### 7. Initialize the repos (the script also does this on first run) +### 6. Initialize both repositories ```sh set -a; source .env; set +a restic init -# Also init the second copy repo if using it +restic -r "$SECOND_RESTIC_REPOSITORY" init --copy-chunker-params --from-repo "$RESTIC_REPOSITORY" ``` -### 8. Status card server +The script initializes the primary repository on first use. It never initializes the +optional second repository automatically because an unreachable SFTP target and an +uninitialized repository are not safely distinguishable. + +### 7. Status card server ```sh sudo docker compose up -d ``` -### 9. Install + enable the timer +### 8. Install + enable the timer ```sh -sudo cp pi-backup.service pi-backup.timer pi-backup-failure.service /etc/systemd/system/ -sudo systemctl daemon-reload -sudo systemctl enable --now pi-backup.timer +sudo bash setup.sh ``` -### 10. Dry run + verify +`setup.sh` renders the service units with the current checkout path, installs them, +and enables the timer. Re-run it after moving the checkout. + +### 9. Run + verify ```sh sudo systemctl start pi-backup.service @@ -180,7 +193,10 @@ Then put state back per service: ## Warnings - **Losing `RESTIC_PASSWORD` = unrecoverable backups.** Keep it secure. -- The second copy is silently skipped if the target SFTP connection fails — +- The second copy is reported as incomplete if its target or an operation fails — don't rely on it as your only backup. +- Staged `.env` copies live under `/root/pi-backup-staging` with mode `0700` and + are removed on normal exit. A forced kill can leave that root-only directory for + the next run to replace. - The script skips missing source paths, so it's safe to enable before every service is deployed; coverage grows automatically as data dirs appear. diff --git a/linux-pi/backup/backup.sh b/linux-pi/backup/backup.sh index 0835881..eadd307 100755 --- a/linux-pi/backup/backup.sh +++ b/linux-pi/backup/backup.sh @@ -8,8 +8,9 @@ set -euo pipefail SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" PI_DIR="$(dirname -- "$SCRIPT_DIR")" +readonly STAGING_BASE="${HOME:-/root}" -if [[ -f "$SCRIPT_DIR/.env" ]]; then +if [[ "${1:-}" != "notify-failure" && -f "$SCRIPT_DIR/.env" ]]; then set -a # shellcheck disable=SC1091 source "$SCRIPT_DIR/.env" @@ -17,7 +18,7 @@ if [[ -f "$SCRIPT_DIR/.env" ]]; then fi : "${SECOND_RESTIC_REPOSITORY:=}" -: "${STAGING_DIR:=/var/tmp/pi-backup-staging}" +readonly STAGING_DIR="$STAGING_BASE/pi-backup-staging" : "${RETENTION_KEEP_DAILY:=7}" : "${RETENTION_KEEP_WEEKLY:=4}" : "${RETENTION_KEEP_MONTHLY:=6}" @@ -25,16 +26,14 @@ fi : "${NTFY_TOPIC:=pi-backup}" : "${KUMA_PUSH_URL:=}" : "${STATUS_JSON:=$SCRIPT_DIR/status/backup-status.json}" -: "${RESTIC_REPOSITORY:?set RESTIC_REPOSITORY in .env}" -: "${RESTIC_PASSWORD:?set RESTIC_PASSWORD in .env}" -export RESTIC_REPOSITORY RESTIC_PASSWORD -export RESTIC_FROM_PASSWORD="$RESTIC_PASSWORD" HOSTTAG="$(hostname)" SNAPSHOT_ID="" SIZE_BYTES=0 DURATION=0 START=$SECONDS +STAGING_READY=false +SECOND_COPY_STATUS="" log() { printf '[backup] %s\n' "$*"; } die() { printf 'error: %s\n' "$*" >&2; exit 1; } @@ -42,23 +41,36 @@ die() { printf 'error: %s\n' "$*" >&2; exit 1; } notify() { local title="$1" priority="$2" tags="$3" msg="$4" [[ -n "${NTFY_URL:-}" ]] || return 0 - local args=(-fsS -H "Title: $title" -H "Priority: $priority" -H "Tags: $tags" -d "$msg") + local args=(-fsS --connect-timeout 5 --max-time 10 -H "Title: $title" -H "Priority: $priority" -H "Tags: $tags" -d "$msg") [[ -n "${NTFY_TOKEN:-}" ]] && args+=(-H "Authorization: Bearer $NTFY_TOKEN") - curl "${args[@]}" "$NTFY_URL/$NTFY_TOPIC" >/dev/null 2>&1 || true + curl "${args[@]}" -- "$NTFY_URL/$NTFY_TOPIC" >/dev/null 2>&1 || true } kuma_push() { local status="$1" msg="$2" ping="${3:-}" [[ -n "$KUMA_PUSH_URL" ]] || return 0 - curl -fsS -G \ + curl -fsS --connect-timeout 5 --max-time 10 -G \ --data-urlencode "status=$status" \ --data-urlencode "msg=$msg" \ --data-urlencode "ping=$ping" \ - "$KUMA_PUSH_URL" >/dev/null 2>&1 || true + -- "$KUMA_PUSH_URL" >/dev/null 2>&1 || true +} + +validate_url() { + local name="$1" value="$2" + [[ -z "$value" || "$value" =~ ^https?://[^[:space:]]+$ ]] || die "$name must be an http(s) URL without whitespace" +} + +notification_config_valid() { + [[ -z "${NTFY_URL:-}" || "${NTFY_URL:-}" =~ ^https?://[^[:space:]]+$ ]] \ + && [[ -z "$KUMA_PUSH_URL" || "$KUMA_PUSH_URL" =~ ^https?://[^[:space:]]+$ ]] \ + && [[ "$NTFY_TOPIC" != *$'\n'* && "$NTFY_TOPIC" != *$'\r'* ]] \ + && [[ "${NTFY_TOKEN:-}" != *$'\n'* && "${NTFY_TOKEN:-}" != *$'\r'* ]] } write_status() { local st="$1" + local notifier_pending="${2:-false}" mkdir -p "$(dirname "$STATUS_JSON")" jq -n \ --arg status "$st" \ @@ -66,39 +78,61 @@ write_status() { --arg snapshot "${SNAPSHOT_ID:-}" \ --argjson size "${SIZE_BYTES:-0}" \ --argjson dur "${DURATION:-0}" \ - '{status:$status, last_run:$last_run, snapshot:$snapshot, repo_size_bytes:$size, duration_seconds:$dur}' \ + --argjson notifier_pending "$notifier_pending" \ + '{status:$status, last_run:$last_run, snapshot:$snapshot, repo_size_bytes:$size, duration_seconds:$dur, notifier_pending:$notifier_pending}' \ >"$STATUS_JSON" } -# systemd OnFailure backstop: record + alert even if the main run died early. +# systemd OnFailure owns alerts so a failed run produces exactly one notification. if [[ "${1:-}" == "notify-failure" ]]; then - write_status failed - notify "Pi backup FAILED" urgent rotating_light "systemd OnFailure — see: journalctl -u pi-backup.service" - kuma_push down "systemd OnFailure — see journalctl -u pi-backup.service" + if command -v jq >/dev/null; then + if jq -e '.status == "failed" and .notifier_pending == true' "$STATUS_JSON" >/dev/null 2>&1; then + status_tmp="${STATUS_JSON}.tmp.$$" + if jq '.notifier_pending = false' "$STATUS_JSON" >"$status_tmp" \ + && mv "$status_tmp" "$STATUS_JSON"; then + : + else + rm -f "$status_tmp" + fi + else + write_status failed || true + fi + fi + if notification_config_valid; then + notify "Pi backup FAILED" urgent rotating_light "systemd OnFailure — see: journalctl -u pi-backup.service" + kuma_push down "systemd OnFailure — see journalctl -u pi-backup.service" + else + printf 'warning: invalid notification configuration; failure status recorded without sending alerts\n' >&2 + fi exit 0 fi STATUS=failed finish() { - local rc=$? if [[ "$STATUS" != success ]]; then DURATION=$((SECONDS - START)) - write_status failed - notify "Pi backup FAILED" urgent rotating_light "exit $rc — see: journalctl -u pi-backup.service" - kuma_push down "exit $rc — see journalctl -u pi-backup.service" + command -v jq >/dev/null && write_status failed true || true fi - rm -rf "$STAGING_DIR" + [[ "$STAGING_READY" == true ]] && rm -rf "$STAGING_DIR" } trap finish EXIT # --- guards ----------------------------------------------------------------- -command -v restic >/dev/null || die "restic not installed (apt install restic)" command -v jq >/dev/null || die "jq not installed (apt install jq)" +write_status running +validate_url NTFY_URL "${NTFY_URL:-}" +validate_url KUMA_PUSH_URL "$KUMA_PUSH_URL" +[[ "$NTFY_TOPIC" != *$'\n'* && "$NTFY_TOPIC" != *$'\r'* ]] || die "NTFY_TOPIC cannot contain a newline" +[[ "${NTFY_TOKEN:-}" != *$'\n'* && "${NTFY_TOKEN:-}" != *$'\r'* ]] || die "NTFY_TOKEN cannot contain a newline" +[[ -n "${RESTIC_REPOSITORY:-}" ]] || die "set RESTIC_REPOSITORY in .env" +[[ -n "${RESTIC_PASSWORD:-}" ]] || die "set RESTIC_PASSWORD in .env" +command -v restic >/dev/null || die "restic not installed (apt install restic)" +export RESTIC_REPOSITORY RESTIC_PASSWORD +export RESTIC_FROM_PASSWORD="$RESTIC_PASSWORD" # --- resolve sources -------------------------------------------------------- CANDIDATES=( "$PI_DIR/adguard/conf" - "$PI_DIR/adguard/work" "$PI_DIR/homepage/config" /etc/motioneye /etc/cups @@ -113,13 +147,15 @@ done [[ ${#SOURCES[@]} -gt 0 ]] || log "no service data on disk yet — backing up staging + .env only" # --- staging: collect .env files ------------------------------------------- +umask 077 rm -rf "$STAGING_DIR" -mkdir -p "$STAGING_DIR/envs" +install -d -m 700 "$STAGING_DIR/envs" +STAGING_READY=true # Capture each service's .env (secrets needed to restore); the repo is encrypted. while IFS= read -r -d '' envf; do svc="$(basename "$(dirname "$envf")")" - cp -a "$envf" "$STAGING_DIR/envs/$svc.env" + install -m 600 "$envf" "$STAGING_DIR/envs/$svc.env" done < <(find "$PI_DIR" -mindepth 2 -maxdepth 2 -name .env -print0) # --- backup ----------------------------------------------------------------- @@ -151,24 +187,28 @@ SIZE_BYTES="$(restic stats --mode raw-data --json | jq -r '.total_size // 0')" # --- second copy (3-2-1-ish) ----------------------------------------------- if [[ -n "$SECOND_RESTIC_REPOSITORY" ]]; then - if ! restic -r "$SECOND_RESTIC_REPOSITORY" cat config >/dev/null 2>&1; then - log "initializing second repo at $SECOND_RESTIC_REPOSITORY" - restic -r "$SECOND_RESTIC_REPOSITORY" init --copy-chunker-params --from-repo "$RESTIC_REPOSITORY" + sync_second_repo() { + restic -r "$SECOND_RESTIC_REPOSITORY" cat config >/dev/null 2>&1 || return 1 + log "copying snapshots → $SECOND_RESTIC_REPOSITORY" + restic -r "$SECOND_RESTIC_REPOSITORY" copy --from-repo "$RESTIC_REPOSITORY" || return 1 + restic -r "$SECOND_RESTIC_REPOSITORY" forget --host "$HOSTTAG" \ + --keep-daily "$RETENTION_KEEP_DAILY" \ + --keep-weekly "$RETENTION_KEEP_WEEKLY" \ + --keep-monthly "$RETENTION_KEEP_MONTHLY" \ + --prune || return 1 + } + + if ! sync_second_repo; then + SECOND_COPY_STATUS=" · second copy incomplete" + log "second repo unavailable or an operation failed — primary backup remains successful" fi - log "copying snapshots → $SECOND_RESTIC_REPOSITORY" - restic -r "$SECOND_RESTIC_REPOSITORY" copy --from-repo "$RESTIC_REPOSITORY" - restic -r "$SECOND_RESTIC_REPOSITORY" forget --host "$HOSTTAG" \ - --keep-daily "$RETENTION_KEEP_DAILY" \ - --keep-weekly "$RETENTION_KEEP_WEEKLY" \ - --keep-monthly "$RETENTION_KEEP_MONTHLY" \ - --prune fi # --- done ------------------------------------------------------------------- DURATION=$((SECONDS - START)) -STATUS=success write_status success +STATUS=success human="$(numfmt --to=iec "$SIZE_BYTES" 2>/dev/null || printf '%s bytes' "$SIZE_BYTES")" -notify "Pi backup OK" default floppy_disk "snapshot $SNAPSHOT_ID · $human · ${DURATION}s" -kuma_push up "snapshot $SNAPSHOT_ID · $human · ${DURATION}s" "$((DURATION * 1000))" -log "done: snapshot $SNAPSHOT_ID, $human, ${DURATION}s" +notify "Pi backup OK" default floppy_disk "snapshot $SNAPSHOT_ID · $human · ${DURATION}s${SECOND_COPY_STATUS}" +kuma_push up "snapshot $SNAPSHOT_ID · $human · ${DURATION}s${SECOND_COPY_STATUS}" "$((DURATION * 1000))" +log "done: snapshot $SNAPSHOT_ID, $human, ${DURATION}s${SECOND_COPY_STATUS}" diff --git a/linux-pi/backup/pi-backup-failure.service b/linux-pi/backup/pi-backup-failure.service deleted file mode 100644 index 56d9f83..0000000 --- a/linux-pi/backup/pi-backup-failure.service +++ /dev/null @@ -1,8 +0,0 @@ -[Unit] -Description=Alert on Pi backup failure - -[Service] -Type=oneshot -# Fires via pi-backup.service's OnFailure= — records a failed status and pings -# ntfy even if backup.sh died before its own error handler could run. -ExecStart=/home/ollie/github/Computer-Setup/linux-pi/backup/backup.sh notify-failure diff --git a/linux-pi/backup/pi-backup-failure.service.template b/linux-pi/backup/pi-backup-failure.service.template new file mode 100644 index 0000000..5fc2a4e --- /dev/null +++ b/linux-pi/backup/pi-backup-failure.service.template @@ -0,0 +1,7 @@ +[Unit] +Description=Alert on Pi backup failure + +[Service] +Type=oneshot +EnvironmentFile=-@BACKUP_ENV@ +ExecStart=/usr/bin/env bash @BACKUP_SCRIPT@ notify-failure diff --git a/linux-pi/backup/pi-backup.service b/linux-pi/backup/pi-backup.service.template similarity index 50% rename from linux-pi/backup/pi-backup.service rename to linux-pi/backup/pi-backup.service.template index f7498e8..4f69d0d 100644 --- a/linux-pi/backup/pi-backup.service +++ b/linux-pi/backup/pi-backup.service.template @@ -1,13 +1,14 @@ [Unit] Description=Pi backup (restic → main server over SSH) +Wants=network-online.target After=network-online.target docker.service OnFailure=pi-backup-failure.service [Service] Type=oneshot -ExecStart=/home/ollie/github/Computer-Setup/linux-pi/backup/backup.sh -# Runs as root to read all service data dirs and drive docker. -# Give restic a HOME so it can use its local cache (root has none under systemd). +ExecStart=/usr/bin/env bash @BACKUP_SCRIPT@ Environment=HOME=/root +TimeoutStartSec=2h +PrivateTmp=yes Nice=10 IOSchedulingClass=idle diff --git a/linux-pi/backup/pi-backup.timer b/linux-pi/backup/pi-backup.timer index 392d6c1..3a5927b 100644 --- a/linux-pi/backup/pi-backup.timer +++ b/linux-pi/backup/pi-backup.timer @@ -2,7 +2,7 @@ Description=Nightly Pi backup [Timer] -# 03:45 — 15 min after the main server's 03:30 backup to avoid contention. +# 03:45 — offset from the main server's 03:30 start; long runs can still overlap. OnCalendar=*-*-* 03:45:00 # Catch up if the machine was off at the scheduled time. Persistent=true diff --git a/linux-pi/backup/setup.sh b/linux-pi/backup/setup.sh new file mode 100755 index 0000000..28feeaa --- /dev/null +++ b/linux-pi/backup/setup.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" +DRY_RUN=false + +case "${1:-}" in + "") ;; + --dry-run) DRY_RUN=true ;; + *) printf 'error: unknown argument: %s\n' "$1" >&2; exit 1 ;; +esac + +if ! [[ "$SCRIPT_DIR" =~ ^/[[:alnum:]_./-]+$ ]]; then + printf 'error: unsupported character in checkout path: %s\n' "$SCRIPT_DIR" >&2 + exit 1 +fi +if [[ "$DRY_RUN" == false ]]; then + [[ $EUID -eq 0 ]] || { printf 'error: run with sudo: sudo bash %s\n' "$0" >&2; exit 1; } + [[ -f "$SCRIPT_DIR/.env" ]] || { printf 'error: create %s/.env before installing\n' "$SCRIPT_DIR" >&2; exit 1; } + chmod 600 "$SCRIPT_DIR/.env" +fi + +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT + +render_unit() { + local src="$1" dest="$2" + sed \ + -e "s|@BACKUP_SCRIPT@|$SCRIPT_DIR/backup.sh|g" \ + -e "s|@BACKUP_ENV@|$SCRIPT_DIR/.env|g" \ + "$src" > "$tmp/$dest" + if [[ "$DRY_RUN" == true ]]; then + printf '[dry-run] install %s as /etc/systemd/system/%s\n' "$src" "$dest" + else + install -o root -g root -m 644 "$tmp/$dest" "/etc/systemd/system/$dest" + fi +} + +render_unit "$SCRIPT_DIR/pi-backup.service.template" pi-backup.service +render_unit "$SCRIPT_DIR/pi-backup-failure.service.template" pi-backup-failure.service +if command -v systemd-analyze >/dev/null; then + systemd-analyze verify "$tmp/pi-backup.service" "$tmp/pi-backup-failure.service" "$SCRIPT_DIR/pi-backup.timer" +fi + +if [[ "$DRY_RUN" == true ]]; then + printf '[dry-run] install %s as /etc/systemd/system/pi-backup.timer\n' "$SCRIPT_DIR/pi-backup.timer" + printf '[dry-run] systemctl daemon-reload\n' + printf '[dry-run] systemctl enable --now pi-backup.timer\n' +else + install -o root -g root -m 644 "$SCRIPT_DIR/pi-backup.timer" /etc/systemd/system/pi-backup.timer + systemctl daemon-reload + systemctl enable --now pi-backup.timer + printf 'Installed and enabled pi-backup.timer\n' +fi diff --git a/linux-pi/backup/test-backup-failure.sh b/linux-pi/backup/test-backup-failure.sh new file mode 100755 index 0000000..4dc612e --- /dev/null +++ b/linux-pi/backup/test-backup-failure.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT + +run_notifier() { + local ntfy_url="${1:-}" + local kuma_push_url="${2:-}" + + STATUS_JSON="$tmp/status.json" \ + NTFY_URL="$ntfy_url" \ + NTFY_TOPIC=pi-backup-test \ + NTFY_TOKEN='' \ + KUMA_PUSH_URL="$kuma_push_url" \ + bash "$SCRIPT_DIR/backup.sh" notify-failure +} + +run_notifier +[[ "$(jq -r '.status' "$tmp/status.json")" == failed ]] + +jq -n '{status:"success", last_run:"previous"}' >"$tmp/status.json" +run_notifier +[[ "$(jq -r '.status' "$tmp/status.json")" == failed ]] + +jq -n '{status:"failed", last_run:"detailed", duration_seconds:42, notifier_pending:true}' >"$tmp/status.json" +run_notifier +[[ "$(jq -r '.last_run' "$tmp/status.json")" == detailed ]] +[[ "$(jq -r '.duration_seconds' "$tmp/status.json")" == 42 ]] +[[ "$(jq -r '.notifier_pending' "$tmp/status.json")" == false ]] + +jq -n '{status:"failed", last_run:"stale", duration_seconds:42, notifier_pending:false}' >"$tmp/status.json" +run_notifier +[[ "$(jq -r '.status' "$tmp/status.json")" == failed ]] +[[ "$(jq -r '.last_run' "$tmp/status.json")" != stale ]] + +jq -n '{status:"running", last_run:"interrupted", notifier_pending:false}' >"$tmp/status.json" +run_notifier +[[ "$(jq -r '.status' "$tmp/status.json")" == failed ]] +[[ "$(jq -r '.last_run' "$tmp/status.json")" != interrupted ]] + +cp "$SCRIPT_DIR/backup.sh" "$tmp/backup.sh" +if STATUS_JSON="$tmp/status.json" NTFY_URL=not-a-url bash "$tmp/backup.sh" >/dev/null 2>&1; then + printf 'FAIL: invalid early notification input unexpectedly succeeded\n' >&2 + exit 1 +fi +[[ "$(jq -r '.status' "$tmp/status.json")" == failed ]] +[[ "$(jq -r '.notifier_pending' "$tmp/status.json")" == true ]] +run_notifier +[[ "$(jq -r '.notifier_pending' "$tmp/status.json")" == false ]] + +rm -f "$tmp/status.json" +run_notifier not-a-url also-not-a-url +[[ "$(jq -r '.status' "$tmp/status.json")" == failed ]] + +printf 'backup failure notifier tests: PASSED\n' diff --git a/linux-pi/cups/.env.example b/linux-pi/cups/.env.example index 91aa589..87652c1 100644 --- a/linux-pi/cups/.env.example +++ b/linux-pi/cups/.env.example @@ -1,3 +1,23 @@ -# Copy to .env and set the value. Tailscale auth for the cups-ts sidecar -# (same OAuth client secret as the other sidecars; needs tag:container). +# Copy to .env and set the values. .env is gitignored (repo is public). + +# --- Tailscale sidecar (docker-compose) --- +# Tailscale auth for the cups-ts sidecar (same OAuth client secret as the other +# sidecars; needs tag:container). TS_AUTHKEY= + +# --- Host cupsd.conf access policy (applied by setup.sh) --- +# These are the private, machine-specific values kept out of the tracked config. +# setup.sh renders them into /etc/cups/cupsd.conf. + +# Space-separated Host names cupsd will answer to (ServerAlias). Include: +# - the tailnet front-door name (what the ts sidecar proxies as), and +# - the LAN/Bonjour names family devices print to (hostname + hostname.local). +CUPS_SERVER_ALIAS="cups-pi..ts.net .local" + +# Must exactly match the subnet pinned in docker-compose.yml. setup.sh adds it +# to both the print and admin access lists. +CUPS_SIDECAR_SUBNET="172.21.0.0/16" + +# Canonical private IPv4 CIDR for the family LAN/WLAN. setup.sh adds it only to +# the root print location; CUPS administration remains unavailable from the LAN. +CUPS_LAN_SUBNET="/24" diff --git a/linux-pi/cups/README.md b/linux-pi/cups/README.md new file mode 100644 index 0000000..b3d9e69 --- /dev/null +++ b/linux-pi/cups/README.md @@ -0,0 +1,110 @@ +# cups/ — Pi print server + HTTPS front door + +The Pi hosts a USB printer through host CUPS on port 631. Family devices print +directly over the home LAN/WLAN, while the `cups-ts` sidecar provides the +operator's remote HTTPS route over Tailscale. + +## Access policy + +`setup.sh` renders three exact access blocks in `/etc/cups/cupsd.conf`: + +- `` permits `localhost`, the pinned sidecar subnet, and the home + LAN/WLAN subnet. This supports family printing and the HTTPS sidecar. +- `` and every descendant admin location permit only + `localhost` and the sidecar subnet. Each is normalized to `AuthType Default` + with `Require user @SYSTEM`. +- Sources outside those ranges are denied. The renderer rejects wildcard + aliases, open networks, non-private networks, non-canonical CIDRs, malformed + hostnames, duplicate aliases, and control-character injection. + +The renderer replaces all TCP `Listen` and `Port` directives with exactly one +`Port 631`, while preserving Unix-socket listeners such as +`Listen /run/cups/cups.sock`. It also replaces every active `ServerAlias` with +the explicit hostnames from `.env`. `ServerAlias *`, `Allow all`, and +internet-wide CIDRs are never accepted. + +## Private configuration + +Copy `.env.example` to the gitignored `.env` and set: + +- `CUPS_SERVER_ALIAS`: lowercase canonical hostnames separated by single + spaces, including the exact Tailscale name and the LAN/Bonjour names. +- `CUPS_SIDECAR_SUBNET`: the bridge CIDR pinned in `docker-compose.yml`. The + setup script requires the exact pin. +- `CUPS_LAN_SUBNET`: the canonical private CIDR used by family LAN/WLAN + clients. It is allowed for printing but not administration. + +Set `.env` to mode `0600`; the installer refuses to source a more broadly +readable file because it also contains the sidecar credential. + +If upgrading from the old policy, replace `CUPS_ALLOW_FROM` with the two subnet +variables above. No private hostname, tailnet name, or LAN address belongs in a +tracked file or LLM transcript. + +## Reviewed deployment + +The ordinary dry-run is intentionally redacted: + +```bash +cd linux-pi/cups +chmod 600 .env +bash setup.sh --dry-run +``` + +It renders and runs `cupsd -t`, then reports only whether a change is pending. +It never prints the rendered configuration, aliases, or CIDRs. + +Prepare the exact candidate and diff as root-only artifacts: + +```bash +sudo bash setup.sh --prepare-review +``` + +The command prints source and candidate hashes but no private values. From a +separate trusted human terminal—not an LLM-controlled terminal—inspect: + +```bash +sudo less /var/lib/cups-policy-review/pending.diff +sudo less /var/lib/cups-policy-review/candidate.conf +``` + +Confirm that the print block contains localhost, the exact pinned sidecar +subnet, and the exact family LAN/WLAN subnet; every admin block must contain +only localhost and the sidecar subnet plus the exact system-user authentication +policy. Confirm that no broad access rule survives. + +Apply exactly what was reviewed by passing both hashes printed by the prepare +step: + +```bash +sudo bash setup.sh --apply-reviewed +``` + +The apply step refuses stale or altered artifacts, validates the candidate +again, backs up the current configuration, disables `cups.socket`, enables and +restarts `cups.service`, and probes CUPS locally with a valid Host header. A +socket, restart, or probe failure restores the previous configuration. Review +artifacts use root ownership with directory mode `0700` and file mode `0600`; +they are removed after a successful apply. + +## Live verification + +After applying, verify without printing private values into an LLM transcript: + +1. `cups.service` is enabled and active, and `cups.socket` is disabled. +2. A family device on LAN/WLAN can discover and print a test page. +3. An HTTP request from the `cups-ts` container to the host CUPS service works. +4. The sidecar HTTPS endpoint works from an authorized tailnet client. +5. A client outside both approved networks is denied. +6. Administrative routes require authentication and are unavailable to an + ordinary LAN-only client. + +The sidecar bridge subnet is pinned in `docker-compose.yml`. If that pin ever +changes, update `.env`, prepare a new review, and apply the newly reviewed +candidate. + +### macOS “Hold for authentication” + +A Mac can display “Hold for authentication” when CUPS actually rejected the +request because of a Host-header or source-network mismatch. Correct the policy +and resume or recreate the job; macOS does not always retry it automatically. diff --git a/linux-pi/cups/docker-compose.yml b/linux-pi/cups/docker-compose.yml index da111ad..57a02a2 100644 --- a/linux-pi/cups/docker-compose.yml +++ b/linux-pi/cups/docker-compose.yml @@ -3,9 +3,8 @@ services: # Decoupled sidecar, proxying :443 -> host.docker.internal:631. # https://cups-pi..ts.net. # - # NOTE: CUPS validates the Host header. If the admin pages 400/"Bad Request" - # through the proxy, add `ServerAlias *` (or the cups-pi hostname) to cupsd.conf - # on the Pi and restart cups. Read-only pages usually work without it. + # cupsd must listen on the Docker bridge and allow its subnet; see ../README.md. + # Keep the exact cups-pi tailnet name in ServerAlias rather than using `*`. cups-ts: image: tailscale/tailscale:latest container_name: cups-ts @@ -28,3 +27,11 @@ services: - 9.9.9.10 - 1.1.1.1 restart: unless-stopped + +# Pin the bridge subnet so the host cupsd.conf `Allow ` rule can never drift +# if this network is recreated. Must match the subnet allowed in /etc/cups/cupsd.conf. +networks: + default: + ipam: + config: + - subnet: 172.21.0.0/16 diff --git a/linux-pi/cups/setup.sh b/linux-pi/cups/setup.sh new file mode 100755 index 0000000..f935df5 --- /dev/null +++ b/linux-pi/cups/setup.sh @@ -0,0 +1,435 @@ +#!/usr/bin/env bash +set -euo pipefail +umask 077 + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" +CUPSD_CONF="${CUPSD_CONF:-/etc/cups/cupsd.conf}" +CUPS_REVIEW_DIR="${CUPS_REVIEW_DIR:-/var/lib/cups-policy-review}" +PINNED_SIDECAR_SUBNET="172.21.0.0/16" + +usage() { + printf 'usage: bash setup.sh --dry-run | --prepare-review | --apply-reviewed \n' >&2 +} + +fail() { + printf 'error: %s\n' "$1" >&2 + exit 1 +} + +require_command() { + command -v "$1" >/dev/null 2>&1 || fail "required command not found: $1" +} + +hash_file() { + local path="$1" + + if command -v sha256sum >/dev/null 2>&1; then + sha256sum "$path" | awk '{print $1}' + elif command -v shasum >/dev/null 2>&1; then + shasum -a 256 "$path" | awk '{print $1}' + else + fail 'sha256sum or shasum is required' + fi +} + +stat_uid() { + local path="$1" + + stat -c '%u' "$path" 2>/dev/null || stat -f '%u' "$path" +} + +stat_mode() { + local path="$1" + + stat -c '%a' "$path" 2>/dev/null || stat -f '%Lp' "$path" +} + +require_root() { + [[ "$(id -u)" == 0 ]] || fail 'run this operation with sudo' +} + +validate_hostname() { + local hostname="$1" + local label + local -a labels + + [[ ${#hostname} -le 253 ]] || return 1 + [[ "$hostname" =~ ^[a-z0-9.-]+$ ]] || return 1 + [[ "$hostname" != .* && "$hostname" != *. && "$hostname" != *..* ]] || return 1 + + IFS='.' read -r -a labels <<< "$hostname" + for label in "${labels[@]}"; do + [[ ${#label} -le 63 ]] || return 1 + [[ "$label" =~ ^[a-z0-9]([a-z0-9-]*[a-z0-9])?$ ]] || return 1 + done +} + +validate_private_cidr() { + local cidr="$1" + local ip prefix octet first second ip_int mask + local -a octets + + [[ "$cidr" =~ ^([0-9]{1,3}\.){3}[0-9]{1,3}/([0-9]{1,2})$ ]] || return 1 + ip="${cidr%/*}" + prefix="${cidr#*/}" + [[ "$prefix" == "$((10#$prefix))" ]] || return 1 + (( prefix >= 16 && prefix <= 30 )) || return 1 + + IFS='.' read -r -a octets <<< "$ip" + for octet in "${octets[@]}"; do + (( 10#$octet <= 255 )) || return 1 + [[ "$octet" == "$((10#$octet))" ]] || return 1 + done + + first=$((10#${octets[0]})) + second=$((10#${octets[1]})) + if ! (( first == 10 || (first == 172 && second >= 16 && second <= 31) || (first == 192 && second == 168) )); then + return 1 + fi + + ip_int=$(( (first << 24) | (second << 16) | (10#${octets[2]} << 8) | 10#${octets[3]} )) + mask=$(( (0xFFFFFFFF << (32 - prefix)) & 0xFFFFFFFF )) + (( (ip_int & mask) == ip_int )) +} + +load_policy() { + local env_file="$SCRIPT_DIR/.env" + local alias_name seen="" + local -a aliases + + [[ -f "$env_file" && ! -L "$env_file" ]] || fail 'create a regular .env from .env.example first' + [[ "$(stat_mode "$env_file")" == 600 ]] || fail 'set .env permissions to 0600 before use' + set -a + # shellcheck disable=SC1090 + source "$env_file" + set +a + + : "${CUPS_SERVER_ALIAS:?set CUPS_SERVER_ALIAS in .env}" + : "${CUPS_LAN_SUBNET:?set CUPS_LAN_SUBNET in .env}" + CUPS_SIDECAR_SUBNET="${CUPS_SIDECAR_SUBNET:-$PINNED_SIDECAR_SUBNET}" + + [[ "$CUPS_SERVER_ALIAS" =~ ^[a-z0-9.-]+(\ [a-z0-9.-]+)*$ ]] \ + || fail 'CUPS_SERVER_ALIAS must contain only canonical lowercase hostnames separated by single spaces' + read -r -a aliases <<< "$CUPS_SERVER_ALIAS" + (( ${#aliases[@]} > 0 )) || fail 'CUPS_SERVER_ALIAS must not be empty' + + for alias_name in "${aliases[@]}"; do + validate_hostname "$alias_name" || fail 'CUPS_SERVER_ALIAS contains an invalid hostname' + [[ " $seen " != *" $alias_name "* ]] || fail 'CUPS_SERVER_ALIAS contains a duplicate hostname' + seen+="${seen:+ }$alias_name" + done + + validate_private_cidr "$CUPS_SIDECAR_SUBNET" || fail 'CUPS_SIDECAR_SUBNET must be a canonical private IPv4 CIDR' + [[ "$CUPS_SIDECAR_SUBNET" == "$PINNED_SIDECAR_SUBNET" ]] \ + || fail 'CUPS_SIDECAR_SUBNET must match the Compose network pin' + validate_private_cidr "$CUPS_LAN_SUBNET" || fail 'CUPS_LAN_SUBNET must be a canonical private IPv4 CIDR between /16 and /30' + [[ ! "$CUPS_LAN_SUBNET" =~ ^172\.21\. ]] \ + || fail 'CUPS_LAN_SUBNET must not overlap the sidecar subnet' + + CUPS_PRINT_ALLOW_FROM="localhost $CUPS_SIDECAR_SUBNET $CUPS_LAN_SUBNET" + CUPS_ADMIN_ALLOW_FROM="localhost $CUPS_SIDECAR_SUBNET" +} + +location_count() { + local path="$1" + local target="$2" + + awk -v target="$target" ' + $0 ~ "^[[:space:]]*[[:space:]]*$" { count++ } + END { print count + 0 } + ' "$path" +} + +validate_source_shape() { + local path="$1" + + [[ "$(location_count "$path" '\\/')" == 1 ]] \ + || fail 'cupsd.conf must contain exactly one root Location block' + [[ "$(location_count "$path" '\\/admin')" == 1 ]] \ + || fail 'cupsd.conf must contain exactly one admin Location block' + [[ "$(location_count "$path" '\\/admin\\/conf')" == 1 ]] \ + || fail 'cupsd.conf must contain exactly one admin/conf Location block' +} + +render_candidate() { + local source="$1" + local destination="$2" + + validate_source_shape "$source" + awk \ + -v aliases="$CUPS_SERVER_ALIAS" \ + -v print_allow="$CUPS_PRINT_ALLOW_FROM" \ + -v admin_allow="$CUPS_ADMIN_ALLOW_FROM" ' + function emit_aliases( count, values, idx) { + count = split(aliases, values, /[ ]+/) + for (idx = 1; idx <= count; idx++) print "ServerAlias " values[idx] + } + function emit_access(values, count, sources, idx) { + print " Order allow,deny" + count = split(values, sources, /[ ]+/) + for (idx = 1; idx <= count; idx++) print " Allow " sources[idx] + } + function emit_admin_policy() { + print " AuthType Default" + print " Require user @SYSTEM" + emit_access(admin_allow) + } + NR == 1 { + if (!network_seen) { + print "Port 631" + network_seen = 1 + } + if (!alias_seen) { + emit_aliases() + alias_seen = 1 + } + } + /^[[:space:]]*ServerAlias([[:space:]]|$)/ { + if (!alias_replaced) { + if (!alias_seen) emit_aliases() + alias_replaced = 1 + } + next + } + /^[[:space:]]*Port([[:space:]]|$)/ { + if (!port_replaced) { + if (!network_seen) print "Port 631" + port_replaced = 1 + } + next + } + /^[[:space:]]*Listen[[:space:]]+\// { print; next } + /^[[:space:]]*Listen([[:space:]]|$)/ { + if (!port_replaced) { + if (!network_seen) print "Port 631" + port_replaced = 1 + } + next + } + /^[[:space:]]*[[:space:]]*$/ { + block = "print" + print + next + } + /^[[:space:]]*[:space:]]+)?>[[:space:]]*$/ { + block = "admin" + print + next + } + block != "" && /^[[:space:]]*<\/Location>[[:space:]]*$/ { + if (block == "print") emit_access(print_allow) + else emit_admin_policy() + block = "" + print + next + } + block == "admin" && /^[[:space:]]*(AuthType|Require)([[:space:]]|$)/ { next } + block != "" && /^[[:space:]]*(Allow|Deny|Order)([[:space:]]|$)/ { next } + { print } + ' "$source" > "$destination" +} + +validate_rendered_policy() { + local path="$1" + local network_count + + network_count="$(awk ' + /^[[:space:]]*Port([[:space:]]|$)/ { count++ } + /^[[:space:]]*Listen([[:space:]]|$)/ && $0 !~ /^[[:space:]]*Listen[[:space:]]+\// { count++ } + END { print count + 0 } + ' "$path")" + [[ "$network_count" == 1 ]] || fail 'rendered policy must contain exactly one TCP listener' + [[ "$(grep -Ec '^[[:space:]]*Port[[:space:]]+631[[:space:]]*$' "$path" || true)" == 1 ]] \ + || fail 'rendered policy must contain exactly one Port 631 directive' + ! grep -Eqi '^[[:space:]]*ServerAlias[[:space:]]+\*([[:space:]]|$)' "$path" \ + || fail 'rendered policy contains a wildcard ServerAlias' + ! grep -Eqi '^[[:space:]]*Allow[[:space:]]+(from[[:space:]]+)?(all|0\.0\.0\.0/0|::/0)([[:space:]]|$)' "$path" \ + || fail 'rendered policy contains an open access rule' + awk ' + /^[[:space:]]*[:space:]]+)?>[[:space:]]*$/ { + admin = 1 + auth = 0 + require_system = 0 + next + } + admin && /^[[:space:]]*AuthType[[:space:]]+Default[[:space:]]*$/ { auth++ } + admin && /^[[:space:]]*Require[[:space:]]+user[[:space:]]+@SYSTEM[[:space:]]*$/ { require_system++ } + admin && /^[[:space:]]*<\/Location>[[:space:]]*$/ { + if (auth != 1 || require_system != 1) invalid = 1 + admin = 0 + } + END { exit invalid } + ' "$path" || fail 'every admin Location must require the system user policy' +} + +find_cupsd() { + local candidate + + if [[ -n "${CUPSD_BIN:-}" ]]; then + [[ -x "$CUPSD_BIN" ]] || fail 'configured cupsd binary is not executable' + return + fi + + CUPSD_BIN="$(command -v cupsd || true)" + for candidate in /usr/sbin/cupsd /sbin/cupsd; do + [[ -z "$CUPSD_BIN" && -x "$candidate" ]] && CUPSD_BIN="$candidate" + done + [[ -n "$CUPSD_BIN" ]] || fail 'cupsd is required to validate the candidate' +} + +validate_candidate() { + local path="$1" + + validate_rendered_policy "$path" + find_cupsd + "$CUPSD_BIN" -t -c "$path" >/dev/null 2>&1 \ + || fail 'cupsd rejected the rendered configuration' +} + +render_to_temp() { + local destination="$1" + + [[ -f "$CUPSD_CONF" && ! -L "$CUPSD_CONF" ]] || fail 'cupsd.conf must be a regular file' + load_policy + render_candidate "$CUPSD_CONF" "$destination" + validate_candidate "$destination" +} + +dry_run() { + local candidate + + candidate="$(mktemp)" + trap 'rm -f "$candidate"' RETURN + render_to_temp "$candidate" + if cmp -s "$CUPSD_CONF" "$candidate"; then + printf 'validation passed; no policy changes are pending\n' + else + printf 'validation passed; policy changes are pending (sensitive diff withheld)\n' + fi +} + +prepare_review() { + local candidate diff_file manifest source_hash source_hash_after candidate_hash diff_status=0 + + require_root + candidate="$(mktemp)" + diff_file="$(mktemp)" + manifest="$(mktemp)" + trap 'rm -f "$candidate" "$diff_file" "$manifest"' RETURN + [[ -f "$CUPSD_CONF" && ! -L "$CUPSD_CONF" ]] || fail 'cupsd.conf must be a regular file' + source_hash="$(hash_file "$CUPSD_CONF")" + render_to_temp "$candidate" + source_hash_after="$(hash_file "$CUPSD_CONF")" + [[ "$source_hash" == "$source_hash_after" ]] || fail 'cupsd.conf changed while preparing the review' + candidate_hash="$(hash_file "$candidate")" + + diff -u --label current --label candidate "$CUPSD_CONF" "$candidate" > "$diff_file" || diff_status=$? + (( diff_status <= 1 )) || fail 'could not create the review diff' + printf 'source %s\ncandidate %s\n' "$source_hash" "$candidate_hash" > "$manifest" + + [[ ! -L "$CUPS_REVIEW_DIR" ]] || fail 'review directory must not be a symbolic link' + install -d -o root -g root -m 700 "$CUPS_REVIEW_DIR" + install -o root -g root -m 600 "$candidate" "$CUPS_REVIEW_DIR/candidate.conf" + install -o root -g root -m 600 "$diff_file" "$CUPS_REVIEW_DIR/pending.diff" + install -o root -g root -m 600 "$manifest" "$CUPS_REVIEW_DIR/manifest" + + printf 'review artifacts prepared with root-only permissions\n' + printf 'source hash: %s\n' "$source_hash" + printf 'candidate hash: %s\n' "$candidate_hash" + printf 'inspect the protected diff outside the LLM transcript, then apply both reviewed hashes\n' +} + +verify_review_artifact() { + local path="$1" + local expected_mode="$2" + + [[ -f "$path" && ! -L "$path" ]] || fail 'review artifact is missing or unsafe' + [[ "$(stat_uid "$path")" == 0 ]] || fail 'review artifacts must be owned by root' + [[ "$(stat_mode "$path")" == "$expected_mode" ]] || fail 'review artifact permissions are unsafe' +} + +rollback_config() { + local backup="$1" + + printf 'apply failed; restoring the previous CUPS configuration\n' >&2 + install -o root -g root -m 640 "$backup" "$CUPSD_CONF" + systemctl disable --now cups.socket >/dev/null 2>&1 || true + systemctl enable cups.service >/dev/null 2>&1 || true + systemctl restart cups.service >/dev/null 2>&1 || true +} + +apply_reviewed() { + local expected_source_hash="$1" + local expected_candidate_hash="$2" + local candidate="$CUPS_REVIEW_DIR/candidate.conf" + local manifest="$CUPS_REVIEW_DIR/manifest" + local recorded_source_hash recorded_candidate_hash current_source_hash backup + local socket_exists=false + + require_root + [[ "$expected_source_hash" =~ ^[a-f0-9]{64}$ ]] || fail 'source hash must be 64 lowercase hexadecimal characters' + [[ "$expected_candidate_hash" =~ ^[a-f0-9]{64}$ ]] || fail 'candidate hash must be 64 lowercase hexadecimal characters' + [[ -d "$CUPS_REVIEW_DIR" && ! -L "$CUPS_REVIEW_DIR" ]] || fail 'review directory is missing or unsafe' + [[ "$(stat_uid "$CUPS_REVIEW_DIR")" == 0 && "$(stat_mode "$CUPS_REVIEW_DIR")" == 700 ]] \ + || fail 'review directory ownership or permissions are unsafe' + verify_review_artifact "$candidate" 600 + verify_review_artifact "$manifest" 600 + verify_review_artifact "$CUPS_REVIEW_DIR/pending.diff" 600 + [[ -f "$CUPSD_CONF" && ! -L "$CUPSD_CONF" ]] || fail 'cupsd.conf must be a regular file' + + recorded_source_hash="$(awk '$1 == "source" { print $2 }' "$manifest")" + recorded_candidate_hash="$(awk '$1 == "candidate" { print $2 }' "$manifest")" + [[ "$recorded_source_hash" =~ ^[a-f0-9]{64}$ && "$recorded_candidate_hash" =~ ^[a-f0-9]{64}$ ]] \ + || fail 'review manifest is invalid' + [[ "$expected_source_hash" == "$recorded_source_hash" ]] || fail 'provided source hash does not match the reviewed source' + [[ "$expected_candidate_hash" == "$recorded_candidate_hash" ]] || fail 'provided candidate hash does not match the reviewed candidate' + current_source_hash="$(hash_file "$CUPSD_CONF")" + [[ "$current_source_hash" == "$recorded_source_hash" ]] || fail 'cupsd.conf changed after review; prepare a new review' + [[ "$(hash_file "$candidate")" == "$recorded_candidate_hash" ]] || fail 'reviewed candidate changed after review' + validate_candidate "$candidate" + + require_command systemctl + require_command curl + backup="$CUPSD_CONF.bak.$(date +%Y%m%d%H%M%S)" + [[ ! -e "$backup" ]] || fail 'backup destination already exists' + install -o root -g root -m 600 "$CUPSD_CONF" "$backup" + install -o root -g root -m 640 "$candidate" "$CUPSD_CONF" + + if systemctl cat cups.socket >/dev/null 2>&1; then + socket_exists=true + fi + if [[ "$socket_exists" == true ]] && ! systemctl disable --now cups.socket >/dev/null 2>&1; then + rollback_config "$backup" + fail 'could not disable CUPS socket activation' + fi + if ! systemctl enable cups.service >/dev/null 2>&1 \ + || ! systemctl restart cups.service >/dev/null 2>&1 \ + || ! curl --fail --silent --show-error --max-time 10 --header 'Host: localhost' http://127.0.0.1:631/ >/dev/null 2>&1; then + rollback_config "$backup" + fail 'CUPS restart or local probe failed' + fi + + rm -f "$candidate" "$CUPS_REVIEW_DIR/pending.diff" "$manifest" + rmdir "$CUPS_REVIEW_DIR" 2>/dev/null || true + printf 'reviewed policy applied; cups.service is active and the local probe passed\n' +} + +mode="${1:-}" +case "$mode" in + --dry-run) + (( $# == 1 )) || { usage; exit 1; } + dry_run + ;; + --prepare-review) + (( $# == 1 )) || { usage; exit 1; } + prepare_review + ;; + --apply-reviewed) + (( $# == 3 )) || { usage; exit 1; } + apply_reviewed "$2" "$3" + ;; + *) + usage + exit 1 + ;; +esac diff --git a/linux-pi/cups/test-setup.sh b/linux-pi/cups/test-setup.sh new file mode 100755 index 0000000..a4b9ad8 --- /dev/null +++ b/linux-pi/cups/test-setup.sh @@ -0,0 +1,247 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT + +fail() { + printf 'FAIL: %s\n' "$1" >&2 + exit 1 +} + +assert_contains() { + local value="$1" + local expected="$2" + + [[ "$value" == *"$expected"* ]] || fail "expected output to contain: $expected" +} + +assert_not_contains() { + local value="$1" + local unexpected="$2" + + [[ "$value" != *"$unexpected"* ]] || fail "output exposed a protected test value" +} + +assert_count() { + local expected="$1" + local pattern="$2" + local path="$3" + local actual + + actual="$(grep -Ec "$pattern" "$path" || true)" + [[ "$actual" == "$expected" ]] || fail "expected $expected matches for $pattern, found $actual" +} + +write_env() { + local aliases="$1" + local sidecar="$2" + local lan="$3" + + printf 'CUPS_SERVER_ALIAS="%s"\nCUPS_SIDECAR_SUBNET="%s"\nCUPS_LAN_SUBNET="%s"\n' \ + "$aliases" "$sidecar" "$lan" > "$work/.env" + chmod 600 "$work/.env" +} + +write_fixture() { + local path="$1" + + printf '%s\n' \ + 'LogLevel warn' \ + 'Listen localhost:631' \ + 'Listen /run/cups/cups.sock' \ + 'Port 8631' \ + 'ServerAlias *' \ + '' \ + ' Order deny,allow' \ + ' Allow all' \ + '' \ + '' \ + ' Allow all' \ + '' \ + '' \ + ' AuthType Default' \ + ' Require user @SYSTEM' \ + ' Order deny,allow' \ + ' Allow all' \ + '' \ + '' \ + ' AuthType Default' \ + ' Require user @SYSTEM' \ + ' Allow all' \ + '' \ + 'Browsing On' > "$path" +} + +write_stub() { + local name="$1" + shift + + printf '%s\n' '#!/usr/bin/env bash' 'set -euo pipefail' "$@" > "$stubs/$name" + chmod +x "$stubs/$name" +} + +run_setup() { + PATH="$stubs:$PATH" \ + CUPSD_BIN="$stubs/cupsd" \ + CUPSD_CONF="$config" \ + CUPS_REVIEW_DIR="$review" \ + SYSTEMCTL_LOG="$systemctl_log" \ + bash "$work/setup.sh" "$@" +} + +run_invalid() { + local description="$1" + local output + + if output="$(run_setup --dry-run 2>&1)"; then + fail "invalid input accepted: $description" + fi + assert_not_contains "$output" 'printer.family.lan' + assert_not_contains "$output" '192.168.50.0/24' +} + +cp "$SCRIPT_DIR/setup.sh" "$work/setup.sh" +chmod +x "$work/setup.sh" +stubs="$work/stubs" +mkdir -p "$stubs" +systemctl_log="$work/systemctl.log" + +write_stub cupsd '[[ "${CUPSD_FAIL:-false}" != true ]]' +write_stub curl '[[ "${CURL_FAIL:-false}" != true ]]' +write_stub systemctl 'printf "%s\n" "$*" >> "$SYSTEMCTL_LOG"' 'exit 0' +write_stub id 'if [[ "${1:-}" == "-u" ]]; then printf "0\n"; else /usr/bin/id "$@"; fi' +write_stub stat \ + 'path=""' \ + 'for argument in "$@"; do path="$argument"; done' \ + 'if [[ "${1:-}" == "-c" && "${2:-}" == "%u" ]]; then printf "0\n"' \ + 'elif [[ "${1:-}" == "-c" && "${2:-}" == "%a" ]]; then if [[ -d "$path" ]]; then printf "700\n"; else printf "600\n"; fi' \ + 'else /usr/bin/stat "$@"; fi' +write_stub install \ + 'directory=false' \ + 'mode=""' \ + 'args=()' \ + 'while (( $# > 0 )); do' \ + ' case "$1" in' \ + ' -d) directory=true; shift ;;' \ + ' -m) mode="$2"; shift 2 ;;' \ + ' -o|-g) shift 2 ;;' \ + ' *) args+=("$1"); shift ;;' \ + ' esac' \ + 'done' \ + 'if [[ "$directory" == true ]]; then mkdir -p "${args[0]}"; target="${args[0]}"' \ + 'else cp "${args[0]}" "${args[1]}"; target="${args[1]}"' \ + 'fi' \ + '[[ -z "$mode" ]] || chmod "$mode" "$target"' + +config="$work/cupsd.conf" +review="$work/review" +write_fixture "$config" +write_env 'printer.family.lan printer printer.local' '172.21.0.0/16' '192.168.50.0/24' + +output="$(run_setup --dry-run 2>&1)" +assert_contains "$output" 'validation passed; policy changes are pending' +for protected_value in printer.family.lan printer.local 172.21.0.0/16 192.168.50.0/24; do + assert_not_contains "$output" "$protected_value" +done + +output="$(run_setup --prepare-review 2>&1)" +assert_contains "$output" 'review artifacts prepared with root-only permissions' +for protected_value in printer.family.lan printer.local 172.21.0.0/16 192.168.50.0/24; do + assert_not_contains "$output" "$protected_value" +done + +candidate="$review/candidate.conf" +[[ -f "$candidate" ]] || fail 'candidate was not prepared' +assert_count 1 '^[[:space:]]*Port[[:space:]]+631[[:space:]]*$' "$candidate" +assert_count 0 '^[[:space:]]*Port[[:space:]]+8631[[:space:]]*$' "$candidate" +assert_count 0 '^[[:space:]]*Listen[[:space:]]+localhost:631[[:space:]]*$' "$candidate" +assert_count 1 '^[[:space:]]*Listen[[:space:]]+/run/cups/cups.sock[[:space:]]*$' "$candidate" +assert_count 3 '^[[:space:]]*ServerAlias[[:space:]]+' "$candidate" +assert_count 0 '^[[:space:]]*Allow[[:space:]]+all[[:space:]]*$' "$candidate" +assert_count 3 '^[[:space:]]*Require[[:space:]]+user[[:space:]]+@SYSTEM[[:space:]]*$' "$candidate" + +print_block="$(awk '/^$/, /^<\/Location>$/' "$candidate")" +admin_block="$(awk '/^$/, /^<\/Location>$/' "$candidate")" +admin_conf_block="$(awk '/^$/, /^<\/Location>$/' "$candidate")" +admin_log_block="$(awk '/^$/, /^<\/Location>$/' "$candidate")" +assert_count 3 '^[[:space:]]*Allow[[:space:]]+' <(printf '%s\n' "$print_block") +assert_count 2 '^[[:space:]]*Allow[[:space:]]+' <(printf '%s\n' "$admin_block") +assert_count 2 '^[[:space:]]*Allow[[:space:]]+' <(printf '%s\n' "$admin_conf_block") +assert_count 2 '^[[:space:]]*Allow[[:space:]]+' <(printf '%s\n' "$admin_log_block") +assert_contains "$print_block" 'Allow 192.168.50.0/24' +assert_contains "$admin_block" 'AuthType Default' +assert_contains "$admin_block" 'Require user @SYSTEM' +assert_not_contains "$admin_block" '192.168.50.0/24' +assert_contains "$admin_conf_block" 'AuthType Default' +assert_contains "$admin_conf_block" 'Require user @SYSTEM' +assert_not_contains "$admin_conf_block" '192.168.50.0/24' +assert_contains "$admin_log_block" 'AuthType Default' +assert_contains "$admin_log_block" 'Require user @SYSTEM' +assert_not_contains "$admin_log_block" '192.168.50.0/24' + +expected="$work/expected.conf" +cp "$candidate" "$expected" +source_hash="$(awk '$1 == "source" { print $2 }' "$review/manifest")" +candidate_hash="$(awk '$1 == "candidate" { print $2 }' "$review/manifest")" +output="$(run_setup --apply-reviewed "$source_hash" "$candidate_hash" 2>&1)" +assert_contains "$output" 'reviewed policy applied' +cmp -s "$config" "$expected" || fail 'apply did not install the reviewed candidate' +assert_contains "$(<"$systemctl_log")" 'disable --now cups.socket' +assert_contains "$(<"$systemctl_log")" 'enable cups.service' +assert_contains "$(<"$systemctl_log")" 'restart cups.service' +for protected_value in printer.family.lan printer.local 172.21.0.0/16 192.168.50.0/24; do + assert_not_contains "$output" "$protected_value" +done + +output="$(run_setup --dry-run 2>&1)" +assert_contains "$output" 'validation passed; no policy changes are pending' +run_setup --prepare-review >/dev/null +cmp -s "$config" "$review/candidate.conf" || fail 'renderer is not idempotent' + +config="$work/cupsd-rollback.conf" +review="$work/review-rollback" +write_fixture "$config" +cp "$config" "$work/rollback-original.conf" +run_setup --prepare-review >/dev/null +source_hash="$(awk '$1 == "source" { print $2 }' "$review/manifest")" +candidate_hash="$(awk '$1 == "candidate" { print $2 }' "$review/manifest")" +if output="$(CURL_FAIL=true run_setup --apply-reviewed "$source_hash" "$candidate_hash" 2>&1)"; then + fail 'failed local probe did not abort apply' +fi +assert_contains "$output" 'restoring the previous CUPS configuration' +cmp -s "$config" "$work/rollback-original.conf" || fail 'rollback did not restore the original configuration' + +config="$work/cupsd-invalid.conf" +review="$work/review-invalid" +write_fixture "$config" + +printf '%s\n' \ + '' \ + ' Allow from all' \ + '' >> "$config" +write_env 'printer.family.lan printer printer.local' '172.21.0.0/16' '192.168.50.0/24' +run_invalid 'open access in an unmanaged specific Location' + +write_fixture "$config" + +write_env 'printer.family.lan printer printer.local' '172.21.0.0/16' '0.0.0.0/0' +run_invalid 'open LAN network' + +write_env 'printer.family.lan printer printer.local' '172.21.0.0/16' '192.168.50.7/24' +run_invalid 'non-canonical LAN network' + +write_env 'printer.family.lan printer printer.local' '172.22.0.0/16' '192.168.50.0/24' +run_invalid 'sidecar network that differs from the Compose pin' + +write_env 'printer.family.lan printer printer.local' '172.21.0.0/16' '172.21.4.0/24' +run_invalid 'LAN network that overlaps the sidecar network' + +write_env 'Printer.family.lan printer printer.local' '172.21.0.0/16' '192.168.50.0/24' +run_invalid 'non-canonical hostname' + +write_env $'printer.family.lan\nAllow all' '172.21.0.0/16' '192.168.50.0/24' +run_invalid 'hostname newline injection' + +printf 'PASS: CUPS policy renderer, privacy, review/apply, rollback, idempotency, and validation\n' diff --git a/linux-pi/homepage/.env.example b/linux-pi/homepage/.env.example index 6696157..11f0776 100644 --- a/linux-pi/homepage/.env.example +++ b/linux-pi/homepage/.env.example @@ -3,6 +3,10 @@ # Tailscale auth for homepage-pi-ts (same OAuth client secret as the other sidecars). TS_AUTHKEY= +# Direct LAN identities accepted by Homepage when Tailscale is unavailable. +HOMEPAGE_VAR_PI_HOSTNAME= +HOMEPAGE_VAR_PI_LAN_IP= + # This homepage's own tailnet hostname (used for HOMEPAGE_ALLOWED_HOSTS + the main # server's link to it). HOMEPAGE_VAR_PI_HOMEPAGE_DOMAIN=homepage-pi..ts.net @@ -15,6 +19,7 @@ HOMEPAGE_VAR_CUPS_DOMAIN=cups-pi..ts.net # Main server's homepage — for the "Main Server" card (link + status monitor), # the reverse of the main server's "Secondary Server (Pi)" card. HOMEPAGE_VAR_MAIN_HOMEPAGE_DOMAIN=homepage..ts.net +HOMEPAGE_VAR_MAIN_HOSTNAME= # AdGuard admin credentials for the Pi's backup-DNS stats widget. HOMEPAGE_VAR_ADGUARD_USER= diff --git a/linux-pi/homepage/config/docker.yaml b/linux-pi/homepage/config/docker.yaml new file mode 100644 index 0000000..98838a8 --- /dev/null +++ b/linux-pi/homepage/config/docker.yaml @@ -0,0 +1,2 @@ +my-docker: + socket: /var/run/docker.sock diff --git a/linux-pi/homepage/config/services.yaml b/linux-pi/homepage/config/services.yaml index 0db101f..1e04b14 100644 --- a/linux-pi/homepage/config/services.yaml +++ b/linux-pi/homepage/config/services.yaml @@ -5,7 +5,7 @@ description: Backup DNS resolver widget: type: adguard - url: https://{{HOMEPAGE_VAR_ADGUARD_PI_DOMAIN}} + url: http://localhost:80 username: "{{HOMEPAGE_VAR_ADGUARD_USER}}" password: "{{HOMEPAGE_VAR_ADGUARD_PASS}}" - motioneye: @@ -42,5 +42,5 @@ - server dashboard: icon: mdi-server href: https://{{HOMEPAGE_VAR_MAIN_HOMEPAGE_DOMAIN}}/ - description: ollie-server — primary home server + description: "{{HOMEPAGE_VAR_MAIN_HOSTNAME}} — primary home server" siteMonitor: https://{{HOMEPAGE_VAR_MAIN_HOMEPAGE_DOMAIN}} diff --git a/linux-pi/homepage/docker-compose.yml b/linux-pi/homepage/docker-compose.yml index 9b685c5..7f8d788 100644 --- a/linux-pi/homepage/docker-compose.yml +++ b/linux-pi/homepage/docker-compose.yml @@ -1,7 +1,7 @@ services: # Decoupled Tailscale sidecar for the Pi's homepage — HTTPS at # https://homepage-pi..ts.net, proxying :443 -> the host's homepage on - # :3000 via host.docker.internal. Mirrors linux-server/homepage/homepage-ts. + # :3001 via host.docker.internal. Mirrors linux-server/homepage/homepage-ts. homepage-pi-ts: image: tailscale/tailscale:latest container_name: homepage-pi-ts @@ -29,16 +29,13 @@ services: image: ghcr.io/gethomepage/homepage:latest container_name: homepage network_mode: host - depends_on: - - homepage-pi-ts volumes: - ./config:/app/config - /var/run/docker.sock:/var/run/docker.sock:ro env_file: - .env environment: - # The proxied hostname must be allow-listed or homepage rejects the request. - HOMEPAGE_ALLOWED_HOSTS: "localhost,${HOMEPAGE_VAR_PI_HOMEPAGE_DOMAIN}" + HOMEPAGE_ALLOWED_HOSTS: "localhost,127.0.0.1,${HOMEPAGE_VAR_PI_HOSTNAME},${HOMEPAGE_VAR_PI_HOSTNAME}.local,${HOMEPAGE_VAR_PI_LAN_IP},${HOMEPAGE_VAR_PI_HOMEPAGE_DOMAIN}" HOSTNAME: "0.0.0.0" - PORT: "3000" + PORT: "3001" restart: unless-stopped diff --git a/linux-pi/homepage/ts-serve.json b/linux-pi/homepage/ts-serve.json index d85ef3a..3f75340 100644 --- a/linux-pi/homepage/ts-serve.json +++ b/linux-pi/homepage/ts-serve.json @@ -2,7 +2,7 @@ "TCP": { "443": { "HTTPS": true } }, "Web": { "${TS_CERT_DOMAIN}:443": { - "Handlers": { "/": { "Proxy": "http://host.docker.internal:3000" } } + "Handlers": { "/": { "Proxy": "http://host.docker.internal:3001" } } } } } diff --git a/linux-server/HTTPS.md b/linux-server/HTTPS.md index d05c96f..54fc59f 100644 --- a/linux-server/HTTPS.md +++ b/linux-server/HTTPS.md @@ -7,7 +7,7 @@ a non-standard port. ## Why subdomains require one Tailscale node per service -A Tailscale node has exactly **one** MagicDNS name (`ollie-server..ts.net`). +A Tailscale node has exactly **one** MagicDNS name (`..ts.net`). You cannot mint arbitrary subdomains of it — `forgejo..ts.net` only exists, resolves, and can get a TLS cert if there is a **node named `forgejo`**. @@ -41,7 +41,7 @@ app generates correct HTTPS URLs. 1. **Enable HTTPS** for the tailnet (DNS → "Enable HTTPS"), so `tailscale serve` can provision Let's Encrypt certs for `*.ts.net`. -2. **MagicDNS** enabled (it is, since you resolve `ollie-server..ts.net`). +2. **MagicDNS** enabled (confirm `..ts.net` resolves). 3. **An auth method for the sidecars — resolved: OAuth client + tag.** Reuses the existing Tailscale OAuth client (`linux-server/tailscale-proxy/.env`, originally created read-only for the device-status proxy): @@ -232,7 +232,7 @@ side of the `ports:` mapping (`host:container`), not the host side. | cockpit | 9090 | ✅ done | host systemd service — **sidecar-only** stack proxies `https+insecure://host.docker.internal:9090`; `cockpit.conf.example`'s `Origins` line turned out to be unnecessary in practice — see Gotchas | | tailscale-web | 8088 | ✅ done | not in the original rollout — added because the homepage Tailscale tile linked plain HTTP. `tailscale web` is a host **systemd user unit**, not a container; `ExecStart` needs `--listen 0.0.0.0:8088 --origin https://tailscale-web..ts.net` so it's reachable via `host.docker.internal` and knows it's reverse-proxied. Don't use port `:5252` — see Gotchas | | watchtower | 8080 | ✅ done | **no UI** — the sidecar fronts only watchtower's token-gated `/v1/metrics` HTTP API (enable `WATCHTOWER_HTTP_API_METRICS=true` + `WATCHTOWER_HTTP_API_TOKEN`); no homepage `href`. Monitor it in Uptime Kuma — see below | -| qbittorrent | 8080 | ✅ done | web UI at container :8080 (`WEBUI_PORT=8080`); works at root (relative URLs). After first login set WebUI "IP address" to `127.0.0.1` (Options → Web UI) so it's reachable only via the serve proxy — the analog of Syncthing's loopback bind; the linuxserver image has no env for it. `Server domains` defaults to `*`, so the proxied Host passes host-header validation. Publish BitTorrent `:6881` tcp+udp on the **sidecar** (raw, not via serve). **Not VPN-routed** — see TODO.md | +| qbittorrent | 8080 | ✅ done | web UI at container :8080 (`WEBUI_PORT=8080`); works at root (relative URLs). After first login set WebUI "IP address" to `127.0.0.1` (Options → Web UI) so it's reachable only via the serve proxy — the analog of Syncthing's loopback bind; the linuxserver image has no env for it. `Server domains` defaults to `*`, so the proxied Host passes host-header validation. Publish BitTorrent `:6881` tcp+udp on the **sidecar** (raw, not via serve). **Not VPN-routed** — see ../docs/TODO.md | Services that also expose **non-HTTP** ports the LAN/tailnet needs (AdGuard DNS `:53`, Syncthing sync `:22000`, Forgejo SSH `:22`) keep those as direct diff --git a/linux-server/README.md b/linux-server/README.md index db6009e..3ad3394 100644 --- a/linux-server/README.md +++ b/linux-server/README.md @@ -70,6 +70,7 @@ Start services in this order. Most just need `docker compose up -d`; exceptions ```sh # Homepage — update .env with your server details first (see post-install.md) cd linux-server/homepage && cp .env.example .env +# Existing .env: add new .env.example keys, including HOMEPAGE_VAR_PI_HOSTNAME. # edit .env, then: docker compose up -d # Access at http://:3000 (or https:// after HTTPS setup) @@ -312,7 +313,7 @@ In NPM admin (`http://:81`): 3. Access at `https://qbittorrent..ts.net/` 4. On first run, the LinuxServer image generates a temporary admin password — find it with `docker compose logs qbittorrent | grep -i password`, log in as `admin`, then change it in **Options → Web UI** (and add the new login to `HOMEPAGE_VAR_QBITTORRENT_PASSWORD` in `linux-server/homepage/.env` for the widget) 5. In **Options → Web UI**, set the "IP address" to `127.0.0.1` so the UI is reachable only through the HTTPS sidecar, not the raw tailnet port - 6. BitTorrent `:6881` (tcp/udp) is published on the sidecar. **Not routed through a VPN** — fine for academic/legal torrents; see [`../TODO.md`](../TODO.md) for the planned Gluetun VPN sidecar before any other use + 6. BitTorrent `:6881` (tcp/udp) is published on the sidecar. **Not routed through a VPN** — fine for academic/legal torrents; see [`../docs/TODO.md`](../docs/TODO.md) for the planned Gluetun VPN sidecar before any other use 17. OpenSpeedTest | [GitHub](https://github.com/openspeedtest/Speed-Test) | [Docs](https://openspeedtest.com/selfhosted-speedtest) 1. Self-hosted browser speed test for the **LAN** (the local-network counterpart to speedtest-tracker's ISP test) — any device opens it in a browser and measures its throughput to the server. Fronted by its own Tailscale HTTPS sidecar (see [`HTTPS.md`](HTTPS.md)), like the other services @@ -334,13 +335,13 @@ In NPM admin (`http://:81`): cp .env.example .env # set RESTIC_PASSWORD (save it in Bitwarden!) + ntfy sudo touch /mnt/wd1tb/.backup-target-ok docker compose up -d # status-card server (loopback :8099) - sudo cp backup.service backup.timer backup-failure.service /etc/systemd/system/ - sudo systemctl daemon-reload && sudo systemctl enable --now backup.timer + bash setup.sh --dry-run + sudo bash setup.sh ``` 3. The status card on Homepage shows last-run time, status, and repo size; failures push to ntfy 19. UPS | [NUT](https://networkupstools.org/) | [Docs](https://networkupstools.org/docs/man/) - 1. Battery-backup monitoring for the CyberPower PR1500LCDRT2U over USB — ntfy alerts on power events, clean shutdown on low battery, auto-restart when wall power returns. Full runbook in [`ups/README.md`](ups/README.md) + 1. Battery-backup monitoring for the CyberPower CST135UC2 over USB — ntfy alerts on power events, clean shutdown on low battery, auto-restart when wall power returns. Full runbook in [`ups/README.md`](ups/README.md) 2. Deploy: ```sh sudo apt install nut # or rerun the root setup.sh --profile server diff --git a/linux-server/apt_packages.md b/linux-server/apt_packages.md index 42df832..9595e76 100644 --- a/linux-server/apt_packages.md +++ b/linux-server/apt_packages.md @@ -65,7 +65,7 @@ Tested on Ubuntu Server LTS. Most packages work on other Debian-based distros. 2. Enabled automatically via systemd socket activation on install; access at `https://:9090` 3. Login with your Linux username and password 19. nut | [apt](https://packages.ubuntu.com/search?keywords=nut) | [Homepage](https://networkupstools.org/) - 1. Network UPS Tools — monitors the CyberPower PR1500LCDRT2U over USB; ntfy alerts + clean shutdown on low battery + 1. Network UPS Tools — monitors the CyberPower CST135UC2 over USB; ntfy alerts + clean shutdown on low battery 2. Needs one-time config after install: `sudo bash linux-server/ups/setup.sh` (see [`ups/README.md`](ups/README.md)) 20. sshpass | [apt](https://packages.ubuntu.com/search?keywords=sshpass) | [Homepage](https://sourceforge.net/projects/sshpass/) 1. Non-interactive SSH password authentication — used by `SSH_and_GPG/add_remote_host.sh` diff --git a/linux-server/backup/README.md b/linux-server/backup/README.md index a7f2afe..70ff1d1 100644 --- a/linux-server/backup/README.md +++ b/linux-server/backup/README.md @@ -38,6 +38,7 @@ gets a ~1s `docker stop`/`start` around a volume copy — the one brief exceptio cd linux-server/backup cp .env.example .env # set RESTIC_PASSWORD (and SAVE IT IN BITWARDEN), NTFY_URL/topic + chmod 600 .env ``` 4. Initialize the repo (the script also does this on first run): ```sh @@ -48,12 +49,10 @@ gets a ~1s `docker stop`/`start` around a volume copy — the one brief exceptio ```sh docker compose up -d ``` -6. Install + enable the timer (system units; adjust the path in the unit files if - the repo isn't at `/home/ulises/github/Computer-Setup`): +6. Render, install, and enable the system timer from the current checkout path: ```sh - sudo cp backup.service backup.timer backup-failure.service /etc/systemd/system/ - sudo systemctl daemon-reload - sudo systemctl enable --now backup.timer + bash setup.sh --dry-run + sudo bash setup.sh ``` 7. Dry run + verify: ```sh diff --git a/linux-server/backup/backup-failure.service b/linux-server/backup/backup-failure.service deleted file mode 100644 index 4b93e8f..0000000 --- a/linux-server/backup/backup-failure.service +++ /dev/null @@ -1,8 +0,0 @@ -[Unit] -Description=Alert on server backup failure - -[Service] -Type=oneshot -# Fires via backup.service's OnFailure= — records a failed status and pings -# ntfy even if backup.sh died before its own error handler could run. -ExecStart=/home/ollie/github/Computer-Setup/linux-server/backup/backup.sh notify-failure diff --git a/linux-server/backup/backup-failure.service.template b/linux-server/backup/backup-failure.service.template new file mode 100644 index 0000000..6779f4a --- /dev/null +++ b/linux-server/backup/backup-failure.service.template @@ -0,0 +1,6 @@ +[Unit] +Description=Alert on server backup failure + +[Service] +Type=oneshot +ExecStart=/usr/bin/env bash @BACKUP_SCRIPT@ notify-failure diff --git a/linux-server/backup/backup.service b/linux-server/backup/backup.service deleted file mode 100644 index 68abcb5..0000000 --- a/linux-server/backup/backup.service +++ /dev/null @@ -1,15 +0,0 @@ -[Unit] -Description=Server backup (restic → external drive) -# Don't run unless the backup drive is mounted. -RequiresMountsFor=/mnt/wd1tb -After=network-online.target docker.service -OnFailure=backup-failure.service - -[Service] -Type=oneshot -ExecStart=/home/ollie/github/Computer-Setup/linux-server/backup/backup.sh -# Runs as root to read all container-owned data dirs and drive docker. -# Give restic a HOME so it can use its local cache (root has none under systemd). -Environment=HOME=/root -Nice=10 -IOSchedulingClass=idle diff --git a/linux-server/backup/backup.service.template b/linux-server/backup/backup.service.template new file mode 100644 index 0000000..37cd7e2 --- /dev/null +++ b/linux-server/backup/backup.service.template @@ -0,0 +1,12 @@ +[Unit] +Description=Server backup (restic → external drive) +RequiresMountsFor=/mnt/wd1tb +After=network-online.target docker.service +OnFailure=backup-failure.service + +[Service] +Type=oneshot +ExecStart=/usr/bin/env bash @BACKUP_SCRIPT@ +Environment=HOME=/root +Nice=10 +IOSchedulingClass=idle diff --git a/linux-server/backup/setup.sh b/linux-server/backup/setup.sh new file mode 100755 index 0000000..8bb29fb --- /dev/null +++ b/linux-server/backup/setup.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" +DRY_RUN=false + +case "${1:-}" in + "") ;; + --dry-run) DRY_RUN=true ;; + *) printf 'error: unknown argument: %s\n' "$1" >&2; exit 1 ;; +esac + +if ! [[ "$SCRIPT_DIR" =~ ^/[[:alnum:]_./-]+$ ]]; then + printf 'error: unsupported character in checkout path: %s\n' "$SCRIPT_DIR" >&2 + exit 1 +fi +if [[ "$DRY_RUN" == false ]]; then + [[ $EUID -eq 0 ]] || { printf 'error: run with sudo: sudo bash %s\n' "$0" >&2; exit 1; } + [[ -f "$SCRIPT_DIR/.env" ]] || { printf 'error: create %s/.env before installing\n' "$SCRIPT_DIR" >&2; exit 1; } + chmod 600 "$SCRIPT_DIR/.env" +fi + +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT + +render_unit() { + local src="$1" dest="$2" + sed "s|@BACKUP_SCRIPT@|$SCRIPT_DIR/backup.sh|g" "$src" > "$tmp/$dest" + if [[ "$DRY_RUN" == true ]]; then + printf '[dry-run] install %s as /etc/systemd/system/%s\n' "$src" "$dest" + else + install -o root -g root -m 644 "$tmp/$dest" "/etc/systemd/system/$dest" + fi +} + +render_unit "$SCRIPT_DIR/backup.service.template" backup.service +render_unit "$SCRIPT_DIR/backup-failure.service.template" backup-failure.service +if command -v systemd-analyze >/dev/null; then + systemd-analyze verify "$tmp/backup.service" "$tmp/backup-failure.service" "$SCRIPT_DIR/backup.timer" +fi + +if [[ "$DRY_RUN" == true ]]; then + printf '[dry-run] install %s as /etc/systemd/system/backup.timer\n' "$SCRIPT_DIR/backup.timer" + printf '[dry-run] systemctl daemon-reload\n' + printf '[dry-run] systemctl enable --now backup.timer\n' +else + install -o root -g root -m 644 "$SCRIPT_DIR/backup.timer" /etc/systemd/system/backup.timer + systemctl daemon-reload + systemctl enable --now backup.timer + printf 'Installed and enabled backup.timer\n' +fi diff --git a/linux-server/forgejo/forgejo-runner-status.service b/linux-server/forgejo/forgejo-runner-status.service deleted file mode 100644 index 9280ab2..0000000 --- a/linux-server/forgejo/forgejo-runner-status.service +++ /dev/null @@ -1,8 +0,0 @@ -[Unit] -Description=Forgejo Actions runner status check (Mac mini) -After=network-online.target docker.service -Wants=network-online.target - -[Service] -Type=oneshot -ExecStart=/home/ollie/github/Computer-Setup/linux-server/forgejo/runner-status.sh diff --git a/linux-server/forgejo/forgejo-runner-status.service.template b/linux-server/forgejo/forgejo-runner-status.service.template new file mode 100644 index 0000000..14690a0 --- /dev/null +++ b/linux-server/forgejo/forgejo-runner-status.service.template @@ -0,0 +1,8 @@ +[Unit] +Description=Forgejo Actions runner status check +After=network-online.target docker.service +Wants=network-online.target + +[Service] +Type=oneshot +ExecStart=/usr/bin/env bash @RUNNER_STATUS_SCRIPT@ diff --git a/linux-server/forgejo/setup-runner-status.sh b/linux-server/forgejo/setup-runner-status.sh new file mode 100755 index 0000000..d7512a0 --- /dev/null +++ b/linux-server/forgejo/setup-runner-status.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" +DRY_RUN=false + +case "${1:-}" in + "") ;; + --dry-run) DRY_RUN=true ;; + *) printf 'error: unknown argument: %s\n' "$1" >&2; exit 1 ;; +esac + +if ! [[ "$SCRIPT_DIR" =~ ^/[[:alnum:]_./-]+$ ]]; then + printf 'error: unsupported character in checkout path: %s\n' "$SCRIPT_DIR" >&2 + exit 1 +fi +if [[ "$DRY_RUN" == false ]]; then + [[ $EUID -eq 0 ]] || { printf 'error: run with sudo: sudo bash %s\n' "$0" >&2; exit 1; } + [[ -f "$SCRIPT_DIR/.env" ]] || { printf 'error: create %s/.env before installing\n' "$SCRIPT_DIR" >&2; exit 1; } + chmod 600 "$SCRIPT_DIR/.env" +fi + +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT +sed "s|@RUNNER_STATUS_SCRIPT@|$SCRIPT_DIR/runner-status.sh|g" \ + "$SCRIPT_DIR/forgejo-runner-status.service.template" > "$tmp/forgejo-runner-status.service" + +if command -v systemd-analyze >/dev/null; then + systemd-analyze verify "$tmp/forgejo-runner-status.service" "$SCRIPT_DIR/forgejo-runner-status.timer" +fi + +if [[ "$DRY_RUN" == true ]]; then + printf '[dry-run] install rendered forgejo-runner-status.service\n' + printf '[dry-run] install forgejo-runner-status.timer\n' + printf '[dry-run] systemctl daemon-reload\n' + printf '[dry-run] systemctl enable --now forgejo-runner-status.timer\n' +else + install -o root -g root -m 644 "$tmp/forgejo-runner-status.service" /etc/systemd/system/forgejo-runner-status.service + install -o root -g root -m 644 "$SCRIPT_DIR/forgejo-runner-status.timer" /etc/systemd/system/forgejo-runner-status.timer + systemctl daemon-reload + systemctl enable --now forgejo-runner-status.timer + printf 'Installed and enabled forgejo-runner-status.timer\n' +fi diff --git a/linux-server/glances/.env.example b/linux-server/glances/.env.example index 5fecc3f..f03d5a7 100644 --- a/linux-server/glances/.env.example +++ b/linux-server/glances/.env.example @@ -3,3 +3,9 @@ # Tailscale auth for the sidecar — same OAuth client secret as the other sidecars # (needs the Auth Keys scope + tag:container in the ACL). TS_AUTHKEY= + +# Glances 4.5.2+ rejects unknown Host headers. Include every LAN and tailnet name +# used to access its Web UI/API; keep real hostnames and addresses in .env only. +# If omitted during an upgrade, Glances permits only local proxy names and logs a +# migration warning until this value is added. +GLANCES_ALLOWED_HOSTS=localhost,127.0.0.1,,.local,,glances..ts.net diff --git a/linux-server/glances/docker-compose.yml b/linux-server/glances/docker-compose.yml index d445d19..f4914b2 100644 --- a/linux-server/glances/docker-compose.yml +++ b/linux-server/glances/docker-compose.yml @@ -30,7 +30,7 @@ services: restart: unless-stopped glances: - image: nicolargo/glances:latest-full + image: nicolargo/glances:4.5.4-full container_name: glances restart: unless-stopped pid: host @@ -38,6 +38,7 @@ services: network_mode: host environment: - GLANCES_OPT=-w + - GLANCES_ALLOWED_HOSTS=${GLANCES_ALLOWED_HOSTS} - TZ=America/Los_Angeles volumes: - /var/run/docker.sock:/var/run/docker.sock:ro @@ -53,4 +54,5 @@ services: - ./entrypoint.sh:/entrypoint.sh:ro - ./rename_disks.py:/rename_disks.py:ro - /dev/disk/by-label:/dev/disk/by-label:ro + - /sys/class/block:/sys/class/block:ro command: /bin/sh /entrypoint.sh diff --git a/linux-server/glances/entrypoint.sh b/linux-server/glances/entrypoint.sh index 092b51e..4476b4b 100755 --- a/linux-server/glances/entrypoint.sh +++ b/linux-server/glances/entrypoint.sh @@ -1,33 +1,38 @@ #!/bin/sh -# Resolves /dev/disk/by-label/ symlinks to current kernel device names and -# monkey-patches the Glances diskio plugin to use persistent label-based names -# (e.g. "Seagate_4TB" instead of "sda"), so Homepage disk I/O widgets survive -# kernel name reordering across reboots. +# The pinned Alpine image has no Bash, so this launcher intentionally uses POSIX sh. +set -eu PYTHON_BIN="/venv/bin/python${PYTHON_VERSION}" -# Build mapping of kernel device name -> label from /dev/disk/by-label/ -DISK_MAPPING="" -for link in /dev/disk/by-label/Seagate_4TB /dev/disk/by-label/WD_1TB /dev/disk/by-label/WD14TB; do - [ -L "$link" ] || continue - label=$(basename "$link") - device=$(readlink "$link" | sed 's|.*/||; s/[0-9]*$//') - [ -n "$device" ] && DISK_MAPPING="${DISK_MAPPING}${device}:${label}," -done -DISK_MAPPING="${DISK_MAPPING%,}" +exec "$PYTHON_BIN" -c ' +import os +import re +import shlex +import sys +from pathlib import Path -if [ -n "$DISK_MAPPING" ]; then - export GLANCES_DISK_MAPPING="$DISK_MAPPING" - exec $PYTHON_BIN -c " -import sys, os -sys.path.insert(0, '/') +sys.path.insert(0, "/") import rename_disks + +allowed_hosts = os.environ.get("GLANCES_ALLOWED_HOSTS", "").strip() +if not allowed_hosts: + print( + "GLANCES_ALLOWED_HOSTS is unset; allowing local proxy names only. Add the host names used by clients to .env.", + file=sys.stderr, + ) + allowed_hosts = "localhost,127.0.0.1,host.docker.internal" +hosts = [host.strip() for host in allowed_hosts.split(",") if host.strip()] +if not hosts or any(re.fullmatch(r"[A-Za-z0-9.-]+", host) is None for host in hosts): + raise SystemExit("GLANCES_ALLOWED_HOSTS must contain only comma-separated hostnames or IPv4 addresses") + +config = Path("/tmp/glances.conf") +allowed_hosts_value = ",".join(hosts) +config.write_text(f"[outputs]\nallowed_hosts={allowed_hosts_value}\n", encoding="utf-8") +rename_disks.patch() + from glances import main -sys.argv = ['glances'] -args = os.environ.get('GLANCES_OPT', '-w').split() -sys.argv.extend(args) + +sys.argv = ["glances", "-C", str(config)] +sys.argv.extend(shlex.split(os.environ.get("GLANCES_OPT", "-w"))) main() -" -else - exec $PYTHON_BIN -m glances ${GLANCES_OPT:--w} -fi +' diff --git a/linux-server/glances/rename_disks.py b/linux-server/glances/rename_disks.py index d0cf42b..803e32a 100644 --- a/linux-server/glances/rename_disks.py +++ b/linux-server/glances/rename_disks.py @@ -1,30 +1,52 @@ -import os, re +import os +import sys +from pathlib import Path -mapping_str = os.environ.get("GLANCES_DISK_MAPPING", "") -disk_aliases = {} -if mapping_str: - for pair in mapping_str.split(","): - if ":" in pair: - kernel, label = pair.split(":", 1) - disk_aliases[kernel.strip()] = label.strip() +DEFAULT_LABELS = ("Seagate_4TB", "WD_1TB", "WD14TB") + + +def resolve_aliases(): + label_dir = Path(os.environ.get("GLANCES_DISK_LABEL_DIR", "/dev/disk/by-label")) + sys_block_dir = Path(os.environ.get("GLANCES_SYS_BLOCK_DIR", "/sys/class/block")) + labels = os.environ.get("GLANCES_DISK_LABELS", ",".join(DEFAULT_LABELS)).split(",") + aliases = {} + + for label in labels: + label = label.strip() + if not label or Path(label).name != label: + continue + try: + device_name = (label_dir / label).resolve(strict=True).name + sys_device = sys_block_dir / device_name + if (sys_device / "partition").exists(): + device_name = sys_device.resolve(strict=True).parent.name + aliases[device_name] = label + except OSError: + continue + + return aliases def patch(): try: import glances.plugins.diskio - orig = glances.plugins.diskio.DiskioPlugin.get_raw + + plugin = glances.plugins.diskio.DiskioPlugin + original = plugin.get_raw def patched(self): - stats = orig(self) + stats = original(self) + aliases = resolve_aliases() for stat in stats: - dn = stat.get("disk_name", "") - if re.match(r"^[a-z]+$", dn) and dn in disk_aliases: - stat["disk_name"] = disk_aliases[dn] + disk_name = stat.get("disk_name", "") + if disk_name in aliases: + stat["disk_name"] = aliases[disk_name] return stats - glances.plugins.diskio.DiskioPlugin.get_raw = patched - except Exception: - pass - + plugin.get_raw = patched + except Exception as exc: + print(f"glances disk alias patch failed: {exc}", file=sys.stderr) + return False -patch() + print("glances disk alias patch installed", file=sys.stderr) + return True diff --git a/linux-server/glances/test_rename_disks.py b/linux-server/glances/test_rename_disks.py new file mode 100644 index 0000000..5d5c2d4 --- /dev/null +++ b/linux-server/glances/test_rename_disks.py @@ -0,0 +1,113 @@ +import contextlib +import io +import os +import sys +import tempfile +import types +import unittest +from pathlib import Path +from unittest.mock import patch + +import rename_disks + + +class RenameDisksTest(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.root = Path(self.temp.name) + self.labels = self.root / "labels" + self.devices = self.root / "devices" + self.sys_block = self.root / "sys-block" + self.labels.mkdir() + self.devices.mkdir() + self.sys_block.mkdir() + self.env = patch.dict( + os.environ, + { + "GLANCES_DISK_LABEL_DIR": str(self.labels), + "GLANCES_SYS_BLOCK_DIR": str(self.sys_block), + }, + ) + self.env.start() + + def tearDown(self): + self.env.stop() + self.temp.cleanup() + + def add_partition(self, label, partition, parent): + (self.devices / partition).touch() + (self.labels / label).symlink_to(self.devices / partition) + sys_parent = self.root / "sys-devices" / parent + sys_partition = sys_parent / partition + sys_partition.mkdir(parents=True) + (sys_partition / "partition").touch() + (self.sys_block / partition).symlink_to(sys_partition) + + def test_resolves_sd_nvme_and_mmc_parents(self): + self.add_partition("Seagate_4TB", "sda1", "sda") + self.add_partition("WD_1TB", "nvme0n1p2", "nvme0n1") + self.add_partition("WD14TB", "mmcblk0p1", "mmcblk0") + + self.assertEqual( + rename_disks.resolve_aliases(), + {"sda": "Seagate_4TB", "nvme0n1": "WD_1TB", "mmcblk0": "WD14TB"}, + ) + + def test_rechecks_late_and_reshuffled_devices(self): + self.assertEqual(rename_disks.resolve_aliases(), {}) + self.add_partition("WD_1TB", "sdb1", "sdb") + self.assertEqual(rename_disks.resolve_aliases(), {"sdb": "WD_1TB"}) + + (self.labels / "WD_1TB").unlink() + self.add_partition("WD_1TB", "nvme1n1p1", "nvme1n1") + self.assertEqual(rename_disks.resolve_aliases(), {"nvme1n1": "WD_1TB"}) + + def test_patch_renames_stats_and_logs_installation(self): + self.add_partition("Seagate_4TB", "sda1", "sda") + + class DiskioPlugin: + def get_raw(self): + return [{"disk_name": "sda"}, {"disk_name": "nvme0n1"}] + + glances = types.ModuleType("glances") + plugins = types.ModuleType("glances.plugins") + diskio = types.ModuleType("glances.plugins.diskio") + diskio.DiskioPlugin = DiskioPlugin + plugins.diskio = diskio + glances.plugins = plugins + + modules = { + "glances": glances, + "glances.plugins": plugins, + "glances.plugins.diskio": diskio, + } + stderr = io.StringIO() + with patch.dict(sys.modules, modules), contextlib.redirect_stderr(stderr): + self.assertTrue(rename_disks.patch()) + self.assertEqual( + DiskioPlugin().get_raw(), + [{"disk_name": "Seagate_4TB"}, {"disk_name": "nvme0n1"}], + ) + + self.assertIn("patch installed", stderr.getvalue()) + + def test_patch_failure_is_visible(self): + glances = types.ModuleType("glances") + plugins = types.ModuleType("glances.plugins") + diskio = types.ModuleType("glances.plugins.diskio") + plugins.diskio = diskio + glances.plugins = plugins + modules = { + "glances": glances, + "glances.plugins": plugins, + "glances.plugins.diskio": diskio, + } + stderr = io.StringIO() + with patch.dict(sys.modules, modules), contextlib.redirect_stderr(stderr): + self.assertFalse(rename_disks.patch()) + + self.assertIn("patch failed", stderr.getvalue()) + + +if __name__ == "__main__": + unittest.main() diff --git a/linux-server/homepage/.env.example b/linux-server/homepage/.env.example index 357bc03..bd66862 100644 --- a/linux-server/homepage/.env.example +++ b/linux-server/homepage/.env.example @@ -76,10 +76,12 @@ HOMEPAGE_VAR_TAILSCALE_WEB_DOMAIN=tailscale-web..ts.net # Fronts NPM's :81 admin UI; NPM still binds host :80/:443 as the non-tailnet edge. HOMEPAGE_VAR_NPM_DOMAIN=npm..ts.net -# Raspberry Pi (ollie-pi4) secondary-server homepage — its own Tailscale sidecar +# Raspberry Pi secondary-server homepage — its own Tailscale sidecar # (homepage-pi..ts.net), served from linux-pi/homepage. The # "Secondary Server (Pi)" card links to it and siteMonitor-pings it. HOMEPAGE_VAR_PI_HOMEPAGE_DOMAIN=homepage-pi..ts.net +# Pi hostname shown as the card's description prefix (kept out of tracked config). +HOMEPAGE_VAR_PI_HOSTNAME= # Tailscale auth for the homepage sidecar — same OAuth client secret as the others. TS_AUTHKEY= diff --git a/linux-server/homepage/config/services.yaml b/linux-server/homepage/config/services.yaml index 590d31e..576925b 100644 --- a/linux-server/homepage/config/services.yaml +++ b/linux-server/homepage/config/services.yaml @@ -2,7 +2,7 @@ - pi dashboard: icon: mdi-raspberry-pi href: https://{{HOMEPAGE_VAR_PI_HOMEPAGE_DOMAIN}}/ - description: ollie-pi4 — motioneye, cups, backup DNS + description: "{{HOMEPAGE_VAR_PI_HOSTNAME}} — motioneye, cups, backup DNS" siteMonitor: https://{{HOMEPAGE_VAR_PI_HOMEPAGE_DOMAIN}} - Core: diff --git a/linux-server/post-install.md b/linux-server/post-install.md index 5fc3b3b..596b83a 100644 --- a/linux-server/post-install.md +++ b/linux-server/post-install.md @@ -174,7 +174,7 @@ Everything Forgejo needs to restore from scratch lives in `FORGEJO_DATA_PATH` (d #### Runner status monitor A host timer (`runner-status.sh`) asks Forgejo whether the Mac mini Actions -runner (see [`../../macOS/forgejo-runner/`](../../macOS/forgejo-runner/)) is +runner (see [`../macOS/forgejo-runner/`](../macOS/forgejo-runner/)) is connected, and surfaces it three ways: the homepage **forgejo-runner** card, an Uptime Kuma push monitor, and an ntfy alert when it drops (and recovers). Optional — skip if you aren't running CI. @@ -190,9 +190,8 @@ Optional — skip if you aren't running CI. ``` - [ ] Install the timer (polls every 2 minutes): ```sh - sudo cp forgejo-runner-status.service forgejo-runner-status.timer /etc/systemd/system/ - sudo systemctl daemon-reload - sudo systemctl enable --now forgejo-runner-status.timer + bash setup-runner-status.sh --dry-run + sudo bash setup-runner-status.sh ``` - [ ] Verify: ```sh @@ -264,6 +263,7 @@ cleanly on low battery. Full runbook in [`ups/README.md`](ups/README.md). sudo bash setup.sh ``` - [ ] Verify: `upsc cyberpower ups.status` prints `OL` +- [ ] Run `bash verify.sh --platform server` from the repo root - [ ] Start the PeaNUT dashboard (`docker compose up -d`, needs `TS_AUTHKEY` in `.env`) — graphs at `https://peanut..ts.net/`, and the homepage **ups** card goes live @@ -285,15 +285,17 @@ Three drives are attached via a TerraMas Thunderbolt DAS enclosure: - **fstab** entries use `nofail,x-systemd.device-timeout=10` — if the DAS is disconnected at boot, the system continues without them. - **Glances disk renaming** — kernel device names (`sda`/`sdb`/`sdc`) can shuffle - when the enclosure is reconnected. The Glances entrypoint - (`glances/entrypoint.sh`) reads `/dev/disk/by-label/` at startup and - monkey-patches the diskio plugin so Homepage widgets see persistent label-based - names instead. See `glances/rename_disks.py`. + when the enclosure is reconnected. The Glances patch resolves + `/dev/disk/by-label/` through sysfs on every refresh, so Homepage widgets see + persistent label-based names even after a late mount or device reshuffle. Patch + installation or API failures are written to the container log. See + `glances/rename_disks.py`. - **Homepage widgets** — the capacity widgets use `fs:/mnt/` (stable mount points); the R/W speed widgets use `disk: