DNS robustness: fix bootstrap deadlock + secondary AdGuard on the Pi - #58
Merged
Conversation
Pin static public resolvers (9.9.9.10, 1.1.1.1) on the adguard-ts sidecar so its OAuth bootstrap can resolve api.tailscale.com without depending on AdGuard Home, which shares this sidecar's network namespace. Previously a cold start (e.g. after a network outage) deadlocked: the sidecar needs DNS to authenticate with the Tailscale control plane, but the only DNS on the box is the AdGuard instance that cannot start until the sidecar's netns is up. Pinning an external resolver for this one container breaks the cycle without routing other traffic around AdGuard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce a linux-pi/ node (parallel to linux-server/) and a backup AdGuard Home on the Raspberry Pi to remove the home network's single point of DNS failure exposed by the 2026-07 outage. Unlike the primary (linux-server/adguard), AdGuard here is host-networked and NOT wrapped in a Tailscale sidecar's netns, so :53 keeps serving LAN clients even when the tailnet or internet is down. A decoupled adguard-pi-ts sidecar provides only the HTTPS admin UI and carries the dns: [9.9.9.10, 1.1.1.1] bootstrap guard from the primary's fix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an adguardhome-sync stack (bakito/adguardhome-sync) that runs on the Pi and pulls the primary's filters, rewrites, upstreams and rules into the secondary on a cron, so the two stay in lockstep. DHCP sync is disabled; credentials and URLs live in a gitignored .env. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add linux-pi/README.md (deploy runbook, resilience rationale, and the "secondary DNS is not clean failover" caveat), point the root CLAUDE.md at the new node, and record the post-outage DNS resilience work in TODO.md: bootstrap-deadlock fix done, Pi secondary added, secondary DHCP still open. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Deploy a homepage instance on ollie-pi4 (host-networked on :3000) with a decoupled homepage-pi-ts sidecar at https://homepage-pi.<tailnet>.ts.net. It carries cards for the Pi's services (backup AdGuard, MotionEye, CUPS) and a resources widget that shows the Pi's own CPU/mem/disk/temp, since homepage runs on the Pi host. Runtime + homepage-auto-generated files are gitignored. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add decoupled Tailscale sidecars fronting the Pi's existing host services — MotionEye (:8765) and CUPS (:631) — at motioneye-pi.<tailnet>.ts.net and cups-pi.<tailnet>.ts.net. The services stay untouched host processes; these stacks add only the HTTPS front doors, each with the same dns: [9.9.9.10, 1.1.1.1] bootstrap guard as the other sidecars. CUPS validates the Host header, so its admin pages may need `ServerAlias *` (noted in the compose + README). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a "Secondary Server (Pi)" group to the main server's homepage that links to the Pi dashboard and siteMonitor-pings it for up/down + latency, driven by a new HOMEPAGE_VAR_PI_HOMEPAGE_DOMAIN. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… hop Extend linux-pi/README with the homepage/motioneye/cups stacks, the main homepage link, deploy steps, and a troubleshooting section for the sidecar -> host.docker.internal:<port> hop that every Pi HTTPS front door depends on (old Docker without host-gateway, or ufw dropping the bridge->host path). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Record the two things learned wiring up the Pi HTTPS front doors: the first HTTPS request to a sidecar hangs briefly while tailscale serve provisions its Let's Encrypt cert (not a fault — retry), and a sidecar's connection bootstraps via DERP then upgrades to a direct LAN path (~5ms), so it is fine for MotionEye video once warmed up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The "Secondary Server (Pi)" card used `widget: {type: siteMonitor}`, which
homepage rejects with "missing widget type". Move it to the top-level
`siteMonitor:` service field so the up/down + latency monitor renders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…bookmarks Add a "Main Server" card to the Pi dashboard that links to and monitors the primary server — the reverse of its "Secondary Server (Pi)" card — via a new HOMEPAGE_VAR_MAIN_HOMEPAGE_DOMAIN. Track an empty bookmarks.yaml so homepage stops auto-generating its Github/Reddit/YouTube example bookmarks (un-ignored so the empty file ships). Also corrects the Pi's motioneye/cups cards to the top-level siteMonitor field, the same fix as the main server. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reorder the main homepage into three rows: Secondary Server (Pi) on top, Management | Network | Storage side-by-side (nested under a header-less "Core" group with style: row), and NAS below with its 2-column disks. No services or widgets changed - the group blocks were re-indented under the row wrapper, comments intact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The disk R/W widgets asked for kernel names (disk:sda/sdb/sdc), but the Glances rename patch (linux-server/glances) reports persistent by-label names, so the widgets errored with "read_bytes undefined". Use disk:WD14TB / Seagate_4TB / WD_1TB to match. Capacity widgets key off mount paths and were unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DAS drives weren't mounted at boot (disconnected), causing Homepage capacity widgets to show no data. Glances restart fixed that. Kernel device names (sda/sdb/sdc) shuffled when the DAS was reconnected, breaking R/W speed widgets. Added a Glances entrypoint that resolves /dev/disk/by-label/ at startup and monkey-patches the diskio plugin to use persistent label-based names (Seagate_4TB, WD_1TB, WD14TB). Changes: - glances/entrypoint.sh — resolves labels → kernel names at container start - glances/rename_disks.py — monkey-patches Glances diskio plugin - glances/docker-compose.yml — mounts entrypoint, rename script, and /dev/disk/by-label into the container - homepage/config/services.yaml — metric: disk:sda/sdb/sdc → disk:<label> - post-install.md — added DAS drives section documenting the setup (cherry picked from commit ba31db2f7e0ac724d2702b4e1b82627c8747f5d3)
# Conflicts: # .gitignore # linux-server/homepage/config/services.yaml
- backup.sh: restic over SFTP to main server DAS (wd1tb primary, wd14tb copy) - systemd timer at 03:45 daily with failure alerts - nginx:alpine status card server on :8099 for homepage widget - LEARNINGS.md: SFTP gotchas (nologin shell output, restic Go SSH client) - Pi homepage: add backup status card under Pi Services
ulises-c
force-pushed
the
feat/dns-robustness
branch
from
July 12, 2026 08:00
99adc73 to
8c8d2d9
Compare
ulises-c
marked this pull request as ready for review
July 12, 2026 08:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A scheduled-maintenance network outage took the home network's only DNS down,
and it couldn't self-heal. Two problems:
sidecar's netns, and the sidecar's OAuth bootstrap needs DNS to reach
api.tailscale.com— but the only DNS is that same AdGuard. Cold start after theoutage deadlocked: sidecar needs DNS → DNS needs the sidecar.
when it goes down.
What this does
Fix the deadlock (
fix(server): break AdGuard/Tailscale DNS bootstrap deadlock)Pin static public resolvers on
adguard-ts(dns: [9.9.9.10, 1.1.1.1]) so thesidecar's bootstrap never depends on AdGuard. Verified in place: the sidecar now
resolves
api.tailscale.comindependently.Add a secondary resolver on the Pi — new
linux-pi/node (parallel tolinux-server/):linux-pi/adguard— backup AdGuard Home, host-networked and independent ofTailscale, so
:53keeps serving LAN clients even when the tailnet/internet isdown. A decoupled
adguard-pi-tssidecar serves only the HTTPS admin UI andcarries the same bootstrap guard. (Deliberately does not copy the primary's
netns coupling.)
linux-pi/adguardhome-sync—bakito/adguardhome-syncon the Pi, pulling theprimary's filters/rewrites/upstreams/rules into the replica on a cron.
linux-pi/README.mddeploy runbook + the "secondary DNS is not cleanfailover" caveat;
CLAUDE.md/TODO.mdupdated.Status / follow-ups
Draft — the deadlock fix is applied and verified on the server; the Pi node is
not deployed yet (owner will run the
linux-pi/README.mdrunbook onollie-pi4:free
:53, fill.envs,docker compose up -d, add the Pi as secondary DNS on therouter). Still open (tracked in TODO): secondary DHCP scope (DHCP is single-homed).
Test plan (on the Pi, post-deploy)
dig @<pi-ip> example.comresolves; a blocked domain returns0.0.0.0(filtering).adguard-pi-ts→dig @<pi-ip>still resolves (DNS independent of Tailscale).https://adguard-pi.<tailnet>.ts.netUI loads; sync makes the Pi's lists match the primary.[server, pi]; stop the primary AdGuard → clients still resolve via the Pi.🤖 Generated with Claude Code