[EcoDev] Node dogfooding — post-storm 0.2.4 green-path + Blend-reachability hardening of the 1click on-ramp

[xAlisher]

TL;DR: EcoDev is dogfooding the post-storm blockchain module 0.2.4 update and hardening our community "1click" node on-ramp so it never declares an unreachable node to Blend. The fork storm surfaced that unreachable nodes joining Blend break the mix network (block proposals route through Blend); the 1click was the on-ramp that put non-technical, NAT'd operators onto Blend without a reachability check. Filing as EcoDev dogfooding feedback + tracking (would tag from-echodev; can't apply the label from this account).

Context

• Storm (Aug 27-30): a batch of nodes joined Blend but were not reachable (wrong IP/port in their declarations, or behind NAT), breaking Blend node-to-node comms and forking the chain. Incident: [Dogfood/incident] Testnet v0.2.1 fork storm (Aug 27–30) — operator impact + investigation #226.
• Fix shipped: blockchain module 0.2.4 (logos-blockchain#3479) adds Blend failure-detection with a clear-broadcast fallback, so proposals no longer stall behind a broken Blend path. New genesis; testnet release stays v0.2.1; bootstrap peers unchanged.
• Root behaviour (code-checked): nothing validates that a node's declared Blend locator is publicly reachable before its SDP declaration is posted. The gate lives in the participate CLI path + libp2p autonat, and setting external_address disables autonat and trusts the operator's IP. So a NAT'd operator can declare unreachable and still join Blend.

What we're tracking

1. Verify the 0.2.4 green-path end to end from a stopped node (wipe state, keep config + keys, logosctl, join Blend only when reachable) and report friction.
2. Add a reachability net-check to the 1click before it declares Blend (mirror participate.rs's bail on unspecified/unreachable; retire the no-public-IP-check toggle; auto-derive external_address).
3. Align the 1click to 0.2.4 (re-pin off the stale July node; new SDP encoding; service-note rename; logosctl join semantics).
4. Follow through on the resource amplifier from the incident (header-only/coalesced block events + drop the per-block stderr dump): #3449, logos-protocol#75.

Note: this reachability behaviour is present in the official config path too; the 1click is called out because it was the on-ramp for non-technical operators. Detailed findings + the per-repo compat issues are tracked internally.

Ask

Would a shared Blend-reachability pre-check helper make sense (so any on-ramp, official or community, gates the declaration the same way), or should each frontend implement its own against participate.rs semantics?

Refs: #226 · logos-blockchain#3479 · #3449 · logos-protocol#75

[xAlisher]

Progress update (2026-09-09). Post-storm recovery onto 0.2.4 dogfooded end to end; the 1-click on-ramp is fixed, and several UI / observability gaps surfaced and were filed upstream.

Green on 0.2.4, three paths verified: CLI (logosctl), the official Basecamp UI, and the new logosctl node guide all sync the live chain. The community 1-click fork can now reach 0.2.4 as well (fix below).

Key findings

• The Basecamp module picker defaults to a higher-numbered pre-release (0.3.0-rc), not 0.2.4 (the live chain), so operators can land on the wrong chain unless they pick 0.2.4 manually.
• The 1-click fork was stuck on 0.2.3 because it bundled its own module clone. Fixed so it resolves the official 0.2.4 module. Validated: the fork UI drives the official 0.2.4 module cleanly, with no RPC drift.
• Recovery path is blocked in the official Basecamp UI: it tells operators to "reset chain state to recover" on a crash or DB corruption but ships no control to do it. This is the exact gap from the Aug storm ([Dogfood/incident] Testnet v0.2.1 fork storm (Aug 27–30) — operator impact + investigation #226). Filed Add in-UI "Reset chain state" recovery control — v0.3 must-have (recovery path blocked; see fork-storm ecosystem#226) logos-blockchain/logos-blockchain-ui#63.
• The 0.2.4 storm fix (Blend failure detection + TSI outage handling) is internal only and not observable. Operators still cannot see Blend delivery health or whether their node is publicly reachable. Filed Expose node health (Blend delivery + reachability) via a read-only status RPC logos-blockchain/logos-blockchain#3502 (read-only status RPC).
• Local UI/module builds produce an un-installable .lgx (forbidden assets/ root entry plus a non-loadable -dev variant), which blocks dev-side dogfooding of UI changes. Filed Local UI/module build produces an un-installable .lgx (assets/ root entry + -dev variant) logos-module-builder#238.

Filed upstream: logos-blockchain/logos-blockchain-ui#63 (reset), logos-blockchain/logos-blockchain#3502 (health RPC), logos-co/logos-module-builder#238 (build tooling). Node-guide feedback shared with the docs team.

Next: Basecamp node-UI feedback to core design (single-column, responsive, tabs not a second sidebar, ordered by the operator's questions; visible primary CTA; honest, live status). Operator recovery guidance (interim manual chain-state clear until the in-UI reset lands).