Skip to content

Commit f4b41ff

Browse files
committed
fix(probes): run the same core as the clients and apply settings live
The probe now installs sing-box-lx (with_xhttp), replacing an upstream core left by an earlier install, and verifies the release checksum. Upstream sing-box rejects a whole configuration containing an XHTTP node, which took the probe blind across every node at once. Also: settings apply without a service restart, check history on the probes tab, atomic binary replacement on reinstall, and the probes menu item appears as soon as the feature is enabled.
1 parent 7896cb4 commit f4b41ff

29 files changed

Lines changed: 1049 additions & 170 deletions

README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -633,16 +633,16 @@ is sent once free space climbs back above the warning level.
633633

634634
A probe is a separate Go binary installed on your own server. It receives a
635635
hidden subscription, runs a real sing-box core and connects to your nodes the
636-
same way a customer does, so it answers the question node status cannot: *does
637-
this node actually work from this network?*
636+
same way a customer does, which shows whether a node is reachable from that
637+
particular network.
638638

639639
Enable it under **Settings → Probes**, then add a probe and run the generated
640640
install command on the host you want to check from. Every result is stored per
641-
vantage point and never overrides `node.status` a single probe cannot tell a
642-
dead node from its own broken uplink.
641+
vantage point and never overrides `node.status`, since a failure can also come
642+
from the probe's own uplink.
643643

644-
Failure codes are actionable rather than generic: `net_unreachable` (port
645-
filtered on the path), `handshake_failed` (dead REALITY destination or DPI),
644+
Each failure code points at a specific cause: `net_unreachable` (port filtered
645+
on the path), `handshake_failed` (dead REALITY destination or DPI),
646646
`auth_rejected` (user not pushed to the running core), `tunnel_no_data` (broken
647647
outbound or ACL), `degraded`, and `core_down` for a failure of the probe's own
648648
core.

README.ru.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -665,13 +665,13 @@ const expected = 'sha256=' + crypto
665665

666666
Пробер — отдельный Go-бинарник, который ставится на ваш собственный сервер. Он
667667
получает скрытую подписку, поднимает настоящее ядро sing-box и подключается к
668-
нодам так же, как это делает клиент. Так он отвечает на вопрос, на который
669-
статус ноды ответить не может: *работает ли эта нода из этой сети?*
668+
нодам так же, как это делает клиент. Это показывает, доступна ли нода из
669+
конкретной сети.
670670

671671
Включается в **Настройки → Проберы**, дальше добавляете пробер и запускаете
672672
сгенерированную команду установки на нужном хосте. Результаты хранятся по точкам
673-
наблюдения и никогда не подменяют `node.status` — один пробер не отличит мёртвую
674-
ноду от собственного сломанного аплинка.
673+
наблюдения и никогда не подменяют `node.status`: сбой может быть и на стороне
674+
самого пробера.
675675

676676
Коды отказов сразу указывают на причину: `net_unreachable` (порт режется на
677677
пути), `handshake_failed` (мёртвый destination REALITY или DPI), `auth_rejected`

docs/mcp-user-guide.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,7 @@ Allowed statement types: `SELECT`, `WITH`, `EXPLAIN`, `DESCRIBE`, `SHOW`. Mutati
195195

196196
> `probes:read` scope required. Requires probes enabled in panel settings.
197197
198-
Returns what external probes observed by dialling your nodes through a real sing-box core, which is the one thing control-plane data cannot answer: whether a client behind a given ISP can actually connect. See the [probes guide](probes.md).
198+
Returns what external probes observed while dialling your nodes through a real sing-box core, which shows what a client behind a given ISP sees. See the [probes guide](probes.md).
199199

200200
**Parameters:**
201201

@@ -207,7 +207,7 @@ Returns what external probes observed by dialling your nodes through a real sing
207207
| `hours` | ❌ No | Look-back window, 1–720 (default 1). Over 24 h reads hourly rollups |
208208
| `limit` | ❌ No | 1–500 (default 100) |
209209

210-
Verdicts: `net_unreachable`, `handshake_failed`, `auth_rejected`, `tunnel_no_data`, `degraded`, `core_down` (the probe's own core failed — this says nothing about the node).
210+
Verdicts: `net_unreachable`, `handshake_failed`, `auth_rejected`, `tunnel_no_data`, `degraded`, `core_down` (the probe's own core failed, so the fault is on the probe host).
211211

212212
<details>
213213
<summary>📖 Example: Why users complain about one node</summary>

docs/mcp-user-guide.ru.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,7 @@ curl -X POST https://your-panel.com/api/mcp \
195195

196196
> Требуется scope: `probes:read`. Проберы должны быть включены в настройках панели.
197197
198-
Возвращает то, что внешние проберы увидели, подключаясь к вашим нодам через настоящее ядро sing-box, — единственное, на что не отвечают данные панели: может ли клиент за конкретным провайдером реально подключиться. Подробности — в [гайде по проберам](probes.ru.md).
198+
Возвращает то, что внешние проберы увидели, подключаясь к вашим нодам через настоящее ядро sing-box: как выглядит подключение для клиента за конкретным провайдером. Подробности — в [гайде по проберам](probes.ru.md).
199199

200200
**Параметры:**
201201

@@ -207,7 +207,7 @@ curl -X POST https://your-panel.com/api/mcp \
207207
| `hours` | ❌ Нет | Окно назад, 1–720 (по умолчанию 1). Больше 24 ч читается из часовых роллапов |
208208
| `limit` | ❌ Нет | 1–500 (по умолчанию 100) |
209209

210-
Вердикты: `net_unreachable`, `handshake_failed`, `auth_rejected`, `tunnel_no_data`, `degraded`, `core_down` (отказало собственное ядро пробера — о ноде это не говорит ничего).
210+
Вердикты: `net_unreachable`, `handshake_failed`, `auth_rejected`, `tunnel_no_data`, `degraded`, `core_down` (отказало собственное ядро пробера, проблема на его хосте).
211211

212212
<details>
213213
<summary>📖 Пример: почему жалуются на конкретную ноду</summary>

docs/probes.md

Lines changed: 21 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -2,25 +2,21 @@
22

33
> Read this in [Russian](probes.ru.md).
44
5-
A probe is a small Go binary you install on **your own** server. It gets a hidden subscription from the panel, starts a real sing-box core, and connects to your nodes exactly like a customer would — then reports what actually happened.
5+
A probe is a small Go binary you install on **your own** server. It gets a hidden subscription from the panel, starts the same core the Click Connect clients run, connects to your nodes exactly like a customer would and reports the result.
66

77
---
88

99
## 📖 Why This Exists
1010

11-
The panel already knows whether a node agent is alive, whether the process is running and how much traffic was accounted. None of that answers the question customers actually ask:
12-
13-
> "Why can't I connect from my ISP?"
14-
15-
A node can be perfectly healthy from the panel's point of view while:
11+
The panel sees its own control plane: whether a node agent is alive, whether the process is running and how much traffic was accounted. A node can look healthy there while:
1612

1713
- the port is filtered by a specific carrier,
1814
- the REALITY masquerade destination is dead, so the TLS handshake never completes,
1915
- the user was never pushed to the running Xray instance, so credentials are refused,
2016
- the tunnel is established but no data flows because of a broken outbound or an ACL rule,
2117
- the exit IP landed on a blacklist, so a specific resource is blocked.
2218

23-
A probe reproduces the client path end to end and names which of these it is.
19+
A probe walks the client path end to end and names which of these happened.
2420

2521
---
2622

@@ -48,7 +44,7 @@ A probe reproduces the client path end to end and names which of these it is.
4844
1. **Enrollment.** The panel issues a single-use token valid for 24 hours. The probe exchanges it once for a permanent token; only SHA-256 hashes are stored for verification.
4945
2. **Profile.** The probe asks what to check: nodes, inbounds, the resource checklist, cadence and the speed-test budget.
5046
3. **Subscription.** Every probe owns a hidden user with its own subscription. That user is excluded from all listings and statistics, so probe traffic never shows up as customer traffic.
51-
4. **Checks.** One local SOCKS listener is opened per checked inbound, which is what pins a measurement to exactly one node inbound rather than to whatever the balancer picked.
47+
4. **Checks.** One local SOCKS listener is opened per checked inbound, so every measurement is pinned to exactly one node inbound.
5248
5. **Reporting.** Results are aggregated locally into windows and shipped gzipped. Undelivered batches are spooled on disk, so a panel outage costs no measurements.
5349

5450
The panel never connects to a probe: everything is initiated by the probe, so it works behind NAT and on residential links.
@@ -65,7 +61,7 @@ Then **Probes → Add probe**, give it a name that describes the vantage point (
6561

6662
```bash
6763
curl -fsSL https://github.com/ClickDevTech/CELERITY-panel/releases/latest/download/celerity-probe-install.sh \
68-
| PANEL_URL='https://panel.example.com' ENROLL_TOKEN='<token>' sh
64+
| sudo PANEL_URL='https://panel.example.com' ENROLL_TOKEN='<token>' sh
6965
```
7066

7167
**Windows (elevated PowerShell):**
@@ -75,7 +71,9 @@ $env:PANEL_URL='https://panel.example.com'; $env:ENROLL_TOKEN='<token>'
7571
irm https://github.com/ClickDevTech/CELERITY-panel/releases/latest/download/celerity-probe-install.ps1 | iex
7672
```
7773

78-
The installer downloads the probe and a sing-box core, enrolls once, and registers a service (systemd, launchd or a Windows service). The enrollment token is passed through the environment, never on the command line, because the process table is readable by other local users.
74+
The installer downloads the probe and a core, enrolls once, and registers a service (systemd, launchd or a Windows service). The enrollment token is passed through the environment, never on the command line, because the process table is readable by other local users.
75+
76+
The core is [sing-box-lx](https://github.com/Leadaxe/sing-box-lx) — the same build the Click Connect apps use. It is upstream sing-box plus the transports the panel can publish; upstream refuses an entire configuration that contains an XHTTP node, so a probe on upstream would go blind on every node at once. An install that finds an upstream core in place replaces it, and the probe writes a warning to its log if the core it starts lacks `with_xhttp` while the fleet has XHTTP nodes. Set `CORE_REPO` before running the installer to pull the core from somewhere else.
7977

8078

8179
| Platform | Service | Data directory |
@@ -103,15 +101,15 @@ The installer downloads the probe and a sing-box core, enrolls once, and registe
103101
| Resource checklist | empty | One `id` + `url` pair per row, checked through every node |
104102

105103

106-
Speed measurement burns real traffic on your own nodes, which is why it is bounded three ways and spends its daily budget round-robin across nodes instead of letting one node consume everything.
104+
Speed measurement burns real traffic on your own nodes, so it is bounded three ways, and its daily budget is spent round-robin across nodes.
107105

108-
**Scale note.** The number of stored series is *probes × nodes × resources*. The panel warns when that product grows past a few thousandat that point lengthen the intervals rather than adding probes.
106+
**Scale note.** The number of stored series is *probes × nodes × resources*. The panel warns when that product grows past a few thousand; at that point lengthen the intervals.
109107

110108
---
111109

112110
## 🔍 Reading the Results
113111

114-
Results are always presented **per vantage point**, and probe verdicts deliberately never change `node.status`. With a single probe there is no way to tell "the node is down" from "the probe's own uplink is broken" — that distinction needs a quorum of probes on different networks.
112+
Results are always presented **per vantage point**, and probe verdicts never change `node.status`. A failed check can also come from the probe's own uplink; telling the two apart needs a quorum of probes on different networks.
115113

116114
Failure codes map directly to an action:
117115

@@ -123,21 +121,21 @@ Failure codes map directly to an action:
123121
| `auth_rejected` | Tunnel stands, credentials refused | User not pushed to the running core — a sync problem |
124122
| `tunnel_no_data` | Authenticated, but nothing flows | Broken outbound or an ACL rule |
125123
| `degraded` | Works, but slowly | Overload, or a bad route to this vantage point |
126-
| `core_down` | The probe's own sing-box was not running | Fix the probe host; this says nothing about the node |
124+
| `core_down` | The probe's own sing-box was not running | The fault is on the probe host |
127125

128126

129-
Resource results are separate on purpose: a blocked resource means a geo-block or a blacklisted exit address, **not** an outage. Only 403 and 451 are treated as a block; a 500 or a transport error is recorded as a failed check without blaming a geo-restriction.
127+
Resource results are kept separate: a blocked resource points to a geo-block or a blacklisted exit address. Only 403 and 451 count as a block; a 500 or a transport error is recorded as a failed check.
130128

131129
Two warnings worth acting on:
132130

133-
- **Same host.** If a probe's egress IP matches one of your nodes, its checks of that node prove nothing — it never leaves the machine. The UI flags this.
131+
- **Same host.** If a probe's egress IP matches one of your nodes, its traffic to that node never leaves the machine. The UI flags this.
134132
- **Virtual nodes.** A virtual node is a `urltest` group. Both the group and its leaves are checked, and the group result records which leaf the balancer actually picked.
135133

136134
---
137135

138136
## 🪝 Alerts
139137

140-
Webhooks fire on state transitions, not once per window, so an alert arrives without waiting for the rollup:
138+
Webhooks fire on state transitions, so an alert arrives without waiting for the rollup:
141139

142140

143141
| Event | Fired when |
@@ -153,17 +151,17 @@ A local core failure (`core_down`) never raises a node alert.
153151

154152
## 🤖 AI Assistant
155153

156-
The `query_probes` MCP tool exposes the same data to an AI client. It requires the `probes:read` scope probe data reveals vantage points and egress addresses, so it does not ride on general statistics access. See the [MCP guide](mcp-user-guide.md).
154+
The `query_probes` MCP tool exposes the same data to an AI client. It requires its own `probes:read` scope, because probe data reveals vantage points and egress addresses. See the [MCP guide](mcp-user-guide.md).
157155

158156
---
159157

160158
## 🔐 Security Model
161159

162160
- A probe holds **client credentials only**. It has no rights on any node, no SSH access and no panel session.
163161
- Tokens are verified against SHA-256 hashes and compared in constant time. The permanent token is additionally stored encrypted so the panel can re-display an install command.
164-
- Deleting a probe removes its hidden user, its subscription and its results immediately, and pushes the removal to the running Xray instances. Since the credentials are plain client credentials, revocation speed is what bounds their lifetime — there is no IP or ASN pinning.
165-
- The traffic cap on the hidden user is the second bound: a leaked probe subscription cannot be used for free traffic beyond it.
166-
- Ingest is idempotent. A redelivered batch is acknowledged without being stored twice, which is what makes at-least-once shipping safe.
162+
- Deleting a probe removes its hidden user, its subscription and its results immediately, and pushes the removal to the running Xray instances. There is no IP or ASN pinning, so revocation speed is what bounds the lifetime of leaked credentials.
163+
- The traffic cap on the hidden user is the second bound: a leaked probe subscription is limited to that amount of traffic.
164+
- Ingest is idempotent. A redelivered batch is acknowledged without being stored twice, which makes at-least-once shipping safe.
167165

168166
---
169167

@@ -175,11 +173,11 @@ The `query_probes` MCP tool exposes the same data to an AI client. It requires t
175173
| Probe stays "awaiting installation" | The enrollment token expired (24 h) or was already used — reissue it from the panel |
176174
| Every node reports `core_down` | sing-box is missing or not executable on the probe host; check the service log |
177175
| Every node reports `auth_rejected` | The probe user did not reach the nodes — run a sync and check the node status |
178-
| One node reports `net_unreachable`, others are fine | Port filtered on the path from this vantage point — that is exactly what the probe is for |
176+
| One node reports `net_unreachable`, others are fine | Port filtered on the path from this vantage point |
179177
| No data after install | The feature is disabled in settings, or the probe cannot reach `PANEL_URL` over HTTPS |
180178

181179

182-
Logs: `journalctl -u celerity-probe -f` (Linux), `tail -f /usr/local/var/celerity-probe/probe.log` (macOS), the service log in the data directory (Windows).
180+
Logs: `sudo journalctl -u celerity-probe -f` (Linux — without root the journal shows nothing), `tail -f /usr/local/var/celerity-probe/probe.log` (macOS), the service log in the data directory (Windows).
183181

184182
---
185183

0 commit comments

Comments
 (0)