You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Maintenance][Alerts] Post-window verification: alert (and ticket) when a device that was online before a maintenance window is still offline after it #5236
Split from #4981 (mega-issue), and the maintenance-window half of #4330. After a scheduled reboot or maintenance window, nothing in Breeze checks that the device came back:
Alert if a server that was previously online is offline at the end of that maintenance window.
When the job runs it creates a ticket to say the server is rebooting, then monitors and waits for the RMM to show it online again. If online it closes the ticket; if offline for more than 20 min it sends an urgent alert via Teams.
Today maintenanceRebootWorker.ts only issues reboots to online devices with a pending-reboot flag during an active window; there is no window-close hook, no "was online before / still offline after" comparison, and no post-maintenance alert or ticket. The generic offline alert (alertConditions/handlers/offline.ts) is typically suppressed during the window, so a server that never returns is silent until someone notices.
Proposed Fix
Snapshot at window open: record which targeted devices were online when the window (or reboot batch) started.
Verify at window close (+ grace): for each device that was online at open and has not heartbeated by windowEnd + graceMinutes (default 20), raise a maintenance_reboot_failed alert at high/critical severity through the normal alert pipeline (so Teams/email/webhook routing and escalation apply, and automations can trigger on it).
Description
Split from #4981 (mega-issue), and the maintenance-window half of #4330. After a scheduled reboot or maintenance window, nothing in Breeze checks that the device came back:
Today
maintenanceRebootWorker.tsonly issues reboots to online devices with a pending-reboot flag during an active window; there is no window-close hook, no "was online before / still offline after" comparison, and no post-maintenance alert or ticket. The generic offline alert (alertConditions/handlers/offline.ts) is typically suppressed during the window, so a server that never returns is silent until someone notices.Proposed Fix
windowEnd + graceMinutes(default 20), raise amaintenance_reboot_failedalert at high/critical severity through the normal alert pipeline (so Teams/email/webhook routing and escalation apply, and automations can trigger on it).Affected Files
apps/api/src/jobs/maintenanceRebootWorker.tsapps/api/src/services/alertConditions/(new handler) or a dedicated post-maintenance jobapps/api/src/db/schema/configurationPolicies.ts(grace + verification flags onconfigPolicyMaintenanceSettings)apps/web/src/components/configurationPolicies/featureTabs/MaintenanceTab.tsxReported By
stressedout9064 on Discord (#feature-suggestions, 2026-09-04 and 2026-08-30) — split from #4981; overlaps #4330 (reboot-and-monitor script). Related: #5235 (window ordering/dependency).