Repository navigation
alerting: resolve infra alerts by editing the open Slack message in place - #140
Merged
Merged
Conversation
…lace A resolve used to post a second message, so every episode left a stale-looking alert plus a separate resolution note in the channel. Now the dispatcher pairs infra's '<base>:open' / '<base>:resolve' delivery IDs: when the open record was delivered via bot token and has a slack_ts, the resolve chat.update's that message with the original alert text plus the resolution banner, and the resolve record is marked delivered against the same ts. Fallbacks: no delivered paired open (shadow-era open, webhook destination) or a permanent update failure (deleted message) post a fresh message as before; transient update failures retry via the existing outbox backoff. - SlackDeliveryPort.update_message: chat.update with the same transient/permanent classification as postMessage - runtime: _deliver_or_update pairing helper in the dispatch path - RecordingSlackPort records updates; protocol gains update_message Signed-off-by: Thang Nguyen <thang.nguyen@inferact.ai>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This was referenced Sep 4, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
An infra episode used to produce two Slack messages: the 🚨 alert, then a separate ✅ resolve. Now the bot edits its own open alert when the episode resolves — the message keeps the original alert text and gains the resolution banner appended below it, so the channel never has stale-looking open alerts.
How
<base>:open/<base>:resolveand the outbox already stores each message'sslack_ts.SlackDeliveryPort.update_message→ Slackchat.update, with the same transient/permanent error classification aschat.postMessage.runtime.py) pairs a…:resolverecord with its delivered…:openrecord and edits it; the resolve row is marked delivered against the original ts.Tests
test_outbox.py: pairing edits in place (no second post), missing open → fresh post, permanent update failure → fresh-post fallbacktest_slack.py:chat.updateURL/payload contract,message_not_foundclassified permanenttest_infra.py: episode tests updated for the new one-message behaviorpytest alerting202 passed, ruff clean, mypy clean except the known local-venv boto3/stub quirks (identical without this change)Deploy
Takes effect on the worker after the next
install.shfrommain. No schema or config changes.Signed-off-by: Thang Nguyen thang.nguyen@inferact.ai