fix: correct MOD namespace reason normalization and rule matching#4
Conversation
Bridge: add NS/VI/AI aliases to _normalize_report_reason so moderation-service kind 1984 reports map to canonical values (nudity, violence, ai_generated). Rule: fix ModerationServiceFlag to match normalized canonical values instead of 'ai_generated'/'deepfake'/'self_harm'/'offensive' (which never matched). Remove BanNostrEvent -- moderation-service uses ['p', sha256] not a real pubkey, so ReportedPubkey/ReportedEventId are unusable for enforcement. Signal only: flag_for_review + ai_classified label. Enforcement stays in ai_classification.sml and label_routing.sml which have real Nostr identifiers.
Osprey rule names become columns in the osprey.osprey_events ClickHouse table. Renaming the rule to ModerationServiceFlag without a corresponding ALTER TABLE made every worker flush fail with "Unrecognized column 'ModerationServiceFlag'", blocking ALL event ingestion (not just kind 1984). Keep the rule name ModerationServiceBan for now — the column already exists and output sink writes succeed. Signal-only semantics (flag_for_review verdict, no BanNostrEvent) are preserved. Follow-up: land a paired column + rule rename via iac-coreconfig + osprey PRs once coordination is in place. Discovered during staging validation of this PR: divine/staging-iterate deploy at 2026-04-17 16:06 UTC, rolled back at 16:52 after ~45 min of failed ClickHouse flushes.
dcadenas
left a comment
dcadenas
left a comment
There was a problem hiding this comment.
Both bugs are fixed correctly with well-justified commentary. The bridge now normalizes MOD namespace labels (NS/VI/AI) into the canonical vocabulary the SML rule actually matches, and the rule no longer calls BanNostrEvent with the sha256-video-hash pubkey and empty event ID that moderation-service kind 1984 produces. The third commit intentionally retains the ModerationServiceBan rule name to keep the ClickHouse osprey_events.ModerationServiceBan column stable — a conscious trade-off that deviates from the PR description's proposed ModerationServiceFlag rename and is documented inline. The ZendeskSink docstring is scope creep relative to the title but doc-only. Only nits below.
Inline comments (outside PR diff):
- divine/nostr-kafka-bridge/main.py:33 — Doc drift: the comment lists canonical values as
csam, nudity, spam, impersonation, illegal, harassment, other, but this PR introducesviolenceandai_generatedas new canonical outputs of_normalize_report_reason. Please add them here (and ideally indivine/rules/rules/reports/auto_hide.sml:5, which also enumerates the canonical set) so the next reader has the full picture.
| # The bridge receives them lowercased after strip().lower() in _normalize_report_reason. | ||
| 'ns': 'nudity', | ||
| 'vi': 'violence', | ||
| 'ai': 'ai_generated', |
There was a problem hiding this comment.
'ai': 'ai_generated' is a very short alias key. Any client that happens to send a freeform reason of literally ai will now be rewritten. That's almost certainly fine given the deliberate vocab of the clients we know about, but it's a lossy mapping that's easy to overlook. A one-line comment clarifying that this mapping is MOD-namespace-specific (and why there's no risk of collision with other clients) would make the assumption explicit.
| # Naming note: the rule is still called `ModerationServiceBan` to match the | ||
| # existing `ModerationServiceBan` column in the `osprey.osprey_events` | ||
| # ClickHouse schema. Rule names become ClickHouse columns, so renaming the | ||
| # rule without a coordinated ALTER TABLE breaks every output sink flush. |
There was a problem hiding this comment.
The schema-compat reason for keeping the ModerationServiceBan name is good, but the follow-up (coordinated ALTER TABLE + rename) is deferred with no tracking link. Please reference an issue number or TODO owner so this doesn't become a forgotten semantic mismatch. Rule name → column name coupling is the kind of thing a future author will trip over when they try to rename it cleanly.
| then=[ | ||
| BanNostrEvent(event_id=ReportedEventId, pubkey=ReportedPubkey, reason='Content flagged by moderation service'), | ||
| DeclareVerdict(verdict='auto_ban'), | ||
| LabelAdd(entity=EventId, label='ai_classified'), |
There was a problem hiding this comment.
Worth calling out in the rule comment: LabelAdd(entity=EventId, label='ai_classified') attaches the label to the 1984 report event itself (moderation-service's own event), not to the video it's reporting — because the content event ID isn't recoverable from this tag structure. That's consistent with the 'signal only' framing, but readers might assume EventId refers to the content being flagged. One clarifying line would prevent confusion.
| a tag/field convention for linking tickets to Nostr events. | ||
| Not implemented yet -- would need to match the relay-manager | ||
| pattern (zendesk_tickets D1 table maps event_id to ticket_id). | ||
| """Log resolution verdicts. Ticket resolution is not yet implemented. |
There was a problem hiding this comment.
The expanded architecture notes are useful, but this change is unrelated to the PR title (MOD namespace normalization / rule matching). Consider splitting into a separate docs-only commit/PR, or at minimum flagging in the PR description that the ZendeskSink change is scoped bundled. Not a blocker — the content is accurate and self-contained.
Addresses review nit from dcadenas on #4. The normalization changes in this PR introduced 'violence' and 'ai_generated' as new canonical outputs of _normalize_report_reason, but the inline comments in main.py and auto_hide.sml still listed the pre-change canonical set. Update both so the next reader has the full picture.
2 participants
Description
Fixes two bugs that prevented the moderation-service kind 1984 signal path from working. The bridge was passing MOD namespace labels (
NS,VI,AI) through without normalization so they never matched SML rule conditions. The rule was also callingBanNostrEventwith identifiers that moderation-service populates with a sha256 hash and empty string — not valid for enforcement.Type of Change
Changes
ns→nudity,vi→violence,ai→ai_generatedto_normalize_report_reasonso MOD namespace labels map to canonical values the rules expectModerationServiceFlag, fixReportReasonmatch values, removeBanNostrEvent(moderation-service kind 1984 uses['p', sha256]— no valid event ID or pubkey), change toflag_for_review+ai_classifiedlabelai_classification.smlandlabel_routing.smlwhich have real Nostr identifiers