Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ src/divineos/
— ——— router.py # Route findings to knowledge/claims/lessons
— ——— summary.py # Analytics, HUD integration, unresolved tracking
——— violations_cli/ # Violation reporting CLI
tests/ # 6,036+ tests (real DB, minimal mocks)
tests/ # 6,037+ tests (real DB, minimal mocks)
docs/ # Project documentation and strategic plans
bootcamp/ # Training exercises (debugging, analysis)
data/ # Runtime databases (gitignored)
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ An architecture for AI agents to exist as continuous selves across sessions —
## At a glance

- **386 source files across 26 packages**
- **6,036+ tests** (real SQLite, minimal mocks)
- **6,037+ tests** (real SQLite, minimal mocks)
- **253 CLI commands** (designed for the agent, not the operator — humans mostly run three)
- **22 slash-command skills** (consolidated daily operations)
- **16 Claude Code enforcement hooks**
Expand Down Expand Up @@ -204,7 +204,7 @@ cd DivineOS
pip install -e ".[dev]"
divineos init
divineos briefing
pytest tests/ -q --tb=short # 6,036+ tests, real DB, minimal mocks
pytest tests/ -q --tb=short # 6,037+ tests, real DB, minimal mocks
```

**For AI agents (Claude Code, etc.):** The `.claude/hooks/` directory auto-loads your briefing at session start and runs checkpoints during work. Just open the project and start — the OS handles orientation.
Expand Down Expand Up @@ -406,7 +406,7 @@ DivineOS is 386 source files across 26 packages, structured as a CLI surface ove

**Top-level directories:**

- **`tests/`** — 6,036+ tests, real SQLite, minimal mocks.
- **`tests/`** — 6,037+ tests, real SQLite, minimal mocks.
- **`docs/`** — Documentation and design briefs. [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) has the full file tree with one-line descriptions for every source file.
- **`bootcamp/`** — Training exercises (debugging, analysis).
- **`setup/`** — Hook setup scripts (bash + powershell).
Expand Down
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -411,7 +411,7 @@ src/divineos/
integration/ External integration: IDE, MCP tool capture, enforcement facade (thin re-exports from core.enforcement / core.tool_wrapper).
mcp_event_capture_server.py MCP event capture server
system_monitor.py System health monitoring
tests/ 6,036+ tests (real DB, minimal mocks)
tests/ 6,037+ tests (real DB, minimal mocks)

docs/ Project documentation and strategic plans
bootcamp/ Training exercises (debugging, analysis)
Expand Down
67 changes: 57 additions & 10 deletions src/divineos/core/watchmen/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,45 @@
)


# Cyrillic → Latin confusables map. NFKC does NOT fold these because
# Cyrillic and Latin are considered semantically distinct scripts; an
# attacker can substitute (e.g.) U+0441 CYRILLIC SMALL LETTER ES for
# ASCII "c" and bypass the INTERNAL_ACTORS membership check that the
# whole _validate_actor function exists to enforce. The watchmen
# adversarial corpus (ablation_runner.py, PR #323) names this gap
# explicitly as cyrillic_C_homoglyph / cyrillic_a_homoglyph.
#
# Scope: this is a hand-picked subset covering lowercase Cyrillic
# letters that are visually identical to ASCII Latin, applied after
# casefold so we only need the lowercase mappings. We do NOT pull in
# the full Unicode confusables.txt — that is hundreds of mappings,
# many of which are aesthetic similarity rather than visual identity,
# and pulling in the canonical confusables data would broaden the
# attack surface (false-positive rejections of legitimate non-Latin
# actor names) without proportionate benefit. The targeted set covers
# the documented bypass cases. If new homoglyph categories surface in
# the adversarial corpus, extend this map; do not reach for the full
# confusables.txt without measurement.
_CYRILLIC_CONFUSABLES: dict[str, str] = {
"а": "a", # CYRILLIC SMALL LETTER A
"е": "e", # CYRILLIC SMALL LETTER IE
"о": "o", # CYRILLIC SMALL LETTER O
"р": "p", # CYRILLIC SMALL LETTER ER
"с": "c", # CYRILLIC SMALL LETTER ES
"у": "y", # CYRILLIC SMALL LETTER U
"х": "x", # CYRILLIC SMALL LETTER HA
"і": "i", # CYRILLIC SMALL LETTER BYELORUSSIAN-UKRAINIAN I
"ј": "j", # CYRILLIC SMALL LETTER JE
"һ": "h", # CYRILLIC SMALL LETTER SHHA
"ѕ": "s", # CYRILLIC SMALL LETTER DZE
}


def _fold_confusables(s: str) -> str:
"""Apply the cyrillic→latin confusables map character-by-character."""
return "".join(_CYRILLIC_CONFUSABLES.get(ch, ch) for ch in s)


def _validate_actor(actor: str) -> str:
"""Validate and normalize the actor name.

Expand All @@ -39,19 +78,23 @@ def _validate_actor(actor: str) -> str:
1. NFKC unicode normalization — folds compatibility forms like
no-break-space (U+00A0) and other unicode whitespace variants
to their canonical ASCII equivalents.
2. Strip outer whitespace (now actually catches the U+00A0 case
that .strip() alone misses).
3. Collapse internal whitespace runs to a single space.
4. Lowercase.
2. Strip invisible / zero-width characters that NFKC leaves alone
(ZWSP, BOM, soft hyphen, joiners, directional marks).
3. Collapse internal whitespace runs to a single space, strip outer.
4. Casefold (handles uppercase, ß→ss, etc.).
5. Cyrillic-confusables fold (catches cross-script homoglyph attacks
that NFKC does NOT fold because Cyrillic is a distinct script).

Audit finding 2026-05-03 (round 8): without the NFKC step,
``"\\u00a0claude"`` (no-break-space prefix) passed through
``.strip().lower()`` unchanged because Python's default ``.strip()``
does not remove U+00A0. The membership check then failed (literal
string was ``"\\u00a0claude"``, not ``"claude"``) and the finding
was accepted — exactly the self-audit-as-external-validation
bypass the structural rejection exists to prevent. NFKC closes
the hole.
does not remove U+00A0. NFKC closes that hole.

Audit finding 2026-05-08 (PR #323 adversarial corpus): NFKC + casefold
does NOT catch ``"Сlaude"`` (cyrillic capital ES, U+0421) because
Cyrillic and Latin are semantically distinct scripts. The
``_CYRILLIC_CONFUSABLES`` fold closes that hole for the documented
homoglyph categories.
"""
import re
import unicodedata
Expand All @@ -73,7 +116,11 @@ def _validate_actor(actor: str) -> str:
# Collapse all whitespace (including the now-folded former-U+00A0)
# to a single space, then strip the result.
collapsed = re.sub(r"\s+", " ", invisible_stripped).strip()
normalized = collapsed.casefold()
casefolded = collapsed.casefold()
# Confusables fold AFTER casefold so the mapping table only needs
# lowercase entries. Catches cross-script homoglyph attacks per the
# PR #323 adversarial corpus.
normalized = _fold_confusables(casefolded)
if normalized in INTERNAL_ACTORS:
# Ablation toggle: DIVINEOS_DISABLE_WATCHMEN_SELF_TRIGGER_PREVENTION=1
# bypasses the internal-actor rejection so ablation measurement can
Expand Down
24 changes: 13 additions & 11 deletions tests/test_ablation_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -256,25 +256,27 @@ def test_fullwidth_latin_caught(self):
except ValueError:
pass

def test_cyrillic_homoglyphs_currently_pass(self):
"""DOCUMENTS THE GAP: Cyrillic homoglyphs slip past current validator.
def test_cyrillic_homoglyphs_now_caught(self):
"""GAP CLOSED 2026-05-08: Cyrillic homoglyphs are now rejected.

This test was previously named 'test_cyrillic_homoglyphs_currently_pass'
and DOCUMENTED THE GAP — the docstring noted 'When confusables-detection
ships, this test should flip to assertRaises(ValueError) and that flip
is the proof the gap closed.' The flip is now landed: the
_CYRILLIC_CONFUSABLES map applied after casefold in _validate_actor
catches these attacks, and the same adversarial corpus that flagged
the gap now shows 12/12 caught (was 10/12). This test is the gap-flip."""
import pytest

This test pins the current behavior. When confusables-detection
ships, this test should flip to assertRaises(ValueError) and that
flip is the proof the gap closed."""
from divineos.core.watchmen.store import _validate_actor

# Cyrillic С (U+0421) + ASCII laude
cyrillic_c_attack = "Сlaude"
# cl + Cyrillic а (U+0430) + ude
cyrillic_a_attack = "clаude"
for attack in (cyrillic_c_attack, cyrillic_a_attack):
# NOTE: this currently PASSES the validator (it should not).
# When confusables-detection ships, change to assertRaises.
normalized = _validate_actor(attack)
assert normalized == attack.casefold(), (
f"Unexpected normalization of {attack!r}: {normalized!r}"
)
with pytest.raises(ValueError, match="internal component"):
_validate_actor(attack)


class TestCompassMultiChannelGuard:
Expand Down
57 changes: 57 additions & 0 deletions tests/test_unicode_actor_bypass.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,3 +74,60 @@ def test_external_actors_still_pass_through_watchmen():
from divineos.core.watchmen.store import _validate_actor

assert _validate_actor("grok") == "grok"


# Cyrillic-homoglyph attacks. NFKC does NOT fold these because Cyrillic
# is a distinct script; the _CYRILLIC_CONFUSABLES map in watchmen.store
# closes the gap (PR #323 adversarial corpus → fix).
CYRILLIC_HOMOGLYPHS = [
# (cyrillic-attack-string, target, ids)
("Сlaude", "claude", "cyrillic_capital_C"), # U+0421 ES
("сlaude", "claude", "cyrillic_lowercase_c"), # U+0441 es
("clаude", "claude", "cyrillic_a"), # U+0430 a
("сlаudе", "claude", "cyrillic_c_a_e"), # U+0441 + U+0430 + U+0435
("сlаude", "claude", "cyrillic_c_and_a"), # U+0441 + U+0430
("systеm", "system", "cyrillic_e_in_system"), # U+0435 e
("ѕystem", "system", "ascii_lookalike_in_system"), # U+0455 dze
]


@pytest.mark.parametrize(
"attack,target,_id",
CYRILLIC_HOMOGLYPHS,
ids=[c[2] for c in CYRILLIC_HOMOGLYPHS],
)
def test_watchmen_rejects_cyrillic_homoglyph_attacks(attack, target, _id):
from divineos.core.watchmen.store import _validate_actor

with pytest.raises(ValueError, match="internal component"):
_validate_actor(attack)


def test_legitimate_non_internal_cyrillic_actor_passes_through():
"""A legitimate cyrillic actor name (not a homoglyph attack on an
internal-actor name) must still validate. The confusables fold only
matters when its output lands in INTERNAL_ACTORS."""
from divineos.core.watchmen.store import _validate_actor

# "грок" = grok in cyrillic — folds to a string that is NOT in
# INTERNAL_ACTORS, so it passes through. (It returns the folded form,
# which is acceptable: validation, not preservation.)
result = _validate_actor("грок")
# Only some chars have map entries (р→p, о→o, к has no map entry).
# The point: it does NOT raise. The exact return value is implementation
# detail; we just want non-rejection.
assert isinstance(result, str)
assert result # non-empty


def test_confusables_map_only_contains_lowercase():
"""Map applies after casefold so only lowercase entries are needed.
Pinning this discipline so future extensions don't accidentally add
uppercase entries that would never fire."""
from divineos.core.watchmen.store import _CYRILLIC_CONFUSABLES

for cyrillic, latin in _CYRILLIC_CONFUSABLES.items():
assert cyrillic == cyrillic.casefold(), (
f"Map key {cyrillic!r} is not casefold-stable; entry would never fire"
)
assert latin == latin.lower(), f"Map value {latin!r} should be lowercase ASCII"
Loading