kg_add rejects commas and slashes in object field with no documentation

[maximillionfrday]

Bug: kg_add silently rejects common characters in the object field

What happened

Calling mempalace_kg_add with an object value containing commas or forward slashes returns a generic error with no explanation of what characters are allowed.

Error message received

object contains invalid characters
object contains invalid path characters

Reproduction

Works:

• "net income 394402.33 in 2021"
• "filed on 2021-01-01 status MFS"

Fails with "invalid characters":

• "income 1305501, expenses 1184062, MFS filing status"
• "2021: charitable 8000, mileage 22456 miles"

Fails with "invalid path characters":

• "filed 2021/01/01"

Expected behavior

Either commas and slashes should be allowed in the object field, or the error message should explicitly state which characters are permitted.

Environment

• MemPalace v3.3.0
• Windows 11
• MCP tool via Claude Code

[yuichiro-nishi]

v3.3.0 input validation: more characters affected + kg_invalidate deadlock + retroactive blocking

I hit the same problem on a research palace (~420 entities, 293 triples) and want to add findings that may help scoping the fix.

Full forbidden character set (verified on v3.3.0, Win11)

Beyond commas and slashes, the following characters all return entity contains invalid characters:

Brackets: ( ) [ ] { } < > （ ） ＜ ＞ 「 」 『 』
Operators: = + × ÷ → ← ⇔ ≠ ≥ ≤ ± ∈ ⊂ / ^ % * & | ＝ ＋
Separators: , ， 、 ; : ： 。 — ─ 〜
Quotes/special: " § ① ② ③ …

/ returns the distinct invalid path characters message (path-traversal layer).

Characters that DO pass (this is the good news)

• ASCII alphanumerics + _ - . ' + space
• CJK (Japanese kanji/hiragana/katakana, Chinese)
• Greek letters (α β γ δ ε λ μ ν ...)
• Latin-extended diacritics (é ö ü, etc.) — Unicode / diacritics rejected in sanitize_name() for KG + MCP writes #637 appears fixed in v3.3.0

Critical: kg_invalidate is blocked by the same validator

This is a retroactive deadlock for users who upgraded from earlier versions:

mempalace_kg_invalidate(subject="gender concept (70)", predicate=..., object=...)
# → {"success": false, "error": "subject contains invalid characters"}

Legacy entities created in v3.0/v3.1 with parens/commas/colons cannot be invalidated through the API. The only escape is direct SQLite editing of ~/.mempalace/knowledge_graph.sqlite3 (UPDATE both entities.id and triples.subject/object in a transaction).

Suggested fix directions (in addition to the documentation fix this issue requests)

1. Document the allowed character set explicitly in error messages: entity contains invalid characters: only [a-zA-Z0-9_\-.' ] + Unicode letters allowed
2. Loosen kg_invalidate / read-side validation — these don't have the same injection risk as kg_add, and need to handle legacy data
3. Ship a mempalace migrate-entities --rename-rule sanitize CLI to help users rescue legacy data without writing custom SQL
4. Consider a Unicode-letter-aware whitelist similar to the fix from Unicode / diacritics rejected in sanitize_name() for KG + MCP writes #637 (jphein's \w regex), but with explicit symbol denylist for path-traversal protection

Adjacent design issue exposed during migration

Auditing my palace, I found 100/420 entities (~24%) were full sentences (e.g. 366-char descriptions stored as object values). They were always referenced by exactly 1 triple — the symptom of kg_add being misused for subject -[predicate]-> "long_descriptive_sentence" instead of subject -[predicate]-> "short_concept_id" + add_drawer(content="long sentence").

Proposal: emit a soft warning in kg_add when len(object) > 80 chars: Hint: object looks like prose — consider storing it in a drawer and using a short concept ID here.

Reproducible sanitization function (sharing in case useful)

import unicodedata, re
CONNECT = set('()<>[]{}（）＜＞「」『』+×÷→←⇔≠≥≤±∈∋⊂⊃∪∩/^%*&|＋')
SEPARATE = set(',，、;:：。—─〜=＝')
DELETE = set('"§')

def sanitize(s):
    s = unicodedata.normalize('NFKC', s)
    out = []
    for ch in s:
        if ch in DELETE: continue
        if ch in CONNECT: out.append('_'); continue
        if ch in SEPARATE: out.append(' '); continue
        if ch in "_-.' ": out.append(ch); continue
        cat = unicodedata.category(ch)
        if cat.startswith('L') or cat.startswith('N'):
            out.append(ch)
        else:
            out.append('_')
    raw = re.sub(r'_+', '_', ''.join(out))
    words = [w.strip('_') for w in raw.split(' ') if w.strip('_')]
    return re.sub(r' +', ' ', ' '.join(words)).strip()

This is deterministic + idempotent (verified across 420 entities, 0 collisions, 0 empty results). Used to migrate the palace before the underlying fix lands.

Happy to share the migration scripts (Phase 0-5: audit → sanitize → SQLite rename → kg-index drawer regen → long-entity shortening) if useful for testing the fix.

Environment

• mempalace 3.3.0 (pip)
• Windows 11 + Python 3.13 + Claude Code MCP