Skip to content

sync identityKey merge crosses type/source boundaries when a downstream mem record shares the identity value #194

Description

@ameet

Summary

A sync profile with a singular identityKey merges/replaces any record resolving to the same identity value, with no scoping by type or by which system created the record. This has now hit us twice: once inside a sync mirror (documented below), and once against curated data in one mem that only shares an identity value with an Attio-sourced record by coincidence of business logic.

First occurrence (already known, documented on our side)

google-calendar_events.json used a singular organizer.email identityKey. 20,136 calendar events fetched per run survived as 89 mirror records, with 19,330 event ids landing inside Attio PEOPLE records that happened to share an organizer.email value — collapsing unrelated calendar and CRM data into one row regardless of type. Fixed on our side by switching to the plural identityKeys form (writes to keys[] without merging).

Second occurrence (mem write path, not just the sync mirror)

Our attioCompanies sync profile (.one/sync/profiles/attio_attioCompanies.json) sets:

"identityKey": "domain"

We separately maintain curated company records in one mem (a different store than the sync mirror) that carry the same domain value as their Attio counterpart by design — that shared domain is how we cross-reference the two. We assumed the identity-merge behavior was scoped to the sync mirror (.one/sync/data/*.db). It is not: the sync engine's mem write path (mem_upsert_by_keys(p_replace=true), confirmed via a comment in our own schema tooling that documents it as "always UPDATEs every matched row" — i.e. a full replace by key, not a merge-patch) applies the same singular-identityKey collapse when it upserts into mem_records.

Result: on one sync run attio, a curated company record whose domain matches an incoming Attio company record gets wholesale-replaced by the raw Attio payload. Fields our own tooling had written onto that record — an attio_id crosswalk key we maintain (not part of Attio's own schema) among others — don't survive the replace, because the incoming Attio-sourced payload never had that field to begin with. From our side this looked like a silent "delinker": a field that existed before a sync run was fully gone after it, with no log line pointing at the sync run as the cause.

Live-confirmed on our data (2026-08-31): two curated company records lost their attio_id key entirely (not set to null/empty — absent) after a sync run, on records where the underlying Attio company's domain matched. We ruled out our own application-level mem write paths (mem add raises loudly on a key conflict; mem update/mem import are merge-patch and preserve omitted fields) before concluding the sync engine's identity-merge was the only mechanism that could produce a fully-absent key.

Ask

Scope identity-merge to the type of the incoming record, not just the identity value. A domain, email, or similar identity value can legitimately collide across two records of different provenance/type without those records being "the same entity" for merge purposes — the calendar/PEOPLE case above is the same failure mode, just inside the sync mirror instead of mem_records. If type-scoping isn't feasible generally, at minimum: document that a singular identityKey merge is not confined to the sync profile's own mirror table, and can overwrite records elsewhere (e.g. one mem) that happen to share the identity value — that scope was not obvious from the sync profile docs and cost us real curated data.

Happy to share the exact record ids / more detail on request.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions