-
Notifications
You must be signed in to change notification settings - Fork 561
feat: Mixpanel cohort sync webhook #8338
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: feat/cohort-sync-amplitude
Are you sure you want to change the base?
Changes from all commits
d636359
753f5db
e37b7cf
7caa299
aab49fc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # Generated by Django 5.2.16 on 2026-08-20 10:06 | ||
|
|
||
| from django.db import migrations, models | ||
|
|
||
|
|
||
| class Migration(migrations.Migration): | ||
|
|
||
| dependencies = [ | ||
| ("cohorts", "0003_cohort_sync_key"), | ||
| ("environments", "0039_use_no_ssrf_url_field"), | ||
| ("segments", "0032_add_segment_rules_data"), | ||
| ] | ||
|
|
||
| operations = [ | ||
| migrations.AddField( | ||
| model_name="cohort", | ||
| name="external_id", | ||
| field=models.CharField(blank=True, max_length=255, null=True), | ||
| ), | ||
| migrations.AlterField( | ||
| model_name="cohort", | ||
| name="source_type", | ||
| field=models.CharField( | ||
| choices=[ | ||
| ("csv", "CSV"), | ||
| ("amplitude", "Amplitude"), | ||
| ("mixpanel", "Mixpanel"), | ||
| ], | ||
| default="csv", | ||
| max_length=50, | ||
| ), | ||
| ), | ||
| ] |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -60,6 +60,27 @@ class AmplitudeListSerializer(serializers.Serializer[None]): | |||||||||||||||||||||||||||
| name = serializers.CharField(max_length=2000) | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| class MixpanelMemberSerializer(serializers.Serializer[None]): | ||||||||||||||||||||||||||||
| # Length mirrors CohortMembership.identifier. | ||||||||||||||||||||||||||||
| mixpanel_distinct_id = serializers.CharField(max_length=2000) | ||||||||||||||||||||||||||||
|
Comment on lines
+63
to
+65
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win Reject identifiers that exceed the identity-store byte limit. Line 65 validates 2000 characters. The downstream identity store permits only 1024 bytes. A multibyte identifier can pass validation, receive a success response, and then fail during identity processing. Validate the UTF-8 byte length before calling the membership service. Proposed validation class MixpanelMemberSerializer(serializers.Serializer[None]):
# Length mirrors CohortMembership.identifier.
mixpanel_distinct_id = serializers.CharField(max_length=2000)
+
+ def validate_mixpanel_distinct_id(self, value: str) -> str:
+ if len(value.encode("utf-8")) > 1024:
+ raise serializers.ValidationError(
+ "Ensure this field is no longer than 1024 bytes."
+ )
+ return value📝 Committable suggestion
Suggested change
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Deferred for now — tracked as a TODO together with the same byte-length check for Amplitude user_ids and CSV uploads. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
🐇 🧠 Learnings usedYou are interacting with an AI system. |
||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| class MixpanelParametersSerializer(serializers.Serializer[None]): | ||||||||||||||||||||||||||||
| mixpanel_cohort_id = serializers.CharField(max_length=255) | ||||||||||||||||||||||||||||
| mixpanel_cohort_name = serializers.CharField(max_length=2000) | ||||||||||||||||||||||||||||
| # An empty page is valid: a first sync of an empty cohort has no members. | ||||||||||||||||||||||||||||
| members = MixpanelMemberSerializer(many=True, allow_empty=True) | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| class MixpanelWebhookSerializer(serializers.Serializer[None]): | ||||||||||||||||||||||||||||
| # "members" carries the full membership on the first sync; | ||||||||||||||||||||||||||||
| # "add_members"/"remove_members" carry changes since the last sync. | ||||||||||||||||||||||||||||
| action = serializers.ChoiceField( | ||||||||||||||||||||||||||||
| choices=["members", "add_members", "remove_members"] | ||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||
| parameters = MixpanelParametersSerializer() | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| class CohortSyncMembersSerializer(serializers.Serializer[None]): | ||||||||||||||||||||||||||||
| # Child length mirrors CohortMembership.identifier. | ||||||||||||||||||||||||||||
| # TODO: this counts characters, but identity data is stored with a | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,11 +1,15 @@ | ||
| from django.urls import path | ||
| from rest_framework.routers import SimpleRouter | ||
|
|
||
| from cohorts.sync_views import AmplitudeCohortSyncViewSet | ||
| from cohorts.sync_views import AmplitudeCohortSyncViewSet, MixpanelCohortSyncView | ||
|
|
||
| app_name = "cohort-sync" | ||
|
|
||
| # SimpleRouter: nothing here is browsed by a person. | ||
| router = SimpleRouter() | ||
| router.register(r"amplitude/lists", AmplitudeCohortSyncViewSet, basename="amplitude") | ||
|
|
||
| urlpatterns = router.urls | ||
| urlpatterns = [ | ||
| path("mixpanel/webhook/", MixpanelCohortSyncView.as_view(), name="mixpanel"), | ||
| *router.urls, | ||
| ] |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Make active cohort lookup and creation atomic.
Concurrent initial
membersrequests can both observe no cohort and create separate active cohorts for the same environment, source type, and external ID. Later incremental requests may select only one duplicate, splitting memberships between cohorts.Add a partial unique constraint for active cohorts with non-null external IDs, and use an atomic get-or-create flow that re-fetches after an integrity conflict.
📍 Affects 2 files
api/cohorts/models.py#L32-L36(this comment)api/cohorts/sync_views.py#L140-L153There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deliberately dropped to keep v1 lean — we'll add the constraint and conflict recovery if duplicate creation is observed in practice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gagantrivedi, understood. The concurrency risk remains deferred from this PR.Would you like me to create a follow-up GitHub issue for the active-cohort uniqueness constraint and atomic conflict recovery?
🐇
✏️ Learnings added
You are interacting with an AI system.