Skip to content

embed pages: page ids up to 2,000 characters, and a first visit creates everything or nothing - #3292

Merged
colinmegill merged 4 commits into
edgefrom
embed-page-id-width
Oct 27, 2026
Merged

colinmegill merged 4 commits into
edgefrom
embed-page-id-width

Conversation

@colinmegill

Copy link
Copy Markdown
Member

What

A page embedding Polis maps to a conversation through its embed key and page id; the first visit to a new page creates the conversation. Until now the page id column held 100 characters while the server inserted the conversation row before storing the id, so every visit to a page with a longer id returned a 500 and left an orphan empty conversation with a join code behind, again on each visit.

  • Migration 000045_page_ids_width: page_ids.page_id and page_ids.site_id widen from varchar(100) to varchar(2000). Metadata-only in Postgres (no rewrite, verified on a populated copy), the unique constraint stays, additive and safe under a rolling replacement, with a 5 s lock timeout. A guarded down migration refuses to narrow if any stored id exceeds the old limit.
  • The embed id parameters are validated at the new limit (1–2,000 characters) and refused with a 400 before any write.
  • The create path runs in one transaction: conversation, page id and join code succeed or fail together; a duplicate-key race between two first visits ends with one conversation and both visitors redirected to it.

Tests

  • Unit: embed id parameter limits.
  • Integration (12 focused, all green): a 150-character id creates once and is reused on the second visit; a 2,001-character id answers 400 and writes nothing; a forced failure in the page-id step leaves no conversation; the duplicate-key race.
  • npm run build, npm run lint, 1,270 unit tests, 1,159 integration tests on a migrated database; old and new server startup against the new schema.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ss9NfEMZS9Qm4FZ2npXDrx

colinmegill and others added 4 commits October 26, 2026 20:20
…es everything or nothing

Every visit to a page with an id over 100 characters returned a 500 and
left an orphan empty conversation. Widen page_ids.page_id and
page_ids.site_id to varchar(2000), keeping the unique site and page pair.

Create the conversation, page mapping and invite in one transaction so
all three succeed or all three roll back. When first visits race, roll
back the losing insert and redirect both visitors to the winner.

Reject missing, empty or overlong site and page ids with a clear 400
before writing anything, while preserving the existing URL parsing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ss9NfEMZS9Qm4FZ2npXDrx
The report builder lists the migrations that sit outside the adoption set; the new migration joins that list and the generated report is refreshed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ss9NfEMZS9Qm4FZ2npXDrx
Measured against a fresh PostgreSQL 17 with the whole migration chain applied: the two page_ids columns widen to 2,000 characters; nothing else changes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ss9NfEMZS9Qm4FZ2npXDrx
…th migration

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ss9NfEMZS9Qm4FZ2npXDrx
@github-actions

Copy link
Copy Markdown
Contributor

Delphi Coverage Report

File Stmts Miss Cover
polismath/init.py 2 0 100%
polismath/benchmarks/bench_pca.py 128 107 16%
polismath/benchmarks/bench_repness.py 81 65 20%
polismath/benchmarks/bench_update_votes.py 38 28 26%
polismath/benchmarks/benchmark_utils.py 34 0 100%
polismath/components/init.py 1 0 100%
polismath/components/config.py 165 70 58%
polismath/conversation/init.py 2 0 100%
polismath/conversation/conversation.py 1293 264 80%
polismath/conversation/group_vote_counts.py 38 2 95%
polismath/conversation/manager.py 131 42 68%
polismath/database/init.py 1 0 100%
polismath/database/dynamodb.py 392 176 55%
polismath/database/postgres.py 414 95 77%
polismath/delphi_storage/init.py 1 0 100%
polismath/delphi_storage/codec.py 295 22 93%
polismath/delphi_storage/embedding_spool.py 58 6 90%
polismath/delphi_storage/golden_corpus.py 60 9 85%
polismath/delphi_storage/import_report.py 51 3 94%
polismath/delphi_storage/legacy_import.py 171 16 91%
polismath/delphi_storage/postgres.py 49 22 55%
polismath/delphi_storage/production_import.py 612 131 79%
polismath/delphi_storage/resource.py 179 32 82%
polismath/delphi_storage/scan_pacing.py 77 0 100%
polismath/delphi_storage/writer.py 196 13 93%
polismath/empty_output.py 29 2 93%
polismath/engine_adapter.py 317 265 16%
polismath/job_child/init.py 364 51 86%
polismath/job_child/census.py 74 9 88%
polismath/job_child/input_bundle.py 101 7 93%
polismath/pca_kmeans_rep/init.py 5 0 100%
polismath/pca_kmeans_rep/clusters.py 257 38 85%
polismath/pca_kmeans_rep/corr.py 98 17 83%
polismath/pca_kmeans_rep/group_k_smoother.py 28 0 100%
polismath/pca_kmeans_rep/legacy_kmeans.py 197 8 96%
polismath/pca_kmeans_rep/pca.py 162 19 88%
polismath/pca_kmeans_rep/repness.py 226 5 98%
polismath/poller/init.py 4 0 100%
polismath/poller/admission.py 318 15 95%
polismath/poller/backfill.py 936 32 97%
polismath/poller/capacity.py 485 27 94%
polismath/poller/capacity_queue.py 248 13 95%
polismath/poller/coordinator_bridge.py 317 317 0%
polismath/poller/math_writer.py 113 3 97%
polismath/poller/promotion.py 112 4 96%
polismath/poller/readiness.py 368 13 96%
polismath/poller/rebuild_child.py 133 5 96%
polismath/poller/service.py 756 54 93%
polismath/poller/worker_pool.py 140 0 100%
polismath/queue/executor.py 221 111 50%
polismath/regression/init.py 4 0 100%
polismath/regression/clojure_comparer.py 188 20 89%
polismath/regression/comparer.py 891 640 28%
polismath/regression/datasets.py 126 27 79%
polismath/regression/recorder.py 36 27 25%
polismath/regression/utils.py 130 85 35%
polismath/replay/init.py 2 0 100%
polismath/replay/attribution_capture.py 49 0 100%
polismath/replay/axis_continuity.py 347 24 93%
polismath/replay/certify.py 999 59 94%
polismath/replay/crosslang.py 128 2 98%
polismath/replay/decision_ties.py 51 8 84%
polismath/replay/diagnostics.py 50 7 86%
polismath/replay/driver.py 73 1 99%
polismath/replay/dsn_admission.py 36 1 97%
polismath/replay/equiv_query.py 10 0 100%
polismath/replay/event_ingress.py 95 1 99%
polismath/replay/fixture_bundle.py 815 62 92%
polismath/replay/fixture_config.py 266 11 96%
polismath/replay/fixture_extract.py 285 12 96%
polismath/replay/fixture_generate.py 103 1 99%
polismath/replay/fixture_samples.py 157 14 91%
polismath/replay/fixture_selection.py 119 0 100%
polismath/replay/fixture_survey.py 147 11 93%
polismath/replay/legacy_pca.py 29 17 41%
polismath/replay/polarity.py 319 25 92%
polismath/replay/poller_equiv.py 776 99 87%
polismath/replay/prodclone.py 193 2 99%
polismath/replay/real_data.py 146 10 93%
polismath/replay/schedule.py 205 10 95%
polismath/replay/served_math.py 159 4 97%
polismath/replay/shard_bench.py 189 65 66%
polismath/replay/stagecompare.py 890 161 82%
polismath/replay/stages.py 256 40 84%
polismath/replay/stepcompare.py 86 1 99%
polismath/replay/store.py 120 11 91%
polismath/replay/tie_capture.py 107 19 82%
polismath/replay/types.py 58 6 90%
polismath/run_math_pipeline.py 284 93 67%
polismath/tools/engine_rebuild.py 230 11 95%
polismath/types.py 6 0 100%
polismath/utils/cli_flags.py 15 0 100%
polismath/utils/clj_hash.py 44 0 100%
polismath/utils/env_flags.py 13 0 100%
polismath/utils/general.py 63 39 38%
polismath/utils/output_profile.py 66 2 97%
polismath/utils/serialization.py 10 5 50%
polismath/utils/vote_convention.py 159 5 97%
umap_narrative/700_datamapplot_for_layer.py 514 470 9%
umap_narrative/701_static_datamapplot_for_layer.py 310 310 0%
umap_narrative/702_consensus_divisive_datamapplot.py 435 275 37%
umap_narrative/801_narrative_report_batch.py 666 218 67%
umap_narrative/803_check_batch_status.py 268 148 45%
umap_narrative/llm_factory_constructor/init.py 2 0 100%
umap_narrative/llm_factory_constructor/model_provider.py 232 106 54%
umap_narrative/narrative_data.py 178 55 69%
umap_narrative/numerical_stages.py 80 14 82%
umap_narrative/polismath_commentgraph/init.py 1 0 100%
umap_narrative/polismath_commentgraph/cli.py 268 268 0%
umap_narrative/polismath_commentgraph/core/init.py 3 3 0%
umap_narrative/polismath_commentgraph/core/clustering.py 108 108 0%
umap_narrative/polismath_commentgraph/core/embedding.py 104 104 0%
umap_narrative/polismath_commentgraph/lambda_handler.py 219 219 0%
umap_narrative/polismath_commentgraph/schemas/init.py 2 0 100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py 80 6 92%
umap_narrative/polismath_commentgraph/tests/conftest.py 17 17 0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py 74 74 0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py 55 55 0%
umap_narrative/polismath_commentgraph/tests/test_storage.py 87 87 0%
umap_narrative/polismath_commentgraph/utils/init.py 9 7 22%
umap_narrative/polismath_commentgraph/utils/converter.py 283 50 82%
umap_narrative/polismath_commentgraph/utils/group_data.py 332 279 16%
umap_narrative/polismath_commentgraph/utils/storage.py 609 481 21%
umap_narrative/reset_conversation.py 162 53 67%
umap_narrative/run_pipeline.py 355 203 43%
umap_narrative/topic_naming.py 148 11 93%
Total 25609 7377 71%

@colinmegill
colinmegill merged commit 4ca9f63 into edge Oct 27, 2026
39 checks passed
@colinmegill
colinmegill deleted the embed-page-id-width branch October 27, 2026 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant