Skip to content

feat(sdk-events): add @dotcms/events for pageviews, content events and experiments - #37799

Merged
oidacra merged 17 commits into
mainfrom
issue-37683-events-prototype
Oct 6, 2026
Merged

oidacra merged 17 commits into
mainfrom
issue-37683-events-prototype

Conversation

@oidacra

@oidacra oidacra commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Proposed Changes

@dotcms/events replaces @dotcms/analytics and @dotcms/experiments with one dotEvents object that runs a single Analytics.js instance. It sends only what dotCMS accepts today: pageviews, conversions, content impressions and content clicks, plus experiments.

import { dotEvents } from '@dotcms/events';

dotEvents.init({ dotcmsUrl: DOTCMS_URL, siteAuth: SITE_AUTH }); // once, in instrumentation-client.ts
dotEvents.conversion('signup'); // from any module: the same object
  • Three entries. @dotcms/events: dotEvents (init, conversion, pageView) and the types its API uses. @dotcms/events/markup: experimentMarkup, what a server-rendered page prints so its experiment runs. @dotcms/events/react: DotCMSExperiment, which prints that markup around its children.
  • ca.min.js for traditional pages (nx run sdk-events:build:standalone): an IIFE that reads the attributes dotCMS already prints in ca/html/analytics_head.html, and decides experiments from the contentlet wrappers dotCMS prints. It is built here but not served yet: Run @dotcms/events on traditional pages as a new opt-in script, /ext/events/events.min.js #37798 (the backend changes that make it the only analytics and experiments script) wires it in.
  • Experiments. A boot script decides returning visitors while the HTML is parsed; the engine decides new visitors once it loads. Both run decideVariant. The engine asks isUserIncluded and redirects to the assigned variant. context.experiments goes only on the events of pages that run an experiment the visitor is in, with every experiment the visitor joined in the session; the events of other pages carry none.
  • Layout. One folder per capability under src/lib (pipeline, contentlets, impressions, clicks, experiments, react, standalone), files named by their role, and the boundaries between them enforced with no-restricted-imports.
  • Next.js 16. SPA navigations come from the browser's Navigation API, so a query-only change is counted and decided. The engine decides marks that stream in (loading.tsx, Suspense), skips routes Activity keeps hidden, keys reveals by experiment and variant, and writes them through an adopted style sheet, which a nonce-based CSP does not block. A visitor who leaves during the wait is neither redirected nor counted under the next page's URL. The README documents Next.js 16 and later.
  • State and errors. Everything is stored in localStorage and sessionStorage under dot_events_*, with no cookies; Analytics.js's own storage stays in memory. onError reports the requests dotCMS rejects or never answers.
  • Example. examples/nextjs-experiments runs on the package.
  • Other SDKs untouched. @dotcms/uve, @dotcms/react and @dotcms/analytics are not changed.

Checklist

  • Tests: 313 unit tests in sdk-events, plus the size, publint and declared-dependency checks in sdk-bundle-budgets.
  • Translations: not applicable.
  • Security Implications Contemplated: no cookies; the script-tag config of ca.min.js accepts only known keys of the right type; the boot script escapes <, so no value can close its script element.

Additional Info

  • Prototype for Create @dotcms/events: one SDK for analytics and experiments #37683 (SDK changes placeholder). Nothing here is released.

  • Sizes, gzip: events-init 28,119 B (budget 30,000), events-react 1,496 B (budget 2,200), ca.min.js 25,636 B.

  • Checked in Chrome against a local dotCMS:

    • on the Next example: pageviews, impressions, conversions and the experiment redirect;
    • on a traditional page served from dotCMS's origin: ca.min.js for a new and a returning visitor, in both variants;
    • on the example's blog streamed through a loading.tsx: new and returning visitors, in both variants, on full loads and client-side navigations, with every event counted through a proxy in front of dotCMS.

    dotCMS rejected no event.

  • libs/sdk/events/CLAUDE.md and README.md document the architecture and the decisions behind it.

This PR fixes: #37683

This PR fixes: #37683

…d experiments

@dotcms/events replaces @dotcms/analytics and @dotcms/experiments with one
events object that runs a single Analytics.js instance. It sends only what
dotCMS accepts today: pageviews, conversions, content impressions and
content clicks.

Entries:
- @dotcms/events: `events` (init, conversion, pageView) and the types its
  API uses.
- @dotcms/events/markup: `experimentMarkup`, what a server-rendered page
  prints so its experiment runs.
- @dotcms/events/react: `DotCMSExperiment`, which prints that markup.
- ca.min.js (build:standalone): the IIFE dotCMS injects into traditional
  pages. It reads the attributes of ca/html/analytics_head.html, and decides
  experiments from the contentlet wrappers dotCMS prints.

Experiments: a boot script decides returning visitors while the HTML is
parsed, and the engine decides new visitors; both run decideVariant. The
engine asks isUserIncluded, redirects to the assigned variant, and adds
context.experiments to every event.

Layout: one folder per capability under src/lib (pipeline, contentlets,
impressions, clicks, experiments, react, standalone), files named by their
role, and the boundaries between them enforced with no-restricted-imports.

State lives in localStorage and sessionStorage under dot_events_*, with no
cookies. onError reports the requests dotCMS rejects or never answers.

The nextjs-experiments example runs on the package.

Refs #37683, #37798
@claude

claude Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @oidacra's task in 1m 25s —— View job


Re-review of @dotcms/events

Rechecked the three findings from the earlier review against the current code, and scanned the engine, store, events entry, navigation, and standalone config for new issues. All prior findings are resolved; no new blocking issues.

  • Recheck prior findings (tracker history patch, navigation listener, config clamp)
  • Scan new/changed code for bugs, security, and error-path issues
  • Post review

Resolved

  • ✅ core-web/libs/sdk/events/src/lib/impressions/tracker.ts:272 — the history.pushState/replaceState monkeypatch is gone. initializePageNavigationHandler now subscribes through onNavigation/onPageRestore (which don't patch history) and keeps the returned detach functions in #stopWatchingNavigations/#stopWatchingRestores; cleanup() (lines 519–522) calls both, so the listeners are released and the instance can be GC'd. The asymmetry with the observers is gone.
  • ✅ core-web/libs/sdk/events/src/lib/pipeline/navigation.ts:51 — onNavigation now returns a real detach (navigation.removeEventListener(...)), and ExperimentsEngine.stop() (engine.ts:611–615) calls stopCountingNavigations?.(), so stop() fully undoes start(). The router-utils fallback still can't truly detach, but it's guarded by an active flag and that limitation is now documented in the JSDoc.
  • ✅ core-web/libs/sdk/events/src/lib/impressions/tracker.ts:101 — visibilityThreshold is now clamped to [0,1] with Math.min(1, Math.max(0, requested)) in resolveImpressionConfig (with a debug warning), so an out-of-range data-analytics-config value no longer throws a RangeError in new IntersectionObserver. This is the better fix location (applies to all config sources, not just the standalone script).
  • ✅ core-web/libs/sdk/events/src/lib/models.ts:55 — the experiments.timeout flicker is addressed: the engine caps the wait at DECISION_TIMEOUT_MS (3 s, waitMs = Math.min(timeoutMs, DECISION_TIMEOUT_MS) in engine.ts:200) and warns above it, so the page can't be shown by the hiding rule and then replaced by a variant.
  • ✅ core-web/libs/sdk/events/src/lib/experiments/boot.ts — the printed boot function is now exercised: boot.spec.ts runs the string buildExperimentBootScript emits through new Function(...) for both a redirect and a reveal, so an outside reference now fails the spec rather than silently at runtime.

Notes (non-blocking, carried from prior review — still accurate)

  • pipeline/sender/queue.ts#sendBatch remains fire-and-forget for normal (non-keepalive) sends: a failed batch drops its events with no retry. This is the documented "nothing is retried" contract — flagging only for visibility, not as a defect.
  • startAutomaticPageViews (events.ts:174) and the engine both register their own onNavigation listener; the one in startAutomaticPageViews is never detached. It's page-lifetime and startAutomaticPageViews runs once, so it's harmless, but it's the one navigation subscription that stop() can't undo.

No 🔴 Critical, 🟠 High, or new 🟡 Medium issues found. The three previous Mediums and the two reviewer comments on models.ts/boot.ts are fixed.
· branch issue-37683-events-prototype

@github-actions github-actions Bot added Area : Frontend PR changes Angular/TypeScript frontend code Area : SDK PR changes SDK libraries labels Sep 29, 2026
State the supported Next.js version in the README. It is not a peer
dependency: nothing in the package imports next, so a peer would only
make npm refuse to install in apps on an older Next.js.

16 rather than 15: Next.js supports 15 only until 21 October 2026, and
instrumentation-client.ts, where the README calls init, needs 15.3.

Refs #37683
- SPA navigations come from the browser's Navigation API, so a change of
  the query alone sends its pageview and is decided; router-utils stays
  as the fallback.
- The engine decides every mark the page renders, skips the ones in routes
  Activity keeps hidden or in content still streaming in, waits for the
  marks of a page its stored rule matches, and decides marks that arrive
  later.
- A visitor who leaves during the wait is neither redirected nor counted
  under the other page's URL.
- Reveals are keyed by experiment and rendered variant, and go through an
  adopted style sheet, which a nonce-based CSP does not block.
  DotCMSExperiment gives its style the nonce too.

The public object is now dotEvents, the same name as window.dotEvents, so
it does not collide with an app's own "events".

The example's blog streams, through a loading.tsx, to show an experiment
on a streamed route next to the catch-all route, which renders in one
piece; its README says what each one shows.

Refs #37683
Comment thread examples/nextjs-experiments/.npmrc Outdated
Comment thread examples/nextjs-experiments/package.json Outdated
- Content clicks are throttled per contentlet, so a click on another
  contentlet right after counts.
- A visitor who goes to another page and back while the assignment is
  pending starts a new visit: the first one is dropped, and the return is
  decided and counted on its own.
- The impressions throttleMs option did nothing, and is removed.
- The isUserIncluded shapes are DotCMSExperimentVariant,
  DotCMSAssignedExperiment and DotCMSIsUserIncludedEntity, the prefix for
  what dotCMS's API returns.
- The trackers, the logger and the activity tracker keep their fields
  private with #. At the es2020 target that costs 977 B gzip of
  events-init; ca.min.js keeps them native. Their specs test behavior
  instead of reading them.
- Comments carried over from @dotcms/analytics now describe this package:
  the plugin order, the contentlet class, the maxNodes default and
  generateSecureId. The click tracker's debug logs read the contentlet's
  data-dot-identifier.
- Specs that depended on the order they ran in set up their own state.

Refs #37683
context.experiments stays cumulative for the session, but events carry it
only while the page on screen runs an experiment the visitor is in. A
visitor who reaches the experiment on /index and moves to a page without
one sends that page's events with no experiments; a later page that runs
another experiment sends both.

Each queued event keeps the context it was created with, in memory and in
sessionStorage. A batch goes out as one request per run of events that
share the site, session, visitor and experiments, so an experiment page's
pageview no longer takes the next page's context when both wait in the
same batch. The device alone does not split a batch: a phone's viewport
changes on scroll.

Refs #37683
The example depended on the local SDK builds: file: paths for every
@dotcms/* package, an .npmrc that copied them in, and a lockfile built
from them. A clean clone could not install without building the SDKs,
and the release pins every @dotcms/* dependency of the examples to the
release version anyway. The dependencies are back on latest, as on
main, with @dotcms/events in place of @dotcms/experiments, and the
lockfile is main's; it gains @dotcms/events when the package is
published.

The README and CLAUDE.md describe @dotcms/events: the Site Auth from
the Content Analytics app in place of the experiments key,
eventsConfig and instrumentation-client.ts in place of
experimentsConfig, and DotCMSExperiment in place of withExperiments.
Until the package is on npm, they install its local build with
npm install --no-save, which leaves package.json and the lockfile
untouched.

Refs #37683
@oidacra
oidacra marked this pull request as ready for review October 2, 2026 14:35
@oidacra

oidacra commented Oct 2, 2026

Copy link
Copy Markdown
Member Author

Claude finished @oidacra's task in 4m 10s —— View job

Review: @dotcms/events

Reviewed the pipeline (queue/sender/http/enricher/identity/activity), the experiments engine (engine/boot/decision/store/dom/api/markup/plugin), both content trackers, events.ts, the standalone config, the React adapter, and the example wiring. The code is careful and heavily tested (313 unit tests), and decideVariant's redirect/ignored logic correctly avoids variant loops in every case I traced. Findings below are non-blocking.

New Issues

  • 🟡 Medium: core-web/libs/sdk/events/src/lib/impressions/tracker.ts:291 — initializePageNavigationHandler() monkeypatches history.pushState/history.replaceState and adds a popstate listener (lines 288–302), but cleanup() (line 499) never restores the patched methods nor removes the popstate listener. After cleanup() the wrapped pushState/replaceState keep invoking the checkPathChange closure, which holds this, so the tracker instance can't be GC'd and keeps reacting to navigations against already-cleared state; re-initializing on the same page would also double-wrap history. In the normal flow cleanup() only runs on pagehide/beforeunload so impact is limited, but the resource handling is asymmetric with the IntersectionObserver/MutationObserver that are torn down. Note the rest of the SDK already detects SPA navigation via pipeline/navigation.ts (Navigation API / router-utils) without patching history. Fix this →
  • 🟡 Medium: core-web/libs/sdk/events/src/lib/pipeline/navigation.ts:28 — onNavigation registers a currententrychange listener (and the router-utils fallback) with no way to detach it, and ExperimentsEngine.stop() (engine.ts:597) only disconnects the mark observer. Each engine/queue/auto-pageview caller leaks one listener for the life of the page. Harmless on a normal page lifecycle, but it means stop() doesn't fully undo start().
  • 🟡 Medium: core-web/libs/sdk/events/src/lib/standalone/config.ts:102 — advanced-config impressions.visibilityThreshold is accepted as any finite number (numbers() only checks Number.isFinite). A value >1 or <0 from data-analytics-config reaches new IntersectionObserver({ threshold }) (tracker.ts:145), which throws a RangeError during plugin init. This is operator misconfiguration on a traditional page, but it would break impression tracking rather than being ignored like other invalid config. Consider clamping/validating to [0,1].

Notes (not blocking)

  • Test gaps are acknowledged in CLAUDE.md (Known Gaps): experiments/plugin.ts, experiments/store.ts, experiments/api.ts have no specs. store.ts#saveAssignments carries real merge logic (expiry, ended-experiment pruning, evaluatedIds accumulation) that would benefit from direct coverage before this leaves prototype.
  • pipeline/sender/queue.ts#sendBatch is fire-and-forget for non-keepalive sends too: a failed normal batch drops its events from both the in-memory queue and sessionStorage with no retry. This matches the documented "nothing is retried" contract, so flagging only for visibility.
  • pipeline/utils.ts exports getPageData, getDeviceData, getUtmData that don't appear to be used by the active enrichment path (enrichPagePayloadOptimized). Worth confirming they aren't dead code carried over from @dotcms/analytics.

No 🔴 Critical or 🟠 High issues found. The two reviewer comments on the example (.npmrc, file: deps in package.json) are resolved in commit 1dff60e0. · branch issue-37683-events-prototype

Fixed.

- A page Chrome keeps in the back/forward cache comes back with its
  content trackers and activity tracking: they clean up only on the
  pagehide that discards the page (onPageDiscard), not on beforeunload,
  which fires before the page goes into that cache too. Before, a
  visitor who went back to such a page sent no impressions or clicks
  from it, and __dotAnalyticsActive__ stayed false.
- The impression tracker learns of SPA navigations through onNavigation
  instead of patching history.pushState and replaceState, and stops
  listening on cleanup. onNavigation returns its unsubscribe, which the
  engine's stop() calls too.
- A visibilityThreshold outside 0 to 1, which IntersectionObserver
  rejects with a RangeError, is clamped to that range with a warning,
  for dotEvents.init and ca.min.js alike.
- getPageData, getDeviceData and getUtmData, carried over from
  @dotcms/analytics and used by nothing, are removed with their specs.
- store.spec.ts covers what the engine stores: how an isUserIncluded
  answer merges into the assignments, expiry, the tab's check, the
  session's experiments and the day off after a 403.

Refs #37683
…cument

The click tracker attached a listener to each contentlet and, for each
one, scanned every contentlet on the page to save its position: n² in
the number of contentlets. At 4x CPU, adding 600 contentlets cost 121 ms
with a 162 ms long task, and 1,000 cost 332 ms with a 414 ms one.

It now listens once, on the document, in the capture phase, and finds
the clicked link or button and its contentlet with closest(). Adding 600
or 1,000 contentlets costs about 3 ms, with no long tasks. Nothing scans
or observes the page for clicks anymore, a contentlet added at any time
counts, an app's stopPropagation() no longer hides a click, and the SDK
no longer writes data-dot-analytics-dom-index on the contentlets:
handleContentletClick reads the position when the click happens, so it
stays right as contentlets are added or moved.

A click inside nested contentlets now counts once, for the innermost;
with a listener on each contentlet it sent one event per nesting level.

Refs #37683
Comment thread core-web/libs/sdk/events/src/lib/experiments/plugin.ts
Comment thread core-web/libs/sdk/events/src/lib/models.ts
Comment thread core-web/libs/sdk/events/src/lib/experiments/boot.ts
A page the browser restores from its back/forward cache comes back as the
visitor left it, so nothing that runs on a load ran again: going back counted
as a pageview when the browser reloaded the page, and not when it restored it.
Which one happens is the browser's call, so the counts depended on it.

On `pageshow` with `persisted` (onPageRestore in pipeline/navigation.ts), the
automatic pageviews now send one, although the URL is the one already counted,
and the impression tracker forgets the page's impressions and observes its
contentlets again, so the ones in view count again, as after a load. A restore
is not a navigation inside the page: the engine's navigation count stays, so
the pageview gets the decision the page already has, with its
context.experiments, or none on a page that runs none.

The experiment goals count pageviews: a session with more than one is not a
bounce, and a session exits from the experiment page when its last pageview is
there. A visitor who goes back to the experiment page and leaves from it now
counts as an exit there, as with a reload.

Checked in Chrome: dotcms.com on next dev and the example's static
/lab/bfcache are restored, and send the pageview and impressions with the
page's experiments (202). Next.js dynamic routes in production answer
no-store, and Chrome reloads them instead, as before. events-init grows
126 B gzip, to 29,148 B.
adrianjm-dotCMS
adrianjm-dotCMS previously approved these changes Oct 2, 2026
From the review of #37799 (events prototype):

- The hiding rule and hideContentlets show the experiment's content after
  DECISION_TIMEOUT_MS (3 s) on their own, while the engine waited
  experiments.timeout. With a longer timeout, a new visitor saw the original
  at 3 s and, when isUserIncluded answered later with another variant, the
  page was replaced by it. The engine now waits 3000 ms at most, reduces a
  longer value and warns in debug, and the JSDoc says so.
- Nothing ran the boot script as the page prints it. boot.spec.ts now runs
  what buildExperimentBootScript prints on its own, through new Function,
  for a redirect and a reveal, so a reference to anything outside the two
  printed functions fails it (checked by adding one).

events-init grows 60 B gzip, to 29,208 B.
The same sections as the other SDK READMEs: overview, installation, the
shared version text, a Next.js quick start, how it works, configuration,
usage, API reference, under the hood, troubleshooting, support,
contributing and licensing. Written for app developers; the internals stay
in CLAUDE.md.

It adds a migration table from @dotcms/analytics and @dotcms/experiments,
and only mentions traditional pages: dotCMS adds the tracking to them on its
own, set up in the Content Analytics app.
From 421 lines (53.7 KB) to 181 (14.1 KB), on the pattern of the client and
react SDKs' CLAUDE.md, as the root CLAUDE.md asks of these files: a quick
reference, not the full design.

It keeps the commands, the structure, what init does, the plugin order,
the queue and the back/forward cache, how experiments are decided and which
events carry them, the traditional-pages switch, and the rules that break
silently. What apps need is in README.md now; the long pageview flow, the
benchmark tables, the layer table that eslint.config.mjs enforces, history
notes and the prototype's known gaps are left out.
adrianjm-dotCMS
adrianjm-dotCMS previously approved these changes Oct 5, 2026
@oidacra
oidacra enabled auto-merge October 5, 2026 19:22
…e build imports

- The package copied every *.md into dist, CLAUDE.md included. It is for
  working on the repo, not for the package's users: only README.md ships
  now. 57 files, 111.7 KB packed.
- @analytics/core and @analytics/storage-utils were declared but never
  imported by the built code; they come with analytics, which depends on
  both. The package now declares analytics, @analytics/queue-utils and
  @analytics/router-utils.

Checked with the release's own build (nx run-many --target=build
--projects='sdk-*' --configuration=production), the bundle budgets,
publint and the package.json shape validator, and by installing the
packed package with npm in an empty project and in the nextjs-experiments
example: both entry formats load, the types check, and analytics gets
its dependencies.
The comment in nextjs-experiments' blog/loading.tsx now says what the file
shows: an experiment on a route that streams. Next.js wraps the page in
<Suspense fallback={<Loading />}> because the file exists, the same as
writing that boundary by hand around the part that waits for dotCMS, and
the SDK works the same either way. The README's line about the blog says
so too.
@oidacra
oidacra added this pull request to the merge queue Oct 6, 2026
Merged via the queue into main with commit 91ed985 Oct 6, 2026
48 checks passed
@oidacra
oidacra deleted the issue-37683-events-prototype branch October 6, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area : Frontend PR changes Angular/TypeScript frontend code Area : SDK PR changes SDK libraries

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Create @dotcms/events: one SDK for analytics and experiments

2 participants