Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion skills/omnibus/adding-warehouse-person-properties/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ the `external_data_schema` value the source needs. A table name alone is not eno

```sql
select column_name, data_type
from information_schema.columns
from system.information_schema.columns
where table_name = '<table name>'
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -516,8 +516,8 @@ maxOrNull
maxState
maxStateOrDefault
maxStateOrNull
MD5
md5
MD5
median
medianArgMax
medianArgMaxOrDefault
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ FROM
argMaxState(properties.$exception_functions.-1, timestamp) AS function_state,
argMaxState(properties.$exception_sources.-1, timestamp) AS source_state,
argMaxState(properties.$lib, timestamp) AS library_state,
least(19, intDiv(dateDiff('seconds', toDateTime(toDateTime('2026-09-11 12:00:00.000000')), timestamp), greatest(1, intDiv(dateDiff('seconds', toDateTime(toDateTime('2026-09-11 12:00:00.000000')), toDateTime(toDateTime('2026-09-12 12:00:42.895843'))), 20)))) AS bin_idx,
least(19, intDiv(dateDiff('seconds', toDateTime(toDateTime('2026-09-12 11:00:00.000000')), timestamp), greatest(1, intDiv(dateDiff('seconds', toDateTime(toDateTime('2026-09-12 11:00:00.000000')), toDateTime(toDateTime('2026-09-13 11:56:47.344259'))), 20)))) AS bin_idx,
count() AS occ,
uniqState(nullIf(e.$session_id, '')) AS sessions_state,
uniqState(coalesce(nullIf(toString(e.event_person_id), '00000000-0000-0000-0000-000000000000'), e.distinct_id)) AS users_state
FROM
events AS e
WHERE
and(equals(e.event, '$exception'), isNotNull(e.properties.$exception_fingerprint), true, greaterOrEquals(e.timestamp, toDateTime(toDateTime('2026-09-11 12:00:00.000000'))), lessOrEquals(e.timestamp, toDateTime(toDateTime('2026-09-12 12:00:42.895843'))), or(greater(position(lower(e.properties.$exception_types), lower('constant')), 0), greater(position(lower(e.properties.$exception_values), lower('constant')), 0), greater(position(lower(e.properties.$exception_sources), lower('constant')), 0), greater(position(lower(e.properties.$exception_functions), lower('constant')), 0), greater(position(lower(e.properties.email), lower('constant')), 0), greater(position(lower(e.person.properties.email), lower('constant')), 0)), equals(properties.tag, 'max_ai'))
and(equals(e.event, '$exception'), isNotNull(e.properties.$exception_fingerprint), true, greaterOrEquals(e.timestamp, toDateTime(toDateTime('2026-09-12 11:00:00.000000'))), lessOrEquals(e.timestamp, toDateTime(toDateTime('2026-09-13 11:56:47.344259'))), or(greater(position(lower(e.properties.$exception_types), lower('constant')), 0), greater(position(lower(e.properties.$exception_values), lower('constant')), 0), greater(position(lower(e.properties.$exception_sources), lower('constant')), 0), greater(position(lower(e.properties.$exception_functions), lower('constant')), 0), greater(position(lower(e.properties.email), lower('constant')), 0), greater(position(lower(e.person.properties.email), lower('constant')), 0)), equals(properties.tag, 'max_ai'))
GROUP BY
fp_hash,
bin_idx) AS ev
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ SELECT
FROM
logs
WHERE
and(and(greaterOrEquals(toStartOfDay(time_bucket), toStartOfDay(assumeNotNull(toDateTime('2025-12-09 00:00:00')))), lessOrEquals(toStartOfDay(time_bucket), toStartOfDay(assumeNotNull(toDateTime('2025-12-10 00:00:00'))))), 1, greaterOrEquals(timestamp, toDateTime('2026-09-11 12:00:43.702465')), indexHint(like(lower(body), '%timeout%')), ilike(toString(body), '%timeout%'), in(severity_text, tuple('warn', 'error', 'fatal')))
and(and(greaterOrEquals(toStartOfDay(time_bucket), toStartOfDay(assumeNotNull(toDateTime('2025-12-09 00:00:00')))), lessOrEquals(toStartOfDay(time_bucket), toStartOfDay(assumeNotNull(toDateTime('2025-12-10 00:00:00'))))), 1, greaterOrEquals(timestamp, toDateTime('2026-09-12 11:56:47.855279')), indexHint(like(lower(body), '%timeout%')), ilike(toString(body), '%timeout%'), in(severity_text, tuple('warn', 'error', 'fatal')))
ORDER BY
timestamp DESC,
uuid DESC
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,18 @@ SELECT
sum(s.console_error_count) AS console_error_count,
max(s.retention_period_days) AS retention_period_days,
plus(dateTrunc('DAY', start_time), toIntervalDay(coalesce(retention_period_days, 30))) AS expiry_time,
date_diff('DAY', toDateTime('2026-09-12 12:00:43.986747'), expiry_time) AS recording_ttl,
greaterOrEquals(max(s._timestamp), toDateTime('2026-09-12 11:55:43.986263')) AS ongoing,
date_diff('DAY', toDateTime('2026-09-13 11:56:48.019822'), expiry_time) AS recording_ttl,
greaterOrEquals(max(s._timestamp), toDateTime('2026-09-13 11:51:48.019482')) AS ongoing,
round(least(greatest(multiply(divide(plus(plus(plus(divide(sum(s.active_milliseconds), 1000), sum(s.click_count)), sum(s.keypress_count)), sum(s.console_error_count)), plus(plus(plus(plus(sum(s.mouse_activity_count), dateDiff('SECOND', start_time, end_time)), sum(s.console_error_count)), sum(s.console_log_count)), sum(s.console_warn_count))), 100), 0), 100), 2) AS activity_score,
coalesce(max(s.surfacing_score), 0.36) AS surfacing_score
FROM
raw_session_replay_events AS s
WHERE
and(greaterOrEquals(s.min_first_timestamp, toDateTime('2026-09-09 00:00:00.000000')), lessOrEquals(s.min_first_timestamp, toDateTime('2026-09-12 12:00:43.986438')))
and(greaterOrEquals(s.min_first_timestamp, toDateTime('2026-09-10 00:00:00.000000')), lessOrEquals(s.min_first_timestamp, toDateTime('2026-09-13 11:56:48.019607')))
GROUP BY
session_id
HAVING
and(greaterOrEquals(expiry_time, toDateTime('2026-09-12 12:00:43.986640')), equals(max(s.is_deleted), 0), greater(active_seconds, 5.0))
and(greaterOrEquals(expiry_time, toDateTime('2026-09-13 11:56:48.019740')), equals(max(s.is_deleted), 0), greater(active_seconds, 5.0))
ORDER BY
start_time DESC,
session_id DESC
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ SELECT
FROM
sessions
WHERE
and(less($start_timestamp, toDateTime('2026-09-12 12:00:50.260096')), greater($start_timestamp, toDateTime('2026-09-11 12:00:45.260565')))
and(less($start_timestamp, toDateTime('2026-09-13 11:56:53.795929')), greater($start_timestamp, toDateTime('2026-09-12 11:56:48.796207')))
ORDER BY
$start_timestamp DESC
LIMIT 50000
Expand Down
31 changes: 23 additions & 8 deletions skills/omnibus/querying-posthog-data/references/guidelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,29 @@ Everything else starts with that workflow.

#### Schema discovery (information_schema)

Don't guess table or column names — they differ per entity and drift over time. Discover the live schema for **every** data group above (system, captured, and data-warehouse tables) by querying `system.information_schema` via `execute-sql`. It exposes four self-describing virtual tables — every column below is selectable:
Don't guess table or column names — they differ per entity and drift over time. Discover the live schema for **every** data group above (system, captured, and data-warehouse tables) by querying `system.information_schema` via `execute-sql`. Four virtual tables carry the schema itself, and each one holds a different set of fields — project a field on the surface that owns it, or the query fails:

- `tables` — table_catalog, table_schema, table_name, table_type, description, row_count. table_type is one of system, data_warehouse, view, posthog (built-in analytics tables like events / persons), or information_schema.
- `columns` — table_schema, table_name, column_name, ordinal_position, data_type, is_nullable, is_array, field_kind, description.
- `relationships` — source_table, source_column, target_table, target_column, relationship_kind, via.
- `data_types` — type_name, description.
- `tables` — one row per table. Fields: table_catalog, table_schema, table_name, table_type, description, row_count, certification. table_type is one of system, data_warehouse, view, posthog (built-in analytics tables like events / persons), or information_schema. `certification` is the settled trust mark (`certified` / `deprecated`) and lives **only** here, not on `columns`.
- `columns` — one row per column. Fields: table_schema, table_name, column_name, ordinal_position, data_type, is_nullable, is_array, field_kind, description, null_fraction, min_value, max_value. The last three are profiling statistics and are filled in for data-warehouse columns only.
- `relationships` — one row per joinable relationship. Fields: source_table, source_column, target_table, target_column, relationship_kind, via, confidence, reasoning.
- `data_types` — one row per HogQL type. Fields: type_name, description.

It describes itself, so the full column set is always discoverable: `SELECT column_name, data_type FROM system.information_schema.columns WHERE table_name = 'system.information_schema.columns'`.
`certification` on `tables` and `confidence` / `reasoning` on `relationships` come from the data catalog. The project catalog, which is what `execute-sql` reads by default, always carries all three. A caller without data catalog access still selects them, and every value reads NULL. A NULL there never means a wrong field name.

The two surfaces differ in what else a NULL means. On `tables`, `certification` reads NULL for a table nobody marked. On `relationships`, `confidence` and `reasoning` hold the review evidence of an accepted relationship proposal. Only a data warehouse join that still matches its proposal carries that evidence. Every built-in join and every field traverser reads NULL for both fields, even on a project with full catalog access. A NULL there means no review evidence, not a broken join: read `source_column` and `target_column`, and use the join.

The same namespace carries six more catalog surfaces, each about project state rather than schema: `metrics`, `certifications` (the full trust-mark review queue, as opposed to the settled `tables.certification` mark), `relationship_proposals`, `data_quality_checks`, `data_quality_check_runs`, and `data_quality_health`. The project serves the three data-quality surfaces only while data quality checks are on for it.

A direct connection queried with `connectionId` is the runtime that drops surfaces. It serves `tables`, `columns`, and `data_types` only, and its `tables` has no `certification`. Leave `certification` out of a `connectionId` query. Do not read `relationships` there before a join, because the surface is absent and the query fails on an unknown table.

Every surface describes itself, so its live field set is always discoverable — ask the catalog instead of trusting the lists above:

```sql
SELECT column_name, data_type
FROM system.information_schema.columns
WHERE table_name = 'system.information_schema.tables'
ORDER BY ordinal_position
```

**List tables** — filter `table_type` to target a group (`system`, `data_warehouse`, `view`, `posthog`):

Expand All @@ -117,12 +132,12 @@ ORDER BY table_name
**Find a table by what its docs say** — names are often opaque (especially data-warehouse tables), so search the `description` text instead of guessing names. The documentation lives in `system.information_schema.tables.description` (the catalog) — not on the `system.data_warehouse_tables` entity, which only holds connection metadata:

```sql
SELECT table_name, description
SELECT table_name, description, certification
FROM system.information_schema.tables
WHERE table_type = 'data_warehouse' AND description ILIKE '%canonical mrr%'
```

Column docs are searchable the same way via `information_schema.columns.description`. Prefer an `ILIKE` filter over dumping the whole catalog and scanning it yourself.
Column docs are searchable the same way via `system.information_schema.columns.description` (the `system.` prefix is required — a bare `information_schema.columns` is an unknown table). Prefer an `ILIKE` filter over dumping the whole catalog and scanning it yourself.

**Inspect a table's columns:**

Expand Down