Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions config/roles/analysts.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# yaml-language-server: $schema=./schema.json
name: analysts
gold_source_id: GRP-002
permissions:
- resource: assets
action: read
- resource: assets
action: write
8 changes: 8 additions & 0 deletions config/roles/engineers.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# yaml-language-server: $schema=./schema.json
name: engineers
gold_source_id: GRP-001
permissions:
- resource: assets
action: read
- resource: systems
action: read
59 changes: 59 additions & 0 deletions config/roles/schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
{
"$defs": {
"PermissionConfig": {
"properties": {
"resource": {
"title": "Resource",
"type": "string"
},
"subresource": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Subresource"
},
"action": {
"title": "Action",
"type": "string"
}
},
"required": [
"resource",
"action"
],
"title": "PermissionConfig",
"type": "object"
}
},
"properties": {
"name": {
"pattern": "^[a-z][a-z0-9-]*$",
"title": "Name",
"type": "string"
},
"gold_source_id": {
"title": "Gold Source Id",
"type": "string"
},
"permissions": {
"items": {
"$ref": "#/$defs/PermissionConfig"
},
"title": "Permissions",
"type": "array"
}
},
"required": [
"name",
"gold_source_id",
"permissions"
],
"title": "RoleConfig",
"type": "object"
}
9 changes: 8 additions & 1 deletion docs/patterns/error-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,14 @@ A `Result[T, E]` is either `Ok(value)` or `Err(error)`, where the error type is

All error types inherit from `AppError`, an abstract base class that enforces a two-level message contract. Each error exposes a `message` property (safe to return to clients) and a `detail` property (for internal logging only, may contain URLs, SQL statements, or driver messages). `__str__` returns `message`, so errors are safe by default anywhere they are converted to strings. Error types store their raw diagnostic data in fields and compute both properties from them.

Error types are domain-specific dataclasses defined in `app/errors.py`. They form unions that describe what can go wrong in each context: `IngestionError` covers fetch failures, schema validation errors, and database errors. Functions declare which error union they can produce, and callers handle each variant explicitly.
Error types are domain-specific dataclasses defined in `app/errors.py`. They form unions that describe what can go wrong in each context:

- `IngestionError = FetchError | RemoteValidationError | ValidationError | DBError` — covers HTTP failures, remote schema parse errors, local validation errors, and database errors.
- `ConfigSyncError = ConfigError | ValidationError | DBError` — covers YAML file errors (missing directory, unreadable file, empty file), local validation errors, and database errors.

`RemoteValidationError` is a distinct error type (not a subtype of `ValidationError`) for upstream data quality failures — produced when an HTTP response body fails Pydantic validation or when a remote service references a resource that cannot be resolved locally. It includes the source URL in its `detail` and maps to HTTP 502. `ValidationError` is for local validation failures (e.g., config files) where no URL context is available, and maps to HTTP 422.

Functions declare which error union they can produce, and callers handle each variant explicitly.

## Route Helpers

Expand Down
4 changes: 4 additions & 0 deletions docs/patterns/ingestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ External data is ingested through a three-phase pipeline that separates fetching

**Phase 2 — Fetch Detail.** For each item in the index, the service calls the detail endpoint to retrieve the full representation. This is validated against a separate Pydantic model for the detail response shape.

Both fetch phases go through the `fetch_json` helper (`app/http.py`), which handles HTTP errors uniformly: transport/status failures become `FetchError` (with the URL), and Pydantic parse failures become `RemoteValidationError` (a URL-aware `ValidationError` subclass). Services declare their source error type as `FetchError | RemoteValidationError`.

**Phase 3 — Convert and Upsert.** A pure function maps each external detail to an `XCreate` model instance. The orchestrator then upserts each converted record into the database, keyed by gold source ID (the record's unique identifier in the external system — see [Gold Source Identity](models.md#gold-source-identity)), and returns `XPublic` instances via `model_validate` after flush.

The separation of index and detail schemas reflects the reality that list and detail endpoints often return different shapes. Keeping the conversion as a pure function (no IO, no session) makes it independently testable. The orchestrator does not commit — the caller (typically the route handler) owns the session lifecycle, which preserves all-or-nothing semantics: if any phase fails, nothing is persisted.
Expand All @@ -18,6 +20,8 @@ When multiple sources need to be ingested atomically, a global ingestion endpoin

Sources that reference records from other sources (e.g., systems referencing assets) resolve those references by gold source ID during their pipeline. If a referenced record cannot be found, the entire source's ingestion is rejected.

The upsert helpers (`upsert_by_gold_source`, `_upsert_permission`) use a select-then-insert pattern. `upsert_by_gold_source` handles `IntegrityError` from concurrent inserts by rolling back and re-querying, so the losing thread returns the canonical record rather than propagating an exception. `_upsert_permission` does not yet handle this case — it is only called from config sync, which is not expected to run concurrently.

## Link Resolution and Syncing

For N:M relationships that cross source boundaries, the ingestion pipeline resolves external IDs to internal IDs using `get_by_gold_source`. After upserting the parent record, a diff-based sync step compares the desired set of linked IDs against the current set in the database, then issues INSERT and DELETE statements against the join table to reconcile the difference. This ensures re-ingestion is idempotent — links are added or removed to match the source of truth without duplicating or orphaning entries.
Expand Down
2 changes: 1 addition & 1 deletion docs/patterns/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Each domain entity uses a family of SQLModel classes that separate concerns acro

## Gold Source Identity

Models that are synced from external systems inherit from `GoldSourceMixin`, which provides `gold_source_id` and `gold_source_type` fields with a composite unique constraint. This allows any synced record to be looked up by its external identity. Each model adds a composite index on these fields for query performance. The mixin provides a `get_by_gold_source` query method that returns a typed `Result[Option[T], DBError]`, consistent with the error handling pattern.
Models that are synced from external systems inherit from `GoldSourceMixin`, which provides `gold_source_id` (string) and `gold_source_type` (`GoldSourceType` enum, stored as `VARCHAR` to avoid a PostgreSQL enum type) fields with a composite unique constraint. This allows any synced record to be looked up by its external identity. The unique constraint implicitly backs gold-source lookups — no separate index is needed. The mixin provides a `get_by_gold_source` query method that returns a typed `Result[Option[T], DBError]`, consistent with the error handling pattern.

## Query Functions and Type Narrowing

Expand Down
8 changes: 4 additions & 4 deletions docs/patterns/routes.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ List endpoints support offset/limit pagination. Lookup endpoints return 404 when

## Response Types

`XPublic` models (defined in `app/models/`) serve directly as API response types. There is no separate response schema layer — `XPublic` is used as the `response_model` in route decorators and as the return type of route handlers.
`XPublic` models (defined in `app/models/`) are the default API response type for individual item endpoints. For simple models (no relationships), routes pass `public_class=XPublic` to query functions (`get_by_id`, `get_paginated`, `get_by_gold_source`), which return the narrowed type directly.

For simple models (no relationships), routes pass `public_class=XPublic` to query functions (`get_by_id`, `get_paginated`, `get_by_gold_source`), which return the narrowed type directly.
For models with ORM relationships that need derived fields (e.g., `asset_ids`), routes query the ORM model and convert via a `_to_public` helper that populates the derived fields from the loaded relationship.

For models with relationships that need derived fields (e.g., `asset_ids`), routes query the ORM model and convert via a `_to_public` helper that populates the derived fields from the loaded relationship.
For endpoints that require joined or enriched data beyond the model's own fields (e.g., `GET /users/{id}` returning the user's assigned role names), a schema class under `app/schemas/` extends `XPublic` with the additional fields. These schemas are used as the `response_model` for that specific endpoint; the list endpoint for the same resource continues to use `XPublic` directly to avoid N+1 queries.

List endpoints use lightweight wrapper schemas (e.g., `AssetListResponse`) under `app/schemas/` that pair a list of `XPublic` items with a `total` count. These are the only response schemas — individual item responses use `XPublic` directly.
List endpoints use lightweight wrapper schemas (e.g., `AssetListResponse`, `UserListResponse`) under `app/schemas/` that pair a list of `XPublic` items with a `total` count.
15 changes: 15 additions & 0 deletions mock-services/devenv.nix
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,19 @@ in
};
};
};

processes.iam-mock = {
exec = "cd $MOCK_SERVICES_DIR && uv run uvicorn iam.app:app --host 0.0.0.0 --port 4012";
process-compose = {
readiness_probe = {
http_get = {
host = "localhost";
port = 4012;
path = "/users";
};
initial_delay_seconds = 2;
period_seconds = 2;
};
};
};
}
Empty file added mock-services/iam/__init__.py
Empty file.
12 changes: 12 additions & 0 deletions mock-services/iam/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
from pathlib import Path

from iam.models import IAMUserIndexItem, IAMUserItem
from mock_helpers import create_mock_app

app, _ = create_mock_app(
title="IAM Mock",
data_path=Path(__file__).parent / "data" / "users.yaml",
full_model=IAMUserItem,
index_model=IAMUserIndexItem,
resource_name="users",
)
18 changes: 18 additions & 0 deletions mock-services/iam/data/users.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
- id: "USR-001"
username: "alice"
email: "alice@example.com"
groups:
- "GRP-001"

- id: "USR-002"
username: "bob"
email: "bob@example.com"
groups:
- "GRP-002"

- id: "USR-003"
username: "charlie"
email: "charlie@example.com"
groups:
- "GRP-001"
- "GRP-002"
14 changes: 14 additions & 0 deletions mock-services/iam/models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
from pydantic import BaseModel


class IAMUserIndexItem(BaseModel):
id: str
username: str
email: str


class IAMUserItem(BaseModel):
id: str
username: str
email: str
groups: list[str]
153 changes: 153 additions & 0 deletions verdict-backend/alembic/versions/0e9a994a92da_create_auth_tables.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
"""create auth tables

Revision ID: 0e9a994a92da
Revises: 3bfa8f122226
Create Date: 2026-03-15 11:57:33.704856

"""

from collections.abc import Sequence

import sqlalchemy as sa
import sqlmodel

from alembic import op

# revision identifiers, used by Alembic.
revision: str = "0e9a994a92da"
down_revision: str | Sequence[str] | None = "3bfa8f122226"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None


def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.create_table(
"app_user",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column("gold_source_id", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.Column("gold_source_type", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.Column("username", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.Column("email", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.Column(
"is_active",
sa.Boolean(),
server_default=sa.text("true"),
nullable=False,
),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("gold_source_type", "gold_source_id"),
)
op.create_table(
"permission",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column("resource", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.Column("subresource", sqlmodel.sql.sqltypes.AutoString(), nullable=True),
sa.Column("action", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
# A single UniqueConstraint("resource","subresource","action") would not prevent
# duplicates when subresource IS NULL, because PostgreSQL treats NULLs as distinct
# in unique constraints. Two partial indexes give correct uniqueness semantics.
op.create_index(
"uq_permission_resource_action_no_subresource",
"permission",
["resource", "action"],
unique=True,
postgresql_where=sa.text("subresource IS NULL"),
)
op.create_index(
"uq_permission_resource_subresource_action",
"permission",
["resource", "subresource", "action"],
unique=True,
postgresql_where=sa.text("subresource IS NOT NULL"),
)
op.create_table(
"role",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column("gold_source_id", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.Column("gold_source_type", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.Column("name", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.Column("description", sqlmodel.sql.sqltypes.AutoString(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("gold_source_type", "gold_source_id"),
sa.UniqueConstraint("name", name="uq_role_name"),
)
op.create_table(
"rolepermission",
sa.Column("role_id", sa.Integer(), nullable=False),
sa.Column("permission_id", sa.Integer(), nullable=False),
sa.ForeignKeyConstraint(
["permission_id"],
["permission.id"],
),
sa.ForeignKeyConstraint(
["role_id"],
["role.id"],
),
sa.PrimaryKeyConstraint("role_id", "permission_id"),
)
op.create_table(
"userrole",
sa.Column("user_id", sa.Integer(), nullable=False),
sa.Column("role_id", sa.Integer(), nullable=False),
sa.ForeignKeyConstraint(
["role_id"],
["role.id"],
),
sa.ForeignKeyConstraint(
["user_id"],
["app_user.id"],
),
sa.PrimaryKeyConstraint("user_id", "role_id"),
)
# ### end Alembic commands ###


def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table("userrole")
op.drop_table("rolepermission")
op.drop_table("role")
op.drop_table("permission")
op.drop_table("app_user")
# ### end Alembic commands ###
Loading
Loading