Define SIP Observability Profile v1.0.0

[frankxai]

What changed

Defines the substrate-neutral SIP Observability Profile:

• authority boundaries across SIS, agentic-ops, starlight-evals, Second Brain OS, Hermes Cockpit, and Token Tracker;
• event and scorecard JSON Schemas v1.0.0;
• required cross-system join keys;
• epistemic boundaries between intent, state change, verification, and external outcome;
• anti-Goodhart requirements;
• Starlight Board verdict: PROCEED-WITH-REVISE.

Why

The estate already measures models, memory, retrieval, safety, content, cost, and fleet state, but those observations do not share a neutral evidence contract. This profile standardizes semantics without mandating Supabase, Langfuse, Hermes, or any other vendor/runtime.

Governance

This is substrate-class. The included Board verdict authorizes a review branch and operational dog-food run, not automatic merge.

Required before merge:

1. Frank acknowledges the authority boundary.
2. Supabase project selection remains private and operational.
3. The first 30-day review checks for unhealthy metric-driven behavior.
4. Surfaces preserve staleness and null denominators.

Verification

• both schemas parse as JSON;
• schemas are byte-aligned with the operational mirror tested in agentic-ops;
• branch is 4 commits ahead / 0 behind main; 4 new files only.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8f7a3091-c632-4c16-a841-295c29d36cb5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/codex/observability-profile-v1

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
site	Ready	Preview, Comment	Jul 16, 2026 6:02pm

[gemini-code-assist]

Code Review

This pull request introduces the SIP Observability Profile, establishing standardized event and scorecard schemas for the Starlight Intelligence System. The review identified several areas to strengthen the data contracts: the 'risks' array in the scorecard schema is currently too permissive, several objects in the event schema allow 'additionalProperties' which may lead to inconsistency, and the 'scorecard_id' field lacks a minimum length constraint to prevent empty identifiers.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The items in the risks array are defined as a generic {"type":"object"}, which allows any object structure. This is too permissive and undermines the purpose of the schema by leaving a key part of the data contract undefined. Please define the structure of a risk object with specific properties (e.g., id, description, severity, status) and set additionalProperties: false.

[gemini-code-assist]

The source object, as well as links (line 13), execution (line 14), quality (line 15), economics (line 16), and items in the evidence array (line 18), are defined with "additionalProperties": true. This allows any extra properties to be added, which can lead to inconsistent data and makes it harder for consumers to reliably parse these events.

For a stricter data contract, it's recommended to set "additionalProperties": false and explicitly define all allowed properties in the schema. If extensibility is truly needed, consider adding a dedicated custom property to these objects to hold arbitrary key-value pairs.

[gemini-code-assist]

The scorecard_id should not be an empty string. It's good practice to add a minLength: 1 constraint to ensure it's a non-empty identifier.

Suggested change

	"scorecard_id": {"type":"string"},
	"scorecard_id": {"type":"string", "minLength": 1},

[frankxai]

CTO/CDO council: refine before ready

The profile is a strong vendor-neutral substrate, and the required join spine is the right direction. I independently verified the current head against main; keep this draft HOLD until these contract gaps are closed:

1. Reconcile the two event contracts. packages/core/schemas/agent-event.schema.json already uses starlight.dev, camelCase, and overlapping AgentEvent semantics; this profile introduces starlightintelligence.org, snake_case, and no migration/boundary map. Declare successor vs separate concern and document the field mapping.
2. Enforce evidence on verification. artifact.verified can currently validate with an empty evidence array. Add a conditional/minimum so a verification event cannot claim quality without evidence.
3. Add conformance fixtures. Commit one valid event and one valid scorecard plus a small schema-validation check in this repo. The private operational mirror is not independently verifiable evidence here.
4. Bound cardinality/privacy surfaces. tags, metrics, and multiple additionalProperties: true bags can become cost and data-leak paths. Add a cardinality budget/namespacing rule and state that free-form values inherit privacy.
5. Tighten scorecard truth and discovery. Add non-empty constraints to anti_goodhart/attestation, define objective/window semantics before calling metrics SLOs, and link the profile from root README + METRICS_TRUTH.md.

Industry note: freeze stable identity/join keys while allowing additive attribute evolution; avoid forcing every emitter into lockstep for harmless schema additions.

No code changes made to this owned Codex lane. Current checks are green and GitHub reports the draft merge state as clean.