Skip to content

Conversation

@Rodriguespn
Copy link
Collaborator

Summary

This PR introduces PostgreSQL version awareness and a comprehensive evaluation framework to the postgres-best-practices skill. The changes ensure AI agents only recommend rules compatible with the target PostgreSQL version and available extensions.

Changes

1. Rule Version & Extension Metadata

Added version constraints and extension requirements to rules that depend on specific PostgreSQL features:

Rule Added Metadata Requirement
query-covering-indexes.md minVersion: "11" INCLUDE clause requires PG11+
schema-partitioning.md minVersion: "10" Declarative partitioning requires PG10+
data-upsert.md minVersion: "9.5" ON CONFLICT requires PG9.5+
lock-skip-locked.md minVersion: "9.5" SKIP LOCKED requires PG9.5+
advanced-jsonb-indexing.md minVersion: "9.4" JSONB type requires PG9.4+
monitor-pg-stat-statements.md extensions: pg_stat_statements Requires extension availability

2. Profile System

Introduced deployment profiles to customize rule filtering based on environment constraints:

Profiles Added:

  • supabase.json: Supabase platform (PG15+, rich extension support including pg_stat_statements, pgjwt, pg_graphql)
  • aurora.json: AWS Aurora PostgreSQL (PG13-16, limited extensions, pg_cron/timescaledb unavailable)
  • self-hosted.json: Generic self-hosted (PG12+, all extensions installable)

Each profile specifies:

  • Minimum/maximum PostgreSQL version
  • Available, installable, and unavailable extensions
  • Environment-specific notes

3. Build System Enhancements

  • New profiles.ts module with version comparison and compatibility checking
  • isRuleCompatibleWithProfile() function filters rules based on version and extension requirements
  • Build now generates profile-specific outputs: AGENTS.supabase.md, AGENTS.aurora.md, AGENTS.self-hosted.md

4. SKILL.md Documentation

Added runtime detection guidance for agents:

  • SQL queries to detect PostgreSQL version
  • Extension availability checking via pg_available_extensions
  • Configuration verification via pg_settings
  • Version compatibility matrix

Evaluation Scenarios

A comprehensive eval framework was added to test agent behavior with version and extension constraints.

Framework

  • Test Runner: Vitest + Vercel AI SDK (Claude integration)
  • Location: skills/postgres-best-practices/evals/
  • Run: npm run eval (requires ANTHROPIC_API_KEY)

Scenarios by Category

Category 1: Core Query Patterns

ID Name Difficulty What it tests
missing-index-detection Missing Index Basic Agent identifies missing indexes on filtered columns
n-plus-one-detection N+1 Query Intermediate Agent recognizes N+1 pattern and recommends JOINs
covering-index-suggestion Covering Index Intermediate Agent suggests INCLUDE clause for covering indexes

Category 2: Version Constraints

ID Name Difficulty What it tests
version-constraint-pg10-no-covering PG10 No Covering Index Intermediate Agent does NOT recommend INCLUDE on PG10 (requires PG11+)
version-constraint-pg93-no-upsert PG9.3 No UPSERT Intermediate Agent does NOT recommend ON CONFLICT on PG9.3 (requires PG9.5+)

Category 3: Extension Requirements

ID Name Difficulty What it tests
extension-available-pg-stat-statements Extension Available Basic Agent recommends pg_stat_statements when available
extension-unavailable-no-pg-stat-statements Extension Unavailable Intermediate Agent suggests alternatives when extension unavailable

@Rodriguespn Rodriguespn requested a review from samrose January 23, 2026 17:35
@Rodriguespn Rodriguespn force-pushed the feat/postgres-version-support-and-evals branch from 33e888a to b5289ff Compare January 23, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants