Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
## Context

This implements "Slice 1: What's Ready?" from the artifact POC analysis. The core insight is using the filesystem as a database - artifact completion is detected by file existence, making the system stateless and version-control friendly.

This module will coexist with the current OpenSpec system as a parallel capability, potentially enabling future migration or integration.

## Goals / Non-Goals

**Goals:**
- Pure dependency graph logic with no side effects
- Stateless state detection (rescan filesystem each query)
- Support glob patterns for multi-file artifacts (e.g., `specs/*.md`)
- Load artifact definitions from YAML schemas
- Calculate topological build order
- Determine "ready" artifacts based on dependency completion

**Non-Goals:**
- CLI commands (Slice 4)
- Multi-change management (Slice 2)
- Template resolution and enrichment (Slice 3)
- Agent integration or Claude commands
- Replacing existing OpenSpec functionality

## Decisions

### Decision: Filesystem as Database
Use file existence for state detection rather than a separate state file.

**Rationale:**
- Stateless - no state corruption possible
- Git-friendly - state derived from committed files
- Simple - no sync issues between state file and actual files

**Alternatives considered:**
- JSON/SQLite state file: More complex, sync issues, not git-friendly
- Git metadata: Too coupled to git, complex implementation

### Decision: Kahn's Algorithm for Topological Sort
Use Kahn's algorithm for computing build order.

**Rationale:**
- Well-understood, O(V+E) complexity
- Naturally detects cycles during execution
- Produces a stable, deterministic order

### Decision: Glob Pattern Support
Support glob patterns like `specs/*.md` in artifact `generates` field.

**Rationale:**
- Allows multiple files to satisfy a single artifact requirement
- Common pattern for spec directories with multiple files
- Uses standard glob syntax

### Decision: Immutable Completed Set
Represent completion state as an immutable Set of completed artifact IDs.

**Rationale:**
- Functional style, easier to reason about
- State derived fresh each query, no mutation needed
- Clear separation between graph structure and runtime state
- Filesystem can only detect binary existence (complete vs not complete)

**Note:** `inProgress` and `failed` states are deferred to future slices. They would require external state tracking (e.g., a status file) since file existence alone cannot distinguish these states.

### Decision: Zod for Schema Validation
Use Zod for validating YAML schema structure and deriving TypeScript types.

**Rationale:**
- Already a project dependency (v4.0.17) used in `src/core/schemas/`
- Type inference via `z.infer<>` - single source of truth for types
- Runtime validation with detailed error messages
- Consistent with existing project patterns (`base.schema.ts`, `config-schema.ts`)

**Alternatives considered:**
- Manual validation: More code, error-prone, no type inference
- JSON Schema: Would require additional dependency, less TypeScript integration
- io-ts: Not already in project, steeper learning curve

### Decision: Two-Level Schema Resolution
Schemas resolve from global user data directory, falling back to package built-ins.

**Resolution order:**
1. `${XDG_DATA_HOME:-~/.local/share}/openspec/schemas/<name>.yaml` - Global user override
2. `<package>/schemas/<name>.yaml` - Built-in defaults

**Rationale:**
- Follows XDG Base Directory Specification (schemas are data, not config)
- Mirrors existing `getGlobalConfigDir()` pattern in `src/core/global-paths.ts`
- Built-ins baked into package, never auto-copied
- Users customize by creating files in global data dir
- Simple - no project-level overrides (can add later if needed)

**XDG compliance:**
- Uses `XDG_DATA_HOME` env var when set (all platforms)
- Unix/macOS fallback: `~/.local/share/openspec/`
- Windows fallback: `%LOCALAPPDATA%/openspec/`

**Alternatives considered:**
- Project-level overrides: Added complexity, not needed initially
- Auto-copy to user space: Creates drift, harder to update defaults
- Config directory (`XDG_CONFIG_HOME`): Schemas are workflow definitions (data), not user preferences (config)

### Decision: Template Field Parsed But Not Resolved
The `template` field is required in schema YAML for completeness, but template resolution is deferred to Slice 3.

**Rationale:**
- Slice 1 focuses on "What's Ready?" - dependency and completion queries only
- Template paths are validated syntactically (non-empty string) but not resolved
- Keeps Slice 1 focused and independently testable

### Decision: Cycle Error Format
Cycle errors list all artifact IDs in the cycle for easy debugging.

**Format:** `"Cyclic dependency detected: A → B → C → A"`

**Rationale:**
- Shows the full cycle path, not just that a cycle exists
- Actionable - developer can see exactly which artifacts to fix
- Consistent with Kahn's algorithm which naturally identifies cycle participants

## Data Structures

**Zod Schemas (source of truth):**

```typescript
import { z } from 'zod';

// Artifact definition schema
export const ArtifactSchema = z.object({
id: z.string().min(1, 'Artifact ID is required'),
generates: z.string().min(1), // e.g., "proposal.md" or "specs/*.md"
description: z.string(),
template: z.string(), // path to template file
requires: z.array(z.string()).default([]),
});

// Full schema YAML structure
export const SchemaYamlSchema = z.object({
name: z.string().min(1, 'Schema name is required'),
version: z.number().int().positive(),
description: z.string().optional(),
artifacts: z.array(ArtifactSchema).min(1, 'At least one artifact required'),
});

// Derived TypeScript types
export type Artifact = z.infer<typeof ArtifactSchema>;
export type SchemaYaml = z.infer<typeof SchemaYamlSchema>;
```

**Runtime State (not Zod - internal only):**

```typescript
// Slice 1: Simple completion tracking via filesystem
type CompletedSet = Set<string>;

// Return type for blocked query
interface BlockedArtifacts {
[artifactId: string]: string[]; // artifact → list of unmet dependencies
}

interface ArtifactGraphResult {
completed: string[];
ready: string[];
blocked: BlockedArtifacts;
buildOrder: string[];
}
```

## File Structure

```
src/core/artifact-graph/
├── index.ts # Public exports
├── types.ts # Zod schemas and type definitions
├── graph.ts # ArtifactGraph class
├── state.ts # State detection logic
├── resolver.ts # Schema resolution (global → built-in)
└── schemas/ # Built-in schema definitions (package level)
├── spec-driven.yaml # Default: proposal → specs → design → tasks
└── tdd.yaml # Alternative: tests → implementation → docs
```

**Schema Resolution Paths:**
- Global user override: `${XDG_DATA_HOME:-~/.local/share}/openspec/schemas/<name>.yaml`
- Package built-in: `src/core/artifact-graph/schemas/<name>.yaml` (bundled with package)

## Risks / Trade-offs

| Risk | Mitigation |
|------|------------|
| Glob pattern edge cases | Use well-tested glob library (fast-glob or similar) |
| Cycle detection | Kahn's algorithm naturally fails on cycles; provide clear error |
| Schema evolution | Version field in schema, validate on load |

## Open Questions

None - all questions resolved in Decisions section.
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
## Why

The current OpenSpec system relies on conventions and AI inference for artifact ordering. A formal artifact graph with dependency awareness would enable deterministic "what's ready?" queries, making the system more predictable and enabling future features like automated pipeline execution.

## What Changes

- Add `ArtifactGraph` class to model artifacts as a DAG with dependency relationships
- Add `ArtifactState` type to track completion status (completed, in_progress, failed)
- Add filesystem-based state detection using file existence and glob patterns
- Add schema YAML parser to load artifact definitions
- Implement topological sort (Kahn's algorithm) for build order calculation
- Add `getNextArtifacts()` to find artifacts ready for creation

## Impact

- Affected specs: New `artifact-graph` capability
- Affected code: `src/core/artifact-graph/` (new directory)
- No changes to existing functionality - this is a parallel module
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
## ADDED Requirements

### Requirement: Schema Loading
The system SHALL load artifact graph definitions from YAML schema files.

#### Scenario: Valid schema loaded
- **WHEN** a valid schema YAML file is provided
- **THEN** the system returns an ArtifactGraph with all artifacts and dependencies

#### Scenario: Invalid schema rejected
- **WHEN** a schema YAML file is missing required fields
- **THEN** the system throws an error with a descriptive message

#### Scenario: Cyclic dependencies detected
- **WHEN** a schema contains cyclic artifact dependencies
- **THEN** the system throws an error listing the artifact IDs in the cycle

#### Scenario: Invalid dependency reference
- **WHEN** an artifact's `requires` array references a non-existent artifact ID
- **THEN** the system throws an error identifying the invalid reference

#### Scenario: Duplicate artifact IDs rejected
- **WHEN** a schema contains multiple artifacts with the same ID
- **THEN** the system throws an error identifying the duplicate

### Requirement: Build Order Calculation
The system SHALL compute a valid topological build order for artifacts.

#### Scenario: Linear dependency chain
- **WHEN** artifacts form a linear chain (A → B → C)
- **THEN** getBuildOrder() returns [A, B, C]

#### Scenario: Diamond dependency
- **WHEN** artifacts form a diamond (A → B, A → C, B → D, C → D)
- **THEN** getBuildOrder() returns A before B and C, and D last

#### Scenario: Independent artifacts
- **WHEN** artifacts have no dependencies
- **THEN** getBuildOrder() returns them in a stable order

### Requirement: State Detection
The system SHALL detect artifact completion state by scanning the filesystem.

#### Scenario: Simple file exists
- **WHEN** an artifact generates "proposal.md" and the file exists
- **THEN** the artifact is marked as completed

#### Scenario: Simple file missing
- **WHEN** an artifact generates "proposal.md" and the file does not exist
- **THEN** the artifact is not marked as completed

#### Scenario: Glob pattern with files
- **WHEN** an artifact generates "specs/*.md" and the specs/ directory contains .md files
- **THEN** the artifact is marked as completed

#### Scenario: Glob pattern empty
- **WHEN** an artifact generates "specs/*.md" and the specs/ directory is empty or missing
- **THEN** the artifact is not marked as completed

#### Scenario: Missing change directory
- **WHEN** the change directory does not exist
- **THEN** all artifacts are marked as not completed (empty state)

### Requirement: Ready Artifact Query
The system SHALL identify which artifacts are ready to be created based on dependency completion.

#### Scenario: Root artifacts ready initially
- **WHEN** no artifacts are completed
- **THEN** getNextArtifacts() returns artifacts with no dependencies

#### Scenario: Dependent artifact becomes ready
- **WHEN** an artifact's dependencies are all completed
- **THEN** getNextArtifacts() includes that artifact

#### Scenario: Blocked artifacts excluded
- **WHEN** an artifact has uncompleted dependencies
- **THEN** getNextArtifacts() does not include that artifact

### Requirement: Completion Check
The system SHALL determine when all artifacts in a graph are complete.

#### Scenario: All complete
- **WHEN** all artifacts in the graph are in the completed set
- **THEN** isComplete() returns true

#### Scenario: Partially complete
- **WHEN** some artifacts in the graph are not completed
- **THEN** isComplete() returns false

### Requirement: Blocked Query
The system SHALL identify which artifacts are blocked and return all their unmet dependencies.

#### Scenario: Artifact blocked by single dependency
- **WHEN** artifact B requires artifact A and A is not complete
- **THEN** getBlocked() returns `{ B: ['A'] }`

#### Scenario: Artifact blocked by multiple dependencies
- **WHEN** artifact C requires A and B, and only A is complete
- **THEN** getBlocked() returns `{ C: ['B'] }`

#### Scenario: Artifact blocked by all dependencies
- **WHEN** artifact C requires A and B, and neither is complete
- **THEN** getBlocked() returns `{ C: ['A', 'B'] }`
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
## 1. Type Definitions
- [x] 1.1 Create `src/core/artifact-graph/types.ts` with Zod schemas (`ArtifactSchema`, `SchemaYamlSchema`) and inferred types via `z.infer<>`
- [x] 1.2 Define `CompletedSet` (Set<string>), `BlockedArtifacts`, and `ArtifactGraphResult` types for runtime state

## 2. Schema Parser
- [x] 2.1 Create `src/core/artifact-graph/schema.ts` with YAML loading and Zod validation via `.safeParse()`
- [x] 2.2 Implement dependency reference validation (ensure `requires` references valid artifact IDs)
- [x] 2.3 Implement duplicate artifact ID detection
- [x] 2.4 Add cycle detection during schema load (error format: "Cyclic dependency detected: A → B → C → A")

## 3. Artifact Graph Core
- [x] 3.1 Create `src/core/artifact-graph/graph.ts` with ArtifactGraph class
- [x] 3.2 Implement `fromYaml(path)` - load graph from schema file
- [x] 3.3 Implement `getBuildOrder()` - topological sort via Kahn's algorithm
- [x] 3.4 Implement `getArtifact(id)` - retrieve single artifact definition
- [x] 3.5 Implement `getAllArtifacts()` - list all artifacts

## 4. State Detection
- [x] 4.1 Create `src/core/artifact-graph/state.ts` with state detection logic
- [x] 4.2 Implement file existence checking for simple paths
- [x] 4.3 Implement glob pattern matching for multi-file artifacts
- [x] 4.4 Implement `detectCompleted(graph, changeDir)` - scan filesystem and return CompletedSet
- [x] 4.5 Handle missing changeDir gracefully (return empty CompletedSet)

## 5. Ready Calculation
- [x] 5.1 Implement `getNextArtifacts(graph, completed)` - find artifacts with all deps completed
- [x] 5.2 Implement `isComplete(graph, completed)` - check if all artifacts done
- [x] 5.3 Implement `getBlocked(graph, completed)` - return BlockedArtifacts map (artifact → unmet deps)

## 6. Schema Resolution
- [x] 6.1 Create `src/core/artifact-graph/resolver.ts` with schema resolution logic
- [x] 6.2 Add `getGlobalDataDir()` to `src/core/global-config.ts` (XDG_DATA_HOME with platform fallbacks)
- [x] 6.3 Implement `resolveSchema(name)` - global (`${XDG_DATA_HOME}/openspec/schemas/`) → built-in fallback

## 7. Built-in Schemas
- [x] 7.1 Create `src/core/artifact-graph/schemas/spec-driven.yaml` (default: proposal → specs → design → tasks)
- [x] 7.2 Create `src/core/artifact-graph/schemas/tdd.yaml` (alternative: tests → implementation → docs)

## 8. Integration
- [x] 8.1 Create `src/core/artifact-graph/index.ts` with public exports

## 9. Testing
- [x] 9.1 Test: Parse valid schema YAML returns correct artifact graph
- [x] 9.2 Test: Parse invalid schema (missing fields) throws descriptive error
- [x] 9.3 Test: Duplicate artifact IDs throws error
- [x] 9.4 Test: Invalid `requires` reference throws error identifying the invalid ID
- [x] 9.5 Test: Cycle in schema throws error listing cycle path (e.g., "A → B → C → A")
- [x] 9.6 Test: Compute build order returns correct topological ordering (linear chain)
- [x] 9.7 Test: Compute build order handles diamond dependencies correctly
- [x] 9.8 Test: Independent artifacts return in stable order
- [x] 9.9 Test: Empty/missing changeDir returns empty CompletedSet
- [x] 9.10 Test: File existence marks artifact as completed
- [x] 9.11 Test: Glob pattern specs/*.md detected as complete when files exist
- [x] 9.12 Test: Glob pattern with empty directory not marked complete
- [x] 9.13 Test: getNextArtifacts returns only root artifacts when nothing completed
- [x] 9.14 Test: getNextArtifacts includes artifact when all deps completed
- [x] 9.15 Test: getBlocked returns artifact with all unmet dependencies listed
- [x] 9.16 Test: isComplete() returns true when all artifacts completed
- [x] 9.17 Test: isComplete() returns false when some artifacts incomplete
- [x] 9.18 Test: Schema resolution finds global override before built-in
- [x] 9.19 Test: Schema resolution falls back to built-in when no global
Loading
Loading