CrucibleFramework

Thin orchestration layer for ML experiment pipelines

---

What's New (v0.5.2 - 2025-12-28)

• Oban-style Repo injection: Host applications now provide their own Repo via config :crucible_framework, repo: MyApp.Repo
• No auto-start by default: Repo is NOT started automatically; set start_repo: true for legacy behavior
• New repo/0 and repo!/0 functions: Access the configured Repo module programmatically
• Dependency updates: crucible_trace bumped to ~> 0.3.1, telemetry to ~> 1.3

What's New (v0.5.1 - 2025-12-27)

• Optional dependencies: crucible_bench and crucible_trace are optional; missing bench errors fast, missing trace disables tracing with a warning
• Examples refresh: New runnable examples plus examples/run_all.sh runner
• Dependency update: crucible_bench bumped to ~> 0.4.0
• Postgres driver: postgrex minimum version raised to >= 0.21.1
• Persistence tests: Integration tests are opt-in via CRUCIBLE_DB_ENABLED=true

What's New (v0.5.0 - 2025-12-27)

• BREAKING: describe/1 callback is now required (removed from @optional_callbacks)
• Schema Module: New Crucible.Stage.Schema for canonical schema definition and validation
• Schema Normalizer: New Crucible.Stage.Schema.Normalizer for legacy schema conversion
• Options Validator: New Crucible.Stage.Validator for runtime options validation
• Registry Enhancements: list_stages_with_schemas/0 and stage_schema/1 for schema access
• Mix Task: New mix crucible.stages command for stage discovery
• Pipeline Validation: Opt-in validate_options: :warn | :error mode in runner
• Conformance Tests: Comprehensive tests for all framework stages

See CHANGELOG.md for the complete migration guide.

What's New (v0.4.1 - 2025-12-26)

• Stage Contract: Enforced describe/1 policy for all stage implementations
• Enhanced Documentation: Comprehensive Crucible.Stage behaviour docs with schema specification
• Runner Documentation: Clarified that Crucible.Pipeline.Runner is the authoritative runner
• Schema Types: Defined type specifications for stage option schemas
• Built-in Stages: Updated all built-in stages with proper describe/1 schemas

What's New (v0.4.0 - 2025-12-25)

• BREAKING: Simplified to pure orchestration layer (~2,000 LOC from ~5,300 LOC)
• Removed: Backend infrastructure (moved to crucible_train)
• Removed: Data loading stages (domain-specific)
• Removed: Analysis adapters (domain-specific)
• Removed: Fairness stages (moved to ExFairness)
• Removed: BackendCall stage (moved to crucible_train)
• Simplified: Context struct with Phoenix-style assigns for domain data
• Updated: crucible_ir ~> 0.2.0 with new Training/Deployment/Feedback IR

Purpose

CrucibleFramework provides:

• Pipeline Execution - Sequential stage execution with Context threading
• Stage Behaviour - Clean interface for composable pipeline stages
• Optional Persistence - Ecto-backed experiment run tracking
• Telemetry Integration - Event emission for observability

This library focuses purely on orchestration. Domain-specific functionality belongs in specialized packages:

Domain	Package
Training	crucible_train (future)
CNS Dialectics	cns_crucible
Fairness	ExFairness
XAI	crucible_xai

---

Quick Start

def deps do
  [
    {:crucible_framework, "~> 0.5.1"}
  ]
end

Define and Run an Experiment

experiment = %CrucibleIR.Experiment{
  id: "my-experiment",
  backend: %CrucibleIR.BackendRef{id: :my_backend},
  pipeline: [
    %CrucibleIR.StageDef{name: :validate},
    %CrucibleIR.StageDef{name: :data_checks},
    %CrucibleIR.StageDef{name: :bench},
    %CrucibleIR.StageDef{name: :report}
  ]
}

{:ok, ctx} = CrucibleFramework.run(experiment)

---

Examples

Runnable scripts live under examples/. Start with:

mix run examples/01_core_pipeline.exs

Run the full set with:

./examples/run_all.sh

See examples/README.md for descriptions and optional dependency notes.

Core Modules

Crucible.Context

Runtime context threaded through pipeline stages. Uses Phoenix-style assigns for domain-specific data:

ctx = %Crucible.Context{
  experiment_id: "exp-1",
  run_id: "run-1",
  experiment: experiment
}

# Add metrics
ctx = Crucible.Context.put_metric(ctx, :accuracy, 0.95)

# Store domain data in assigns (training stages, CNS stages, etc.)
ctx = Crucible.Context.assign(ctx, :dataset, my_data)
ctx = Crucible.Context.assign(ctx, :backend_session, session)
ctx = Crucible.Context.assign(ctx, :snos, extracted_snos)

# Track stage completion
ctx = Crucible.Context.mark_stage_complete(ctx, :data_load)

Context Helper Functions

Category	Functions
Metrics	put_metric/3, get_metric/3, update_metric/3, merge_metrics/2, has_metric?/2
Outputs	add_output/2, add_outputs/2
Artifacts	put_artifact/3, get_artifact/3, has_artifact?/2
Assigns	assign/2, assign/3
Stages	mark_stage_complete/2, stage_completed?/2, completed_stages/1

Crucible.Stage

Behaviour for pipeline stages. All stages must implement both run/2 and describe/1:

defmodule MyApp.Stage.CustomStage do
  @behaviour Crucible.Stage

  @impl true
  def run(%Crucible.Context{} = ctx, opts) do
    # Do work, update ctx
    {:ok, updated_ctx}
  end

  @impl true
  def describe(_opts) do
    %{
      name: :custom,
      description: "My custom stage",
      required: [:input_path],
      optional: [:format, :verbose],
      types: %{
        input_path: :string,
        format: {:enum, [:json, :csv]},
        verbose: :boolean
      },
      defaults: %{
        format: :json,
        verbose: false
      }
    }
  end
end

Stage Contract

All stages must implement describe/1 returning a canonical schema:

Field	Required	Type	Description
name	Yes	atom	Stage identifier
description	Yes	string	Human-readable description
required	Yes	list of atoms	Required option keys
optional	Yes	list of atoms	Optional option keys
types	Yes	map	Type specifications for options
defaults	No	map	Default values for optional fields
version	No	string	Stage version
__extensions__	No	map	Domain-specific metadata

Use mix crucible.stages to list available stages and their schemas:

$ mix crucible.stages
$ mix crucible.stages --name bench

Built-in Stages

Stage	Purpose
Crucible.Stage.Validate	Pre-flight pipeline validation
Crucible.Stage.DataChecks	Lightweight data validation (reads from assigns[:examples])
Crucible.Stage.Guardrails	Safety checks via adapters
Crucible.Stage.Bench	Statistical analysis (requires crucible_bench)
Crucible.Stage.Report	Output generation

Crucible.Registry

Stage module resolution from config:

# In config.exs
config :crucible_framework,
  stage_registry: %{
    validate: Crucible.Stage.Validate,
    bench: Crucible.Stage.Bench,
    my_stage: MyApp.Stage.Custom
  }

---

Architecture

CrucibleFramework.run(experiment)
    |
    v
Crucible.Pipeline.Runner
    |
    +-> Stage 1: Validate
    +-> Stage 2: CustomDataLoader (domain-specific)
    +-> Stage 3: CustomBackendCall (domain-specific)
    +-> Stage 4: Bench
    +-> Stage 5: Report
    |
    v
{:ok, final_context}

Domain-Specific Stages

Training, CNS, and other domain-specific stages should be implemented in their respective packages and registered via config:

# crucible_train would provide:
config :crucible_framework,
  stage_registry: %{
    data_load: CrucibleTrain.Stage.DataLoad,
    backend_call: CrucibleTrain.Stage.BackendCall,
    # ...
  }

# cns_crucible would provide:
config :crucible_framework,
  stage_registry: %{
    cns_extract: CnsCrucible.Stage.SNOExtraction,
    cns_topology: CnsCrucible.Stage.TopologyAnalysis,
    # ...
  }

---

Configuration

Database Configuration (Oban Pattern)

CrucibleFramework uses dynamic repo injection - your host application provides the Repo:

# config/config.exs
config :crucible_framework,
  repo: MyApp.Repo,  # Required: host app's Repo module
  stage_registry: %{
    validate: Crucible.Stage.Validate,
    data_checks: Crucible.Stage.DataChecks,
    guardrails: Crucible.Stage.Guardrails,
    bench: Crucible.Stage.Bench,
    report: Crucible.Stage.Report
  },
  guardrail_adapter: Crucible.Stage.Guardrails.Noop

# Your host app's Repo configuration
config :my_app, MyApp.Repo,
  database: "my_app_dev",
  username: "postgres",
  password: "postgres",
  hostname: "localhost"

Then start your Repo in your application's supervision tree:

# lib/my_app/application.ex
children = [
  MyApp.Repo,
  # ... other children
]

Migrations

Copy migrations from deps/crucible_framework/priv/repo/migrations/ or run:

mix crucible_framework.install

Legacy Mode

For backwards compatibility, set start_repo: true to auto-start CrucibleFramework.Repo:

config :crucible_framework,
  start_repo: true,
  ecto_repos: [CrucibleFramework.Repo]

config :crucible_framework, CrucibleFramework.Repo,
  database: "crucible_dev",
  username: "crucible_dev",
  password: "crucible_dev_pw",
  hostname: "localhost"

---

Dependencies

• crucible_ir - Shared experiment IR structs (v0.2.0+)
• crucible_bench - Statistical testing (optional; required for Crucible.Stage.Bench)
• crucible_trace - Causal reasoning traces (optional)

Optional Dependencies

CrucibleFramework runs without the optional packages below; they enable specific features.

crucible_bench

• Enables Crucible.Stage.Bench and statistical testing helpers
• If missing and :bench is used, the stage returns {:error, {:missing_dependency, :crucible_bench}}

crucible_trace

• Enables trace lifecycle helpers and enable_trace: true in the runner
• If missing, tracing is disabled and a warning is logged; export/load helpers return nil or {:error, {:missing_dependency, :crucible_trace}}

Enabling optional packages

def deps do
  [
    {:crucible_framework, "~> 0.5.1"},
    {:crucible_bench, "~> 0.4.0"},
    {:crucible_trace, "~> 0.3.0"}
  ]
end

If you do not need bench or tracing, omit those deps and remove :bench from your pipeline (or from stage_registry) to keep the core slim. See examples/02_bench_optional.exs and examples/03_trace_optional.exs for optional-dep usage.

---

Development

# Setup
mix deps.get && mix compile

# Tests
mix test

# Integration tests (persistence; requires CRUCIBLE_DB_ENABLED=true)
CRUCIBLE_DB_ENABLED=true MIX_ENV=test mix test --include integration

# Quality checks
mix format
mix credo --strict
mix dialyzer

---

Related Repositories

Repository	Purpose
crucible_ir	Shared IR structs
crucible_bench	Statistical testing
crucible_trace	Causal transparency
cns	CNS dialectical reasoning
cns_crucible	CNS + Crucible integration

---

License

MIT. See LICENSE.