Skip to content

Repository files navigation

CloseLoop AI — Multi-Agent Timing Closure Copilot

"Cursor for chip design." — Built for WeaveHacks 4, June 2026.

A full-stack visual sandbox where a team of AI agents finds, diagnoses, and fixes chip-design timing violations in real time. Watch the red critical path turn green as ECO fixes ripple through the graph.


What problem does this solve?

Chip physical-design timing closure is one of the most time-consuming steps in semiconductor development. According to industry data (Cadence Certus, TDF), full design closure traditionally takes weeks to months. Engineers manually read thousands of lines of OpenSTA timing reports, identify root causes, and iteratively apply ECO (engineering change order) fixes.

CloseLoop AI automates this loop with six specialized agents:

Agent Role
STA Agent Reads timing graph, finds worst path, explains root causes
Fix Agent Recommends concrete ECO fixes (upsize, buffer insertion, etc.) with PPA impact
PPA Agent Validates power/area/congestion tradeoffs before any fix is applied
Orchestrator Manages the closure loop, decides when to stop (≤ 5 iterations)
Explainability Translates results into plain English for non-engineers
MetaAgent After each run, reviews Weave traces vs ground-truth slack deltas and proposes prompt edits the user can accept to bias the next run (closed self-improvement loop)

Research baseline

This work is informed by and compared against:

  • Nexus (arXiv:2502.19091) — multi-agent timing closure on FPGA (VTR benchmarks), 30% power saving.
  • ChatEDA (IEEE TCAD 2024, arXiv:2308.10204) — LLM agent for RTL-to-GDSII flow.
  • Manual baseline — industry data from Cadence Certus documentation and TDF case studies.

CloseLoop AI differentiates by being:

  1. Open-source
  2. ASIC-targeted (Nexus is FPGA-only)
  3. Self-correcting — orchestrator detects diminishing returns and stops
  4. Self-improving — MetaAgent compares predicted vs actual slack improvements, surfaces failure modes, and applies prompt edits that change the next run's behavior (verified end-to-end)
  5. Live visualization — only system with a visual sandbox
  6. Real OpenROAD design support — ships with the canonical gcd_nangate45 test design from OpenROAD's CI suite as a selectable preset; not just synthetic toys

How to run

git clone <repo>
cd closeloop-ai
npm install --legacy-peer-deps   # see "Why --legacy-peer-deps" below
npm run dev

Open http://localhost:3000. The demo design loads automatically.

Zero API keys required — the app runs in full mock mode by default.

Why --legacy-peer-deps? CopilotKit's dependency tree pins zod@3, while @openai/agents requires zod@4. We let both coexist by nesting zod@4 inside the @openai/agents* packages (handled automatically by the postinstall hook at scripts/nest-zod-v4.sh). Node's module resolution finds the right version from each caller's location at runtime. See the "Agentic orchestration" section below for details.

Real-API mode (going live)

cp .env.example .env
# Edit .env and add your keys

.env:

# Required for real agents
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o            # ← use gpt-4o; do NOT use gpt-5.x for the
                               # closure loop (17-80s per agent call, too slow
                               # for a demo, and rejects temperature overrides)

# Optional — falls back to in-memory store if missing
# Redis Cloud (TLS):  rediss://default:PASSWORD@HOST:PORT
# Local:              redis://localhost:6379
REDIS_URL=

# Optional — falls back to no-op tracer if missing
WANDB_API_KEY=...
WANDB_PROJECT=closeloop-ai

# Set to anything other than "true" to enable real mode
NEXT_PUBLIC_MOCK_MODE=

Each key is independent — missing keys quietly fall back to their mock counterpart, so you can roll out incrementally (e.g. real OpenAI + mock Redis on day 1, add real Redis on day 2).

Verify integrations:

# Hit the health endpoint
npm run dev
curl http://localhost:3000/api/health | jq

# Or run the self-test (also makes one GPT-4o call ~$0.01)
npm run test:real

The self-test reports environment, Redis ping, an end-to-end STA agent call, and Weave status. Use it before demo'ing.

Performance (measured on demo design with gpt-4o):

Stage Latency
Orchestrator ~1.3s
STA agent ~2.9s
Fix agent ~2.8s
PPA agent ~2.6s (runs in parallel with apply)
Full closure loop ~9.7s for 1-iteration closure

All agents use strict JSON-schema enforcement (zod via the Agents SDK, or hand-written JSON Schemas in the fallback path) — the model is server-side constrained to return valid output, no retries needed.

Cost note: in real mode, a single full closure loop makes 4-20 GPT-4o calls (≈ $0.05-0.20 of tokens depending on iteration count). The orchestrator hard-stops at 5 iterations and on closure/diminishing returns to bound this.

Profile your own setup:

npm run profile:sdk     # default path (uses @openai/agents)
npm run profile:real    # fallback path (hand-rolled OpenAI calls)

Redis security: if you accidentally commit or paste a Redis password, rotate it immediately in your provider's console. .env is gitignored — keep credentials there.


Demo script (for judges)

  1. The app loads the demo design — an 8-cell ALU datapath in nangate45 ASIC technology. The red critical path is visible: FF1 → NAND_X1 → BUF_X1 (weak) → INV_X1 → FF2. Initial WNS = -0.226ns (timing violation).
  2. Click Judge Mode 👁 for a 5-step non-technical walkthrough.
  3. Or click Run STA to invoke the STA agent. Watch the agent timeline at the bottom fill in. The Timing Panel shows the cell-by-cell delay breakdown, the SlackGauge animates, and root causes appear.
  4. The Fix Agent auto-runs after STA. Two fix cards appear in the right panel: Upsize BUF_X1 → BUF_X4 and Insert BUF_X2 at NET_LONG_1 midpoint.
  5. Click Apply Fix. The red path turns green; slack jumps to +0.148ns (closure achieved). A new buffer node slides in at the midpoint of the long wire.
  6. Click Run Full Closure to see the orchestrator run the entire loop automatically — every agent call shows up as a nested op in the live W&B Weave trace tree (link below).
  7. Use the Design dropdown in the header to switch to OpenROAD gcd (real, 426 cells) — this is the canonical small ASIC benchmark from the OpenROAD project's CI. The chip graph now shows real cell IDs (_677_, _357_, …) at their real placed coordinates, on the 21-deep NAND chain that's the actual gcd critical path. Run Full Closure again — agents reason about real cells.
  8. Scroll to the Agent Self-Critique panel and click Critique Last Run. The MetaAgent compares predicted vs actual slack improvements from the run, surfaces specific failure modes, and proposes prompt edits. Click one or more, then Apply — the next closure run will splice these into the live agent prompts.
  9. Scroll to the Benchmark Comparison panel — CloseLoop AI vs Nexus vs ChatEDA vs Manual. We close in 0.24 minutes (14 seconds) vs Nexus's 3.2 minutes and Manual's 180 minutes (208× faster than manual).
  10. Open the CopilotKit Chat sidebar (bottom-right) and ask: "Explain what just happened to a non-engineer."
  11. Browse the W&B trace tree at the URL printed by /api/health — every agent call from the demo is captured with full structured inputs/outputs.

Why this is not a fake hardcoded demo

Timing is recomputed from physical formulas on every fix application:

  • Cell delay: baseDelay / sqrt(driveStrength) + fanout_penalty
  • Wire delay: cubic in length — (length/100)³ × 0.05 ns (RC model)
  • The final slack depends on which cells are upsized, where buffers are inserted, and the length of the resulting net segments.
  • The MCMM panel applies real PVT derating per corner — the SS/0.9V/125C worst-case corner is automatically identified.
  • You can paste your own OpenSTA timing report via the parser endpoint (POST /api/report/parse).

Open the JS console while you click around — you will see [MOCK REDIS], [MOCK WEAVE], [MOCK AGENT] traces showing exactly what would be sent to real systems.


Architecture

┌─────────────────────────────────────────────┐
│  Next.js 14 App Router (TypeScript)         │
│  ┌──────────────┐    ┌──────────────────┐  │
│  │ Dashboard UI │←──→│ Zustand store    │  │
│  │ (React Flow) │    └──────────────────┘  │
│  └──────┬───────┘                          │
│         │ fetch                            │
│  ┌──────▼───────────────────────────────┐  │
│  │  /api/* route handlers               │  │
│  └──────┬───────────────────────────────┘  │
│         │                                  │
│  ┌──────▼───────────────────────────────┐  │
│  │  lib/apiRouter — switches mock/real  │  │
│  └──┬────────────┬─────────────┬────────┘  │
│     ▼            ▼             ▼           │
│  Agents      Redis         Weave           │
│ (5 mock or  (in-mem or    (no-op or        │
│  GPT-4o)    real ioredis)  real W&B)       │
└─────────────────────────────────────────────┘

Core directories

  • app/ — Next.js routes (dashboard + 14 API endpoints): design/{demo,current,load}, sta/analyze, fix/{recommend,apply}, closure/run, iterations, report/parse, benchmark, mcmm, health, copilotkit/{,threads}, critique/{run,apply} (new).
  • components/ — UI panels (ChipGraph, TimingPanel, AgentTimeline, FixCards, JudgeMode, MCMMPanel, BenchmarkPanel, CopilotChat, SelfCritique).
  • lib/agents/sdk/OpenAI Agents SDK agents (default real path).
  • lib/agents/real/ — Hand-rolled OpenAI structured-output agents (fallback).
  • lib/agents/meta/ — MetaAgent (closes the self-improvement loop).
  • lib/agents/promptOverrides.ts — Redis-backed agent prompt overrides spliced in at call time via withOverride().
  • lib/realDesign/.def parser + OpenROAD gcd loader (cell coordinates, net routing, critical-path subgraph extraction).
  • lib/weaveBootstrap.ts — single-init Weave client + instrumentOpenAIAgents()
    • wrapOpenAI() + weaveOp() helper.
  • lib/timing/ — Timing engine (formulas, ECO budget, report parser).
  • lib/mock/ — Mock agents, mock Redis, mock Weave.
  • lib/benchmark/ — Research-paper baseline data.
  • data/ — Demo design JSON, sample OpenSTA report, real-gcd/ (real OpenROAD gcd_nangate45.{def,v,sdc} fetched verbatim from upstream).
  • scripts/ — Profiling scripts and the nest-zod-v4.sh postinstall hook.

Agentic orchestration

The closure loop is orchestrated using the OpenAI Agents SDK (@openai/agents@0.11.6):

  • Each agent (STA, Fix, PPA, Orchestrator, Explainability) is defined as new Agent({ name, instructions, outputType: ZodSchema, model }) in lib/agents/sdk/sdkAgents.ts.
  • outputType schemas (in lib/agents/sdk/schemas.ts) drive OpenAI's strict structured-output mode — the model's response is JSON-schema validated server-side before it reaches us.
  • A tiered model strategy applies: reasoning-heavy agents (STA, Orchestrator) get OPENAI_MODEL (default gpt-4o); lighter agents default to gpt-4o-mini. Override per-agent with OPENAI_MODEL_STA, OPENAI_MODEL_FIX, etc.
  • The closure loop is hand-rolled around Runner.run() calls because we need explicit hard-stop guards (≤ 5 iterations, +0.02ns margin, diminishing-returns detection) that the SDK doesn't model directly.
  • PPA runs in parallel with applying fixes — it's advisory only, so there's no need to block the apply step on it.

Fallback path

Set CLOSELOOP_AGENT_BACKEND=real in .env to use a hand-rolled implementation in lib/agents/real/ that calls OpenAI directly with response_format: { type: "json_schema", strict: true }. Same correctness guarantees, no SDK dependency. Useful for debugging or environments where the nested-zod trick isn't desired.

The zod v3 + v4 coexistence trick

CopilotKit's tree pins zod@^3.25.76; @openai/agents requires zod@^4.0.0. We don't pick one — we give each its own copy:

  • Top level: node_modules/zod = v3 (used by CopilotKit, ag-ui, etc.)
  • Nested: node_modules/@openai/agents*/node_modules/zod = v4 (used by the Agents SDK)

Node's require('zod') walks up from the caller's directory, so each package finds the right version automatically. The postinstall hook (scripts/nest-zod-v4.sh) installs the nested copies after every npm install --legacy-peer-deps. Idempotent and safe to re-run.


Sponsor integrations

  • OpenAI — Both real backends call OpenAI: the SDK path through @openai/agents, the fallback through openai directly. Mock fallback when OPENAI_API_KEY is missing.
  • RedisgetRedisClient() returns ioredis when REDIS_URL is set (TLS auto-detected from rediss://), otherwise an in-memory Map-backed implementation with the same surface. Live client is wrapped in a graceful-fallback Proxy so any per-op failure (connection blip, quota) silently uses the in-memory mock for that one call without failing the request.
  • W&B Weaveweave@0.15.1 is the real production SDK. Every agent function is weaveOp(fn, name)-wrapped at module load so calling them from inside the also-wrapped closure_loop automatically produces the nested trace tree (per the docs at docs.wandb.ai/weave/guides/tracking/trace-tree). instrumentOpenAIAgents() is called once after init so every model call inside an Agent.run() shows up as a child of the agent that invoked it. wrapOpenAI() does the same for the hand-rolled fallback path. MetaAgent's runs appear as meta_agent ops alongside the rest.
  • CopilotKit — Real <CopilotSidebar> from @copilotkit/react-ui with /api/copilotkit runtime endpoint backed by OpenAIAdapter. The sidebar reads live design state via useCopilotReadable and triggers agents via useCopilotAction (run_sta_analysis, apply_recommended_fixes, explain_to_judge, run_full_closure_loop, show_benchmark_comparison).
  • W&B MCP Server — wandb-mcp-server is wired into the user's Claude Code session. The 20 MCP tools (query_weave_traces_tool, count_weave_traces_tool, query_wandb_tool, etc.) let the developer inspect, count, and diff their own traces from chat — useful for closing the dev loop on agent regressions.

Real OpenROAD design support

In addition to the synthetic 8-cell demo, the app ships with real gcd_nangate45 data fetched verbatim from The-OpenROAD-Project/OpenROAD/test/. Switch via the Design dropdown in the header.

Real-design stats:

  • 734 total components, 426 logic cells after filtering FILLCELL + TAPCELL + PHY_EDGE placeholders.
  • Clock period 0.485 ns (target 2.06 GHz) from gcd_nangate45.sdc.
  • DIEAREA 32.7 × 32.7 µm, 21 placement rows on the FreePDK45_38x28_10R_NP_162NW_34O site.
  • 497 nets with real routed metal2/metal3 segments — wire lengths are computed from the actual routing coordinates, not faked.

How it's wired:

  • lib/realDesign/defParser.ts — minimal LEF/DEF parser. Handles COMPONENTS (with PLACED/FIXED positions), NETS (with pin connections and per-segment routed metal), DIEAREA, UNITS. Computes per-net wire length by summing |dx|+|dy| across routed coordinates.
  • lib/realDesign/cellMapping.ts — maps ~50 distinct nangate45 standard cells (DFF_X1, NAND2_X4, OAI21_X1, …) into the 7 high-level buckets the UI knows about, with drive-strength-aware delay/power/area estimates.
  • lib/realDesign/gcdLoader.ts — DFS finds the worst FF→FF combinational path through the real netlist using real wire delays (the loader uses the same cubic-RC model as the timing engine so the numbers stay consistent). Extracts a ≤ 25-cell visualization subgraph (critical path + 1-hop neighbors) while the timing computation reflects the full path. Projects real DBU coordinates onto the React Flow canvas.

When agents run on the real design, they reason about the actual gcd critical path: _677_ (DFF_X1) → 21-deep NAND chain → _706_ (DFF_X2), WNS = -0.822 ns at the worst-case ss_0p9v_125c signoff corner.


Per-agent model routing (mix OpenAI + W&B Inference + fine-tuned LoRA)

Different agents have very different cost/latency/quality tradeoffs. The reasoning-heavy ones (STA, Orchestrator, MetaAgent) benefit from gpt-4o; the structured-transformation ones (Fix, PPA, Explainability) do fine on much smaller open-source models for ~10× the cost reduction. The model router at lib/llm/router.ts makes this per-agent configurable via env vars.

URI scheme

openai:gpt-4o
openai:gpt-4o-mini
wandb:meta-llama/Llama-3.1-8B-Instruct
wandb:meta-llama/Llama-3.3-70B-Instruct
wandb:microsoft/Phi-4-mini-instruct
wandb:deepseek-ai/DeepSeek-R1-0528          (reasoning, could replace gpt-4o)
wandb-artifact:///entity/project/fix-lora:latest   (your fine-tuned LoRA)

All three providers are OpenAI-API-compatible — the router just swaps the baseURL and auth key. Every client is wrapped with weave.wrapOpenAI() so each chat.completions.create shows up as a Weave child of whatever agent op is on the call stack.

Env vars

# Reasoning-heavy stays on OpenAI (defaults are already this)
MODEL_STA=openai:gpt-4o
MODEL_ORCHESTRATOR=openai:gpt-4o
MODEL_META=openai:gpt-4o

# Lightweight transformations on W&B Inference
MODEL_FIX=wandb:meta-llama/Llama-3.1-8B-Instruct
MODEL_PPA=wandb:microsoft/Phi-4-mini-instruct
MODEL_EXPLAINABILITY=wandb:meta-llama/Llama-3.1-8B-Instruct

# Or use your fine-tune (no other code changes)
MODEL_FIX=wandb-artifact:///syed-omer-shah-intel/closeloop-ai/fix-lora:latest

When any agent is routed to a non-OpenAI provider, the backend automatically swaps from the @openai/agents SDK path to the hand-rolled realAgents path (which constructs a fresh OpenAI-compatible client per call, so it handles mixed providers cleanly). The SDK path is only used when every agent is on OpenAI. This switch is invisible to the UI.

What you see in the demo

  • The AgentTimeline cards show a CPU badge with the actual model that ran each step (e.g. gpt-4o, Llama-3.1-8B-Instruct (W&B), fix-lora:latest (fine-tune)).
  • The /api/llm/routing endpoint returns the full per-agent config so you can verify routing without burning tokens.
  • The /api/health endpoint includes the same data under integrations.llmRouting.perAgent and reports the active backend.

Verified end-to-end

With MODEL_FIX=wandb:meta-llama/Llama-3.1-8B-Instruct and STA/Orch on gpt-4o, a single closure-loop run on the synthetic demo:

  • Closed in 11.8s with WNS -0.226 → +0.058ns (within margin of the all-gpt-4o baseline).
  • Per the W&B trace tree (verified via the wandb MCP), the Fix agent's openai.chat.completions.create child trace has inputs.model="meta-llama/Llama-3.1-8B-Instruct"; STA/Orchestrator children have inputs.model="gpt-4o".

Fine-tuned models — actually shipped

The Fix-agent fine-tune is wired up end-to-end on W&B Serverless SFT (CoreWeave GPUs). The pipeline lives in scripts/:

# 1. Generate distillation dataset (gpt-4o teacher, ~$0.25 for 20 examples)
npm run finetune:build 20

# 2. Upload as a W&B artifact (Python CLI does multipart upload)
wandb artifact put data/fix-training-set.jsonl \
    --type training-data \
    --name syed-omer-shah-intel/closeloop-ai/fix-agent-training-set

# 3. Submit SFT job on Llama 3.1 8B base
npm run sft:submit

# 4. Monitor (training runs on CoreWeave GPUs; free during public preview)
npm run sft:watch

# 5. Once training produces non-zero-step checkpoints, the `:latest`
#    alias rolls forward. Activate by setting in .env:
MODEL_FIX=wandb-artifact:///syed-omer-shah-intel/closeloop-ai/fix-lora:latest

A/B benchmark

Same closure loop on the synthetic demo design across three Fix-agent backends (STA/Orchestrator/PPA stayed on gpt-4o so the Fix agent is the only variable). Initial WNS = -0.226 ns.

Fix-agent backend Wall time Iterations Final WNS Closure? Predict-vs-actual err (iter 1)
OpenAI gpt-4o-mini (baseline) 11.1 s 1 +0.187 ns +0.107 ns
W&B Inference Llama 3.1 8B (base) 15.3 s 1 +0.103 ns -0.039 ns
W&B Serverless SFT fix-lora (LoRA on Llama 3.1 8B) 34.3 s 2 +0.256 ns +0.373 ns

How to read these numbers honestly:

  • All three close timing. The closure-loop architecture is robust to the Fix agent's model choice — even an under-trained LoRA still proposes fixes that move the design toward closure (the timing engine validates each fix's actual impact, so bad proposals get caught and the loop iterates).
  • gpt-4o-mini wins on wall time because OpenAI's serving is faster than W&B's Llama inference for this prompt size (~7 s vs ~9 s per call).
  • Base Llama 3.1 8B is the most calibrated in this snapshot — its iter-1 prediction error is the smallest (-0.039 ns vs +0.107 ns for gpt-4o-mini), meaning its estimatedSlackImprovement matched what the timing engine actually measured most closely. This is the kind of signal MetaAgent uses to drive prompt edits.
  • The fix-lora result reflects training in progress. At measurement time the SFT job was still queued on CoreWeave with only a step-0 checkpoint (= base Llama weights with no LoRA adapter applied). The iter-1 prediction error is high (+0.373 ns) because we're effectively measuring base Llama with the wandb-artifact:/// routing path layered on top of it. As training produces higher-step checkpoints, :latest rolls forward automatically and the same env var picks them up — no code change required.

In other words: the routing + tracing + provider-switching path is fully verified, the training pipeline is submitted and queued, and the demo can run on any of the three backends today with one env-var flip. When the LoRA training completes, the Fix-agent prediction error should drop below gpt-4o-mini's, because the LoRA was distilled from gpt-4o (the more expensive teacher) on this exact task.


Self-improvement loop (MetaAgent)

After any closure run, the Agent Self-Critique panel can invoke a post-mortem MetaAgent that:

  1. Compares each Fix proposal's estimatedSlackImprovement against the slack delta the timing engine actually measured after applying it.
  2. Identifies per-agent failure modes (over-confident predictions, fixes targeting non-critical cells, PPA verdicts that proved wrong, …).
  3. Emits a structured CritiqueOutput:
    • summary — short prose verdict on the run's health.
    • findings[] — each tagged with agent, issue, evidence, severity, and a numerical predictionErrorNs.
    • suggestedPromptDeltas[] — short, actionable system-prompt additions (not rewrites) per agent.
    • meanAbsErrorNs — computed locally from ground truth, never trusted to the LLM.
  4. User selects which deltas to accept → /api/critique/apply persists them per-agent in Redis. The next closure loop's agent factories (makeSTAAgent, makeFixAgent, …) call withOverride() which splices the addition onto the base system prompt before passing it to new Agent({ instructions, … }). The override is appended with a timestamp tag so the model knows it came from a prior run's post-mortem.
  5. The MetaAgent itself is weaveOp('meta_agent')-wrapped, so its runs show up as a top-level meta_agent op in the Weave UI alongside the closure_loop traces it analyzed.

Verified end-to-end: in a representative test, Run 1's Fix agent predicted +0.370 ns improvement and actually got +0.357 ns (over by 0.013 ns). MetaAgent flagged the over-confidence and emitted a FIX-agent delta. After applying it, Run 2's Fix agent predicted only +0.300 ns (measurably more conservative — actual still +0.357 ns), proving the override pipeline changes agent behavior.


Cadence Certus parity features

  1. Multi-Corner Multi-Mode (MCMM) analysis — 3 corners (tt_1p0v_25c, ss_0p9v_125c worst-case, ff_1p1v_m40c) with PVT derating.
  2. ECO legalization checker — every fix gets a routability score; warnings flagged if score < 50.
  3. Hierarchical timing — cells grouped by block (ALU, Control).
  4. Signoff correlation — 5% signoff margin applied; paths that pass OpenSTA but fail signoff are flagged in the MCMM panel.
  5. Congestion-aware fix recommendation — ECO budget tracks congestion impact per fix.
  6. TNS-driven optimization — both WNS and TNS tracked in the PPA chart.

Constraints honored

  • ≤ 25 nodes rendered in the chip graph (synthetic demo: 8; real gcd: 25 via the critical-path subgraph extractor).
  • No auth, no login.
  • ≤ 5 closure iterations (orchestrator hard-stops; also stops on +0.02 ns margin or diminishing returns).
  • No real EDA engine — timing is recomputed from physics-consistent formulas every fix application (cubic RC model on real or synthetic wire lengths).
  • 100% offline in mock mode for demo safety.

License

MIT — built at WeaveHacks 4, 2026.

About

WeaveHacks 4 Hackathon project

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages