Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,30 +159,37 @@ LOCAL_MODEL_NAME=llama3.1
## 4. 에이전트 설정 (agents.yaml)

`backend/agents.yaml`에서 에이전트별로 provider와 모델을 독립적으로 지정할 수 있다.
모델은 상단 `models` **티어 별칭**(judge/code)으로 관리한다 — 신모델 출시 시 별칭 두 줄만
바꾸면 전체 에이전트에 반영된다. 특정 에이전트만 다른 모델을 쓰려면 `model:` 로 직접 지정(override).

```yaml
# 동시 실행 에이전트 수 제한
max_concurrent: 2

# 모델 티어 별칭 — 여기 두 줄만 바꾸면 전체 반영
models:
judge: claude-opus-4-8 # 판단·설계·리뷰
code: claude-sonnet-5 # 코드 생성

architect:
provider: claude-cli
model: claude-opus-4-6
model_tier: judge
timeout_seconds: 300
on_timeout: escalate
max_retries_on_timeout: 1
max_tokens: 8192

backend_coder:
provider: claude-cli
model: claude-sonnet-4-6 # 더 저렴한 모델로 교체 가능
model_tier: code
timeout_seconds: 600
on_timeout: retry
max_retries_on_timeout: 1
max_tokens: 16384

frontend_coder:
provider: claude-cli
model: claude-haiku-4-5 # 빠르고 저렴 — 프론트 코딩에 충분
model_tier: code # 더 저렴한 모델을 쓰려면 model: <id> 로 override
timeout_seconds: 600
on_timeout: retry
max_retries_on_timeout: 1
Expand Down
29 changes: 18 additions & 11 deletions backend/agents.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,16 @@ max_concurrent: 2 # 동시 실행 에이전트 수 제한
# - compact prompt (skeleton 이미 채워진 후): orchestrator 300s 충분
# 메인 레포 개발자 환경: 이 값들로 충분. CI/느린 환경: 1.5배 권장.

# ── 모델 티어 별칭 ──────────────────────────────────────────────
# 신모델 출시 시 아래 두 줄만 바꾸면 전체 에이전트에 반영된다.
# 각 에이전트는 `model_tier: judge|code` 로 참조하거나, `model:` 로 직접 지정(override) 가능.
models:
judge: claude-opus-4-8 # 판단·설계·리뷰: architect · designer · orchestrator · reviewer · qa
code: claude-sonnet-5 # 코드 생성: 모든 coder (backend · frontend · mobile 4종)

architect:
provider: claude-cli
model: claude-opus-4-6
model_tier: judge
prompt_path: agents/architect/CLAUDE.md
timeout_seconds: 600
on_timeout: retry
Expand All @@ -18,7 +25,7 @@ architect:

designer:
provider: claude-cli
model: claude-opus-4-6
model_tier: judge
prompt_path: agents/designer/CLAUDE.md
timeout_seconds: 600
on_timeout: retry
Expand All @@ -29,7 +36,7 @@ designer:

orchestrator:
provider: claude-cli
model: claude-opus-4-6
model_tier: judge
prompt_path: agents/orchestrator/CLAUDE.md
timeout_seconds: 300
on_timeout: retry
Expand All @@ -40,7 +47,7 @@ orchestrator:

backend_coder:
provider: claude-cli
model: claude-sonnet-4-6
model_tier: code
prompt_path: agents/backend_coder/CLAUDE.md
timeout_seconds: 600
on_timeout: retry
Expand All @@ -51,7 +58,7 @@ backend_coder:

frontend_coder:
provider: claude-cli
model: claude-haiku-4-5-20251001
model_tier: code
prompt_path: agents/frontend_coder/CLAUDE.md
timeout_seconds: 600
on_timeout: retry
Expand All @@ -63,7 +70,7 @@ frontend_coder:
# M0 first gate — React Native + Expo mobile coder (M1 에서 본 작업)
mobile_coder_rn:
provider: claude-cli
model: claude-haiku-4-5-20251001
model_tier: code
prompt_path: agents/mobile_coder_rn/CLAUDE.md
timeout_seconds: 600
on_timeout: retry
Expand All @@ -75,7 +82,7 @@ mobile_coder_rn:
# M0-B — Flutter mobile coder (M2 에서 본 작업, Dart 비주류라 sonnet)
mobile_coder_flutter:
provider: claude-cli
model: claude-sonnet-4-6
model_tier: code
prompt_path: agents/mobile_coder_flutter/CLAUDE.md
timeout_seconds: 600
on_timeout: retry
Expand All @@ -87,7 +94,7 @@ mobile_coder_flutter:
# M0-B — Android Kotlin native (M3 에서 본 작업, Gradle 빌드 시간 고려 900s)
mobile_coder_android:
provider: claude-cli
model: claude-sonnet-4-6
model_tier: code
prompt_path: agents/mobile_coder_android/CLAUDE.md
timeout_seconds: 900
on_timeout: retry
Expand All @@ -99,7 +106,7 @@ mobile_coder_android:
# M0-B — iOS Swift native (M3, Windows 호스트는 SwiftLint + swift build 만)
mobile_coder_ios:
provider: claude-cli
model: claude-sonnet-4-6
model_tier: code
prompt_path: agents/mobile_coder_ios/CLAUDE.md
timeout_seconds: 900
on_timeout: retry
Expand All @@ -110,7 +117,7 @@ mobile_coder_ios:

reviewer:
provider: claude-cli
model: claude-opus-4-6
model_tier: judge
prompt_path: agents/reviewer/CLAUDE.md
timeout_seconds: 300
on_timeout: escalate
Expand All @@ -121,7 +128,7 @@ reviewer:

qa:
provider: claude-cli
model: claude-opus-4-6
model_tier: judge
prompt_path: agents/qa/CLAUDE.md
timeout_seconds: 600
on_timeout: retry
Expand Down
2 changes: 2 additions & 0 deletions backend/docs/prompt-evaluation-2026-06-10.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@ rn 은 `권위순서 → 자율결정금지 → 골든원칙` 순, flutter/andro
agents.yaml 이 claude-opus-4-6 / sonnet-4-6 / haiku-4-5. 현재 최신 opus-4-8. 단 스킬 경로
(Agent model="sonnet")엔 무관, Orchestra 경로만 해당 → H2 판단 후 결정.

> **2026-07-01 해결**: agents.yaml 을 `model_tier` 별칭(`models: {judge: claude-opus-4-8, code: claude-sonnet-5}`)으로 전환. haiku 미사용(코딩 전부 sonnet code tier). 신모델 출시 시 `models` 두 줄만 교체 → 버전 노후 재발 방지. (config `_resolve_model_tiers`)

### (신규 발견) 미러 divergence
repo `skills/` 와 런타임 `~/.claude/skills/` 가 심링크가 아닌 별도 복사본이고 현재 내용이 다름.
한 쪽만 수정하면 런타임에 반영 안 됨 → 수정 후 미러 동기화 필수.
Expand Down
14 changes: 7 additions & 7 deletions backend/src/dashboard/routes/agents.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@

router = APIRouter(prefix="/api/agents", tags=["agents"])

# agents.yaml에서 허용된 모델 목록 (claude_client 의존 제거)
# 대시보드에서 에이전트 모델 override 시 허용되는 값 (claude_client 의존 제거).
# agents.yaml 의 `models` 티어 별칭이 가리키는 현재 모델 + 대체 선택지.
_ALLOWED_MODELS: frozenset[str] = frozenset(
{
"claude-opus-4-5",
"claude-sonnet-4-5",
"claude-haiku-3-5",
"claude-3-5-sonnet-20241022",
"claude-3-5-haiku-20241022",
"claude-3-opus-20240229",
"claude-opus-4-8", # judge tier (현재)
"claude-sonnet-5", # code tier (현재)
"claude-haiku-4-5-20251001", # 저렴/빠른 대체
"claude-opus-4-6", # 이전 세대 fallback
"claude-sonnet-4-6", # 이전 세대 fallback
}
)

Expand Down
35 changes: 34 additions & 1 deletion backend/src/orchestrator/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ class AgentConfig(BaseModel):
"""Runtime configuration for a single agent."""

provider: Provider
model: str
model: str # concrete 모델 문자열. model_tier 사용 시 로더가 models 별칭으로 채운다.
model_tier: str | None = None # judge|code 등 — agents.yaml 의 models 맵 키 (introspection 용)
prompt_path: str
timeout_seconds: int = 300
on_timeout: OnTimeout = OnTimeout.ESCALATE
Expand Down Expand Up @@ -87,6 +88,36 @@ def all_agents(self) -> dict[str, AgentConfig]:
}


def _resolve_model_tiers(raw: dict) -> dict:
"""`models` 별칭 맵으로 각 에이전트의 `model_tier` 를 concrete `model` 로 해석.

- 상단 `models: {judge: ..., code: ...}` 를 정의하면 에이전트는 `model_tier: judge`
한 줄로 참조 → 신모델 출시 시 `models` 두 줄만 바꾸면 전체 반영.
- 에이전트에 `model` 이 명시돼 있으면 그대로 둔다 (override 허용, backward compatible).
- `models` 블록이 없고 모두 concrete `model` 인 구형 yaml 도 그대로 동작.
"""
models = raw.pop("models", {})
if not isinstance(models, dict):
raise ValueError(f"'models' must be a mapping, got {type(models).__name__}")

for name, spec in raw.items():
if not isinstance(spec, dict):
continue # max_concurrent 등 스칼라는 스킵
if spec.get("model"):
continue # 명시적 model 우선
tier = spec.get("model_tier")
if tier is None:
raise ValueError(f"agent {name!r}: 'model' 또는 'model_tier' 중 하나가 필요합니다")
if tier not in models:
raise ValueError(
f"agent {name!r}: model_tier {tier!r} 가 models 에 없습니다 "
f"(정의된 tier: {sorted(models)})"
)
spec["model"] = models[tier]

return raw


def load_agents_config(path: str | Path) -> OrchestratorConfig:
"""Read agents.yaml and parse into OrchestratorConfig."""
path = Path(path)
Expand All @@ -99,4 +130,6 @@ def load_agents_config(path: str | Path) -> OrchestratorConfig:
if not isinstance(raw, dict):
raise ValueError(f"invalid agents.yaml format: expected dict, got {type(raw).__name__}")

raw = _resolve_model_tiers(raw)

return OrchestratorConfig(**raw)
6 changes: 3 additions & 3 deletions backend/tests/orchestrator/test_agent_matching.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ def _make_agent(
) -> AgentConfig:
return AgentConfig(
provider="claude-cli",
model="claude-sonnet-4-6",
model="claude-sonnet-5",
prompt_path="agents/test/CLAUDE.md",
requires_capabilities=capabilities or [],
requires_profile_ids=profile_ids or [],
Expand All @@ -38,7 +38,7 @@ def _make_full_agents_yaml(tmp_path: Path) -> Path:
"""agents.yaml 형식 전체 agent dict — load_agents_config 용."""
base_agent: dict = {
"provider": "claude-cli",
"model": "claude-sonnet-4-6",
"model": "claude-sonnet-5",
"prompt_path": "agents/test/CLAUDE.md",
"timeout_seconds": 300,
"on_timeout": "escalate",
Expand Down Expand Up @@ -334,7 +334,7 @@ def test_missing_requires_fields_default_to_empty(self, tmp_path: Path) -> None:
"""agents.yaml without requires_* still loads cleanly."""
base_agent: dict = {
"provider": "claude-cli",
"model": "claude-sonnet-4-6",
"model": "claude-sonnet-5",
"prompt_path": "agents/test/CLAUDE.md",
}
data = {
Expand Down
60 changes: 58 additions & 2 deletions backend/tests/orchestrator/test_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
OnTimeout,
OrchestratorConfig,
Provider,
_resolve_model_tiers,
load_agents_config,
)

Expand Down Expand Up @@ -159,8 +160,8 @@ def test_load_real_agents_yaml(self) -> None:
pytest.skip("agents.yaml 없음")

cfg = load_agents_config(real_path)
assert cfg.architect.model == "claude-opus-4-6"
assert cfg.backend_coder.model == "claude-sonnet-4-6"
assert cfg.architect.model == "claude-opus-4-8"
assert cfg.backend_coder.model == "claude-sonnet-5"
assert cfg.orchestrator.on_timeout == OnTimeout.RETRY

def test_all_prompt_paths_resolve(self) -> None:
Expand All @@ -185,3 +186,58 @@ def test_all_prompt_paths_resolve(self) -> None:
assert not missing, "prompt_path 파일이 없습니다 (.gitignore 누락 의심):\n" + "\n".join(
missing
)


class TestModelTierResolution:
"""model_tier 별칭 해석 (`_resolve_model_tiers`)."""

def test_tier_maps_to_concrete_model(self) -> None:
raw = {
"models": {"judge": "claude-opus-4-8", "code": "claude-sonnet-5"},
"architect": {"provider": "claude-cli", "model_tier": "judge", "prompt_path": "a"},
"backend_coder": {"provider": "claude-cli", "model_tier": "code", "prompt_path": "b"},
"max_concurrent": 2,
}
out = _resolve_model_tiers(raw)
assert out["architect"]["model"] == "claude-opus-4-8"
assert out["backend_coder"]["model"] == "claude-sonnet-5"
assert "models" not in out # 별칭 블록은 소비됨
assert out["max_concurrent"] == 2 # 스칼라는 그대로

def test_explicit_model_overrides_tier(self) -> None:
raw = {
"models": {"code": "claude-sonnet-5"},
"backend_coder": {
"provider": "claude-cli",
"model": "custom-model",
"model_tier": "code",
"prompt_path": "b",
},
}
out = _resolve_model_tiers(raw)
assert out["backend_coder"]["model"] == "custom-model" # 명시적 model 우선

def test_unknown_tier_raises(self) -> None:
raw = {
"models": {"judge": "claude-opus-4-8"},
"architect": {
"provider": "claude-cli",
"model_tier": "nonexistent",
"prompt_path": "a",
},
}
with pytest.raises(ValueError, match="model_tier"):
_resolve_model_tiers(raw)

def test_missing_model_and_tier_raises(self) -> None:
raw = {"models": {}, "architect": {"provider": "claude-cli", "prompt_path": "a"}}
with pytest.raises(ValueError, match="필요"):
_resolve_model_tiers(raw)

def test_legacy_concrete_model_without_models_block(self) -> None:
"""models 블록 없는 구형 yaml — concrete model 그대로 동작."""
raw = {
"architect": {"provider": "claude-cli", "model": "claude-opus-4-8", "prompt_path": "a"}
}
out = _resolve_model_tiers(raw)
assert out["architect"]["model"] == "claude-opus-4-8"
2 changes: 1 addition & 1 deletion harness/profiles/python-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ lessons_applied:
- **사용자 튜너블** (모델명, 비용 단가) → `pyproject.toml [tool.<name>]` 섹션
```toml
[tool.hijack]
default_model = "claude-sonnet-4-6"
default_model = "claude-sonnet-5"
cost_per_input_token = 3e-6
```
- **런타임 환경값** (API 키, 엔드포인트) → 환경변수 (§4 보안 규칙에 따름)
Expand Down