Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ function transformSummary(raw: Record<string, unknown>): ModelUsageSummary {
agents,
top_models_by_usage,
top_models_by_cost,
cost_by_phase: (raw.cost_by_phase ?? {}) as Record<string, number>,
};
}

Expand Down
61 changes: 61 additions & 0 deletions apps/frontend/src/renderer/__tests__/phase-costs.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
/**
* Tests for orderPhaseCosts (P5.T3): ordering the model-usage
* "Cost by phase" breakdown for display.
*/

import { describe, expect, it } from 'vitest';
import { orderPhaseCosts } from '../lib/phase-costs';

describe('orderPhaseCosts', () => {
it('returns [] for empty, null, or undefined input', () => {
expect(orderPhaseCosts({})).toEqual([]);
expect(orderPhaseCosts(null)).toEqual([]);
expect(orderPhaseCosts(undefined)).toEqual([]);
});

it('orders known phases by pipeline order regardless of input order', () => {
const result = orderPhaseCosts({
validation: 0.5,
planning: 1,
coding: 2,
});
expect(result.map((p) => p.phase)).toEqual([
'planning',
'coding',
'validation',
]);
});

it('drops zero and negative costs', () => {
const result = orderPhaseCosts({
planning: 0,
coding: 1.25,
validation: -3,
});
expect(result).toEqual([{ phase: 'coding', cost: 1.25 }]);
});

it('appends unknown phases after known ones, sorted alphabetically', () => {
const result = orderPhaseCosts({
zeta: 1,
coding: 2,
unknown: 3,
planning: 4,
});
expect(result.map((p) => p.phase)).toEqual([
'planning',
'coding',
'unknown',
'zeta',
]);
});

it('ignores non-finite costs', () => {
const result = orderPhaseCosts({
planning: Number.NaN,
coding: Number.POSITIVE_INFINITY,
validation: 0.99,
});
expect(result).toEqual([{ phase: 'validation', cost: 0.99 }]);
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { ScrollArea } from '../ui/scroll-area';
import { useToast } from '../../hooks/use-toast';
import { ModelUsageCard } from './ModelUsageCard';
import { CostChart } from './CostChart';
import { orderPhaseCosts } from '../../lib/phase-costs';
import type {
ModelUsageSummary,
ModelUsageTrendPoint,
Expand Down Expand Up @@ -333,6 +334,37 @@ export function ModelUsageDashboard({ projectId }: ModelUsageDashboardProps) {
</div>
)}

{/* Cost by Phase */}
{summary &&
(() => {
const phaseCosts = orderPhaseCosts(summary.cost_by_phase);
if (phaseCosts.length === 0) return null;
return (
<div className="rounded-lg border border-border bg-card p-6">
<h2 className="text-lg font-semibold mb-4">
{t('model-usage:dashboard.costByPhase')}
</h2>
<div className="space-y-2">
{phaseCosts.map(({ phase, cost }) => (
<div
key={phase}
className="flex items-center justify-between text-sm"
>
<span className="text-muted-foreground">
{t(`model-usage:dashboard.phases.${phase}`, {
defaultValue: phase,
})}
</span>
<span className="font-medium text-green-600">
${cost.toFixed(2)}
</span>
</div>
))}
</div>
</div>
);
})()}
Comment thread
coderabbitai[bot] marked this conversation as resolved.

{/* Top Models by Usage */}
{summary && <ModelListSection title={t('model-usage:dashboard.topByUsage')} models={summary.top_models_by_usage} limit={5} />}

Expand Down
45 changes: 45 additions & 0 deletions apps/frontend/src/renderer/lib/phase-costs.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
/**
* Ordering for the model-usage "Cost by phase" breakdown.
*
* The backend keys `cost_by_phase` by execution phase (see
* apps/backend/core/token_stats.py / save_token_stats callers). We render the
* phases in pipeline order, with any unrecognized keys ('unknown', future
* phases) appended alphabetically so nothing is silently dropped.
*/

export interface PhaseCost {
phase: string;
cost: number;
}

/** Canonical pipeline order of known phases. */
export const PHASE_ORDER: readonly string[] = [
'planning',
'coding',
'test_generation',
'validation',
'performance_profiling',
];

/**
* Turn a `cost_by_phase` map into an ordered list, dropping non-positive
* entries. Known phases come first in pipeline order; unknown keys follow,
* sorted alphabetically.
*/
export function orderPhaseCosts(
costByPhase: Record<string, number> | undefined | null,
): PhaseCost[] {
if (!costByPhase) return [];
const entries = Object.entries(costByPhase).filter(
([, cost]) => Number.isFinite(cost) && cost > 0,
);
entries.sort(([a], [b]) => {
const ia = PHASE_ORDER.indexOf(a);
const ib = PHASE_ORDER.indexOf(b);
if (ia !== -1 && ib !== -1) return ia - ib;
if (ia !== -1) return -1;
if (ib !== -1) return 1;
return a.localeCompare(b);
});
return entries.map(([phase, cost]) => ({ phase, cost }));
}
9 changes: 9 additions & 0 deletions apps/frontend/src/shared/i18n/locales/en/model-usage.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,15 @@
"totalCost": "Total Cost",
"modelsUsed": "Models Used",
"usageByAgent": "Model Usage by Agent",
"costByPhase": "Cost by Phase",
"phases": {
"planning": "Planning",
"coding": "Coding",
"test_generation": "Test generation",
"validation": "QA",
"performance_profiling": "Profiling",
"unknown": "Other"
},
"topByUsage": "Top Models by Usage",
"topByCost": "Top Models by Cost",
"allModels": "All Models",
Expand Down
9 changes: 9 additions & 0 deletions apps/frontend/src/shared/i18n/locales/fr/model-usage.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,15 @@
"totalCost": "Coût total",
"modelsUsed": "Modèles utilisés",
"usageByAgent": "Utilisation par agent",
"costByPhase": "Coût par phase",
"phases": {
"planning": "Planification",
"coding": "Codage",
"test_generation": "Génération de tests",
"validation": "QA",
"performance_profiling": "Profilage",
"unknown": "Autre"
},
"topByUsage": "Top modèles par utilisation",
"topByCost": "Top modèles par coût",
"allModels": "Tous les modèles",
Expand Down
5 changes: 5 additions & 0 deletions apps/frontend/src/shared/types/model-usage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,11 @@ export interface ModelUsageSummary {
// Top models (computed in IPC handler from models array)
top_models_by_usage: ModelMetrics[];
top_models_by_cost: ModelMetrics[];

// USD cost keyed by execution phase (planning/coding/validation/…),
// passed through from the backend summary. Optional so summaries cached
// before this field existed still validate.
cost_by_phase?: Record<string, number>;
}

// ============================================
Expand Down
3 changes: 2 additions & 1 deletion docs/strategy/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
|---|---|---|---|
| T1 | Надёжный захват usage + запись модели/провайдера в статистику | `agents/session.py`, `core/token_stats.py` (поля `model`, `provider`) | `token_stats.json` содержит токены + модель + провайдер по фазам |
| T2 | Подключить агрегацию: per-spec и проектный `.auto-claude/model_usage_summary.json` | `analysis/model_usage_analytics.py`, вызов после `save_token_stats()` | Считается стоимость по ролям и фазам |
| T3 | Дашборд: посчитать $ через `model-costs.ts`, показать (переиспользуя графики) + строку «Cost» в `ContextViewer.tsx` | `apps/frontend/src/main/ipc-handlers/token-stats-handler.ts` + компоненты | «Сборка стоила $X — планирование/код/QA …» |
| T3 | Дашборд: разрез стоимости по фазам в живом Model Usage дашборде | `model-usage-handlers.ts` (проброс `cost_by_phase`), `phase-costs.ts`, `ModelUsageDashboard.tsx` | «Cost by Phase»: planning/coding/QA/… с $-суммами из `model_usage_summary.json`. NB: `ContextViewer.tsx` из плана — мёртвый (нигде не монтируется), поэтому карточка добавлена в реально видимый Model Usage дашборд |
| T4 | ✅ Бюджет-гард: лимит на спеку, останов/предупреждение при превышении | `core/budget_guard.py`; проверка в `qa/loop.py` и цикле coder-сессий | `AUTO_CODE_COST_LIMIT` (USD): низкий лимит останавливает прогон с понятным сообщением, на 80% — предупреждение |
| T5 *(позже)* | Роутинг дешёвой модели на тривиальные подзадачи | `phase_config.py` (`get_model_for_agent`/`lock_agent_model`) | Простые подзадачи идут на haiku, записано в статистике |

Expand Down Expand Up @@ -146,6 +146,7 @@
- ✅ **P3·T4** — тумблер автономности в настройках → инжектит `AUTO_CODE_AUTONOMY` в окружение сборки (явный env приоритетнее).
- ✅ **P5·T2** — агрегация стоимости подключена: `save_token_stats()` пишет запись в `cost_report.json` спеки (роль+фаза+$) и обновляет проектный `.auto-claude/model_usage_summary.json` (`cost_by_phase` добавлен; попутно починены сериализация summary и `+00:00Z`-таймстампы записей).
- ✅ **P3·T2** — гейт планнера: `AUTO_CODE_AUTONOMY=off` блокирует не-Claude planner (`ProviderError` до создания сессии, после runner-роутинга); Claude-путь не затронут.
- ✅ **P5·T3** — разрез стоимости по фазам в живом Model Usage дашборде: `getModelUsageSummary` теперь пробрасывает `cost_by_phase` из бэкенда (`transformSummary`), новый чистый хелпер `orderPhaseCosts` упорядочивает фазы, карточка «Cost by Phase» показывает planning/coding/QA/… с $-суммами. ContextViewer из плана оказался мёртвым кодом — вынес в реально видимый дашборд.
- ✅ **P5·T4** — бюджет-гард `AUTO_CODE_COST_LIMIT` (USD): `core/budget_guard.py` сверяет накопленную стоимость спеки с лимитом на входе каждой итерации coder- и QA-циклов; превышение останавливает прогон с понятным сообщением, 80%+ — предупреждение. Opt-in и fail-open.

Отчёт доверия теперь несёт: **вердикт · тесты · дифф · out-of-scope · confidence · uncertainty** — и виден в UI.
Expand Down
Loading