You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adds the weekly cost/model watchdog: a Monday-morning schedule that pulls the AI Gateway spend by surface and model, compares it to the previous week, checks the current model landscape, and proposes per-surface model adjustments (or states that nothing needs to change).
apps/evi/agent/schedules/cost-watchdog.ts: weekly cron (Monday 08:00 UTC), clear of the digest (05:00), upstream-sync (Mon/Thu 07:00) and self-review (Wed 08:00). Monday keeps the report aligned to complete calendar weeks.
apps/evi/agent/skills/cost-watchdog/SKILL.md: the procedure, grounded in the actual report shape (group_by: tag returns the evi:env:* total plus one evi:surface:* row per surface). It pulls the last 7 full days and the 7 days before for drift, researches the model landscape on the web with sources, and delivers the full report as a Linear document with the top finding in the thread. Material recommendations become Linear issues; a quiet week is a one-line "nothing to improve".
apps/evi/docs/notes.md: records the report shape so the next reader does not rediscover it.
Checks, run in /workspace/repo on the branch:
lint: clean (the schedule file; markdown is not linted)
typecheck: clean after pnpm run dev:prepare rebuilds packages/evlog/dist. The sandbox dist was stale (predated the distinctIdField addition in the posthog adapter), which initially surfaced as a typecheck error in agent/hooks/evlog.ts on clean main; rebuilding the workspace package clears it. CI runs the same build step.
test: 91 passed (15 files)
No changeset: the change is confined to apps/evi, which never needs one.
Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.
This PR includes no changesets
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Addressed the review comment on cost-watchdog/SKILL.md: added a Resources section with real links so a run goes straight to the sources instead of searching, and pointed the model-landscape step at them.
https://ai-gateway.vercel.sh/v1/models — the AI Gateway model catalog as JSON, pricing and capabilities per model in one fetch (verified live, no auth needed)
https://vercel.com/ai-gateway/models — the human-readable models browser
https://vercel.com/docs/ai-gateway/models-and-providers — the docs
https://arena.ai/leaderboard — ex-LMArena, blind A/B human-preference Elo
https://artificialanalysis.ai/ — independent Intelligence Index, Cost per Task and speed
web_search remains only as a fallback for what the links do not cover. No code changed; lint/typecheck/test unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes EVL-248.
Adds the weekly cost/model watchdog: a Monday-morning schedule that pulls the AI Gateway spend by surface and model, compares it to the previous week, checks the current model landscape, and proposes per-surface model adjustments (or states that nothing needs to change).
apps/evi/agent/schedules/cost-watchdog.ts: weekly cron (Monday 08:00 UTC), clear of the digest (05:00), upstream-sync (Mon/Thu 07:00) and self-review (Wed 08:00). Monday keeps the report aligned to complete calendar weeks.apps/evi/agent/skills/cost-watchdog/SKILL.md: the procedure, grounded in the actual report shape (group_by: tagreturns theevi:env:*total plus oneevi:surface:*row per surface). It pulls the last 7 full days and the 7 days before for drift, researches the model landscape on the web with sources, and delivers the full report as a Linear document with the top finding in the thread. Material recommendations become Linear issues; a quiet week is a one-line "nothing to improve".apps/evi/docs/notes.md: records the report shape so the next reader does not rediscover it.Checks, run in
/workspace/repoon the branch:pnpm run dev:preparerebuildspackages/evlog/dist. The sandbox dist was stale (predated thedistinctIdFieldaddition in the posthog adapter), which initially surfaced as a typecheck error inagent/hooks/evlog.tson cleanmain; rebuilding the workspace package clears it. CI runs the same build step.No changeset: the change is confined to
apps/evi, which never needs one.