fix(langfuse): count daily summary stats in SQL instead of loading 24h of call trackers

[cmd-err]

Problem

langfuse_score_monitor builds the daily Slack summary at 23:30 IST. _get_daily_call_stats ran SELECT lct.* over the last 24h of lead_call_tracker with no LIMIT (~143k rows, 1.3 GB on disk, full JSON columns), decoded every row into a model, and only counted status / outcome / provider.

• The pod running it grows from ~0.6 GB to ~5.4 GB in under a minute. The node (e2-standard-2, ~6 GB usable, 4Gi request, no memory limit) runs out of memory and the kubelet evicts the pod. In-flight requests get 502s, which fires the Buddy LB 5xx alert.
• The "summary sent" Redis key is only set after success, so each 10-min run retries on another pod. That's 6 evictions per night, 23:30–00:30 IST, every night since 22 Sep.
• The last Slack summary went out on 21 Sep (that night the load already took 2m13s; it was 17–20s on 18–20 Sep).

Fix

A new get_daily_summary_stats_query returns one row with the same counts, using the same rules as the old Python loops:

• calls: FINISHED count; NO_ANSWER / CONFIRM / CANCEL / ADDRESS_UPDATED / BUSY counts; provider split (TWILIO / EXOTEL / PLIVO)
• leads, grouped per request_id: total, picked (finished > no_answer), confirmed, cancelled, address-updated

_get_daily_call_stats now reads this row. The derived percentages, the stats dict and the Slack message are unchanged, so the summary loses nothing. Memory is now flat no matter how many calls there are.

Removed dead code

get_all_lead_call_trackers and the lead_call_tracker accessor get_lead_based_analytics are deleted, along with their query builders and the app/database/accessor/__init__.py exports. The summary was their only caller (checked with a repo-wide grep; the dynamic import_module sites only load intent and UI modules). An unbounded full-row fetch by date range is exactly the pattern that caused this, so it shouldn't be left around for reuse. The analytics dashboard's own get_lead_based_analytics handler (analytics/handlers.py) is a different function and is unchanged.

Verification

• The aggregate SQL, run read-only on prod for the exact 21 Sep window, reproduces the stats logged by the old code (only calls_attempted differs, by +7, for calls that finished after the snapshot). It runs in ~0.15s vs 2m13s.
• New test tests/test_langfuse_daily_summary_stats.py pins the output to the exact dict prod logged on 21 Sep, plus the zero fallback.
• pytest tests: 3586 passed. pyrefly check: 0 errors. black / isort / autoflake clean.

Deploy note

This needs to be live before 23:30 IST tonight, or the same evictions repeat.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Analytics
  • Daily call and lead summaries now use pre-aggregated counts for calls, outcomes, providers, and leads within the reporting period.
  • Existing derived metrics and percentages continue to be calculated from these totals. When summary data is unavailable, the displayed counts use zero-value defaults.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 2c817542-c560-44e1-8e60-bb6befe9bdeb

📥 Commits

Reviewing files that changed from the base of the PR and between 7e8c8d3 and aef8f97.

📒 Files selected for processing (5)

• app/database/accessor/__init__.py
• app/database/accessor/breeze_buddy/lead_call_tracker.py
• app/database/queries/breeze_buddy/lead_call_tracker.py
• app/services/langfuse/tasks/score_monitor/score.py
• tests/test_langfuse_daily_summary_stats.py

💤 Files with no reviewable changes (1)

• app/database/accessor/init.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

Walkthrough

The score monitor now gets daily call and lead counts from a date-bounded SQL aggregate through a new database accessor. The change removes the previous call-list and per-lead analytics accessors and adds tests for aggregate mapping and empty results.

Changes

Daily Statistics Aggregation

Layer / File(s)	Summary
Daily SQL aggregation app/database/queries/breeze_buddy/lead_call_tracker.py	A date-bounded query aggregates call outcomes, provider counts, and lead metrics. The previous call-list and per-lead analytics queries are removed.
Aggregate accessor and exports app/database/accessor/breeze_buddy/lead_call_tracker.py, app/database/accessor/__init__.py	A new accessor returns the first aggregate record or None, and the removed accessors are no longer exported.
Score monitor integration and tests app/services/langfuse/tasks/score_monitor/score.py, tests/test_langfuse_daily_summary_stats.py	The score monitor reads the aggregate for a 24-hour window. Tests check mapped statistics, the requested window, and empty results.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant ScoreMonitor._get_daily_call_stats
  participant get_daily_summary_stats
  participant Database
  ScoreMonitor._get_daily_call_stats->>get_daily_summary_stats: Request aggregate for the 24-hour date range
  get_daily_summary_stats->>Database: Execute date-bounded aggregate query
  Database-->>get_daily_summary_stats: Return aggregate row or no rows
  get_daily_summary_stats-->>ScoreMonitor._get_daily_call_stats: Return first row or None

Loading

Suggested reviewers: manas-narra

Merge Risk: ⚪ Minimal · up to aef8f

The daily summary’s checked counts retain their previous meaning, with no identified issue requiring a fix before merge.

Security Architecture Review

Security architecture risk: 🔵 Low · up to aef8f

The change addresses a reported pod-memory failure without evidence of broader data access or a new externally reachable path. Its remaining risk is whether the database can execute the daily aggregation reliably at production volume.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The examined execution path can read daily counts across the existing date-bounded call-tracker population and send aggregate figures to Slack. Database contention could affect services sharing that database, but its production capacity and isolation were not established.

Trust Boundaries and Controls

• inferred — No attacker-supplied bounds or new external route were identified on the examined summary path: scheduling determines the dates, and the accessor passes them as query parameters. Database role and row-level-security controls remain unverified.

Resilience and Maintainability Implications

• inferred — Returning one aggregate row removes the full-record transfer associated with the reported pod failure, while making database execution cost the key remaining availability question. The available tests do not establish peak-volume database behavior.

Hardening Proposals

• proposed — Before relying on the change at peak volume, validate the production query plan and existing statement-timeout safeguards so the memory fix does not shift a failure into the shared database.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 78.57% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 4 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: replacing 24-hour call-tracker loading with SQL aggregation for Langfuse daily summary statistics.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{A rabbit checks the daily count,
Then hops through rows that add and mount.
Call outcomes settle in their place,
Lead totals join the measured pace.
With tests in hand, I thump hooray!}

_{Comment @coderabbitai help to get the list of available commands.}