Skip to content

Fix broken CI test suite and close major test coverage gaps#3

Merged
projectamazonph merged 1 commit into
mainfrom
claude/test-coverage-analysis-jogslk
Jul 16, 2026
Merged

Fix broken CI test suite and close major test coverage gaps#3
projectamazonph merged 1 commit into
mainfrom
claude/test-coverage-analysis-jogslk

Conversation

@projectamazonph

Copy link
Copy Markdown
Owner

Problem

A test-coverage audit found that CI's Tests step (CI=true npx vitest run) was silently failing to load 7 of the richest test files because they imported from bun:test instead of vitest. Digging further, the 4 live-server integration suites (auth, resources, questions/interview/AI, user-paths) were unconditionally skipped under CI=true and, when actually run, turned out to authenticate via an x-user-id header that had been deliberately removed as a security fix — they were exercising an auth path that no longer exists in the app.

Solution

  • Fixed the 7 bun:testvitest imports so those suites actually load and run.
  • Rewrote the 4 integration suites to authenticate via the real JWT session-cookie flow instead of the removed x-user-id header.
  • Wired those suites into CI for real: a Postgres service container, schema push + seed, build, boot the app, run the suite against the live server, tear down (previously they only ran against a manually-started local dev server, never in CI).
  • Made login/register/general API rate limits configurable via env vars (production defaults unchanged) so the integration suite's request volume doesn't trip production-tuned limits during the CI run.
  • Fixed two pre-existing data-drift bugs the tests surfaced along the way: seed data uses 'PPC VA' not 'Amazon PPC VA', and 'PPC Fundamentals' isn't a valid skillArea enum value.
  • Added new tests for previously-uncovered surface: component render tests (MockInterview, ResumeLab, OnboardingQuiz), direct unit tests (session, rate-limit, sanitize), and route tests (subscription webhook/manage, verify-email, questions/count) — all importing the real implementation rather than reimplementing route logic in-test.
  • Added @vitest/coverage-v8 and a test:coverage script.

Scope

Included: test infrastructure, CI workflow, the two rate-limit env-var additions in login/register/middleware.ts, and new test files.
Excluded: no product/UI behavior changes. Did not refactor the existing "shadow stub" test files (tests that reimplement route logic in-file rather than importing it) — that's a larger, separate effort.

Acceptance criteria

  • Observable behavior matches the requirement — CI now actually runs the full suite it claims to run
  • Important failure paths are handled — auth/ownership/validation failure paths covered in new tests
  • AI-generated content meets guardrails policy — n/a, no AI-generation logic changed
  • Subscription tier access controls work correctly — covered by new subscription/manage tests

Validation

  • Type checking: npx tsc --noEmit — clean
  • Lint: npx eslint . — 0 errors (pre-existing warnings only, none in changed files)
  • Unit tests: CI=true npx vitest run — 331 passed, 97 skipped (integration suites, correctly gated without a live server)
  • Full CI sequence dry-run (Postgres, db push, seed, unit tests, build, start server, live-server integration tests): 331 unit + 81 integration tests passing, verified locally end-to-end before pushing

Risk and rollback

Low risk: additive test coverage plus one CI workflow change and small, backward-compatible env-var-gated rate-limit config (defaults unchanged, so production behavior is identical unless the new env vars are explicitly set). Rollback is a plain revert of this commit.

Documentation

No user-facing docs changed. docs/TESTING.md describes the test suite's intent, which this PR makes actually true in CI; a follow-up doc update noting the live-server CI job could be worthwhile but wasn't required for this fix.


Generated by Claude Code

- Fix 7 test files importing from 'bun:test' instead of 'vitest',
  which made npx vitest run (what CI actually runs) fail to load
  them entirely — CI was passing on a fraction of the suite.
- Rewrite the 4 live-server integration suites (auth, resources,
  questions/interview/AI, user-paths) to authenticate via the real
  JWT session cookie flow; they still used the x-user-id header
  shortcut that was removed as a security fix, so they were
  exercising an auth path that no longer exists.
- Wire those integration suites into CI: spin up a Postgres service,
  push + seed the schema, build, start the app, and run them against
  the live server instead of skipping unconditionally under CI=true.
- Make login/register/general API rate limits configurable via env
  vars (defaults unchanged) so the integration suite's request volume
  doesn't trip production-tuned limits.
- Add real render tests (MockInterview, ResumeLab, OnboardingQuiz),
  direct unit tests (session, rate-limit, sanitize), and route tests
  (subscription webhook/manage, verify-email, questions/count) that
  import the actual implementation instead of reimplementing it.
- Add @vitest/coverage-v8 and a test:coverage script.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CyerEYcFhnBf8oaqqSSCSg
@projectamazonph
projectamazonph merged commit 3377687 into main Jul 16, 2026
3 checks passed
@vercel

vercel Bot commented Jul 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
interview-lab Ready Ready Preview, Comment Jul 16, 2026 1:00pm

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@projectamazonph, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 5 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 522c9282-551c-42e1-9eec-3ec18a1ed9d2

📥 Commits

Reviewing files that changed from the base of the PR and between 3e07910 and 5bd4344.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (27)
  • .github/workflows/ci.yml
  • __tests__/api/assessments.test.ts
  • __tests__/api/auth-login.test.ts
  • __tests__/api/auth-register.test.ts
  • __tests__/api/auth-verify-email.test.ts
  • __tests__/api/auth.test.ts
  • __tests__/api/profile-dashboard.test.ts
  • __tests__/api/questions-count.test.ts
  • __tests__/api/questions-interview-ai.test.ts
  • __tests__/api/questions.test.ts
  • __tests__/api/resources.test.ts
  • __tests__/api/resume-coverletter.test.ts
  • __tests__/api/subscription-manage.test.ts
  • __tests__/api/subscription-webhook.test.ts
  • __tests__/api/subscription.test.ts
  • __tests__/api/user-paths.test.ts
  • __tests__/components/mock-interview.test.tsx
  • __tests__/components/onboarding-quiz.test.tsx
  • __tests__/components/resume-lab.test.tsx
  • __tests__/lib/rate-limit.test.ts
  • __tests__/lib/sanitize.test.ts
  • __tests__/lib/session.test.ts
  • package.json
  • src/app/api/auth/login/route.ts
  • src/app/api/auth/register/route.ts
  • src/middleware.ts
  • vitest.config.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/test-coverage-analysis-jogslk

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants