Bug: Missing caching on career pathway queries causes slow response times and database overload

[anshul23102]

Problem Statement

DevPath does not implement caching for career pathway data, causing redundant database queries on every user request. This leads to slow response times, high database load, and poor user experience especially for popular career paths requested frequently.

Root Cause Analysis

Career data queries hit database every request without caching layer. No Redis or in-memory cache. No cache invalidation strategy. Database queries not optimized with indexes. No pagination on large result sets.

Solution Overview

1. Implement Redis caching for career pathway data
2. Set appropriate TTL (time-to-live) based on update frequency (3600s for hourly updates)
3. Implement cache invalidation on data updates
4. Add database query optimization with indexes
5. Implement pagination for large datasets
6. Add cache warming on application startup
7. Monitor cache hit/miss rates
8. Use compression for large cached objects

Type of Change

• Bug fix (performance)
• New feature
• Breaking change
• Documentation update

Testing Done

Environment

• Node.js 18.x+
• Redis
• Database with indexes

Manual Testing Steps

Test Case 1: Response time without cache

1. Request career pathway data
2. Measure response time

Expected: Response time < 100ms with cache
Actual: Response time 1000ms+ hitting database every time

Related Issue

Improves application performance and reduces database load.

Suggested Labels

bug, level:intermediate, gssoc26, performance, caching, database-optimization

[HimanshuGaur14]

Hey @komalharshita , this is my proposed solution,

• Add a Redis middleware in your API route handler (e.g., routes/careers.js) to check cache before querying, and update db/queries.js to add indexes and pagination parameters.
• On data mutations in controllers/careerController.js, call redis.del(cacheKey) to invalidate stale cache, and add a scripts/warmCache.js startup script that pre-loads popular career paths into Redis on app boot.

Can you assign this to me Under gssoc26, thank you for your time.

[nensii21]

/assign