Skip to content

Complete Real-Time Metrics Dashboard with Throughput Tracking#23

Merged
AndrGab merged 8 commits into
AndrGab:mainfrom
WalkingDevFlag:feature/throughput-metrics
Oct 11, 2025
Merged

Complete Real-Time Metrics Dashboard with Throughput Tracking#23
AndrGab merged 8 commits into
AndrGab:mainfrom
WalkingDevFlag:feature/throughput-metrics

Conversation

@WalkingDevFlag
Copy link
Copy Markdown
Contributor

  • Added a live dashboard tracking key system and CDC metrics with WebSocket updates every 2 seconds.
  • Metrics include connected users, messages per minute, real Kafka topics count, CDC event throughput, and data bytes per second.
  • Charts visualize operation distribution (CREATE/UPDATE/DELETE) and real-time CDC activity.
  • Backend captures message timestamps, CDC event types, and throughput with smart rolling windows.
  • Responsive design with full-width layout for all screen sizes.
  • Kafka topics count fetched dynamically via AIOKafkaAdminClient with error handling.
  • Added CDC throughput metrics (events/sec, bytes/sec) with automatic byte formatting.
  • Tested with real database operations and multiple responsive breakpoints.

Files modified:
connection_manager.py, websockets.py, script.js, styles.css, index.html

…osing brace in slideInLeft animation) - Fixed JavaScript duplicate createMessageElement function - Improved message display consistency - Resolved modal and layout rendering issues
Features:
- Real-time metrics dashboard with WebSocket updates
- Tab navigation between Messages and Dashboard
- System metrics: Connected users, CDC events, messages/min, Kafka topics
- Operation distribution chart (percentage-based)
- Real-time CDC activity tracking
- 24h operation history

Fixes:
- Fixed header sizing consistency across different viewports
- Added nav-tabs-container wrapper for proper alignment
- Reverted to percentage-based fill for operation bars
- Fixed sidebar layout in messages tab
- Improved responsive design for tabs
- Show user count instead of names to prevent overflow
Improvements:
- Added 4 breakpoints: 1024px, 768px, 480px, 360px
- Large tablets (1024px): Adjusted layouts and reduced padding
- Tablets (768px): Stack elements vertically, hide tab text, show icons
- Mobile (480px): Compact spacing, hide unnecessary elements, single column grids
- Small mobile (360px): Ultra-compact mode for smallest devices

Layout fixes:
- Fixed charts container to use min(100%, 400px) for better mobile support
- Fixed stats grid to use min(100%, 280px) for responsive cards
- Added proper overflow handling for long text
- Ensured all grids collapse to single column on mobile

Typography scaling:
- Progressive font size reduction across breakpoints
- Headers scale from 1.5rem -> 1.25rem -> 1rem -> 0.875rem
- Body text scales appropriately for readability

Dashboard responsive:
- Metrics cards stack on mobile
- Charts take full width on tablets and mobile
- Operation bars maintain visibility with reduced height
- System info grid collapses to single column

Added utility classes:
- Text truncation and word wrap utilities
- Scrollable containers with touch support
- Image responsiveness
- Smooth scrolling
- Prevent horizontal overflow

All UI elements now resize well across:
- Desktop (1400px+)
- Laptop (1024px - 1400px)
- Tablet (768px - 1024px)
- Mobile (480px - 768px)
- Small mobile (360px - 480px)
- Extra small (< 360px)
Changes:
- Removed max-width: 1400px from .app-container
- Removed max-width: 1200px from .header-content
- Removed max-width: 1200px from .nav-tabs-container
- Removed max-width: 1200px from .footer-content
- Removed max-width: 800px from .message-form
- Changed all to width: 100% for full viewport utilization
- Removed box-shadow from app-container (no longer needed without margin)
- Removed margin: 0 auto centering from all main containers
- Updated responsive breakpoints to remove redundant max-width overrides

The app now fills the entire browser width, eliminating side margins
and making better use of available screen real estate on all devices.
Problem:
- Kafka Topics metric was hardcoded to 128 in the dashboard
- No actual connection to Kafka to fetch real topic count
- Misleading data displayed to users

Solution:
- Added AIOKafkaAdminClient integration to ConnectionManager
- Implemented fetch_kafka_topics_count() async method
- Fetches actual topic count from Kafka cluster
- Filters out internal topics (starting with __)
- Caches topic count to avoid repeated failures
- Added error handling for Kafka connection issues

Backend changes:
- Updated ConnectionManager to track kafka_topics_count
- Added kafka_bootstrap_servers configuration
- Integrated Kafka admin client calls in metrics endpoint
- Returns actual topic count in metrics API response

Frontend changes:
- Updated JavaScript to display metrics.kafka_topics
- Changed initial HTML value from '128' to '-' (loading state)
- Shows actual count once Kafka data is fetched

Features:
- Real-time topic count updates every 2 seconds
- Graceful fallback if Kafka is unavailable
- Filters internal Kafka topics for accurate user-facing count
- Uses existing aiokafka dependency (no new packages)

Testing:
- Verified with actual Kafka cluster (shows 4 topics)
- Tested error handling when Kafka is unavailable
- Dashboard now displays real data instead of hardcoded value
@WalkingDevFlag
Copy link
Copy Markdown
Contributor Author

@AndrGab pls review

Copy link
Copy Markdown
Owner

@AndrGab AndrGab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are doing a great job ❤️
thx

@AndrGab AndrGab merged commit 8537f5e into AndrGab:main Oct 11, 2025
3 checks passed
@WalkingDevFlag WalkingDevFlag deleted the feature/throughput-metrics branch October 11, 2025 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants