An end-to-end Machine Learning pipeline designed to monitor real-time data streams and flag statistical irregularities. This system integrates a predictive model with a REST API and a simulation dashboard to visualize anomaly detection as it happens.
The project is divided into four core components:
- Model Trainer (
train_model.py): Script to preprocess data and train the detection algorithm. - Inference Engine (
anomaly_api.py): A Flask/FastAPI backend that serves the trained model via REST endpoints. - Real-time Simulator (
realtime_simulator.py): A script that generates synthetic data streams to test the model's limits. - Web Dashboard: Located in
templatesandstatics, providing a visual representation of the data flow and anomaly spikes.
The system evaluates incoming values and assigns a classification based on the learned decision boundary.
| Timestamp | Data Value | Status | Label |
|---|---|---|---|
| 21:55:06 | -0.7259 | β Normal | model=0 |
| 21:55:04 | 0.0103 | β Normal | model=0 |
| 21:55:03 | -5.3221 | π¨ Anomaly | model=1 |
| 21:55:02 | 5.4508 | π¨ Anomaly | model=1 |
Clone the repository and initialize a virtual environment:
git clone https://github.com/Vbhhacl/advanced-anomaly-detection-system.git
cd advanced-anomaly-detection-system
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
First, ensure the model is trained, then start the API and Simulator:
python train_model.py
python anomaly_api.py
python realtime_simulator.py
βββ statics/ # CSS and JS for the dashboard
βββ templates/ # HTML templates
βββ anomaly_api.py # API for model serving
βββ anomaly_model.pkl # Serialized ML model
βββ realtime_simulator.py # Data stream generator
βββ requirements.txt # Project dependencies
βββ train_model.py # Model training logic
βββ .gitignore # Version control exclusions
- Language: Python 3.x
- ML Libraries: Scikit-Learn, NumPy, Pandas
- API Framework: Flask / FastAPI
- Frontend: HTML5, CSS3, JavaScript
Author: Vbhhacl