🛡️ Web Log Anomaly Detection
Machine Learning + Cybersecurity • CSIC 2010 Web Traffic Logs
This project implements a complete end-to-end Web Log Anomaly Detection System for detecting malicious HTTP traffic using RandomForest, IsolationForest, and a modern Streamlit dashboard, with a Flask API backend.
It is designed as an academic yet production-ready pipeline for cybersecurity log analysis.
🚀 Features
✅ Data Pipeline
-Parsing CSIC 2010 raw HTTP traffic
-Normalization & cleaning
-Feature engineering (URL entropy, length, user-agent features, HTTP method, etc.)
-CSV export for training
✅ Machine Learning Models
-RandomForest Classifier (supervised)
-IsolationForest (unsupervised)
-Saved models in .joblib
✅ Evaluation & Visualizations
-Confusion matrix
-Classification report
-Feature importance plot
-Anomaly score distribution
-Heatmap RF vs ISO
-✅ Flask API
Supports:
-/predict-csv → upload CSV logs
-/predict-log → upload raw Apache access logs
✅ Streamlit Dashboard
-Dark premium theme
-Sidebar with logo
-Upload CSV or Apache logs
-RF/ISO prediction distributions
-Real-time heatmap
-Anomaly explorer table
-"About" page with project information
This project uses the CSIC 2010 HTTP dataset, containing:
-Normal HTTP requests
-Malicious / abnormal requests
-Anomalies such as SQL Injection, XSS, buffer overflow patterns
Source:
-https://www.isi.csic.es/dataset/
🧠 Machine Learning Approach
🔹 Supervised: RandomForest
Used to classify each HTTP request as:
-0 = Normal
-1 = Anomalous
🔹 Unsupervised: IsolationForest
Used for anomaly score estimation:
Detects outliers even without labels.
⚙️ Installation:
1️⃣ Clone the repo
git clone https://github.com/ImaneBARAKAT/web_log_anomaly_detection.git
cd web_log_anomaly_detection
2️⃣ Install dependencies
pip install -r requirements.txt
3️⃣ Run Flask API
cd api
python app.py
4️⃣ Run Streamlit Dashboard
cd app
streamlit run streamlit_app.py
📜 License
MIT License – free to use, modify and distribute.