Add unified requirements.txt and setup instructions for Python dependencies

[Surge77]

📋 Unified Requirements Management & Setup Instructions

---

🎯 Overview

The repository currently lacks a single, authoritative dependency management and setup flow. That causes onboarding friction and inconsistent runtime behaviour across contributors and users. This document centralizes dependency definitions, setup instructions, dev tooling, and CI checks so the project is reproducible and easy to contribute to.

---

🔍 Current State

• Individual scripts in python-scripts/ (and other directories) use third-party libraries without documented requirements.
• Only one requirements.txt exists at sql-injection-lab/requirements.txt (contains Flask and pytest).
• No repository root-level requirements.txt or requirements-dev.txt.
• No setup instructions for Python environment configuration or platform notes.
• Dependencies are not version-pinned, increasing the risk of breakage.

---

📚 Dependencies Identified

Core Dependencies (used across scripts)

• scapy — Network packet manipulation
• requests — HTTP requests
• beautifulsoup4 — HTML parsing
• cryptography — SSL/TLS analysis
• geoip2 — IP geolocation
• urllib3 — HTTP client utilities
• paramiko — SSH connections (implied)

Optional / Development Dependencies

• pytest>=7.0 — Testing framework
• Flask>=2.0 — Web framework (sql injection lab)

Note: exact minimum versions should be derived from static analysis & running the test matrix (Phase 1).

---

🎯 Proposed Solution (High-level)

1. Add a repository root requirements.txt with pinned (minimum) versions for core runtime dependencies.
2. Add requirements-dev.txt for developer tools and testing libraries.
3. Add comprehensive setup docs: README.md additions + SETUP.md with platform-specific notes.
4. Optionally add pyproject.toml or setup.py for packaging/distribution, and a Dockerfile for reproducible environments.
5. Add CI checks to validate dependency installation and run smoke tests.

---

📋 Tasks & Phases

Phase 1: Core Requirements

1. Analyze every Python file to enumerate import statements and map them to PyPI package names.
2. Determine minimum versions by running scripts on a clean environment and noting any version-specific features/errors.
3. Create requirements.txt at repository root with pinned minimum versions (use package>=min_version,<next_major where appropriate).
4. Test the requirements.txt by creating a fresh virtual environment and installing.
5. Update sql-injection-lab/requirements.txt to reference or include the root file (see examples).

Deliverables: requirements.txt (root), verification notes, updated sql-injection-lab/requirements.txt.

---

Phase 2: Documentation & Setup

1. README.md / SETUP.md: Add step-by-step environment setup (venv, pip, Windows/Linux/macOS specifics).
2. Platform Notes: Npcap installation (Windows), sudo/capabilities for Scapy (Linux), macOS permissions.
3. Troubleshooting: Common permission errors, binary dependency issues, SSL issues, pip wheels vs. source builds.
4. Activation scripts: Provide shell and PowerShell snippets for activating venvs.

Deliverables: SETUP.md, patches to README.md.

---

Phase 3: Development Environment

1. Create requirements-dev.txt — contains pytest, linters, pre-commit hooks, black, isort, mypy (optional)
2. Pre-commit: Add .pre-commit-config.yaml and example hooks.
3. CI Workflow: Add GitHub Actions workflow to validate installs and run unit tests on PRs.
4. Virtualenv helper scripts: scripts/activate_venv.sh and scripts/activate_venv.ps1.

Deliverables: requirements-dev.txt, .pre-commit-config.yaml, .github/workflows/ci.yml, helper scripts.

---

Phase 4: Advanced (Optional)

• pyproject.toml with build-system metadata and dependency groups.
• Dockerfile and docker-compose.yml for development.
• One-command install script: scripts/bootstrap.sh / scripts/bootstrap.ps1.

---

🎁 Benefits

Contributors

• Consistent environments and faster onboarding.

Users

• Clear, tested setup steps and platform guidance.

Maintainers

• Easier dependency tracking, versioning, and security scanning.

---

🔧 Technical Considerations

• Python support: Target Python 3.8+ unless older syntax mandates otherwise.
• Pinning policy: Use minimum pinned versions with a compatibility upper bound (e.g., requests>=2.28.0,<3.0.0).
• Platform-specific notes: Document packet-capture privileges (Linux capabilities or running as root) for scapy, and Npcap installer for Windows.
• Security: Prefer well-maintained packages; add pip-audit or safety in requirements-dev.txt.

---

⚙️ Example Files & Snippets

Save these as examples in the repo (e.g., examples/requirements_samples/).

Example: requirements.txt (root)

# Core runtime dependencies (minimum pinned versions)
scapy>=2.4.5,<3.0
requests>=2.28.0,<3.0
beautifulsoup4>=4.11.1,<5.0
cryptography>=39.0.0,<40.0
geoip2>=4.6.0,<5.0
urllib3>=1.26.14,<2.0
paramiko>=2.11.0,<3.0

Example: requirements-dev.txt

-r requirements.txt
pytest>=7.0
black
isort
mypy
pre-commit
pip-audit

Example: sql-injection-lab/requirements.txt (wrapper)

# include core requirements from root
-r ../requirements.txt
# lab-specific additions
Flask>=2.0
pytest>=7.0

---

🧭 README / SETUP.md — Key Sections to Add

1. Quickstart (30 seconds): create venv, install deps, run a sample script.
2. Full Setup (detailed): platform-specific steps and troubleshooting.
3. Running tests and lints.
4. Contributing guide with dev workflow and commit hooks.
5. How to add new dependencies (policy for pinning and PR process).

---

🛠️ Example SETUP.md Outline (detailed)

1. Prerequisites

   • Python 3.8+ installed
   • pip up-to-date: python -m pip install --upgrade pip

2. Linux / macOS

   python -m venv .venv
   source .venv/bin/activate
   pip install -r requirements-dev.txt

   • For scapy raw sockets: either run scripts as root or set capabilities (Linux):

     sudo setcap cap_net_raw,cap_net_admin=eip $(which python3)

3. Windows (PowerShell)

   python -m venv .venv
   .\.venv\Scripts\Activate.ps1
   pip install -r requirements-dev.txt

   • Install Npcap (WinPcap-compatible) for packet capture.

4. Virtual Environment Activation Shortcuts

   • scripts/activate_venv.sh and scripts/activate_venv.ps1 provided.

5. Troubleshooting

   • Permission errors with Scapy: use sudo or capabilities.
   • Binary wheel build failures: install system-level build tools (build-essential / python3-dev).
   • pip cache issues: pip cache purge then reinstall.

---

[Surge77]

If you find this valuable please assign this issue to me thnks!

[SohamProg]

Please assign this issue to me. Thanks!

[amandewatnitrr]

@SohamProg & @Surge77 assigned to both. PR is based on quality, and first come, first serve basis.