Multi-threaded HTTP Server

This project is a multi-threaded HTTP/1.1 server built from scratch in Python using low-level socket programming. It handles multiple concurrent clients, serves static and binary files for GET requests, processes JSON data for POST requests, and includes essential security and connection management features.

Features

1. Multi-threaded Architecture: Uses a thread pool to handle multiple client connections concurrently.
2. GET Request Handling:
   • Serves HTML files for in-browser rendering.
   • Serves binary files (images, text files) as attachments for download.
3. POST Request Handling:
   • Accepts and validates application/json content.
   • Saves POSTed JSON data to a new file in resources/uploads/.
4. Connection Management: Supports persistent connections (keep-alive) with idle timeouts and request limits.
5. Security:
   • Path Traversal Protection to prevent access to files outside the resources directory.
   • Host Header Validation to reject malformed or unauthorized requests.
6. Configuration: Host, port, and thread pool size configurable via command-line arguments.

Directory Structure

http-server-project/
├── server.py
├── setup_test_files.sh
├── README.md
└── resources/
    ├── uploads/
    ├── index.html
    ├── about.html
    ├── contact.html
    ├── sample.txt
    ├── test.txt
    ├── logo.png
    ├── photo.jpg
    └── large_photo.jpg (>1MB)

Setup and Running Instructions

Prerequisites

• Python 3.x

Installation

1. Clone the repository:

   git clone <your-repository-url>
   cd http-server-project

2. Run the setup script (creates all required test files):

   chmod +x setup_test_files.sh
   ./setup_test_files.sh

   Or manually create the directory structure:

   mkdir -p resources/uploads

3. Add test files to the resources/ directory:

   • At least 3 HTML files (index.html, about.html, contact.html)
   • At least 2 PNG images
   • At least 2 JPEG images (including one >1MB for testing)
   • At least 2 text files

Running the Server

Default configuration (127.0.0.1:8080, 10 threads):

python3 server.py

Custom configuration:

python3 server.py <port> <host> <max_threads>

Examples:

# Run on port 8000, all interfaces, 20 threads
python3 server.py 8000 0.0.0.0 20

# Run on custom port with default host
python3 server.py 9000

To stop the server: Press Ctrl + C in the terminal where the server is running.

Testing the Server

Browser Testing 🌐

Once your server is running, test these URLs in your web browser:

Test Case	URL	Expected Result
Homepage	http://localhost:8080/	Displays index.html
HTML Page	http://localhost:8080/about.html	Displays about.html
Another Page	http://localhost:8080/contact.html	Displays contact.html
Image Download	http://localhost:8080/photo.jpg	Downloads photo.jpg as binary
PNG Download	http://localhost:8080/logo.png	Downloads logo.png as binary
Text Download	http://localhost:8080/sample.txt	Downloads sample.txt as binary
Large File	http://localhost:8080/large_photo.jpg	Downloads large image (>1MB)
404 Error	http://localhost:8080/nonexistent.html	Shows 404 Not Found error

Command Line Testing 🖥️

1. Test POST Request (JSON upload):

curl -X POST -H "Content-Type: application/json" \
  --data '{"user": "test", "id": 123}' \
  http://localhost:8080/upload

Expected: 201 Created with JSON response containing filepath

2. Test Invalid JSON POST:

curl -X POST -H "Content-Type: application/json" \
  --data '{invalid json}' \
  http://localhost:8080/upload

Expected: 400 Bad Request

3. Test Unsupported Content-Type:

curl -X POST -H "Content-Type: text/plain" \
  --data 'some text' \
  http://localhost:8080/upload

Expected: 415 Unsupported Media Type

4. Test Path Traversal Protection:

curl -v http://localhost:8080/../etc/passwd

Expected: 403 Forbidden

5. Test Host Header Validation:

curl -v -H "Host: evil.com" http://localhost:8080/

Expected: 403 Forbidden

6. Test Missing Host Header:

curl -v -H "Host:" http://localhost:8080/

Expected: 400 Bad Request

7. Test Unsupported Method:

curl -X PUT http://localhost:8080/index.html

Expected: 405 Method Not Allowed

8. Test Concurrent Downloads (run simultaneously in separate terminals):

# Terminal 1:
curl -o download1.jpg http://localhost:8080/photo.jpg

# Terminal 2:
curl -o download2.jpg http://localhost:8080/photo.jpg

# Terminal 3:
curl -o download3.jpg http://localhost:8080/large_photo.jpg

Expected: All files download successfully and match originals

9. Verify Binary File Integrity:

# Download a file
curl -o downloaded_photo.jpg http://localhost:8080/photo.jpg

# Compare checksums (on Linux/Mac)
md5sum resources/photo.jpg downloaded_photo.jpg
# Or on Mac:
md5 resources/photo.jpg downloaded_photo.jpg

# Both should have identical checksums

10. Test Keep-Alive Connection:

curl -v --keepalive-time 60 http://localhost:8080/index.html

Expected: Response includes Connection: keep-alive and Keep-Alive: timeout=30, max=100

Architecture

Thread Pool Implementation

The server uses a producer-consumer model with a thread-safe queue:

• Producer: The main server thread accepts incoming client connections and adds them to a shared queue.Queue.
• Consumers: A pool of worker threads (default: 10) continuously fetch client sockets from the queue and handle their requests.
• Synchronization: Python's queue.Queue provides built-in thread-safe operations, preventing race conditions. A threading.Lock is used to safely update the active thread counter.

Benefits:

• Efficient resource management with a fixed number of threads
• Prevents server overload by queuing excess connections
• Proper cleanup of resources when connections close
• Scalable to handle many concurrent clients

Flow:

1. Main thread accepts connection → adds to queue
2. Idle worker thread retrieves connection from queue
3. Worker processes all requests on that connection
4. Connection closed → worker returns to pool

Binary Transfer Implementation

Binary files (images, text files) are handled to preserve data integrity:

1. Binary Read Mode: Files are opened in binary mode ('rb') to prevent data corruption.
2. Content-Type Header: Set to application/octet-stream to indicate binary data.
3. Content-Disposition Header: attachment; filename="..." triggers browser download.
4. Chunked Reading: Files are read efficiently in chunks (suitable for large files).
5. Content-Length: Exact file size is specified in bytes for proper transfer completion.

Example Response Headers:

HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 45678
Content-Disposition: attachment; filename="photo.jpg"
Date: Fri, 10 Oct 2025 10:30:15 GMT
Server: Multi-threaded HTTP Server
Connection: keep-alive
Keep-Alive: timeout=30, max=100

HTTP Protocol Features

• HTTP/1.1 Compliance: Proper request/response formatting per RFC 7230
• Persistent Connections: Keep-alive support with 30-second idle timeout
• Request Limits: Maximum 100 requests per persistent connection
• Content Negotiation: Different handling for HTML (render) vs binary (download)
• Status Codes: Comprehensive error handling (400, 403, 404, 405, 415, 500, 503)

Connection Management

Keep-Alive Behavior:

• HTTP/1.1 connections default to keep-alive
• Client can request Connection: close to close after response
• Server enforces 30-second idle timeout
• Maximum 100 requests per connection before forced close
• Timeout prevents resource exhaustion from idle connections

Connection Flow:

1. Client connects → socket accepted
2. Request received → parsed and validated
3. Response sent → connection remains open (if keep-alive)
4. Client can send more requests on same connection
5. Timeout or max requests → connection closed

Security

1. Path Traversal Protection

Prevents attackers from accessing files outside the resources directory:

• Requested paths are normalized using os.path.abspath()
• Validated to ensure they remain within the allowed directory
• Blocks malicious patterns: .., ./, absolute paths
• Returns 403 Forbidden for unauthorized access attempts

Implementation:

def is_path_safe(self, path):
    """Check for path traversal vulnerabilities"""
    requested_path = os.path.abspath(os.path.join(self.resources_dir, path))
    return requested_path.startswith(self.resources_dir)

Blocked Examples:

• GET /../etc/passwd → 403 Forbidden
• GET /../../sensitive.txt → 403 Forbidden
• GET //etc/hosts → 403 Forbidden
• GET /./././../config → 403 Forbidden

2. Host Header Validation

Ensures requests are intended for this server:

• Every request must include a valid Host header
• Header value must match the server's configured address
• Missing headers return 400 Bad Request
• Mismatched headers return 403 Forbidden
• Special handling for 0.0.0.0 binding (accepts localhost/127.0.0.1)

Valid Examples:

• Host: localhost:8080
• Host: 127.0.0.1:8080
• Host: 192.168.1.100:8080 (if server bound to that IP)

Invalid Examples:

• Missing Host header → 400 Bad Request
• Host: evil.com → 403 Forbidden
• Host: malicious.site:8080 → 403 Forbidden

3. Content Type Validation

• POST requests must include Content-Type: application/json
• Only JSON payloads are accepted for uploads
• Invalid content types return 415 Unsupported Media Type
• JSON parsing errors return 400 Bad Request

4. Request Size Limits

• Maximum request size: 8192 bytes
• Prevents memory exhaustion from large requests
• Larger requests are truncated or rejected

Error Handling

Status Code	Meaning	When It Occurs
200	OK	Successful GET request
201	Created	Successful POST request, file created
400	Bad Request	Malformed request, missing Host header, invalid JSON
403	Forbidden	Path traversal attempt, Host mismatch
404	Not Found	Requested resource doesn't exist
405	Method Not Allowed	PUT, DELETE, or other unsupported methods
415	Unsupported Media Type	Non-JSON POST or unsupported file type
500	Internal Server Error	Unexpected server-side errors
503	Service Unavailable	Thread pool exhausted (with Retry-After header)

Logging

The server implements comprehensive logging with timestamps:

Server Startup:

[2025-10-10 10:30:00] HTTP Server started on http://127.0.0.1:8080
[2025-10-10 10:30:00] Thread pool size: 10
[2025-10-10 10:30:00] Serving files from '/path/to/resources' directory
[2025-10-10 10:30:00] Press Ctrl+C to stop the server

Request Processing:

[2025-10-10 10:30:15] [Thread-1] Connection from 127.0.0.1:54321
[2025-10-10 10:30:15] [Thread-1] Request: GET /photo.jpg HTTP/1.1
[2025-10-10 10:30:15] [Thread-1] Host validation: localhost:8080 ✓
[2025-10-10 10:30:15] [Thread-1] Sending binary file: photo.jpg (45678 bytes)
[2025-10-10 10:30:15] [Thread-1] Response: 200 OK (45678 bytes transferred)
[2025-10-10 10:30:15] [Thread-1] Connection: keep-alive

Thread Pool Status (every 30 seconds):

[2025-10-10 10:35:00] Thread pool status: 8/10 active

Security Events:

[2025-10-10 10:35:30] [Thread-3] Forbidden path access attempt: /../etc/passwd
[2025-10-10 10:35:35] [Thread-4] Host validation: evil.com:8080 ✗

Connection Queue:

[2025-10-10 10:40:00] Warning: Thread pool saturated, queuing connection
[2025-10-10 10:40:05] Connection dequeued, assigned to Thread-5

Known Limitations

• Supports only GET and POST methods (no PUT, DELETE, PATCH, etc.)
• HTTP/1.1 only (HTTP/2 and HTTP/3 not supported)
• No HTTPS/TLS encryption
• Request size limited to 8192 bytes
• Basic request parsing (may not handle all edge cases)
• No support for multipart form data
• No compression (gzip, deflate, brotli)
• No caching mechanisms (ETag, Last-Modified)
• No range requests for partial content
• Error pages are basic HTML (not customizable)
• No virtual host support
• No access control lists or authentication

Dependencies

• Python 3.x standard library only
• No external packages required

Core modules used:

• socket - Low-level networking
• threading - Thread pool implementation
• queue - Thread-safe connection queue
• json - JSON parsing for POST requests
• os - File system operations and path validation
• datetime - Timestamp generation
• email.utils - RFC 7231 date formatting

Performance Considerations

• Thread Pool Size: Default 10 threads. Adjust based on expected load:
  • Low traffic: 5-10 threads
  • Medium traffic: 10-20 threads
  • High traffic: 20-50 threads
• Connection Queue: Unbounded queue prevents connection rejection but may consume memory under extreme load
• Keep-Alive: Reduces overhead for multiple requests from same client
• Buffer Size: 8192 bytes is optimal for most use cases
• File I/O: Binary files read in single operation (suitable for files <100MB)

Troubleshooting

Problem: Address already in use error

• Solution: Port is already in use. Either wait 60 seconds or use a different port:

  python3 server.py 8081

Problem: Permission denied on port <1024

• Solution: Ports below 1024 require root/admin privileges. Use port ≥1024 or run with sudo:

  sudo python3 server.py 80

Problem: Cannot access server from other machines

• Solution: Bind to 0.0.0.0 instead of 127.0.0.1:

  python3 server.py 8080 0.0.0.0

Problem: Files not downloading correctly

• Solution: Ensure files are in resources/ directory and paths are correct

Problem: 403 Forbidden errors

• Solution: Check Host header matches server address, verify path doesn't contain ..

Future Improvements

• HTTPS/TLS support
• HTTP/2 protocol support
• Request body streaming for large uploads
• Gzip compression for responses
• ETag and caching support
• Range requests for video streaming
• WebSocket support
• Rate limiting per IP
• Authentication and authorization
• Custom error pages
• Configuration file support
• Request/response logging to file
• Metrics and monitoring endpoints