Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
agent-os/
.claude/
.serena/
logs/

# Custom
config/config.php
Expand Down
35 changes: 35 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,41 @@ All notable changes to BigDump are documented in this file.

> **Note**: BigDump was originally created by Alexey Ozerov in 2003. Version 2.x is a complete MVC refactoring by w3spi5 (2025).

## [2.24] - 2025-01-05 - Smart Table Reset & Bug Fixes

### Added in 2.24

- **Smart Table Reset**: Automatic DROP of tables before import
- Pre-scans SQL file for CREATE TABLE statements (supports .sql, .gz, .bz2)
- Automatically drops existing tables before import to prevent "Table already exists" errors
- New methods in `FileAnalysisService`: `findCreateTables()`, `dropTablesForFile()`
- Security: Table name validation with regex, backtick quoting
- Graceful failure: import continues even if Smart Reset fails

- **Smart Table Reset Tests**: New test file `tests/SmartTableResetTest.php`
- 7 tests covering table extraction, gzip support, deduplication, security validation

### Fixed in 2.24

- **Session cleanup after SSE error**: Session now properly cleared after import errors
- Previously, failed imports left stale session data causing silent failures on retry
- Added `clearSessionDirect()` call after error event in `sseImport()`

- **Navigation links**: All "Back to Home" links now use `$scriptUri`
- Fixed `href="../"` in `error.php` (line 88)
- Fixed `href="/"` in `import.php` (line 345) - was going to server root!
- Fixed `href="./"` in `layout.php` and `layout_phar.php` (header logo)
- All navigation now works correctly regardless of installation path

### Changed in 2.24

- **Header styling**: Theme-aware pastel gradients for light and dark modes
- Light mode: soft yellow/orange gradient (`#fcd34d → #fdba74 → #fbbf24`)
- Dark mode: muted amber/orange gradient (`#d97706 → #ea580c → #c2410c`)
- CSS class-based theming (`.header-gradient`, `.header-title`, `.header-subtitle`)

---

## [2.23] - 2025-12-31 - Single-File PHAR Distribution

### Added in 2.23
Expand Down
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# BigDump 2.23 - Staggered MySQL Dump Importer
# BigDump 2.24 - Staggered MySQL Dump Importer

[![PHP Version](https://img.shields.io/badge/php-8.1+-yellow.svg)](https://php.net/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Package Version](https://img.shields.io/badge/version-2.23-blue.svg)](https://php.net/)
[![Package Version](https://img.shields.io/badge/version-2.24-blue.svg)](https://php.net/)
[![Build Assets](https://img.shields.io/badge/build-GitHub_Actions-2088FF.svg)](https://github.com/w3spi5/bigdump/actions)
[![PHAR](https://img.shields.io/badge/PHAR-single--file-purple.svg)](https://github.com/w3spi5/bigdump/releases)

<p align="center">
<img src="docs/logo.png" alt="BigDump Logo" width="400">
<img src="assets/img/logo.png" alt="BigDump Logo" width="400">
</p>

BigDump is a PHP tool for importing large MySQL dumps on web servers with strict execution time limits. Originally created by Alexey Ozerov in 2003, this major version 2 is a complete refactoring using object-oriented MVC architecture.
Expand Down Expand Up @@ -153,7 +153,7 @@ This can provide **10-50x speedup** when importing the optimized file.
- `.sql.gz` - Gzip compressed SQL
- `.sql.bz2` - Bzip2 compressed SQL

## PHAR Distribution (v2.23) — Easiest Install
## PHAR Distribution (v2.24) — Easiest Install

Download `bigdump.phar` from [Releases](https://github.com/w3spi5/bigdump/releases) and upload to your server. That's it!

Expand Down Expand Up @@ -350,8 +350,10 @@ bigdump/
│ │ ├── css/tailwind.css
│ │ └── js/*.js
│ ├── icons.svg # SVG icon sprite
│ └── img/
│ └── logo.png
│ ├── img/
│ │ └── logo.png
│ └── docs/
│ └── demov2.2.png # Screenshot
├── src/
│ ├── Config/Config.php
│ ├── Controllers/BigDumpController.php
Expand Down Expand Up @@ -394,8 +396,6 @@ bigdump/
│ └── workflows/
│ ├── build-assets.yml # CI asset pipeline
│ └── build-phar.yml # PHAR build & release
├── docs/
│ └── logo.png
├── CHANGELOG.md
├── LICENSE
└── README.md
Expand Down Expand Up @@ -480,10 +480,12 @@ If uploading large files (>500MB) fails with HTTP 500 error but smaller files wo
- **Original**: Alexey Ozerov (http://www.ozerov.de/bigdump) — Created in 2003
- **MVC Refactoring**: Version 2 by [w3spi5](https://github.com/w3spi5) — 2025

> 🔗 This fork is officially linked from the [original BigDump page](https://www.ozerov.de/bigdump).

---

## Screenshots

<p align="center">
<img src="docs/demov2.2.png" alt="BigDump Screenshot" width="800">
<img src="assets/docs/demov2.2.png" alt="BigDump Screenshot" width="800">
</p>
Binary file modified assets/img/logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
25 changes: 24 additions & 1 deletion config/config.example.php
Original file line number Diff line number Diff line change
Expand Up @@ -56,18 +56,35 @@
* - Safe for shared hosting environments
* - 64KB file buffer, 2000 INSERT batch size, 16MB max batch bytes
* - COMMIT after every batch
* - 5000 lines per session (v2.25+)
*
* 'aggressive':
* - Targets 128MB PHP memory_limit (REQUIRED: 128MB+ memory_limit)
* - +20-30% throughput improvement on INSERT-heavy dumps
* - 128KB file buffer, 5000 INSERT batch size, 32MB max batch bytes
* - COMMIT every 3 batches
* - 10000 lines per session (v2.25+)
*
* WARNING: If memory_limit < 128MB, aggressive mode automatically falls
* back to conservative to prevent memory exhaustion. Check logs for warnings.
*/
'performance_profile' => 'conservative',

/**
* Auto-aggressive mode threshold (v2.25+).
*
* Files larger than this threshold automatically use aggressive profile,
* even if conservative is configured. This provides optimal performance
* for large imports without requiring manual configuration changes.
*
* Set to 0 to disable auto-aggressive mode.
* Default: 104857600 (100MB)
*
* Requirements:
* - PHP memory_limit >= 128MB (otherwise auto-upgrade is skipped)
*/
'auto_profile_threshold' => 104857600, // 100MB

/**
* File read buffer size (in bytes).
* Controls how much data is read from the SQL file per I/O operation.
Expand Down Expand Up @@ -141,6 +158,11 @@

/**
* Number of lines to process per session (base value).
*
* v2.25+ defaults (profile-dependent):
* - Conservative: 5000 (increased from 3000)
* - Aggressive: 10000 (increased from 5000)
*
* With auto-tuning enabled (default), this is dynamically adjusted
* based on available RAM (NVMe-optimized profiles):
* < 512 MB -> 10,000 lines
Expand All @@ -150,7 +172,7 @@
* < 8 GB -> 150,000 lines
* > 8 GB -> 200,000 lines
*/
'linespersession' => 3000,
'linespersession' => 5000,

/**
* Force a specific batch size (bypasses auto-tuning).
Expand Down Expand Up @@ -310,6 +332,7 @@

/**
* Minimum batch size for auto-tuner.
* v2.25: Increased from 3000 to 5000 for better performance.
*/
'min_batch_size' => 5000,

Expand Down
84 changes: 84 additions & 0 deletions dist/bigdump-config.example.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
<?php

/**
* BigDump PHAR Configuration
*
* Copy this file to bigdump-config.php (in the same directory as bigdump.phar)
* and modify the values below.
*
* @package BigDump
*/

return [
// =========================================================================
// DATABASE CONFIGURATION (REQUIRED)
// =========================================================================

/**
* MySQL server.
* Format: 'hostname' or 'hostname:port' or 'localhost:/path/to/socket'
*/
'db_server' => 'localhost',

/**
* Database name.
*/
'db_name' => '',

/**
* MySQL username.
*/
'db_username' => '',

/**
* MySQL password.
*/
'db_password' => '',

/**
* Connection charset.
* Must match the dump file charset.
*/
'db_connection_charset' => 'utf8mb4',

// =========================================================================
// PERFORMANCE PROFILE
// =========================================================================

/**
* Performance profile selection.
* 'conservative' (default): Safe for shared hosting, 64MB memory
* 'aggressive': For dedicated servers, requires 128MB+ memory
*/
'performance_profile' => 'conservative',

// =========================================================================
// UPLOAD DIRECTORY
// =========================================================================

/**
* Upload directory for SQL dump files.
* Path is relative to the bigdump.phar location.
* Leave empty to use default 'uploads/' directory.
*/
'upload_dir' => './uploads/',

// =========================================================================
// IMPORT CONFIGURATION
// =========================================================================

/**
* AJAX mode (true = no page refresh during import).
*/
'ajax' => true,

/**
* Number of lines to process per session.
*/
'linespersession' => 3000,

/**
* Debug mode (shows detailed error traces).
*/
'debug' => false,
];
Binary file added dist/bigdump.phar
Binary file not shown.
Loading