diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..92bf941 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,223 @@ +# Node-Firebird Roadmap + +This document outlines the future development direction for the `node-firebird` library. Our primary goals are to modernize the codebase, implement support for the latest Firebird features, and improve the overall developer experience. + +> **Note:** The issue and PR lists below may be incomplete. Please check the [GitHub issues page](https://github.com/hgourvest/node-firebird/issues) for the most up-to-date list of open items, and feel free to open new issues or add a 👍 to existing ones to help prioritize. + +--- + +## Guiding Principles + +- **Stability first:** fix hangs, leaks, and "callback never called" scenarios before adding new API surface. +- **Compatibility:** keep the current callback API stable while adding promise/TypeScript improvements. +- **Framework-friendly:** make it straightforward to use in web frameworks (Express, Fastify, etc.) without connection leaks. +- **Incremental delivery:** ship small, reviewable changes rather than large rewrites. + +--- + +## 1. User-Reported Issues (Prioritized) + +These items come directly from current open issues and should be tracked as roadmap deliverables. Priority is based on user impact and frequency of reports. + +### P0 — Correctness / hang / callback never called + +- **[Issue #387](https://github.com/hgourvest/node-firebird/issues/387) — BLOB callback never runs** + Goal: ensure query callbacks always settle (success or error) and that BLOB streaming cannot stall silently. + Deliverables: + - Add internal watchdog/timeout for pending requests (opt-in first, then default). + - Add debug logging hooks (connection + statement lifecycle) to diagnose stalls. + - Add regression test: BLOB read path must either resolve or error deterministically. + +- **[Issue #357](https://github.com/hgourvest/node-firebird/issues/357) — Pool connections hanging after idle time** + Goal: pool should detect dead sockets and recover cleanly. + Deliverables: + - Health-check on checkout (lightweight ping or keepalive). + - Better handling of server disconnects + reconnect backoff. + - Ensure `db.detach()` always returns connection to pool in all code paths. + +- **[Issue #343](https://github.com/hgourvest/node-firebird/issues/343) — Pool connection errors** + Goal: pool should provide reliable acquisition and clear error messages. + Deliverables: + - Clarify pool error semantics and add a retry strategy option. + - Add tests for pool exhaustion and stale idle connections. + +- **[Issue #313](https://github.com/hgourvest/node-firebird/issues/313) — Sequential heap limit / allocation failed** + Goal: reduce memory pressure and document safe usage patterns. + Deliverables: + - Validate that `sequentially` does not retain rows unintentionally. + - Provide streaming patterns and "do/don't" guidance in docs. + +### P1 — Behavior fixes / sharp edges + +- **[Issue #341](https://github.com/hgourvest/node-firebird/issues/341) — RETURNING failure leads to uncaught error** + Goal: isolate statement failures and reset connection state correctly after errors. + Deliverables: + - Harden error handling: ensure connection state machine resets on statement failure. + - Add regression test for failure + subsequent query on the same connection. + +- **[Issue #329](https://github.com/hgourvest/node-firebird/issues/329) — Pool idle connection deletion** + Goal: make pool idle cleanup safe and observable. + Deliverables: + - Document and/or fix idle cleanup behavior. + - Add observable events/metrics: "connection closed due to idle timeout", etc. + +### P2 — Questions / documentation gaps (triage) + +These may be closed with a clear explanation or resolved with a small doc/code fix. + +- **[Issue #353](https://github.com/hgourvest/node-firebird/issues/353)** — LIST() function support question +- **[Issue #348](https://github.com/hgourvest/node-firebird/issues/348)** — Protocol version hard-coded? +- **[Issue #335](https://github.com/hgourvest/node-firebird/issues/335)** — BLOB loading slowly +- **[Issue #336](https://github.com/hgourvest/node-firebird/issues/336)** — Default encoding option (UTF-8 vs latin1) +- **[Issue #332](https://github.com/hgourvest/node-firebird/issues/332)** — LIKE clause error in SELECT +- **[Issue #320](https://github.com/hgourvest/node-firebird/issues/320)** — Deno compatibility + +--- + +## 2. Express.js Support (First-Class Integration) + +The library already works with Express, but "support" should mean **documented, safe-by-default patterns** that prevent connection leaks and hangs in a request/response lifecycle. + +### Deliverables + +- **New docs section: "Using node-firebird with Express.js"** + - Recommended architecture: create a single pool at app startup and reuse it. + - Request lifecycle pattern: acquire connection → run queries → always release in `finally`. + - Transaction middleware example (commit on success, rollback on error). + - Error handling: map Firebird errors to HTTP status codes without exposing internals. + - BLOB streaming example: stream BLOBs directly to `res` and ensure `db.detach()` on `finish`/`close`. + +- **Optional helper utilities (non-breaking additions)** + - `pool.withConnection(async (db) => { ... })` — guarantees release even on error. + - `db.withTransaction(async (tx) => { ... })` — auto-commit or auto-rollback. + +### Acceptance Criteria + +Provide at least **two copy-paste ready examples**: + +1. **Standard JSON API endpoint** — query rows and return JSON, with proper connection release. +2. **BLOB streaming download endpoint** — pipe a BLOB column to the HTTP response, with cleanup on client disconnect. + +--- + +## 3. TypeScript Roadmap (Detailed) + +Goal: improve TypeScript support without breaking existing JavaScript/callback users. Each phase is independently deliverable. + +### Phase A — Accurate typings for the current API (no runtime changes) + +Deliver or refresh `.d.ts` definitions covering: +- Connection options (`host`, `port`, `database`, `user`, `password`, `role`, `charset`, `pageSize`, `wireCrypt`, `wireCompression`, `dbCryptConfig`, `sessionTimeZone`, `retryConnectionInterval`, `blobAsText`, `lowercase_keys`, `encoding`, etc.) +- Pool object and its methods (`pool.get`, `pool.destroy`) +- Database / transaction / statement objects and all their callbacks +- Event emitter events (`row`, `result`, `attach`, `detach`, `reconnect`, `error`, `transaction`, `commit`, `rollback`) +- Result shapes: `db.query` → `Array>`, `db.execute` → `unknown[][]` +- Blob columns (appear as functions at runtime; typings must model that explicitly) + +**Tradeoffs and constraints to be aware of:** +- Query result shapes are dynamic (depend on the SQL); TypeScript cannot infer column names automatically. Users must cast or supply their own row types. +- Blob columns being functions is a runtime quirk; the typings will be accurate but may surprise users new to the library. +- Some options and event payloads vary by server version or protocol; typings must be permissive in those areas to avoid false type errors. +- Maintaining `.d.ts` files separately from the JavaScript source creates a risk of drift; a linting step should be added to CI to catch obvious mismatches. + +### Phase B — Dual API: callbacks + promises + +Provide promise-returning wrappers alongside the existing callbacks: +- `Firebird.attachAsync`, `pool.getAsync`, `db.queryAsync`, `db.executeAsync`, `transaction.commitAsync`, `transaction.rollbackAsync`, etc. + +**Tradeoffs and risks:** +- Promise wrappers can hide resource-leak bugs if callers forget `finally`; `withConnection` / `withTransaction` helpers (see Express section) mitigate this. +- Promise wrappers must not change execution ordering or serialization semantics, especially around `sequentially`. +- Mixing callbacks and promises in the same codebase increases the surface for subtle bugs; docs should recommend one style per project. +- Rejected promises that are not caught will produce `UnhandledPromiseRejection` warnings; users need to be aware of this difference from callback-style errors. + +### Phase C — Modern TypeScript ergonomics (optional / future) + +- Consider publishing dual CJS + ESM package exports, or documenting CJS-only stance clearly. +- Add opt-in generic helpers: + - `db.query>(sql, params)` for user-supplied row shapes. + - `db.queryAsync(...): Promise` + +**Constraints:** +- Full ESM migration can be a **breaking change** depending on consumer build tooling; it may require a major version bump and a migration guide. +- Generic row typing is only as good as the types the user supplies; it does not validate SQL at compile time. +- A major TypeScript rewrite would be a large undertaking. Incremental phases (A → B → C) are preferred to reduce risk and keep the library usable throughout the transition. + +### Modern JavaScript Classes (prerequisite / parallel track) + +Before or alongside the TypeScript work, refactor the prototype-based codebase to use ES6 `class` syntax. This improves readability and makes a future TypeScript migration less disruptive. + +**Benefits:** cleaner code structure, easier to understand inheritance. +**Risk:** a large mechanical refactor may introduce subtle behavioral regressions; must be accompanied by thorough test coverage. + +--- + +## 4. Protocol Implementation Status + +| Firebird Version | Protocol Versions | Status | +| :--- | :--- | :--- | +| 2.5 | 10, 11, 12, 13 | ✅ Implemented | +| 3.0 | 14, 15 | ✅ Implemented | +| 4.0 | 16, 17 | ✅ Protocol 16 Implemented / 🚧 Protocol 17 Missing | +| 5.0 | 18 | ❌ Not Implemented | +| 6.0 | N/A | ❌ Not Implemented | + +### Firebird 3 Support + +- **Protocol Versions 14 and 15:** ✅ Implemented. +- **Enhanced Authentication:** ✅ Implemented — Srp256 (SHA-256) alongside Srp (SHA-1) and Legacy_Auth. +- **Wire Protocol Encryption:** ✅ Implemented — Arc4 (RC4) stream cipher using SRP session keys. +- **Wire Protocol Compression:** ✅ Implemented — zlib compression for protocol 13+. +- **Packed (NULL-aware) Row Data:** ✅ Implemented — null bitmap for protocol 13+. +- **op_cond_accept Handling:** ✅ Implemented. +- **UTF-8 User Identification:** ✅ Implemented. +- **Database Encryption Callback:** ✅ Implemented — `op_crypt_key_callback` support; `dbCryptConfig` accepts plain text or base64-encoded keys. + +### Firebird 4 Support + +- **Protocol Versions 16 and 17:** ✅ Protocol 16 Implemented / 🚧 Protocol 17 Missing. +- **Statement Timeout:** ✅ Implemented (Protocol 16+). +- **`INT128` Data Type:** ✅ Implemented. +- **Time Zone Support:** ✅ Implemented — `TIME WITH TIME ZONE`, `TIMESTAMP WITH TIME ZONE`, `sessionTimeZone` option (Protocol 16+). +- **`DECFLOAT` Data Type:** ❌ TODO — `DECFLOAT(16)` and `DECFLOAT(34)`. + +### Firebird 5 Support + +- **Protocol Version 18:** ❌ TODO. +- **Bidirectional Cursors:** ❌ TODO — scrollable cursors for remote database access. +- **`RETURNING` Multiple Rows:** ❌ TODO — DML returning multiple rows. +- **`SKIP LOCKED`:** ❌ TODO — `SELECT WITH LOCK`, `UPDATE`, and `DELETE`. +- **Parallel Workers Information:** ❌ TODO. + +### Firebird 6 and Beyond + +- **Native `JSON` Data Type:** ❌ TODO — native JSON storage support. +- **SQL-Standard `ROW` Type:** ❌ TODO — structured data types as columns or variables. +- **SQL-Compliant JSON Functions:** ❌ TODO — `JSON_VALUE`, `JSON_QUERY`, `JSON_EXISTS`, `JSON_OBJECT`. +- **Tablespaces:** ❌ TODO — physical storage location control. +- **SQL Schemas:** ❌ TODO — standard schema namespace support. +- **Enhanced Collation Support:** ❌ TODO — collations declared as part of the data type. + +--- + +## 5. In-Flight PRs + +These are open pull requests that are close to being merged and represent near-term deliverables. + +- **[PR #385](https://github.com/hgourvest/node-firebird/pull/385)** — Use native `BigInt` instead of the `big-integer` library +- **[PR #383](https://github.com/hgourvest/node-firebird/pull/383)** — `DECFLOAT` data type support + +--- + +## Suggested Release Buckets + +| Target | Items | +| :--- | :--- | +| Next patch | P0 bug fixes: #387, #357, #343, #341 | +| Next minor | Express.js docs + helpers; TS Phase A typings; in-flight PRs #385, #383 | +| Future minor | TS Phase B (promise wrappers); P1 + P2 issues; Protocol 17 / Firebird 5 | +| Future major | ESM/CJS rework; TS Phase C generics; full class-based refactor (only if breaking) | + +--- + +We believe these changes will make `node-firebird` a more robust, modern, and developer-friendly library for accessing Firebird databases. Contributions are welcome — please check the [open issues](https://github.com/hgourvest/node-firebird/issues) and [contributing guide](README.md) for details. diff --git a/Roadmap.md b/Roadmap.md deleted file mode 100644 index 7315f3f..0000000 --- a/Roadmap.md +++ /dev/null @@ -1,87 +0,0 @@ -# Node-Firebird Roadmap - -This document outlines the future development direction for the `node-firebird` library. Our primary goals are to modernize the codebase, implement support for the latest Firebird features, and improve the overall developer experience. - -## Protocol Implementation Status - -The following table summarizes the current and planned implementation status of the Firebird wire protocol for each major version. - -| Firebird Version | Protocol Versions | Status | -| :--- | :--- | :--- | -| 2.5 | 10, 11, 12, 13 | ✅ Implemented | -| 3.0 | 14, 15 | ✅ Implemented | -| 4.0 | 16, 17 | ✅ Protocol 16 Implemented / 🚧 Protocol 17 Missing | -| 5.0 | 18 | ❌ Not Implemented | -| 6.0 | N/A | ❌ Not Implemented | - -## Firebird 3 Support - -Firebird 3 introduced Protocol 13, which brought significant changes focusing on security and performance. The following features have been implemented: - -- **Protocol Versions 14 and 15:** ✅ Implemented - newer wire protocol versions are now supported. -- **Enhanced Authentication:** ✅ Implemented - Srp256 (SHA-256) authentication plugin is now supported alongside Srp (SHA-1) and Legacy_Auth. -- **Wire Protocol Encryption:** ✅ Implemented - Arc4 (RC4) stream cipher encryption for all network traffic using SRP session keys. -- **Wire Protocol Compression:** ✅ Implemented - zlib compression support for protocol version 13+. -- **Packed (NULL-aware) Row Data:** ✅ Implemented - null bitmap support for protocol version 13+. -- **op_cond_accept Handling:** ✅ Implemented - proper handling of conditional accept with authentication continuation. -- **UTF-8 User Identification:** ✅ Implemented - all user identification is properly handled with UTF-8 encoding via `isc_dpb_utf8_filename` flag for Firebird 3+. -- **Database Encryption Callback:** ✅ Implemented - support for database encryption key callback (`op_crypt_key_callback`) during the connect phase, allowing connections to encrypted databases. The `dbCryptConfig` connection option supports both plain text and base64-encoded encryption keys. - -## Firebird 4 Support - -Firebird 4 introduced Protocol versions 16 and 17, continuing to build upon the foundation of Firebird 3. Key features to implement include: - -- **Protocol Versions 16 and 17:** ✅ Protocol 16 Implemented - support for statement timeout, INT128, and timezones. 🚧 Protocol 17 Missing. -- **Statement Timeout:** ✅ Implemented - support for `op_execute` with statement timeout (Protocol 16+). -- **`INT128` Data Type:** ✅ Implemented - support for 128-bit integer data type. -- **Time Zone Support:** ✅ Implemented - support for `TIME WITH TIME ZONE`, `TIMESTAMP WITH TIME ZONE`, and `sessionTimeZone` connection option (Protocol 16+). -- **`DECFLOAT` Data Type:** ❌ TODO - support for `DECFLOAT(16)` and `DECFLOAT(34)`. - -## Firebird 5 Support - -Firebird 5 introduces Protocol version 18 and a host of new SQL features and performance improvements: - -- **Protocol Version 18:** ❌ TODO - implement the latest protocol version. -- **Bidirectional Cursors:** ❌ TODO - implement support for scrollable cursors for remote database access. -- **`RETURNING` Multiple Rows:** ❌ TODO - enhance DML operations to support returning multiple rows. -- **`SKIP LOCKED`:** ❌ TODO - add support for the `SKIP LOCKED` clause in `SELECT WITH LOCK`, `UPDATE`, and `DELETE` statements. -- **Parallel Workers Information:** ❌ TODO - support for parallel workers information in SQL information items. - - -## Firebird 6 and Beyond - -As Firebird 6 and future versions are released, we will continue to monitor new features and plan for their implementation. Key areas of interest include: - -- **Native `JSON` Data Type:** Implement support for the new native `JSON` type (often handled as optimized binary storage). -- **SQL-Standard `ROW` Type:** Support for structured data types (records) as columns or variables. -- **SQL-Compliant JSON Functions:** Implement client-side support for `JSON_VALUE`, `JSON_QUERY`, `JSON_EXISTS`, and `JSON_OBJECT`. -- **Tablespaces:** Add support for tablespaces to control physical storage locations. -- **SQL Schemas:** Implement support for SQL-standard schemas for better namespace organization. -- **Enhanced Collation Support:** Support for collations defined directly as part of the data type declaration. - -## Codebase Refactoring - -The current codebase is functional but could be significantly improved by adopting modern JavaScript and TypeScript features. - -### Modern JavaScript Classes - -The existing codebase is written in a prototype-based style. We plan to refactor the entire library to use modern JavaScript classes (`class` syntax). This will improve readability, maintainability, and make the code easier to understand for new contributors. - -**Benefits:** - -- Improved code structure and organization. -- Easier to reason about inheritance and object-oriented patterns. -- Better alignment with modern JavaScript best practices. - -### TypeScript Rewrite - -A full rewrite of the library in TypeScript is a long-term goal. This would bring the benefits of static typing, improved developer tooling, and a more robust codebase. - -**Benefits:** - -- **Type Safety:** Catch errors at compile-time instead of runtime. -- **Improved Autocomplete:** Better editor support and developer experience. -- **Self-documenting Code:** Types make the code easier to understand and use. -- **Easier Refactoring:** Static analysis makes it easier to refactor code with confidence. - -We believe these changes will make `node-firebird` a more robust, modern, and developer-friendly library for accessing Firebird databases.