Schema support

devdoc/jdp/jdp-2025-06-schema-support.adoc

@@ -0,0 +1,149 @@
+= jdp-2025-06: Schema Support
+
+// SPDX-FileCopyrightText: Copyright 2025 Mark Rotteveel
+// SPDX-License-Identifier: LicenseRef-PDL-1.0
+
+== Status
+
+* Draft
+* Proposed for: Jaybird 7, potential partial backport to Jaybird 5
+
+== Type
+
+* Feature-Specification
+
+== Context
+
+Firebird 6.0 introduces support for schemas.
+To quote from `README.schemas.md` (of an early snapshot):
+
+____
+Firebird 6.0 introduces support for schemas in the database.
+Schemas are not an optional feature, so every Firebird 6 database has at least a `SYSTEM` schema, reserved for Firebird system objects (`RDB$*` and `MON$*`).
+
+User objects live in different schemas, which may be the automatically created `PUBLIC` schema or user-defined ones.
+It is not allowed (except for indexes) to create or modify objects in the `SYSTEM` schema.
+____
+
+Important details related to schemas:
+
+* Search path, defaults to `PUBLIC, SYSTEM`.
+The session default can be configured with `isc_dpb_search_path` (string DPB item).
+The current search path can be altered with `SET SEARCH_PATH TO ...`.
+`ALTER SESSION RESET` reverts to the session default.
+* If `SYSTEM` is not on the search path, it is automatically appended to the search path to be searched last
+* The "`current`" schema cannot be set separately;
+the first valid (i.e. existing) schema listed in the search path is considered the current schema.
+* `CURRENT_SCHEMA` and `RDB$GET_CONTEXT('SYSTEM', 'CURRENT_SCHEMA')` return the first valid schema from the search path
+* `RDB$GET_CONTEXT('SYSTEM', 'SEARCH_PATH')` returns the current search path
+* Objects not qualified with a schema name will be resolved using the current search path.
+This is done -- with some exceptions -- at prepare time.
+* TPB has new item `isc_tpb_lock_table_schema` to specify the schema of a table to be locked (1 byte length + string data)
+* Gbak has additional options to include/exclude (skip) schema data in backup or restore, similar to existing options to include/exclude tables
+* Gstat has additional options to specify a schema for operations involving tables
+* For validation, `val_sch_incl` and `val_sch_excl` (I don't think we use the related items `val_tab_incl`/`val_tab_excl` in Jaybird, so might not be relevant)
+* Stored procedure resolution:
+** Unqualified stored procedures (`<name>`) are searched on the search path
+** Qualified stored procedures (`<schema-or-package>.<name>`) are first located by schema and name, and if not found, searched on the search path by package and name.
+** Scope-specified stored procedures are either only located by schema (`<schema>%SCHEMA.<name>`) or only searched on the search path by package and name (`<package>%PACKAGE.<name>`), not both.
+** Fully-qualified packaged stored procedures (`<schema>.<package>.<name>`) are located by schema, package and name
+
+JDBC defines various methods, parameters, and return values or result set columns that are related to schemas.
+
+Jaybird 5 is the "`long-term support`" version for Java 8.
+
+[NOTE]
+====
+This document is in flux, and will be updated during implementation of the feature.
+====
+
+== Decision
+
+Jaybird 7 will implement schema support for Firebird 6.0.
+When Jaybird 7 is used on Firebird 5.0 or older, it will behave as before (no schemas at all).
+
+Further details can be found in <<consequences>>.
+
+Schema support will _not_ be backported to Jaybird 6 as the required changes are simply too large.
+It would mean that Jaybird 6 would contain most of Jaybird 7, and upgrading to a -- theoretical -- Jaybird 6.1 would have similar risks and compatibility issues as upgrading to Jaybird 7.
+
+Decision on a (partial) backport to Jaybird 5 -- as the "`long-term support`" version for Java 8 -- is still pending (e.g. as a Jaybird 5.1.x), and may be the subject of a separate JDP.
+We might only do that on demand and/or when someone is willing to sponsor the work.
+
+[#consequences]
+== Consequences
+
+The following changes are made to Jaybird to support schemas when connecting to Firebird 6.0 or higher:
+
+* Connection property `searchPath` (alias `search_path`, `isc_dpb_search_path`) to configure the default session search path.
++
+On Firebird 5.0 and older, this will be silently ignored.
+* In internal queries in Jaybird, and fully qualified object names, we'll use the regular -- unquoted -- identifier `SYSTEM`, even though `SYSTEM` is a SQL:2023 reserved word, to preserve dialect 1 compatibility.
+* `Connection`
+** `getSchema()` will return the result of `select CURRENT_SCHEMA from SYSTEM.RDB$DATABASE`;
+the connection will not store this value
+** `setSchema(String)` will query the current search path, and if not previously called, it will prepend the schema name to the search path, otherwise it will _replace_ the previously prepended schema name.
+The schema name is stored _only_ for this replacement operation (i.e. it will not be returned by `getSchema`!)
+*** The name must match exactly as is stored in the metadata (it is always case-sensitive!)
+*** Jaybird will take care of quoting, and will always quote on dialect 3
+*** Existence of the schema is **not** checked, so it is possible the current schema does not change with this operation, as `CURRENT_SCHEMA` reports the first _valid_ schema
+*** JDBC specifies that "`__Calling ``setSchema`` has no effect on previously created or prepared Statement objects.__`";
+Jaybird cannot honour this requirement for plain `Statement`, as schema resolution is on prepare time (which for plain `Statement` is on execute), and not always for `CallableStatement`, as the implementation may delay actual prepare until execution, though we do try to identify the procedure when the callable statement is created and use that to fully-qualify the procedure.
+* Request `isc_info_sql_relation_schema` after preparing a query, record it in `FieldDescriptor`, and return it were relevant for JDBC (e.g. `ResultSetMetaData.getSchemaName(int)`)
+** For Firebird 5.0 and older, we need to ensure that JDBC methods continue to report the correct value (i.e. `""` for schema-less objects)
+* A Firebird 6.0 variant of the `DatabaseMetaData` and other internal metadata queries needs to be written to address at least the following things:
+** Explicitly qualify metadata tables with `SYSTEM`, so the queries will work even if another schema on the search path contains tables with the same names as the system tables.
+** Returning schema names, and qualified object names where relevant (e.g. in `DatabaseMetaData` result sets)
+** Include schema names in joins to ensure matching the right objects
+** Allow searching for schema or schema pattern as specified in JDBC, or were needed for internal metadata queries
+** `getCatalogs`;
+it is not possible to identify the schema(s) within the confines of JDBC.
++
+We considered adding a column that lists the schema(s) that contain the package name, but we don't think it will see use in practice.
+* `FirebirdConnection`
+** Added method `String getSearchPath()` to obtain the search path as reported by `RBB$GET_CONTEXT('SYSTEM', 'SEARCH_PATH')`, or `null` if schemas are not supported
+** Added method `List<String> getSearchPathList()` to obtain the search path as a list of unquoted object names, or empty list if schemas are not supported
+** Added methods `setSearchPath(String)` and `setSearchPathList` with overloads `(String...)` and `(List<String>)` to set the search path.
+* `FBCallableStatement`
+** On creating the instance, the stored procedure is parsed and identified in the database metadata, including selectability, unless `ignoreProcedureType` is `true`
+*** Parsing of callable statements is changed to be able to identify schema, package and procedure name, including scope specifiers
+** Jaybird emulates the lookup rules as used by Firebird, and -- if found -- records the identified procedure so subsequent internal prepares refer to the same procedure, even if the search path changes;
+this fulfills the JDBC requirements that a `CallableStatement` is not sensitive to current schema changes, but only *if* Jaybird is able to identify the procedure, behaviour is undefined if the procedure was not found.
+** The API of `StoredProcedureMetaData` (internal API) is changed to not report selectability, but to update the `FBProcedureCall` instance with selectability and other information, like identified schema and/or package.
+** For qualified *and* unambiguous procedure reference, the selectability is cached *per connection*, for unqualified or ambiguous procedure reference, the lookup is performed on each `Connection.prepareCall`, to account for search path changes
+** Support for packages was missing in the handling of callable statements, and is added, also for older versions
+* Effects for management API
+** `StatisticsManager`
+*** `getTableStatistics` received an overload to also accept a list of schemas (`sts_schema`)
+** `FBTableStatisticsManager`/`TableStatistics`
+*** Internally `ObjectReference` is used for the table instead of a String
+*** The key of the map returned by `getTableStatistics()` is a qualified table reference (i.e. `{<table-name> | <quoted-schema>.<quoted-table-name>}`.
+For schemaless tables, the unquoted table name is used as the key for backwards compatibility when used against Firebird 5.0 and older.
+*** `TableStatistics` received two extra accessors:
+**** `schema()` with the schema, or empty string if schemaless (or not found)
+**** `tableReference()` with the qualified table reference (i.e. `[<quoted-schema>.]<quoted-table-name>` (contrary to the key of getTableStatistics, it's always quoted!))
+* TODO: Add information to Jaybird manual
+
+Note to self: use `// TODO Add schema support` in places that you identify need to get/improve schema support, while working on schema support elsewhere
+
+[appendix]
+== License Notice
+
+The contents of this Documentation are subject to the Public Documentation License Version 1.0 (the “License”);
+you may only use this Documentation if you comply with the terms of this License.
+A copy of the License is available at https://firebirdsql.org/en/public-documentation-license/.
+
+The Original Documentation is "`jdp-2025-06: Schema Support`".
+The Initial Writer of the Original Documentation is Mark Rotteveel, Copyright © 2025.
+All Rights Reserved.
+(Initial Writer contact(s): mark (at) lawinegevaar (dot) nl).
+
+////
+Contributor(s): ______________________________________.
+Portions created by ______ are Copyright © _________ [Insert year(s)].
+All Rights Reserved.
+(Contributor contact(s): ________________ [Insert hyperlink/alias]).
+////
+
+The exact file history is recorded in our Git repository;
+see https://github.com/FirebirdSQL/jaybird

src/docs/asciidoc/release_notes.adoc

@@ -508,6 +508,74 @@ Artificial testing on local WiFi with small blobs (200 bytes) shows a 30,000-45,
 
 This optimization was backported to Jaybird 5.0.8 and Jaybird 6.0.2.
 
+[#schemas]
+=== Schema support
+
+Firebird 6.0 introduces schemas, and Jaybird 7 provides support for schemas as defined in the JDBC specification.
+
+Changes include:
+
+* Connection property `searchPath` sets the initial search path of the connection.
+The search path is the list of schemas that will be searched for schema-bound objects if they are not explicitly qualified with a schema name.
+The first _valid_ schema is the current schema of the connection.
++
+The value of `searchPath` is a comma-separated list of schema names.
+Schema names that are case-sensitive or otherwise non-regular identifiers, must be quoted.
+Unknown schema names are ignored.
+If `SYSTEM` is not included, the server will automatically add it as the last schema.
+* `DatabaseMetaData`
+** Methods accepting a `schema` (exact match if not `null`) or `schemaPattern` (`LIKE` match if not `null`) will return no rows for value empty (`++""++`) on Firebird 6.0 and higher;
+use `null` or -- `schemaPattern` only -- `"%"` to match all schemas
+** `getCatalogs` -- when `useCatalogAsPackage=true` -- returns all (distinct) package names over all schemas.
+Within the limitations and specification of the JDBC API, this method cannot be used to find out which schema(s) contain a specific package name.
+// TODO Maybe add a custom column with a list of schema names?
+** `getColumnPrivileges` and `getTablePrivileges` received an additional column, `JB_GRANTEE_SCHEMA`, which is non-``null`` for grantees that are schema-bound (e.g. procedures).
++
+As this is a non-standard column, we recommend to always retrieve it by name.
+** `getProcedures` received two additional columns.
++
+As these are non-standard columns, we recommend to always retrieve them by name.
++
+*** `JB_PROCEDURE_TYPE` -- value of column `RDB$PROCEDURE_TYPE`
++
+For the possible values, the following constants were added to `FirebirdDatabaseMetaData`:
++
+**** `jbProcedureTypeUnknown` (`0`)
+**** `jbProcedureTypeSelectable` (`1`)
+**** `jbProcedureTypeExecutable` (`2`).
+*** `JB_PROCEDURE_SOURCE` -- value of column `RDB$PROCEDURE_SOURCE`: the body of the stored procedure;
+this column is `null` for procedures in a package.
+** `getProcedureSourceCode` is deprecated, the recommended replacement is `DatabaseMetaData.getProcedures(String, String, String)`, column `JB_PROCEDURE_SOURCE`.
+** `getTriggerSourceCode`/`getViewSourceCode` now also have an overload accepting the schema;
+the overloads without a `schema` parameter, or `schema` is `null` will return the source code of the first match found (schema order is undefined).
+This behaviour also applies for -- now deprecated -- `getProcedureSourceCode(String)`.
++
+The `schema` parameter is ignored on Firebird 5.0 and older.
+** `getSchemas()` returns all defined schemas
+** `getSchemas(String catalog, String schemaPattern)` returns all schemas matching the `LIKE` pattern `schemaPattern`, with the following caveats
+*** `catalog` non-empty will return no rows -- even if `useCatalogAsPackage` is `true`;
+we recommend to always use `null` for `catalog`
+* `ResultSetMetaData`
+** `getSchemaName` reports the schema if the column is backed by a table, otherwise empty string (`""`)
+* `FirebirdConnection`/`FBConnection`
+** Added method `String getSearchPath()` to obtain the search path as reported by `RBB$GET_CONTEXT('SYSTEM', 'SEARCH_PATH')`, or `null` if schemas are not supported
+** Added method `List<String> getSearchPatList()` to obtain the search path as a list of unquoted object names, or empty list if schemas are not supported
+** Added methods `setSearchPath(String)` and `setSearchPathList(String...)/(List<String>)` where added to set the search path;
+these methods throw `SQLFeatureNotSupportedException` if schemas are not supported.
+* `StatisticsManager`
+** `getTableStatistics`
+*** `getTableStatistics(String[] tableNames)` was changed to accept varargs (`getTableStatistics(String... tableNames)`)
+*** Added overload `getTableStatistics(List<String> tableNames)` with same behaviour as `getTableStatistics(String... tableNames)`
+*** Added overload `getTableStatistics(List<String> schemas, List<String> tableNames)` -- if `schemas` is non-empty, on Firebird 6.0 and higher, it will restrict the search for tables to the specified schemas
+* `FBTableStatisticsManager` (experimental feature)
+** For schema-bound tables, the key of the map returned by `getTableStatistics()` is a fully qualified and quoted table reference (i.e. `<quoted-schema>.<quoted-table-name>`).
+For schemaless tables (Firebird 5.0 and older, or tables that were not found), the key is still the unquoted `<table-name>`.
+** The static method `toKey(String schema, String tableName)` can be used to create a table reference in the same format as the key of the map returned by `getTableStatistics()`.
+The `schema` can be `null` or empty string for schemaless tables (i.e. Firebird 5.0 or older)
+** The `TableStatistics` object received additional accessors:
+*** `schema()` with the schema, or empty string for schemaless (Firebird 5.0 or older) or if the table was not found
+*** `tableReference()` with the fully qualified and quoted table reference (i.e. `[<quoted-schema>.]<quoted-table-name>`)
+
 // TODO add major changes
 
 [#other-fixes-and-changes]
@@ -654,7 +722,7 @@ If you are confronted with such a change, let us know on {firebird-java}[firebir
 
 The following changes might cause issues, though we think this is unlikely:
 
-// TODO Document unlikely breaking changes, or remove section
+// TODO Document unlikely breaking changes, or comment out this section
 
 [#breaking-changes-for-jaybird-8]
 === Breaking changes for Jaybird 8

src/main/org/firebirdsql/ds/AbstractConnectionPropertiesDataSource.java

@@ -518,6 +518,16 @@ public void setMaxBlobCacheSize(int maxBlobCacheSize) {
         FirebirdConnectionProperties.super.setMaxBlobCacheSize(maxBlobCacheSize);
     }
 
+    @Override
+    public String getSearchPath() {
+        return FirebirdConnectionProperties.super.getSearchPath();
+    }
+
+    @Override
+    public void setSearchPath(String searchPath) {
+        FirebirdConnectionProperties.super.setSearchPath(searchPath);
+    }
+
     @SuppressWarnings("deprecation")
     @Deprecated(since = "5")
     @Override