For Oracle, change the data type for BLOB column to support storing up to 2GB #3070
Conversation
Torch3333
force-pushed
the
oracle_blob_change
branch
from
c02b715 to
58f82af
Compare
...integration-test/java/com/scalar/db/storage/jdbc/JdbcDatabaseColumnValueIntegrationTest.java
Show resolved
Hide resolved
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request changes the default data type for BLOB columns in Oracle from RAW(2000) to BLOB to support storing up to 2GB of data. This change aligns Oracle's BLOB handling with that of Db2 and addresses previous concerns about storage limitations. As a result of using the native BLOB type, several limitations are introduced for Oracle: BLOB columns can no longer be used as primary keys, clustering keys, secondary indexes, or in conditions and orderings for cross-partition scans. The changes include updating the RdbEngine for Oracle, adding new error codes, and modifying integration tests to respect these new limitations. A specific workaround for handling large BLOBs (>32KB) in Oracle has also been implemented. The changes are thorough and well-tested. I have one minor suggestion to improve a comment's clarity.
...integration-test/java/com/scalar/db/storage/jdbc/JdbcDatabaseColumnValueIntegrationTest.java
Outdated
Show resolved
Hide resolved
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Pull Request Overview
This PR updates the Oracle JDBC adapter to change the default data type mapping for BLOB columns from RAW(2000) to native Oracle BLOB type, which can store up to 2GB instead of just 2KB. This addresses the restrictive storage limitation of the previous mapping.
- Changes Oracle BLOB column mapping from
RAW(2000)toBLOBfor increased storage capacity - Introduces restrictions preventing BLOB columns from being used as keys, secondary indexes, or in cross-partition scan conditions/ordering
- Updates integration tests to disable BLOB-related functionality where Oracle no longer supports it
Reviewed Changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| RdbEngineOracle.java | Implements new BLOB data type mapping and validation methods for Oracle limitations |
| JdbcOperationChecker.java | Adds cross-partition scan condition checking for BLOB columns |
| Multiple test files | Updates integration tests to handle Oracle BLOB limitations and adds new BLOB size testing |
| CoreError.java | Adds error messages for Oracle BLOB limitations |
| Multiple admin/storage test files | Updates to disable BLOB column features where Oracle doesn't support them |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
...src/main/java/com/scalar/db/api/DistributedStorageCrossPartitionScanIntegrationTestBase.java
Show resolved
Hide resolved
core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineOracle.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineOracle.java
Outdated
Show resolved
Hide resolved
...integration-test/java/com/scalar/db/storage/jdbc/JdbcDatabaseColumnValueIntegrationTest.java
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Torch3333
requested review from
a team,
brfrn169,
feeblefakie and
komamitsu
and removed request for
a team
core/src/main/java/com/scalar/db/storage/jdbc/JdbcOperationChecker.java
Outdated
Show resolved
Hide resolved
# Conflicts: # core/src/main/java/com/scalar/db/common/CoreError.java
# Conflicts: # core/src/main/java/com/scalar/db/common/CoreError.java
core/src/main/java/com/scalar/db/storage/jdbc/JdbcOperationChecker.java
Outdated
Show resolved
Hide resolved
…p to 2GB (#3070) # Conflicts: # core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineOracle.java # core/src/test/java/com/scalar/db/storage/jdbc/JdbcAdminTestBase.java # Conflicts: # core/src/main/java/com/scalar/db/common/CoreError.java # core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineOracle.java # core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineStrategy.java
…p to 2GB (#3070) # Conflicts: # core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineOracle.java # core/src/test/java/com/scalar/db/storage/jdbc/JdbcAdminTestBase.java # Conflicts: # core/src/main/java/com/scalar/db/common/CoreError.java # core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineOracle.java # core/src/main/java/com/scalar/db/storage/jdbc/RdbEngineStrategy.java
4 participants
Description
This changes the default mapping for Oracle BLOB column to a data type that allows storing up to 2GB. The former data type
RAW(2000)allowed a maximum data size of 2000 bytes. This changes addresses a similar concerns regarding Db2 that the original data type only allowed to store up 2KB which was deemed too restrictive.The new Oracle data type used is named
BLOBand its maximum capacity is 2GB which also coincides with the maximum capacity that the backing objectbyte[]of BlobColumn object can hold.Because of limitations of the Oracle
BLOBdata type, ScalarDB BLOB column can no longer be used as :wherecondition in a Get or Scan operationRelated issues and/or PRs
Changes made
RAW(2000)toBLOBChecklist
Additional notes (optional)
Release notes
When using Oracle, the default data type used for ScalarDB BLOB column from Oracle is changed from
RAW(2000)toBLOBto allow storing data up to 2GB. This introduces new limitations: a BLOB column can no longer be used as a partition key, clustering key, secondary index, or a condition in a Get or Scan operation.