diff --git a/.changeset/great-experts-repair.md b/.changeset/great-experts-repair.md new file mode 100644 index 00000000..f605e0ac --- /dev/null +++ b/.changeset/great-experts-repair.md @@ -0,0 +1,17 @@ +--- +"@cipherstash/protect-dynamodb": major +"@cipherstash/protect": major +"@cipherstash/schema": major +--- + +Added JSON and INT data type support and update FFI to v0.17.1 with x86_64 musl environment platform support. + +* Update @cipherstash/protect-ffi from 0.16.0 to 0.17.1 with support for x86_64 musl platforms. +* Add searchableJson() method to schema for JSON field indexing (the search operations still don't work but this interface exists) +* Refactor type system: EncryptedPayload → Encrypted, add JsPlaintext +* Add comprehensive test suites for JSON, integer, and basic encryption +* Update encryption format to use 'k' property for searchable JSON +* Remove deprecated search terms tests for JSON fields +* Simplify schema data types to text, int, json only +* Update model helpers to handle new encryption format +* Fix type safety issues in bulk operations and model encryption \ No newline at end of file diff --git a/README.md b/README.md index 87c8dfeb..5f90f02c 100644 --- a/README.md +++ b/README.md @@ -985,10 +985,59 @@ const bulkDecryptedResult = await protectClient ## Supported data types -Protect.js currently supports encrypting and decrypting text. -Other data types like booleans, dates, ints, floats, and JSON are well-supported in other CipherStash products, and will be coming to Protect.js soon. +Protect.js supports a number of different data types with support for additional types on the roadmap. + +| JS/TS Type | Available | Notes | +|--|--|--| +| `string` | ✅ | +| `number` | ✅ | +| `json` (opaque) | ✅ | | +| `json` (searchable) | ⚙️ | Coming soon | +| `bigint` | ⚙️ | Coming soon | +| `boolean`| ⚙️ | Coming soon | +| `date` | ⚙️ | Coming soon | + +If you need support for ther data types please [raise an issue](https://github.com/cipherstash/protectjs/issues) and we'll do our best to add it to Protect.js. + +### Type casting + +When encrypting types other than `string`, Protect requires the data type to be specified explicitly using the `dataType` function on the column definition. + +For example, to handle encryption of a `number` field called `score`: + +```ts +const users = csTable('users', { + score: csColumn('score').dataType('number') +}) +``` + +This means that any JavaScript/TypeScript `number` will encrypt correctly but if an attempt to encrypt a value of a different type is made the operation will fail with an error. +This is particularly important for searchable index schemes that require data types (and their encodings) to be consistent. + +In an unencrypted setup, this type checking is usually handled by the database (the column type in a table) but when the data is encrypted, the database can't determine what type the plaintext value should be so we must specify it in the Protect schema instead. + +> [!IMPORTANT] +> If the data type of a column is set to `bigint`, floating point numbers will be converted to integers (via truncation). + +### Handling of null and special values + +There are some important special cases to be aware of when encrypting values with Protect.js. +For example, encrypting `null` or `undefined` will just return a `null`/`undefined` value. + +When `dataType` is `number`, attempting to encrypt `NaN`, `Infinity` or `-Infinity` will fail with an error. +Encrypting `-0.0` will coerce the value into `0.0`. + +The table below summarizes these cases. + +| Data type | Plaintext | Encryption | +|--|--|--| +|`any`| `null` | `null` | +| `any` | `undefined` | `undefined` | +| `number` | `-0.0` | Encryption of `0.0` | +| `number` | `NaN` | _Error_ | +| `number` | `Infinity` | _Error_| +| `number` | `-Infinity` | _Error_| -Until support for other data types are available, you can express interest in this feature by adding a :+1: on this [GitHub Issue](https://github.com/cipherstash/protectjs/issues/48). ## Searchable encryption diff --git a/docs/prompts/init-protect.md b/docs/prompts/init-protect.md new file mode 100644 index 00000000..df45ea33 --- /dev/null +++ b/docs/prompts/init-protect.md @@ -0,0 +1,111 @@ +# Implementing Protect.js into a Node.js application + +Your task is to introduce Protect.js into a Node.js application. Protect.js requires the Node.js runtime so it will still work with framweworks like Next.js and Tanstack Start, since they support running code that only executes on the server. + +--- + +## Installing Protect.js + +Determine what package manager the application is using. This will either be `pnpm`, `npm`, or `bun` and then use the appropriate package manager to add Protect.js to the application: + +```bash +npm install @cipehrstash/protect +# or +pnpm add @cipehrstash/protect +# or +bun add @cipehrstash/protect +``` + +If you detect a mono repo, you need to ask the user which application they want Protect.js installed in. + +--- + +## Adding scafolding for the Protect.js client, schemas, and example code + +In the rool of the application (if the application is configred to use something like `src` then this is where these operations will occur), you need to add a `protect` directory with the following files/content. If the application uses TypeScript use the `.ts` extension, else use the `.js` extension. + +`protect/schemas.(ts/js)` +```js +import { csTable, csColumn } from '@cipherstash/protect` + +export const protectedExample = csTable('example_table', { + sensitiveData: csColumn('sensitiveData'), +} +``` + +`protect/index.(ts/js)` +```js +import { protect } from '@cipehrstash/protect' +import { * as protectSchemas } from './schemas' + +export const protectClient = protect({ + schemas: [...protectSchemas] +}) +``` + +`protect/example.(ts/js)` +```js +import { protectClient } from './index' +import { * as protectSchemas } from './schemas' + +const sensitiveData = "Let's encrypt some data." + +/** + * There is no need to wrap any protectClient method in a try/catch as it will always return a Result pattern. + * --- + * The Result will either contain a `failure` OR a `data` key. You should ALWAYS check for the `failure` key first. + * If the `failure` key is present, you should handle the error accordingly. + * If the `data` key is present, the operation was successful. + */ +// +const encryptResult = protectClient.encrypt(sensitiveData, { + table: protectSchemas.protectedExample + column: protectSchemas.protectedExample.sensitiveData +}) + +if (encryptResult.failure) { + // You as the developer can determine exactly how you want to handle the failure scenario. +} + +const encryptedData = encryptResult.data + +/** + * The encryptedData is a JSON object which is unique to CipherStash. + * You can store this data in any data store that supports JSON, however, only PostgreSQL supports searchable encryption operations. + * --- + * When storing in PostgreSQL, the database instance needs to be configured to Encrypt Query Language (EQL). + * More information can be found in the [Encrypt Query Language (EQL) documentation](https://github.com/cipherstash/encrypt-query-language). + * After EQL is installed, you can use the `eql_v2_encrypted` data type to store the encrypted data. + * ```sql + * CREATE TABLE protected_example ( + * id SERIAL PRIMARY KEY, + * sensitive_data eql_v2_encrypted + * ) + * ``` + **/ +console.log('encryptedData:', encryptedData) + +const decryptResult = protectClient.decrypt(encryptedData) + +if (decryptResult.failure) { + // Again, you as the developer can determine exactly how you want to handle the failure scenario. +} + +const decryptedData = decryptResult.data + +/** + * The decryptedData is the plaintext data. + */ +console.log('decryptedData:', decryptedData) +``` + +--- + +After the initialization process, you need to also analyze the application code base to determine the following: + +1. What database and general data store (if any) does the application connect to and store data? +2. Does the application use a specific ORM (like Drizzle ORM, Prisma, or Sequalize)? +3. What types of data inputs does this application ingest that could be considered sensitive data? +4. If the application is ingesting sensitive data, where does the application store the data? + +Use these answer in combonation with the answers provided a CIPHERSTASH_GETTING_STARTED.md file which will be used to help the user determine the next steps based on the application specific details. \ No newline at end of file diff --git a/packages/protect-dynamodb/__tests__/dynamodb.test.ts b/packages/protect-dynamodb/__tests__/dynamodb.test.ts index 1cd036a7..0b8ee27a 100644 --- a/packages/protect-dynamodb/__tests__/dynamodb.test.ts +++ b/packages/protect-dynamodb/__tests__/dynamodb.test.ts @@ -8,10 +8,16 @@ const schema = csTable('dynamo_cipherstash_test', { firstName: csColumn('firstName').equality(), lastName: csColumn('lastName').equality(), phoneNumber: csColumn('phoneNumber'), + json: csColumn('json').dataType('json'), + jsonSearchable: csColumn('jsonSearchable').dataType('json'), + //.searchableJson('users/jsonSearchable'), example: { protected: csValue('example.protected'), deep: { protected: csValue('example.deep.protected'), + protectNestedJson: csValue('example.deep.protectNestedJson').dataType( + 'json', + ), }, }, }) @@ -30,7 +36,7 @@ describe('protect dynamodb helpers', () => { }) }) - it('should encrypt columns', async () => { + it('should encrypt and decrypt a model', async () => { const testData = { id: '01ABCDEFGHIJKLMNOPQRSTUVWX', email: 'test.user@example.com', @@ -39,6 +45,22 @@ describe('protect dynamodb helpers', () => { firstName: 'John', lastName: 'Smith', phoneNumber: '555-555-5555', + json: { + name: 'John Doe', + age: 30, + preferences: { + theme: 'dark', + notifications: true, + }, + }, + jsonSearchable: { + name: 'John Doe', + age: 30, + preferences: { + theme: 'dark', + notifications: true, + }, + }, companyName: 'Acme Corp', batteryBrands: ['Brand1', 'Brand2'], metadata: { role: 'admin' }, @@ -48,6 +70,9 @@ describe('protect dynamodb helpers', () => { deep: { protected: 'deep protected', notProtected: 'deep not protected', + protectNestedJson: { + hello: 'world', + }, }, }, } @@ -80,6 +105,16 @@ describe('protect dynamodb helpers', () => { expect(encryptedData.example.notProtected).toBe('I am not protected') expect(encryptedData.example.deep.notProtected).toBe('deep not protected') expect(encryptedData.metadata).toEqual({ role: 'admin' }) + + const decryptResult = await protectDynamo.decryptModel( + encryptedData, + schema, + ) + if (decryptResult.failure) { + throw new Error(`Decryption failed: ${decryptResult.failure.message}`) + } + + expect(decryptResult.data).toEqual(testData) }) it('should handle null and undefined values', async () => { diff --git a/packages/protect-dynamodb/src/helpers.ts b/packages/protect-dynamodb/src/helpers.ts index 8566dcff..c52d0a15 100644 --- a/packages/protect-dynamodb/src/helpers.ts +++ b/packages/protect-dynamodb/src/helpers.ts @@ -1,4 +1,8 @@ -import type { EncryptedPayload } from '@cipherstash/protect' +import type { + Encrypted, + ProtectTable, + ProtectTableColumn, +} from '@cipherstash/protect' import type { ProtectDynamoDBError } from './types' export const ciphertextAttrSuffix = '__source' export const searchTermAttrSuffix = '__hmac' @@ -83,7 +87,7 @@ export function toEncryptedDynamoItem( typeof attrValue === 'object' && 'c' in (attrValue as object)) ) { - const encryptPayload = attrValue as EncryptedPayload + const encryptPayload = attrValue as Encrypted if (encryptPayload?.c) { const result: Record = {} if (encryptPayload.hm) { @@ -92,6 +96,13 @@ export function toEncryptedDynamoItem( result[`${attrName}${ciphertextAttrSuffix}`] = encryptPayload.c return result } + + // TODO: Need to implement the new Encrypt payload type when FFI is updated + if (encryptPayload?.sv) { + const result: Record = {} + result[`${attrName}${ciphertextAttrSuffix}`] = encryptPayload.sv + return result + } } // Handle nested objects recursively @@ -122,8 +133,8 @@ export function toEncryptedDynamoItem( } export function toItemWithEqlPayloads( - decrypted: Record, - encryptedAttrs: string[], + decrypted: Record, + encryptSchemas: ProtectTable, ): Record { function processValue( attrName: string, @@ -139,24 +150,42 @@ export function toItemWithEqlPayloads( return {} } + const encryptConfig = encryptSchemas.build() + const encryptedAttrs = Object.keys(encryptConfig.columns) + const columnName = attrName.slice(0, -ciphertextAttrSuffix.length) + // Handle encrypted payload if ( attrName.endsWith(ciphertextAttrSuffix) && - (encryptedAttrs.includes( - attrName.slice(0, -ciphertextAttrSuffix.length), - ) || - isNested) + (encryptedAttrs.includes(columnName) || isNested) ) { - const baseName = attrName.slice(0, -ciphertextAttrSuffix.length) + // TODO: Implement the standardized typing for Encrypted Payloads through the FFI interface + const i = { c: columnName, t: encryptConfig.tableName } + const v = 2 + + // Nested values are not searchable, so we can just return the standard EQL payload. + // Worth noting, that encryptConfig.columns[columnName] will be undefined if isNested is true. + if ( + !isNested && + encryptConfig.columns[columnName].cast_as === 'jsonb' && + encryptConfig.columns[columnName].indexes.ste_vec + ) { + return { + [columnName]: { + i, + v, + k: 'sv', + sv: attrValue, + }, + } + } + return { - [baseName]: { + [columnName]: { + i, + v, + k: 'ct', c: attrValue, - bf: null, - hm: null, - i: { c: 'notUsed', t: 'notUsed' }, - k: 'notUsed', - ob: null, - v: 2, }, } } diff --git a/packages/protect-dynamodb/src/index.ts b/packages/protect-dynamodb/src/index.ts index 6cd02f0c..433cda6b 100644 --- a/packages/protect-dynamodb/src/index.ts +++ b/packages/protect-dynamodb/src/index.ts @@ -1,5 +1,5 @@ import type { - EncryptedPayload, + Encrypted, ProtectTable, ProtectTableColumn, SearchTerm, @@ -42,7 +42,7 @@ export function protectDynamoDB( }, decryptModel>( - item: Record, + item: Record, protectTable: ProtectTable, ) { return new DecryptModelOperation( @@ -54,7 +54,7 @@ export function protectDynamoDB( }, bulkDecryptModels>( - items: Record[], + items: Record[], protectTable: ProtectTable, ) { return new BulkDecryptModelsOperation( diff --git a/packages/protect-dynamodb/src/operations/bulk-decrypt-models.ts b/packages/protect-dynamodb/src/operations/bulk-decrypt-models.ts index 7bab00c0..e72e231c 100644 --- a/packages/protect-dynamodb/src/operations/bulk-decrypt-models.ts +++ b/packages/protect-dynamodb/src/operations/bulk-decrypt-models.ts @@ -1,7 +1,7 @@ import { type Result, withResult } from '@byteslice/result' import type { Decrypted, - EncryptedPayload, + Encrypted, ProtectClient, ProtectTable, ProtectTableColumn, @@ -17,12 +17,12 @@ export class BulkDecryptModelsOperation< T extends Record, > extends DynamoDBOperation[]> { private protectClient: ProtectClient - private items: Record[] + private items: Record[] private protectTable: ProtectTable constructor( protectClient: ProtectClient, - items: Record[], + items: Record[], protectTable: ProtectTable, options?: DynamoDBOperationOptions, ) { @@ -37,9 +37,8 @@ export class BulkDecryptModelsOperation< > { return await withResult( async () => { - const encryptedAttrs = Object.keys(this.protectTable.build().columns) const itemsWithEqlPayloads = this.items.map((item) => - toItemWithEqlPayloads(item, encryptedAttrs), + toItemWithEqlPayloads(item, this.protectTable), ) const decryptResult = await this.protectClient diff --git a/packages/protect-dynamodb/src/operations/decrypt-model.ts b/packages/protect-dynamodb/src/operations/decrypt-model.ts index 56ccf157..862e434d 100644 --- a/packages/protect-dynamodb/src/operations/decrypt-model.ts +++ b/packages/protect-dynamodb/src/operations/decrypt-model.ts @@ -1,7 +1,7 @@ import { type Result, withResult } from '@byteslice/result' import type { Decrypted, - EncryptedPayload, + Encrypted, ProtectClient, ProtectTable, ProtectTableColumn, @@ -17,12 +17,12 @@ export class DecryptModelOperation< T extends Record, > extends DynamoDBOperation> { private protectClient: ProtectClient - private item: Record + private item: Record private protectTable: ProtectTable constructor( protectClient: ProtectClient, - item: Record, + item: Record, protectTable: ProtectTable, options?: DynamoDBOperationOptions, ) { @@ -35,8 +35,10 @@ export class DecryptModelOperation< public async execute(): Promise, ProtectDynamoDBError>> { return await withResult( async () => { - const encryptedAttrs = Object.keys(this.protectTable.build().columns) - const withEqlPayloads = toItemWithEqlPayloads(this.item, encryptedAttrs) + const withEqlPayloads = toItemWithEqlPayloads( + this.item, + this.protectTable, + ) const decryptResult = await this.protectClient .decryptModel(withEqlPayloads as T) diff --git a/packages/protect-dynamodb/src/operations/search-terms.ts b/packages/protect-dynamodb/src/operations/search-terms.ts index 74b9032d..2bd6c246 100644 --- a/packages/protect-dynamodb/src/operations/search-terms.ts +++ b/packages/protect-dynamodb/src/operations/search-terms.ts @@ -39,6 +39,12 @@ export class SearchTermsOperation extends DynamoDBOperation { ) } + if (term?.k !== 'ct') { + throw new Error( + 'Tried to create search term with an invalid encrypted payload', + ) + } + if (!term?.hm) { throw new Error('expected encrypted search term to have an HMAC') } diff --git a/packages/protect-dynamodb/src/types.ts b/packages/protect-dynamodb/src/types.ts index a00ea1be..594d6404 100644 --- a/packages/protect-dynamodb/src/types.ts +++ b/packages/protect-dynamodb/src/types.ts @@ -1,7 +1,5 @@ -import type { Result } from '@byteslice/result' import type { - Decrypted, - EncryptedPayload, + Encrypted, ProtectClient, ProtectTable, ProtectTableColumn, @@ -40,12 +38,12 @@ export interface ProtectDynamoDBInstance { ): BulkEncryptModelsOperation decryptModel>( - item: Record, + item: Record, protectTable: ProtectTable, ): DecryptModelOperation bulkDecryptModels>( - items: Record[], + items: Record[], protectTable: ProtectTable, ): BulkDecryptModelsOperation diff --git a/packages/protect/__tests__/basic-protect.test.ts b/packages/protect/__tests__/basic-protect.test.ts new file mode 100644 index 00000000..6277a842 --- /dev/null +++ b/packages/protect/__tests__/basic-protect.test.ts @@ -0,0 +1,44 @@ +import 'dotenv/config' +import { csColumn, csTable } from '@cipherstash/schema' +import { beforeAll, describe, expect, it } from 'vitest' +import { protect } from '../src' + +const users = csTable('users', { + email: csColumn('email').freeTextSearch().equality().orderAndRange(), + address: csColumn('address').freeTextSearch(), + json: csColumn('json').dataType('json'), +}) + +let protectClient: Awaited> + +beforeAll(async () => { + protectClient = await protect({ + schemas: [users], + }) +}) + +describe('encryption and decryption', () => { + it('should encrypt and decrypt a payload', async () => { + const email = 'hello@example.com' + + const ciphertext = await protectClient.encrypt(email, { + column: users.email, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('c') + + const a = ciphertext.data + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: email, + }) + }, 30000) +}) diff --git a/packages/protect/__tests__/bulk-protect.test.ts b/packages/protect/__tests__/bulk-protect.test.ts index d0217317..893bea86 100644 --- a/packages/protect/__tests__/bulk-protect.test.ts +++ b/packages/protect/__tests__/bulk-protect.test.ts @@ -326,53 +326,6 @@ describe('bulk encryption and decryption', () => { expect(decryptedData.data).toHaveLength(0) }, 30000) - - it('should handle encrypted payloads with only "c" key', async () => { - // First encrypt some data - const plaintexts = [ - { id: 'user1', plaintext: 'alice@example.com' }, - { id: 'user2', plaintext: 'bob@example.com' }, - { id: 'user3', plaintext: 'charlie@example.com' }, - ] - - const encryptedData = await protectClient.bulkEncrypt(plaintexts, { - column: users.email, - table: users, - }) - - if (encryptedData.failure) { - throw new Error(`[protect]: ${encryptedData.failure.message}`) - } - - // Remove all keys except "c" from each encrypted payload - const modifiedEncryptedData = encryptedData.data.map((item) => ({ - id: item.id, - data: { - c: item.data?.c, - } as EncryptedPayload, - })) - - // Now decrypt the modified data - const decryptedData = await protectClient.bulkDecrypt( - modifiedEncryptedData, - ) - - if (decryptedData.failure) { - throw new Error(`[protect]: ${decryptedData.failure.message}`) - } - - // Verify structure - expect(decryptedData.data).toHaveLength(3) - expect(decryptedData.data[0]).toHaveProperty('id', 'user1') - expect(decryptedData.data[0]).toHaveProperty('data', 'alice@example.com') - expect(decryptedData.data[1]).toHaveProperty('id', 'user2') - expect(decryptedData.data[1]).toHaveProperty('data', 'bob@example.com') - expect(decryptedData.data[2]).toHaveProperty('id', 'user3') - expect(decryptedData.data[2]).toHaveProperty( - 'data', - 'charlie@example.com', - ) - }, 30000) }) describe('bulk operations with lock context', () => { diff --git a/packages/protect/__tests__/json-protect.test.ts b/packages/protect/__tests__/json-protect.test.ts new file mode 100644 index 00000000..24841d21 --- /dev/null +++ b/packages/protect/__tests__/json-protect.test.ts @@ -0,0 +1,1217 @@ +import 'dotenv/config' +import { csColumn, csTable, csValue } from '@cipherstash/schema' +import { beforeAll, describe, expect, it } from 'vitest' +import { LockContext, protect } from '../src' + +const users = csTable('users', { + email: csColumn('email').freeTextSearch().equality().orderAndRange(), + address: csColumn('address').freeTextSearch(), + json: csColumn('json').dataType('json'), + metadata: { + profile: csValue('metadata.profile').dataType('json'), + settings: { + preferences: csValue('metadata.settings.preferences').dataType('json'), + }, + }, +}) + +type User = { + id: string + email?: string | null + createdAt?: Date + updatedAt?: Date + address?: string | null + json?: Record | null + metadata?: { + profile?: Record | null + settings?: { + preferences?: Record | null + } + } +} + +let protectClient: Awaited> + +beforeAll(async () => { + protectClient = await protect({ + schemas: [users], + }) +}) + +describe('JSON encryption and decryption', () => { + it('should encrypt and decrypt a simple JSON payload', async () => { + const json = { + name: 'John Doe', + age: 30, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should encrypt and decrypt a complex JSON payload', async () => { + const json = { + user: { + id: 123, + name: 'Jane Smith', + email: 'jane@example.com', + preferences: { + theme: 'dark', + notifications: true, + language: 'en-US', + }, + tags: ['premium', 'verified'], + metadata: { + created: '2023-01-01T00:00:00Z', + lastLogin: '2023-12-01T10:30:00Z', + }, + }, + settings: { + privacy: { + public: false, + shareData: true, + }, + features: { + beta: true, + experimental: false, + }, + }, + array: [1, 2, 3, { nested: 'value' }], + nullValue: null, + booleanValue: true, + numberValue: 42.5, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should handle null JSON payload', async () => { + const ciphertext = await protectClient.encrypt(null, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify null is preserved + expect(ciphertext.data).toBeNull() + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: null, + }) + }, 30000) + + it('should handle empty JSON object', async () => { + const json = {} + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should handle JSON with special characters', async () => { + const json = { + message: 'Hello "world" with \'quotes\' and \\backslashes\\', + unicode: '🚀 emoji and ñ special chars', + symbols: '!@#$%^&*()_+-=[]{}|;:,.<>?/~`', + multiline: 'Line 1\nLine 2\tTabbed', + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) +}) + +describe('JSON model encryption and decryption', () => { + it('should encrypt and decrypt a model with JSON field', async () => { + const decryptedModel = { + id: '1', + email: 'test@example.com', + address: '123 Main St', + json: { + name: 'John Doe', + age: 30, + preferences: { + theme: 'dark', + notifications: true, + }, + }, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('k') + expect(encryptedModel.data.address).toHaveProperty('k') + expect(encryptedModel.data.json).toHaveProperty('k') + + // Verify non-encrypted fields remain unchanged + expect(encryptedModel.data.id).toBe('1') + expect(encryptedModel.data.createdAt).toEqual(new Date('2021-01-01')) + expect(encryptedModel.data.updatedAt).toEqual(new Date('2021-01-01')) + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should handle null JSON in model', async () => { + const decryptedModel = { + id: '2', + email: 'test2@example.com', + address: '456 Oak St', + json: null, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('k') + expect(encryptedModel.data.address).toHaveProperty('k') + expect(encryptedModel.data.json).toBeNull() + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should handle undefined JSON in model', async () => { + const decryptedModel = { + id: '3', + email: 'test3@example.com', + address: '789 Pine St', + json: undefined, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('k') + expect(encryptedModel.data.address).toHaveProperty('k') + expect(encryptedModel.data.json).toBeUndefined() + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) +}) + +describe('JSON bulk encryption and decryption', () => { + it('should bulk encrypt and decrypt JSON payloads', async () => { + const jsonPayloads = [ + { id: 'user1', plaintext: { name: 'Alice', age: 25 } }, + { id: 'user2', plaintext: { name: 'Bob', age: 30 } }, + { id: 'user3', plaintext: { name: 'Charlie', age: 35 } }, + ] + + const encryptedData = await protectClient.bulkEncrypt(jsonPayloads, { + column: users.json, + table: users, + }) + + if (encryptedData.failure) { + throw new Error(`[protect]: ${encryptedData.failure.message}`) + } + + // Verify structure + expect(encryptedData.data).toHaveLength(3) + expect(encryptedData.data[0]).toHaveProperty('id', 'user1') + expect(encryptedData.data[0]).toHaveProperty('data') + expect(encryptedData.data[0].data).toHaveProperty('k') + expect(encryptedData.data[1]).toHaveProperty('id', 'user2') + expect(encryptedData.data[1]).toHaveProperty('data') + expect(encryptedData.data[1].data).toHaveProperty('k') + expect(encryptedData.data[2]).toHaveProperty('id', 'user3') + expect(encryptedData.data[2]).toHaveProperty('data') + expect(encryptedData.data[2].data).toHaveProperty('k') + + // Now decrypt the data + const decryptedData = await protectClient.bulkDecrypt(encryptedData.data) + + if (decryptedData.failure) { + throw new Error(`[protect]: ${decryptedData.failure.message}`) + } + + // Verify decrypted data + expect(decryptedData.data).toHaveLength(3) + expect(decryptedData.data[0]).toHaveProperty('id', 'user1') + expect(decryptedData.data[0]).toHaveProperty('data', { + name: 'Alice', + age: 25, + }) + expect(decryptedData.data[1]).toHaveProperty('id', 'user2') + expect(decryptedData.data[1]).toHaveProperty('data', { + name: 'Bob', + age: 30, + }) + expect(decryptedData.data[2]).toHaveProperty('id', 'user3') + expect(decryptedData.data[2]).toHaveProperty('data', { + name: 'Charlie', + age: 35, + }) + }, 30000) + + it('should handle mixed null and non-null JSON in bulk operations', async () => { + const jsonPayloads = [ + { id: 'user1', plaintext: { name: 'Alice', age: 25 } }, + { id: 'user2', plaintext: null }, + { id: 'user3', plaintext: { name: 'Charlie', age: 35 } }, + ] + + const encryptedData = await protectClient.bulkEncrypt(jsonPayloads, { + column: users.json, + table: users, + }) + + if (encryptedData.failure) { + throw new Error(`[protect]: ${encryptedData.failure.message}`) + } + + // Verify structure + expect(encryptedData.data).toHaveLength(3) + expect(encryptedData.data[0]).toHaveProperty('id', 'user1') + expect(encryptedData.data[0]).toHaveProperty('data') + expect(encryptedData.data[0].data).toHaveProperty('k') + expect(encryptedData.data[1]).toHaveProperty('id', 'user2') + expect(encryptedData.data[1]).toHaveProperty('data') + expect(encryptedData.data[1].data).toBeNull() + expect(encryptedData.data[2]).toHaveProperty('id', 'user3') + expect(encryptedData.data[2]).toHaveProperty('data') + expect(encryptedData.data[2].data).toHaveProperty('k') + + // Now decrypt the data + const decryptedData = await protectClient.bulkDecrypt(encryptedData.data) + + if (decryptedData.failure) { + throw new Error(`[protect]: ${decryptedData.failure.message}`) + } + + // Verify decrypted data + expect(decryptedData.data).toHaveLength(3) + expect(decryptedData.data[0]).toHaveProperty('id', 'user1') + expect(decryptedData.data[0]).toHaveProperty('data', { + name: 'Alice', + age: 25, + }) + expect(decryptedData.data[1]).toHaveProperty('id', 'user2') + expect(decryptedData.data[1]).toHaveProperty('data', null) + expect(decryptedData.data[2]).toHaveProperty('id', 'user3') + expect(decryptedData.data[2]).toHaveProperty('data', { + name: 'Charlie', + age: 35, + }) + }, 30000) + + it('should bulk encrypt and decrypt models with JSON fields', async () => { + const decryptedModels = [ + { + id: '1', + email: 'test1@example.com', + address: '123 Main St', + json: { + name: 'Alice', + preferences: { theme: 'dark' }, + }, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + }, + { + id: '2', + email: 'test2@example.com', + address: '456 Oak St', + json: { + name: 'Bob', + preferences: { theme: 'light' }, + }, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + }, + ] + + const encryptedModels = await protectClient.bulkEncryptModels( + decryptedModels, + users, + ) + + if (encryptedModels.failure) { + throw new Error(`[protect]: ${encryptedModels.failure.message}`) + } + + // Verify encrypted fields for each model + expect(encryptedModels.data[0].email).toHaveProperty('k') + expect(encryptedModels.data[0].address).toHaveProperty('k') + expect(encryptedModels.data[0].json).toHaveProperty('k') + expect(encryptedModels.data[1].email).toHaveProperty('k') + expect(encryptedModels.data[1].address).toHaveProperty('k') + expect(encryptedModels.data[1].json).toHaveProperty('k') + + // Verify non-encrypted fields remain unchanged + expect(encryptedModels.data[0].id).toBe('1') + expect(encryptedModels.data[0].createdAt).toEqual(new Date('2021-01-01')) + expect(encryptedModels.data[0].updatedAt).toEqual(new Date('2021-01-01')) + expect(encryptedModels.data[1].id).toBe('2') + expect(encryptedModels.data[1].createdAt).toEqual(new Date('2021-01-01')) + expect(encryptedModels.data[1].updatedAt).toEqual(new Date('2021-01-01')) + + const decryptedResult = await protectClient.bulkDecryptModels( + encryptedModels.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModels) + }, 30000) +}) + +describe('JSON encryption with lock context', () => { + it('should encrypt and decrypt JSON with lock context', async () => { + const userJwt = process.env.USER_JWT + + if (!userJwt) { + console.log('Skipping lock context test - no USER_JWT provided') + return + } + + const lc = new LockContext() + const lockContext = await lc.identify(userJwt) + + if (lockContext.failure) { + throw new Error(`[protect]: ${lockContext.failure.message}`) + } + + const json = { + name: 'John Doe', + age: 30, + preferences: { + theme: 'dark', + notifications: true, + }, + } + + const ciphertext = await protectClient + .encrypt(json, { + column: users.json, + table: users, + }) + .withLockContext(lockContext.data) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient + .decrypt(ciphertext.data) + .withLockContext(lockContext.data) + + if (plaintext.failure) { + throw new Error(`[protect]: ${plaintext.failure.message}`) + } + + expect(plaintext.data).toEqual(json) + }, 30000) + + it('should encrypt JSON model with lock context', async () => { + const userJwt = process.env.USER_JWT + + if (!userJwt) { + console.log('Skipping lock context test - no USER_JWT provided') + return + } + + const lc = new LockContext() + const lockContext = await lc.identify(userJwt) + + if (lockContext.failure) { + throw new Error(`[protect]: ${lockContext.failure.message}`) + } + + const decryptedModel = { + id: '1', + email: 'test@example.com', + json: { + name: 'John Doe', + preferences: { theme: 'dark' }, + }, + } + + const encryptedModel = await protectClient + .encryptModel(decryptedModel, users) + .withLockContext(lockContext.data) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('k') + expect(encryptedModel.data.json).toHaveProperty('k') + + const decryptedResult = await protectClient + .decryptModel(encryptedModel.data) + .withLockContext(lockContext.data) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should bulk encrypt JSON with lock context', async () => { + const userJwt = process.env.USER_JWT + + if (!userJwt) { + console.log('Skipping lock context test - no USER_JWT provided') + return + } + + const lc = new LockContext() + const lockContext = await lc.identify(userJwt) + + if (lockContext.failure) { + throw new Error(`[protect]: ${lockContext.failure.message}`) + } + + const jsonPayloads = [ + { id: 'user1', plaintext: { name: 'Alice', age: 25 } }, + { id: 'user2', plaintext: { name: 'Bob', age: 30 } }, + ] + + const encryptedData = await protectClient + .bulkEncrypt(jsonPayloads, { + column: users.json, + table: users, + }) + .withLockContext(lockContext.data) + + if (encryptedData.failure) { + throw new Error(`[protect]: ${encryptedData.failure.message}`) + } + + // Verify structure + expect(encryptedData.data).toHaveLength(2) + expect(encryptedData.data[0]).toHaveProperty('id', 'user1') + expect(encryptedData.data[0]).toHaveProperty('data') + expect(encryptedData.data[0].data).toHaveProperty('k') + expect(encryptedData.data[1]).toHaveProperty('id', 'user2') + expect(encryptedData.data[1]).toHaveProperty('data') + expect(encryptedData.data[1].data).toHaveProperty('k') + + // Decrypt with lock context + const decryptedData = await protectClient + .bulkDecrypt(encryptedData.data) + .withLockContext(lockContext.data) + + if (decryptedData.failure) { + throw new Error(`[protect]: ${decryptedData.failure.message}`) + } + + // Verify decrypted data + expect(decryptedData.data).toHaveLength(2) + expect(decryptedData.data[0]).toHaveProperty('id', 'user1') + expect(decryptedData.data[0]).toHaveProperty('data', { + name: 'Alice', + age: 25, + }) + expect(decryptedData.data[1]).toHaveProperty('id', 'user2') + expect(decryptedData.data[1]).toHaveProperty('data', { + name: 'Bob', + age: 30, + }) + }, 30000) +}) + +describe('JSON nested object encryption', () => { + it('should encrypt and decrypt nested JSON objects', async () => { + const protectClient = await protect({ schemas: [users] }) + + const decryptedModel = { + id: '1', + email: 'test@example.com', + metadata: { + profile: { + name: 'John Doe', + age: 30, + preferences: { + theme: 'dark', + notifications: true, + }, + }, + settings: { + preferences: { + language: 'en-US', + timezone: 'UTC', + }, + }, + }, + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('k') + expect(encryptedModel.data.metadata?.profile).toHaveProperty('k') + expect(encryptedModel.data.metadata?.settings?.preferences).toHaveProperty( + 'c', + ) + + // Verify non-encrypted fields remain unchanged + expect(encryptedModel.data.id).toBe('1') + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should handle null values in nested JSON objects', async () => { + const protectClient = await protect({ schemas: [users] }) + + const decryptedModel = { + id: '2', + email: 'test2@example.com', + metadata: { + profile: null, + settings: { + preferences: null, + }, + }, + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify null fields are preserved + expect(encryptedModel.data.email).toHaveProperty('k') + expect(encryptedModel.data.metadata?.profile).toBeNull() + expect(encryptedModel.data.metadata?.settings?.preferences).toBeNull() + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should handle undefined values in nested JSON objects', async () => { + const protectClient = await protect({ schemas: [users] }) + + const decryptedModel = { + id: '3', + email: 'test3@example.com', + metadata: { + profile: undefined, + settings: { + preferences: undefined, + }, + }, + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify undefined fields are preserved + expect(encryptedModel.data.email).toHaveProperty('k') + expect(encryptedModel.data.metadata?.profile).toBeUndefined() + expect(encryptedModel.data.metadata?.settings?.preferences).toBeUndefined() + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) +}) + +describe('JSON edge cases and error handling', () => { + it('should handle very large JSON objects', async () => { + const largeJson = { + data: Array.from({ length: 1000 }, (_, i) => ({ + id: i, + name: `User ${i}`, + email: `user${i}@example.com`, + metadata: { + preferences: { + theme: i % 2 === 0 ? 'dark' : 'light', + notifications: i % 3 === 0, + }, + }, + })), + metadata: { + total: 1000, + created: new Date().toISOString(), + }, + } + + const ciphertext = await protectClient.encrypt(largeJson, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: largeJson, + }) + }, 30000) + + it('should handle JSON with circular references (should fail gracefully)', async () => { + const circularObj: Record = { name: 'test' } + circularObj.self = circularObj + + try { + await protectClient.encrypt(circularObj, { + column: users.json, + table: users, + }) + // This should not reach here as JSON.stringify should fail + expect(true).toBe(false) + } catch (error) { + // Expected to fail due to circular reference + expect(error).toBeDefined() + } + }, 30000) + + it('should handle JSON with special data types', async () => { + const json = { + string: 'hello', + number: 42, + boolean: true, + null: null, + array: [1, 2, 3], + object: { nested: 'value' }, + date: new Date('2023-01-01T00:00:00Z'), + // Note: Functions and undefined are not JSON serializable + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + // Date objects get serialized to strings in JSON + const expectedJson = { + ...json, + date: '2023-01-01T00:00:00.000Z', + } + + expect(plaintext).toEqual({ + data: expectedJson, + }) + }, 30000) +}) + +describe('JSON performance tests', () => { + it('should handle large numbers of JSON objects efficiently', async () => { + const largeJsonArray = Array.from({ length: 100 }, (_, i) => ({ + id: i, + data: { + name: `User ${i}`, + preferences: { + theme: i % 2 === 0 ? 'dark' : 'light', + notifications: i % 3 === 0, + }, + metadata: { + created: new Date().toISOString(), + version: i, + }, + }, + })) + + const jsonPayloads = largeJsonArray.map((item, index) => ({ + id: `user${index}`, + plaintext: item, + })) + + const encryptedData = await protectClient.bulkEncrypt(jsonPayloads, { + column: users.json, + table: users, + }) + + if (encryptedData.failure) { + throw new Error(`[protect]: ${encryptedData.failure.message}`) + } + + // Verify structure + expect(encryptedData.data).toHaveLength(100) + + // Decrypt the data + const decryptedData = await protectClient.bulkDecrypt(encryptedData.data) + + if (decryptedData.failure) { + throw new Error(`[protect]: ${decryptedData.failure.message}`) + } + + // Verify all data is preserved + expect(decryptedData.data).toHaveLength(100) + + for (let i = 0; i < 100; i++) { + expect(decryptedData.data[i].id).toBe(`user${i}`) + expect(decryptedData.data[i].data).toEqual(largeJsonArray[i]) + } + }, 5000) +}) + +describe('JSON advanced scenarios', () => { + it('should handle JSON with deeply nested arrays', async () => { + const json = { + users: [ + { + id: 1, + name: 'Alice', + roles: [ + { name: 'admin', permissions: ['read', 'write', 'delete'] }, + { name: 'user', permissions: ['read'] }, + ], + }, + { + id: 2, + name: 'Bob', + roles: [{ name: 'user', permissions: ['read'] }], + }, + ], + metadata: { + total: 2, + lastUpdated: new Date().toISOString(), + }, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should handle JSON with mixed data types in arrays', async () => { + const json = { + mixedArray: ['string', 42, true, null, { nested: 'object' }, [1, 2, 3]], + metadata: { + types: ['string', 'number', 'boolean', 'null', 'object', 'array'], + }, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should handle JSON with Unicode and international characters', async () => { + const json = { + international: { + chinese: '你好世界', + japanese: 'こんにちは世界', + korean: '안녕하세요 세계', + arabic: 'مرحبا بالعالم', + russian: 'Привет мир', + emoji: '🚀 🌍 💻 🎉', + }, + metadata: { + languages: ['Chinese', 'Japanese', 'Korean', 'Arabic', 'Russian'], + encoding: 'UTF-8', + }, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should handle JSON with scientific notation and large numbers', async () => { + const json = { + numbers: { + integer: 1234567890, + float: Math.PI, + scientific: 1.23e10, + negative: -9876543210, + zero: 0, + verySmall: 1.23e-10, + }, + metadata: { + precision: 'high', + format: 'scientific', + }, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should handle JSON with boolean edge cases', async () => { + const json = { + booleans: { + true: true, + false: false, + stringTrue: 'true', + stringFalse: 'false', + numberOne: 1, + numberZero: 0, + emptyString: '', + nullValue: null, + }, + metadata: { + type: 'boolean_edge_cases', + }, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) +}) + +describe('JSON error handling and edge cases', () => { + it('should handle malformed JSON gracefully', async () => { + // This test ensures the library handles JSON serialization errors + const invalidJson = { + valid: 'data', + // This will cause JSON.stringify to fail + circular: null as unknown, + } + + // Create a circular reference + invalidJson.circular = invalidJson + + try { + await protectClient.encrypt(invalidJson, { + column: users.json, + table: users, + }) + expect(true).toBe(false) // Should not reach here + } catch (error) { + expect(error).toBeDefined() + expect(error).toBeInstanceOf(Error) + } + }, 30000) + + it('should handle empty arrays and objects', async () => { + const json = { + emptyArray: [], + emptyObject: {}, + nestedEmpty: { + array: [], + object: {}, + }, + mixedEmpty: { + data: 'present', + empty: [], + null: null, + }, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should handle JSON with very long strings', async () => { + const longString = 'A'.repeat(10000) // 10KB string + const json = { + longString, + metadata: { + length: longString.length, + type: 'long_string', + }, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) + + it('should handle JSON with all primitive types', async () => { + const json = { + string: 'hello world', + number: 42, + float: 3.14, + boolean: true, + null: null, + array: [1, 2, 3], + object: { key: 'value' }, + nested: { + level1: { + level2: { + level3: 'deep value', + }, + }, + }, + } + + const ciphertext = await protectClient.encrypt(json, { + column: users.json, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('k') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: json, + }) + }, 30000) +}) diff --git a/packages/protect/__tests__/number-protect.test.ts b/packages/protect/__tests__/number-protect.test.ts new file mode 100644 index 00000000..8a417c8d --- /dev/null +++ b/packages/protect/__tests__/number-protect.test.ts @@ -0,0 +1,877 @@ +import 'dotenv/config' +import { csColumn, csTable, csValue } from '@cipherstash/schema' +import { beforeAll, describe, expect, it, test } from 'vitest' +import { LockContext, protect } from '../src' + +const users = csTable('users', { + email: csColumn('email').freeTextSearch().equality().orderAndRange(), + address: csColumn('address').freeTextSearch(), + age: csColumn('age').dataType('number').equality().orderAndRange(), + score: csColumn('score').dataType('number').equality().orderAndRange(), + metadata: { + count: csValue('metadata.count').dataType('number'), + level: csValue('metadata.level').dataType('number'), + }, +}) + +type User = { + id: string + email?: string + createdAt?: Date + updatedAt?: Date + address?: string + age?: number + score?: number + metadata?: { + count?: number + level?: number + } +} + +let protectClient: Awaited> + +beforeAll(async () => { + protectClient = await protect({ + schemas: [users], + }) +}) + +const cases = [ + 25, 0, -42, 2147483647, + 77.9, 0.0, -117.123456, + 1e15, -1e15, // Very large floats + 9007199254740991 // Max safe integer in JavaScript +] + +describe('Number encryption and decryption', () => { + test.each(cases)('should encrypt and decrypt a number: %d', async (age) => { + const ciphertext = await protectClient.encrypt(age, { + column: users.age, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('c') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: age, + }) + }, 30000) + + it('should handle null integer', async () => { + const ciphertext = await protectClient.encrypt(null, { + column: users.age, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify null is preserved + expect(ciphertext.data).toBeNull() + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: null, + }) + }, 30000) + + // Special case + it('should treat a negative zero valued float as 0.0', async () => { + const score = -0.0 + + const ciphertext = await protectClient.encrypt(score, { + column: users.score, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('c') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: 0.0, + }) + }, 30000) + + // Special case + it('should error for a NaN float', async () => { + const score = NaN + + const result = await protectClient.encrypt(score, { + column: users.score, + table: users, + }); + + expect(result.failure).toBeDefined() + expect(result.failure?.message).toContain('Cannot encrypt NaN value') + }, 30000) + + // Special case + it('should error for Infinity', async () => { + const score = Infinity + + const result = await protectClient.encrypt(score, { + column: users.score, + table: users, + }) + + expect(result.failure).toBeDefined() + expect(result.failure?.message).toContain('Cannot encrypt Infinity value') + }, 30000) + + // Special case + it('should error for -Infinity', async () => { + const score = -Infinity + + const result = await protectClient.encrypt(score, { + column: users.score, + table: users, + }) + + expect(result.failure).toBeDefined() + expect(result.failure?.message).toContain('Cannot encrypt Infinity value') + }, 30000) +}) + +describe('Model encryption and decryption', () => { + it('should encrypt and decrypt a model with number fields', async () => { + const decryptedModel = { + id: '1', + email: 'test@example.com', + address: '123 Main St', + age: 30, + score: 95, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('c') + expect(encryptedModel.data.address).toHaveProperty('c') + expect(encryptedModel.data.age).toHaveProperty('c') + expect(encryptedModel.data.score).toHaveProperty('c') + + // Verify non-encrypted fields remain unchanged + expect(encryptedModel.data.id).toBe('1') + expect(encryptedModel.data.createdAt).toEqual(new Date('2021-01-01')) + expect(encryptedModel.data.updatedAt).toEqual(new Date('2021-01-01')) + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should handle null numbers in model', async () => { + const decryptedModel: User = { + id: '2', + email: 'test2@example.com', + address: '456 Oak St', + age: undefined, + score: undefined, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('c') + expect(encryptedModel.data.address).toHaveProperty('c') + expect(encryptedModel.data.age).toBeUndefined() + expect(encryptedModel.data.score).toBeUndefined() + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should handle undefined numbers in model', async () => { + const decryptedModel = { + id: '3', + email: 'test3@example.com', + address: '789 Pine St', + age: undefined, + score: undefined, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('c') + expect(encryptedModel.data.address).toHaveProperty('c') + expect(encryptedModel.data.age).toBeUndefined() + expect(encryptedModel.data.score).toBeUndefined() + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) +}) + +describe('Bulk encryption and decryption', () => { + it('should bulk encrypt and decrypt number payloads', async () => { + const intPayloads = [ + { id: 'user1', plaintext: 25 }, + { id: 'user2', plaintext: 30.7 }, + { id: 'user3', plaintext: -35.123 }, + ] + + const encryptedData = await protectClient.bulkEncrypt(intPayloads, { + column: users.age, + table: users, + }) + + if (encryptedData.failure) { + throw new Error(`[protect]: ${encryptedData.failure.message}`) + } + + // Verify structure + expect(encryptedData.data).toHaveLength(3) + expect(encryptedData.data[0]).toHaveProperty('id', 'user1') + expect(encryptedData.data[0]).toHaveProperty('data') + expect(encryptedData.data[0].data).toHaveProperty('c') + expect(encryptedData.data[1]).toHaveProperty('id', 'user2') + expect(encryptedData.data[1]).toHaveProperty('data') + expect(encryptedData.data[1].data).toHaveProperty('c') + expect(encryptedData.data[2]).toHaveProperty('id', 'user3') + expect(encryptedData.data[2]).toHaveProperty('data') + expect(encryptedData.data[2].data).toHaveProperty('c') + + expect(encryptedData.data[0].data?.k).toBe('ct') + expect(encryptedData.data[1].data?.k).toBe('ct') + expect(encryptedData.data[2].data?.k).toBe('ct') + + // Verify all encrypted values are different + const getCiphertext = (data: any) => { + if (data?.k === 'ct') return data.c + return data?.c + } + + expect(getCiphertext(encryptedData.data[0].data)).not.toBe( + getCiphertext(encryptedData.data[1].data), + ) + expect(getCiphertext(encryptedData.data[1].data)).not.toBe( + getCiphertext(encryptedData.data[2].data), + ) + expect(getCiphertext(encryptedData.data[0].data)).not.toBe( + getCiphertext(encryptedData.data[2].data), + ) + + // Now decrypt the data + const decryptedData = await protectClient.bulkDecrypt(encryptedData.data) + + if (decryptedData.failure) { + throw new Error(`[protect]: ${decryptedData.failure.message}`) + } + + // Verify decrypted data + expect(decryptedData.data).toHaveLength(3) + expect(decryptedData.data[0]).toHaveProperty('id', 'user1') + expect(decryptedData.data[0]).toHaveProperty('data', 25) + expect(decryptedData.data[1]).toHaveProperty('id', 'user2') + expect(decryptedData.data[1]).toHaveProperty('data', 30.7) + expect(decryptedData.data[2]).toHaveProperty('id', 'user3') + expect(decryptedData.data[2]).toHaveProperty('data', -35.123) + }, 30000) + + it('should handle mixed null and non-null numbers in bulk operations', async () => { + const intPayloads = [ + { id: 'user1', plaintext: 25 }, + { id: 'user2', plaintext: null }, + { id: 'user3', plaintext: 35 }, + ] + + const encryptedData = await protectClient.bulkEncrypt(intPayloads, { + column: users.age, + table: users, + }) + + if (encryptedData.failure) { + throw new Error(`[protect]: ${encryptedData.failure.message}`) + } + + // Verify structure + expect(encryptedData.data).toHaveLength(3) + expect(encryptedData.data[0]).toHaveProperty('id', 'user1') + expect(encryptedData.data[0]).toHaveProperty('data') + expect(encryptedData.data[0].data).toHaveProperty('c') + expect(encryptedData.data[1]).toHaveProperty('id', 'user2') + expect(encryptedData.data[1]).toHaveProperty('data') + expect(encryptedData.data[1].data).toBeNull() + expect(encryptedData.data[2]).toHaveProperty('id', 'user3') + expect(encryptedData.data[2]).toHaveProperty('data') + expect(encryptedData.data[2].data).toHaveProperty('c') + + // Now decrypt the data + const decryptedData = await protectClient.bulkDecrypt(encryptedData.data) + + if (decryptedData.failure) { + throw new Error(`[protect]: ${decryptedData.failure.message}`) + } + + // Verify decrypted data + expect(decryptedData.data).toHaveLength(3) + expect(decryptedData.data[0]).toHaveProperty('id', 'user1') + expect(decryptedData.data[0]).toHaveProperty('data', 25) + expect(decryptedData.data[1]).toHaveProperty('id', 'user2') + expect(decryptedData.data[1]).toHaveProperty('data', null) + expect(decryptedData.data[2]).toHaveProperty('id', 'user3') + expect(decryptedData.data[2]).toHaveProperty('data', 35) + }, 30000) + + it('should bulk encrypt and decrypt models with number fields', async () => { + const decryptedModels = [ + { + id: '1', + email: 'test1@example.com', + address: '123 Main St', + age: 25, + score: 85, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + }, + { + id: '2', + email: 'test2@example.com', + address: '456 Oak St', + age: 30, + score: 90, + createdAt: new Date('2021-01-01'), + updatedAt: new Date('2021-01-01'), + }, + ] + + const encryptedModels = await protectClient.bulkEncryptModels( + decryptedModels, + users, + ) + + if (encryptedModels.failure) { + throw new Error(`[protect]: ${encryptedModels.failure.message}`) + } + + // Verify encrypted fields for each model + expect(encryptedModels.data[0].email).toHaveProperty('c') + expect(encryptedModels.data[0].address).toHaveProperty('c') + expect(encryptedModels.data[0].age).toHaveProperty('c') + expect(encryptedModels.data[0].score).toHaveProperty('c') + expect(encryptedModels.data[1].email).toHaveProperty('c') + expect(encryptedModels.data[1].address).toHaveProperty('c') + expect(encryptedModels.data[1].age).toHaveProperty('c') + expect(encryptedModels.data[1].score).toHaveProperty('c') + + // Verify non-encrypted fields remain unchanged + expect(encryptedModels.data[0].id).toBe('1') + expect(encryptedModels.data[0].createdAt).toEqual(new Date('2021-01-01')) + expect(encryptedModels.data[0].updatedAt).toEqual(new Date('2021-01-01')) + expect(encryptedModels.data[1].id).toBe('2') + expect(encryptedModels.data[1].createdAt).toEqual(new Date('2021-01-01')) + expect(encryptedModels.data[1].updatedAt).toEqual(new Date('2021-01-01')) + + const decryptedResult = await protectClient.bulkDecryptModels( + encryptedModels.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModels) + }, 30000) +}) + +describe('Encryption with lock context', () => { + it('should encrypt and decrypt number with lock context', async () => { + const userJwt = process.env.USER_JWT + + if (!userJwt) { + console.log('Skipping lock context test - no USER_JWT provided') + return + } + + const lc = new LockContext() + const lockContext = await lc.identify(userJwt) + + if (lockContext.failure) { + throw new Error(`[protect]: ${lockContext.failure.message}`) + } + + const age = 42 + + const ciphertext = await protectClient + .encrypt(age, { + column: users.age, + table: users, + }) + .withLockContext(lockContext.data) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('c') + + const plaintext = await protectClient + .decrypt(ciphertext.data) + .withLockContext(lockContext.data) + + if (plaintext.failure) { + throw new Error(`[protect]: ${plaintext.failure.message}`) + } + + expect(plaintext.data).toEqual(age) + }, 30000) + + it('should encrypt model with lock context', async () => { + const userJwt = process.env.USER_JWT + + if (!userJwt) { + console.log('Skipping lock context test - no USER_JWT provided') + return + } + + const lc = new LockContext() + const lockContext = await lc.identify(userJwt) + + if (lockContext.failure) { + throw new Error(`[protect]: ${lockContext.failure.message}`) + } + + const decryptedModel = { + id: '1', + email: 'test@example.com', + age: 30, + score: 95, + } + + const encryptedModel = await protectClient + .encryptModel(decryptedModel, users) + .withLockContext(lockContext.data) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('c') + expect(encryptedModel.data.age).toHaveProperty('c') + expect(encryptedModel.data.score).toHaveProperty('c') + + const decryptedResult = await protectClient + .decryptModel(encryptedModel.data) + .withLockContext(lockContext.data) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should bulk encrypt numbers with lock context', async () => { + const userJwt = process.env.USER_JWT + + if (!userJwt) { + console.log('Skipping lock context test - no USER_JWT provided') + return + } + + const lc = new LockContext() + const lockContext = await lc.identify(userJwt) + + if (lockContext.failure) { + throw new Error(`[protect]: ${lockContext.failure.message}`) + } + + const intPayloads = [ + { id: 'user1', plaintext: 25 }, + { id: 'user2', plaintext: 30 }, + ] + + const encryptedData = await protectClient + .bulkEncrypt(intPayloads, { + column: users.age, + table: users, + }) + .withLockContext(lockContext.data) + + if (encryptedData.failure) { + throw new Error(`[protect]: ${encryptedData.failure.message}`) + } + + // Verify structure + expect(encryptedData.data).toHaveLength(2) + expect(encryptedData.data[0]).toHaveProperty('id', 'user1') + expect(encryptedData.data[0]).toHaveProperty('data') + expect(encryptedData.data[0].data).toHaveProperty('c') + expect(encryptedData.data[1]).toHaveProperty('id', 'user2') + expect(encryptedData.data[1]).toHaveProperty('data') + expect(encryptedData.data[1].data).toHaveProperty('c') + + // Decrypt with lock context + const decryptedData = await protectClient + .bulkDecrypt(encryptedData.data) + .withLockContext(lockContext.data) + + if (decryptedData.failure) { + throw new Error(`[protect]: ${decryptedData.failure.message}`) + } + + // Verify decrypted data + expect(decryptedData.data).toHaveLength(2) + expect(decryptedData.data[0]).toHaveProperty('id', 'user1') + expect(decryptedData.data[0]).toHaveProperty('data', 25) + expect(decryptedData.data[1]).toHaveProperty('id', 'user2') + expect(decryptedData.data[1]).toHaveProperty('data', 30) + }, 30000) +}) + +describe('Nested object encryption', () => { + it('should encrypt and decrypt nested number objects', async () => { + const protectClient = await protect({ schemas: [users] }) + + const decryptedModel = { + id: '1', + email: 'test@example.com', + metadata: { + count: 100, + level: 5, + }, + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify encrypted fields + expect(encryptedModel.data.email).toHaveProperty('c') + expect(encryptedModel.data.metadata?.count).toHaveProperty('c') + expect(encryptedModel.data.metadata?.level).toHaveProperty('c') + + // Verify non-encrypted fields remain unchanged + expect(encryptedModel.data.id).toBe('1') + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should handle null values in nested objects with number fields', async () => { + const protectClient = await protect({ schemas: [users] }) + + const decryptedModel: User = { + id: '2', + email: 'test2@example.com', + metadata: { + count: undefined, + level: undefined, + }, + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify null fields are preserved + expect(encryptedModel.data.email).toHaveProperty('c') + expect(encryptedModel.data.metadata?.count).toBeUndefined() + expect(encryptedModel.data.metadata?.level).toBeUndefined() + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) + + it('should handle undefined values in nested objects with number fields', async () => { + const protectClient = await protect({ schemas: [users] }) + + const decryptedModel = { + id: '3', + email: 'test3@example.com', + metadata: { + count: undefined, + level: undefined, + }, + } + + const encryptedModel = await protectClient.encryptModel( + decryptedModel, + users, + ) + + if (encryptedModel.failure) { + throw new Error(`[protect]: ${encryptedModel.failure.message}`) + } + + // Verify undefined fields are preserved + expect(encryptedModel.data.email).toHaveProperty('c') + expect(encryptedModel.data.metadata?.count).toBeUndefined() + expect(encryptedModel.data.metadata?.level).toBeUndefined() + + const decryptedResult = await protectClient.decryptModel( + encryptedModel.data, + ) + + if (decryptedResult.failure) { + throw new Error(`[protect]: ${decryptedResult.failure.message}`) + } + + expect(decryptedResult.data).toEqual(decryptedModel) + }, 30000) +}) + +describe('Search terms', () => { + it('should create search terms for number fields', async () => { + const searchTerms = [ + { + value: 25, + column: users.age, + table: users, + }, + { + value: 100, + column: users.score, + table: users, + }, + ] + + const searchTermsResult = await protectClient.createSearchTerms(searchTerms) + + if (searchTermsResult.failure) { + throw new Error(`[protect]: ${searchTermsResult.failure.message}`) + } + + expect(searchTermsResult.data).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + c: expect.any(String), + }), + ]), + ) + }, 30000) + + it('should create search terms with composite-literal return type for numbers', async () => { + const searchTerms = [ + { + value: 42, + column: users.age, + table: users, + returnType: 'composite-literal' as const, + }, + ] + + const searchTermsResult = await protectClient.createSearchTerms(searchTerms) + + if (searchTermsResult.failure) { + throw new Error(`[protect]: ${searchTermsResult.failure.message}`) + } + + const result = searchTermsResult.data[0] as string + expect(result).toMatch(/^\(.*\)$/) + expect(() => JSON.parse(result.slice(1, -1))).not.toThrow() + }, 30000) + + it('should create search terms with escaped-composite-literal return type for numbers', async () => { + const searchTerms = [ + { + value: 99, + column: users.score, + table: users, + returnType: 'escaped-composite-literal' as const, + }, + ] + + const searchTermsResult = await protectClient.createSearchTerms(searchTerms) + + if (searchTermsResult.failure) { + throw new Error(`[protect]: ${searchTermsResult.failure.message}`) + } + + const result = searchTermsResult.data[0] as string + expect(result).toMatch(/^".*"$/) + const unescaped = JSON.parse(result) + expect(unescaped).toMatch(/^\(.*\)$/) + expect(() => JSON.parse(unescaped.slice(1, -1))).not.toThrow() + }, 30000) +}) + +describe('Performance tests', () => { + it('should handle large numbers of numbers efficiently', async () => { + const largeNumArray = Array.from({ length: 100 }, (_, i) => ({ + id: i, + data: { + age: i + 18, // Ages 18-117 + score: (i % 100) + 1, // Scores 1-100 + }, + })) + + const numPayloads = largeNumArray.map((item, index) => ({ + id: `user${index}`, + plaintext: item.data.age, + })) + + const encryptedData = await protectClient.bulkEncrypt(numPayloads, { + column: users.age, + table: users, + }) + + if (encryptedData.failure) { + throw new Error(`[protect]: ${encryptedData.failure.message}`) + } + + // Verify structure + expect(encryptedData.data).toHaveLength(100) + + // Decrypt the data + const decryptedData = await protectClient.bulkDecrypt(encryptedData.data) + + if (decryptedData.failure) { + throw new Error(`[protect]: ${decryptedData.failure.message}`) + } + + // Verify all data is preserved + expect(decryptedData.data).toHaveLength(100) + + for (let i = 0; i < 100; i++) { + expect(decryptedData.data[i].id).toBe(`user${i}`) + expect(decryptedData.data[i].data).toEqual(largeNumArray[i].data.age) + } + }, 60000) +}) + +describe('Advanced scenarios', () => { + it('should handle boundary values', async () => { + const boundaryValues = [ + Number.MIN_SAFE_INTEGER, + -2147483648, // Min 32-bit signed integer + -1, + 0, + 1, + 2147483647, // Max 32-bit signed integer + Number.MAX_SAFE_INTEGER, + ] + + for (const value of boundaryValues) { + const ciphertext = await protectClient.encrypt(value, { + column: users.age, + table: users, + }) + + if (ciphertext.failure) { + throw new Error(`[protect]: ${ciphertext.failure.message}`) + } + + // Verify encrypted field + expect(ciphertext.data).toHaveProperty('c') + + const plaintext = await protectClient.decrypt(ciphertext.data) + + expect(plaintext).toEqual({ + data: value, + }) + } + }, 30000) +}) + +const invalidPlaintexts = [ + '400', + 'aaa', + '100a', + '73.51', + {}, + [], + [123], + { num: 123 }, +]; + +describe('Invalid or uncoercable values', () => { + test.each(invalidPlaintexts)('should fail to encrypt', async (input) => { + const result = await protectClient.encrypt(input, { + column: users.age, + table: users, + }) + + expect(result.failure).toBeDefined() + expect(result.failure?.message).toContain('Unsupported conversion') + }, 30000) +}) diff --git a/packages/protect/__tests__/protect.test.ts b/packages/protect/__tests__/protect-ops.test.ts similarity index 97% rename from packages/protect/__tests__/protect.test.ts rename to packages/protect/__tests__/protect-ops.test.ts index 8a7abfab..c7a2e276 100644 --- a/packages/protect/__tests__/protect.test.ts +++ b/packages/protect/__tests__/protect-ops.test.ts @@ -25,29 +25,7 @@ beforeAll(async () => { }) }) -describe('encryption and decryption', () => { - it('should encrypt and decrypt a payload', async () => { - const email = 'hello@example.com' - - const ciphertext = await protectClient.encrypt(email, { - column: users.email, - table: users, - }) - - if (ciphertext.failure) { - throw new Error(`[protect]: ${ciphertext.failure.message}`) - } - - // Verify encrypted field - expect(ciphertext.data).toHaveProperty('c') - - const plaintext = await protectClient.decrypt(ciphertext.data) - - expect(plaintext).toEqual({ - data: email, - }) - }, 30000) - +describe('encryption and decryption edge cases', () => { it('should return null if plaintext is null', async () => { const ciphertext = await protectClient.encrypt(null, { column: users.email, diff --git a/packages/protect/__tests__/supabase.test.ts b/packages/protect/__tests__/supabase.test.ts index 45a1526c..74683085 100644 --- a/packages/protect/__tests__/supabase.test.ts +++ b/packages/protect/__tests__/supabase.test.ts @@ -2,7 +2,7 @@ import 'dotenv/config' import { csColumn, csTable } from '@cipherstash/schema' import { describe, expect, it } from 'vitest' import { - type EncryptedPayload, + type Encrypted, bulkModelsToEncryptedPgComposites, encryptedToPgComposite, isEncryptedPayload, @@ -61,7 +61,7 @@ describe('supabase', () => { throw new Error(`[protect]: ${error.message}`) } - const dataToDecrypt = data[0].encrypted as EncryptedPayload + const dataToDecrypt = data[0].encrypted as Encrypted const plaintext = await protectClient.decrypt(dataToDecrypt) await supabase.from('protect-ci').delete().eq('id', insertedData[0].id) diff --git a/packages/protect/package.json b/packages/protect/package.json index 3af089db..4480b032 100644 --- a/packages/protect/package.json +++ b/packages/protect/package.json @@ -55,11 +55,11 @@ }, "dependencies": { "@byteslice/result": "^0.2.0", - "@cipherstash/protect-ffi": "0.16.0", + "@cipherstash/protect-ffi": "0.17.1", "@cipherstash/schema": "workspace:*", "zod": "^3.24.2" }, "optionalDependencies": { "@rollup/rollup-linux-x64-gnu": "4.24.0" } -} +} \ No newline at end of file diff --git a/packages/protect/src/ffi/index.ts b/packages/protect/src/ffi/index.ts index c5ea50dc..bacd41c3 100644 --- a/packages/protect/src/ffi/index.ts +++ b/packages/protect/src/ffi/index.ts @@ -1,5 +1,5 @@ import { type Result, withResult } from '@byteslice/result' -import { newClient } from '@cipherstash/protect-ffi' +import { type JsPlaintext, newClient } from '@cipherstash/protect-ffi' import { type EncryptConfig, type ProtectTable, @@ -15,8 +15,7 @@ import type { Client, Decrypted, EncryptOptions, - EncryptPayload, - EncryptedPayload, + Encrypted, SearchTerm, } from '../types' import { BulkDecryptOperation } from './operations/bulk-decrypt' @@ -92,7 +91,10 @@ export class ProtectClient { * await eqlClient.encrypt(plaintext, { column, table }) * await eqlClient.encrypt(plaintext, { column, table }).withLockContext(lockContext) */ - encrypt(plaintext: EncryptPayload, opts: EncryptOptions): EncryptOperation { + encrypt( + plaintext: JsPlaintext | null, + opts: EncryptOptions, + ): EncryptOperation { return new EncryptOperation(this.client, plaintext, opts) } @@ -102,7 +104,7 @@ export class ProtectClient { * await eqlClient.decrypt(encryptedData) * await eqlClient.decrypt(encryptedData).withLockContext(lockContext) */ - decrypt(encryptedData: EncryptedPayload): DecryptOperation { + decrypt(encryptedData: Encrypted): DecryptOperation { return new DecryptOperation(this.client, encryptedData) } diff --git a/packages/protect/src/ffi/model-helpers.ts b/packages/protect/src/ffi/model-helpers.ts index 3003a845..7cde0dd6 100644 --- a/packages/protect/src/ffi/model-helpers.ts +++ b/packages/protect/src/ffi/model-helpers.ts @@ -1,12 +1,14 @@ import { - type Encrypted, + type Encrypted as CipherStashEncrypted, + type DecryptBulkOptions, + type JsPlaintext, decryptBulk, encryptBulk, } from '@cipherstash/protect-ffi' import type { ProtectTable, ProtectTableColumn } from '@cipherstash/schema' import { isEncryptedPayload } from '../helpers' import type { GetLockContextResponse } from '../identify' -import type { Client, Decrypted, EncryptedPayload } from '../types' +import type { Client, Decrypted, Encrypted } from '../types' import type { AuditData } from './operations/base-operation' /** @@ -14,8 +16,8 @@ import type { AuditData } from './operations/base-operation' */ export function extractEncryptedFields>( model: T, -): Record { - const result: Record = {} +): Record { + const result: Record = {} for (const [key, value] of Object.entries(model)) { if (isEncryptedPayload(value)) { @@ -48,7 +50,7 @@ export function extractOtherFields>( */ export function mergeFields( otherFields: Record, - encryptedFields: Record, + encryptedFields: Record, ): T { return { ...otherFields, ...encryptedFields } as T } @@ -201,7 +203,11 @@ function prepareFieldsForEncryption>( continue } - if (typeof value === 'object' && !isEncryptedPayload(value)) { + if ( + typeof value === 'object' && + !isEncryptedPayload(value) && + !columnPaths.includes(fullKey) + ) { // Only process nested objects if they're in the schema if (columnPaths.some((path) => path.startsWith(fullKey))) { processNestedFields( @@ -253,7 +259,7 @@ export async function decryptModelFields>( const bulkDecryptPayload = Object.entries(operationFields).map( ([key, value]) => ({ id: key, - ciphertext: (value as EncryptedPayload)?.c ?? '', + ciphertext: value as CipherStashEncrypted, }), ) @@ -397,7 +403,7 @@ export async function decryptModelFieldsWithLockContext< const bulkDecryptPayload = Object.entries(operationFields).map( ([key, value]) => ({ id: key, - ciphertext: (value as EncryptedPayload)?.c ?? '', + ciphertext: value as CipherStashEncrypted, lockContext: lockContext.context, }), ) @@ -564,7 +570,11 @@ function prepareBulkModelsForOperation>( continue } - if (typeof value === 'object' && !isEncryptedPayload(value)) { + if ( + typeof value === 'object' && + !isEncryptedPayload(value) && + !columnPaths.includes(fullKey) + ) { // Only process nested objects if they're in the schema if (columnPaths.some((path) => path.startsWith(fullKey))) { processNestedFields( @@ -600,6 +610,7 @@ function prepareBulkModelsForOperation>( const processEncryptedFields = ( obj: Record, prefix = '', + columnPaths: string[] = [], ) => { for (const [key, value] of Object.entries(obj)) { const fullKey = prefix ? `${prefix}.${key}` : key @@ -609,9 +620,17 @@ function prepareBulkModelsForOperation>( continue } - if (typeof value === 'object' && !isEncryptedPayload(value)) { + if ( + typeof value === 'object' && + !isEncryptedPayload(value) && + !columnPaths.includes(fullKey) + ) { // Recursively process nested objects - processEncryptedFields(value as Record, fullKey) + processEncryptedFields( + value as Record, + fullKey, + columnPaths, + ) } else if (isEncryptedPayload(value)) { // This is an encrypted field const id = index.toString() @@ -749,7 +768,7 @@ export async function bulkDecryptModels>( const bulkDecryptPayload = operationFields.flatMap((fields, modelIndex) => Object.entries(fields).map(([key, value]) => ({ id: `${modelIndex}-${key}`, - ciphertext: (value as EncryptedPayload)?.c ?? '', + ciphertext: value as CipherStashEncrypted, })), ) @@ -836,7 +855,7 @@ export async function bulkDecryptModelsWithLockContext< const bulkDecryptPayload = operationFields.flatMap((fields, modelIndex) => Object.entries(fields).map(([key, value]) => ({ id: `${modelIndex}-${key}`, - ciphertext: (value as EncryptedPayload)?.c ?? '', + ciphertext: value as CipherStashEncrypted, lockContext: lockContext.context, })), ) diff --git a/packages/protect/src/ffi/operations/bulk-decrypt.ts b/packages/protect/src/ffi/operations/bulk-decrypt.ts index 2480266e..2ddc2b0c 100644 --- a/packages/protect/src/ffi/operations/bulk-decrypt.ts +++ b/packages/protect/src/ffi/operations/bulk-decrypt.ts @@ -1,5 +1,6 @@ import { type Result, withResult } from '@byteslice/result' import { + type Encrypted as CipherStashEncrypted, type DecryptResult, decryptBulkFallible, } from '@cipherstash/protect-ffi' @@ -20,9 +21,7 @@ const createDecryptPayloads = ( .filter(({ data }) => data !== null) .map(({ id, data, originalIndex }) => ({ id, - ciphertext: (typeof data === 'object' && data !== null - ? data.c - : data) as string, + ciphertext: data as CipherStashEncrypted, originalIndex, ...(lockContext && { lockContext }), })) diff --git a/packages/protect/src/ffi/operations/bulk-encrypt.ts b/packages/protect/src/ffi/operations/bulk-encrypt.ts index 5e223965..e89f827e 100644 --- a/packages/protect/src/ffi/operations/bulk-encrypt.ts +++ b/packages/protect/src/ffi/operations/bulk-encrypt.ts @@ -1,5 +1,5 @@ import { type Result, withResult } from '@byteslice/result' -import { encryptBulk } from '@cipherstash/protect-ffi' +import { type JsPlaintext, encryptBulk } from '@cipherstash/protect-ffi' import type { ProtectColumn, ProtectTable, @@ -14,7 +14,7 @@ import type { BulkEncryptedData, Client, EncryptOptions, - EncryptedPayload, + Encrypted, } from '../../types' import { noClientError } from '../index' import { ProtectOperation } from './base-operation' @@ -31,7 +31,7 @@ const createEncryptPayloads = ( .filter(({ plaintext }) => plaintext !== null) .map(({ id, plaintext, originalIndex }) => ({ id, - plaintext: plaintext as string, + plaintext: plaintext as JsPlaintext, column: column.getName(), table: table.tableName, originalIndex, @@ -47,7 +47,7 @@ const createNullResult = ( const mapEncryptedDataToResult = ( plaintexts: BulkEncryptPayload, - encryptedData: EncryptedPayload[], + encryptedData: Encrypted[], ): BulkEncryptedData => { const result: BulkEncryptedData = new Array(plaintexts.length) let encryptedIndex = 0 diff --git a/packages/protect/src/ffi/operations/decrypt.ts b/packages/protect/src/ffi/operations/decrypt.ts index eb6dae62..23fdde0b 100644 --- a/packages/protect/src/ffi/operations/decrypt.ts +++ b/packages/protect/src/ffi/operations/decrypt.ts @@ -1,17 +1,20 @@ import { type Result, withResult } from '@byteslice/result' -import { decrypt as ffiDecrypt } from '@cipherstash/protect-ffi' +import { + type JsPlaintext, + decrypt as ffiDecrypt, +} from '@cipherstash/protect-ffi' import { type ProtectError, ProtectErrorTypes } from '../..' import { logger } from '../../../../utils/logger' import type { LockContext } from '../../identify' -import type { Client, EncryptedPayload } from '../../types' +import type { Client, Encrypted } from '../../types' import { noClientError } from '../index' import { ProtectOperation } from './base-operation' -export class DecryptOperation extends ProtectOperation { +export class DecryptOperation extends ProtectOperation { private client: Client - private encryptedData: EncryptedPayload + private encryptedData: Encrypted - constructor(client: Client, encryptedData: EncryptedPayload) { + constructor(client: Client, encryptedData: Encrypted) { super() this.client = client this.encryptedData = encryptedData @@ -23,7 +26,7 @@ export class DecryptOperation extends ProtectOperation { return new DecryptOperationWithLockContext(this, lockContext) } - public async execute(): Promise> { + public async execute(): Promise> { return await withResult( async () => { if (!this.client) { @@ -41,7 +44,7 @@ export class DecryptOperation extends ProtectOperation { }) return await ffiDecrypt(this.client, { - ciphertext: this.encryptedData.c, + ciphertext: this.encryptedData, unverifiedContext: metadata, }) }, @@ -54,7 +57,7 @@ export class DecryptOperation extends ProtectOperation { public getOperation(): { client: Client - encryptedData: EncryptedPayload + encryptedData: Encrypted auditData?: Record } { return { @@ -65,9 +68,7 @@ export class DecryptOperation extends ProtectOperation { } } -export class DecryptOperationWithLockContext extends ProtectOperation< - string | null -> { +export class DecryptOperationWithLockContext extends ProtectOperation { private operation: DecryptOperation private lockContext: LockContext @@ -81,7 +82,7 @@ export class DecryptOperationWithLockContext extends ProtectOperation< } } - public async execute(): Promise> { + public async execute(): Promise> { return await withResult( async () => { const { client, encryptedData } = this.operation.getOperation() @@ -107,7 +108,7 @@ export class DecryptOperationWithLockContext extends ProtectOperation< } return await ffiDecrypt(client, { - ciphertext: encryptedData.c, + ciphertext: encryptedData, unverifiedContext: metadata, lockContext: context.data.context, serviceToken: context.data.ctsToken, diff --git a/packages/protect/src/ffi/operations/encrypt.ts b/packages/protect/src/ffi/operations/encrypt.ts index 13bb48a4..2d15175c 100644 --- a/packages/protect/src/ffi/operations/encrypt.ts +++ b/packages/protect/src/ffi/operations/encrypt.ts @@ -1,5 +1,8 @@ import { type Result, withResult } from '@byteslice/result' -import { encrypt as ffiEncrypt } from '@cipherstash/protect-ffi' +import { + type JsPlaintext, + encrypt as ffiEncrypt, +} from '@cipherstash/protect-ffi' import type { ProtectColumn, ProtectTable, @@ -9,22 +12,21 @@ import type { import { type ProtectError, ProtectErrorTypes } from '../..' import { logger } from '../../../../utils/logger' import type { LockContext } from '../../identify' -import type { - Client, - EncryptOptions, - EncryptPayload, - EncryptedPayload, -} from '../../types' +import type { Client, EncryptOptions, Encrypted } from '../../types' import { noClientError } from '../index' import { ProtectOperation } from './base-operation' -export class EncryptOperation extends ProtectOperation { +export class EncryptOperation extends ProtectOperation { private client: Client - private plaintext: EncryptPayload + private plaintext: JsPlaintext | null private column: ProtectColumn | ProtectValue private table: ProtectTable - constructor(client: Client, plaintext: EncryptPayload, opts: EncryptOptions) { + constructor( + client: Client, + plaintext: JsPlaintext | null, + opts: EncryptOptions, + ) { super() this.client = client this.plaintext = plaintext @@ -38,7 +40,7 @@ export class EncryptOperation extends ProtectOperation { return new EncryptOperationWithLockContext(this, lockContext) } - public async execute(): Promise> { + public async execute(): Promise> { logger.debug('Encrypting data WITHOUT a lock context', { column: this.column.getName(), table: this.table.tableName, @@ -54,6 +56,14 @@ export class EncryptOperation extends ProtectOperation { return null } + if (typeof this.plaintext === 'number' && isNaN(this.plaintext)) { + throw new Error('[protect]: Cannot encrypt NaN value') + } + + if (typeof this.plaintext === 'number' && !Number.isFinite(this.plaintext)) { + throw new Error('[protect]: Cannot encrypt Infinity value') + } + const { metadata } = this.getAuditData() return await ffiEncrypt(this.client, { @@ -72,7 +82,7 @@ export class EncryptOperation extends ProtectOperation { public getOperation(): { client: Client - plaintext: EncryptPayload + plaintext: JsPlaintext | null column: ProtectColumn | ProtectValue table: ProtectTable } { @@ -85,7 +95,7 @@ export class EncryptOperation extends ProtectOperation { } } -export class EncryptOperationWithLockContext extends ProtectOperation { +export class EncryptOperationWithLockContext extends ProtectOperation { private operation: EncryptOperation private lockContext: LockContext @@ -95,7 +105,7 @@ export class EncryptOperationWithLockContext extends ProtectOperation> { + public async execute(): Promise> { return await withResult( async () => { const { client, plaintext, column, table } = diff --git a/packages/protect/src/helpers/index.ts b/packages/protect/src/helpers/index.ts index c1687303..00744274 100644 --- a/packages/protect/src/helpers/index.ts +++ b/packages/protect/src/helpers/index.ts @@ -1,16 +1,13 @@ -import type { Encrypted } from '@cipherstash/protect-ffi' -import type { EncryptedPayload } from '../types' +import type { Encrypted } from '../types' export type EncryptedPgComposite = { - data: EncryptedPayload + data: Encrypted } /** * Helper function to transform an encrypted payload into a PostgreSQL composite type */ -export function encryptedToPgComposite( - obj: EncryptedPayload, -): EncryptedPgComposite { +export function encryptedToPgComposite(obj: Encrypted): EncryptedPgComposite { return { data: obj, } @@ -47,13 +44,15 @@ export function bulkModelsToEncryptedPgComposites< /** * Helper function to check if a value is an encrypted payload */ -export function isEncryptedPayload(value: unknown): value is EncryptedPayload { +export function isEncryptedPayload(value: unknown): value is Encrypted { if (value === null) return false // TODO: this can definitely be improved if (typeof value === 'object') { const obj = value as Encrypted - return 'v' in obj && 'c' in obj && 'i' in obj + return ( + obj !== null && 'v' in obj && ('c' in obj || 'sv' in obj) && 'i' in obj + ) } return false diff --git a/packages/protect/src/types.ts b/packages/protect/src/types.ts index 9a3a9b52..a491ed94 100644 --- a/packages/protect/src/types.ts +++ b/packages/protect/src/types.ts @@ -1,4 +1,8 @@ -import type { Encrypted, newClient } from '@cipherstash/protect-ffi' +import type { + Encrypted as CipherStashEncrypted, + JsPlaintext, + newClient, +} from '@cipherstash/protect-ffi' import type { ProtectColumn, ProtectTable, @@ -11,14 +15,20 @@ import type { */ export type Client = Awaited> | undefined +/** + * Type to represent an encrypted payload + */ +export type Encrypted = CipherStashEncrypted | null + /** * Represents an encrypted payload in the database + * @deprecated Use `Encrypted` instead */ export type EncryptedPayload = Encrypted | null /** * Represents an encrypted data object in the database - * @deprecated Use `EncryptedPayload` instead + * @deprecated Use `Encrypted` instead */ export type EncryptedData = Encrypted | null @@ -26,7 +36,7 @@ export type EncryptedData = Encrypted | null * Represents a value that will be encrypted and used in a search */ export type SearchTerm = { - value: string + value: JsPlaintext column: ProtectColumn table: ProtectTable returnType?: 'eql' | 'composite-literal' | 'escaped-composite-literal' @@ -34,17 +44,16 @@ export type SearchTerm = { /** * The return type of the search term based on the return type specified in the `SearchTerm` type - * If the return type is `eql`, the return type is `EncryptedPayload` + * If the return type is `eql`, the return type is `Encrypted` * If the return type is `composite-literal`, the return type is `string` where the value is a composite literal * If the return type is `escaped-composite-literal`, the return type is `string` where the value is an escaped composite literal */ -export type EncryptedSearchTerm = EncryptedPayload | string +export type EncryptedSearchTerm = Encrypted | string /** * Represents a payload to be encrypted using the `encrypt` function - * We currently only support the encryption of strings */ -export type EncryptPayload = string | null +export type EncryptPayload = JsPlaintext | null /** * Represents the options for encrypting a payload using the `encrypt` function @@ -58,21 +67,21 @@ export type EncryptOptions = { * Type to identify encrypted fields in a model */ export type EncryptedFields = { - [K in keyof T as T[K] extends EncryptedPayload ? K : never]: T[K] + [K in keyof T as T[K] extends Encrypted ? K : never]: T[K] } /** * Type to identify non-encrypted fields in a model */ export type OtherFields = { - [K in keyof T as T[K] extends EncryptedPayload | null ? never : K]: T[K] + [K in keyof T as T[K] extends Encrypted ? never : K]: T[K] } /** * Type to represent decrypted fields in a model */ export type DecryptedFields = { - [K in keyof T as T[K] extends EncryptedPayload | null ? K : never]: string + [K in keyof T as T[K] extends Encrypted ? K : never]: string } /** @@ -85,12 +94,12 @@ export type Decrypted = OtherFields & DecryptedFields */ export type BulkEncryptPayload = Array<{ id?: string - plaintext: string | null + plaintext: JsPlaintext | null }> -export type BulkEncryptedData = Array<{ id?: string; data: EncryptedPayload }> -export type BulkDecryptPayload = Array<{ id?: string; data: EncryptedPayload }> -export type BulkDecryptedData = Array> +export type BulkEncryptedData = Array<{ id?: string; data: Encrypted }> +export type BulkDecryptPayload = Array<{ id?: string; data: Encrypted }> +export type BulkDecryptedData = Array> type DecryptionSuccess = { error?: never diff --git a/packages/schema/README.md b/packages/schema/README.md index 8304e48b..71b889bd 100644 --- a/packages/schema/README.md +++ b/packages/schema/README.md @@ -136,12 +136,6 @@ Set the data type for a column using `.dataType()`: const column = csColumn('field') .dataType('text') // text (default) .dataType('int') // integer - .dataType('big_int') // big integer - .dataType('small_int') // small integer - .dataType('real') // real number - .dataType('double') // double precision - .dataType('boolean') // boolean - .dataType('date') // date .dataType('jsonb') // JSON binary ``` @@ -189,12 +183,6 @@ const column = csColumn('field').freeTextSearch({ }) ``` -### JSON Field with Prefix - -```typescript -const column = csColumn('metadata').josn('meta') -``` - ## Type Safety The schema builder provides full TypeScript support: @@ -288,7 +276,7 @@ Creates a column definition. - `.equality(tokenFilters?: TokenFilter[])`: Enable equality index - `.freeTextSearch(opts?: MatchIndexOpts)`: Enable text search - `.orderAndRange()`: Enable order and range index -- `.josn(prefix: string)`: Enable JSON field with prefix +- `.searchableJson()`: Enable searchable JSON index ### `csValue(valueName: string)` diff --git a/packages/schema/__tests__/schema.test.ts b/packages/schema/__tests__/schema.test.ts index 7dcae853..d1d99a51 100644 --- a/packages/schema/__tests__/schema.test.ts +++ b/packages/schema/__tests__/schema.test.ts @@ -35,7 +35,7 @@ describe('Schema with nested columns', () => { // Verify email column configuration expect(columns.email).toEqual({ - cast_as: 'text', + cast_as: 'string', indexes: { match: expect.any(Object), unique: expect.any(Object), @@ -45,7 +45,7 @@ describe('Schema with nested columns', () => { // Verify address column configuration expect(columns.address).toEqual({ - cast_as: 'text', + cast_as: 'string', indexes: { match: expect.any(Object), }, @@ -53,13 +53,13 @@ describe('Schema with nested columns', () => { // Verify nested field configuration expect(columns['example.field']).toEqual({ - cast_as: 'text', + cast_as: 'string', indexes: {}, }) // Verify deeply nested field configuration expect(columns['example.nested.deep']).toEqual({ - cast_as: 'text', + cast_as: 'string', indexes: {}, }) }) @@ -119,14 +119,28 @@ describe('Schema with nested columns', () => { // Verify complex nested column with multiple indexes expect(config.tables.complex['content.metadata.tags']).toEqual({ - cast_as: 'text', + cast_as: 'string', indexes: {}, }) // Verify deeply nested column with order and range expect(config.tables.complex['content.metadata.stats.views']).toEqual({ - cast_as: 'text', + cast_as: 'string', indexes: {}, }) }) + + // NOTE: Leaving this test commented out until stevec indexing for JSON is supported. + /*it('should handle ste_vec index for JSON columns', () => { + const users = csTable('users', { + json: csColumn('json').dataType('jsonb').searchableJson(), + } as const) + + const config = buildEncryptConfig(users) + + expect(config.tables.users.json.indexes).toHaveProperty('ste_vec') + expect(config.tables.users.json.indexes.ste_vec?.prefix).toEqual( + 'users/json', + ) + })*/ }) diff --git a/packages/schema/src/index.ts b/packages/schema/src/index.ts index 1431403f..a47434c3 100644 --- a/packages/schema/src/index.ts +++ b/packages/schema/src/index.ts @@ -3,19 +3,16 @@ import { z } from 'zod' // ------------------------ // Zod schemas // ------------------------ +// export type CastAs = +// | 'bigint' +// | 'boolean' +// | 'date' +// | 'number' +// | 'string' +// | 'json' const castAsEnum = z - .enum([ - 'big_int', - 'boolean', - 'date', - 'real', - 'double', - 'int', - 'small_int', - 'text', - 'jsonb', - ]) - .default('text') + .enum(['bigint', 'boolean', 'date', 'number', 'string', 'json']) + .default('string') const tokenFilterSchema = z.object({ kind: z.literal('downcase'), @@ -114,7 +111,7 @@ export class ProtectValue { constructor(valueName: string) { this.valueName = valueName - this.castAsValue = 'text' + this.castAsValue = 'string' } /** @@ -149,7 +146,7 @@ export class ProtectColumn { constructor(columnName: string) { this.columnName = columnName - this.castAsValue = 'text' + this.castAsValue = 'string' } /** @@ -198,12 +195,13 @@ export class ProtectColumn { } /** - * Enable a STE Vec index, requires a prefix. + * Enable a STE Vec index, uses the column name for the index. */ - josn(prefix: string) { - this.indexesValue.ste_vec = { prefix } + // NOTE: Leaving this commented out until stevec indexing for JSON is supported. + /*searchableJson() { + this.indexesValue.ste_vec = { prefix: this.columnName } return this - } + }*/ build() { return { @@ -249,7 +247,25 @@ export class ProtectTable { colName: string, ) => { if (builder instanceof ProtectColumn) { - builtColumns[colName] = builder.build() + const builtColumn = builder.build() + + // Hanlde building the ste_vec index for JSON columns so users don't have to pass the prefix. + if ( + builtColumn.cast_as === 'json' && + builtColumn.indexes.ste_vec?.prefix === 'enabled' + ) { + builtColumns[colName] = { + ...builtColumn, + indexes: { + ...builtColumn.indexes, + ste_vec: { + prefix: `${this.tableName}/${colName}`, + }, + }, + } + } else { + builtColumns[colName] = builtColumn + } } else { for (const [key, value] of Object.entries(builder)) { if (value instanceof ProtectValue) { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 284f33dc..b36b44a9 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -456,8 +456,8 @@ importers: specifier: ^0.2.0 version: 0.2.0 '@cipherstash/protect-ffi': - specifier: 0.16.0 - version: 0.16.0 + specifier: 0.17.1 + version: 0.17.1 '@cipherstash/schema': specifier: workspace:* version: link:../schema @@ -975,33 +975,38 @@ packages: '@changesets/write@0.4.0': resolution: {integrity: sha512-CdTLvIOPiCNuH71pyDu3rA+Q0n65cmAbXnwWH84rKGiFumFzkmHNT8KHTMEchcxN+Kl8I54xGUhJ7l3E7X396Q==} - '@cipherstash/protect-ffi-darwin-arm64@0.16.0': - resolution: {integrity: sha512-RXxhgGemIbFMPH9NcpIncI9uICwEwJxsYekIaZ/DEsT9wUTGmPVkAyscGlellHx5xUkDtrvxIQY086GlcNON8w==} + '@cipherstash/protect-ffi-darwin-arm64@0.17.1': + resolution: {integrity: sha512-hVQJxDDtdXCSX5+r5PCk1Tl2fz/jbaKEtOph/MDz5gSLsxoA9cpgH01TQqmVQdQLV5kKHDBXalvKk6+2b5EgMw==} cpu: [arm64] os: [darwin] - '@cipherstash/protect-ffi-darwin-x64@0.16.0': - resolution: {integrity: sha512-H5LG2mzeQ6Fy3sPqBw1D/IZ5QVe+pZNYRR1u2wiSc6UR7fdlRZPzYNPWcTuHDJvI4vEY3DZjYCUaNNeeGtX0IA==} + '@cipherstash/protect-ffi-darwin-x64@0.17.1': + resolution: {integrity: sha512-+eJnIxXz7VPT+l0I2pyhzo2lPNjxZzu6qUHhejWJsdvh5yYgA0yAa/MZKXOJ1EyuTgVqBDYsE0YSE8KpNgByEw==} cpu: [x64] os: [darwin] - '@cipherstash/protect-ffi-linux-arm64-gnu@0.16.0': - resolution: {integrity: sha512-iHsdJKX6IAe8EoPMdH/zQycH8WXNC7IKoYVI6CdvelJjhScIMfKxeuQolDgEYSQe5ujLfCA6Ujk0bqyKPwiS7Q==} + '@cipherstash/protect-ffi-linux-arm64-gnu@0.17.1': + resolution: {integrity: sha512-qXWvDZmRYiBM/SCPPk7sRxw3UB7Nxb/yAbY7R/FimWFaKH7H086sYAsmXXT/iTe0OSYAdomLj/c1d4gBDvN4bA==} cpu: [arm64] os: [linux] - '@cipherstash/protect-ffi-linux-x64-gnu@0.16.0': - resolution: {integrity: sha512-desQSbnHAB8i8GfQ/kYMojX92McVsejnCf2uCfW9ilkJ+HVtFVfrYp1R2Y2zCtQtvdfuYel07UiFpogIWGp/2Q==} + '@cipherstash/protect-ffi-linux-x64-gnu@0.17.1': + resolution: {integrity: sha512-uEwqmRIovx+h+gv0v3uS6xd5AD21eFFvWq7DjWcdeIsHiBK9nQrsdMCIxhkSYGK37kzc+fRX0wwd8R2cNV6Jzw==} cpu: [x64] os: [linux] - '@cipherstash/protect-ffi-win32-x64-msvc@0.16.0': - resolution: {integrity: sha512-9aZAepWdPUnBeNM8tWL7D0VCqrylwySZbw0qChVDzEfDMrxOnIjrTht54kI14FVKfRHWugm7FFbzjjfLwS4M5A==} + '@cipherstash/protect-ffi-linux-x64-musl@0.17.1': + resolution: {integrity: sha512-0BrhKuL7QTJQbmpioQF6A0wVUA+exrpjUENosUWCOjRf6Pn3fdkNexh2KfwtVaxEnv5E8ro824kH28ud8k7Cvg==} + cpu: [x64] + os: [linux] + + '@cipherstash/protect-ffi-win32-x64-msvc@0.17.1': + resolution: {integrity: sha512-3nl9xcjbd2UMMhhldwZWU+W9rPFQgUty/6JGsphVZtRlP2g034ZHo6hyXWxMcj3tsDluzB37L6Kv/dEGYAILlA==} cpu: [x64] os: [win32] - '@cipherstash/protect-ffi@0.16.0': - resolution: {integrity: sha512-sG5yo0Q277oQGXHd4T4hA9Jdf19AI+0sLzEOvcjjvb/T9AZplsUZJd1ZU0oQUzSrJ4JGzOahyQ6CeMrYvdmdeg==} + '@cipherstash/protect-ffi@0.17.1': + resolution: {integrity: sha512-FGf/GMC+6Qx9KnwaWCdo3a94YaCz2SbqtucHjw2V64h/FXxZf+yNJuVXgfqQz8pNWuz+LgUKojUuXJ2nUmM7KQ==} '@clerk/backend@1.25.5': resolution: {integrity: sha512-nnBpr7oSq5iATWRExuljEfp7xa90KE1OUgaGCSmtZYF0T9TWHGkZHYqkQhD4XjiqlR2XsrsQ/UzPfmHM1Km7+Q==} @@ -7813,30 +7818,34 @@ snapshots: human-id: 4.1.1 prettier: 2.8.8 - '@cipherstash/protect-ffi-darwin-arm64@0.16.0': + '@cipherstash/protect-ffi-darwin-arm64@0.17.1': + optional: true + + '@cipherstash/protect-ffi-darwin-x64@0.17.1': optional: true - '@cipherstash/protect-ffi-darwin-x64@0.16.0': + '@cipherstash/protect-ffi-linux-arm64-gnu@0.17.1': optional: true - '@cipherstash/protect-ffi-linux-arm64-gnu@0.16.0': + '@cipherstash/protect-ffi-linux-x64-gnu@0.17.1': optional: true - '@cipherstash/protect-ffi-linux-x64-gnu@0.16.0': + '@cipherstash/protect-ffi-linux-x64-musl@0.17.1': optional: true - '@cipherstash/protect-ffi-win32-x64-msvc@0.16.0': + '@cipherstash/protect-ffi-win32-x64-msvc@0.17.1': optional: true - '@cipherstash/protect-ffi@0.16.0': + '@cipherstash/protect-ffi@0.17.1': dependencies: '@neon-rs/load': 0.1.82 optionalDependencies: - '@cipherstash/protect-ffi-darwin-arm64': 0.16.0 - '@cipherstash/protect-ffi-darwin-x64': 0.16.0 - '@cipherstash/protect-ffi-linux-arm64-gnu': 0.16.0 - '@cipherstash/protect-ffi-linux-x64-gnu': 0.16.0 - '@cipherstash/protect-ffi-win32-x64-msvc': 0.16.0 + '@cipherstash/protect-ffi-darwin-arm64': 0.17.1 + '@cipherstash/protect-ffi-darwin-x64': 0.17.1 + '@cipherstash/protect-ffi-linux-arm64-gnu': 0.17.1 + '@cipherstash/protect-ffi-linux-x64-gnu': 0.17.1 + '@cipherstash/protect-ffi-linux-x64-musl': 0.17.1 + '@cipherstash/protect-ffi-win32-x64-msvc': 0.17.1 '@clerk/backend@1.25.5(react-dom@19.0.0(react@19.0.0))(react@19.0.0)': dependencies: