refactor(dapi,dpp)!: move dapi-client and Identifier off Buffer to Uint8Array#3680
refactor(dapi,dpp)!: move dapi-client and Identifier off Buffer to Uint8Array#3680PastaPastaPasta wants to merge 9 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (7)
🚧 Files skipped from review as they are similar to previous changes (6)
📝 WalkthroughWalkthroughProject-wide migration from Node Buffer to Uint8Array for binary data, adding shared byte helpers, exposing DAPIClient.bytes, refactoring wasm Identifier and WASM loader, and updating tests/docs and downstream consumers. Changesjs-dapi-client Uint8Array migration
wasm-dpp Uint8Array refactor
Sequence Diagram(s)(skipped) Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Suggested labels
Suggested reviewers
Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
|
|
✅ DashSDKFFI.xcframework built for this PR.
SwiftPM (host the zip at a stable URL, then use): .binaryTarget(
name: "DashSDKFFI",
url: "https://your.cdn.example/DashSDKFFI.xcframework.zip",
checksum: "bfa0ce328feabc7b4f6d66a72c6cdc47229bcc9ec155eaa86e41747b372d302b"
)Xcode manual integration:
|
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
PastaPastaPasta
force-pushed
the
claude/esm-2-uint8array
branch
from
0b8c68e to
0eb8b2c
Compare
PastaPastaPasta
changed the title
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
PastaPastaPasta
force-pushed
the
claude/esm-2-uint8array
branch
from
0eb8b2c to
29cf55f
Compare
PastaPastaPasta
force-pushed
the
claude/esm-2-uint8array
branch
5 times, most recently
from
28e63f7 to
afa4da3
Compare
… API
Adds lib/utils/bytes.js helper (hexToBytes/bytesToHex/base64ToBytes/bytesToBase64/concatBytes/bytesEqual) and converts all Buffer.* call sites in dapi-client lib/ to Uint8Array, with corresponding test updates. Package stays CJS.
Production exceptions where Buffer is retained: BlockHeadersReader passes Buffer to dashcore-lib's BlockHeader (its BufferReader needs .readInt32LE), and GetIdentitiesContractKeysResponse passes Buffer to wasm-dpp's Identifier.from (it explicitly requires Node Buffer).
createGrpcTransportError now handles both raw bytes (grpc-js path) and base64 strings (grpc-web path) for drive-error-data-bin, stack-bin, and dash-serialized-consensus-error-bin metadata fields, restoring the dual-format behavior that Buffer.from(x, 'base64') used to provide implicitly.
Test updates: spec files that construct expected protobuf requests now wrap .toBuffer() with new Uint8Array(...) to match production's normalization (sinon calledOnceWithExactly distinguishes Buffer from plain Uint8Array).
Breaking change for direct consumers: response object byte fields are now Uint8Array. Callers that do response.field.toString('hex') will fail — use bytesToHex(response.field) from lib/utils/bytes instead. Buffer.isBuffer(response.field) now returns false; use response.field instanceof Uint8Array.
Test results: dapi-client 315/315, wallet-lib 377/377, js-dash-sdk 60/60 — downstream consumers continue passing without modification (they exercise dapi-client mostly via mocks).
PastaPastaPasta
force-pushed
the
claude/esm-2-uint8array
branch
from
afa4da3 to
e703e90
Compare
Identifier constructor and static from() now accept Uint8Array in addition to Buffer. Since Buffer extends Uint8Array in Node, existing Buffer callers continue to work unchanged — this is a widening, not a swap. Lets dapi-client and other web-facing consumers pass plain Uint8Array (e.g. from protobuf getters) without an explicit Buffer.from wrap. Error message updated from 'Identifier expects Buffer' to 'Identifier expects Uint8Array'.
wasm-dpp's Identifier.from now accepts Uint8Array directly, so the explicit Buffer.from wrap around entry.getIdentityId() is no longer needed. Removes one of two remaining Buffer escape hatches in dapi-client (the BlockHeader path through dashcore-lib remains, pending an upstream change there).
…ment
- Replace deprecated String.prototype.substr with slice in hexToBytes.
- Add light input validation to bytesToHex, base64ToBytes, bytesToBase64, concatBytes, and bytesEqual — they now throw informative TypeErrors instead of opaque runtime errors on non-bytes input.
- Export the helpers as DAPIClient.bytes so the recommended migration path (bytesToHex(field) instead of field.toString('hex')) is reachable via the package's public surface instead of a deep import into lib/utils/bytes.
- README: add a Browser usage section explaining that response fields are Uint8Array, that DAPIClient.bytes is the conversion helper, and that two internal paths (BlockHeadersProvider via dashcore-lib, Identifier.from via wasm-dpp) still construct Buffer — browser consumers must ensure Buffer is reachable at runtime via their bundler or an explicit polyfill until dashcore-lib's BufferReader accepts Uint8Array.
PastaPastaPasta
force-pushed
the
claude/esm-2-uint8array
branch
from
55fea88 to
250a79c
Compare
PastaPastaPasta
changed the title
Identifier no longer extends Buffer at the prototype level. It now extends Uint8Array, so new Identifier(x), id.toString('hex'), id.toString('base64'), id.toString('base58'), id.toBytes(), and id.equals(other) work without a Buffer polyfill at runtime. The wasm-dpp module loader also drops Buffer.from in favor of an inline atob-based decode.
toBuffer() is retained as a @deprecated compatibility shim that returns Buffer.from(this); it still requires the Buffer global to exist at call time but no longer at module load time. External callers migrating off Buffer should prefer toBytes() going forward.
BREAKING for any code that relied on Identifier extending Buffer: Buffer.isBuffer(id), id instanceof Buffer, and Buffer-specific methods on id (.readUInt32LE, .equals, etc.) no longer work. The one such internal callsite (wallet-lib's IdentitySyncWorker) is migrated to check instanceof Uint8Array. The error message string changed from 'Identifier expects Buffer' to 'Identifier expects Uint8Array' (accepted-type check itself remains a widening — Buffer extends Uint8Array).
bs58 (via safe-buffer) is the remaining transitive Buffer dependency in wasm-dpp; eliminating it requires replacing bs58 with a Uint8Array-native base58 library and is out of scope here.
…to .toBytes() Mechanical follow-up to the Identifier extends Uint8Array commit. All internal test code now uses the Buffer-free .toBytes() accessor on Identifier instances (.getId(), .getOwnerId(), .getContractId(), .getDataContractId(), and direct identityId/contractId/ownerId variables). The deprecated .toBuffer() method is retained for external compatibility but no longer used inside the monorepo. Also strips redundant new Uint8Array(...) wraps around .toBytes() — toBytes already returns a fresh Uint8Array, so the wrap was a copy of a copy.
PastaPastaPasta
changed the title
PastaPastaPasta
changed the title
PastaPastaPasta
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Acted on real findings, dropped false positives.
Fixed:
- wallet-lib getTransaction: Buffer.from(response.getBlockHash()).toString('hex') instead of relying on Uint8Array.toString('hex') which ignores the encoding arg. After PR 2's migration, GetTransactionResponse.getBlockHash() returns a plain Uint8Array, so the previous code returned a comma-delimited decimal string instead of hex.
- GetProtocolVersionUpgradeVoteStatusResponse: use versionSignal.getProTxHash_asU8() instead of new Uint8Array(versionSignal.getProTxHash()). The default protobuf getter yields a base64 string under grpc-web; new Uint8Array(string) does not base64-decode, it iterates chars and produces zeros. _asU8 explicitly returns Uint8Array bytes in both grpc-js and grpc-web.
- wallet-lib Transport.d.ts: getIdentityByPublicKeyHash signature updated from (publicKeyHash: Buffer): Promise<Buffer[]> to (publicKeyHash: Uint8Array): Promise<Uint8Array>. Matches the post-migration runtime contract.
- wasm-dpp Document.spec.js #getDataContractId: was calling getOwnerId() and asserting against $ownerId. Now actually exercises getDataContractId.
- GetIdentityByPublicKeyHashResponse JSDoc: @param renamed from identities (plural) to identity (singular); @returns Uint8Array[] corrected to Uint8Array. Both predate this PR; cleaned up while touching the file.
- GetIdentityContractNonce.spec.js, GetIdentityNonce.spec.js: proof-only constructor-pass-through tests now use BigInt(0) (matches the actual nonce type) instead of new Uint8Array(0).
- GetIdentityKeys.spec.js: proof-only test now uses [] (matches the array-of-keys contract) instead of new Uint8Array(0).
- PlatformMethodsFacade.spec.js: getIdentityNonce second arg is now {} (options) instead of a second Uint8Array. Predates the migration.
- SimplifiedMasternodeListProvider catch block: guards bytesToHex call when simplifiedMNListDiffBytes is undefined, so the original error isn't masked by a secondary TypeError from the new bytesToHex input validation.
Skipped (false positives):
- GetDataContractResponse, GetIdentityResponse, GetIdentityByPublicKeyHashResponse createFromProto 'new Uint8Array(undefined)' findings: the existing tests explicitly assert new Uint8Array(0) for the proof-only path, and new Uint8Array(undefined) === new Uint8Array(0). Current behavior matches contract.
- wasm-dpp loadDpp atob fallback: atob has been a Node global since 16.0.0, and the package's engines.node is >=18.18 (PR 1). No fallback needed.
- validateDocumentsBatchTransitionBasicFactory.spec.js .toBuffer() nitpick: inside a describe.skip block; dead code.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
thepastaclaw
left a comment
thepastaclaw
left a comment
There was a problem hiding this comment.
Code Review
Both prior findings are confirmed FIXED at HEAD 4ed8428. Remaining in-scope items are: (1) inconsistent dashcore-lib boundary in wallet-lib getTransaction.js — the new Uint8Array transaction payload is passed straight into new Transaction(...), while every other in-tree call site at the same boundary still normalizes via Buffer.from(...); (2) the protobuf bytes-decoding pattern new Uint8Array(proto.getX()) used in several PR-touched response classes will silently corrupt bytes under grpc-web (the very target this PR enables) — the author already fixed one instance in GetProtocolVersionUpgradeVoteStatusResponse.js with a comment explaining the issue; (3) a JSDoc/.d.ts contract mismatch on getIdentityByPublicKeyHash.js. None are strictly blocking — codex-ffi-engineer verified dashcore-lib 0.22 accepts Uint8Array via BufferUtil — but the inconsistency and grpc-web pattern should be addressed.
🟡 2 suggestion(s) | 💬 1 nitpick(s)
1 additional finding(s) omitted (not in diff).
Out-of-scope follow-up suggestions (2)
These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.
- Audit all dapi-client response classes for grpc-web byte-decoding compatibility — The
new Uint8Array(proto.getXxx())pattern that the author fixed inGetProtocolVersionUpgradeVoteStatusResponse.jsexists across many sibling response classes (GetStatusResponse.jsnodeId/proTxHash/latestBlockHash/latestAppHash/earliestBlockHash/earliestAppHash,WaitForStateTransitionResultResponse.js,response/Proof.jsgrovedbProof/quorumHash/signature,core/getTransaction/GetTransactionResponse.jsblockHash,getBlockchainStatusFactory.js). Some of these are touched by this PR; the broader audit (and choosing between_asU8()everywhere vs. a base64-string-aware helper) is bigger than this PR.- Follow-up: Open a separate issue to systematically migrate all dapi-client bytes-field decoders to
_asU8()getters (or a wrapper that handles both representations).
- Follow-up: Open a separate issue to systematically migrate all dapi-client bytes-field decoders to
- Verify wallet-lib → dashcore-lib byte boundaries now that responses are Uint8Array —
packages/wallet-lib/src/transport/DAPIClientTransport/methods/getBlockByHash.jsandgetBlockByHeight.jspass the dapi-client response (now Uint8Array) straight intonew Block(...)from@dashevo/dashcore-lib. The PR explicitly preservedBuffer.from(header)at the analogousBlockHeadersReaderboundary. Worth confirming whetherdashcore-lib.Blockaccepts a Uint8Array, or whether explicitBuffer.from(...)wraps should be added for consistency with the rest of the codebase.- Follow-up: Audit wallet-lib → dashcore-lib byte boundaries (Block, BlockHeader, Transaction inputs) and either standardize on
Buffer.from(...)wrappers or wait for upstream dashcore-lib#315 to widen the inputs.
- Follow-up: Audit wallet-lib → dashcore-lib byte boundaries (Block, BlockHeader, Transaction inputs) and either standardize on
| @@ -22,7 +22,9 @@ module.exports = async function getTransaction(txid) { | |||
|
|
|||
| return { | |||
| transaction: new Transaction(response.getTransaction()), | |||
There was a problem hiding this comment.
🟡 Suggestion: getTransaction() passes raw Uint8Array to dashcore-lib.Transaction while sibling call sites still normalize via Buffer.from(...)
@dashevo/dapi-client now constructs GetTransactionResponse.transaction as a plain Uint8Array (packages/js-dapi-client/lib/methods/core/getTransaction/GetTransactionResponse.js:78), and this wrapper hands the result directly to new Transaction(response.getTransaction()). Every other raw-transaction entry point in this same package still routes through Buffer.from(...) before crossing into dashcore-lib: packages/wallet-lib/src/plugins/Workers/TransactionsSyncWorker/utils.js:56 does new Transaction(Buffer.from(rawTransaction)), and packages/wallet-lib/src/transport/FixtureTransport/methods/getTransaction.js:6 does new Transaction(Buffer.from(txFile.transaction, 'hex')). The PR also intentionally preserves Buffer.from(...) at the analogous BlockHeadersReader boundary. The dapi-client unit tests still build GetTransactionResponse with Buffer values, so the new typed-array runtime shape isn't actually exercised by tests. Either wrap with Buffer.from(...) here for consistency and safety, or drop the wraps at the other two sites with a comment.
| transaction: new Transaction(response.getTransaction()), | |
| transaction: new Transaction(Buffer.from(response.getTransaction())), |
source: ['codex']
| return new GetIdentityResponse( | ||
| Buffer.from(proto.getV0().getIdentity()), | ||
| new Uint8Array(proto.getV0().getIdentity()), | ||
| metadata, | ||
| proof, | ||
| ); |
There was a problem hiding this comment.
🟡 Suggestion: new Uint8Array(proto.getIdentity()) will silently corrupt bytes under grpc-web — use getIdentity_asU8()
The google-protobuf JS binding returns bytes fields either as Uint8Array (grpc-js) or as a base64 string (grpc-web, depending on deserialization). The author already addressed this exact boundary at packages/js-dapi-client/lib/methods/platform/getProtocolVersionUpgradeVoteStatus/GetProtocolVersionUpgradeVoteStatusResponse.js:44-48 by switching to getProTxHash_asU8(), with a comment stating the default getter would yield a base64 string under grpc-web and new Uint8Array(string) does NOT base64-decode it. Same issue applies here and in GetIdentityByPublicKeyHashResponse.js:29-32, GetStatusResponse.js:96-106, WaitForStateTransitionResultResponse.js:41, response/Proof.js:51-53, and core/getTransaction/GetTransactionResponse.js:78-79. Since this PR's stated purpose is to enable browser/grpc-web bundles, the _asU8() accessor should be applied consistently at the bytes-field boundaries this PR touches. Granted, the pre-PR Buffer.from(string) was also wrong in this scenario, but this PR's whole point is to make these paths actually work in the browser — so the fix belongs here.
| return new GetIdentityResponse( | |
| Buffer.from(proto.getV0().getIdentity()), | |
| new Uint8Array(proto.getV0().getIdentity()), | |
| metadata, | |
| proof, | |
| ); | |
| return new GetIdentityResponse( | |
| proto.getV0().getIdentity_asU8(), | |
| metadata, | |
| proof, | |
| ); |
source: ['claude']
…fer-wrap tx Bytes-field boundaries used new Uint8Array(getXxx()), which silently yields an empty array under grpc-web where protobuf stores bytes as a base64 string (new Uint8Array(string) does not base64-decode). Switch to the _asU8() accessors, matching the pattern already used for proTxHash in GetProtocolVersionUpgradeVoteStatusResponse. Also wrap the raw transaction in Buffer.from before passing to dashcore-lib's Transaction in wallet-lib getTransaction, consistent with the other raw-transaction call sites. Addresses CodeRabbit review findings.
|
✅ Review complete (commit 7885bb3) |
thepastaclaw
left a comment
thepastaclaw
left a comment
There was a problem hiding this comment.
Code Review
PR resolves two of three prior findings: getTransaction now Buffer-wraps the payload before handing to dashcore-lib, and six response classes (GetTransaction, GetIdentity, GetIdentityByPublicKeyHash, GetStatus, Proof, WaitForStateTransitionResult) now use _asU8(). However the sweep missed four sibling byte-decoding sites that were also touched by this PR (getBlockchainStatusFactory, getBlockByHashFactory, getBlockByHeightFactory, GetDocumentsResponse), each of which still uses the new Uint8Array(proto.getX()) pattern that silently corrupts bytes under grpc-web. JSDoc/.d.ts contract mismatch on getIdentityByPublicKeyHash.js also still valid.
🟡 4 suggestion(s) | 💬 1 nitpick(s)
1 additional finding(s) omitted (not in diff).
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/js-dapi-client/lib/methods/core/getBlockByHashFactory.js`:
- [SUGGESTION] packages/js-dapi-client/lib/methods/core/getBlockByHashFactory.js:30-32: `new Uint8Array(response.getBlock())` corrupts the block payload under grpc-web — use `getBlock_asU8()`
This factory was modified by this PR (e703e9024f changed `Buffer.from(response.getBlock())` to `new Uint8Array(response.getBlock())`). The google-protobuf JS binding returns `bytes` fields as a `Uint8Array` under grpc-js but as a base64 `string` under grpc-web. `new Uint8Array(string)` does NOT base64-decode — it coerces the string length to a numeric and produces a zero-length / bogus typed array, silently corrupting the block bytes returned to callers. The latest delta in this PR explicitly fixed this same pattern in `GetTransactionResponse`, `GetIdentityResponse`, `GetIdentityByPublicKeyHashResponse`, `GetStatusResponse`, `Proof`, and `WaitForStateTransitionResultResponse` using the `_asU8()` accessor for exactly this reason. Apply the same fix here; the generated bindings expose `getBlock_asU8()` for this purpose.
In `packages/js-dapi-client/lib/methods/core/getBlockByHeightFactory.js`:
- [SUGGESTION] packages/js-dapi-client/lib/methods/core/getBlockByHeightFactory.js:31-33: `new Uint8Array(response.getBlock())` corrupts the block payload under grpc-web — use `getBlock_asU8()`
Same root cause as `getBlockByHashFactory.js`: this PR (e703e9024f) replaced `Buffer.from(response.getBlock())` with `new Uint8Array(response.getBlock())`. Under grpc-web `response.getBlock()` can be a base64 `string`, and `new Uint8Array(string)` does not decode the payload — it yields garbage bytes. The PR already standardized on the `_asU8()` accessor in six sibling response classes; the same fix applies here. The generated `GetBlockResponse` exposes `getBlock_asU8()`.
In `packages/js-dapi-client/lib/methods/core/getBlockchainStatusFactory.js`:
- [SUGGESTION] packages/js-dapi-client/lib/methods/core/getBlockchainStatusFactory.js:34-46: `new Uint8Array(response.getChain().getBestBlockHash() / getChainWork())` corrupts bytes under grpc-web — use `_asU8()`
This factory was modified by this PR (e703e9024f changed `Buffer.from(...)` to `new Uint8Array(...)` for `bestBlockHash` and `chainWork`). Under grpc-web the protobuf `bytes` getters return a base64 `string` rather than a `Uint8Array`, and `new Uint8Array(string)` does not base64-decode — it produces a zero-length or NaN-length typed array, silently corrupting the values. The latest delta in this PR (7885bb39) explicitly fixed this exact pattern in six sibling response classes with the comment "new Uint8Array(string) does NOT base64-decode, silently losing bytes." The `GetBlockchainStatusResponse` chain message exposes `getBestBlockHash_asU8()` and `getChainWork_asU8()`; this site was missed in the fix-up sweep.
In `packages/js-dapi-client/lib/methods/platform/getDocuments/GetDocumentsResponse.js`:
- [SUGGESTION] packages/js-dapi-client/lib/methods/platform/getDocuments/GetDocumentsResponse.js:31-36: `getDocumentsList().map((d) => new Uint8Array(d))` corrupts each document under grpc-web — use `getDocumentsList_asU8()`
This response class was modified by this PR (e703e9024f). Each entry returned by `documents.getDocumentsList()` is a protobuf `bytes` value, which under grpc-web can be a base64 `string`. Wrapping each entry in `new Uint8Array(string)` does not base64-decode it; it produces garbage bytes for every document returned. The PR's latest delta already standardized on `_asU8()` accessors elsewhere for exactly this reason. The generated bindings expose `getDocumentsList_asU8()` which returns an array of `Uint8Array` regardless of grpc-js vs grpc-web representation.
| const blockBinaryArray = response.getBlock(); | ||
|
|
||
| return Buffer.from(blockBinaryArray); | ||
| return new Uint8Array(blockBinaryArray); |
There was a problem hiding this comment.
🟡 Suggestion: new Uint8Array(response.getBlock()) corrupts the block payload under grpc-web — use getBlock_asU8()
This factory was modified by this PR (e703e90 changed Buffer.from(response.getBlock()) to new Uint8Array(response.getBlock())). The google-protobuf JS binding returns bytes fields as a Uint8Array under grpc-js but as a base64 string under grpc-web. new Uint8Array(string) does NOT base64-decode — it coerces the string length to a numeric and produces a zero-length / bogus typed array, silently corrupting the block bytes returned to callers. The latest delta in this PR explicitly fixed this same pattern in GetTransactionResponse, GetIdentityResponse, GetIdentityByPublicKeyHashResponse, GetStatusResponse, Proof, and WaitForStateTransitionResultResponse using the _asU8() accessor for exactly this reason. Apply the same fix here; the generated bindings expose getBlock_asU8() for this purpose.
| const blockBinaryArray = response.getBlock(); | |
| return Buffer.from(blockBinaryArray); | |
| return new Uint8Array(blockBinaryArray); | |
| return response.getBlock_asU8(); |
source: ['codex']
| const blockBinaryArray = response.getBlock(); | ||
|
|
||
| return Buffer.from(blockBinaryArray); | ||
| return new Uint8Array(blockBinaryArray); |
There was a problem hiding this comment.
🟡 Suggestion: new Uint8Array(response.getBlock()) corrupts the block payload under grpc-web — use getBlock_asU8()
Same root cause as getBlockByHashFactory.js: this PR (e703e90) replaced Buffer.from(response.getBlock()) with new Uint8Array(response.getBlock()). Under grpc-web response.getBlock() can be a base64 string, and new Uint8Array(string) does not decode the payload — it yields garbage bytes. The PR already standardized on the _asU8() accessor in six sibling response classes; the same fix applies here. The generated GetBlockResponse exposes getBlock_asU8().
| const blockBinaryArray = response.getBlock(); | |
| return Buffer.from(blockBinaryArray); | |
| return new Uint8Array(blockBinaryArray); | |
| return response.getBlock_asU8(); |
source: ['codex']
| if (response.getChain()) { | ||
| if (response.getChain() | ||
| .getBestBlockHash()) { | ||
| responseObject.chain.bestBlockHash = Buffer.from(response.getChain() | ||
| responseObject.chain.bestBlockHash = new Uint8Array(response.getChain() | ||
| .getBestBlockHash()); | ||
| } | ||
|
|
||
| if (response.getChain() | ||
| .getChainWork()) { | ||
| responseObject.chain.chainWork = Buffer.from(response.getChain() | ||
| responseObject.chain.chainWork = new Uint8Array(response.getChain() | ||
| .getChainWork()); | ||
| } | ||
| } |
There was a problem hiding this comment.
🟡 Suggestion: new Uint8Array(response.getChain().getBestBlockHash() / getChainWork()) corrupts bytes under grpc-web — use _asU8()
This factory was modified by this PR (e703e90 changed Buffer.from(...) to new Uint8Array(...) for bestBlockHash and chainWork). Under grpc-web the protobuf bytes getters return a base64 string rather than a Uint8Array, and new Uint8Array(string) does not base64-decode — it produces a zero-length or NaN-length typed array, silently corrupting the values. The latest delta in this PR (7885bb3) explicitly fixed this exact pattern in six sibling response classes with the comment "new Uint8Array(string) does NOT base64-decode, silently losing bytes." The GetBlockchainStatusResponse chain message exposes getBestBlockHash_asU8() and getChainWork_asU8(); this site was missed in the fix-up sweep.
| if (response.getChain()) { | |
| if (response.getChain() | |
| .getBestBlockHash()) { | |
| responseObject.chain.bestBlockHash = Buffer.from(response.getChain() | |
| responseObject.chain.bestBlockHash = new Uint8Array(response.getChain() | |
| .getBestBlockHash()); | |
| } | |
| if (response.getChain() | |
| .getChainWork()) { | |
| responseObject.chain.chainWork = Buffer.from(response.getChain() | |
| responseObject.chain.chainWork = new Uint8Array(response.getChain() | |
| .getChainWork()); | |
| } | |
| } | |
| if (response.getChain()) { | |
| if (response.getChain() | |
| .getBestBlockHash()) { | |
| // Use _asU8 so we get bytes regardless of the underlying protobuf | |
| // representation (grpc-js: Uint8Array; grpc-web: base64 string). | |
| // new Uint8Array(string) does NOT base64-decode, silently losing bytes. | |
| responseObject.chain.bestBlockHash = response.getChain() | |
| .getBestBlockHash_asU8(); | |
| } | |
| if (response.getChain() | |
| .getChainWork()) { | |
| responseObject.chain.chainWork = response.getChain() | |
| .getChainWork_asU8(); | |
| } | |
| } |
source: ['claude', 'codex']
| return new GetDocumentsResponse( | ||
| documents !== undefined | ||
| ? documents.getDocumentsList().map((document) => Buffer.from(document)) : [], | ||
| ? documents.getDocumentsList().map((document) => new Uint8Array(document)) : [], | ||
| metadata, | ||
| proof, | ||
| ); |
There was a problem hiding this comment.
🟡 Suggestion: getDocumentsList().map((d) => new Uint8Array(d)) corrupts each document under grpc-web — use getDocumentsList_asU8()
This response class was modified by this PR (e703e90). Each entry returned by documents.getDocumentsList() is a protobuf bytes value, which under grpc-web can be a base64 string. Wrapping each entry in new Uint8Array(string) does not base64-decode it; it produces garbage bytes for every document returned. The PR's latest delta already standardized on _asU8() accessors elsewhere for exactly this reason. The generated bindings expose getDocumentsList_asU8() which returns an array of Uint8Array regardless of grpc-js vs grpc-web representation.
| return new GetDocumentsResponse( | |
| documents !== undefined | |
| ? documents.getDocumentsList().map((document) => Buffer.from(document)) : [], | |
| ? documents.getDocumentsList().map((document) => new Uint8Array(document)) : [], | |
| metadata, | |
| proof, | |
| ); | |
| return new GetDocumentsResponse( | |
| documents !== undefined | |
| ? documents.getDocumentsList_asU8() : [], | |
| metadata, | |
| proof, | |
| ); |
source: ['codex']
4 participants
Summary
Replaces
BufferwithUint8Arrayacross@dashevo/dapi-client's public API. Rewrites@dashevo/wasm-dpp'sIdentifierto extendUint8Arraydirectly (instead ofBuffer) and removesBuffer.fromfrom the wasm-dpp module loader. After this PR, both packages' module-load and core construction paths are Buffer-free; only the deprecatedIdentifier#toBuffer()shim still requires aBufferglobal at call time. The SPV path through dashcore-lib will become Buffer-free once dashpay/dashcore-lib#315 lands and dapi-client picks up the new release.This is PR 2 of 3 in the stacked series. PR 1 (#3679) is merged.
Why
Removes
Bufferfrom the dapi-client and wasm-dpp surfaces so consumers don't need aBufferpolyfill in browser bundles.Bufferis Node-specific;Uint8Arrayworks everywhere.What changes
@dashevo/dapi-clientlib/utils/bytes.js:hexToBytes,bytesToHex,base64ToBytes,bytesToBase64,concatBytes,bytesEqual(CJS exports). Input-validated with informativeTypeErrors; usesString.prototype.slicenot the deprecatedsubstr.DAPIClient.bytesso the migration path is reachable from the package's public surface (no deep imports).Buffer.*call sites inlib/converted toUint8Arrayper the translation table below.createGrpcTransportError: explicit dual-format handling that the previousBuffer.from(x, 'base64')provided implicitly —typeof x === 'string' ? base64ToBytes(x) : new Uint8Array(x)for the three metadata-bin fields (grpc-js sends raw bytes; grpc-web sends base64 strings).Buffer.fromis retained:BlockHeadersReader→new BlockHeader(Buffer.from(header))because dashcore-lib'sBufferReaderstill requiresBuffer. Tracked upstream: dashpay/dashcore-lib#315. Once that lands and we bump the dep here, this wrap goes away.@dashevo/wasm-dppUint8Arrayinstead ofBuffer.Buffer.isBuffer(id)now returnsfalse;id instanceof Uint8Arrayreturnstrue.id.toBytes()returns a plainUint8Arraycopy.id.toBuffer()retained as@deprecatedcompatibility shim — callsBuffer.from(this). RequiresBufferonly at call time, no longer at module load.id.toString(encoding)now handles'base58'/'base64'/'hex'explicitly via inline byte helpers — noBuffermethod dependency. ThrowsIdentifierError: Unsupported encoding: ...for anything else.Identifier.from(value, encoding)acceptsUint8Arraydirectly andstringwith'base58'(default) or'base64'. The encoding parameter type is narrowed to only the supported values.id.equals(other)byte-by-byte loop, no Buffer methods.lib/index.ts) replacesBuffer.from(wasmBase, 'base64')with an inlineatob-based decode.loadDpp()no longer requires aBufferglobal.@dashevo/wallet-libIdentitySyncWorker.js:98migratesBuffer.isBuffer(identityBuffer)→identityBuffer instanceof Uint8Array. Comment updated.Test migrations
All internal test code that called
.toBuffer()on an Identifier-derived value is migrated to.toBytes()(36 test files, ~100 mechanical one-line swaps). The deprecated.toBuffer()shim is intentionally exercised in one place —wasm-dpp/test/unit/Identifier.spec.js#toBufferdescribe block — with an explanatory comment.README
New "Browser usage" section in dapi-client documenting that response fields are
Uint8Array, thatDAPIClient.bytesis the conversion helper, and that one internal path (BlockHeadersProvider via dashcore-lib) still requires aBufferpolyfill until upstream changes land.Breaking changes
@dashevo/dapi-clientpublic APIresponse.field.toString('hex')DAPIClient.bytes.bytesToHex(response.field)response.field.toString('base64')DAPIClient.bytes.bytesToBase64(response.field)Buffer.isBuffer(response.field)response.field instanceof Uint8Arrayresponse.field.slice(a, b)response.field.subarray(a, b)In Node,
Buffer extends Uint8Array, so anything that just reads bytes (passes tocrypto, writes to a stream, length checks, indexed access) continues to work identically.@dashevo/wasm-dppIdentifierBuffer.isBuffer(id)returnedtruefalseid instanceof Bufferreturnedtruefalseid.readUInt32LE(...)etc. (Buffer methods)idno longer has Buffer-only methodsid.toBuffer()id.toBytes()id.toString('hex')/id.toString('base64')id.toString('utf8'),'ascii', etc.IdentifierError: Unsupported encoding(was silently inherited from Buffer)'Identifier expects Buffer''Identifier expects Uint8Array'The remaining transitive Buffer dependency in wasm-dpp is
bs58(viasafe-buffer) — eliminating it requires replacingbs58with a Uint8Array-native base58 library and is out of scope here.Test plan
JS packages (@dashevo/dapi-client) / TestsJS packages (@dashevo/wasm-dpp) / TestsJS packages (@dashevo/wallet-lib) / TestsJS packages (dash) / TestsStack
Summary by CodeRabbit
New Features
Bug Fixes / Improvements
Breaking Changes
Documentation