Add const XDR serialization by leighmcculloch · Pull Request #562 · stellar/rs-stellar-xdr

leighmcculloch · 2026-07-28T11:45:57Z

Note

Part of a stack of PRs that must merge in this order.

A first group of PRs change contract specs so that they are produced at compile time instead of at proc-macro execution time. This provides the foundation to construct the specs from information that is only known at compile time, like the fully qualified name of a type:

A second group of PRs changes the type names that the sdk stores in specs are fully qualified type names, and then the cli reduces them down to unique simple identifiers. During the contract build the types are given names like ::mycrate::mymod::MyType instead of MyType. Then the cli reduces them back down to simple names after spec shaking. Qualified type names make it possible to uniquely identify types in the spec, even when they have the same name. This resolves several problems with contract specs the type identify problem (stellar/rs-soroban-sdk#1570), type aliases limitations (stellar/rs-soroban-sdk#1857 stellar/rs-soroban-sdk#1063), and optimise spec shaking data section size (stellar/rs-soroban-sdk#1978):

Note that downstream clients and SDKs should see no, or little, change because the cli will during the build process reduce the fully qualified names back to simple unique names. Contracts that had colliding type names, which meant they could not be used with clients, will now work.

A third group of PRs are an optimisation to spec shaking v2, and will use the new unique type names to shake type specs by reachability, producing a dependency graph (thanks @mootz12), so that only spec entries that can't be reached from fns, like errors and events, get spec markers from dead-code-elimination:

What

Add a const feature giving every generated type an alternative XDR encoding that can be used in const contexts.

Why

XDR encoding is used by soroban-sdk to encode contract specs during proc-macro execution. The soroban-sdk will be changing when it encodes contracts specs to during compilation so that the specs can contain information only available during compile.

Known limitations

The const implementation is a separate implementation because a const implementation cannot use traits, which means it can't stream to a writer. It is not practical to maintain a single implementation that offers the same features of both. So instead there are two implementations and a test ensures they work identically.

I would have really liked to fuzz test the const xdr encoding with the existing xdr encoding, however I don't see a way to do that. We can't convert from owned to ref because of the way the ref types are structured. And we can't generate ref types from arbitrary because there would be nothing to own the values. Open to feedback on this, it's possible a slightly different way of architecting the ref types might make this possible.

…ases

Copilot

Pull request overview

Adds feature-gated const XDR serialization for generated types, enabling encoding during compile-time evaluation.

Changes:

Adds generated const serialization APIs and type mappings.
Updates enum, struct, union, and newtype templates.
Regenerates XDR types with const serialization support.
Adds parity testing against runtime WriteXdr.

Required changes:

Add parity coverage for a fixed non-opaque array, ideally including a nested array.
Declare the encoded test array as const so the complete serializer is verified under CTFE.

Reviewed changes

Copilot reviewed 2 out of 487 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`xdr-generator-rust/generator/templates/union.rs.jinja`	Generates const serialization for unions.
`xdr-generator-rust/generator/templates/typedef_newtype.rs.jinja`	Generates const serialization for newtypes.
`xdr-generator-rust/generator/templates/struct.rs.jinja`	Generates const serialization for structs.
`xdr-generator-rust/generator/templates/enum.rs.jinja`	Generates const serialization for enums.
`xdr-generator-rust/generator/templates/definition.rs.jinja`	Integrates generated const definitions.
`xdr-generator-rust/generator/templates/const_to_xdr.rs.jinja`	Defines public const encoding APIs.
`xdr-generator-rust/generator/src/types.rs`	Maps types to const-compatible references.
`xdr-generator-rust/generator/src/main.rs`	Registers const serializer generation.
`tests/const_writer.rs`	Tests const/runtime serialization parity; coverage and CTFE proof need strengthening.
`src/generated/uint256.rs`	Adds generated const serialization.
`src/generated/u_int256_parts.rs`	Adds generated const serialization.
`src/generated/u_int128_parts.rs`	Adds generated const serialization.
`src/generated/tx_set_component_type.rs`	Adds generated const serialization.
`src/generated/tx_demand_vector.rs`	Adds generated const serialization.
`src/generated/tx_advert_vector.rs`	Adds generated const serialization.
`src/generated/trust_line_flags.rs`	Adds generated const serialization.
`src/generated/transaction_set.rs`	Adds generated const serialization.
`src/generated/transaction_set_v1.rs`	Adds generated const serialization.
`src/generated/transaction_result.rs`	Adds generated const serialization.
`src/generated/transaction_result_set.rs`	Adds generated const serialization.
`src/generated/transaction_result_code.rs`	Adds generated const serialization.
`src/generated/transaction_meta_v1.rs`	Adds generated const serialization.
`src/generated/transaction_event_stage.rs`	Adds generated const serialization.
`src/generated/time_point.rs`	Adds generated const serialization.
`src/generated/thresholds.rs`	Adds generated const serialization.
`src/generated/threshold_indexes.rs`	Adds generated const serialization.
`src/generated/survey_message_response_type.rs`	Adds generated const serialization.
`src/generated/survey_message_command_type.rs`	Adds generated const serialization.
`src/generated/string64.rs`	Adds generated const serialization.
`src/generated/stellar_value_type.rs`	Adds generated const serialization.
`src/generated/soroban_credentials_type.rs`	Adds generated const serialization.
`src/generated/signer_key_type.rs`	Adds generated const serialization.
`src/generated/signature.rs`	Adds generated const serialization.
`src/generated/signature_hint.rs`	Adds generated const serialization.
`src/generated/set_options_result_code.rs`	Adds generated const serialization.
`src/generated/send_more.rs`	Adds generated const serialization.
`src/generated/scp_statement.rs`	Adds generated const serialization.
`src/generated/scp_statement_type.rs`	Adds generated const serialization.
`src/generated/sc_val_type.rs`	Adds generated const serialization.
`src/generated/sc_string.rs`	Adds generated const serialization.
`src/generated/sc_spec_type.rs`	Adds generated const serialization.
`src/generated/sc_spec_type_vec.rs`	Adds generated const serialization.
`src/generated/sc_spec_type_udt.rs`	Adds generated const serialization.
`src/generated/sc_spec_type_tuple.rs`	Adds generated const serialization.
`src/generated/sc_spec_type_result.rs`	Adds generated const serialization.
`src/generated/sc_spec_type_option.rs`	Adds generated const serialization.
`src/generated/sc_spec_type_map.rs`	Adds generated const serialization.
`src/generated/sc_spec_event_data_format.rs`	Adds generated const serialization.
`src/generated/sc_spec_entry_kind.rs`	Adds generated const serialization.
`src/generated/sc_nonce_key.rs`	Adds generated const serialization.
`src/generated/sc_meta_v0.rs`	Adds generated const serialization.
`src/generated/sc_meta_kind.rs`	Adds generated const serialization.
`src/generated/sc_error_type.rs`	Adds generated const serialization.
`src/generated/sc_error_code.rs`	Adds generated const serialization.
`src/generated/sc_env_meta_kind.rs`	Adds generated const serialization.
`src/generated/sc_contract_instance.rs`	Adds generated const serialization.
`src/generated/sc_bytes.rs`	Adds generated const serialization.
`src/generated/sc_address_type.rs`	Adds generated const serialization.
`src/generated/s_error.rs`	Adds generated const serialization.
`src/generated/revoke_sponsorship_type.rs`	Adds generated const serialization.
`src/generated/public_key_type.rs`	Adds generated const serialization.
`src/generated/price.rs`	Adds generated const serialization.
`src/generated/precondition_type.rs`	Adds generated const serialization.
`src/generated/pool_id.rs`	Adds generated const serialization.
`src/generated/payment_result_code.rs`	Adds generated const serialization.
`src/generated/payment_op.rs`	Adds generated const serialization.
`src/generated/operation_type.rs`	Adds generated const serialization.
`src/generated/operation_result_code.rs`	Adds generated const serialization.
`src/generated/offer_entry_flags.rs`	Adds generated const serialization.
`src/generated/message_type.rs`	Adds generated const serialization.
`src/generated/memo_type.rs`	Adds generated const serialization.
`src/generated/manage_sell_offer_result_code.rs`	Adds generated const serialization.
`src/generated/manage_offer_effect.rs`	Adds generated const serialization.
`src/generated/manage_data_result_code.rs`	Adds generated const serialization.
`src/generated/manage_data_op.rs`	Adds generated const serialization.
`src/generated/manage_buy_offer_result_code.rs`	Adds generated const serialization.
`src/generated/liquidity_pool_type.rs`	Adds generated const serialization.
`src/generated/liabilities.rs`	Adds generated const serialization.
`src/generated/ledger_upgrade_type.rs`	Adds generated const serialization.
`src/generated/ledger_key_data.rs`	Adds generated const serialization.
`src/generated/ledger_header_flags.rs`	Adds generated const serialization.
`src/generated/ledger_entry_type.rs`	Adds generated const serialization.
`src/generated/ledger_entry_changes.rs`	Adds generated const serialization.
`src/generated/ledger_entry_change_type.rs`	Adds generated const serialization.
`src/generated/ip_addr_type.rs`	Adds generated const serialization.
`src/generated/int256_parts.rs`	Adds generated const serialization.
`src/generated/int128_parts.rs`	Adds generated const serialization.
`src/generated/inflation_result_code.rs`	Adds generated const serialization.
`src/generated/host_function_type.rs`	Adds generated const serialization.
`src/generated/frozen_ledger_keys.rs`	Adds generated const serialization.
`src/generated/flood_demand.rs`	Adds generated const serialization.
`src/generated/flood_advert.rs`	Adds generated const serialization.
`src/generated/extension_point.rs`	Adds generated const serialization.
`src/generated/error_code.rs`	Adds generated const serialization.
`src/generated/envelope_type.rs`	Adds generated const serialization.
`src/generated/encrypted_body.rs`	Adds generated const serialization.
`src/generated/duration.rs`	Adds generated const serialization.
`src/generated/dont_have.rs`	Adds generated const serialization.
`src/generated/data_entry.rs`	Adds generated const serialization.
`src/generated/crypto_key_type.rs`	Adds generated const serialization.
`src/generated/create_contract_args.rs`	Adds generated const serialization.
`src/generated/create_account_result_code.rs`	Adds generated const serialization.
`src/generated/contract_id_preimage_type.rs`	Adds generated const serialization.
`src/generated/contract_executable_type.rs`	Adds generated const serialization.
`src/generated/contract_event_v0.rs`	Adds generated const serialization.
`src/generated/contract_event_type.rs`	Adds generated const serialization.
`src/generated/contract_data_durability.rs`	Adds generated const serialization.
`src/generated/contract_cost_type.rs`	Adds generated const serialization.
`src/generated/contract_cost_params.rs`	Adds generated const serialization.
`src/generated/contract_code_entry.rs`	Adds generated const serialization.
`src/generated/config_upgrade_set.rs`	Adds generated const serialization.
`src/generated/config_setting_id.rs`	Adds generated const serialization.
`src/generated/clawback_result_code.rs`	Adds generated const serialization.
`src/generated/claimant_v0.rs`	Adds generated const serialization.
`src/generated/claimant_type.rs`	Adds generated const serialization.
`src/generated/claimable_balance_id_type.rs`	Adds generated const serialization.
`src/generated/claimable_balance_flags.rs`	Adds generated const serialization.
`src/generated/claim_predicate_type.rs`	Adds generated const serialization.
`src/generated/claim_atom_type.rs`	Adds generated const serialization.
`src/generated/change_trust_result_code.rs`	Adds generated const serialization.
`src/generated/bump_sequence_result_code.rs`	Adds generated const serialization.
`src/generated/bucket_metadata.rs`	Adds generated const serialization.
`src/generated/bucket_list_type.rs`	Adds generated const serialization.
`src/generated/bucket_entry_type.rs`	Adds generated const serialization.
`src/generated/binary_fuse_filter_type.rs`	Adds generated const serialization.
`src/generated/auth.rs`	Adds generated const serialization.
`src/generated/auth_cert.rs`	Adds generated const serialization.
`src/generated/asset_type.rs`	Adds generated const serialization.
`src/generated/asset_code4.rs`	Adds generated const serialization.
`src/generated/asset_code12.rs`	Adds generated const serialization.
`src/generated/allow_trust_result_code.rs`	Adds generated const serialization.
`src/generated/account_merge_result_code.rs`	Adds generated const serialization.
`src/generated/account_flags.rs`	Adds generated const serialization.
`Cargo.toml`	Declares the `const` feature.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+use common::{tx_env_owned, tx_env_ref};
+
+#[test]
+fn ref_const_and_owned_encode_same() {


+    let r_xdr: [u8; R.const_xdr_len()] = R.const_to_xdr();
+    let o_xdr = o.to_xdr(Limits::none()).unwrap();
+    assert_eq!(r_xdr, o_xdr.as_slice());


leighmcculloch force-pushed the const-xdr-serialization branch from 0c7bcaa to 807385f Compare August 6, 2026 12:03

leighmcculloch changed the title ~~Add const XDR serialization on Ref types~~ Add const XDR serialization on View types Aug 6, 2026

leighmcculloch force-pushed the const-xdr-serialization branch from 807385f to 595950c Compare August 6, 2026 12:44

leighmcculloch force-pushed the generated-ref-types branch from 8834869 to 94be53b Compare August 25, 2026 05:17

leighmcculloch force-pushed the const-xdr-serialization branch from 595950c to 1510e34 Compare August 25, 2026 06:37

leighmcculloch force-pushed the generated-ref-types branch from 25bfab3 to afca646 Compare August 25, 2026 13:21

leighmcculloch force-pushed the const-xdr-serialization branch 2 times, most recently from 449b58f to 5f7c121 Compare August 25, 2026 13:33

leighmcculloch force-pushed the generated-ref-types branch from afca646 to 87358a1 Compare August 25, 2026 14:51

leighmcculloch force-pushed the const-xdr-serialization branch 2 times, most recently from 10d93ec to 2f59f68 Compare August 25, 2026 23:26

leighmcculloch force-pushed the const-xdr-serialization branch 2 times, most recently from 5737305 to 4012fd7 Compare September 7, 2026 00:06

leighmcculloch force-pushed the generated-ref-types branch from 0382122 to d4f6049 Compare September 7, 2026 00:14

leighmcculloch force-pushed the const-xdr-serialization branch 11 times, most recently from b93d88e to c28e3b4 Compare September 7, 2026 15:57

leighmcculloch added 4 commits September 8, 2026 02:11

put const XDR encoding on ConstWriter

6b51bfd

render const writer methods from the template

4db69f5

carry the Ref rename into the const writer

c0e860d

check const bytes against a hand-built owned

552499c

leighmcculloch force-pushed the const-xdr-serialization branch from c28e3b4 to 552499c Compare September 8, 2026 02:17

leighmcculloch added 4 commits September 8, 2026 13:08

shrink const xdr test suite to single envelope test that covers all b…

d2318f0

…ases

hand-write the scalar option and vec writers

b2bc25d

drop the const writer's residual module path

2c50bb3

simplify the const writer template

0c498db

leighmcculloch marked this pull request as ready for review September 8, 2026 06:23

Copilot AI balanced review requested due to automatic review settings September 8, 2026 06:23

Copilot started reviewing on behalf of leighmcculloch September 8, 2026 06:24 View session

Copilot AI reviewed Sep 8, 2026

View reviewed changes

leighmcculloch mentioned this pull request Sep 8, 2026

Add const XDR serialization #580

Merged

leighmcculloch closed this Sep 8, 2026

leighmcculloch mentioned this pull request Sep 10, 2026

Remove the single-file generator #582

Merged

Navigation Menu

Add const XDR serialization - #562

Add const XDR serialization#562
leighmcculloch wants to merge 8 commits into
generated-ref-typesfrom
const-xdr-serialization

leighmcculloch commented Jul 28, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

leighmcculloch commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Known limitations

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

leighmcculloch commented Jul 28, 2026 •

edited

Loading