Skip to content

Bound PartitionInfo snapshot buffers via config_node_snapshot_buffer_size_max - #18580

Merged
JackieTien97 merged 1 commit into
apache:masterfrom
CRZbulabula:improve/bounded-confignode-snapshot-buffers
Sep 4, 2026
Merged

JackieTien97 merged 1 commit into
apache:masterfrom
CRZbulabula:improve/bounded-confignode-snapshot-buffers

Conversation

@CRZbulabula

@CRZbulabula CRZbulabula commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Taking a ConfigNode snapshot always allocated a fixed 32MB heap buffer (PARTITION_TABLE_BUFFER_SIZE hard-coded in PartitionInfo), regardless of the actual partition table size. On edge deployments the default heap is only 224MB and ConfigNode snapshots are triggered frequently, so the fixed allocation can cause GC spikes when the partition table is small.

Changes

  • Add SnapshotStreamFactory + ReusableBufferedOutputStream (node-commons) providing capped and reusable buffered streams for snapshot files. The buffer size is bounded by the new config_node_snapshot_buffer_size_max parameter (default 4MB; 0 disables buffering), and write buffers are pooled per thread via ThreadLocal + SoftReference.
  • PartitionInfo: replace the fixed 32MB buffer with the factory. The read buffer is sized from the file size and capped by the parameter, and the write buffer is capped and reused across snapshots on the same thread.
  • Edge distribution: config_node_snapshot_buffer_size_max=8192, documented in iotdb-system.properties.template.
  • Scope is confined to PartitionInfo: no other snapshot code paths are changed.

Tests

  • New SnapshotStreamFactoryTest (buffer cap, disabled-buffering passthrough, round-trip with buffer wrap-around, large-write bypass, buffer reuse, idempotent close).
  • New ConfigNodeConfigTest (default alignment with the factory cap, setter validation).
  • PartitionInfoTest runs all snapshot round-trips with a 64KB buffer cap, including a snapshot larger than the write buffer.

@JackieTien97 JackieTien97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because the new stream-based template snapshot recovery regresses an accepted vector-schema form. The remaining inline comments cover a low-priority buffer-cap contract issue and cleanup/documentation items.

@CRZbulabula
CRZbulabula force-pushed the improve/bounded-confignode-snapshot-buffers branch from 109f543 to 4401337 Compare September 4, 2026 06:25
@CRZbulabula CRZbulabula changed the title Bound ConfigNode snapshot buffers and avoid whole-file heap reads Size PartitionInfo snapshot buffer from the file size instead of a fixed 32MB Sep 4, 2026
@CRZbulabula
CRZbulabula force-pushed the improve/bounded-confignode-snapshot-buffers branch from 4401337 to 864a759 Compare September 4, 2026 06:32
@CRZbulabula CRZbulabula changed the title Size PartitionInfo snapshot buffer from the file size instead of a fixed 32MB Bound PartitionInfo snapshot buffers via config_node_snapshot_buffer_size_max Sep 4, 2026
…size_max

- Add SnapshotStreamFactory (node-commons) providing capped and reusable
  buffered streams for ConfigNode snapshot files, bounded by the new
  config_node_snapshot_buffer_size_max parameter (default 4MB; 0 disables
  buffering). Write buffers are pooled per thread via SoftReference.
- PartitionInfo routes its snapshot load/save streams through the factory,
  replacing the previous fixed 32MB buffer. The read buffer is sized from
  the file size and the write buffer is capped by the configured limit.
- Edge distribution caps the snapshot buffer at 8KB.
- Scope is confined to PartitionInfo: no other snapshot code paths are
  changed.
@CRZbulabula
CRZbulabula force-pushed the improve/bounded-confignode-snapshot-buffers branch from 864a759 to 41525a7 Compare September 4, 2026 06:40
@JackieTien97
JackieTien97 merged commit fd39898 into apache:master Sep 4, 2026
39 checks passed
@CRZbulabula
CRZbulabula deleted the improve/bounded-confignode-snapshot-buffers branch September 4, 2026 08:18
JackieTien97 pushed a commit that referenced this pull request Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants