Skip to content

stream: serverStream has 2 scattered bools wasting 8B and causing false sharing with mu #9349

Description

@gidotencate

Summary

serverStream (in stream.go) has two bool fields placed between aligned fields. Each bool forces alignment padding before the next field, wasting 8 bytes total. Additionally, serverHeaderBinlogged at offset 240 shares a 64-byte cache line with mu at offset 244, causing false-sharing overhead on every mu.Lock() under concurrent load.

This is the same pattern fixed for clientStream in #9280 / #9281.

Layout (DWARF- and runtime-verified)

Current size: 256B → 256B allocator size class

Field Offset Padding after Guarded by mu?
recvFirstMsg 168 7 B (before 8B-aligned int) No — "set after the first message is received"
serverHeaderBinlogged 240 3 B (before sync.Mutex, align 4) No — "doesn't need to be synchronized"

mu at offset 244 "protects trInfo.tr after the service handler runs." serverHeaderBinlogged is written in SendHeader() and Send() on every binlogged stream — frequently, and without holding mu.

serverHeaderBinlogged (offset 240) and mu (offset 244) share the same 64-byte cache line (bytes 192–255).

After grouping bools at tail

Verified by constructing an equivalent mirror struct and calling unsafe.Sizeof:

Metric Current Proposed
Struct size 256 B 248 B
Allocator size class 256 B 256 B (no change)
Savings per serverStream 8 B

Unlike csAttempt (#9347) and addrConnStream (#9348), this fix does not drop a size class. The primary benefit is eliminating the cache-line conflict between serverHeaderBinlogged and mu.

Cache-line false-sharing benchmark

One goroutine spins mu.Lock()/mu.Unlock() while N stressor goroutines spin-write serverHeaderBinlogged (the bool sharing cache line 3 with mu):

goos: linux / goarch: amd64 / cpu: AMD Ryzen 7 7800X3D
benchtime=3s, count=6, cpu=16

BenchmarkLayoutMuLockServerStream/stressors=0-16     3.94 ns/op   (baseline)
BenchmarkLayoutMuLockServerStream/stressors=1-16    20.61 ns/op   5.2× slower
BenchmarkLayoutMuLockServerStream/stressors=2-16    46.11 ns/op  11.7× slower
BenchmarkLayoutMuLockServerStream/stressors=4-16    81.89 ns/op  20.8× slower

In production, serverHeaderBinlogged is written during every SendHeader()/Send() call on binlog-enabled streams, while mu is locked in the stream cleanup path. The penalty is real whenever server-side sends and stream finalisation overlap.

Impact

  • Memory: 8 B saved per serverStream. Stays in the 256 B size class, but reduces the internal struct footprint.
  • False sharing: Moving serverHeaderBinlogged off mu's cache line eliminates the cache-miss penalty on mu.Lock() caused by concurrent unsynchronised writes to serverHeaderBinlogged.

Proposed fix

Group both bools at the tail of serverStream, adding a // Not guarded by mu comment section with per-field notes, following the pattern in #9281. Add a TestServerStreamSize assertion to guard against regressions.

/cc @easwars @mbissa

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions