Skip to content

stream: csAttempt has 4 scattered bools wasting 16B and causing false sharing with mu #9347

Description

@gidotencate

Summary

csAttempt (in stream.go) has four bool fields interspersed among pointer- and interface-sized fields. Each bool forces alignment padding before the next field, wasting 16 bytes total. Additionally, decompressorSet at offset 160 shares a 64-byte cache line with mu at offset 164, causing false-sharing overhead on every mu.Lock() call under concurrent load.

This is the same pattern fixed for clientStream in #9280 / #9281.

Layout (DWARF- and runtime-verified)

Current size: 232B → 240B allocator size class

Field Offset Padding after Guarded by mu?
finished 120 7 B (before 16B-aligned interface) Yes
decompressorSet 160 3 B (before sync.Mutex, align 4) No
allowTransparentRetry 224 0 B (consecutive with drop) No
drop 225 6 B tail No

decompressorSet (offset 160) and mu (offset 164) are on the same 64-byte cache line (bytes 128–191).

After grouping bools at tail

Verified by constructing an equivalent mirror struct and calling unsafe.Sizeof:

Metric Current Proposed
Struct size 232 B 216 B
Allocator size class 240 B 224 B
Savings per csAttempt 16 B

Cache-line false-sharing benchmark

One goroutine spins mu.Lock()/mu.Unlock() (the hot path in finish()) while N stressor goroutines spin-write decompressorSet (the bool sharing its cache line). decompressorSet is written once per stream in recvMsg(), without holding mu. Under load, these accesses overlap across concurrent streams.

goos: linux / goarch: amd64 / cpu: AMD Ryzen 7 7800X3D
benchtime=3s, count=6, cpu=16

BenchmarkLayoutMuLockCSAttempt/stressors=0-16     3.94 ns/op   (baseline)
BenchmarkLayoutMuLockCSAttempt/stressors=1-16    18.02 ns/op   4.6× slower
BenchmarkLayoutMuLockCSAttempt/stressors=2-16    37.93 ns/op   9.6× slower
BenchmarkLayoutMuLockCSAttempt/stressors=4-16    78.34 ns/op  19.9× slower

Impact

  • Allocator: 16 B saved per RPC attempt, unconditionally. Goes into the 224 B size class instead of 240 B — better span utilisation and ~6.7% less GC-scanned heap per object.
  • False sharing: Eliminating decompressorSet from mu's cache line reduces the cache-miss penalty on mu.Lock() whenever another goroutine is receiving the first response message on a concurrent attempt.

Proposed fix

Group all 4 bools at the tail of csAttempt, split into // Not guarded by mu (decompressorSet, allowTransparentRetry, drop) and // Guarded by mu (finished) sections with per-field comments, following the same convention as #9281. Add a TestCSAttemptSize assertion to guard against regressions.

/cc @easwars @mbissa

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions