compiler: preemptively indirect large params/results instead of letting LLVM do it#5526
compiler: preemptively indirect large params/results instead of letting LLVM do it#5526jakebailey wants to merge 8 commits into
Conversation
|
FYI @deadprogram, this is the "horrorshow" from #5477 (comment), though it's less bad when split apart. |
| // Extract the params from the struct. | ||
| forwardParams := []llvm.Value{} | ||
| zero := llvm.ConstInt(b.ctx.Int32Type(), 0, false) | ||
| for i := range getParams(callback.Signature) { |
There was a problem hiding this comment.
I have to be at least a bit amused by the loops I'm replacing, which are all written completely differently despite being identical.
|
@jakebailey can you pull the latest |
createDefer builds an LLVM struct containing the deferred function and its arguments. createRunDefers separately reconstructs the same struct type before loading the fields. Keep each LLVM value together with its type while building a deferred call record, and load all argument fields through one helper. This removes the duplicate lists of field types and field loads.
Map, channel, index, and goroutine lowering each create allocas, store SSA values into them, load results, and emit lifetime intrinsics. Add helpers for these operations and use runtimeValueResult for runtime calls that write a value and an optional comma-ok result. The generated LLVM IR is unchanged.
getFunction and getLLVMFunctionType duplicate the construction of LLVM result types for functions with zero, one, or multiple results. Move this code to getLLVMResultType. Also split the existing parameter expansion code into expandDirectFormalParamType so callers can request the current flattened parameter types directly.
createFunctionCall and createGo currently lower arguments before they determine whether the call is direct, an interface invoke, or through a function value. Resolve the callee, function type, and context first, then append the arguments in the same order as before. This does not change the generated LLVM IR.
createInstruction records each LLVM value in locals, emits stack object tracking, and constructs function returns inline. Move these operations to setValue and createReturn. This keeps the instruction switch from having to know how an SSA value or function result is emitted.
Add compiler coverage for large aggregate parameters and results, including function values, interfaces, maps, channels, selects, deferred calls, goroutines, phis, and multiple results.
LLVM ComputeValueVTs recursively expands arrays and structs into one value type per scalar leaf. SelectionDAG call lowering allocates data structures proportional to this count, which makes very large values exhaust memory or crash LLVM. Count scalar leaves and use pointers for internal parameters and results when the count exceeds 1024. A result pointer is the first parameter, and aggregate parameters point to read-only memory. Exported function types are unchanged. Keep these SSA values in memory and copy them with memcpy when needed. Handle calls, interfaces, maps, channels, selects, defers, goroutines, phis, and multiple results. Update the expected compiler IR and re-enable the native compress/flate tests.
90d5bcd to
12d73ea
Compare
|
Here is edited version of the useful parts of some generated feedback: createIndirectStorage calls runtime.alloc (GC-tracked heap allocation) rather than using stack allocas. For frequently-called functions with large params, this adds GC pressure. The comment in the PR description says "The indirection happens on the stack where possible" but looking at the code, createIndirectStorage always calls runtime.alloc. The getValueStorage helper does use createTemporaryAlloca for small types but falls through to getValuePointer -> createIndirectStorage for large ones. Is there a reason stack allocation (alloca) wasn't used for the indirect storage? It seems like for values that don't escape, a stack alloca would be more efficient. isLLVMValueType filtering determines whether a Go type maps to a "pure" LLVM value type. Types containing interfaces, maps, slices etc. are excluded from indirect treatment because their LLVM representation is already a small struct of pointers. However, a struct containing a large array AND an interface field would return false from isLLVMValueType and skip indirection entirely, even though the array component is huge. Is this intentional? prependIndirectResult in defer replay, when replaying deferred calls, prependIndirectResult allocates storage for the result, but deferred calls discard their return values. This allocates memory that's never read. It's harmless but wasteful. getGoroutineCallArgument correctly copies indirect params to new storage (since the goroutine outlives the caller), but uses createIndirectStorage (heap alloc). For go readLargeValue(value), this means an extra 1025-byte heap allocation + memcpy. This is correct but worth noting for performance. |
So, yes, but these are allocs that would normally have crashed the compiler, so I'm not sure that I'm actually regressing much?
I don't think this is true, but maybe I can add a test.
Yes, but I don't think there's a way around this one.
IIRC this happened before, it's just that I'm doing it earlier, right? |
|
I have some fixes I'll push in, but, I have to compile LLVM 22 first so that'll take me a bit 😄 |
2 participants
#5477 revealed a preexisting bug(?) with large types.
When there are too many parameters or results (common! it's not like every result can fit in registers), LLVM indirects things. It turns out that the result value it makes is a flattened struct, one element per struct field and array element. So then each of those elements then gets its own LLVM value to construct it, which means that it scales up worse and worse the more array elements there are. In SelectionDAG this also hits a hard representation limit:
SDNode::NumValuesis anunsigned short, and overflows at 65,536 flattened values.https://go.dev/cl/707355 added a struct with a massive array that's passed by value:
This makes the compiler very unhappy. In this case,
tokenshas 65,858 scalar leaves, so direct lowering can crash LLVM in addition to being very slow.The only way I can think to avoid this is to just do the indirection ourselves before handing it over to LLVM so that it doesn't hit this limit. TinyGo now does this for internal aggregate params/results above 1024 flattened scalar leaves, which is intentionally well below LLVM's 65,535-value representation limit to avoid the compile-time/memory cliff before the crash boundary. Exported function types are unchanged.
This is a pretty invasive change. I tried to split this stack into enough individual (and I think reasonable) commits with abstractions to make this reviewable one commit at a time.
Temporary backing storage goes through TinyGo's normal allocation path: non-escaping bounded allocations may still be stack-promoted by the allocation optimizer, while escaping storage remains heap-backed. It's unlikely anyone will actually hit this code, though, given before they would have been very much timing things out?