[stdlib] Optimize compiler times by prevent huge loop unrolling when filling inline arrays. #4046
Conversation
|
Could we make unroll_threshold into a parameter that's defaulted to something a bit more reasonable, like 64? That way people can customize it, since unrolling 1000 iterations is already icache pollution. |
| @parameter | ||
| for i in range(size): | ||
| for i in range(unrolled): |
There was a problem hiding this comment.
We probably want to unroll the inner loop, not the outer one.
There was a problem hiding this comment.
Done: msaelices@bb401c4
| @@ -140,8 +140,21 @@ struct InlineArray[ | |||
| _inline_array_construction_checks[size]() | |||
| __mlir_op.`lit.ownership.mark_initialized`(__get_mvalue_as_litref(self)) | |||
|
|
|||
| alias unroll_threshold = 1000 | |||
| alias unrolled = size // unroll_threshold | |||
There was a problem hiding this comment.
math.align_down makes intent more clear.
There was a problem hiding this comment.
Sure: msaelices@bb401c4
Unroll the inner loop Signed-off-by: Manuel Saelices <[email protected]>
… a parameter Signed-off-by: Manuel Saelices <[email protected]>
msaelices
force-pushed
the
inlinearray-compiler-slowness
branch
from
57c599e to
edadb98
Compare
Done: msaelices@edadb98 |
|
@owenhilyard Thanks for the review. Could you please take another look? |
| @@ -131,7 +132,7 @@ struct InlineArray[ | |||
|
|
|||
| @always_inline | |||
| @implicit | |||
| fn __init__(out self, fill: Self.ElementType): | |||
| fn __init__[batch_size: Int = 100](out self, fill: Self.ElementType): | |||
There was a problem hiding this comment.
I think this should be a power of 2 in order to keep the chunks aligned well. 32 or 64 is good enough to not bloat the icache but still be fast, especially since processing the loop variable will happen in parallel with the vector ops on any superscalar processor, which is most of them at this point.
There was a problem hiding this comment.
If it's an inline array of bytes, then 100 will need to do some really odd things with instructions, especially on AVX512 since it will move 64, then 32, then 4.
There was a problem hiding this comment.
Done: msaelices@212caea
Signed-off-by: Manuel Saelices <[email protected]>
The
InlineArray(fill=...)constructor loops for the initialization of all the pointee items in the array in comptime.This makes the compiler really slow when the size of the inline array is >2k, and even shows a warning if the size is bigger than 64k:
IMO, I don't find useful to have 64k
init_pointee_copylines when we can unroll by batches, keeping the compiler fast and the actual runtime still fast.