-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimized string.Replace(char, char) #67049
Changes from 4 commits
549d7c4
7dfe855
5232726
a442549
4e99ac4
30889ac
8627f6f
ed83650
5d92816
0ee90f4
0a7ca74
c65192a
35679cb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -968,13 +968,11 @@ private string ReplaceCore(string oldValue, string? newValue, CompareInfo? ci, C | |||||||||||||||||||||||||||
// | ||||||||||||||||||||||||||||
public string Replace(char oldChar, char newChar) | ||||||||||||||||||||||||||||
{ | ||||||||||||||||||||||||||||
if (oldChar == newChar) | ||||||||||||||||||||||||||||
return this; | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
int firstIndex = IndexOf(oldChar); | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
if (firstIndex < 0) | ||||||||||||||||||||||||||||
int firstIndex; | ||||||||||||||||||||||||||||
if (oldChar == newChar || (firstIndex = IndexOf(oldChar)) < 0) | ||||||||||||||||||||||||||||
{ | ||||||||||||||||||||||||||||
return this; | ||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
int remainingLength = Length - firstIndex; | ||||||||||||||||||||||||||||
string result = FastAllocateString(Length); | ||||||||||||||||||||||||||||
|
@@ -988,35 +986,56 @@ public string Replace(char oldChar, char newChar) | |||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
// Copy the remaining characters, doing the replacement as we go. | ||||||||||||||||||||||||||||
ref ushort pSrc = ref Unsafe.Add(ref Unsafe.As<char, ushort>(ref _firstChar), copyLength); | ||||||||||||||||||||||||||||
ref ushort pDst = ref Unsafe.Add(ref Unsafe.As<char, ushort>(ref result._firstChar), copyLength); | ||||||||||||||||||||||||||||
ref ushort pSrc = ref Unsafe.Add(ref Unsafe.As<char, ushort>(ref _firstChar), (nint)(uint)copyLength); | ||||||||||||||||||||||||||||
ref ushort pDst = ref Unsafe.Add(ref Unsafe.As<char, ushort>(ref result._firstChar), (nint)(uint)copyLength); | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. off-topic: gosh, the same line with raw pointers is basically ushort* pDst = ((ushort*)result._firstChar)[copyLength] "Safe" Unsafe is killing me 🤦♂️ There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here 🙈 It reads (and writes) like a mess. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Might become slightly more readable if the We could also expose an internal only |
||||||||||||||||||||||||||||
nint i = 0; | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
if (Vector.IsHardwareAccelerated && remainingLength >= Vector<ushort>.Count) | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Imagine a scenario where we are on an AVX2 machine and have This effectively pessimizes the support added to "backtrack" so the "trailing" elements can be handled via vectorization. It would likely be better to check that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good idea 👍🏻
I follow your idea (in general), but is There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, should be There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done with 8627f6f |
||||||||||||||||||||||||||||
{ | ||||||||||||||||||||||||||||
Vector<ushort> oldChars = new Vector<ushort>(oldChar); | ||||||||||||||||||||||||||||
Vector<ushort> newChars = new Vector<ushort>(newChar); | ||||||||||||||||||||||||||||
Vector<ushort> oldChars = new(oldChar); | ||||||||||||||||||||||||||||
Vector<ushort> newChars = new(newChar); | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
do | ||||||||||||||||||||||||||||
Vector<ushort> original; | ||||||||||||||||||||||||||||
Vector<ushort> equals; | ||||||||||||||||||||||||||||
Vector<ushort> results; | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
nint lengthToExamine = (nint)(uint)(remainingLength - Vector<ushort>.Count); | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
if (lengthToExamine > 0) | ||||||||||||||||||||||||||||
danmoseley marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||
{ | ||||||||||||||||||||||||||||
Vector<ushort> original = Unsafe.ReadUnaligned<Vector<ushort>>(ref Unsafe.As<ushort, byte>(ref pSrc)); | ||||||||||||||||||||||||||||
Vector<ushort> equals = Vector.Equals(original, oldChars); | ||||||||||||||||||||||||||||
Vector<ushort> results = Vector.ConditionalSelect(equals, newChars, original); | ||||||||||||||||||||||||||||
Unsafe.WriteUnaligned(ref Unsafe.As<ushort, byte>(ref pDst), results); | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
pSrc = ref Unsafe.Add(ref pSrc, Vector<ushort>.Count); | ||||||||||||||||||||||||||||
pDst = ref Unsafe.Add(ref pDst, Vector<ushort>.Count); | ||||||||||||||||||||||||||||
remainingLength -= Vector<ushort>.Count; | ||||||||||||||||||||||||||||
do | ||||||||||||||||||||||||||||
{ | ||||||||||||||||||||||||||||
original = Unsafe.ReadUnaligned<Vector<ushort>>(ref Unsafe.As<ushort, byte>(ref Unsafe.Add(ref pSrc, i))); | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
We can look at making that function public in .NET 8 to provide parity with the Vector64/128/256 functions. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, makes sense. Should this happen in this PR or separate (I image there are more places that could profit from that new API). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think adding it here is fine, we can then update other places to use it separately. We are actually using "local helpers" in a few of the other places already so the readability isn't bad there today. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Vector.LoadUnsafe doesn't have char overload so it still needs the Unsafe.As mess 😉 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Only when creating the We could also provide a specialized There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
👍 at least internal would be nice to have There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||||||||||||||||||||||||
equals = Vector.Equals(original, oldChars); | ||||||||||||||||||||||||||||
results = Vector.ConditionalSelect(equals, newChars, original); | ||||||||||||||||||||||||||||
Unsafe.WriteUnaligned(ref Unsafe.As<ushort, byte>(ref Unsafe.Add(ref pDst, i)), results); | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here with |
||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
i += Vector<ushort>.Count; | ||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||
while (i < lengthToExamine); | ||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||
while (remainingLength >= Vector<ushort>.Count); | ||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
for (; remainingLength > 0; remainingLength--) | ||||||||||||||||||||||||||||
{ | ||||||||||||||||||||||||||||
ushort currentChar = pSrc; | ||||||||||||||||||||||||||||
pDst = currentChar == oldChar ? newChar : currentChar; | ||||||||||||||||||||||||||||
// There are [0, Vector<ushort>.Count) elements remaining now. | ||||||||||||||||||||||||||||
// As the operation is idempotent, and we know that in total there are at least Vector<ushort>.Count | ||||||||||||||||||||||||||||
// elements available, we read a vector from the very end of the string, perform the replace | ||||||||||||||||||||||||||||
// and write to the destination at the very end. | ||||||||||||||||||||||||||||
// Thus we can eliminate the scalar processing of the remaining elements. | ||||||||||||||||||||||||||||
// We perform this operation even if there are 0 elements remaining, as it is cheaper than the | ||||||||||||||||||||||||||||
// additional check which would introduce a branch here. | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Can you quantify this? Even with good branch prediction it's still more expensive? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's hard to pour this statement into numbers, as with a BDN-benchmark the branch predictor will very likely do a great job (they got really smart over the last generation of cpus). In contrast to real-world usage I assume that it is more likely to have When I start working on Vector128/256 support for |
||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps worth adding an assert that current There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hm, I think in this case a test should fail? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tests cover these cases, so I don't see a need for the Debug.Assert -- but I'll add it of course if you want. runtime/src/libraries/Common/tests/Tests/System/StringTests.cs Lines 4566 to 4578 in 8ed8517
|
||||||||||||||||||||||||||||
pSrc = ref Unsafe.Add(ref pSrc, 1); | ||||||||||||||||||||||||||||
pDst = ref Unsafe.Add(ref pDst, 1); | ||||||||||||||||||||||||||||
i = (nint)(uint)this.Length - Vector<ushort>.Count; | ||||||||||||||||||||||||||||
original = Unsafe.ReadUnaligned<Vector<ushort>>(ref Unsafe.As<char, byte>(ref Unsafe.Add(ref _firstChar, i))); | ||||||||||||||||||||||||||||
equals = Vector.Equals(original, oldChars); | ||||||||||||||||||||||||||||
results = Vector.ConditionalSelect(equals, newChars, original); | ||||||||||||||||||||||||||||
Unsafe.WriteUnaligned(ref Unsafe.As<char, byte>(ref Unsafe.Add(ref result._firstChar, i)), results); | ||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||
else | ||||||||||||||||||||||||||||
{ | ||||||||||||||||||||||||||||
for (; i < (nint)(uint)remainingLength; ++i) | ||||||||||||||||||||||||||||
{ | ||||||||||||||||||||||||||||
ushort currentChar = Unsafe.Add(ref pSrc, i); | ||||||||||||||||||||||||||||
Unsafe.Add(ref pDst, i) = currentChar == oldChar ? newChar : currentChar; | ||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||
} | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
return result; | ||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isn't it against the guidelines to perform an assignment inside an
if
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know, but it gives nice machine code here 😉
It's about collapsing the epilogs for the first checks (
oldChar == newChar
, andfirstIndex < 0
).Maybe I think too complicated now, but the other option would be using
goto
for this.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At the very least it would be good to have a comment calling out the assignment why it is being done here.
Otherwise, at a glance it may look like a potential bug or comparison using
==
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But, the below is also the more "natural" pattern and more readable:
Ideally the JIT would handle such a pattern "correctly" and optimize it down accordingly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tannergooding so what's my action here?
(I'm leaning towards the last option, for perf-reasons -- except JIT issue will be fixed for .NET 7 😉)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think writing it naturally and filing an issue for the JIT is the best choice and don't expect the cost to be significant here.
If the cost is more significant, then adding a comment calling out the assignment and why as well as filing an issue for the JIT is the next best option.
If the issue is actually being fixed for .NET 7, that's all the more reason to do the first approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the codegen issue tracked by #8883?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code change here (back to where it was) with 30889ac
If I read the issue from the previous comment correct, so this should cover that case.