[API Proposal]: Get a Read-only view of a StringBuilder body and clear the StringBuilder.

[jkoritzinsky]

Background and motivation

The Roslyn and runtime teams are looking at improving the experience of writing formatted text, primarily around source generators. An important feature for Roslyn is a mechanism to, without a copy, get a read-only view of any text that no one else can write to. The lack of an API to provide this has been blocking dotnet/roslyn#61326.

The Roslyn team has mentioned recently that they are okay with a non-optimal copy-based API for downlevel scenarios as long as there is a no-copy API on the horizon. This proposal provides such an API.

The following is updated from #97570 (comment)

API Proposal

namespace System.Text;

public class StringBuilder
{
    public StringBuilder MoveChunksToStringBuilder();
}

This API will create a new StringBuilder instance initialized to the same state as the original instance, and then reset the fields of the original instance back to the same state as a freshly created instance.

This clearing behavior is in contrast to the behavior when Clear is called, where the internal array is not thrown away.

API Usage

Roslyn would use it with an API similar to the following:

public abstract class SourceText
{
    public static SourceText DrainFrom(StringBuilder stringBuilder)
    {
        return new StringBuilderText(stringBuilder.MoveChunksToStringBuilder());
    }
}

(StringBuilderText implementation, not able to wrap user-provided StringBuilders until this StringBuilder API is added)

New StringBuilder versus array

An allocation is needed somewhere so that chunk enumeration can start multiple times after the original StringBuilder is cleared.

Allocating a new StringBuilder is cheaper than allocating an array of chunks, both because the array would be variable sized and because populating the array requires following the whole linked list chain. In comparison, the implementation of MoveChunksToStringBuilder needs only allocate a new StringBuilder and set a handful of fields directly in the old and new instances.

Allocating a new StringBuilder is also a more composable operation, opening the door to any use case where you need a cheap way to be sure that no one else can modify the contents of the StringBuilder that you hold, even if your use case goes beyond chunk enumeration (possibly involving your own mutations).

Naming

MoveChunks is preferred over Move or Drain because it is clearer that the chunks are leaving the current instance and landing as-is in the new instance without being compacted into a single chunk. MoveChunks is preferred over DrainChunks because Move represents the O(1) operation on the chunks that is happening, whereas Drain sounds more involved. (And in prior art, Drain is in fact more costly than Move; ImmutableArray<T>.Builder.DrainToImmutable freshly allocates and copies elements when the presizing doesn't match.)

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

The Roslyn and runtime teams are looking at improving the experience of writing formatted text, primarily around source generators. An important feature for Roslyn is a mechanism to, without a copy, get a read-only view of any text that no one else can write to. The lack of an API to provide this has been blocking dotnet/roslyn#61326.

The Roslyn team has mentioned recently that they are okay with a non-optimal copy-based API for downlevel scenarios as long as there is a no-copy API on the horizon. This proposal provides such an API.

API Proposal

namespace System.Text;

public class StringBuilder
{
    public System.Buffers.ReadOnlySequence<char> ToReadOnlySequenceAndClear();
}

This API will create a read-only sequence over the chunks in the StringBuilder's content, reset the internal buffer of the builder to a new backing array, and remove references to previous chuncks.

This clearing behavior is in contrast to the behavior when Clear is called, where the internal array is not thrown away.

API Usage

ReadOnlySequence<char> GetReadOnlyView(StringBuilder builder)
{
    return builder.ToReadOnlySequenceAndClear();
}

Alternative Designs

Instead of enhancing StringBuilder as-is, we could make a type that wraps StringBuilder and provides a more fully-featured API set for the formatted text writing and provides this sort of API (built on top of StringBuilder.GetChunks() and the fact that it would own the StringBuilder instance).

Risks

No response

Author:	jkoritzinsky
Assignees:	-
Labels:	api-suggestion, area-System.Runtime
Milestone:	-

[jkoritzinsky]

cc: @CyrusNajmabadi @stephentoub

[stephentoub]

Related to #87362

(As with that one, this would necessitate moving ReadOnlySequence into corelib.)

[MichalPetryka]

I don't see how the Clear behaviour here makes sense, seems like you effectively want it to alloc a new StringBuilder instead.

[CyrusNajmabadi]

I don't see how the Clear behaviour here makes sense, seems like you effectively want it to alloc a new StringBuilder instead.

We want to transfer ownership of hte underlying data. That's only safe if after the operation the SB itself doesn't point at the data (hence, it is "cleared").

[lambdageek]

I don't see how the Clear behaviour here makes sense, seems like you effectively want it to alloc a new StringBuilder instead.

We want to transfer ownership of hte underlying data. That's only safe if after the operation the SB itself doesn't point at the data (hence, it is "cleared").

I'm still unclear on the scenario. Either there's a single reference to the string builder instance - in which case it's ok to have 2 separate operations - one that gives a view of the underlying chunks and a separate clear operation (different from the existing Clear which mutates the internal array).

Or there are multiple references - in which case don't they already need some kind of coordination mechanism? Otherwise how could they possibly call Append twice and have any confidence that they're still writing to the same buffer.

It seems the "and clear" part can be separate.

[jkoritzinsky]

I don't see how the Clear behaviour here makes sense, seems like you effectively want it to alloc a new StringBuilder instead.

We want to transfer ownership of hte underlying data. That's only safe if after the operation the SB itself doesn't point at the data (hence, it is "cleared").

I'm still unclear on the scenario. Either there's a single reference to the string builder instance - in which case it's ok to have 2 separate operations - one that gives a view of the underlying chunks and a separate clear operation (different from the existing Clear which mutates the internal array).

This is the expected scenario. In the Roslyn source generator scenario, the user would build up the source in the StringBuilder and then pass the resulting sequence between phases (so the StringBuilder would be single-use or a pooled instance, never saved in a field for later usage based on previous state).

Or there are multiple references - in which case don't they already need some kind of coordination mechanism? Otherwise how could they possibly call Append twice and have any confidence that they're still writing to the same buffer.

Today, there's not a good mechanism for coordination when using the GetChunks method while also clearing the buffer, which is what this API effectively enables.

It seems the "and clear" part can be separate.

We could just introduce a ResetBuffers() that would release all the internal buffers instead of what "Clear" does and then Roslyn could provide the "create a ReadOnlySequence and reset" mechanism on top (possibly internally).

[reflectronic]

A similar API #72625 went from "ToImmutableAndClear" to "DrainToImmutable" in API review, perhaps this 'drain' naming pattern should be used here as well--DrainToReadOnlySequence?