[Question] [Microsoft.Extensions.AI] Stateful tools

[raffaeler]

I am trying to figure out how to manage stateful tools.

Scenario

• I register each tool in the DI as Transient.
• A conversation starts determining a first level of context that is valid only for this conversation
• The conversation and the tool work is long-running and can be persisted and rehydrated in memory later on as it needs some human response that can take a long time
• At a certain time, the LLM request to invoke a tool. But it can need to invoke it multiple times. As it is Transient, it will be instantiated right before its invocation. Being able to keep a separate tool state for each instance can be important in some case. For example the tool may invoke another LLM and keep the token usage in its state.
• Since tools can be invoked in parallel, working on different instances is necessary for stateful tools.

Problem

In this scenario I set the state right after creating the tool instance, but the list of AIFunction that I provided to the LLM at the beginning captured the tool instance, therefore I have to re-create the AIFunction every time I need to invoke the tool.

Possible solutions?

At this point I see two possible solutions:

• adding an object to the tool signature (which must be invisible to the LLM) so that I can provide the context when I invoke the tool manually
  or
• adding the possibility to specify the tool instance in the AIFunction.AIInvokeAsync method.

Am I missing other solutions?

Thanks