Using Semantic Kernel with Web Applications.

[kengorman]

A Challenging Use Case We All May be Faced With

The console/test examples provided in the semantic-kernel source are clear and concise. In my two decades of .NET, I've never seen a more flexible and powerful library. However, an area I'm interested in discussing is how developers using Semantic Kernel are integrating the flow and lifetime of Kernel instances when dealing with web applications.

As an example, I build and maintain a .NET core application with an API, and a React front end.
If Alice logs in to the client-side application and initiates a chat, I forward the message to my API, then to my business layer which creates a Kernel instance and generates a reply.
What then should happen to that kernel instance on the server? Do I want to maintain that specific kernel in state (Redis?) and use a session id to know that this kernel is associated only to Alice? And how will that scale with multiple users? Or instead, should I save the conversation in state and create a new Kernel instance for the following requests and just keep appending the previous conversation to the next request sent to the model by referencing the session id? Related to memory and scaling, should I consider keeping my Semantic Kernel application in an entirely separate app service or function app?

Apologies for all the questions, in a sense this is an application architecture topic. But I think using Semantic Kernel in this manner (web application) no longer falls entirely in the realm of 'disconnected, stateless web applications' we've been creating in the past. Some mistakes may be made (possibly by me) along the way when implementing AI in this manner.
Thank you!

[jmservera]

LLM models are stateless by nature, so what you may want to maintain is the memory (depending on how your users interact with it); here's an example of Semantic Kernel using Redis for this.
Also, this video helped me a lot on integrating OpenAI in already existing web apps using .Net Core: https://youtu.be/jrNfKeGSuCg?t=981

[jmservera]

Check this other thread also: #7090 (comment)

[kengorman]

Hi Juan, thanks for the response! But for my needs, my question is based around handling Kernel instance state in a web application. I was luckily at the build conference in Seattle, and I attended this breakout session. By coincidence, I'm the person asking Vin Kamat a question towards the end of the session. In Vin's case, it looks like they're using SignalR to help with session management. Is this what Microsoft is recommending, or are there other patterns in the event SignalR is not an option? Thanks, Ken

[sophialagerkranspandey]

We recommend that you have a kernel per request (i.e. create a transient kernel).
@jmservera +1 what you'd added. The only thing you would need to persist per session is history which would be just text.