How can I stream to a frontend using SSE? #1313
Replies: 4 comments 4 replies
-
Thanks for the issue! Could you expand a little bit on what SSE is? |
Beta Was this translation helpful? Give feedback.
-
Yes, I'm sorry about that, SSE stands for server-sent events, which is, from what I understand, how ChatGPT streams information to its UI. |
Beta Was this translation helpful? Give feedback.
-
Got it. Check if my understanding is correct: on the server side (Python) you would have to create a data stream as part of the response object with mime type "text/event-stream"? So, something similar to: @app.route("/stream")
def stream():
def generate():
while True:
yield "data: {}\n\n".format("Hello, World!")
return Response(generate(), mimetype="text/event-stream") The So the only thing we need to produce from the autogen side is a Python generator that runs in its own thread listening to a concurrent pipe and yields data from that pipe. I think this can be achieved by registering a middleware (#1240) to the agent you want to stream response from, and every time you catch its response in the middleware you then asynchronously writes the response to the concurrent pipe the generator is listening to. This way your Flaks code you can call the agent and directly returns the generator in your Response object. Of course you need to close the concurrent pipe in the middleware once you have done with it. You are welcome to contribute to the effort there with an implementation of a middleware that takes input a concurrent pipe, writes response message to that pipe, and has a method to close it once finished. |
Beta Was this translation helpful? Give feedback.
-
@ekzhu Another idea might be to try capturing and streaming the stdout. |
Beta Was this translation helpful? Give feedback.
-
Describe the issue
I would like to host Autogen on Flask, end then stream the responses to a frontend using SSE. I am not sure how to do that. SSE can return streaming messages if it uses a generator, but I am not sure how to get the code out from the deep innards of Autogen (client.py, conversable_agent.py) and bring it up to the top so Flask can return that stream. Any help on this would be much appreciated. @ragyabraham
Steps to reproduce
No response
Screenshots and logs
No response
Additional Information
No response
Beta Was this translation helpful? Give feedback.
All reactions