Skip to content

All the user to specify the stream argument of Agent._run_model somewhere #371

@nsfinkelstein

Description

@nsfinkelstein

Enhancement Description

The default orchestrator calls Agent._run_model without a streaming argument: https://github.com/PrefectHQ/ControlFlow/blob/main/src/controlflow/orchestration/orchestrator.py#L375.

In effect, the default of True is always used unless the user constructs a new Orchestrator.

Use Case

Users may not be able to use a specific LLM API in streaming mode, but may have authorization to it otherwise.

Proposed Implementation

A stream parameter can be passed to the agent at construction and set on the object, and then Agent._run_model function could then use then use that value. Arguably whether or not an agent should be used in streaming mode is an Agent-level decision rather than a call-level decision, so it may make sense to specify it at construction rather than in the function that triggers that LLM call.

https://github.com/PrefectHQ/ControlFlow/blob/main/src/controlflow/agents/agent.py#L284

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementEnhance an existing feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions