Is it possible too run a pipe line considering Rate limit of LLMs. #17697
Unanswered
Sparshsing
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
When running a pipeline involving LLMs (for ex. TitleExtractor, SummaryExtractor), is there a way that only a certain number of nodes can be processed in a minute?
We could first extract the nodes from documents and then run them in batches as per rate limit. But is there a standard functionality for this, as even node parsers could sometimes use LLMs?
Beta Was this translation helpful? Give feedback.
All reactions