Add bulk consumption helper/method for stream processing in client python #949
Description
Problem to Solve
The current python client offers a helper for stream consumption (process_message), processing events one by one. This one-at-a-time approach is causing issues with certain stream connectors:
- Rate Limit Issues: Sending too many requests to external solutions (e.g., one indicator creation equals one external request).
- Performance Issues: Handling indicators individually is inefficient, especially when many external solutions support bulk upload APIs.
Current Workaround
N/A
Proposed Solution
Introduce a new helper or method for bulk event consumption. This should allow specification of:
- Maximum number of events to retrieve at once.
- A timeout to return control if the maximum count isn't reached in a timely manner.