Skip to content

Add request splitting feature #77

Open
@matthewhanson

Description

@matthewhanson

It's useful to be able to take a large request with many pages into smaller requests so they can be run in parallel or asynchronously. Datetime is convenient way to split up requests.

The code in the Cirrus stac-api feeder code is one example of doing this with sat-search.

The ItemSearch class should have a function(s) to return an array of Request objects. The simplest of these can just take in a num_requests parameter that will take the datetime range and split it into num_requests requests.

A more complicated function would take in the approximate number of Items to return in each request. A series of requests can be made to get number of hits to divide up the requests roughly equally (as in the above code link).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions