Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,24 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

### Added

- Spatial search support for collections via `bbox` parameter on `/collections` endpoint. Collections are now indexed with a `bbox_shape` field (GeoJSON polygon) derived from their spatial extent for efficient geospatial queries when created or updated. [#481](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/481)
- Introduced SFEOS Tools (`sfeos_tools/`) - An installable Click-based CLI package for managing SFEOS deployments. Initial command `add-bbox-shape` adds the `bbox_shape` field to existing collections for spatial search compatibility. Install with `pip install sfeos-tools[elasticsearch]` or `pip install sfeos-tools[opensearch]`. [#481](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/481)
- CloudFerro logo to sponsors and supporters list [#485](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/485)
- Latest news section to README [#485](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/485)

### Changed

### Fixed

[v6.5.1] - 2025-09-30
## [v6.5.1] - 2025-09-30

### Fixed

- Issue where token, query param was not being passed to POST collections search logic [#483](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/483)
- Issue where datetime param was not being passed from POST collections search logic to Elasticsearch [#483](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/483)
- Collections search tests to ensure both GET /collections and GET/POST /collections-search endpoints are tested [#483](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/483)

[v6.5.0] - 2025-09-29
## [v6.5.0] - 2025-09-29

### Added

Expand Down
76 changes: 74 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ The following organizations have contributed time and/or funding to support the

<div style="max-height: 200px; overflow-y: auto; padding: 10px; border: 1px solid #eaecef; border-radius: 6px; margin-bottom: 16px;">

- 10/04/2025: The [CloudFerro](https://cloudferro.com/) logo has been added to the sponsors and supporters list above. Their sponsorship of the ongoing collections search extension work has been invaluable. This is in addition to the many other important changes and updates their developers have added to the project.
- **10/12/2025:** Collections search **bbox** functionality added! The collections search extension now supports bbox queries. Collections will need to be updated via the API or with the new **[SFEOS-tools](#sfeos-tools-cli)** CLI package to support geospatial discoverability. Thanks again to **CloudFerro** for their sponsorship of this work!
- **10/04/2025:** The **[CloudFerro](https://cloudferro.com/)** logo has been added to the sponsors and supporters list above. Their sponsorship of the ongoing collections search extension work has been invaluable. This is in addition to the many other important changes and updates their developers have added to the project.

</div>

Expand Down Expand Up @@ -105,6 +106,7 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI
- [Interacting with the API](#interacting-with-the-api)
- [Configure the API](#configure-the-api)
- [Collection Pagination](#collection-pagination)
- [SFEOS Tools CLI](#sfeos-tools-cli)
- [Ingesting Sample Data CLI Tool](#ingesting-sample-data-cli-tool)
- [Elasticsearch Mappings](#elasticsearch-mappings)
- [Managing Elasticsearch Indices](#managing-elasticsearch-indices)
Expand Down Expand Up @@ -160,9 +162,21 @@ These endpoints support advanced collection discovery features including:
- Collections are matched if their temporal extent overlaps with the provided datetime parameter
- This allows for efficient discovery of collections based on time periods

- **Spatial Filtering**: Filter collections by their spatial extent using the `bbox` parameter
- Example: `/collections?bbox=-10,35,40,70` (finds collections whose spatial extent intersects with this bounding box)
- Example: `/collections?bbox=-180,-90,180,90` (finds all collections with global coverage)
- Supports both 2D bounding boxes `[minx, miny, maxx, maxy]` and 3D bounding boxes `[minx, miny, minz, maxx, maxy, maxz]` (altitude values are ignored for spatial queries)
- Collections are matched if their spatial extent (stored in the `extent.spatial.bbox` field) intersects with the provided bbox parameter
- **Implementation Note**: When collections are created or updated, a `bbox_shape` field is automatically generated from the collection's spatial extent and indexed as a GeoJSON polygon for efficient geospatial queries
- **Migrating Legacy Collections**: Collections created before this feature was added will not be discoverable via bbox search until they have the `bbox_shape` field added. You can either:
- Update each collection via the API (PUT `/collections/{collection_id}` with the existing collection data)
- Run the migration tool (see [SFEOS Tools CLI](#sfeos-tools-cli) for installation and connection options):
- `sfeos-tools add-bbox-shape --backend elasticsearch --no-ssl`
- `sfeos-tools add-bbox-shape --backend opensearch --host db.example.com --no-ssl`

These extensions make it easier to build user interfaces that display and navigate through collections efficiently.

> **Configuration**: Collection search extensions (sorting, field selection, free text search, structured filtering, and datetime filtering) for the `/collections` endpoint can be disabled by setting the `ENABLE_COLLECTIONS_SEARCH` environment variable to `false`. By default, these extensions are enabled.
> **Configuration**: Collection search extensions (sorting, field selection, free text search, structured filtering, datetime filtering, and spatial filtering) for the `/collections` endpoint can be disabled by setting the `ENABLE_COLLECTIONS_SEARCH` environment variable to `false`. By default, these extensions are enabled.
>
> **Configuration**: The custom `/collections-search` endpoint can be enabled by setting the `ENABLE_COLLECTIONS_SEARCH_ROUTE` environment variable to `true`. By default, this endpoint is **disabled**.

Expand Down Expand Up @@ -470,6 +484,64 @@ The system uses a precise naming convention:
curl -X "GET" "http://localhost:8080/collections?limit=1&token=example_token"
```

## SFEOS Tools CLI

- **Overview**: SFEOS Tools is an installable CLI package for managing and maintaining SFEOS deployments.

- **Installation**:
```shell
# For Elasticsearch (from PyPI)
pip install sfeos-tools[elasticsearch]

# For OpenSearch (from PyPI)
pip install sfeos-tools[opensearch]

# For local development
pip install -e sfeos_tools[elasticsearch]
# or
pip install -e sfeos_tools[opensearch]
```

- **Available Commands**:
- `add-bbox-shape`: Add bbox_shape field to existing collections for spatial search support

- **Basic Usage**:
```shell
sfeos-tools add-bbox-shape --backend elasticsearch
sfeos-tools add-bbox-shape --backend opensearch
```

- **Connection Options**: Configure database connection via CLI flags or environment variables:
- `--host`: Database host (default: `localhost` or `ES_HOST` env var)
- `--port`: Database port (default: `9200` or `ES_PORT` env var)
- `--use-ssl` / `--no-ssl`: Use SSL connection (default: `true` or `ES_USE_SSL` env var)
- `--user`: Database username (default: `ES_USER` env var)
- `--password`: Database password (default: `ES_PASS` env var)

- **Examples**:
```shell
# Local Docker Compose (no SSL)
sfeos-tools add-bbox-shape --backend elasticsearch --no-ssl

# Remote server with SSL
sfeos-tools add-bbox-shape \
--backend elasticsearch \
--host db.example.com \
--port 9200 \
--user admin \
--password secret

# Cloud deployment with environment variables
ES_HOST=my-es-cluster.cloud.com ES_PORT=9243 ES_USER=elastic ES_PASS=changeme \
sfeos-tools add-bbox-shape --backend elasticsearch

# Using --help for more information
sfeos-tools --help
sfeos-tools add-bbox-shape --help
```

For more details, see the [SFEOS Tools README](./sfeos_tools/README.md).

## Ingesting Sample Data CLI Tool

- **Overview**: The `data_loader.py` script provides a convenient way to load STAC items into the database.
Expand Down
21 changes: 21 additions & 0 deletions sfeos_tools/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2025 Jonathan Healy and CloudFerro S.A.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
113 changes: 113 additions & 0 deletions sfeos_tools/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# SFEOS Tools

CLI tools for managing [stac-fastapi-elasticsearch-opensearch](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch) deployments.

## Installation

### For Elasticsearch

```bash
pip install sfeos-tools[elasticsearch]
```

Or for local development:
```bash
pip install -e sfeos_tools[elasticsearch]
```

### For OpenSearch

```bash
pip install sfeos-tools[opensearch]
```

Or for local development:
```bash
pip install -e sfeos_tools[opensearch]
```

### For Development (both backends)

```bash
pip install sfeos-tools[dev]
```

Or for local development:
```bash
pip install -e sfeos_tools[dev]
```

## Usage

After installation, the `sfeos-tools` command will be available:

```bash
# View available commands
sfeos-tools --help

# View version
sfeos-tools --version

# Get help for a specific command
sfeos-tools add-bbox-shape --help
```

## Commands

### add-bbox-shape

Add `bbox_shape` field to existing collections for spatial search support.

**Basic usage:**

```bash
# Elasticsearch
sfeos-tools add-bbox-shape --backend elasticsearch

# OpenSearch
sfeos-tools add-bbox-shape --backend opensearch
```

**Connection options:**

```bash
# Local Docker Compose (no SSL)
sfeos-tools add-bbox-shape --backend elasticsearch --no-ssl

# Remote server with SSL
sfeos-tools add-bbox-shape \
--backend elasticsearch \
--host db.example.com \
--port 9200 \
--user admin \
--password secret

# Using environment variables
ES_HOST=my-cluster.cloud.com ES_PORT=9243 ES_USER=elastic ES_PASS=changeme \
sfeos-tools add-bbox-shape --backend elasticsearch
```

**Available options:**

- `--backend`: Database backend (elasticsearch or opensearch) - **required**
- `--host`: Database host (default: localhost or ES_HOST env var)
- `--port`: Database port (default: 9200 or ES_PORT env var)
- `--use-ssl / --no-ssl`: Use SSL connection (default: true or ES_USE_SSL env var)
- `--user`: Database username (default: ES_USER env var)
- `--password`: Database password (default: ES_PASS env var)

## Development

To develop sfeos-tools locally:

```bash
# Install in editable mode with dev dependencies
pip install -e ./sfeos_tools[dev]

# Run the CLI
sfeos-tools --help
```

## License

MIT License - see the main repository for details.
56 changes: 56 additions & 0 deletions sfeos_tools/setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
"""Setup for SFEOS Tools."""

from setuptools import find_packages, setup

with open("README.md", "r", encoding="utf-8") as f:
long_description = f.read()

setup(
name="sfeos-tools",
version="0.1.0",
description="CLI tools for managing stac-fastapi-elasticsearch-opensearch deployments",
long_description=long_description,
long_description_content_type="text/markdown",
author="Jonathan Healy, CloudFerro S.A.",
license="MIT",
url="https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch",
packages=find_packages(),
python_requires=">=3.8",
install_requires=[
"click>=8.0.0",
],
extras_require={
"elasticsearch": [
"stac_fastapi_core",
"sfeos_helpers",
"stac_fastapi_elasticsearch",
],
"opensearch": [
"stac_fastapi_core",
"sfeos_helpers",
"stac_fastapi_opensearch",
],
"dev": [
"stac_fastapi_core",
"sfeos_helpers",
"stac_fastapi_elasticsearch",
"stac_fastapi_opensearch",
],
},
entry_points={
"console_scripts": [
"sfeos-tools=sfeos_tools.cli:cli",
],
},
classifiers=[
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
],
)
3 changes: 3 additions & 0 deletions sfeos_tools/sfeos_tools/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
"""SFEOS Tools - Utilities for managing stac-fastapi-elasticsearch-opensearch deployments."""

__version__ = "0.1.0"
Loading