Gated catalogs: adopt STAC Authentication incl. the tokenExchange flow (was: access: extension proposal)

[cayetanobv]

Status (2026-07-07) — repurposed after upstream agreement

This PR began as a proposal for a Portolan-owned access: extension (one field, access:credentials, carrying the credential-vending pointer). The discussion below with @m-mohr (STAC Authentication maintainer) resolved into a better path: standardize the missing piece upstream, then adopt the upstream extension wholesale. Both upstream PRs are open:

• OAI/OpenAPI-Specification#5428 — tokenExchange OAuth flow (RFC 8693) in the OAuth Flows Object
• stac-extensions/authentication#44 — mirrors it in the STAC Authentication extension

What this PR now does

Adopts STAC Authentication (auth:) into the Portolan profile for gated catalogs — no Portolan-owned authentication namespace:

• spec/extensions/authentication.md — profile note: the two-step gated read (identity via openIdConnect, credential vending via oauth2/tokenExchange), fields used (all upstream), client algorithm, the chaining convention (machine-readable linkage deliberately left to the upstream discussion on feat(hooks): add auto-fetch for core dependency docs via gitingest #44), validation rules for portolan check, security considerations (consumer IdP, no secrets in metadata, least-privilege credentials).
• spec/examples/authentication-collection.json — a real-world-shaped example (anonymized from a running implementation): two schemes, two gated assets (GeoParquet + Apache Iceberg table metadata) sharing one exchange. Validates against the released v1.1.0 schema as-is.
• The earlier access: files are removed; the shape remains in this PR's history as the recorded fallback if the upstream path stalls.

Export-only-IAM publishers (#120) are unaffected: they omit the exchange scheme (optionally declaring an s3 hint); presigned-URL publishers use the extension's existing signedUrl type. Enforcement stays outside Portolan — this is discovery metadata only.

Draft until

stac-extensions/authentication#44 is merged and a version is tagged (the example pins v1.1.0 for now — unknown flow keys validate as generic flow objects; strict tokenUrl enforcement arrives with the new version). Per ADR-0048, the CLI implementation (emission + the validation rules above) joins this PR before it leaves draft.

Open questions from the original proposal — reconciled

1. Per-asset override → auth:refs is already per-asset; nothing to add.
2. Standardized request/response body → answered by RFC 8693.
3. Contribute the vending step upstream as a scheme/flow → happening (the two PRs above).
4. Schema-host domain → moot; there is no Portolan-owned schema anymore.

Relates to #120 (Access Control & Visibility) and #485 (Git-Backed Catalog extension).

Summary by CodeRabbit

• Documentation
  • Added guidance for authenticated, access-controlled catalogs using STAC Authentication.
  • Expanded the specification navigation to include the new authentication extension page.
  • Added an example collection showing how secure asset access is configured and referenced.

[coderabbitai]

📝 Walkthrough

Walkthrough

Changes

This PR adds documentation for a new "auth:" STAC extension. It introduces spec/extensions/authentication.md describing gated catalog access using OIDC identity tokens exchanged for short-lived, prefix-scoped credentials via OAuth 2.0 Token Exchange, along with validation expectations for portolan check. An example STAC Collection JSON (authentication-collection.json) demonstrates auth:schemes and auth:refs usage on assets. spec/README.md gains a new "Extensions" navigation entry linking to the new documentation page.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant publisher_oidc
  participant credential_exchange
  participant Storage
  Client->>publisher_oidc: Request identity token
  publisher_oidc-->>Client: OIDC identity token
  Client->>credential_exchange: OAuth2 tokenExchange with identity token
  credential_exchange-->>Client: Short-lived scoped credentials
  Client->>Storage: Read asset with credentials

Loading

Estimated code review effort: Low

Related issues: None provided.

Related PRs: None provided.

Suggested labels: documentation, spec

Suggested reviewers: None provided.

Poem:
A rabbit hops through gated doors,
With tokens traded, three times four.
OIDC whispers, "Who goes there?"
Exchange replies with creds to share.
Docs now written, README linked—
This burrow's access, neatly synced. 🐇🔐

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: adopting STAC Authentication with token exchange for gated catalogs.
Description check	✅ Passed	The description covers the repurposed scope, technical details, and related issues, but it omits an explicit breaking-changes section and checklist.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[m-mohr]

Do I understand correctly, that is is only meant to provide another token for object storage access? It feels like this should be solvable via the auth extension natively, but I'm also not sure whether I understand the usecase/issue correctly yet. Is this a somewhat two-step auth process?

[cayetanobv]

Yes — two steps, and the key nuance is that the second token is a different kind of credential:

1. Authenticate (pure auth:, nothing new): the client resolves the openIdConnect scheme → OIDC discovery → login → identity token.
2. Exchange (access:credentials): the client POSTs that identity token to a vending endpoint and receives short-lived, scoped object-storage credentials — e.g. S3-style {key, secret, session_token} or a downscoped GCS token, read-only, prefix-scoped — or 403 if the publisher hasn't entitled that identity.

The reason step 2 returns storage credentials rather than another bearer token: the readers are query engines (DuckDB, GDAL /vsis3/, PyIceberg…) doing direct range reads against object storage, with no server in the data path. For Apache Iceberg in particular the engine discovers the file list dynamically (metadata.json → manifests → N parquet files), so it needs one credential valid for a whole prefix rather than per-request auth. Same shape as Iceberg REST "vended credentials" or Delta Sharing: bearer token in → scoped temp storage creds out.

On "solvable via the auth extension natively" — we mapped our case onto the existing scheme types before adding anything:

• s3 covers consumers who already are cloud IAM principals (the export-only-IAM model — which we support by simply omitting access:). Our consumers are identified by OIDC (self-registration, federation) and are not pre-provisioned in the storage vendor's IAM.
• openIdConnect identifies them perfectly — but object storage doesn't validate IdP tokens, so identity alone can't read the bucket.
• signedUrl is the closest mechanism (and our presigned protocol is essentially that flavor), but it authorizes per object, and for Iceberg the engine discovers the file list from manifests mid-read — it needs one credential valid for the table's whole prefix.

The missing piece is the bridge: exchange a verified identity token for temporary, scoped storage credentials, with entitlement semantics (grant → creds, no grant → 403). That exchange step is the entire extension.

And honestly, upstream is where we'd love this to end up — our open question 3 is exactly "contribute this as a new scheme type (e.g. tokenExchange) to the Authentication extension and drop the namespace." Current plan is to incubate it here against a running implementation (broker + clients) and, once proven end-to-end, propose it upstream with working code rather than a spec-only idea. If a new scheme type sounds like the right home to you, that's extremely useful signal — happy to sketch what it would look like on the auth side.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@spec/extensions/access.md`:
- Around line 62-63: The wording in the access extension description uses
“datasets,” which conflicts with the STAC terminology guideline. Update the text
in the extension docs to use STAC terms consistently, especially in the sentence
about public content, and choose the appropriate STAC object term such as
Catalog or Collection instead of dataset.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 4bd1011c-ef4c-4f87-82ac-b98ca5daf434

📥 Commits

Reviewing files that changed from the base of the PR and between aa87e60 and 2f301dd.

📒 Files selected for processing (4)

• spec/README.md
• spec/examples/access-collection.json
• spec/extensions/access.md
• spec/schema/access/v1.0.0/schema.json

[coderabbitai]

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Use STAC terminology instead of "datasets".

✏️ Proposed fix

-  extension. Public datasets omit it.
+  extension. Public Collections/Catalogs omit it.

As per coding guidelines, **/*.{py,md,yaml,yml,json}: "Use STAC terminology exclusively: use 'Catalog', 'Collection', 'Item', and 'Asset' instead of 'dataset'".

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	- **Additive / backward-compatible.** A client that doesn't understand the
	extension still reads valid STAC. Public datasets omit it.
	- **Additive / backward-compatible.** A client that doesn't understand the
	extension still reads valid STAC. Public Collections/Catalogs omit it.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@spec/extensions/access.md` around lines 62 - 63, The wording in the access
extension description uses “datasets,” which conflicts with the STAC terminology
guideline. Update the text in the extension docs to use STAC terms consistently,
especially in the sentence about public content, and choose the appropriate STAC
object term such as Catalog or Collection instead of dataset.

Source: Coding guidelines

[m-mohr]

Thanks, this is helpful and makes the use case clearer to me.

One question to help decide the right home: how common is this exact two-step pattern across providers in practice (OIDC/OAuth login, then exchange for short-lived storage credentials), versus this being specific to a specific platform?

If this is broadly common, I wonder whether the best long-term path is to standardize the pattern upstream first (for example in the OpenAPI/OAuth ecosystem, if not already covered there), and then bring it into STAC Authentication downstream. STAC Authentication is based on OpenAPI and has no two-step semantic, it's encoded implicitly in the specific authentication types such as OIDC.
If it is less common or provider-specific, your current incubation path in Portolan first seems reasonable.

[cayetanobv]

Broadly common — and the exchange itself is already standardized; it's the discovery that isn't:

• The pattern: OAuth 2.0 Token Exchange is RFC 8693, and every major cloud implements it at STS (AWS AssumeRoleWithWebIdentity, GCP Workload Identity Federation, Azure federated credentials): OIDC token in → short-lived storage credentials out.
• The data ecosystem converged on it too, as "credential vending": Iceberg REST catalogs (vended-credentials in LoadTable — Polaris, Unity, S3 Tables, R2) and Delta Sharing.
• What has no standard anywhere is the discovery descriptor: in all of those, the exchange endpoint is implicit (the catalog itself, the share server, a pre-registered STS). Nothing lets a catalog entry advertise "exchange your token here, credentials come back in format Y" — and cross-ecosystem discovery metadata is exactly STAC's turf.

So the plan your framing suggests: incubate here, profile the endpoint on RFC 8693 (that also answers our open question 2), and bring the proven descriptor to STAC Authentication downstream — likely as a scheme type like tokenExchange.

[m-mohr]

As RFC 8693 exists, it should probably be easy to propose an addition to OpenAPI and then once they have something in a draft (doesn't necessarily need to be released), we can downstream it to STAC. That would be the ideal way, indeed. Whether this needs to go through IETF or not, I'm not sure.

Edit: I just found OAI/OpenAPI-Specification#4807, but it wasn't really picked up yet. I guess the most promising path would be to open a PR to https://github.com/OAI/OpenAPI-Specification with a specific proposal for discussion.

PS: No offence, but I prefer to communicate with a human directly.

[cayetanobv]

Thanks for finding openapi's PR 4807 , I did some digging on that side:

• State of PR 4807: it's exactly this feature, and Ralf already sketched the shape (a tokenExchange field in the OAuth Flows Object, tokenUrl required) , but that was a year ago and nothing moved since. No PR for token exchange has ever been opened on the OpenAPI repo. The device authorization flow is the same kind of addition, first PR in 2020, closed unmerged, and the flow only shipped (v3.2.0) five years later. A tokenExchange flow could sit in that queue for years, and this is worring me on timing
• So I'd treat the two tracks as parallel, not sequential:
  • I'm happy to pick up #4807 and turn Ralf's sketch into a concrete PR to the OpenAPI spec , following the deviceAuthorization pattern from 3.2.0 it's a small change.
  • But I'd rather not gate the STAC/Portolan side on when OpenAPI merges or releases.
• Meanwhile we keep incubating the field in Portolan, so there's a running end-to-end implementation to point at from both proposals.

What do you think?

Some thoughts about "how common is this and why it wasn't added to open specs": I think the exchange itself is everywhere (cloud STS, Iceberg REST vended credentials, Delta Sharing, and I see it first-hand at CARTO: in custom apps we build for clients, this exact pattern login against the org's IdP, then exchange the token for short-lived storage credentials to read data directly), but until data catalogs, nobody consumed it via a description document; the client was always an SDK that already knew the endpoint or read from hardcoded envs in the config. Catalogs read by generic clients are what make the discoverable variant matter, which is probably why the gap survived this long.

PS: not offended at all, I'm using AI to explore and write faster responses, something common in this Project (but I wrote this response directly). Anyway, we'll be talking in person in 2 days :)

[m-mohr]

Thanks for the additional context on industry usage. Also finding it in the OpenAPI forum and the RFC adds enough weight that it's worth implementing it.

I agree with your plan. Let's create an OpenAPI PR, then a STAC Authentication extension PR that mirrors the OpenAPI PR (potentially tagging a beta release so that we have validation) and then we can implement/use it in Portolan. I think that makes the access extension obsolete, right?

PS: Appreciated.

[cayetanobv]

Thanks for the additional context on industry usage. Also finding it in the OpenAPI forum and the RFC adds enough weight that it's worth implementing it.

I agree with your plan. Let's create an OpenAPI PR, then a STAC Authentication extension PR that mirrors the OpenAPI PR (potentially tagging a beta release so that we have validation) and then we can implement/use it in Portolan. I think that makes the access extension obsolete, right?

PS: Appreciated.

Great, thanks for your feedback. I'll prepare the PRs.

[cayetanobv]

@m-mohr, the PRs I opened regarding the proposed plan:

• Add OAuth 2.0 Token Exchange (RFC 8693) to the supported OAuth flows OAI/OpenAPI-Specification#5428
• Add tokenExchange OAuth2 flow (OAuth 2.0 Token Exchange, RFC 8693) stac-extensions/authentication#44