feat(api): add an authorized artifact-content and storage-provenance contract

[jeffspahr]

Feature description

PR #13986 replaces MLMD-backed artifact discovery with the native Artifact API, but artifact bytes are still read through a frontend-server route whose authority is assembled from caller-visible URI coordinates, an opaque providerInfo string, namespace-prefix heuristics, and optional per-namespace proxying.

That is not a complete replacement for MLMD's trusted ownership and storage context. The native API should expose an authenticated artifact-content contract keyed by artifact identity, with storage resolution and authorization performed from trusted server-side state.

Related:

• feat(backend): replace MLMD tracking with native task and artifact APIs. Part of #11760 #13986 — MLMD removal and native frontend migration
• feat(backend): add MLMD-to-native data migration and startup gate #14029 — historical MLMD-to-native migration and startup gate
• chore(frontend): scale and harden native runtime metadata views after MLMD removal #14028 — frontend scale/hardening follow-ups
• chore(backend): carry resiliency and scale guarantees into the post-MLMD architecture #14020 — runtime API resiliency and retry guarantees

Why this is a follow-up rather than a frontend patch in #13986

The current PR can and should close bounded frontend-server regressions: canonical namespace-prefix validation, view-role RBAC, query-safe links, request fan-out, and failure isolation. It cannot safely manufacture trusted storage provenance from browser-supplied values.

A durable fix requires a backend/API contract, authorization semantics, persisted or derivable trusted storage configuration, generated-client changes, and compatibility tests across S3, MinIO, GCS, custom roots, single-user, multi-user, and historical artifacts. Adding another query parameter or broadening the frontend server's credentials would enlarge the trust boundary and could turn compatibility failures into cross-tenant reads or credential use.

Verified gaps

1. Namespace-local Artifact rows are not sufficient ownership proof for custom roots

An authorized user can create/import an Artifact row in their namespace whose URI points at another location. An exact URI match in the caller's namespace therefore proves that the row is visible to the caller; it does not prove that centrally privileged object-store credentials may read the referenced bytes.

The private-artifacts/<namespace>/... convention provides independent ownership evidence for the standard root. Custom roots need equally trusted provenance or namespace-isolated credentials.

2. Opaque providerInfo crosses the browser/proxy trust boundary

Artifact URI query parameters are currently converted in the browser into provider JSON and forwarded through the central UI to the namespace artifact proxy. If accepted as authority, that JSON can select an endpoint and, in secret-backed mode, name credential Secret keys. Depending on the proxy service account's permissions, this permits unauthorized credential use/request signing and endpoint or SSRF control.

The browser may carry a hint, but it must not establish which endpoint or credentials the server is authorized to use.

3. Query-bearing URIs lose exact ownership identity

For a stored URI such as s3://bucket/key?endpoint=..., the browser separates the query from the object key before calling the frontend server. The ownership validator reconstructs s3://bucket/key, so an exact native Artifact lookup cannot match the stored row. Strict ownership mode and custom-root proxy reads therefore reject legitimate records.

Reversing an opaque provider JSON string into a URI is not a safe canonicalization contract.

4. Namespace-prefix fallback is not bound to a trusted root

The fallback recognizes private-artifacts/<namespace>/... below any scheme/authority. If shared credentials can read another bucket or endpoint containing the same path, matching the namespace segment alone is not sufficient ownership evidence.

The fallback should be bound to a trusted configured root and retired for migrated records once #14029 is complete.

5. Custom-root credentials have no safe default multi-user path

The central UI correctly refuses to read tenant Secrets. The current alternative is the optional per-namespace artifact proxy, which is disabled by default and already documented as carrying security, maintenance, and scalability costs.

The replacement architecture needs a supported namespace-local credential path without requiring the browser to transport credentials/configuration or the central UI service account to read tenant Secrets.

providers.<provider>.default supplies storage settings and credentials only after a URI is admitted by the configured pipeline root or an explicit override; it is not authorization for arbitrary outside-root paths.

6. Artifact content lacks bounded transfer metadata

The native Artifact model does not provide a content-read operation with authoritative size, media type, range/peek semantics, or a server-enforced limit. Selected HTML/Markdown and legacy viewer sources may therefore be buffered in full before rendering.

Proposed API shape

The exact transport is open for design, but it should provide an artifact-ID-based operation such as:

• a streaming GetArtifactContent/HTTP content endpoint;
• a short-lived presigned URL minted after authorization; or
• a server-side proxy with equivalent bounded streaming behavior.

The operation should:

1. Accept artifact identity, not caller-selected credentials.
2. Load the Artifact record and authorize access to its run/namespace.
3. Resolve storage configuration from trusted launcher/run/artifact provenance.
4. Bind the resolved scheme, authority, bucket, key, query configuration, and credential scope to that artifact.
5. Support custom roots without granting the central UI broad tenant Secret access.
6. Return or enforce media type, content length when known, byte ranges/peek, maximum preview size, and download disposition.
7. Preserve streaming and cancellation; do not buffer arbitrary objects in the API server.
8. Emit auditable errors that distinguish authorization, missing record, invalid trusted configuration, missing object, and transient provider failure without exposing secrets.

Security requirements

• Never treat a caller-created Artifact row alone as authorization to use centrally privileged credentials for an arbitrary URI.
• Never accept browser-supplied providerInfo, endpoint, region, Secret name, or Secret key as trusted authority.
• Resolve tenant credentials inside the tenant's authorization boundary, or mint a narrowly scoped read capability after server-side validation.
• Bind any prefix fallback to the configured scheme/authority/bucket/root and claimed namespace.
• Protect against SSRF, DNS rebinding, redirects to untrusted hosts, credential forwarding, path traversal, encoded separators, and signed-query leakage.
• Keep authorization fail-closed for custom roots and unknown provenance.
• Ensure cache keys include artifact identity, authorization identity/context, namespace, content range, and configuration revision; do not share authorization results across tenants.

Acceptance criteria

• A documented native artifact-content API is keyed by artifact ID and authorizes against trusted run/namespace ownership.
• Standard roots, custom roots, S3, MinIO, and GCS work in single-user and multi-user deployments.
• Browser-supplied provider JSON cannot select credentials, Secret keys, or an untrusted endpoint.
• Query-bearing stored URIs retain one canonical identity from persistence through authorization and content resolution.
• Namespace-prefix compatibility fallback is bound to a trusted configured root and has a documented retirement plan after feat(backend): add MLMD-to-native data migration and startup gate #14029.
• Tenant credentials are resolved without the central UI service account reading arbitrary tenant Secrets.
• Preview reads have server-enforced bounds; downloads support streaming/range behavior without whole-object buffering.
• Errors distinguish permanent authorization/configuration failures from transient provider failures and contain corrective action.
• Tests cover forged Artifact rows, forged provider info, wrong roots/buckets, endpoint injection/SSRF, cross-namespace reads, stale configuration, redirects, malformed encoding, and large objects.
• Historical migrated artifacts from feat(backend): add MLMD-to-native data migration and startup gate #14029 use the same content path without weakening ownership checks.

Test plan

• Table-driven unit tests for canonical URI/root normalization across every supported provider.
• API authorization tests for owner, namespace viewer, namespace editor, unauthorized identity, deleted run, and mismatched artifact/run namespace.
• Adversarial tests for caller-selected endpoint/Secret/provider values and tenant-created rows pointing at another root.
• Integration tests using default and custom roots in at least two namespaces.
• Streaming/range/peek tests for empty, small, large, missing, and interrupted objects.
• Upgrade tests that read historical non-prefix artifacts after feat(backend): add MLMD-to-native data migration and startup gate #14029 migration.

---

Love this idea? Give it a 👍.

[github-actions]

🤖 AI Issue Quality Review

⚠️ Validation Failed: Issue title must follow the correct format: <type>(<area>): <title contents> where type is bug, chore, or feature.

[jeffspahr]

Two current-route constraints should be covered explicitly during this API transition:

1. View-role compatibility: a namespace viewer who may read a run must receive the minimum native Artifact get/list permissions needed for server-side ownership validation and artifact-content authorization. Mutating verbs remain editor-only. Add an RBAC matrix test so UI authorization and Artifact API authorization cannot drift.
2. volume:// semantics: the existing frontend route interprets a caller-supplied volume name as a mount on the serving UI pod and currently skips artifact ownership validation. The native content API must not expose central UI pod mounts to authenticated tenants. Either reject this source in multi-user mode or resolve it only inside a namespace-isolated, artifact-owned execution boundary with explicit provenance.

Additional acceptance criteria:

• Namespace viewers can read authorized artifact metadata/content without receiving artifact mutation permissions.
• Authenticated tenants cannot use volume:// to enumerate or read mounts belonging to the central UI/API serving pod.
• Tests cover view/edit/unauthorized roles for every supported content source, including rejection of direct central volume:// reads.

[jeffspahr]

The two interim deployment-policy decisions previously called out here are now decided and implemented in #13986 commit 672b7d33b:

• namespace viewers receive native artifact metadata get/list (no mutation verbs);
• authenticated direct central-UI volume:// reads fail closed, while single-user and namespace-isolated proxy reads remain supported.

#14034 records the rationale, alternatives, compatibility impact, and rollout guidance. This does not reduce #14031's scope: the long-term trusted artifact-content/storage-provenance API remains necessary for custom roots, namespace-local credentials, streaming/range/size metadata, and an explicit durable model for volume-backed artifacts.

[jeffspahr]

An additional compatibility requirement from the #13986 live-head review sharpens the trusted-provenance scope in this issue:

Distinguish historical storage context from native launcher policy

The transitional frontend server cannot reliably infer whether an artifact read should follow current native launcher policy or historical stored session context:

• Treating a missing kfp-launcher ConfigMap as a legacy signal permits any outside-root URI to fall back to environment credentials, while the native launcher deliberately rejects outside-root artifacts without an explicit query or provider override.
• Treating ConfigMap presence as a native signal rejects legitimate historical outside-root links before their stored provider context can be used.
• A browser query parameter, providerInfo, artifact metadata field, or caller-selected legacy=true flag is not trusted provenance: users able to create artifacts or requests could forge it and select credentials/endpoints.

The native artifact-content contract therefore needs an immutable, server-established provenance/credential-resolution mode associated with artifact identity. The exact representation is open, but it must distinguish at least:

1. Native launcher-resolved artifact: enforce the pipeline root, artifact URI query, and explicit override semantics from the launcher configuration/revision that authorized creation.
2. Migrated historical artifact: use only storage context imported by the trusted feat(backend): add MLMD-to-native data migration and startup gate #14029 migration path, subject to the artifact's namespace/run authorization and without exposing opaque credentials to the browser.

Additional acceptance criteria:

• Content authorization/resolution chooses native versus historical policy from trusted persisted provenance keyed by artifact identity, never from ConfigMap presence or a caller-provided flag.
• Ordinary Artifact API create/update permissions cannot forge or mutate the provenance mode, credential reference, endpoint, or configuration revision used for content reads.
• Native artifacts outside their authorized launcher root remain rejected when the ConfigMap is absent, matching runtime launcher behavior.
• Migrated historical outside-root artifacts remain readable when their trusted migrated storage context is valid, regardless of whether a current kfp-launcher ConfigMap exists.
• Migration records enough canonical URI/provider/configuration provenance to resolve historical reads without forwarding browser-supplied providerInfo.
• Tests cover all four ConfigMap/provenance combinations (present/absent × native/historical), forged provenance fields, changed roots/endpoints after migration, and rollback.

This cannot be closed safely by the #13986 frontend bridge alone; the current ConfigMap-presence heuristic is necessarily ambiguous until the API owns this provenance.

[jeffspahr]

Direct legacy/native provenance residual from #13986

The latest #13986 review identified the concrete compatibility case behind this issue:

• In single-user/direct mode, a historical outside-root artifact may carry legacy
  store_session_info/providerInfo that is sufficient to read the object.
• When a kfp-launcher ConfigMap is present, the frontend resolves launcher policy first. An
  outside-root URI with no trusted query or matching override is rejected before the legacy
  provider fallback, so the preview/download returns 500.
• Relaxing that validation whenever browser-supplied provider information exists is not safe. The
  current request has no trusted native-versus-legacy discriminator, so the same rule would allow a
  native request to bypass configured launcher-root policy and select storage authority from the
  browser.
• Treating ConfigMap absence as provenance is also incorrect: native launchers still apply their
  compiled default root when the ConfigMap is absent.

The #13986 decision is therefore to keep native launcher-root enforcement fail-closed, document the
legacy limitation, and resolve it through this issue rather than add a frontend heuristic.

Additional acceptance coverage for the artifact-content contract:

• Persist or derive a trusted native/legacy storage-provenance signal; do not accept a caller
  flag or opaque provider blob as that signal.
• Native outside-root artifact without a ConfigMap is rejected under the same compiled default
  root policy used by the launcher.
• Native artifact under the default root is readable with and without a ConfigMap.
• Historical legacy outside-root artifact with trusted migrated provider information remains
  readable when launcher configuration exists.
• A forged native request containing legacy-shaped provider information cannot bypass root,
  endpoint, credential, or ownership policy.

[jeffspahr]

Additional detail for gap 3 (canonical ownership identity), from reviewing the frontend artifact-coordinate work in #13986.

The frontend server now enforces a canonical artifact-URI key on both artifact routes: raw ?/# are rejected as object-key characters, and the key must equal encodeURI(decodeURIComponent(key)). That gives one spelling per object, so artifact identity and the ownership check cannot diverge — it also blocks encoded alias spellings such as root/%73ecret that would otherwise decode past a prefix check.

The ? half of that contract already matches the producer: GenerateOutputURI splits the pipeline root on ? to separate provider query parameters (backend/src/common/util/artifact.go:28-43).

The escaping half is not guaranteed on the producer side. The same function builds the URI as:

return fmt.Sprintf("%s/%s%s", strings.TrimRight(pipelineRoot, "/"), path.Join(paths...), query)

pipelineRoot is passed through verbatim and the path segments are joined raw, with no percent-encoding. So a pipeline_root (or path segment) containing a character that requires escaping — a space, a non-ASCII character, or a literal % — persists a non-canonical URI. Task and output names are sanitized, so a user-supplied pipeline_root is the realistic case.

The practical consequence today is narrow: such an artifact is readable by download but rejected by the preview route, because the persisted spelling is not the canonical one the reader requires.

Worth deciding explicitly as part of the content contract here, since it is a producer/consumer agreement rather than something the reader can fix on its own:

• canonicalize at write time in GenerateOutputURI, so persisted URIs are always canonically escaped; or
• canonicalize on read as part of the artifact-content operation, so stored spelling and lookup identity are normalized in one trusted place.

Either way this belongs under the existing acceptance criterion "Query-bearing stored URIs retain one canonical identity from persistence through authorization and content resolution" — the same criterion should cover path escaping, not only the query string.