fix iceberg extension REST catalog file binder exception

[ericyuanhui]

fix #871

Root Cause

The REST Catalog implementation was added in the Iceberg extension, but the core LOAD FROM binder still assumes that the source must be a filesystem path.

The failure occurs before the extension table function is bound:

LOAD FROM
  -> bindFileScanSource()
  -> bindFilePaths()
  -> filesystem glob
  -> binder error

Solution

Update bindFileScanSource() to recognize qualified Iceberg REST Catalog table names before filesystem globbing.

When the following conditions are met:

• the Iceberg extension is loaded;
• iceberg_warehouse is configured;
• file_format='iceberg';
• the source matches namespace.table or iceberg_catalog.namespace.table;

the binder bypasses filesystem globbing and passes the source directly to the Iceberg extension.

Filesystem path validation is also skipped for these Catalog names because they are logical table identifiers, not local files.

Scope and Compatibility

The change is intentionally limited to Iceberg REST Catalog identifiers.

Existing behavior remains unchanged for:

• Parquet, CSV, JSON, and other file formats;
• regular Iceberg filesystem paths;
• metadata file paths;
• wildcard file patterns;
• REST-like strings when the Iceberg REST configuration is not enabled.

No changes are made to the Iceberg extension implementation itself.

Result

After rebuilding Ladybug, the following queries can be resolved through the REST Catalog:

CALL iceberg_warehouse='warehouse';
CALL iceberg_endpoint='http://127.0.0.1:8181';
CALL iceberg_authorization_type='none';

LOAD FROM 'perf_demo.User'
(file_format='iceberg')
RETURN count(*);

LOAD FROM 'perf_demo.Session'
(file_format='iceberg')
RETURN count(*);

LOAD FROM 'perf_demo.User_owns_Session'
(file_format='iceberg')
RETURN count(*);

The fully qualified form is also supported:

LOAD FROM 'iceberg_catalog.perf_demo.User'
(file_format='iceberg')
RETURN count(*);

[adsharma]

The idea sounds good, but the knowledge of an extension (iceberg) leaks into the main repo. How about we design a callback or the extension does this eagerly and sets a bit somewhere for the main repo to understand it shouldn't glob?

[ericyuanhui]

The idea sounds good, but the knowledge of an extension (iceberg) leaks into the main repo. How about we design a callback or the extension does this eagerly and sets a bit somewhere for the main repo to understand it shouldn't glob?

This solution looks much cleaner. I'll go ahead and refactor accordingly. Thank you for your suggestion!

[adsharma]

I'll hold the 0.20.2 release for a few hours if you want to get this in.

[ericyuanhui]

I'll hold the 0.20.2 release for a few hours if you want to get this in.

Unfortunately, I am unlikely to meet the timeline. I have been occupied with other urgent tasks and have not yet made the modifications. I apologize for the delay.

[ericyuanhui]

The root cause is that the current LOAD FROM binding pipeline assumes every file source is a filesystem path.

For a statement such as:

LOAD FROM 'perf_demo.User' (file_format='iceberg') RETURN count(*);

the flow is currently:

LOAD FROM
  -> Binder::bindFileScanSource()
  -> Binder::bindFilePaths()
  -> VirtualFileSystem::glob()
  -> “No file found that matches the pattern”
  -> Iceberg table function bind callback is never reached

The Iceberg extension only receives control later, when Ladybug selects and binds the iceberg_scan table function. By that point, the core binder has already attempted local filesystem globbing and rejected perf_demo.User.

Therefore, this cannot be fixed solely inside the current Iceberg extension while preserving the existing namespace.table / iceberg_catalog.namespace.table syntax.

The existing extension mechanisms do not provide a hook before Binder::bindFilePaths() is called. In particular:

• BinderExtension is intended for binding extension statements, not for classifying FileScanSource paths.
• An Iceberg table function bind callback is invoked after path expansion and validation.
• A VFS-based workaround cannot reliably solve this for the current syntax, because VFS routing is based on the path string and does not receive the parsed file_format='iceberg' option. Treating every dotted identifier as an Iceberg source would incorrectly intercept ordinary local file paths.

A single global “Iceberg is loaded, do not glob” flag is also insufficient. Iceberg still supports ordinary filesystem-backed tables, metadata paths, and wildcard paths, all of which must retain the normal globbing behavior. Whether a source is a logical REST catalog identifier depends on the individual statement’s source, file_format, and current REST catalog configuration—not merely on whether the extension has been loaded.

@adsharma

[adsharma]

  source string
     │
     ▼
  extensionFormat?  (file_format declared && FileTypeUtils::fromString → UNKNOWN)
     │ yes → bindFilePaths(paths, passThroughOnNoMatch=true)
     ▼
  VirtualFileSystem::glob(path)          ← globbing is STILL attempted, always
     │
     ├─ matches found (existing dir/file, or wildcard expansion)
     │      → use the expanded paths          (globbing case, works as before)
     │
     └─ empty (nonexistent path, or a logical name like 'default.lineitem')
            → passThroughOnNoMatch: keep the raw string verbatim
     ▼
  getScanFunction → FileType::UNKNOWN → lookup "ICEBERG_SCAN" → extension bind
     ▼
  iceberg bindFuncHelper: catalog-qualified name? → REST query
                          otherwise        → DuckDB iceberg_scan('<source>')