feat(rust): add datafusion catalog_provider through namespace

[majin1102]

This is the first step of #4779

[codecov]

Codecov Report

❌ Patch coverage is 59.81308% with 86 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-namespace-datafusion/src/catalog.rs	51.47%	29 Missing and 4 partials ⚠️
.../lance-namespace-datafusion/src/namespace_level.rs	67.10%	23 Missing and 2 partials ⚠️
rust/lance-namespace-datafusion/src/schema.rs	62.85%	12 Missing and 1 partial ⚠️
.../lance-namespace-datafusion/src/session_builder.rs	68.96%	6 Missing and 3 partials ⚠️
rust/lance-namespace-datafusion/src/error.rs	0.00%	6 Missing ⚠️

📢 Thoughts on this report? Let us know!

[majin1102]

Hi @wjones127 @jackye1995 @yanghua @westonpace @Xuanwo ,

This PR is driven by #4779. In our production scenario, we’d like to leverage DataFusion to deliver a better SQL experience that supports search, analytics, and AI-related operators (e.g., via UDFs/UDTFs).

I’ve already discussed this with @jackye1995 offline, and we agreed to try this with a dedicated crate for the DataFusion namespace integration. This PR introduces the foundational approach.

I’ll follow up with additional issues covering next steps—such as Python bindings, URL-based tables, SQL extensions, etc.

Please take a look when you have time!