feat: add Error::External variant for preserving user errors #5606
Conversation
PR Review: feat: add Error::External variant for preserving user errorsThis PR adds functionality to preserve user-defined errors passed through Lance APIs. The implementation is clean and the test coverage is comprehensive. No blocking issues foundThe code quality is good, error handling is consistent with existing patterns, and tests cover the key scenarios. Minor observations (non-blocking)
Overall this is a well-designed addition that addresses a real user need for error recovery in streaming APIs. |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
wjones127
force-pushed
the
feat/external-error-variant
branch
2 times, most recently
from
fe93b6a to
c18f61c
Compare
rust/lance-core/src/error.rs
Outdated
| // Check if the ArrowError wraps an external error and extract it | ||
| match *arrow_err { | ||
| ArrowError::ExternalError(source) => { | ||
| // Try to downcast to lance_core::Error first | ||
| match source.downcast::<Self>() { | ||
| Ok(lance_err) => *lance_err, | ||
| Err(source) => Self::External { source }, | ||
| } | ||
| } | ||
| other => Self::Arrow { | ||
| message: other.to_string(), | ||
| location, | ||
| }, | ||
| } |
There was a problem hiding this comment.
Would we get this automatically if we returned LanceError::from(arrow_err)?
There was a problem hiding this comment.
I think so. That's a good idea for simplification.
When users pass streams with custom error types (e.g., wrapped in ArrowError::ExternalError), the original error was being converted to a string and lost. This adds an Error::External variant that preserves the boxed error, allowing users to downcast and recover their original error type. Changes: - Add #[snafu(transparent)] External variant to Error enum - Update From<ArrowError> to extract inner error from ExternalError - Update From<DataFusionError> to handle External and nested cases - Add helper methods: external(), external_source(), into_external() 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
When a lance_core::Error is converted to ArrowError or DataFusionError and then back, the original error type is now recovered via downcasting instead of being wrapped in External or converted to strings. This enables workflows where iterators/streams use lance_core::Error internally but go through Arrow/DataFusion error types at API boundaries. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
wjones127
force-pushed
the
feat/external-error-variant
branch
from
92bfa6d to
be8e177
Compare
2 participants
Summary
Error::Externalvariant with#[snafu(transparent)]to preserve user errors passed through streamsFrom<ArrowError>to extract inner error fromExternalErrorvariantFrom<DataFusionError>to handleExternaland nestedArrowError::ExternalErrorexternal(),external_source(),into_external()Test plan
lance-coreInsertBuilder::execute_streamandMergeInsertJob::execute_reader🤖 Generated with Claude Code