Repository navigation
Preserve typed columns for empty datasets - #204
Merged
Merged
Conversation
When no partitions are selected, initialize each dataset column from the discovered schema, matching the single-file reader's empty-column behavior. Cover metadata-only datasets, filters excluding every file, logical overrides, table conversions, and reading again after close. AI disclosure: This work was prepared with assistance from OpenAI Codex.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #204 +/- ##
==========================================
+ Coverage 86.81% 86.84% +0.03%
==========================================
Files 10 10
Lines 1600 1604 +4
==========================================
+ Hits 1389 1393 +4
Misses 211 211 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reading a dataset whose filter excludes every file, or a directory containing only schema metadata, currently throws
BoundsErrorwhen accessing a column or converting throughTables.columntableorTables.rowtable. The schema is valid, but the loader never initializes its columns.Initialize an empty vector for each declared column when no partitions are selected, matching the single-file reader's existing behavior. Preserve the discovered schema and logical-type overrides, and document that a schema must still be available from metadata or an existing data file.
Validation: the unchanged implementation passes 21 controls and errors on 30 public empty-dataset operations on both actual Julia 1.3.1 and 1.13.1. The fix passes all 51 controls plus 420 permanent checks on both versions, covering both constructors, fallback and metadata schemas, metadata-only directories, all column names and types, row/column conversion, unused callbacks, and reading again after close. Both full suites pass with four threads and bounds checking: 4,588 checks on Julia 1.3.1 and 4,574 on Julia 1.13.1. An independent agent privately reviewed the complete diff and schema/iteration/loading/close paths and found no blocker.
git diff --checkpasses. No package dependencies, exports, or version changes.Hosted validation at
aebbf1274af9d2f57d4274acd27ea9e01ff2fce7: all nine Julia 1.3/current/nightly jobs on Linux, macOS, and Windows and both coverage checks pass. Each uploader log confirms acceptance for this repository and commit; all nine reports are processed at 86.84%. No coverage threshold was changed.Normally merged as
c67ff831c2afc5ab412da1bd475b13fef3e0906e. The merged tree matches the fully tested PR tree. All nine exact-merge native jobs pass, and all nine correctly addressed coverage reports are accepted and processed at 86.84%.Co-authored by Codex
AI disclosure: This work was prepared with assistance from OpenAI Codex.