Skip to content

feat(rust/sedona-schema): Add Int8, UInt64, Int64 band data types, don't panic in BandMetadataRef#589

Merged
Kontinuation merged 5 commits intoapache:mainfrom
Kontinuation:pr3-int8-band-datatype
Feb 13, 2026
Merged

feat(rust/sedona-schema): Add Int8, UInt64, Int64 band data types, don't panic in BandMetadataRef#589
Kontinuation merged 5 commits intoapache:mainfrom
Kontinuation:pr3-int8-band-datatype

Conversation

@Kontinuation
Copy link
Member

@Kontinuation Kontinuation commented Feb 10, 2026

Summary

  • Add three new BandDataType variants: Int8, UInt64, Int64
  • Derive Hash and Copy on both BandDataType and StorageType enums for ergonomic use
  • Update match arms in BandMetadataRefImpl::data_type() to decode the new ordinals
  • Update data_type() and storage_type() to return Result, don't panic when observing unknown values
  • Extend test coverage and test data generators (generate_random_band_data, get_nodata_value_for_type) to handle all 10 band data types

@Kontinuation Kontinuation marked this pull request as draft February 10, 2026 14:02
@Kontinuation Kontinuation force-pushed the pr3-int8-band-datatype branch from e53444b to a9fd5ec Compare February 11, 2026 12:46
@Kontinuation Kontinuation requested a review from Copilot February 11, 2026 14:33
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the BandDataType enum to support three additional data types (Int8, UInt64, Int64) and aligns all enum discriminants with GDAL's GDALDataType ordinals for non-complex types.

Changes:

  • Renumbered existing BandDataType variants from 0-6 to 1-7 and added Int8 (14), UInt64 (12), Int64 (13)
  • Derived Hash and Copy traits on BandDataType and StorageType for better ergonomics
  • Updated test data generators and test cases to handle all 10 band data types

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
rust/sedona-schema/src/raster.rs Updated BandDataType and StorageType enum definitions with new variants and trait derivations
rust/sedona-raster/src/array.rs Updated match arms in BandMetadataRefImpl::data_type() to decode new ordinals
rust/sedona-raster/src/builder.rs Extended test coverage with new data type test cases
rust/sedona-testing/src/rasters.rs Refactored generate_random_band_data using macro and added support for new data types

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Kontinuation Kontinuation marked this pull request as ready for review February 11, 2026 14:37
Copy link
Member

@paleolimbot paleolimbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

It's great to align these with GDAL's model but probably better for the Rust enum / internal serialization to be separated from the integer storage.

Comment on lines 159 to 160
_ => panic!(
"Unknown band data type: {}",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this return an error or is this validated before we get here? (i.e., do we risk a panic for a corrupted raster array?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I found that storage_type also need to be fixed, I have changed both storage_type() and data_type() to return Result<...>.

Comment on lines 605 to +608
// Test all BandDataType variants
let test_cases = vec![
(BandDataType::UInt8, vec![1u8, 2u8, 3u8, 4u8]),
(BandDataType::Int8, vec![255u8, 254u8, 253u8, 252u8]), // -1, -2, -3, -4 as i8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible this may be a good time to add test for a corrupted band identifier to make sure it doesn't panic

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added test_invalid_band_metadata_returns_err for testing this

Comment on lines 91 to 93
/// Ordinals match GDALDataType for real-valued pixel types only.
/// GDT_Unknown (0) and complex types (CInt16=8, CInt32=9, CFloat32=10, CFloat64=11)
/// are intentionally omitted.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could handle these with an Other(usize) variant (whose internal representation is opaque bytes).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need Other variant for now since we decided to not enforce the values of BandDataType to match GDAL. I believe that it is a better design.

Comment on lines 94 to 96
#[repr(u16)]
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, PartialEq, Eq, Hash, Copy)]
pub enum BandDataType {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably more flexible without the #[repr(u16)] since at some point we may need a variant to handle other types of data (e.g., generic fixed size). It's great to align our representation to GDAL's but it is probably better scoped as try_to_gdal_band_data_type() -> u16 and try_from_gdal_band_data_type(gdal_type: u16).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The values here were happened to be the same as GDAL band data types. We'll have functions for converting between then rather than implicitly relying on it. https://github.com/Kontinuation/sedona-db/blob/0e73e17a79332cf7e58b9881084d86dd3f3531c1/rust/sedona-raster-gdal/src/gdal_common.rs#L50-L88

I decided to simply define the values without GDAL awareness just as before.

@Kontinuation Kontinuation changed the title feat(rust/sedona-schema): Add Int8, UInt64, Int64 band data types and align ordinals with GDAL feat(rust/sedona-schema): Add Int8, UInt64, Int64 band data types, don't panic in BandMetadataRef Feb 12, 2026
Copy link
Member

@paleolimbot paleolimbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@Kontinuation Kontinuation merged commit 66b58c3 into apache:main Feb 13, 2026
17 checks passed
@paleolimbot paleolimbot added this to the 0.3.0 milestone Feb 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants