Repository navigation
Conversation
f253b68 to
6cb25c8
Compare
|
I think it's definitely good to add this information but i'm not quite sure about the format. Did we just not to copy the compatibility section for V3? Having multiple roles for support based on the version seems a little harder to read imho. |
I think it will be easier to compare between versions. With more versions incoming, it's hard to maintain them copying almost identical table every time. |
f1c6d5e to
1ef76e3
Compare
|
@RussellSpitzer @nssalian I'm using tabbed table now. Each Table Spec gets its own tab or one merged tab if their contents are identical. |
1ef76e3 to
b02246d
Compare
b7de469 to
3b666b3
Compare
wgtmac
left a comment
wgtmac
left a comment
There was a problem hiding this comment.
AI has proof-read the tables and reported some findings. Perhaps maintainers can double check? @kevinjqliu @zeroshade @liurenjie1024
9ddbaf3 to
ff94458
Compare
Track format v3 capabilities in per-version tabs and refresh the implementation status against the released Java 1.12.0, PyIceberg 0.12.0, Rust 0.11.0, Go 0.7.0, and C++ 0.4.0 libraries, so the page describes functionality users can rely on today. Generated-by: Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ff94458 to
fd6a513
Compare
nssalian
left a comment
nssalian
left a comment
There was a problem hiding this comment.
Thanks for the update @manuzhang. I checked, using AI, the cells against the release tags. The latest changes look right. 14 cells still don't match the released code. They were already wrong on main, but it'd be good to fix them here. Most are inline. Two are on rows outside the diff:
- Line 295, REST
updateNamespaceProperties: Rust should be N.update_namespacestill returnsFeatureUnsupportedin 0.11.0 (crates/catalog/rest/src/catalog.rs:1035-1045). - Lines 384-389, Glue views: Java should be N.
GlueCatalogdoesn't implementViewCatalogin 1.12.0 (GlueCatalog.java:83-84).
There was a problem hiding this comment.
Rust should be N. In 0.11.0 the REST update_namespace still returns FeatureUnsupported("Updating namespace not supported yet!")
| | Avro | Y | N | N | N | Y | | ||
|
|
||
| ## File IO | ||
|
|
||
| | Storage | Java | PyIceberg | Rust | Go | C++ | | ||
| |-------------------|------|-----------|------|----|------| | ||
| | Local Filesystem | Y | Y | Y | Y | Y | | ||
| | Hadoop Filesystem | Y | Y | Y | Y | N | | ||
| | S3 Compatible | Y | Y | Y | Y | N | | ||
| | Hadoop Filesystem | Y | Y | Y | N | N | |
There was a problem hiding this comment.
Rust should be N here too. In 0.11.0, parse_scheme in crates/storage/opendal/src/resolving.rs:56-69 accepts memory, file, s3, gcs, oss, azdls and hf only, and there's no hdfs feature in any Cargo.toml.
| |------------------------|------|-----------|------|----|-----| | ||
| | Append data | Y | Y | Y | Y | Y | | ||
| | Write position deletes | Y | N | N | Y | Y | | ||
| | Write equality deletes | Y | N | Y | Y | Y | |
There was a problem hiding this comment.
Rust can write equality-delete files, but 0.11.0 has no transaction that commits them, so I think this should be N in both the V2 and V3 tabs.
| | Operation | Java | PyIceberg | Rust | Go | C++ | | ||
| |-------------------|------|-----------|------|----|-----| | ||
| | Append data files | Y | Y | Y | Y | Y | | ||
| | Rewrite files | Y | Y | N | Y | Y | |
There was a problem hiding this comment.
PyIceberg 0.12.0 doesn't have these as far as I can tell. UpdateSnapshot only exposes fast_append, merge_append, overwrite and delete. Nothing produces a replace snapshot, and there's no rewrite_manifests. The V3 tab already says N, so lines 174-175 and 183-184 could match it.
| | Expire snapshots | Y | Y | Y | Y | Y | | ||
| | Manage snapshots | Y | Y | N | Y | Y | | ||
|
|
||
| === "V3" |
There was a problem hiding this comment.
nit: this tab is identical to "V1 - V2" above, so by the merge rule it could be a single "V1 - V3" tab like REST.
Refresh implementation status against the released Java 1.12.0, PyIceberg 0.12.0, Rust 0.11.0, Go 0.7.0, and C++ 0.4.0 libraries, and add format V3 coverage using per-version tabs. A data type counts as supported only when the library handles it in schemas and can read and write its values. Cells reflect released source and tests rather than
main, so capabilities that exist only in unreleased commits are markedN. Closes #17308.Test plan: Markdown lint across site and versioned source docs; focused MkDocs build and rendered-table/tab/link checks;
git diff --check. The full lint wrapper failed refreshing dependencies; the focused build disabled remote asset downloads after font timeouts. Implementation tests were inspected, not rerun. Review-comment corrections were checked against each library's tagged release source.AI Disclosure