Skip to content

chore(deps): bump the production-dependencies group with 6 updates #2851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 11, 2024
Merged

chore(deps): bump the production-dependencies group with 6 updates #2851

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
14002c2
chore(deps): bump the production-dependencies group with 6 updates
dependabot[bot] Jun 10, 2024
2f9764b
fix: pin deltalake and adapt to breaking change
jaidisido Jun 11, 2024
f351427
fix: upgrade cryptography in poetry.lock
jaidisido Jun 11, 2024
5108904
fix: update test_read_deltalake_versioned test
jaidisido Jun 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions awswrangler/s3/_write_deltalake.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ def to_deltalake(
mode: Literal["error", "append", "overwrite", "ignore"] = "append",
dtype: dict[str, str] | None = None,
partition_cols: list[str] | None = None,
overwrite_schema: bool = False,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deltalake package introduced a breaking change where overwrite_schema boolean argument was deprecated and replaced with a literal argument

schema_mode: Literal["overwrite"] | None = None,
lock_dynamodb_table: str | None = None,
s3_allow_unsafe_rename: bool = False,
boto3_session: boto3.Session | None = None,
Expand All @@ -81,8 +81,8 @@ def to_deltalake(
(e.g. ``{'col name':'bigint', 'col2 name': 'int'})``
partition_cols: list[str], optional
List of columns to partition the table by. Only required when creating a new table.
overwrite_schema: bool
If True, allows updating the schema of the table.
schema_mode: str, optional
If set to "overwrite", allows replacing the schema of the table. Set to "merge" to merge with existing schema.
lock_dynamodb_table: str | None
DynamoDB table to use as a locking provider.
A locking mechanism is needed to prevent unsafe concurrent writes to a delta lake directory when writing to S3.
Expand Down Expand Up @@ -130,6 +130,6 @@ def to_deltalake(
data=table,
partition_by=partition_cols,
mode=mode,
overwrite_schema=overwrite_schema,
schema_mode=schema_mode,
storage_options=storage_options,
)
73 changes: 37 additions & 36 deletions poetry.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ jsonpath-ng = { version = "^1.5.3", optional = true }
# Other
openpyxl = { version = "^3.0.0", optional = true }
progressbar2 = { version = "^4.0.0", optional = true }
deltalake = { version = ">=0.6.4,<0.18.0", optional = true }
deltalake = { version = ">=0.18.0,<0.19.0", optional = true }
geopandas = [
{ version = "^0.13.2", markers = "python_version < \"3.12\"", optional = true },
{ version = "^0.14.1", markers = "python_version >= \"3.12\"", optional = true }
Expand Down
Loading