SqlTable Fails When Running Pulumi Up after Creation

[fernandodakko]

Describe what happened

Creating a Databricks SqlTable fails when running pulumi up/preview after first time creation.

Sample program

def create_bronze_liquidity_snap(catalog: str, schema: str, wh_id: str):
    table_name = "liquidity_snap"
    full_name = f"{catalog}.{schema}.{table_name}"
    table = databricks.SqlTable(
        resource_name=f"uc_table_{full_name}",
        opts=pulumi.ResourceOptions(protect=True),
        args=databricks.SqlTableArgs(
            catalog_name=catalog,
            schema_name=schema,
            name=table_name,
            table_type="MANAGED",
            data_source_format="DELTA",
            owner="Data Engineering",
            cluster_keys=["exchange", "timestamp"],
            warehouse_id=wh_id,
            properties={
                'delta.enableDeletionVectors': "true"
            },
            columns=[
                databricks.SqlTableColumnArgs(
                    name="exchange",
                    type="STRING",
                    nullable=False,
                ),
                databricks.SqlTableColumnArgs(
                    name="raw_address",
                    type="STRING",
                    nullable=False,
                ),
                databricks.SqlTableColumnArgs(
                    name="timestamp",
                    type="TIMESTAMP",
                    nullable=False,
                ),
                databricks.SqlTableColumnArgs(
                    name="lower_tick",
                    type="INTEGER",
                    nullable=False,
                ),
                databricks.SqlTableColumnArgs(
                    name="upper_tick",
                    type="INTEGER",
                    nullable=False,
                ),
                databricks.SqlTableColumnArgs(
                    name="amount",
                    type="DECIMAL(18, 8)",
                    nullable=False,
                ),
                databricks.SqlTableColumnArgs(
                    name="amount0",
                    type="DECIMAL(18, 8)",
                    nullable=False,
                ),
                databricks.SqlTableColumnArgs(
                    name="amount1",
                    type="DECIMAL(18, 8)",
                    nullable=False,
                )
            ]
        )
    )

Log output

sdk-v2/provider2.go:520: sdk.helper_schema: changing the 'type' of an existing column is not supported: [email protected]

Affected Resource(s)

SqlTable

Output of pulumi about

CLI          
Version      3.141.0
Go Version   go1.23.3
Go Compiler  gc

Plugins
KIND      NAME        VERSION
resource  aws         6.59.1
resource  databricks  1.56.0
language  python      unknown

Host     
OS       darwin
Version  15.1
Arch     arm64

... stack name and urns here that's private info

Dependencies:
NAME               VERSION
pip                24.3.1
pulumi_aws         6.59.1
pulumi_databricks  1.56.0
setuptools         75.5.0
wheel              0.45.0

Additional context

Edit. I found the issue. If I remove the space in DECIMAL(18, 8) it works. It seems to be a parsing problem. LEaving the issue here because DECIMAL(X,Y) should be the same as DECIMAL(X, Y)

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[fernandodakko]

I'm also running into the same issue when creating a table with complex types. This one doesn't have spaces. It seems the SqlTable object is buggy when parsing the SQL

databricks.SqlTableColumnArgs(
                    name="asks",
                    type="ARRAY<STRUCT<amount DECIMAL(18,8), price DECIMAL(18,6)>>",
                    nullable=False,
                ),
                databricks.SqlTableColumnArgs(
                    name="bids",
                    type="ARRAY<STRUCT<amount DECIMAL(18,8), price DECIMAL(18,6)>>",
                    nullable=False,
),

[guineveresaenger]

Hi @fernandodakko - thank you for filing this issue, and thank you also for updating with your workaround. We're sorry you're having trouble!

This is almost certainly an upstream terraform provider, or even Databricks API issue, as the error comes straight from Databricks.

In fact there are quite a few upstream issues relating to the table type. In particular you might find the comments on this issue helpful. It does sound like formatting here is fussy.

If none of the upstream issues hit your use case, we'd encourage you to file a bug there. Any upstream fixes will be available in this provider on the next release.