Skip to content

[Bug]: Cloud_Spanner_to_GCS_Avro template ignores 'tableNames', always exports all tables #3917

Description

@Ziemowit-LH

Related Template(s)

Cloud_Spanner_to_GCS_Avro

Template Version

latest

What happened?

Template ignores tableNames parameter.

gcloud dataflow jobs run spanner-export-job \
  --gcs-location=gs://dataflow-templates/latest/Cloud_Spanner_to_GCS_Avro \
  --service-account-email=tmp-df-spanner-import@${PROJECT}.iam.gserviceaccount.com \
  --project=${PROJECT} \
  --region=${REGION} \
  --network=${NETWORK} \
  --subnetwork=${SUBNETWORK} \
  --disable-public-ips \
  --parameters="^|^instanceId=${INSTANCE_ID}|databaseId=${DATABASE_ID}|tableNames=table1,table2|shouldExportRelatedTables=false|outputDir=${EXPORT_LOCATION}"

Export always contains all tables.
Even if tableNames contains a single, independent table.

Export fails (as expected) when nonexistent table is defined.

Resulting export manifest:

{
  "tables": [{
    "name": "mutex",
    "manifestFile": "mutex-manifest.json"
  }, {
    "name": "ObserverVersion",
    "manifestFile": "ObserverVersion-manifest.json"
  }, {
    "name": "MigrationsHistory",
    "manifestFile": "MigrationsHistory-manifest.json"
  }, {
    "name": "table1",
    "manifestFile": "table1-manifest.json"
  }, {
    "name": "table2",
    "manifestFile": "table2-manifest.json"
  }],
  "databaseOptions": [{
    "optionName": "version_retention_period",
    "optionType": "STRING",
    "optionValue": "1h"
  }]
}

Relevant log output

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions