Skip to content

Add yu-iskw/dbt-bigquery-federation package - #448

Open
yu-iskw wants to merge 1 commit into
dbt-labs:mainfrom
yu-iskw:cursor/add-dbt-bigquery-federation-2540
Open

Add yu-iskw/dbt-bigquery-federation package#448
yu-iskw wants to merge 1 commit into
dbt-labs:mainfrom
yu-iskw:cursor/add-dbt-bigquery-federation-2540

Conversation

@yu-iskw

@yu-iskw yu-iskw commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Description

dbt_bigquery_federation makes BigQuery EXTERNAL_QUERY practical from dbt. It plans type conversions from Git-reviewed schema pins and renders federated table expressions for Cloud SQL PostgreSQL, AlloyDB PostgreSQL, and Spanner GoogleSQL.

https://docs.cloud.google.com/bigquery/docs/federated-queries-intro

Hub identity after indexing:

  • GitHub repo: yu-iskw/dbt-bigquery-federation
  • Hub package name: yu-iskw/dbt_bigquery_federation

A semver git tag (for example v0.1.0) still needs to be created so hubcap can publish the first version after merge.

Link to your package's repository: https://github.com/yu-iskw/dbt-bigquery-federation

Checklist

This checklist is a cut down version of the best practices that we have identified as the package hub has grown. Although meeting these checklist items is not a prerequisite to being added to the Hub, we have found that packages which don't conform provide a worse user experience.

Real world usage

Using coding agents to help you build your package is totally fine, but please don't submit it to the package hub until you've used it in your own production environment for an extended period of time.

  • (Required): I have been using this package in production and am satisfied with its behavior.

First run experience

  • (Required): The package includes a licence file detectable by GitHub, such as the Apache 2.0 or MIT licence.
  • The package includes a README which explains how to get started with the package and customise its behaviour
  • The README indicates which data warehouses/platforms are expected to work with this package

Customisability

  • The package uses ref or source, instead of hard-coding table references.

This is a macro package, not a data-transformation package. Call sites pass connection / schema / table, and pins live in vars.dbt_bigquery_federation.

Dependencies

Dependencies on dbt Core

  • The package has set a supported require-dbt-version range in dbt_project.yml. Example: A package which depends on functionality added in dbt Core 1.2 should set its require-dbt-version property to [">=1.2.0", "<2.0.0"].

Dependencies on other packages defined in packages.yml:

  • Dependencies are imported from the dbt Package Hub when available, as opposed to a git installation.
  • Dependencies contain the widest possible range of supported versions, to minimise issues in dependency resolution.
  • In particular, dependencies are not pinned to a patch version unless there is a known incompatibility.

No root packages.yml dependencies.

Interoperability

  • The package does not override dbt Core behaviour in such a way as to impact other dbt resources (models, tests, etc) not provided by the package.
  • The package uses the cross-database macros built into dbt Core where available, such as {{ dbt.except() }} and {{ dbt.type_string() }}.
  • The package disambiguates its resource names to avoid clashes with nodes that are likely to already exist in a project. For example, packages should not provide a model simply called users.

Versioning

  • (Required): The package's git tags validates against the regex defined in hubcap/version.py (examples).
  • The package's version follows the guidance of Semantic Versioning 2.0.0. (Note in particular the recommendation for production-ready packages to be version 1.0.0 or above)

Register the BigQuery EXTERNAL_QUERY federation macros on dbt Hub so hubcap can index future semver tags.

Co-authored-by: Yu Ishikawa <yu-iskw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants