Skip to content

Added dbt-streams-task to hub - #414

Open
AidanHarveyNelson wants to merge 1 commit into
dbt-labs:mainfrom
AidanHarveyNelson:feature/add-dbt-streams-task
Open

Added dbt-streams-task to hub#414
AidanHarveyNelson wants to merge 1 commit into
dbt-labs:mainfrom
AidanHarveyNelson:feature/add-dbt-streams-task

Conversation

@AidanHarveyNelson

@AidanHarveyNelson AidanHarveyNelson commented Mar 21, 2026

Copy link
Copy Markdown

Description

This package is aimed at helping users manage streams and tasks within DBT. User's can declare there table SQL normally and the package will create a merge task that will ensure that the data is loaded in on a schedule they define. They are also able to create streams to better facilitate incremental loads and have these streams mentioned previously reference them.

No more managing these objects in a different repo it will now be possible to do it all inside DBT

Link to your package's repository: https://github.com/AidanHarveyNelson/dbt-streams-tasks

Checklist

This checklist is a cut down version of the best practices that we have identified as the package hub has grown. Although meeting these checklist items is not a prerequisite to being added to the Hub, we have found that packages which don't conform provide a worse user experience.

First run experience

  • (Required): The package includes a licence file detectable by GitHub, such as the Apache 2.0 or MIT licence.
  • The package includes a README which explains how to get started with the package and customise its behaviour
  • The README indicates which data warehouses/platforms are expected to work with this package

Customisability

  • The package uses ref or source, instead of hard-coding table references.

Packages for data transformation (delete if not relevant):

  • provide a mechanism (such as variables) to customise the location of source tables.
  • do not assume database/schema names in sources.

Dependencies

Dependencies on dbt Core

  • The package has set a supported require-dbt-version range in dbt_project.yml. Example: A package which depends on functionality added in dbt Core 1.2 should set its require-dbt-version property to [">=1.2.0", "<2.0.0"].

Dependencies on other packages defined in packages.yml:

  • Dependencies are imported from the dbt Package Hub when available, as opposed to a git installation.
  • Dependencies contain the widest possible range of supported versions, to minimise issues in dependency resolution.
  • In particular, dependencies are not pinned to a patch version unless there is a known incompatibility.

Interoperability

  • The package does not override dbt Core behaviour in such a way as to impact other dbt resources (models, tests, etc) not provided by the package.
  • The package uses the cross-database macros built into dbt Core where available, such as {{ dbt.except() }} and {{ dbt.type_string() }}.
  • The package disambiguates its resource names to avoid clashes with nodes that are likely to already exist in a project. For example, packages should not provide a model simply called users.

Versioning

  • (Required): The package's git tags validates against the regex defined in hubcap/version.py (examples).
  • The package's version follows the guidance of Semantic Versioning 2.0.0. (Note in particular the recommendation for production-ready packages to be version 1.0.0 or above)

@joellabes

Copy link
Copy Markdown
Contributor

@AidanHarveyNelson thanks for submitting! Have you been using this in a production environment for a while? How has it gone?

@dbeatty10

Copy link
Copy Markdown
Contributor

@AidanHarveyNelson could you answer the questions above? That will help us review your package submission.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants