The dbt-bigquery transformer uses SQL to transform data stored in your warehouse.

Getting Started #

Prerequisites #

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project

Installation and configuration #

Using the Command Line Interface #

  1. Add the dbt-bigquery transformer to your project using meltano add : 

    meltano add transformer dbt-bigquery
  2. Configure the settings below using meltano config .

Next steps #

Follow the remaining steps of the Getting Started guide:

  1. Transform loaded data for analysis

If you run into any issues, learn how to get help.

Settings #

Settings for dbt itself can be configured through dbt_project.yml as usual, which can be found at transform/dbt_project.yml in your Meltano project. dbt also has adapter-specific documentation for BigQuery.

These and other supported settings are documented below. To quickly find the setting you're looking for, use the Table of Contents at the top of the page.

Project Directory (project_dir) #

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set project_dir <project_dir>

export DBT_BIGQUERY_PROJECT_DIR=<project_dir>

Profiles Directory (profiles_dir) #

  • Environment variable: DBT_PROFILES_DIR, alias: DBT_BIGQUERY_PROFILES_DIR
  • Default: $MELTANO_PROJECT_ROOT/transform/profiles/bigquery

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set profiles_dir <profiles_dir>

export DBT_PROFILES_DIR=<profiles_dir>

Authentication Method (auth_method) #

The auth method to use. One of: “oauth”, “oauth-secrets”, or “service-account”

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set auth_method <auth_method>

export DBT_BIGQUERY_AUTH_METHOD=<auth_method>

Project (project) #

The BigQuery project ID.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set project <project>

export DBT_BIGQUERY_PROJECT=<project>

dataset #

The dataset to use.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set dataset <dataset>

export DBT_BIGQUERY_DATASET=<dataset>

Refresh Token (refresh_token) #

The refresh token, if authenticating via oauth-secrets method.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set refresh_token <refresh_token>

export DBT_BIGQUERY_REFRESH_TOKEN=<refresh_token>

Client ID (client_id) #

The client id to use, if authenticating via oauth-secrets method.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set client_id <client_id>

export DBT_BIGQUERY_CLIENT_ID=<client_id>

Client Secret (client_secret) #

The client secret to use, if authenticating via oauth-secrets method.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set client_secret <client_secret>

export DBT_BIGQUERY_CLIENT_SECRET=<client_secret>

Token URI (token_uri) #

The token redirect URI

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set token_uri <token_uri>

export DBT_BIGQUERY_TOKEN_URI=<token_uri>

keyfile #

The path to the keyfile.json` to use, if authenticating via service-account method.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config dbt-bigquery set keyfile <keyfile>

export DBT_BIGQUERY_KEYFILE=<keyfile>

Looking for help? #

If you're having trouble getting the dbt-bigquery transformer to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!