Section 1: How dbt thinks
Your first project: sources, models and ref
Let us create a project and build a model on top of raw data.
python -m pip install dbt-core dbt-duckdb
dbt init subscriptions
cd subscriptions
dbt debug
Declare your sources
Raw tables are declared once as sources. This gives them a name inside dbt and lets you add tests and freshness checks.
# models/staging/_sources.yml
sources:
- name: app
schema: raw
tables:
- name: customers
- name: payments
A staging model
Staging models do light, one to one cleanup: rename columns, cast types, convert cents to currency units. One staging model per source table.
-- models/staging/stg_payments.sql
select
id as payment_id,
customer_id,
cast(amount_cents as decimal(12, 2)) / 100 as amount,
lower(status) as status,
cast(created_at as timestamp) as paid_at
from {{ source('app', 'payments') }}
Build on top with ref
-- models/marts/fct_customer_revenue.sql
select
customer_id,
count(*) as payments,
sum(amount) as revenue
from {{ ref('stg_payments') }}
where status = 'succeeded'
group by customer_id
ref('stg_payments') does two things. At compile time it becomes the fully qualified name of that relation in the current target. At planning time it tells dbt that fct_customer_revenue depends on stg_payments, so dbt always builds them in the right order.
Never hard code a table name like analytics.stg_payments. If you do, dbt cannot see the dependency, and your model will read from the wrong schema when you switch between development and production targets.
Run dbt run and you will see both models built. Run dbt docs generate followed by dbt docs serve and open the lineage graph: source, staging, mart, connected by arrows you never had to draw.