What a dbt model actually is
-- models/stg_events.sql
select
event_id,
user_id,
event_type,
amount
from {{ source('article_demo', 'raw_events') }}
A SELECT statement, in a .sql file, referencing a raw table through source() instead of a hardcoded name. That's the entire model. No CREATE VIEW, no CREATE TABLE, nothing procedural, dbt decides what to actually build from this based on separate configuration, and the model file itself stays as plain, readable SQL as if it were a query in any SQL client.
A second model, depending on the first
-- models/mart_user_totals.sql
select
user_id,
count(*) as event_count,
sum(amount) as net_amount
from {{ ref('stg_events') }}
group by user_id
ref('stg_events') instead of a hardcoded table name is the entire mechanism dbt uses to understand that this model depends on the other one. No separate configuration file lists the dependency graph, it's inferred directly from which models reference which.
Running it for real
$ dbt run
1 of 2 START sql view model stg_events ........... [RUN]
1 of 2 OK created sql view model stg_events ...... [CREATE VIEW in 0.11s]
2 of 2 START sql view model mart_user_totals ..... [RUN]
2 of 2 OK created sql view model mart_user_totals [CREATE VIEW in 0.05s]
Done. PASS=2 WARN=0 ERROR=0 SKIP=0 NO-OP=0 REUSED=0 TOTAL=2
Both real views got created in a real database, in the correct order, stg_events before mart_user_totals, entirely because of the ref() call, never because of an explicitly stated order anywhere. Querying the result directly confirms it's real, correct, aggregated data:
user_id | event_count | net_amount
---------+-------------+------------
101 | 2 | 62.49
102 | 2 | 10.00
103 | 2 | 90.25
Tests, in the same tool, against the same models
# models/schema.yml
models:
- name: stg_events
columns:
- name: event_id
tests: [unique, not_null]
- name: mart_user_totals
columns:
- name: user_id
tests: [unique]
$ dbt test
1 of 3 PASS not_null_stg_events_event_id
2 of 3 PASS unique_mart_user_totals_user_id
3 of 3 PASS unique_stg_events_event_id
Done. PASS=3 WARN=0 ERROR=0 SKIP=0 NO-OP=0 REUSED=0 TOTAL=3
Three real assertions about the data, checked with a single command, using the exact same model definitions the pipeline itself runs on. A genuinely broken pipeline, a duplicate event_id sneaking into the raw source, or a user_id somehow appearing twice in the aggregated mart, fails here, with a real non-zero exit code any CI system can act on, rather than being discovered downstream by whoever eventually notices a report looks wrong.
Why this matters more than the syntax
The real shift dbt represents isn't the SQL, teams have always written SQL to build reporting tables. It's treating that SQL like software: every model lives in a real file, in real version control, reviewable in a real pull request, with real, automated tests attached directly to the same definitions instead of living in a separate spreadsheet or a person's memory of what "should" be true. The dependency graph being inferred from ref() calls, rather than hand-maintained, means it can never silently drift out of sync with what the models actually do, the way a manually-drawn pipeline diagram always eventually does.
The takeaway
A dbt project is close to the simplest possible shape a data pipeline can take: SQL files, a real dependency graph inferred automatically from how those files reference each other, and tests that run against the exact same models in production. Nothing here needed a scheduler, a custom framework, or infrastructure beyond a database connection, confirmed directly by the real dbt run and dbt test output above.
Comments
Loading comments...