You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Part of #2072 (Migrate mart and reporting models to use the dimensional layer). As more sub-issues under this epic move from triage into actual dimensional-layer migrations, we need a repeatable, field-level QA safeguard to confirm a migrated model produces identical output to the model it replaces, before it's deployed to production.
Requirements:
Build the pre-migration and post-migration versions of a model side-by-side against real production data, with zero manual data copying.
Compare output with field-level precision (not just row counts or spot checks).
This repo already has the infrastructure to build against real production data with no data copy: the dev_local DuckDB target reads production Iceberg tables directly via ol-dbt local register (Glue-backed views), and override_ref/override_source transparently fall back to those views when a model isn't built locally yet. What's missing is the comparison step itself.
Plan/Design
Add dbt-labs/dbt_audit_helper to src/ol_dbt/packages.yml for row/column-level relation comparison macros (compare_relations, compare_column_values, compare_row_counts).
Add a new ol-dbt diff command in src/ol_dbt_cli/ol_dbt_cli/commands/diff.py (following the existing pattern in impact.py/local_dev.py), e.g. ol-dbt diff --old <model> --new <model> [--target dev_local|dev_production], wrapping dbt run-operation audit_helper.compare_relations (and friends) and surfacing a pass/fail summary plus mismatched rows for inspection.
Register the command in src/ol_dbt_cli/ol_dbt_cli/cli.py and add tests under src/ol_dbt_cli/tests/test_diff.py.
Out of scope: evaluating third-party tools like Datafold's data-diff — same-engine comparison via dbt_audit_helper covers this need without extra infrastructure.
Description/Context
Part of #2072 (Migrate mart and reporting models to use the dimensional layer). As more sub-issues under this epic move from triage into actual dimensional-layer migrations, we need a repeatable, field-level QA safeguard to confirm a migrated model produces identical output to the model it replaces, before it's deployed to production.
Requirements:
This repo already has the infrastructure to build against real production data with no data copy: the
dev_localDuckDB target reads production Iceberg tables directly viaol-dbt local register(Glue-backed views), andoverride_ref/override_sourcetransparently fall back to those views when a model isn't built locally yet. What's missing is the comparison step itself.Plan/Design
dbt-labs/dbt_audit_helpertosrc/ol_dbt/packages.ymlfor row/column-level relation comparison macros (compare_relations,compare_column_values,compare_row_counts).ol-dbt diffcommand insrc/ol_dbt_cli/ol_dbt_cli/commands/diff.py(following the existing pattern inimpact.py/local_dev.py), e.g.ol-dbt diff --old <model> --new <model> [--target dev_local|dev_production], wrappingdbt run-operation audit_helper.compare_relations(and friends) and surfacing a pass/fail summary plus mismatched rows for inspection.src/ol_dbt_cli/ol_dbt_cli/cli.pyand add tests undersrc/ol_dbt_cli/tests/test_diff.py.data-diff— same-engine comparison viadbt_audit_helpercovers this need without extra infrastructure.