Skip to content

Add ol-dbt diff command to QA migrated mart/reporting models against pre-migration output #2407

Description

@quazi-h

Description/Context

Part of #2072 (Migrate mart and reporting models to use the dimensional layer). As more sub-issues under this epic move from triage into actual dimensional-layer migrations, we need a repeatable, field-level QA safeguard to confirm a migrated model produces identical output to the model it replaces, before it's deployed to production.

Requirements:

  • Build the pre-migration and post-migration versions of a model side-by-side against real production data, with zero manual data copying.
  • Compare output with field-level precision (not just row counts or spot checks).
  • Usable by any agent/contributor working a migration ticket under [Epic] Migrate mart and reporting models to use the dimensional layer #2072, not a one-off local script.

This repo already has the infrastructure to build against real production data with no data copy: the dev_local DuckDB target reads production Iceberg tables directly via ol-dbt local register (Glue-backed views), and override_ref/override_source transparently fall back to those views when a model isn't built locally yet. What's missing is the comparison step itself.

Plan/Design

  • Add dbt-labs/dbt_audit_helper to src/ol_dbt/packages.yml for row/column-level relation comparison macros (compare_relations, compare_column_values, compare_row_counts).
  • Add a new ol-dbt diff command in src/ol_dbt_cli/ol_dbt_cli/commands/diff.py (following the existing pattern in impact.py/local_dev.py), e.g. ol-dbt diff --old <model> --new <model> [--target dev_local|dev_production], wrapping dbt run-operation audit_helper.compare_relations (and friends) and surfacing a pass/fail summary plus mismatched rows for inspection.
  • Register the command in src/ol_dbt_cli/ol_dbt_cli/cli.py and add tests under src/ol_dbt_cli/tests/test_diff.py.
  • Out of scope: evaluating third-party tools like Datafold's data-diff — same-engine comparison via dbt_audit_helper covers this need without extra infrastructure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions