dbt-testgen: Automate Your dbt Test YAML Generation

dbt-testgen is a powerful dbt package designed to automatically generate dbt test YAML files based on real data. Inspired by dbt-codegen, this tool simplifies the process of creating comprehensive tests for your data models.

Installation

To get started with dbt-testgen, ensure you have dbt version 1.2.x or higher. Add the following to your packages.yml file:

packages:
 - package: kgmcquate/testgen
 version: 1.0.3

Supported Test Types

dbt-testgen can generate a variety of tests using built-in tests, dbt_utils, and dbt-expectations, including:

Uniqueness
Not Null
String Length
Range
Accepted Values
Recency

Usage

The dbt configuration YAML is generated by a Jinja macro called get_test_suggestions. You can run it with the following command:

“from dbt 1.9, tests is defined as data_tests. So update the tests in generated yml file to data_tests”

dbt compile -q --inline "{{ testgen.get_test_suggestions(ref('stg_system__account_manager')) }}"

Here’s an example output:

models:
 - name: stg_system__account_manager
 columns:
 - name: account_id
 tests:
 - unique
 - not_null
 - name: account_manager_flag
 tests:
 - accepted_values:
 values:
 - '1'

Advanced Usage

For more advanced configurations, you can set the following arguments:

Example Command

Here’s an example of running the command with advanced arguments:

dbt compile -q --inline "{{ testgen.get_test_suggestions(ref('stg_system__account_manager'), exclude_types=['float'], tests=['uniqueness', 'accepted_values', 'range', 'string_length'], limit=1000000000) }}"

This will generate the following output:

models:
 - name: stg_system__account_manager
 columns:
 - name: account_id
 tests:
 - unique
 - not_null
 - name: account_manager_flag
 tests:
 - accepted_values:
 values:
 - '1'

With dbt-testgen, you can streamline your testing process and ensure your data models are thoroughly validated with minimal effort. Happy testing!

References

[**GitHub - kgmcquate/dbt-testgen: Generate DBT tests based on sample data**
*Generate DBT tests based on sample data. Contribute to kgmcquate/dbt-testgen development by creating an account on…*github.com](https://github.com/kgmcquate/dbt-testgen "https://github.com/kgmcquate/dbt-testgen")

This article was originally published at https://medium.com/@aradsouza/dbt-testgen-automate-your-dbt-test-yaml-generation-f26de83c0537