columns (5)

Catalog rules use both the manifest.json and catalog.json artifacts. Catalog rules require an active database connection to be generated using dbt docs generate. These files can become out of sync during development (for example, when running dbtective in pre-commit hooks), especially if files are moved or renamed and only one of the commands generating manifest.json is run. For more information, see the dbt documentation on manifest.json.

To ensure your catalog is up to date, delete it from the dbt target folder and regenerate it using dbt docs generate. Future updates to dbtective will include an option to automate this process with a specific flag. It's also possible to disable `catalog` based rules using the `--only-manifest` flag.

Configuration

• type: Must be columns_all_documented.
• applies_to: (optional) List of dbt object types to include.
  • Default: ["models", "seeds", "snapshots", "sources", "semantic_models"]
  • Options: models, seeds, snapshots, sources, macros,semantic_models

Example Config

catalog_tests:
  - name: "all_columns_should_be_documented"
    type: "columns_all_documented"
    description: "Everything must have a description."
    # severity: "warning"  (optional)
    # applies_to: ['models', 'seeds']  (optional)
    # includes: ["path/to/include/*"]
    # excludes: ["path/to/exclude/*"]

[[catalog_tests]]
name = "all_columns_should_be_documented"
type = "columns_all_documented"
description = "Everything must have a description."
# severity = "warning"  # (optional)
# applies_to = ["models", "seeds"]  # (optional)
# includes = ["path/to/include/*"]
# excludes = ["path/to/exclude/*"]

[[tool.dbtective.catalog_tests]]
name = "all_columns_should_be_documented"
type = "columns_all_documented"
description = "Everything must have a description."
# severity = "warning"  # (optional)
# applies_to = ["models", "seeds"]  # (optional)
# includes = ["path/to/include/*"]
# excludes = ["path/to/exclude/*"]

models:
  - name: model_without_columns_documented
    columns:
      - column_1
      - column_2
  # Example if the model has 2 columns
  - name: model_with_missing_documentation_for_column_2
    columns:
      - column_1
  - name: model_without_columns_documented

Catalog rules use both the manifest.json and catalog.json artifacts. Catalog rules require an active database connection to be generated using dbt docs generate. These files can become out of sync during development (for example, when running dbtective in pre-commit hooks), especially if files are moved or renamed and only one of the commands generating manifest.json is run. For more information, see the dbt documentation on manifest.json.

To ensure your catalog is up to date, delete it from the dbt target folder and regenerate it using dbt docs generate. Future updates to dbtective will include an option to automate this process with a specific flag. It's also possible to disable `catalog` based rules using the `--only-manifest` flag.

Configuration

• type: Must be columns_name_convention.
• applies_to: (optional) List of dbt object types to include.
  • Default: ["models", "seeds", "snapshots"]
  • Options: models, seeds, snapshots, sources
• pattern: The naming convention pattern to enforce. Can be one of the following presets or a custom regex pattern.
  • Presets:
    • snake_case: lowercase letters, numbers, and underscores (e.g., user_id, created_at)
    • kebab-case: lowercase letters, numbers, and hyphens (e.g., user-id, created-at)
    • camelCase: starts with a lowercase letter, followed by uppercase letters for new words (e.g., userId, createdAt)
    • PascalCase: starts with an uppercase letter, followed by uppercase letters for new words (e.g., UserId, CreatedAt)
  • Custom Regex: Any valid regex pattern to match against column names.
• data_types: (optional) List of SQL data types to filter columns by. Only columns with these data types will be checked included in the naming convention rule. If not specified, all columns are included. This can cause mismatches when --only-manifest is being used!
  • Default: All data types
  • Example: If you want all datetime columns to end with ‘dt’, you can set data_types: ['date', 'date_time', 'timestamp', 'timestamptz'] with pattern .*_dt$
  • Available types: integer, big_int, small_int, tiny_int, decimal, numeric, float, double, real, string, text, varchar, char, date, date_time, time, timestamp, timestamptz, boolean, json, jsonb, array, object, variant, binary, varbinary, uuid, interval
• use_database_columns: (optional) Whether to check column names from the database catalog (true) or from the manifest/dbt code (false). This is useful for case-insensitive databases (Snowflake). Updating casing in dbt code does not update the actual materialized table. Set to false to validate against the column names as written in your dbt project instead of as they appear in the database. This option is independent of the --only-manifest flag.
  • Default: true

Example Config

catalog_tests:
  # Basic snake_case check
  - name: "columns_snake_case"
    type: "columns_name_convention"
    description: "All column names must be snake_case."
    pattern: "snake_case"
    # severity: "warning"  (optional)
    # applies_to: ['models', 'sources']  (optional)
    # includes: ["path/to/include/*"]  (optional)
    # excludes: ["path/to/exclude/*"]  (optional)

  # Custom regex pattern
  - name: "columns_custom_pattern"
    type: "columns_name_convention"
    description: "All columns must match custom pattern."
    pattern: "^[a-z][a-z0-9_]*$"

    # Use dbt project column names instead of the materialized table
  # Useful for case-insensitive databases
  - name: "columns_snake_case_dbt_project"
    type: "columns_name_convention"
    description: "All column names must be snake_case."
    pattern: "snake_case"
    use_database_columns: false

# Basic snake_case check
[[catalog_tests]]
name = "columns_snake_case"
type = "columns_name_convention"
description = "All column names must be snake_case."
pattern = "snake_case"
# severity = "warning"  # (optional)
# applies_to = ["models", "sources"]  # (optional)
# includes = ["path/to/include/*"]  # (optional)
# excludes = ["path/to/exclude/*"]  # (optional)

# Custom regex pattern
[[catalog_tests]]
name = "columns_custom_pattern"
type = "columns_name_convention"
description = "All columns must match custom pattern."
pattern = "^[a-z][a-z0-9_]*$"

# Use dbt project column names instead of the materialized table
# Useful for case-insensitive databases
[[catalog_tests]]
name = "columns_snake_case_dbt_project"
type = "columns_name_convention"
description = "All column names must be snake_case."
pattern = "snake_case"
use_database_columns = false

# Basic snake_case check
[[tool.dbtective.catalog_tests]]
name = "columns_snake_case"
type = "columns_name_convention"
description = "All column names must be snake_case."
pattern = "snake_case"
# severity = "warning"  # (optional)
# applies_to = ["models", "sources"]  # (optional)
# includes = ["path/to/include/*"]  # (optional)
# excludes = ["path/to/exclude/*"]  # (optional)

# Custom regex pattern
[[tool.dbtective.catalog_tests]]
name = "columns_custom_pattern"
type = "columns_name_convention"
description = "All columns must match custom pattern."
pattern = "^[a-z][a-z0-9_]*$"

# Use dbt project column names instead of the materialized table
# Useful for case-insensitive databases
[[tool.dbtective.catalog_tests]]
name = "columns_snake_case_dbt_project"
type = "columns_name_convention"
description = "All column names must be snake_case."
pattern = "snake_case"
use_database_columns = false

-- Example model SQL
SELECT
    snake_case,           -- PASS: snake_case
    camelCase,            -- PASS: camelCase
    PascalCase            -- PASS: PascalCase
FROM users

Catalog rules use both the manifest.json and catalog.json artifacts. Catalog rules require an active database connection to be generated using dbt docs generate. These files can become out of sync during development (for example, when running dbtective in pre-commit hooks), especially if files are moved or renamed and only one of the commands generating manifest.json is run. For more information, see the dbt documentation on manifest.json.

To ensure your catalog is up to date, delete it from the dbt target folder and regenerate it using dbt docs generate. Future updates to dbtective will include an option to automate this process with a specific flag. It's also possible to disable `catalog` based rules using the `--only-manifest` flag.

Configuration

• type: Must be columns_have_description.
• applies_to: (optional) List of dbt object types to include.
  • Default: ["models", "seeds", "snapshots", "sources"]
  • Options: models, seeds, snapshots, sources

Example Config

catalog_tests:
  - name: "all_columns_must_have_descriptions"
    type: "columns_have_description"
    description: "All documented columns must have non-empty descriptions."
    # severity: "warning"  (optional)
    # applies_to: ['models', 'seeds']  (optional)
    # includes: ["path/to/include/*"]
    # excludes: ["path/to/exclude/*"]

[[catalog_tests]]
name = "all_columns_must_have_descriptions"
type = "columns_have_description"
description = "All documented columns must have non-empty descriptions."
# severity = "warning"  # (optional)
# applies_to = ["models", "seeds"]  # (optional)
# includes = ["path/to/include/*"]
# excludes = ["path/to/exclude/*"]

[[tool.dbtective.catalog_tests]]
name = "all_columns_must_have_descriptions"
type = "columns_have_description"
description = "All documented columns must have non-empty descriptions."
# severity = "warning"  # (optional)
# applies_to = ["models", "seeds"]  # (optional)
# includes = ["path/to/include/*"]
# excludes = ["path/to/exclude/*"]

models:
  - name: customers
    columns:
      - name: id
        description: "Customer ID" # PASS: has description
      - name: name
        description: "" # FAIL: empty description
      - name: email
        # FAIL: no description field

Catalog rules use both the manifest.json and catalog.json artifacts. Catalog rules require an active database connection to be generated using dbt docs generate. These files can become out of sync during development (for example, when running dbtective in pre-commit hooks), especially if files are moved or renamed and only one of the commands generating manifest.json is run. For more information, see the dbt documentation on manifest.json.

To ensure your catalog is up to date, delete it from the dbt target folder and regenerate it using dbt docs generate. Future updates to dbtective will include an option to automate this process with a specific flag. It's also possible to disable `catalog` based rules using the `--only-manifest` flag.

Identifies columns that match specified “invalid” patterns and flags them as violations, suggesting they should be renamed to the canonical name. You can also define exceptions for columns that should be allowed even when matched. Can be both regex or strings.

Configuration

• type: Must be columns_canonical_name.
• canonical: The preferred/canonical column name (e.g., zip_code).
• invalid_names: List of patterns that should be flagged as violations. Each pattern can be:
  • Strings: An exact string match (e.g., postal_code)
  • Regex: A pattern starting with ^, ending with $, or containing .* or .+ (e.g., ^zip.*)
• exceptions: (optional) List of patterns to exclude from violations. Columns matching these patterns will not be flagged even if they match invalid_names. Uses the same literal/regex format as invalid_names.
• applies_to: (optional) List of dbt object types to include.
  • Default: ["models", "seeds", "snapshots"]
  • Options: models, seeds, snapshots, sources

Example Config

catalog_tests:
  - name: "canonical_zip_code"
    type: "columns_canonical_name"
    description: "All zip-related columns should be named 'zip_code'."
    canonical: "zip_code"
    invalid_names:
      - "postal_code" # literal match
      - "^zip" # regex: matches zipcode, zip_cd, etc.
    # exceptions:
    #   - "zip_code_legacy"  # allow this specific column
    # severity: "warning"  (optional)
    # applies_to: ['models', 'sources']  (optional)
    # includes: ["path/to/include/*"]  (optional)
    # excludes: ["path/to/exclude/*"]  (optional)

[[catalog_tests]]
name = "canonical_zip_code"
type = "columns_canonical_name"
description = "All zip-related columns should be named 'zip_code'."
canonical = "zip_code"
invalid_names = ["postal_code", "^zip"]
# exceptions = ["zip_code_legacy"]  # (optional)
# severity = "warning"  # (optional)
# applies_to = ["models", "sources"]  # (optional)
# includes = ["path/to/include/*"]  # (optional)
# excludes = ["path/to/exclude/*"]  # (optional)

[[tool.dbtective.catalog_tests]]
name = "canonical_zip_code"
type = "columns_canonical_name"
description = "All zip-related columns should be named 'zip_code'."
canonical = "zip_code"
invalid_names = ["postal_code", "^zip"]
# exceptions = ["zip_code_legacy"]  # (optional)
# severity = "warning"  # (optional)
# applies_to = ["models", "sources"]  # (optional)
# includes = ["path/to/include/*"]  # (optional)
# excludes = ["path/to/exclude/*"]  # (optional)

SELECT
    zip_code,            -- PASS: canonical name
    postal_code,         -- FAIL: matches invalid_names literal
    zipcode,             -- FAIL: matches invalid_names regex ^zip
    zip_code_legacy,     -- PASS: matches exception
    other_column         -- PASS: not in invalid_names
FROM source_table

Catalog rules use both the manifest.json and catalog.json artifacts. Catalog rules require an active database connection to be generated using dbt docs generate. These files can become out of sync during development (for example, when running dbtective in pre-commit hooks), especially if files are moved or renamed and only one of the commands generating manifest.json is run. For more information, see the dbt documentation on manifest.json.

To ensure your catalog is up to date, delete it from the dbt target folder and regenerate it using dbt docs generate. Future updates to dbtective will include an option to automate this process with a specific flag. It's also possible to disable `catalog` based rules using the `--only-manifest` flag.

Configuration

• type: Must be columns_have_data_type.
• min_coverage: (optional) Minimum percentage of columns that must have data types defined per dbt-object.
  • Default: 100 (all columns must have data types)
  • Example: 90 means at least 90% of columns must have data types
• applies_to: (optional) List of dbt object types to include.
  • Default: ["models", "seeds", "snapshots", "sources"]
  • Options: models, seeds, snapshots, sources

Example Config

catalog_tests:
  - name: "columns_have_data_type"
    type: "columns_have_data_type"
    description: "All columns must have data types defined."
    # min_coverage: 100  (default: all columns, use e.g. 90 for 90%)
    # severity: "warning"  (optional)
    # applies_to: ['models', 'seeds']  (optional)
    # includes: ["path/to/include/*"]
    # excludes: ["path/to/exclude/*"]

[[catalog_tests]]
name = "columns_have_data_type"
type = "columns_have_data_type"
description = "All columns must have data types defined."
# min_coverage = 100  # (default: all columns, use e.g. 90 for 90%)
# severity = "warning"  # (optional)
# applies_to = ["models", "seeds"]  # (optional)
# includes = ["path/to/include/*"]
# excludes = ["path/to/exclude/*"]

[[tool.dbtective.catalog_tests]]
name = "columns_have_data_type"
type = "columns_have_data_type"
description = "All columns must have data types defined."
# min_coverage = 100  # (default: all columns, use e.g. 90 for 90%)
# severity = "warning"  # (optional)
# applies_to = ["models", "seeds"]  # (optional)
# includes = ["path/to/include/*"]
# excludes = ["path/to/exclude/*"]

models:
  - name: customers
    columns:
      - name: id
        data_type: integer # PASS: has data type
      - name: name
        data_type: varchar # PASS: has data type
      - name: email
        # FAIL: no data_type field