sources_have_freshness
Rule: sources_have_freshness
Manifest Rule
sources_have_freshness details
This rule ensures that dbt sources have a freshness configuration defined. Freshness defines the acceptable amount of time between the most recent record and now, for a table to be considered "fresh". At least one of `warn_after` or `error_after` must have a non-null count.
Configuration
- type: Must be
sources_have_freshness. - applies_to: (optional) List of dbt object types to check.
- Default:
["sources"] - Options:
sources
- Default:
Common Rule Config
- name: Human-readable name of the rule.
- severity:
"error"(fail) or"warning"(warn only).- (optional, defaults to
"error"if not specified)
- (optional, defaults to
- description: Human-readable explanation of the rule.
- includes: List of patterns to explicitly include for this rule.
Paths are relative to theoriginal_file_pathfrom the manifest.
Pattern syntax:*matches any characters except/(within a single directory)**matches any characters including/(across directories)^at the start anchors to the beginning of the path$at the end anchors to the end of the path- Without anchors, pattern matches if it appears anywhere in the path (contains)
Examples:^models/staging/- paths starting withmodels/staging/orders- paths containingordersanywhere.sql$- paths ending with.sql^models/*.sql$- SQL files directly inmodels/folder^models/**/*.sql$- SQL files in any subfolder ofmodels/
- excludes: List of patterns to explicitly exclude from this rule.
Uses the same pattern syntax asincludes.
Examples:^models/legacy/- exclude legacy models folder_deprecated- exclude paths containing_deprecated^tests/- exclude test files - model_materializations: Filter models by materialization type. Only applies when
applies_toincludesmodels.
(optional, if not specified all materializations are included)
Built-in types:table,view,incremental,ephemeral,materialized_view. Custom materializations are also supported.
Example:["table", "incremental"]
Example Config
manifest_tests:
- name: "sources_have_freshness"
type: "sources_have_freshness"
description: "All sources must have freshness configured."[[manifest_tests]]
name = "sources_have_freshness"
type = "sources_have_freshness"
description = "All sources must have freshness configured."[[tool.dbtective.manifest_tests]]
name = "sources_have_freshness"
type = "sources_have_freshness"
description = "All sources must have freshness configured."Relevant dbt code
sources:
- name: jaffle_shop
freshness:
warn_after:
count: 24
period: hour
error_after:
count: 48
period: hour
loaded_at_field: updated_at
tables:
- name: orders
- name: customers
- name: stripe
freshness:
warn_after:
count: 12
period: hour
loaded_at_field: created_at
tables:
- name: payments