Skip to content

Support on_warning_callback in watcher execution mode #2979

Description

@pankajkoti

Summary

on_warning_callback is honored in local, virtualenv, kubernetes, and gcp_gke execution modes but not in watcher. This issue tracks adding support so warning callbacks behave consistently across execution modes. See the Warning behavior docs.

Current behavior

The callback is threaded into each watcher test/source task's task_args, but the consumer sensors never store or invoke it, so it is silently dropped. The producer also classifies warn as success (correct, since a warning must not fail the task), which currently means the warn detail is discarded. One case already works: TestBehavior.AFTER_ALL remaps its single test task to local, so that task honors the callback. The gap is the default AFTER_EACH test warnings and source-freshness warnings.

Desired behavior

In watcher mode, when warn results occur, the matching consumer task invokes on_warning_callback with the same enriched context local mode provides (test_names and test_results in the Airflow context), while the task still succeeds. Parity should hold across all three watcher variants (WATCHER, WATCHER_KUBERNETES, WATCHER_GCP_GKE).

Scope

  • In scope: test warnings (AFTER_EACH) and source-freshness warnings.
  • Out of scope: TestBehavior.BUILD warnings. Watcher mode has no build consumer (DbtBuildWatcherOperator raises NotImplementedError), so that would be a separate effort.

Implementation notes

The warn data already reaches the producer via two paths (verified against dbt-core 1.9.0):

  • Test warns flow through the JSON log stream (store_dbt_resource_status_from_log receives node_status="warn" and routes it into push_test_result_or_aggregate, which currently collapses it to pass/fail).
  • Source-freshness warns come from _apply_source_freshness / _push_source_freshness_results, not the log parser.

The work is to capture the warn detail at both entry points, then have the consumer sensors read it and invoke the callback on both the synchronous poke path and the deferred execute_complete path. The message-field extraction needs an empirical check across dbt versions.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    dbt:testPrimarily related to dbt test command or functionalityenhancementNew feature or requestexecution:callbackTasks related to callback when executing tasksexecution:watcherRelated to Watcher execution mode

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions