Environment
| Field |
Value |
| Astronomer Cosmos version |
1.15.1 |
| dbt Core version |
1.12.0 |
| dbt adapter versions |
Not applicable (rendering from a committed manifest.json, no adapter invoked) |
| LoadMode |
DBT_MANIFEST |
| ExecutionMode |
LOCAL (rendering only, not relevant to the bug) |
| InvocationMode |
Not applicable |
| Airflow version |
3.3.0 |
| Python |
3.11 |
| Operating System |
macOS (Darwin) |
Summary
In LoadMode.DBT_MANIFEST, a selector that combines the graph operator (+) with a bare folder name or package name (for example my_folder+ or my_package+) resolves to an empty set. Cosmos resolves the graph-operator root by exact node name only, so folder and package tokens match nothing and the select or exclude silently has no effect. The same selectors work under dbt ls, which delegates resolution to dbt.
For exclude this is a correctness issue. The intended nodes are not removed and remain in the rendered DAG.
Background: what #2357 addressed
PR #2357 ("Fix inclusion of package models, and selecting/excluding") added manifest-mode support for:
- Including installed-package models in the graph with correct file paths.
package:<name> select and exclude.
- Bare identifiers resolving the dbt way (by package name, node name, or folder name).
- Select and exclude by folder name.
All of the above work. The gap below is the graph-operator (+) case layered on top of the folder and package tokens.
What still needs addressing
Graph-operator root resolution should match dbt. The token attached to + should resolve through the same methods as a bare identifier (node name, folder name, package name, fqn, path), not by exact node name alone.
How to reproduce
Project: fhir-dbt-analytics (public), loaded from its target/manifest.json with LoadMode.DBT_MANIFEST.
from cosmos.config import ProjectConfig, RenderConfig, ExecutionConfig
from cosmos.constants import LoadMode
from cosmos.dbt.graph import DbtGraph
def n_models(selector):
g = DbtGraph(
project=ProjectConfig(dbt_project_path=DBT_DIR, manifest_path=DBT_DIR / "target" / "manifest.json"),
render_config=RenderConfig(load_method=LoadMode.DBT_MANIFEST, select=[selector]),
execution_config=ExecutionConfig(dbt_project_path=DBT_DIR),
)
g.load(method=LoadMode.DBT_MANIFEST)
return sum(1 for x in g.filtered_nodes.values() if x.resource_type.value == "model")
Results:
| kind |
selector |
models selected |
status |
| node name |
metric_definition |
1 |
PASS |
| node + descendants |
metric_definition+ |
7 |
PASS |
| folder name |
post_processing |
6 |
PASS |
| folder + descendants |
post_processing+ |
0 |
FAIL |
| path |
path:models/post_processing |
6 |
PASS |
| path + descendants |
path:models/post_processing+ |
11 |
PASS |
| bare package |
fhir_dbt_utils |
25 |
PASS |
| bare package + descendants |
fhir_dbt_utils+ |
0 |
FAIL |
| package selector |
package:fhir_dbt_utils |
25 |
PASS |
| package selector + descendants |
package:fhir_dbt_utils+ |
182 |
PASS |
So <node>+, path:...+, and package:...+ all work. Only <folder>+ and <package>+ (a bare non-node token plus the operator) return nothing.
Root cause
In cosmos/dbt/selector.py, GraphSelector resolves its root through a node_by_name[node.name] lookup (exact node name). If the token is not a node name it logs Selector <x> not found and returns without selecting anything. Explicit method prefixes inside a graph selector (path:, package:) are handled, which is why path:...+ and package:...+ work, but bare folder, package, and fqn tokens are not. The non-graph bare-identifier path already resolves folder and package names, so the two paths are inconsistent.
Impact
Any <folder>+ or <package>+ selector in manifest mode silently matches nothing, which diverges from dbt ls. On exclude the targeted nodes are not excluded.
Workaround
Add an explicit method prefix so the graph selector can resolve the root. Use package:<name>+ or path:<folder>+ instead of the bare <name>+.
Suggested fix
When the GraphSelector root token is not an exact node name, fall back to the same resolution the non-graph bare-identifier path already uses (folder name, package name, fqn), so <folder>+ and <package>+ behave as they do under dbt. Adding manifest-mode selector tests for these two shapes would lock in the parity.
Are you willing to submit a PR?
Yes, I am willing to submit a PR.
Environment
manifest.json, no adapter invoked)DBT_MANIFESTLOCAL(rendering only, not relevant to the bug)Summary
In
LoadMode.DBT_MANIFEST, a selector that combines the graph operator (+) with a bare folder name or package name (for examplemy_folder+ormy_package+) resolves to an empty set. Cosmos resolves the graph-operator root by exact node name only, so folder and package tokens match nothing and the select or exclude silently has no effect. The same selectors work underdbt ls, which delegates resolution to dbt.For
excludethis is a correctness issue. The intended nodes are not removed and remain in the rendered DAG.Background: what #2357 addressed
PR #2357 ("Fix inclusion of package models, and selecting/excluding") added manifest-mode support for:
package:<name>select and exclude.All of the above work. The gap below is the graph-operator (
+) case layered on top of the folder and package tokens.What still needs addressing
Graph-operator root resolution should match dbt. The token attached to
+should resolve through the same methods as a bare identifier (node name, folder name, package name, fqn, path), not by exact node name alone.How to reproduce
Project:
fhir-dbt-analytics(public), loaded from itstarget/manifest.jsonwithLoadMode.DBT_MANIFEST.Results:
metric_definitionmetric_definition+post_processingpost_processing+path:models/post_processingpath:models/post_processing+fhir_dbt_utilsfhir_dbt_utils+package:fhir_dbt_utilspackage:fhir_dbt_utils+So
<node>+,path:...+, andpackage:...+all work. Only<folder>+and<package>+(a bare non-node token plus the operator) return nothing.Root cause
In
cosmos/dbt/selector.py,GraphSelectorresolves its root through anode_by_name[node.name]lookup (exact node name). If the token is not a node name it logsSelector <x> not foundand returns without selecting anything. Explicit method prefixes inside a graph selector (path:,package:) are handled, which is whypath:...+andpackage:...+work, but bare folder, package, and fqn tokens are not. The non-graph bare-identifier path already resolves folder and package names, so the two paths are inconsistent.Impact
Any
<folder>+or<package>+selector in manifest mode silently matches nothing, which diverges fromdbt ls. Onexcludethe targeted nodes are not excluded.Workaround
Add an explicit method prefix so the graph selector can resolve the root. Use
package:<name>+orpath:<folder>+instead of the bare<name>+.Suggested fix
When the
GraphSelectorroot token is not an exact node name, fall back to the same resolution the non-graph bare-identifier path already uses (folder name, package name, fqn), so<folder>+and<package>+behave as they do under dbt. Adding manifest-mode selector tests for these two shapes would lock in the parity.Are you willing to submit a PR?
Yes, I am willing to submit a PR.