Description
Cosmos supports configuring the dbt project root either through:
ProjectConfig.dbt_project_path; or
- separate
RenderConfig.dbt_project_path and ExecutionConfig.dbt_project_path values.
However, ProjectConfig only stores its configured model, seed, and snapshot relative paths when ProjectConfig.dbt_project_path is set.
When the project root comes from RenderConfig/ExecutionConfig, explicit relative paths are silently discarded. Under LoadMode.CUSTOM, LegacyDbtProject consequently falls back to the default models, seeds, and snapshots directories and may render an incomplete or empty DAG.
This is a pre-existing limitation of the singular models_relative_path/seeds_relative_path/snapshots_relative_path arguments, not a regression introduced by #2875. The new plural arguments added in #2875 inherit the same behaviour.
Reproduction
Given a project containing:
split_path_project/
└── custom/
└── models/
└── my_model.sql
the following configuration discards custom/models:
project_config = ProjectConfig(
manifest_path=manifest_path,
project_name="split_path_project",
models_relative_paths=["custom/models"],
)
render_config = RenderConfig(
load_method=LoadMode.CUSTOM,
dbt_project_path=project_dir,
)
execution_config = ExecutionConfig(
dbt_project_path=project_dir,
)
Observed:
assert project_config.models_paths == []
When the custom parser runs, it searches the default models/ directory instead of custom/models/, so my_model is not included in the rendered graph.
The deprecated singular argument has the same limitation:
ProjectConfig(
manifest_path=manifest_path,
project_name="split_path_project",
models_relative_path="custom/models",
)
Expected behaviour
Relative model, seed, and snapshot paths should be retained independently of where the absolute project root is configured.
For DAG rendering, Cosmos should resolve them against the effective rendering project path, including when that path comes from RenderConfig.dbt_project_path.
Possible approach
- Preserve the normalized relative paths even when
ProjectConfig.dbt_project_path is absent.
- Resolve them against
RenderConfig.project_path when loading through LoadMode.CUSTOM.
- Preserve the distinction between an omitted value and an explicitly empty list.
Suggested tests
- Custom parser with the project root supplied through
RenderConfig and ExecutionConfig.
- Nested relative paths such as
custom/models.
- Multiple model, seed, and snapshot directories.
- An explicitly empty relative-path list, ensuring it does not fall back to the default directory.
- Backward compatibility for the deprecated singular arguments.
Description
Cosmos supports configuring the dbt project root either through:
ProjectConfig.dbt_project_path; orRenderConfig.dbt_project_pathandExecutionConfig.dbt_project_pathvalues.However,
ProjectConfigonly stores its configured model, seed, and snapshot relative paths whenProjectConfig.dbt_project_pathis set.When the project root comes from
RenderConfig/ExecutionConfig, explicit relative paths are silently discarded. UnderLoadMode.CUSTOM,LegacyDbtProjectconsequently falls back to the defaultmodels,seeds, andsnapshotsdirectories and may render an incomplete or empty DAG.This is a pre-existing limitation of the singular
models_relative_path/seeds_relative_path/snapshots_relative_patharguments, not a regression introduced by #2875. The new plural arguments added in #2875 inherit the same behaviour.Reproduction
Given a project containing:
the following configuration discards
custom/models:Observed:
When the custom parser runs, it searches the default
models/directory instead ofcustom/models/, somy_modelis not included in the rendered graph.The deprecated singular argument has the same limitation:
Expected behaviour
Relative model, seed, and snapshot paths should be retained independently of where the absolute project root is configured.
For DAG rendering, Cosmos should resolve them against the effective rendering project path, including when that path comes from
RenderConfig.dbt_project_path.Possible approach
ProjectConfig.dbt_project_pathis absent.RenderConfig.project_pathwhen loading throughLoadMode.CUSTOM.Suggested tests
RenderConfigandExecutionConfig.custom/models.