-
Notifications
You must be signed in to change notification settings - Fork 710
Add new rule to validate pattern directory structure #4662
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
c324fb4
Validate pattern directory structure
shatakshiiii af9ed4c
Merge branch 'main' into validate_pattern_dir
shatakshiiii 71e2f99
Add creator as a dep
shatakshiiii 833a9ac
Updates for schema validation
shatakshiiii 6943f03
Revert "Updates for schema validation"
shatakshiiii 05a51cd
Merge branch 'main' into validate_pattern_dir
shatakshiiii 0cbe1fd
Add pattern rule
shatakshiiii 2dde0cd
pre-commit fixes
shatakshiiii 8800130
update rules number
shatakshiiii 1e84250
Updates for pattern rule
shatakshiiii 23a3792
Merge branch 'main' into validate_pattern_dir
shatakshiiii 66719e3
Remove duplicate pattern entry from config file
shatakshiiii 6fddb40
Changes for making pattern as file kind
shatakshiiii f720399
Fixes
ssbarnea d1979c6
Fixes in the pattern rule
shatakshiiii b1a73bc
Merge branch 'main' into validate_pattern_dir
shatakshiiii 231c523
Improve the get_playbook_file function
shatakshiiii 0941a8c
Validation for pattern name
shatakshiiii f86d646
clean up
shatakshiiii 5ef4ee5
Use metadata as tag
shatakshiiii 2f829ba
Update docs
shatakshiiii 4164a3d
Improve pattern rule descriptions for better clarity
shatakshiiii 0856d3d
Merge branch 'main' into validate_pattern_dir
shatakshiiii c6eb7ae
Add usage details in pattern.md
shatakshiiii File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
44 changes: 44 additions & 0 deletions
44
examples/collections/extensions/patterns/invalid_pattern/pattern.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| { | ||
| "schema_version": "1.0", | ||
| "name": "weather_forecasting", | ||
| "title": "Weather Forecasting", | ||
| "description": "This pattern is designed to help get the weather forecast for a given airport code. It creates a project, EE, and job templates in automation controller to get the weather forecast.", | ||
| "short_description": "This pattern is designed to help get the weather forecast for a given airport code.", | ||
| "tags": ["weather", "forecasting"], | ||
| "aap_resources": { | ||
| "controller_project": { | ||
| "name": "Weather Forecasting", | ||
| "description": "Project for the Weather Forecasting pattern" | ||
| }, | ||
| "controller_execution_environment": { | ||
| "name": "Weather Forecasting", | ||
| "description": "EE for the Weather Forecasting pattern", | ||
| "image_name": "weather-demo-ee", | ||
| "pull": "missing" | ||
| }, | ||
| "controller_labels": ["weather", "forecasting"], | ||
| "controller_job_templates": [ | ||
| { | ||
| "name": "Get Weather Forecast", | ||
| "description": "This job template gets the weather at the location of a provided airport code.", | ||
| "execution_environment": "Weather Forecasting", | ||
| "playbook": "site.yml", | ||
| "primary": true, | ||
| "labels": ["weather", "forecasting"], | ||
| "survey": { | ||
| "name": "Weather Forecasting", | ||
| "description": "Survey to configure the weather forecasting pattern", | ||
| "spec": [ | ||
| { | ||
| "type": "text", | ||
| "question_name": "Location", | ||
| "question_description": "Enter the airport code for which you want to get the weather forecast", | ||
| "variable": "location", | ||
| "required": true | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| } |
Empty file.
Empty file.
44 changes: 44 additions & 0 deletions
44
examples/collections/extensions/patterns/valid_pattern/meta/pattern.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| { | ||
| "schema_version": "1.0", | ||
| "name": "weather_forecasting", | ||
| "title": "Weather Forecasting", | ||
| "description": "This pattern is designed to help get the weather forecast for a given airport code. It creates a project, EE, and job templates in automation controller to get the weather forecast.", | ||
| "short_description": "This pattern is designed to help get the weather forecast for a given airport code.", | ||
| "tags": ["weather", "forecasting"], | ||
| "aap_resources": { | ||
| "controller_project": { | ||
| "name": "Weather Forecasting", | ||
| "description": "Project for the Weather Forecasting pattern" | ||
| }, | ||
| "controller_execution_environment": { | ||
| "name": "Weather Forecasting", | ||
| "description": "EE for the Weather Forecasting pattern", | ||
| "image_name": "weather-demo-ee", | ||
| "pull": "missing" | ||
| }, | ||
| "controller_labels": ["weather", "forecasting"], | ||
| "controller_job_templates": [ | ||
| { | ||
| "name": "Get Weather Forecast", | ||
| "description": "This job template gets the weather at the location of a provided airport code.", | ||
| "execution_environment": "Weather Forecasting", | ||
| "playbook": "site.yml", | ||
| "primary": true, | ||
| "labels": ["weather", "forecasting"], | ||
| "survey": { | ||
| "name": "Weather Forecasting", | ||
| "description": "Survey to configure the weather forecasting pattern", | ||
| "spec": [ | ||
| { | ||
| "type": "text", | ||
| "question_name": "Location", | ||
| "question_description": "Enter the airport code for which you want to get the weather forecast", | ||
| "variable": "location", | ||
| "required": true | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| } | ||
Empty file.
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # pattern | ||
|
|
||
| This rule aims to validate Ansible pattern directory structure. | ||
shatakshiiii marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| """Implementation of PatternRule.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import sys | ||
| from typing import TYPE_CHECKING, Any | ||
|
|
||
| from ansiblelint.rules import AnsibleLintRule | ||
|
|
||
| if TYPE_CHECKING: | ||
| from ansiblelint.errors import MatchError | ||
| from ansiblelint.file_utils import Lintable | ||
|
|
||
|
|
||
| class PatternRule(AnsibleLintRule): | ||
| """Rule for checking pattern directory.""" | ||
|
|
||
| id = "pattern" | ||
| description = "Confirm that pattern has valid directory structure." | ||
| severity = "MEDIUM" | ||
| tags = ["metadata"] | ||
| version_changed = "25.7.0" | ||
|
|
||
| def matchplay(self, file: Lintable, data: dict[str, Any]) -> list[MatchError]: | ||
| """Return matches found for a specific play (entry in playbook).""" | ||
| if file.kind != "pattern": | ||
| return [] | ||
|
|
||
| results = [] | ||
|
|
||
| pattern_dir = file.path.parent.parent.resolve() | ||
| meta_dir = pattern_dir / "meta" | ||
|
|
||
| # Check if meta directory exists | ||
| if not meta_dir.is_dir(): | ||
| results.append( | ||
| self.create_matcherror( | ||
| message=( | ||
| f"Pattern directory '{pattern_dir}' contains pattern.json but is missing the required 'meta' directory." | ||
| ), | ||
| tag=self.id, | ||
shatakshiiii marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| filename=file, | ||
| ), | ||
| ) | ||
| return results | ||
|
|
||
| # Define required files relative to the pattern dir | ||
| required_paths = [ | ||
| pattern_dir / "README.md", | ||
| pattern_dir / "playbooks" / "site.yml", | ||
| ] | ||
| missing = [ | ||
| str(p.relative_to(pattern_dir)) for p in required_paths if not p.exists() | ||
| ] | ||
|
|
||
| # Check execution_environments directory if it exists | ||
| ee_dir = pattern_dir / "execution_environments" | ||
| if ee_dir.exists(): | ||
| expected_file = ee_dir / "execution_environment.yml" | ||
| # Must contain only execution_environment.yml | ||
| files = list(ee_dir.iterdir()) | ||
| if not expected_file.exists(): | ||
| missing.append("execution_environments/execution_environment.yml") | ||
| if len(files) != 1 or files[0].name != "execution_environment.yml": | ||
| results.append( | ||
| self.create_matcherror( | ||
| message=( | ||
| f"'execution_environments' directory in '{pattern_dir}' must contain only 'execution_environment.yml' file." | ||
| ), | ||
| tag=self.id, | ||
shatakshiiii marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| filename=file, | ||
| ), | ||
| ) | ||
|
|
||
| if missing: | ||
| results.append( | ||
| self.create_matcherror( | ||
| message=( | ||
| f"Pattern directory '{pattern_dir}' is missing required files: {', '.join(missing)}" | ||
| ), | ||
| tag=self.id, | ||
shatakshiiii marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| filename=file, | ||
| ), | ||
| ) | ||
|
|
||
| return results | ||
|
|
||
|
|
||
| if "pytest" in sys.modules: | ||
| import pytest | ||
|
|
||
| from ansiblelint.rules import RulesCollection # pylint: disable=ungrouped-imports | ||
| from ansiblelint.runner import Runner | ||
|
|
||
| @pytest.mark.parametrize( | ||
| ("file", "expected"), | ||
| ( | ||
| pytest.param( | ||
| "examples/collections/extensions/patterns/valid_pattern/meta/pattern.json", | ||
| ["pattern"], | ||
| id="valid-pattern", | ||
| ), | ||
| pytest.param( | ||
| "examples/collections/extensions/patterns/invalid_pattern/pattern.json", | ||
| ["pattern"], | ||
| id="invalid-pattern", | ||
| ), | ||
| ), | ||
| ) | ||
| def test_pattern( | ||
| default_rules_collection: RulesCollection, | ||
| file: str, | ||
| expected: list[str], | ||
| ) -> None: | ||
| """Validate that rule works as intended.""" | ||
| results = Runner(file, rules=default_rules_collection).run() | ||
|
|
||
| assert len(results) == len(expected) | ||
| for index, result in enumerate(results): | ||
| assert result.rule.id == PatternRule.id, result | ||
| assert result.tag == expected[index] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| """Tests for the pattern feature.""" | ||
|
|
||
| import shutil | ||
| import subprocess | ||
| import sys | ||
| import tempfile | ||
| from pathlib import Path | ||
|
|
||
| LINT_BIN = Path(sys.executable).parent / "ansible-lint" | ||
|
|
||
|
|
||
| def test_creator_scaffolded_pattern() -> None: | ||
| """Validate creator scaffolded pattern. | ||
|
|
||
| Scaffold a pattern using ansible-creator. Run ansible-lint | ||
| on the scaffolded pattern and validate lint results. | ||
|
|
||
| Args: | ||
| monkeypatch: Monkeypatch fixture. | ||
| """ | ||
| # Create a tmp dir and copy an existing collection into it | ||
| collection_src = Path("examples/collections/broken_no_runtime") | ||
| with tempfile.TemporaryDirectory() as tmpdir: | ||
| collection_dest = Path(tmpdir) / collection_src.name | ||
| shutil.copytree(collection_src, collection_dest) | ||
|
|
||
| # Scaffold a pattern using ansible-creator | ||
| result = subprocess.run( | ||
| [ | ||
| "ansible-creator", | ||
| "add", | ||
| "resource", | ||
| "pattern", | ||
| "sample_pattern", | ||
| collection_dest, | ||
| ], | ||
| capture_output=True, | ||
| text=True, | ||
| check=False, | ||
| ) | ||
| assert result.returncode == 0, f"Pattern scaffolding failed: {result.stderr}" | ||
|
|
||
| # Run ansible-lint on the scaffolded pattern | ||
| pattern_path = collection_dest | ||
|
|
||
| lint_result = subprocess.run( | ||
| [str(LINT_BIN), pattern_path], | ||
| capture_output=True, | ||
| text=True, | ||
| env={"NO_COLOR": "1"}, | ||
| check=False, | ||
| ) | ||
| assert lint_result.returncode == 0, f"ansible-lint failed: {lint_result.stderr}" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.