Skip to content

refactor: introduce golden file testing to simplify test expectation updates#6148

Open
rindrics wants to merge 2 commits into
kubernetes-sigs:masterfrom
rindrics:use-golden-test
Open

refactor: introduce golden file testing to simplify test expectation updates#6148
rindrics wants to merge 2 commits into
kubernetes-sigs:masterfrom
rindrics:use-golden-test

Conversation

@rindrics

Copy link
Copy Markdown

Summary

This PR introduces golden file testing across test packages (api/resource, api/resmap, and api/krusty) to simplify updating test expectations when YAML output order changes. This change prepares for PR #6046, which will modify Resource.AsYAML() behavior and is expected to change the output order of generated YAML resources.

Background

PR #6046 changes Resource.AsYAML() to use RNode.MustString() instead of sigs.k8s.io/yaml.JSONToYAML. This change will preserve the original YAML field order instead of sorting fields alphabetically, which will affect the output order of generated resources.

While both orderings are valid YAML, this change requires updating many test expectations to match the new output order. To make these updates easier and more maintainable, we are introducing golden file testing, which separates test expectations from test code and allows for easy updates using a simple flag.

Benefits

  1. Easier test expectation updates: When PR fix: preserve original YAML formatting in resource output #6046 changes YAML output order, test expectations can be updated by simply running tests with -update-golden flag instead of manually editing test code
  2. Better test maintainability: Golden files separate test expectations from test logic, making it easier to review changes and understand what the tests are verifying
  3. Improved readability: Test code becomes more concise without large inline YAML strings
  4. Future-proof: Golden files make it easier to handle future changes to output formatting or ordering

Testing

All tests pass with the new golden file infrastructure:

# Run all tests
cd api && go test -v -timeout 45m -cover ./... -ldflags "-X sigs.k8s.io/kustomize/api/provenance.buildDate=2023-01-31T23:38:41Z -X sigs.k8s.io/kustomize/api/provenance.version=(test)"

# Update golden files (when needed after PR #6046)
go test ./api/resource -update-golden -ldflags "..."
go test ./api/resmap -update-golden -ldflags "..."
go test ./api/krusty -update-golden -ldflags "..."

Related PRs and Issues

Migration Notes

This PR does not change any test behavior or expected outputs. It only changes how test expectations are stored and compared. After PR #6046 is merged, golden files can be updated using the -update-golden flag to reflect the new YAML output order.

…lity

- Use golden file comparisons instead of inline expected strings
- Use direct comparison for TestPluginEnvironment (not suited for golden files)
- Handle AsYAML() error in TestDropLocalNodes instead of discarding it
- Normalize spaces to underscores in golden file paths for subtest compatibility
- Remove lazy flag registration from isUpdateGolden (already handled in TestMain)
- Remove actual output from golden-file-missing errors to avoid leaking secrets in
CI logs
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

This PR has multiple commits, and the default merge method is: merge.
You can request commits to be squashed using the label: tide/merge-method-squash

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 13, 2026
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Rindrics
Once this PR has been reviewed and has the lgtm label, please assign koba1t for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
k8s-ci-robot requested review from koba1t and sarab97 May 13, 2026 05:34
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

Hi @rindrics. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 13, 2026
@rindrics
rindrics marked this pull request as ready for review May 13, 2026 05:44
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 13, 2026
@rindrics

Copy link
Copy Markdown
Author

@koba1t @sarab97 @varshaprasad96 would you accept this PR to run CI tests?

@koba1t

koba1t commented May 16, 2026

Copy link
Copy Markdown
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 16, 2026
@rindrics

rindrics commented Jul 3, 2026

Copy link
Copy Markdown
Author

@koba1t @sarab97 @varshaprasad96
Is there anything I can do to help complete the status check and move this pull request forward?
thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants