Introduce config opt-in NLB provisioning with Security Groups

[mtulio]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR introduces managed security group support for Service type-LoadBalancer NLB through a global cloud-config option. Key changes:

• Managed Security Groups: Introduces NLBSecurityGroupMode: Managed cloud-config to automatically provision Security Groups for new NLBs
• Automatic Lifecycle Management: Controller creates, manages, and deletes security groups with proper AWS tagging
• Backward Compatibility: Existing Service NLBs without security groups remain unchanged (no retrofitting)
• BYO Security Group Limitation: Explicitly blocks service.beta.kubernetes.io/aws-load-balancer-security-groups and service.beta.kubernetes.io/aws-load-balancer-extra-security-groups annotations for NLB

Rationale for BYO SG Limitation:

BYO Security Group support for NLB is intentionally blocked in this release due to:

1. AWS API complexity with NLB security group management mixing with the managed SG feature
2. Security group leak bug (see #1208 / #1209) logic will be reused in NLB BYO SG feature

Which issue(s) this PR fixes:
Refs #1151

Special notes for your reviewer:

Future Work:
Managed SG is important to empower users to bypass managed SG, and enhance security control boundaries with user-managed security groups on NLBs. BYO Security Group support for NLB will be considered in a future release after:

• The managed SG be merged with NLB SG management logic
• Resolution of security group leak bug (#1209)
• Comprehensive field testing of managed SG functionality
• Community feedback on managed SG approach

The changes introducing the global, opt-in, cloud-config for enabling Security Group (SG) on NLB creation (similar CLB), is for users/administrators who intentionally wants to enforce SG across all new services - following AWS recommendations, and ALBC defaults. This won't change the default CCM behavior if the configuration isn't added.

Done checklist:

• introduce cloud-config to signalize the controller to always provision Security Group on Service type-LoadBalancer NLB
• Create Security Group when NLB is created, and global config is added
• Delete Security Group when NLB is removed
• Update Service port and ensure SG rules are updated
• Review SG rules logic
• Elaborate an e2e scenario that allows to update the cloud-config to exercise NLB+SG feature
• Resolve pending question related to the approach
• Create User Documentation for the feature

Related changes isolated from this PR:

https://github.com/kubernetes/cloud-provider-aws/pull/1207
https://github.com/kubernetes/cloud-provider-aws/issues/1206
https://github.com/kubernetes/cloud-provider-aws/pull/1163
https://github.com/kubernetes/cloud-provider-aws/pull/1205
https://github.com/kubernetes/cloud-provider-aws/pull/1159
https://github.com/kubernetes/cloud-provider-aws/pull/1153
https://github.com/kubernetes/cloud-provider-aws/pull/1215

Does this PR introduce a user-facing change?:

Add opt-in NLB Security Group provisioning support. Introduces support for provisioning Security Groups to Service type-LoadBalancer NLB by default through global cloud-config (`NLBSecurityGroupMode: Managed`). The managed security group approach is opt-in and includes automatic life cycle management of Security Groups and rules.

[k8s-ci-robot]

This issue is currently awaiting triage.

If cloud-provider-aws contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

Hi @mtulio. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[mtulio]

Almost done. A few pending items in TODO, and raising questions in key points from a broad group.

[mtulio]

Hey @kmala, I still have some items to review in this PR, but tests are passing locally. Would you mind stamping ok-to-test to validate if there is no local addiction from my env? Thanks

[kmala]

/ok-to-test

[mtulio]

/test all

[elmiko]

i'm caught up in some bug work currently, but i will try to make time for a review this friday (29 Aug).

[mtulio]

Last changes:

• doc strings reviewed
• addressed the suggestion Introduce config opt-in NLB provisioning with Security Groups #1158 (comment) (increase error handling on IsNLBSecurityGroupModeManaged)
• reviewed of error msg descriptions, and related unit tests

[elmiko]

i think this is looking good, but i have not tried it in a live cluster. i would love to test drive it next week, but i can't guarantee that i will have the time.

[mtulio]

i think this is looking good, but i have not tried it in a live cluster. i would love to test drive it next week, but i can't guarantee that i will have the time.

Thanks, @elmiko . I just sent to your way the creds of a live v1.33 cluster with operands turned off, so you can save time setting up the environment :)

[mtulio]

Hello all, I just reviewed all comments to ensure everything is resolved. No pending items so far. Awaiting further review, if there are some. Thanks!

[elmiko]

@mtulio demonstrated this change for me today, in general things work as expected. we did notice some odd behavior when removing the annotation, a new lb was created, but we think the bug is not related to this code change.

/lgtm

[mtulio]

@mtulio demonstrated this change for me today, in general things work as expected. we did notice some odd behavior when removing the annotation, a new lb was created, but we think the bug is not related to this code change.

/lgtm

Thanks for peer review, Mike. I just recorded the steps in the issue (already existed in controller) #1254 . cc @kmala

[mtulio]

Hey @kmala , would you mind taking a look at this PR, please? Thanks!

[JoelSpeed]

Also gone through this today

/lgtm

Thanks @mtulio

[kmala]

/lgtm

[mtulio]

@mtulio: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cloud-provider-aws-check e55c57f link true /test pull-cloud-provider-aws-check
pull-cloud-provider-aws-test e55c57f link true /test pull-cloud-provider-aws-test
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

failures caused by missing unit test from recent changes which moved BYO SG annotations to dedicated validation functions, as well unit test failure caused by sdk v2 bump. Both are fixed in my last commit, additionally I moved BYO SG unit tests to the validation functions.

Awaiting new CI signals.

[mtulio]

CI is green! Hey @elmiko @kmala are you ok to go here?

[kmala]

/lgtm

[JoelSpeed]

/approve

Based on myself, @elmiko and @kmala having reviewed

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JoelSpeed

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [JoelSpeed]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mtulio]

Thanks @elmiko @kmala @JoelSpeed for your support on this PR.