Skip to content

Commit 48ace56

Browse files
feat(eks-v2-alpha): add support for EKS hybrid nodes (#36749)
### Issue # (if applicable) ### Reason for this change This change allows users to specify the networking primitives for an EKS Hybrid nodes cluster through L2 constructs. ### Description of changes This PR introduces two new top-level fields called remoteNodeNetworks and remotePodNetworks in the Cluster construct. Together, these allow users to specify the exact CIDRs ranges they want to use for their on-premises nodes and (optionally) pods. The Hybrid nodes feature requires that none of the node and pod CIDRs overlap with each other so I have also added validations for that. Network utils from EC2 had to be exported so they can be used in eks-v2-alpha, which is in a different package (@aws-cdk/aws-eks-v2-alpha). It cannot be used directly without an export like in eks v1 which is in aws-cdk-lib. Because of that, they are marked as internal. Similar to #32389 This aligns EKS v2 with v1. I also confirmed it works with EKS auto mode. ### Describe any new or updated permissions being added None ### Description of how you validated changes Integ test deployed ### Checklist - [X] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1 parent cb1658f commit 48ace56

17 files changed

+2542
-93
lines changed

packages/@aws-cdk/aws-eks-v2-alpha/README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -711,6 +711,31 @@ declare const cluster: eks.Cluster;
711711
const clusterEncryptionConfigKeyArn = cluster.clusterEncryptionConfigKeyArn;
712712
```
713713

714+
### Hybrid Nodes
715+
716+
When you create an Amazon EKS cluster, you can configure it to leverage the [EKS Hybrid Nodes](https://aws.amazon.com/eks/hybrid-nodes/) feature, allowing you to use your on-premises and edge infrastructure as nodes in your EKS cluster. Refer to the Hyrid Nodes [networking documentation](https://docs.aws.amazon.com/eks/latest/userguide/hybrid-nodes-networking.html) to configure your on-premises network, node and pod CIDRs, access control, etc before creating your EKS Cluster.
717+
718+
Once you have identified the on-premises node and pod (optional) CIDRs you will use for your hybrid nodes and the workloads running on them, you can specify them during cluster creation using the `remoteNodeNetworks` and `remotePodNetworks` (optional) properties:
719+
720+
```ts
721+
import { KubectlV34Layer } from '@aws-cdk/lambda-layer-kubectl-v34';
722+
723+
new eks.Cluster(this, 'Cluster', {
724+
version: eks.KubernetesVersion.V1_34,
725+
remoteNodeNetworks: [
726+
{
727+
cidrs: ['10.0.0.0/16'],
728+
},
729+
],
730+
remotePodNetworks: [
731+
{
732+
cidrs: ['192.168.0.0/16'],
733+
},
734+
],
735+
});
736+
```
737+
738+
714739
## Permissions and Security
715740

716741
In the new EKS module, `ConfigMap` is deprecated. Clusters created by the new module will use `API` as authentication mode. Access Entry will be the only way for granting permissions to specific IAM users and roles.

packages/@aws-cdk/aws-eks-v2-alpha/lib/cluster.ts

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import type * as kms from 'aws-cdk-lib/aws-kms';
88
import * as ssm from 'aws-cdk-lib/aws-ssm';
99
import type { IResource, Duration, ArnComponents } from 'aws-cdk-lib/core';
1010
import { Annotations, CfnOutput, CfnResource, Resource, Tags, Token, Stack, UnscopedValidationError, FeatureFlags } from 'aws-cdk-lib/core';
11+
import { ValidationError } from 'aws-cdk-lib/core/lib/errors';
1112
import { memoizedGetter } from 'aws-cdk-lib/core/lib/helpers-internal';
1213
import { MethodMetadata, addConstructMetadata } from 'aws-cdk-lib/core/lib/metadata-resource';
1314
import { propertyInjectable } from 'aws-cdk-lib/core/lib/prop-injectable';
@@ -433,6 +434,19 @@ export interface ClusterCommonOptions {
433434
* If not defined, kubectl provider will not be created by default.
434435
*/
435436
readonly kubectlProviderOptions?: KubectlProviderOptions;
437+
438+
/**
439+
* IPv4 CIDR blocks defining the expected address range of hybrid nodes
440+
* that will join the cluster.
441+
* @default - none
442+
*/
443+
readonly remoteNodeNetworks?: RemoteNodeNetwork[];
444+
445+
/**
446+
* IPv4 CIDR blocks for Pods running Kubernetes webhooks on hybrid nodes.
447+
* @default - none
448+
*/
449+
readonly remotePodNetworks?: RemotePodNetwork[];
436450
}
437451

438452
/**
@@ -527,6 +541,26 @@ export class EndpointAccess {
527541
}
528542
}
529543

544+
/**
545+
* Remote network configuration for hybrid nodes
546+
*/
547+
export interface RemoteNodeNetwork {
548+
/**
549+
* IPv4 CIDR blocks for the remote node network
550+
*/
551+
readonly cidrs: string[];
552+
}
553+
554+
/**
555+
* Remote network configuration for pods on hybrid nodes
556+
*/
557+
export interface RemotePodNetwork {
558+
/**
559+
* IPv4 CIDR blocks for the remote pod network
560+
*/
561+
readonly cidrs: string[];
562+
}
563+
530564
/**
531565
* Options for configuring EKS Auto Mode compute settings.
532566
* When enabled, EKS will automatically manage compute resources like node groups and Fargate profiles.
@@ -1150,6 +1184,8 @@ export class Cluster extends ClusterBase {
11501184

11511185
this.tagSubnets();
11521186

1187+
this.validateRemoteNetworkConfig(props);
1188+
11531189
// this is the role used by EKS when interacting with AWS resources
11541190
this.role = props.role || new iam.Role(this, 'Role', {
11551191
assumedBy: new iam.ServicePrincipal('eks.amazonaws.com'),
@@ -1261,6 +1297,14 @@ export class Cluster extends ClusterBase {
12611297
enabled: autoModeEnabled,
12621298
},
12631299
},
1300+
...(props.remoteNodeNetworks ? {
1301+
remoteNetworkConfig: {
1302+
remoteNodeNetworks: props.remoteNodeNetworks,
1303+
...(props.remotePodNetworks ? {
1304+
remotePodNetworks: props.remotePodNetworks,
1305+
}: {}),
1306+
},
1307+
} : {}),
12641308
resourcesVpcConfig: {
12651309
securityGroupIds: [securityGroup.securityGroupId],
12661310
subnetIds,
@@ -1679,6 +1723,81 @@ export class Cluster extends ClusterBase {
16791723
return autoModeEnabled;
16801724
}
16811725

1726+
private validateRemoteNetworkConfig(props: ClusterProps) {
1727+
if (!props.remoteNodeNetworks) {
1728+
if (props.remotePodNetworks) {
1729+
throw new ValidationError('remotePodNetworks cannot be specified without remoteNodeNetworks also being specified', this);
1730+
}
1731+
return;
1732+
}
1733+
1734+
this.validateNetworkCidrs(props.remoteNodeNetworks, 'node');
1735+
1736+
if (!props.remotePodNetworks) return;
1737+
1738+
this.validateNetworkCidrs(props.remotePodNetworks, 'pod');
1739+
this.validateCrossNetworkOverlap(props.remoteNodeNetworks, props.remotePodNetworks);
1740+
}
1741+
1742+
/**
1743+
* Validates all CIDR rules for a single network type (within same network + across networks).
1744+
*/
1745+
private validateNetworkCidrs(networks: RemoteNodeNetwork[] | RemotePodNetwork[], networkType: 'node' | 'pod') {
1746+
const resolvedCidrs = networks.map(n => n.cidrs.filter(c => !Token.isUnresolved(c)));
1747+
1748+
// Within same network
1749+
resolvedCidrs.forEach((cidrs, index) => {
1750+
for (let i = 0; i < cidrs.length; i++) {
1751+
for (let j = i + 1; j < cidrs.length; j++) {
1752+
if (ec2.NetworkUtils.validateCidrPairOverlap(cidrs[i], cidrs[j])) {
1753+
throw new ValidationError(
1754+
`CIDR ${cidrs[i]} should not overlap with another CIDR in remote ${networkType} network #${index + 1}`,
1755+
this,
1756+
);
1757+
}
1758+
}
1759+
}
1760+
});
1761+
1762+
// Across different networks
1763+
for (let i = 0; i < resolvedCidrs.length; i++) {
1764+
if (resolvedCidrs[i].length === 0) continue;
1765+
for (let j = i + 1; j < resolvedCidrs.length; j++) {
1766+
if (resolvedCidrs[j].length === 0) continue;
1767+
const [overlap, cidr1, cidr2] = ec2.NetworkUtils.validateCidrBlocksOverlap(resolvedCidrs[i], resolvedCidrs[j]);
1768+
if (overlap) {
1769+
throw new ValidationError(
1770+
`CIDR block ${cidr1} in remote ${networkType} network #${i + 1} should not overlap with CIDR block ${cidr2} in remote ${networkType} network #${j + 1}`,
1771+
this,
1772+
);
1773+
}
1774+
}
1775+
}
1776+
}
1777+
1778+
/**
1779+
* Validates that node network CIDRs do not overlap with pod network CIDRs.
1780+
*/
1781+
private validateCrossNetworkOverlap(nodeNetworks: RemoteNodeNetwork[], podNetworks: RemotePodNetwork[]) {
1782+
for (const nodeNetwork of nodeNetworks) {
1783+
const nodeCidrs = nodeNetwork.cidrs.filter(c => !Token.isUnresolved(c));
1784+
if (nodeCidrs.length === 0) continue;
1785+
1786+
for (const podNetwork of podNetworks) {
1787+
const podCidrs = podNetwork.cidrs.filter(c => !Token.isUnresolved(c));
1788+
if (podCidrs.length === 0) continue;
1789+
1790+
const [overlap, nodeCidr, podCidr] = ec2.NetworkUtils.validateCidrBlocksOverlap(nodeCidrs, podCidrs);
1791+
if (overlap) {
1792+
throw new ValidationError(
1793+
`Remote node network CIDR block ${nodeCidr} should not overlap with remote pod network CIDR block ${podCidr}`,
1794+
this,
1795+
);
1796+
}
1797+
}
1798+
}
1799+
}
1800+
16821801
private addNodePoolRole(id: string): iam.Role {
16831802
const role = new iam.Role(this, id, {
16841803
assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),

0 commit comments

Comments
 (0)