Skip to content

[Feature] Support KubeletExtraConfig for Windows nodegroups #5649

Open
@cthrasher

Description

@cthrasher

When trying to create a Windows nodegroup, I got the error:

Error: couldn't create cluster provider from options: kubeletExtraConfig is not supported for WindowsServer2019FullContainer nodegroups (path=nodeGroups[0].kubeletExtraConfig)

However, the aws-eks-best-practices documentation specifically calls out the need to do this to get around the lack of an out-of-memory process killer in Windows itself:

If eksctl is the deployment tool, check the following documentation to customize the kubelet configuration https://eksctl.io/usage/customizing-the-kubelet/

The CloudFormation template referenced in that documentation takes the kubelet arguments as a parameter named BootstrapArguments and injects them into a custom bootstrap script written in Powershell:

<powershell>
[string]$EKSBinDir = "$env:ProgramFiles\Amazon\EKS"
[string]$EKSBootstrapScriptName = 'Start-EKSBootstrap.ps1'
[string]$EKSBootstrapScriptFile = "$EKSBinDir\$EKSBootstrapScriptName"
[string]$cfn_signal = "$env:ProgramFiles\Amazon\cfn-bootstrap\cfn-signal.exe"
& $EKSBootstrapScriptFile -EKSClusterName ${ClusterName} ${BootstrapArguments} 3>&1 4>&1 5>&1 6>&1
$LastError = if ($?) { 0 } else { $Error[0].Exception.HResult }
& $cfn_signal --exit-code=$LastError --stack="${AWS::StackName}" --resource="NodeGroup" --region=${AWS::Region}
</powershell>

In order to set up the nodegroup with these memory settings, we have to be able to specify kubeletExtraConfig with Windows nodes. The specific arguments they call out would look like this:

    kubeletExtraConfig:
      kubeReserved:
        memory: "0.5Gi"
        ephemeral-storage: "1Gi"
      systemReserved:
        memory: "1.5Gi"
        ephemeral-storage: "1Gi"
      evictionHard:
        memory.available: "200Mi"
        nodefs.available: "10%"

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions