Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
552f798
xpmt: AKS machine API integration
comtalyst Aug 13, 2025
384fcbc
leftover merge conflict
comtalyst Aug 13, 2025
9c863b9
remove AKS machines pool management
comtalyst Aug 14, 2025
eadbfbe
correct image id header
comtalyst Aug 14, 2025
1abf4ba
fix: vm name format
comtalyst Aug 30, 2025
9e3939f
fix: drop assumption on using vm name for GET AKS machine
comtalyst Aug 30, 2025
92676e7
test: fix one small unit test setup
comtalyst Aug 30, 2025
32bc93a
chore: resolve more merge conflicts
comtalyst Sep 8, 2025
a213bcf
feat: (from rebase) support FIPs + generic Ubuntu image family
comtalyst Sep 8, 2025
e125779
chore: fix json casing
comtalyst Sep 8, 2025
22a84da
feat: ETag support on AKS machine + in-place update
comtalyst Sep 8, 2025
706d0e9
chore: improve a comment
comtalyst Sep 8, 2025
1e888b6
refactor: share code of setAdditionalAnnotationsForNewNodeClaim acros…
comtalyst Sep 9, 2025
939f48b
chore: split InstancePromise definition to a separate file
comtalyst Sep 9, 2025
f2c8b87
refactor: common basic validation for retrieved AKS machine
comtalyst Sep 9, 2025
d0bddc3
chore: naming improvements
comtalyst Sep 9, 2025
e20d790
feat: handle cases where zone is not specified
comtalyst Sep 9, 2025
433b5d2
refactor: some naming standardization
comtalyst Sep 9, 2025
bfa171c
chore: add a note on GET after initial PUT
comtalyst Sep 9, 2025
2bf1283
chore: standardized AKS machine instance provider name
comtalyst Sep 9, 2025
f2ff173
refactor: rework artifact streaming enablement determination logic(?)
comtalyst Sep 10, 2025
18ae208
test: fix custom VNet test per new VNet validation
comtalyst Sep 10, 2025
a0a16f8
test: fix managed tags to the correct karpenter.azure.com_cluster
comtalyst Sep 11, 2025
da64b28
test: add more unit tests for NIC garbage collection
comtalyst Sep 11, 2025
8140155
chore: minor unit tests reorganization
comtalyst Sep 11, 2025
5fa7468
test: fix small unit test
comtalyst Sep 11, 2025
aa18897
refactor: split test into multiple files
comtalyst Sep 11, 2025
74f416e
refactor: categorize older tests
comtalyst Sep 11, 2025
7dd7d09
chore: some linter fixes
comtalyst Sep 11, 2025
3bec1d1
chore: more linter fixes
comtalyst Sep 12, 2025
e6afdb0
refactor: creation timestamp, SIG option validation, labels/naming TO…
comtalyst Sep 12, 2025
041d304
test: update unit tests
comtalyst Sep 13, 2025
56a2971
fix: formalize agentpool not found logic on list
comtalyst Sep 13, 2025
6d09d14
fix: rework tags for creationtimestamp/name
comtalyst Sep 15, 2025
e88e1aa
test: fix one forgotten unit test break
comtalyst Sep 15, 2025
08a5a97
feat: finalize NotFound API behaviors
comtalyst Sep 15, 2025
2a29a76
fix: timestamp formats + misc + unit tests
comtalyst Sep 15, 2025
c21cd5b
chore: fix linters + some unit tests
comtalyst Sep 15, 2025
bb6d5aa
chore: update dev tools to support aksmachineapi
comtalyst Sep 16, 2025
9cfc49c
fix: small fix for dev tools
comtalyst Sep 16, 2025
5cc840c
temp: configure E2E actions for aks machine api
comtalyst Sep 16, 2025
9c74045
chore: resolved a finished TODO
comtalyst Sep 16, 2025
433e0ea
temp: use default VNET subnet ID
comtalyst Sep 16, 2025
6b80b4b
fix: do not use auxiliary token client on ProvisionModeAKSMachineAPI
comtalyst Sep 16, 2025
bac0c11
chore: fix role assignment for aksmachine
comtalyst Sep 16, 2025
95b320a
fix: also rehydrate machines in list
comtalyst Sep 16, 2025
91cf3fc
fix: give default control of VnetSubnetID to server-side
comtalyst Sep 19, 2025
fe18503
refactor: rework method layering + error messages
comtalyst Sep 22, 2025
1dc3191
refactor: misc
comtalyst Sep 22, 2025
624f86d
feat: reachability toggle with no-client
comtalyst Sep 22, 2025
0545a79
refactor: small linter change
comtalyst Sep 22, 2025
e32b069
refactor: rename AKS_MACHINES_REACHABLE to MANAGE_EXISTING_AKS_MACHINES
comtalyst Sep 23, 2025
2172594
refactor: use custom azure-sdk-for-go-extensions
comtalyst Sep 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/actions/e2e/create-cluster/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ inputs:
required: false
location:
description: "The azure location to run the e2e test in"
default: "eastus"
default: "westus2"
runs:
using: "composite"
steps:
Expand Down Expand Up @@ -80,3 +80,13 @@ runs:
run: |
make az-perm
make az-perm-acr
make az-perm-aksmachine
- name: add aks machines pool
shell: bash
env:
AZURE_SUBSCRIPTION_ID: ${{ inputs.subscription-id }}
AZURE_CLUSTER_NAME: ${{ inputs.cluster_name }}
AZURE_RESOURCE_GROUP: ${{ inputs.resource_group }}
AKS_MACHINES_POOL_NAME: testmpool
run: |
make az-add-aksmachinespool
2 changes: 1 addition & 1 deletion .github/actions/e2e/install-karpenter/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ runs:
run: az account set --subscription ${{ inputs.subscription-id }}
- name: configure Helm chart values
shell: bash
run: AZURE_CLUSTER_NAME=${{ inputs.cluster_name }} AZURE_RESOURCE_GROUP=${{ inputs.resource_group }} AZURE_LOCATION=${{ inputs.location }} make az-configure-values
run: AZURE_CLUSTER_NAME=${{ inputs.cluster_name }} AZURE_RESOURCE_GROUP=${{ inputs.resource_group }} AZURE_LOCATION=${{ inputs.location }} AKS_MACHINES_POOL_NAME=testmpool make az-configure-values-aksmachine
- name: deploy karpenter to cluster
shell: bash
run: AZURE_ACR_NAME=${{ inputs.acr_name }} make az-run
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/e2e-matrix-trigger.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,4 @@ jobs:
secrets:
E2E_CLIENT_ID: ${{ secrets.E2E_CLIENT_ID }}
E2E_TENANT_ID: ${{ secrets.E2E_TENANT_ID }}
E2E_SUBSCRIPTION_ID: ${{ secrets.E2E_SUBSCRIPTION_ID }}
E2E_SUBSCRIPTION_ID: ${{ secrets.E2E_STAGING_SUBSCRIPTION_ID }}
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"go.testTimeout": "120s",
"go.testTimeout": "240s",
"gopls": {
"build.directoryFilters": [
"-hack"
Expand Down
21 changes: 20 additions & 1 deletion Makefile-az.mk
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ else
endif

AZURE_ACR_SUFFIX ?= azurecr.io
AZURE_SIG_SUBSCRIPTION_ID ?= $(AZURE_SUBSCRIPTION_ID)
AZURE_SIG_SUBSCRIPTION_ID ?= 10945678-1234-1234-1234-123456789012
AZURE_CLUSTER_NAME ?= $(COMMON_NAME)
AZURE_RESOURCE_GROUP_MC = MC_$(AZURE_RESOURCE_GROUP)_$(AZURE_CLUSTER_NAME)_$(AZURE_LOCATION)

Expand All @@ -22,6 +22,8 @@ KARPENTER_FEDERATED_IDENTITY_CREDENTIAL_NAME ?= KARPENTER_FID
CUSTOM_VNET_NAME ?= $(AZURE_CLUSTER_NAME)-vnet
CUSTOM_SUBNET_NAME ?= nodesubnet

AKS_MACHINES_POOL_NAME ?= testmpool

.DEFAULT_GOAL := help # make without arguments will show help

az-all: az-login az-create-workload-msi az-mkaks-cilium az-create-federated-cred az-perm az-perm-acr az-configure-values az-build az-run az-run-sample ## Provision the infra (ACR,AKS); build and deploy Karpenter; deploy sample Provisioner and workload
Expand All @@ -30,6 +32,8 @@ az-all-cniv1: az-login az-create-workload-msi az-mkaks-cniv1 az-cre

az-all-cni-overlay: az-login az-create-workload-msi az-mkaks-overlay az-create-federated-cred az-perm az-perm-acr az-configure-values az-build az-run az-run-sample ## Provision the infra (ACR,AKS); build and deploy Karpenter; deploy sample Provisioner and workload

az-all-aksmachine: az-login az-create-workload-msi az-mkaks-cilium az-create-federated-cred az-perm az-perm-acr az-perm-aksmachine az-add-aksmachinespool az-configure-values-aksmachine az-build az-run az-run-sample

az-all-perftest: az-login az-create-workload-msi az-mkaks-perftest az-create-federated-cred az-perm az-perm-acr az-configure-values
$(MAKE) az-mon-deploy
$(MAKE) az-pprof-enable
Expand Down Expand Up @@ -135,12 +139,18 @@ az-mkaks-savm: az-mkrg ## Create experimental cluster with standalone VMs (+ ACR
az aks get-credentials --resource-group $(AZURE_RESOURCE_GROUP) --name $(AZURE_CLUSTER_NAME) --overwrite-existing
skaffold config set default-repo $(AZURE_ACR_NAME).$(AZURE_ACR_SUFFIX)/karpenter

az-add-aksmachinespool:
hack/deploy/add-aks-machines-pool.sh $(AZURE_SUBSCRIPTION_ID) $(AZURE_RESOURCE_GROUP) $(AZURE_CLUSTER_NAME) $(AKS_MACHINES_POOL_NAME)

az-rmrg: ## Destroy test ACR and AKS cluster by deleting the resource group (use with care!)
az group delete --name $(AZURE_RESOURCE_GROUP)

az-configure-values: ## Generate cluster-related values for Karpenter Helm chart
hack/deploy/configure-values.sh $(AZURE_CLUSTER_NAME) $(AZURE_RESOURCE_GROUP) $(KARPENTER_SERVICE_ACCOUNT_NAME) $(AZURE_KARPENTER_USER_ASSIGNED_IDENTITY_NAME)

az-configure-values-aksmachine: ## Generate cluster-related values for Karpenter Helm chart
hack/deploy/configure-values.sh $(AZURE_CLUSTER_NAME) $(AZURE_RESOURCE_GROUP) $(KARPENTER_SERVICE_ACCOUNT_NAME) $(AZURE_KARPENTER_USER_ASSIGNED_IDENTITY_NAME) aksmachineapi $(AKS_MACHINES_POOL_NAME)

az-mkvmssflex: ## Create VMSS Flex (optional, only if creating VMs referencing this VMSS)
az vmss create --name $(AZURE_CLUSTER_NAME)-vmss --resource-group $(AZURE_RESOURCE_GROUP_MC) --location $(AZURE_LOCATION) \
--instance-count 0 --orchestration-mode Flexible --platform-fault-domain-count 1 --zones 1 2 3
Expand All @@ -156,6 +166,15 @@ az-perm: ## Create role assignments to let Karpenter manage VMs and Network
az role assignment create --assignee-object-id $(KARPENTER_USER_ASSIGNED_CLIENT_ID) --assignee-principal-type "ServicePrincipal" --scope /subscriptions/$(AZURE_SUBSCRIPTION_ID)/resourceGroups/$(AZURE_RESOURCE_GROUP_MC) --role "Managed Identity Operator"
@echo Consider "make az-configure-values"!

az-perm-aksmachine: ## Create role assignments for AKS machine API operations
$(eval KARPENTER_USER_ASSIGNED_CLIENT_ID=$(shell az identity show --resource-group "${AZURE_RESOURCE_GROUP}" --name "${AZURE_KARPENTER_USER_ASSIGNED_IDENTITY_NAME}" --query 'principalId' -otsv))
az role assignment create --assignee-object-id $(KARPENTER_USER_ASSIGNED_CLIENT_ID) --assignee-principal-type "ServicePrincipal" --scope /subscriptions/$(AZURE_SUBSCRIPTION_ID)/resourceGroups/$(AZURE_RESOURCE_GROUP) --role "Azure Kubernetes Service Contributor Role"
az role assignment create --assignee-object-id $(KARPENTER_USER_ASSIGNED_CLIENT_ID) --assignee-principal-type "ServicePrincipal" --scope /subscriptions/$(AZURE_SUBSCRIPTION_ID)/resourceGroups/$(AZURE_RESOURCE_GROUP_MC) --role "Network Contributor"
$(eval CLUSTER_IDENTITY=$(shell az aks show --resource-group "${AZURE_RESOURCE_GROUP}" --name "${AZURE_CLUSTER_NAME}" --query 'identity.principalId' -otsv))
az role assignment create --assignee-object-id $(CLUSTER_IDENTITY) --assignee-principal-type "ServicePrincipal" --scope /subscriptions/$(AZURE_SUBSCRIPTION_ID)/resourceGroups/$(AZURE_RESOURCE_GROUP_MC) --role "Virtual Machine Contributor"
az role assignment create --assignee-object-id $(CLUSTER_IDENTITY) --assignee-principal-type "ServicePrincipal" --scope /subscriptions/$(AZURE_SUBSCRIPTION_ID)/resourceGroups/$(AZURE_RESOURCE_GROUP_MC) --role "Network Contributor"
az role assignment create --assignee-object-id $(CLUSTER_IDENTITY) --assignee-principal-type "ServicePrincipal" --scope /subscriptions/$(AZURE_SUBSCRIPTION_ID)/resourceGroups/$(AZURE_RESOURCE_GROUP_MC) --role "Managed Identity Operator"

az-perm-sig: ## Create role assignments when testing with SIG images
$(eval KARPENTER_USER_ASSIGNED_CLIENT_ID=$(shell az identity show --resource-group "${AZURE_RESOURCE_GROUP}" --name "${AZURE_KARPENTER_USER_ASSIGNED_IDENTITY_NAME}" --query 'principalId' -otsv))
az role assignment create --assignee-object-id $(KARPENTER_USER_ASSIGNED_CLIENT_ID) --assignee-principal-type "ServicePrincipal" --role "Reader" --scope /subscriptions/$(AZURE_SIG_SUBSCRIPTION_ID)/resourceGroups/AKS-Ubuntu/providers/Microsoft.Compute/galleries/AKSUbuntu
Expand Down
6 changes: 4 additions & 2 deletions cmd/controller/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@ func main() {

aksCloudProvider := cloudprovider.New(
op.InstanceTypesProvider,
op.InstanceProvider,
op.VMInstanceProvider,
op.AKSMachineProvider,
op.EventRecorder,
op.GetClient(),
op.ImageProvider,
Expand All @@ -79,7 +80,8 @@ func main() {
op.GetClient(),
op.EventRecorder,
aksCloudProvider,
op.InstanceProvider,
op.VMInstanceProvider,
op.AKSMachineProvider,
// TODO: still need to refactor ImageProvider side of things.
op.KubernetesVersionProvider,
op.ImageProvider,
Expand Down
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ require (
github.com/Azure/aks-middleware v0.0.41
github.com/Azure/azure-kusto-go v0.16.1
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible
github.com/Azure/azure-sdk-for-go-extensions v0.2.0
github.com/Azure/azure-sdk-for-go-extensions v0.2.1-0.20250924093303-adf3c22e7580
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.11.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute v1.0.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.7.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v4 v4.8.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v7 v7.3.0-beta.1
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be split out into a separate PR probably that just does this?

github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork v1.1.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resourcegraph/armresourcegraph v0.9.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armsubscriptions v1.3.0
Expand Down
6 changes: 4 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ github.com/Azure/azure-sdk-for-go v68.0.0+incompatible h1:fcYLmCpyNYRnvJbPerq7U0
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-sdk-for-go-extensions v0.2.0 h1:Wcgvuz5isRhcXWfciv66m4AES8QdomdcpTTOYBnXIc4=
github.com/Azure/azure-sdk-for-go-extensions v0.2.0/go.mod h1:ryW/ApW8CPlJeqhzt0JTEE8rGWgsvCmfQTeFYE/SHX8=
github.com/Azure/azure-sdk-for-go-extensions v0.2.1-0.20250924093303-adf3c22e7580 h1:GuhclDDvkIHVd6eDdqhmBgSQY11O06pCfZ+oxYJUDkk=
github.com/Azure/azure-sdk-for-go-extensions v0.2.1-0.20250924093303-adf3c22e7580/go.mod h1:so4bhVd3qbJ1bKqQj72P4JFpyRxe2USAJzdJZsgYG+0=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.0 h1:ci6Yd6nysBRLEodoziB6ah1+YOzZbZk+NYneoA6q+6E=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.0/go.mod h1:QyVsSSN64v5TGltphKLQ2sQxe4OBQg0J1eKRcVBnfgE=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.11.0 h1:MhRfI58HblXzCtWEZCO0feHs8LweePB3s90r7WaR1KU=
Expand All @@ -26,12 +28,12 @@ github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerregistry/armconta
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerregistry/armcontainerregistry v1.2.0/go.mod h1:E7ltexgRDmeJ0fJWv0D/HLwY2xbDdN+uv+X2uZtOx3w=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v2 v2.4.0 h1:1u/K2BFv0MwkG6he8RYuUcbbeK22rkoZbg4lKa/msZU=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v2 v2.4.0/go.mod h1:U5gpsREQZE6SLk1t/cFfc1eMhYAlYpEzvaYXuDfefy8=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v4 v4.8.0 h1:0nGmzwBv5ougvzfGPCO2ljFRHvun57KpNrVCMrlk0ns=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v4 v4.8.0/go.mod h1:gYq8wyDgv6JLhGbAU6gg8amCPgQWRE+aCvrV2gyzdfs=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v5 v5.0.0 h1:5n7dPVqsWfVKw+ZiEKSd3Kzu7gwBkbEBkeXb8rgaE9Q=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v5 v5.0.0/go.mod h1:HcZY0PHPo/7d75p99lB6lK0qYOP4vLRJUBpiehYXtLQ=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v6 v6.6.0 h1:xkWEcbsnJWid3rOf/S/LOHy1I55JA+4kw/f8Tnm+Onc=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v6 v6.6.0/go.mod h1:OWKfCmX4X3Vp2w7GSx1LZn8566tOHJBA6K0IAUVNYx0=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v7 v7.3.0-beta.1 h1:u8eQ8MNpywW+XHIPMh51gKW2ks4bJ8V15x/cR4hIXrw=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v7 v7.3.0-beta.1/go.mod h1:W47WhHDtkHadDTg9O+mOU2R0A0CrQKBsRk74gPDNs7c=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal v1.0.0 h1:lMW1lD/17LUA5z1XTURo7LcVG2ICBPlyMHjIUrcFZNQ=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal v1.0.0/go.mod h1:ceIuwmxDWptoW3eCqSXlnPsZFKh4X+R38dWPv7GS9Vs=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal/v2 v2.0.0 h1:PTFGRSlMKCQelWwxUyYVEUqseBJVemLyqWJjvMyt0do=
Expand Down
64 changes: 64 additions & 0 deletions hack/deploy/add-aks-machines-pool.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/usr/bin/env bash
set -euo pipefail

if [ "$#" -ne 4 ]; then
echo "Usage: $0 <subscription-id> <resource-group> <cluster-name> <nodepool-name>"
echo "This script adds a nodepool with 'machines' mode to an existing AKS cluster using Azure REST API."
echo "Example:"
echo " $0 00000000-0000-0000-0000-000000000000 robin-aks-xpmt karpenter-sh-08302025 testmpool"
exit 1
fi

AZURE_SUBSCRIPTION_ID=$1
RESOURCE_GROUP=$2
CLUSTER_NAME=$3
NODEPOOL_NAME=$4

echo "Adding nodepool '$NODEPOOL_NAME' with machines mode to cluster '$CLUSTER_NAME' in resource group '$RESOURCE_GROUP'..."

# Get access token
echo "Getting access token..."
ACCESS_TOKEN=$(az account get-access-token --query accessToken --output tsv)

# REST API endpoint
API_VERSION="2025-07-01"
URL="https://management.azure.com/subscriptions/${AZURE_SUBSCRIPTION_ID}/resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.ContainerService/managedClusters/${CLUSTER_NAME}/agentPools/${NODEPOOL_NAME}?api-version=${API_VERSION}"

# Request body with machines mode
REQUEST_BODY=$(cat <<EOF
{
"properties": {
"mode": "Machines",
}
}
EOF
)

echo "Making REST API call..."
echo "URL: $URL"
echo "Request Body:"
echo "$REQUEST_BODY"

# Make the REST API call
RESPONSE=$(curl -X PUT \
-H "Authorization: Bearer $ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-d "$REQUEST_BODY" \
"$URL" \
-w "\n%{http_code}" \
-s)

# Extract HTTP status code from response
HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
RESPONSE_BODY=$(echo "$RESPONSE" | head -n -1)

echo "HTTP Status Code: $HTTP_CODE"
echo "Response:"
echo "$RESPONSE_BODY" | jq '.' 2>/dev/null || echo "$RESPONSE_BODY"

if [[ "$HTTP_CODE" == "200" ]] || [[ "$HTTP_CODE" == "201" ]]; then
echo "✅ Successfully added nodepool '$NODEPOOL_NAME' with machines mode"
else
echo "❌ Failed to add nodepool. HTTP Status: $HTTP_CODE"
exit 1
fi
16 changes: 12 additions & 4 deletions hack/deploy/configure-values.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ set -euo pipefail
# This script interrogates the AKS cluster and Azure resources to generate
# the karpenter-values.yaml file using the karpenter-values-template.yaml file as a template.

if [ "$#" -ne 4 ]; then
echo "Usage: $0 <cluster-name> <resource-group> <karpenter-service-account-name> <karpenter-user-assigned-identity-name>"
if [ "$#" -lt 4 ] || [ "$#" -gt 6 ]; then
echo "Usage: $0 <cluster-name> <resource-group> <karpenter-service-account-name> <karpenter-user-assigned-identity-name> [provision-mode] [aks-machines-pool-name]"
exit 1
fi

Expand All @@ -14,6 +14,8 @@ CLUSTER_NAME=$1
AZURE_RESOURCE_GROUP=$2
KARPENTER_SERVICE_ACCOUNT_NAME=$3
AZURE_KARPENTER_USER_ASSIGNED_IDENTITY_NAME=$4
PROVISION_MODE=${5:-}
AKS_MACHINES_POOL_NAME=${6:-testmpool}

# Optional values through env vars:
LOG_LEVEL=${LOG_LEVEL:-"info"}
Expand Down Expand Up @@ -68,9 +70,15 @@ NODE_IDENTITIES=$(jq -r ".identityProfile.kubeletidentity.resourceId" <<< "$AKS_
KARPENTER_USER_ASSIGNED_CLIENT_ID=$(az identity show --resource-group "${AZURE_RESOURCE_GROUP}" --name "${AZURE_KARPENTER_USER_ASSIGNED_IDENTITY_NAME}" --query 'clientId' -otsv)
KUBELET_IDENTITY_CLIENT_ID=$(jq -r ".identityProfile.kubeletidentity.clientId // empty" <<< "$AKS_JSON")

export CLUSTER_NAME AZURE_LOCATION AZURE_RESOURCE_GROUP_MC KARPENTER_SERVICE_ACCOUNT_NAME \
# For Machine API mode
if [[ "${PROVISION_MODE:-}" == "aksmachineapi" ]]; then
USE_SIG="true"
AZURE_SIG_SUBSCRIPTION_ID=109a5e88-712a-48ae-9078-9ca8b3c81345
fi

export CLUSTER_NAME AZURE_LOCATION AZURE_RESOURCE_GROUP AZURE_RESOURCE_GROUP_MC KARPENTER_SERVICE_ACCOUNT_NAME \
CLUSTER_ENDPOINT BOOTSTRAP_TOKEN SSH_PUBLIC_KEY VNET_SUBNET_ID KARPENTER_USER_ASSIGNED_CLIENT_ID NODE_IDENTITIES AZURE_SUBSCRIPTION_ID NETWORK_PLUGIN NETWORK_PLUGIN_MODE NETWORK_POLICY \
LOG_LEVEL VNET_GUID KUBELET_IDENTITY_CLIENT_ID
LOG_LEVEL VNET_GUID KUBELET_IDENTITY_CLIENT_ID PROVISION_MODE USE_SIG AZURE_SIG_SUBSCRIPTION_ID AKS_MACHINES_POOL_NAME

# get karpenter-values-template.yaml, if not already present (e.g. outside of repo context)
if [ ! -f karpenter-values-template.yaml ]; then
Expand Down
10 changes: 8 additions & 2 deletions karpenter-values-template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,19 @@ controller:

# managed karpenter settings
- name: USE_SIG
value: "false"
value: "${USE_SIG}"
- name: SIG_ACCESS_TOKEN_SERVER_URL
value: ""
- name: SIG_ACCESS_TOKEN_SCOPE
value: ""
- name: SIG_SUBSCRIPTION_ID
value: ""
value: "${AZURE_SIG_SUBSCRIPTION_ID}"
- name: PROVISION_MODE
value: ${PROVISION_MODE}
- name: AKS_MACHINES_POOL_NAME
value: "${AKS_MACHINES_POOL_NAME}"
- name: ARM_RESOURCE_GROUP
value: ${AZURE_RESOURCE_GROUP}
serviceAccount:
name: ${KARPENTER_SERVICE_ACCOUNT_NAME}
annotations:
Expand Down
1 change: 1 addition & 0 deletions pkg/apis/v1beta1/labels.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ var (

AnnotationAKSNodeClassHash = apis.Group + "/aksnodeclass-hash"
AnnotationAKSNodeClassHashVersion = apis.Group + "/aksnodeclass-hash-version"
AnnotationAKSMachineResourceID = apis.Group + "/aks-machine-resource-id" // resource ID of the associated AKS machine
Comment thread
comtalyst marked this conversation as resolved.
)

const (
Expand Down
Loading