Add EKS AutoMode support

[richardcase]

/kind feature
/priority important-soon
/triage accepted
/help

Describe the solution you'd like

Add support for EKS Auto mode to CAPA.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[k8s-ci-robot]

@richardcase:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/kind feature
/priority important-soon
/triage accepted
/help

Describe the solution you'd like

Add support for EKS Auto mode to CAPA.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[richardcase]

Whoever picks this up we should think about the API changes upfront. Does it fit into AWSManagedControlPlane or do we need a new resource kind?

[seanschneeweiss]

This will require the upgrade to AWS SDK v2 in #2225. We created a fork to include EKS Auto Mode support in the v1 SDK: c445/aws-sdk-go@309a05d
The fork of SDK v1 was just a quick starter for us. I guess for CAPA upstream the SDK v2 and the CRD change is the way forward.

IMHO it fits into AWSManagedControlPlane. On the long run, a mixed mode of EKS Auto Mode and non EKS Auto Mode might be supported by EKS.

Node Pools in EKS Auto Mode differ from Amazon EKS Managed Node Groups but can coexist in the same cluster. src

I wished this could be aligned with ACK controller for EKS, but there is no progress yet: aws-controllers-k8s/community#2222

The AWS SDK exposes Auto Mode settings as follows:

ComputeConfig *ComputeConfigRequest

type ComputeConfigRequest struct {
	// Request to enable or disable the compute capability on your EKS Auto Mode
	// cluster. If the compute capability is enabled, EKS Auto Mode will create
	// and delete EC2 Managed Instances in your Amazon Web Services account.
	Enabled *bool `locationName:"enabled" type:"boolean"`
	// Configuration for node pools that defines the compute resources for your
	// EKS Auto Mode cluster. For more information, see EKS Auto Mode Node Pools
	// in the EKS User Guide.
	NodePools []*string `locationName:"nodePools" type:"list"`
	// The ARN of the IAM Role EKS will assign to EC2 Managed Instances in your
	// EKS Auto Mode cluster. This value cannot be changed after the compute capability
	// of EKS Auto Mode is enabled. For more information, see the IAM Reference
	// in the EKS User Guide.
	NodeRoleArn *string `locationName:"nodeRoleArn" type:"string"`
}

Hint about node pools:

nodePools: EKS Auto Mode includes general-purpose and system default Node Pools. src & docs

StorageConfig *StorageConfigRequest

type StorageConfigRequest struct {
	// Request to configure EBS Block Storage settings for your EKS Auto Mode cluster.
	BlockStorage *BlockStorage `locationName:"blockStorage" type:"structure"`
}

type BlockStorage struct {
	// Indicates if the block storage capability is enabled on your EKS Auto Mode
	// cluster. If the block storage capability is enabled, EKS Auto Mode will create
	// and delete EBS volumes in your Amazon Web Services account.
	Enabled *bool `locationName:"enabled" type:"boolean"`
}

KubernetesNetworkConfig *KubernetesNetworkConfigRequest

type KubernetesNetworkConfigRequest struct {
	// Indicates the current configuration of the load balancing capability on your
	// EKS Auto Mode cluster. For example, if the capability is enabled or disabled.
	ElasticLoadBalancing *ElasticLoadBalancing
}

type ElasticLoadBalancing struct {
	// Indicates if the load balancing capability is enabled on your EKS Auto Mode
	// cluster. If the load balancing capability is enabled, EKS Auto Mode will
	// create and delete load balancers in your Amazon Web Services account.
	Enabled *bool `locationName:"enabled" type:"boolean"`
}

API draft
Draft of a potential autoMode type:

autoModeConfig:
  nodeRoleArn: arn:aws:iam::123456789012:role/eks-node-role
  nodePools:
    generalPurpose: true
    system: true
  blockStorage: true
  elasticLoadBalancing: true

Logic for ComputeConfig:

# ComputeConfig always requires nodeRoleArn to be set.
if nodeRoleArn != nil {
  ComputeConfig.Enabled = true
}

# NodePools can be set only if ComputeConfig is enabled by setting nodeRoleArn.
if nodePools != nil {
  # nodeRoleArn must be set.
}

_{Sean Schneeweiss [email protected], Mercedes-Benz Tech Innovation GmbH, Provider Information}