Eks

---
name: eks
description: "Expert coverage of Amazon Elastic Kubernetes Service (EKS): managed control plane, Karpenter, Fargate, IRSA, Pod Identity, VPC CNI, EKS add-ons, EKS Anywhere, and Auto Mode. Use for \"EKS\", \"Amazon EKS\", \"Karpenter\", \"Fargate\", \"IRSA\", \"VPC CNI\", \"EKS Auto Mode\", \"EKS Anywhere\", \"eksctl\"."
license: MIT
---

# Amazon EKS

This skill covers Amazon Elastic Kubernetes Service (EKS), including:

- EKS managed control plane (HA, upgrades, API server configuration)
- Compute options (Managed Node Groups, self-managed nodes, Fargate, EKS Auto Mode)
- Karpenter node provisioning (NodePool, EC2NodeClass, consolidation, spot)
- IAM integration (IRSA, EKS Pod Identity, node IAM roles)
- Networking (VPC CNI, pod networking, security groups for pods, prefix delegation)
- EKS add-ons (CoreDNS, kube-proxy, VPC CNI, EBS CSI, EFS CSI, ADOT, GuardDuty)
- EKS Anywhere (bare metal, VMware, Nutanix, Snow)
- Storage (EBS CSI, EFS CSI, FSx Lustre CSI)
- Observability (Container Insights, ADOT, Prometheus managed service)
- Security (envelope encryption, cluster endpoint access, security groups)

## How to Approach Tasks

1. **Classify** the request:
   - **Architecture** -- Load `references/architecture.md` for control plane, networking, compute, storage details
   - **Node provisioning** -- Karpenter vs Managed Node Groups vs Fargate decision and configuration
   - **IAM/Security** -- IRSA, Pod Identity, encryption, endpoint access
   - **Networking** -- VPC CNI configuration, pod IP management, security groups for pods
   - **Troubleshooting** -- Node join failures, pod networking issues, IAM permission errors

2. **Identify context** -- Which compute type? Which CNI mode? Which EKS version? Ask if unclear.

3. **Load context** -- Read the reference file for deep technical detail.

4. **Apply** -- Provide AWS CLI, eksctl, or Terraform examples as appropriate.

5. **Validate** -- Suggest `aws eks describe-cluster`, `kubectl get nodes`, CloudWatch metrics.

## EKS Control Plane

- Fully managed: AWS runs API server and etcd across 3 AZs
- SLA: 99.95% uptime
- Version support: 14 months standard, 12 months extended ($0.60/hr)
- Upgrade: one minor version at a time; control plane upgrades are in-place (rolling API server update)
- Cluster endpoint: public, private, or public+private. Private endpoint uses VPC interface endpoint.
- Envelope encryption: optional KMS key for encrypting Kubernetes Secrets in etcd

## Compute Options

### Managed Node Groups

AWS manages the EC2 Auto Scaling Group, AMI selection, and node draining during upgrades:

```bash
eksctl create nodegroup \
  --cluster=prod \
  --name=workers \
  --node-type=m6i.xlarge \
  --nodes=4 \
  --nodes-min=2 \
  --nodes-max=10 \
  --node-ami-family=AmazonLinux2023 \
  --managed
```

AMI families: `AmazonLinux2023` (recommended), `AmazonLinux2`, `Bottlerocket`, `Ubuntu`, `Windows`.

**Update strategy**: rolling update by default. Surge (`maxUnavailable`) controls how many nodes update simultaneously.

### Karpenter

Karpenter is the de facto standard for EKS node autoscaling (replaces Cluster Autoscaler). Current: v1.10.x.

```yaml
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      nodeClassRef:
        apiVersion: karpenter.k8s.aws/v1
        kind: EC2NodeClass
        name: general
      requirements:
      - key: karpenter.sh/capacity-type
        operator: In
        values: ["spot", "on-demand"]
      - key: karpenter.k8s.aws/instance-category
        operator: In
        values: ["m", "c", "r"]
      - key: karpenter.k8s.aws/instance-generation
        operator: Gt
        values: ["5"]
      - key: kubernetes.io/arch
        operator: In
        values: ["amd64"]
  limits:
    cpu: "1000"
    memory: "4000Gi"
  disruption:
    consolidationPolicy: WhenUnderutilized
    consolidateAfter: 30s
    budgets:
    - nodes: "10%"
```

```yaml
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: general
spec:
  amiFamily: AL2023
  role: KarpenterNodeRole-prod
  subnetSelectorTerms:
  - tags:
      karpenter.sh/discovery: prod
  securityGroupSelectorTerms:
  - tags:
      karpenter.sh/discovery: prod
  blockDeviceMappings:
  - deviceName: /dev/xvda
    ebs:
      volumeSize: 100Gi
      volumeType: gp3
      iops: 3000
      throughput: 125
      encrypted: true
```

**Karpenter advantages over Cluster Autoscaler**:
- Provisions in seconds (not minutes)
- Selects optimal instance type per pod requirements (not limited to node group instance types)
- Consolidates underutilized nodes automatically
- Native spot instance support with automatic fallback to on-demand
- No node group management overhead

### Fargate

Serverless pods -- no node management:

```yaml
# Fargate profile
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: prod
fargateProfiles:
  - name: serverless
    selectors:
    - namespace: serverless
    - namespace: kube-system
      labels:
        k8s-app: coredns
```

**Fargate limitations**:
- No DaemonSets (no nodes to schedule on)
- No GPU workloads
- No privileged containers
- Max 4 vCPU, 30 GB memory per pod
- Higher per-pod cost than EC2 at sustained usage
- Cold start latency (~30-60 seconds)

### EKS Auto Mode

Combines managed node groups with Karpenter-like auto-provisioning. AWS manages compute, networking, and storage infrastructure automatically. Simplest operational model, recommended for teams without deep Kubernetes expertise.

## IAM Integration

### IRSA (IAM Roles for Service Accounts)

```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: s3-reader
  namespace: production
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/S3ReaderRole
```

**How IRSA works**: EKS runs an OIDC provider. When a pod uses the annotated ServiceAccount, kubelet injects a projected ServiceAccount token. The AWS SDK exchanges this OIDC token for temporary IAM credentials via STS AssumeRoleWithWebIdentity.

**Trust policy on the IAM role**:
```json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::ACCOUNT:oidc-provider/oidc.eks.REGION.amazonaws.com/id/CLUSTER_ID"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "oidc.eks.REGION.amazonaws.com/id/CLUSTER_ID:sub": "system:serviceaccount:production:s3-reader",
        "oidc.eks.REGION.amazonaws.com/id/CLUSTER_ID:aud": "sts.amazonaws.com"
      }
    }
  }]
}
```

### EKS Pod Identity (newer alternative to IRSA)

Simpler setup -- no OIDC provider management. Uses the EKS Pod Identity Agent add-on:

```bash
aws eks create-pod-identity-association \
  --cluster-name prod \
  --namespace production \
  --service-account s3-reader \
  --role-arn arn:aws:iam::ACCOUNT:role/S3ReaderRole
```

No annotation on the ServiceAccount needed. The association is managed cluster-side.

## Networking

### VPC CNI

Default EKS CNI. Pods get real VPC IP addresses from the node's subnet:

**Secondary IP mode** (default): each ENI has multiple secondary IPs assigned to pods. Limited by instance type's max ENIs and IPs-per-ENI.

**Prefix delegation**: assigns /28 prefixes to ENIs instead of individual IPs. Dramatically increases pod density (up to 110+ pods per node on m5.xlarge vs ~29 in secondary IP mode).

```bash
kubectl set env daemonset aws-node -n kube-system ENABLE_PREFIX_DELEGATION=true
kubectl set env daemonset aws-node -n kube-system WARM_PREFIX_TARGET=1
```

**Custom networking**: pods use a different subnet than the node's primary ENI. Useful when node subnets have limited IP space.

**Security groups for pods**: assign AWS security groups directly to pods (in addition to NetworkPolicy):
```yaml
apiVersion: vpcresources.k8s.aws/v1beta1
kind: SecurityGroupPolicy
metadata:
  name: my-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: database
  securityGroups:
    groupIds:
    - sg-0123456789abcdef0
```

## EKS Add-ons

Managed add-ons with automatic version management:

| Add-on | Purpose |
|--------|---------|
| CoreDNS | Cluster DNS |
| kube-proxy | Service networking |
| Amazon VPC CNI | Pod networking |
| EBS CSI Driver | EBS volume provisioning |
| EFS CSI Driver | EFS volume mounting |
| ADOT | AWS Distro for OpenTelemetry |
| GuardDuty Agent | Threat detection |
| Snapshot Controller | Volume snapshots |

```bash
aws eks create-addon --cluster-name prod --addon-name aws-ebs-csi-driver \
  --service-account-role-arn arn:aws:iam::ACCOUNT:role/EBSCSIRole
```

## Common Patterns

### Multi-AZ High Availability

```yaml
# Topology spread across AZs
topologySpreadConstraints:
- maxSkew: 1
  topologyKey: topology.kubernetes.io/zone
  whenUnsatisfiable: DoNotSchedule
  labelSelector:
    matchLabels:
      app: myapp

# Karpenter NodePool with multi-AZ
requirements:
- key: topology.kubernetes.io/zone
  operator: In
  values: ["us-east-1a", "us-east-1b", "us-east-1c"]
```

### Cost Optimization

- Use Karpenter with spot instances for fault-tolerant workloads
- Use Fargate for burst/batch workloads with unpredictable scheduling
- Right-size with VPA recommendations
- Use Savings Plans for baseline capacity, spot for variable
- Enable Karpenter consolidation to bin-pack underutilized nodes

## Reference Files

- `references/architecture.md` -- Managed control plane internals, Karpenter deep dive, Fargate architecture, IRSA/Pod Identity mechanics, VPC CNI modes, add-on management, EKS Anywhere. Read for architecture and design questions.