map localhost to both 127.0.0.1 and ::1

[mmerickel]

The current /etc/hosts on a node is:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost6 localhost6.localdomain6

This causes issues when using kubectl port-forward on a pod that is not listening on ipv4 interfaces. For example, when using kubectl port-forward on an istio proxy pod in an EKS-IPv6 cluster. When manually adding localhost to ::1 the problem goes away.

Example failure when running curl localhost:15000/config_dump:

kubectl port-forward -n istio-ingressgateway-private istio-ingressgateway-78ddddf548-nlf58 15000:15000
Forwarding from 127.0.0.1:15000 -> 15000
Forwarding from [::1]:15000 -> 15000
Handling connection for 15000
E0305 15:32:56.165931   80794 portforward.go:409] an error occurred forwarding 15000 -> 15000: error forwarding port 15000 to pod 2ff00ec557ed84d8928988130bae68a3a38ef77d0f27908db9b8acf771542875, uid : failed to execute portforward in network namespace "/var/run/netns/cni-a56faa1d-177f-2320-d9a4-a839886fdd7b": failed to connect to localhost:15000 inside namespace "2ff00ec557ed84d8928988130bae68a3a38ef77d0f27908db9b8acf771542875", IPv4: dial tcp4 127.0.0.1:15000: connect: connection refused IPv6 dial tcp6: address localhost: no suitable address found
error: lost connection to pod

Related: istio/istio#49496

[cartermckinnon]

I need to look into why this entry is used for the v6 loopback address on AL2 -- it seems like only a convention that varies by distro, but it could be load-bearing.

FWIW, it looks like cloud-init would create the ::1 localhost entry as you suggested: https://github.com/canonical/cloud-init/blob/0c80569929c8b40d244c49af2aac75c6928f06ab/templates/hosts.redhat.tmpl#L22 if you use manage_etc_hosts in your cloud config.

[mmerickel]

Using terraform-aws-eks module with AL2023 I'm able to get the localhost entries defined using manage_etc_hosts as suggested above with the following config:

      cloudinit_pre_nodeadm = [
        {
          # This is required for the ipv6 nodes since localhost isn't properly mapped
          # to ::1, causing kubectl port-forward to not work right on pods that are
          # only listening on ipv6 interfaces.
          # https://github.com/awslabs/amazon-eks-ami/issues/1714#issuecomment-1984232868
          content_type = "text/cloud-config"
          content      = <<-EOT
            manage_etc_hosts: true
          EOT
        },
      ]

Hope this helps someone and that you consider fixing this issue in the AMI itself.