Simplify project by scoping it to GKE (while maximizing Kubernetes provider agnosticity)

[bwplotka]

As discussed in #780 (comment) we propose to remove support for EKS and kind. No one tests or maintain that part and it makes prombench more complex than it's needed.

We are using GKE and there is a way to minimize use of GKE or GCP specific features (e.g. Kubernetes API only), so any downstream (e.g. forks) users could implement their own providers. Really, the only provider agnostic logic should be for creating nodepools.

Acceptance Criteria

• Remove non GKE resources from https://github.com/prometheus/test-infra/tree/master/prombench/manifests
• Remove infra or replace it with some other CLI that might help us for scenario building e.g. prombenchctl.

WDYT?

If you are using prombench on EKS or kind and you would be impacted by this change, let us know!

Disclaimer: I work for Google.

[kakkoyun]

We should aim to clearly separate Kubernetes providers from the Kubernetes stack itself, if this distinction isn’t already in place. This separation ensures that users can easily test the stack on their preferred Kubernetes setup, whether it’s Kind, Minikube, or other solutions.

For development purposes, we can maintain a straightforward Kind configuration to simplify local testing.

That said, I’m also happy to restrict official support to GKE since it’s the only provider we actively test against. However, to improve maintainability, we should consider transitioning to infrastructure-as-code tools like Terraform for managing IaaS resources, instead of relying on custom configuration code.

[bboreham]

Agreed, we should retain kind as an option for local testing.

[bwplotka]

Cool, let's then use kind in our CI, otherwise it will get stale. Added issue about this: #795

If we keep kind then we can't really simplify our scripts and infra CLI much, so closing this idea for now. Thanks for super quick feedback @bboreham and @kakkoyun ! 💪🏽