[feature request] alternative registry to txt and aws, like redis or etcd ?

[mcarbonneaux]

What would you like to be added:

The only available registry are TXT, AWS Dynamo or AWS Service Discovery. in reality when your are not on AWS the only available option is TXT...

Can you add an alternative local registry option like redis, etcd, consul or equivalent?

Why is this needed:

In a non-aws environment (like on-premises), it is useful to not store metadata in TXT fields to avoid overloading DNS with changes.
In on-premises kubernetes cluster, it would be useful to store metadata in a local (in cluster) redis/ectd/consul cluster for example.

[ivankatliarchuk]

It would be nice to have cloud agnostic storage support.

Many providers enforce API limits for various reasons, which restricts the number of records we can add, update, or delete.

Challenges with External DNS and API Limits: The AWS Example

Disclaimer; AWS used as an example replace with any provider. And only AWS have some options, but not sure how useful they are at the moment

Currently, external DNS faces significant challenges when dealing with API rate limits in scenarios like AWS Route 53.

AWS Route 53 has a default API rate limit of 5 requests per second, applied account-wide. This effectively hinders scaling Route 53 usage unless you create a separate AWS account for each hosted zone.

Key Limitations:

• The API limit does not consider:
  • The number of Route 53 hosted zones.
  • The type of operation (e.g., read vs. write).
  • Consumer roles (e.g., Role A vs. Role B).

As a result, if your account manages multiple zones or records and has several consumers of the Route 53 API, it's easy to reach Denial of Service (DoS) scenarios.

Example Scenario:

Imagine an account with:

• Over 100 hosted zones (a mix of public and private zones).
• Hundreds of records in some zones.
• Multiple EKS clusters, each with its own instance of Kubernetes external-dns managing DNS records for services.

When external-dns starts up:

1. It queries all hosted zones via API calls.
2. It retrieves and validates records for services (triggering more API operations per record).

Additional Complications:

Beyond external-dns, other tools like Terraform, Ansible, and Crossplane controllers may also manage DNS records. With multiple tools accessing the same account-wide API limit, it’s possible for the limit to be consumed entirely for several minutes. During this period, other services needing to query the Route 53 API (e.g., for reads) may become completely blocked.

It would be extremely helpful if we have at least one more cloud agnostic registry

• postgress
• etcd
• redis
• ...

[ivankatliarchuk]

This could also help standardise the registry interface, and we should explore options to support additional registries as add-ons, similar to how webhooks are implemented for providers.

[ivankatliarchuk]

Found this half-done pull request #4598

Very relevant

[mcarbonneaux]

at the base is implementation of the interface:

external-dns/registry/registry.go

Lines 30 to 36 in 9619e6b

	type Registry interface {
	Records(ctx context.Context) ([]*endpoint.Endpoint, error)
	ApplyChanges(ctx context.Context, changes *plan.Changes) error
	AdjustEndpoints(endpoints []*endpoint.Endpoint) ([]*endpoint.Endpoint, error)
	GetDomainFilter() endpoint.DomainFilterInterface
	OwnerID() string
	}

with very litle information on each function :

external-dns/registry/registry.go

Lines 26 to 29 in 9619e6b

	// Registry is an interface which should enables ownership concept in external-dns
	// Records() returns ALL records registered with DNS provider
	// each entry includes owner information
	// ApplyChanges(changes *plan.Changes) propagates the changes to the DNS Provider API and correspondingly updates ownership depending on type of registry being used

[akrizs]

up this one,, feel like the registry solutions today by injecting a txt record into the zone at AWS isn't particularly safe, exposing paths and/or pod/service names,, if one were to name services/ingresses with internal customer ID's then they would be exposed on the DNS via a txt record...

maybe that is an antipattern to use those types of id's in the service names, but somehow they need to be distinguished in the cluster.

having an extra etcd storage wouldn't be that of a big deal and connect external-dns to that.

p.s was testing out the config in external-dns and set the registry to noop to try to avoid exposing info and it deleted all of the records in my zone,, haha!

guess it was a combination of noop and setting the policy to sync,,

[lucasfcnunes]

p.s was testing out the config in external-dns and set the registry to noop to try to avoid exposing info and it deleted all of the records in my zone,, haha!

guess it was a combination of noop and setting the policy to sync,,

That's very concerning...

[ivankatliarchuk]

Noop registry is tricky, mainly for providers not supporting mechanisms to track ownership.