Too many calls 429 in Auzre App Configuration, how to solve it?

[anirugu]

I am seeing lot of occurrences of 429 when bounded context (microservice) is trying to get app config. I have some feature in featuremanager.

I am interested to know more about the process I can troubleshoot this issue. Is there any way I can solve it. I try to debug the code locally. By reducing the cache expiration to 1 seconds, my thought was to make it more calls to configuration so I can see them in logs.

Currently I am not seeing the configuration related logs in Azure app insights. Are there any hints on what can cause 429.

I have lot of assumption in my head regarding 429. Either there are too many pods which is calling the Azure app config at same time so that might cause, might be some E2E tests which is starting app and those start is flooding the Azure app config call.

Here is the implementation which look very normal. Any clue to find the root cause of the issue.

        return builder.AddAzureAppConfiguration(options => connectFunc(options)
            .UseFeatureFlags()
            .ConfigureRefresh(refresh =>
            {
                const string keyPrefix = ".appconfig.featureflag/";
                var featureFlags = //Method to get all features from Enums
                foreach (var featureFlag in featureFlags)
                {
                    refresh
                        .Register(keyPrefix + featureFlag, true);
                }

                if (cacheTime >= 1)
                {
                    refresh
                        .SetCacheExpiration(TimeSpan.FromSeconds(cacheTime));
                }
            }), true);

[amerjusupovic]

Hi @anirugu, the first thing I want to clarify about your implementation is that you shouldn't need to call ConfigureRefresh this way, because calling UseFeatureFlags already selects all of the flags with no label, adding them to your configuration and setting their refresh time to 30 seconds by default. You can change that time by changing the FeatureFlagOptions.CacheExpirationInterval value.

Besides these additional requests, it also depends what your cache expiration is normally, as a shorter time can potentially mean more requests to App Configuration depending on your application and how often you trigger a refresh.

You can check the descriptions of your 429 exceptions and reference this document to learn more about why you got them. You can also use the link about monitoring at the bottom of that page to check your request quota usage. Let me know if this helps or if you have any more questions.

[anirugu]

Thanks @amerjusupovic for reply.

Setting FeatureFlagOptions.CacheExpirationInterval in UseFeatureFlags looking good. I will try it.

I am also thinking to try using App Configuration Kubernetes Provider https://learn.microsoft.com/en-us/azure/azure-app-configuration/quickstart-azure-kubernetes-service
Will it help reducing traffic to Azure?

[RichardChen820]

It will help since your app does not have to talk to App Configuration directly, just need to refer to the ConfigMap that generated by the Kubernetes Provider.

But Kubernetes Provider does not support Feature Management yet, we are working on rolling out a new version to support it. It will come very soon