Keys with double underscores not correctly converted into ASP.NET Core Options

[brgrz]

The ASP.NET Configuration provider will correctly load and convert the following environment variables that include double underscores into hierarchical configuration and also populate the Options by using GetRequiredSection() method.

However this does not work with configuration loaded from Azure App Configuration

Example env vars:

Auth__Client_id=client123
Auth__ApiHost=https://auth0.com

The same two keys exist in our AAC.

When the app loads and the configuration provider loads the values this is how it looks in the Configuration. I've attached screenshot from the debugger to visualize the difference in how the environment variables were loaded and how the AAC configuration keys were loaded.

Unfortunately the way AAC keys were loaded means that the GetRequiredSection("Auth") method on the Configuration instance fails with an exception of non existing "Auth" section.

We would very much like to use double underscores for separators within the AAC because it allows us to override those values with values from environment variables which usually come in a form with two underscores.

[brgrz]

Also note, after further testing I can testify to say using the colon : for separator does work as expected with regards to using the Options pattern. I would consider not being able to use double underscores in the same way as a bug in configuration loader either in the AddAzureAppConfiguration() extension or the actual ASP.NET Core configuration loader.

This is a code snippet used to load from the Azure App Configuration endpoint

builder.Configuration.AddAzureAppConfiguration(options =>
{
				options.Connect(builder.Configuration.GetRequiredConnectionString("AzureAppConfiguration"));
				options.Select($"{ConfigPrefix}*", env);
				options.TrimKeyPrefix(ConfigPrefix);
});

[zhenlan]

.NET uses : as the separator for configuration key names. However, Linux doesn't allow : in environment variable names. Therefore, the .NET environment configuration provider converts double underscores to colons. This is why you see colons instead of double underscores under the debugger. This special handling is done by the .NET environment configuration provider.

When using the Azure App Configuration provider, you don’t need such special conversion. You should use : when creating key-values.

using the colon : for separator does work as expected with regards to using the Options pattern

Not sure how it is unexpected, but we have a document that uses the Options pattern:
https://learn.microsoft.com/en-us/azure/azure-app-configuration/quickstart-aspnet-core-app

[zhenlan]

@brgrz please let me if there is anything else I can help with. Closing for now.

[brgrz]

Therefore, the .NET environment configuration provider converts double underscores to colons. This is why you see colons instead of double underscores under the debugger.

This is actually the issue. When we use double underscores in env variable keys .NET correctly turns them into colons and creates a hierarchy.

If we use double underscores in Azure App Config keys then this conversion does not happen but I think it should. I shouldn't be required to rewrite key values with colons.

Since AAC supports double underscores we would like to keep using them as we already use them as said in env vars and keeps things simpler (we can copy/share key values in various places).

[zhenlan]

@brgrz can you please share the reason you can't use : as key separator in App Configuration given : is what .NET expects?

App Configuration has minimal character limits, allowing it to be used across various platforms and languages. However, it is the user’s responsibility to ensure that the key-values are authored in a way that is supported by the target platform or language.

[brgrz]

@brgrz can you please share the reason you can't use : as key separator in App Configuration given : is what .NET expects?

Again, .NET allows for double underscores and correctly turns them into colons if those double__underscored keys come from env variables. If, however, they come from Azure App Configuration they don't get turned into colons.

You are asking me why I can't use : as a separator. I actually could use colon as a separator in Azure App Configuration but prefer to use double underscores because we also have some settings in env variables and those only support double underscores if you target multiplatform. And to keep things similar/consistent it is easier to use double underscores everywhere.

In short, double underscores are supported by .NET and are multiplatform but when such keys are loaded from Azure App Configuration the middleware does not turn double underscores into colons (and as such GetOptions() pattern fails).

And I believe that is a bug within Azure App Configuration middleware.

[brgrz]

From

https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-9.0#configuration-keys-and-values

Hierarchical keys
Within the Configuration API, a colon separator (:) works on all platforms.
In environment variables, a colon separator may not work on all platforms. A double underscore, __, is supported by all platforms and is automatically converted into a colon :.
In Azure Key Vault, hierarchical keys use -- as a separator. The Azure Key Vault configuration provider automatically replaces -- with a : when the secrets are loaded into the app's configuration.

The behaviour is documented here - how it should work - but it doesn't when we use AAC to load settings.

[zhiyuanliang-ms]

Map api mentioned in this thread can be a workaround solution.