How credentials refreshing works

[Veetaha]

Having thought a bit more about #1282, it just occurred to me that if aiobotocore loads credentials when the client is created, it stores the static values of aws_access_key_id, aws_secret_access_key, aws_session_token resolved at that time somewhere in the client object and never refreshes them, right?

I'm rather unexperienced with Python and boto yet, but I know JS and Rust SDKs don't do that. They load credentials lazily on the first API call and their client constructors are pure, infallible and synchronous. This credentials lazy loading model makes a lot of sense to me. I don't understand why [aio]botocore operates with static credential values, since they can expire pretty quickly and an auto-refreshing credentials provider must be used directly in the client.

Maybe I'm missing something, because I saw the answer #1119 (reply in thread) to a similar discussion where @thehesiod wrote a comment that using the client indefinitely is safe. Then why is it safe? Maybe I'm reading the code wrong? I can't unsee the fact that credentials are loaded eagerly, and they are cached indefinitely here:

aiobotocore/aiobotocore/session.py

Lines 89 to 93 in f13f103

	if self._credentials is None:
	self._credentials = await self._components.get_component(
	'credential_provider'
	).load_credentials()
	return self._credentials

I also don't see that the credentials track the expiration timestamp at all. For example, the credentials object in Rust SDK has an expires_after property (code link), while in botocore the Credentials object doesn't seem to have that property at all (code link). Maybe that method property somehow does the magic?

Also boto3 docs mention that credentials refreshing should be done automatically in the AssumeRole provider section, but the current eager credentials loading behaviour somehow contradicts this statement.

---

UPD looks like load_credentials() returns another instance of credentials (not the one that VSCode was hinting me), that have builtin logic of refreshing. However, I still don't understand why credentials are eagerly loaded.

---

On a related note, what is the official way to create auto-refreshable credentials for AssumeRole flow? set_credentials() on the session object accepts static credential values, but what I need is a way to specify the credential provider that will refresh the credentials instead.

I've found this answer on Stackoverflow, and it suggests setting the internal _credentials property of the Session which looks like a hack, and not the intended way it should work.

[thehesiod]

this is a discussion for botocore and not aiobotocore. botocore has a concept of refreshable and non refreshable credential types. if it's coming from disk it's not considered refreshable for example as they don't expect they will change

[thehesiod]

btw to see how refreshing is supposed to work put a breakpoint here: https://github.com/aio-libs/aiobotocore/blob/master/aiobotocore/signers.py#L164

[Veetaha]

Oh, I completely forgot that Python has a good debugger. Now it's much easier to see what happens, thanks!

I suppose you don't have an answer as to why credentials are loaded eagerly during client creation and wether it's expected to use _credentials to configure a custom credentials provider?

[thehesiod]

botocore question ;) we just adapt, don't design

…

On Fri, Jan 31, 2025, 10:33 AM Vitalii Kryvenko ***@***.***> wrote: Oh, I completely forgot that Python has a good debugger. Now it's much easier to see what happens, thanks! I suppose you don't have an answer as to why credentials are loaded eagery during client creation and wether it's fine to use _credentials to configure a custom credentials provider? — Reply to this email directly, view it on GitHub <#1283 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6Q77IE4G6CA4P72WTIWPT2NO6XNAVCNFSM6AAAAABWGYTEMOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMBSGEYDGNY> . You are receiving this because you were mentioned.Message ID: ***@***.***>