[client] Implement MCP OAuth scope selection and step-up authorization

[dogacancolak]

Implements the MCP specification's OAuth scope selection strategy and step-up authorization flow for handling scope challenges.

Specification Sections Implemented

1. Scope Selection Strategy - Priority order for selecting scopes during authorization:

   • Use scope from WWW-Authenticate header (if provided)
   • Use all scopes from PRM scopes_supported (if available)
   • Omit scope parameter (if neither is available)

2. Scope Challenge Handling - Step-up authorization when 403 with insufficient_scope is received:

   • Extract updated scope from WWW-Authenticate header
   • Trigger re-authorization with new scope
   • Retry request with new tokens

Changes Made

Core Implementation:

• Added _extract_field_from_www_auth() - Generic WWW-Authenticate header parser
• Added _extract_scope_from_www_auth() - Extracts scope per RFC6750
• Added _select_scopes() - Implements MCP scope selection priority
• Added 403 handler in async_auth_flow() - Step-up authorization flow
• Removed fallback to authorization server metadata scopes

Testing

Comprehensive test coverage added in tests/client/test_auth.py:

Scope Selection Tests:

• WWW-Authenticate scope prioritized over PRM
• PRM scopes used when WWW-Authenticate has no scope
• Scope omitted when neither provides values
• Various WWW-Authenticate header formats (quoted/unquoted)

Step-Up Authorization Tests:

• 403 with insufficient_scope triggers step-up flow
• Scope correctly updated from WWW-Authenticate
• 403 without insufficient_scope does not trigger step-up

Breaking Changes

• No longer falls back to authorization server metadata scopes_supported

---

Checklist

• Follows MCP specification
• Code follows style guidelines
• All tests pass locally
• Added appropriate error handling

[jennsun]

Can we confirm if updating the Python SDK will update the Claude Client, or will we also need to make updates in the TypeScript SDK for these to reflect in Claude.ai? In the Typescript SDK, for example, we are not retrieving scopes from the PRM endpoint: https://github.com/modelcontextprotocol/typescript-sdk/blob/main/src/client/auth.ts#L328
CC: @pcarleton

[pcarleton]

see this latest change to the spec:

When implementing authorization flows, MCP clients SHOULD follow the principle of least privilege by requesting
only the scopes necessary for their intended operations. During the initial authorization handshake, MCP clients
SHOULD follow this priority order for scope selection:

1. Use scope parameter from the initial WWW-Authenticate header in the 401 response, if provided
2. If scope is not available, use all scopes defined in scopes_supported from the Protected Resource Metadata document, omitting the scope parameter if scopes_supported is undefined.

This approach accommodates the general-purpose nature of MCP clients, which typically lack domain-specific knowledge to make informed decisions about individual scope selection. Requesting all available scopes allows the authorization server and end-user to determine appropriate permissions during the consent process.

Are you able to update this PR to:

1. Get scope from www-authenticate
2. Omit scope if not present in PRM? (currently impl falls back to scopes_supported from AS metadata)

[dogacancolak]

Updated. Although the complete priority order is:

1. Explicitly configured scope in the client
2. www-authenticate scope
3. PRM scope
4. Omit

[pcarleton]

Hm I don't think (1) is correct there. If we always use what's configured on the client, there's no path for upgrading scope if the initial request didn't have the correct scope.

We had some discussion in this across a few PR's (example) and in discord about similar issues.

[dogacancolak]

According to this spec https://modelcontextprotocol.io/specification/draft/basic/authorization#scope-challenge-handling the step-up only happens when the server returns 403. The changes in this PR only take effect upon a 401.

I can definitely update the 401 path by not prioritizing the configured scope on the client, but the step-up logic remains unhandled in the MCP client in this SDK. I can create a separate PR for that if that makes sense

[dogacancolak]

I can make it part of this PR too considering this is already a breaking change

[dogacancolak]

working on adding step-up auth flow

[dogacancolak]

done