Refactor azure-keyvault-secrets to patch-based structure

[Copilot]

Description

Restructures azure-keyvault-secrets from handwritten-wrapper-on-generated to patch-based customization pattern, matching azure-keyvault-securitydomain reference implementation.

Changes

New Structure

• _patch.py - Root patch containing:
  • ApiVersion enum (migrated from _shared/client_base.py)
  • SecretClient implementation with challenge auth, custom polling, and all secret operations
  • API version formatting utilities
• models/_patch.py - Handwritten models (SecretProperties, KeyVaultSecret, DeletedSecret, KeyVaultSecretIdentifier)
• _internal/__init__.py - Internal utilities bridge (challenge auth, polling, ID parsing)
• _operations/_patch.py - Operations patch (minimal)
• aio/_patch.py - Async patch (minimal, preserves existing _client.py)

Updated Import Pattern

All __init__.py files now use patch-based imports with TYPE_CHECKING guards:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from ._patch import *

from ._patch import __all__ as _patch_all
from ._patch import *
from ._patch import patch_sdk as _patch_sdk

__all__ = [...]
_patch_sdk()

Backward Compatibility

Public API unchanged - all classes, methods, and signatures identical. Old _client.py and _models.py preserved for transition.

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• pypi.org
  • Triggering command: /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/bin/pip pip install isodate -q (dns block)
  • Triggering command: /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/bin/python /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/bin/python /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/lib/python3.9/site-packages/pip/__pip-REDACTED__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-37382dib/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i REDACTED -- setuptools>=40.8.0 (dns block)
  • Triggering command: /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/bin/python /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/bin/python /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/lib/python3.9/site-packages/pip/__pip-REDACTED__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-36i5iycv/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i REDACTED -- setuptools>=40.8.0 (dns block)
• scanning-api.github.com
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

Objective

Refactor azure-keyvault-secrets from the old handwritten-layer-on-generated-code structure to the new generation-focused structure with a _patch.py patching layer. Use azure-keyvault-securitydomain as the reference implementation for the target structure.

Background

Current Structure (Old Format):

• Handwritten code lives outside _generated/ directory
• Generated code is isolated in _generated/ subdirectory
• Handwritten _client.py wraps generated client
• Handwritten _models.py wraps generated models
• Manual exports in __init__.py

Target Structure (New Format - based on securitydomain):

• Generated code is in the main namespace (no _generated/ subdirectory)
• _patch.py files at various levels modify/extend generated code
• _internal/ directory for handwritten utilities that patches import
• __init__.py uses automatic patch-based imports
• models/ directory (not _generated/models/) with its own _patch.py

Detailed Refactoring Requirements

1. Root Package Structure (azure/keyvault/secrets/)

Update __init__.py

• Change from manual imports to patch-based pattern
• Add TYPE_CHECKING block for type hints
• Import from _patch with try/except pattern
• Call _patch_sdk() at module initialization
• Pattern to follow:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from ._patch import *

from ._client import SecretClient  # type: ignore
from ._version import VERSION

__version__ = VERSION

try:
    from ._patch import __all__ as _patch_all
    from ._patch import *
except ImportError:
    _patch_all = []
from ._patch import patch_sdk as _patch_sdk

__all__ = [
    "SecretClient",
]
__all__.extend([p for p in _patch_all if p not in __all__])

_patch_sdk()

Create _patch.py

This is the main handwritten customization file. It should include:

1. Imports:

   • Import generated SecretClient as base class
   • Import utilities from _internal/ (challenge auth, polling, etc.)
   • Import models from models/

2. ApiVersion enum:

   • Move from _shared/client_base.py
   • Include all supported API versions with V7_5 (or latest) as default

3. Enhanced SecretClient class:

   • Inherit from generated client
   • Override __init__ to add:
     • Challenge authentication policy
     • HTTP logging policy with Key Vault headers
     • API version handling
     • vault_url property
   • Keep existing method signatures but delegate to generated operations where possible
   • Add handwritten methods like send_request with API version formatting
   • Maintain polling customizations for delete/recover operations

4. Helper functions:

   • _format_api_version() function (similar to securitydomain)

5. Export list:

__all__: List[str] = [
    "SecretClient",
    "ApiVersion",
]

6. patch_sdk() function:

def patch_sdk():
    """Do not remove from this file.
    
    `patch_sdk` is a last resort escape hatch that allows you to do customizations
    you can't accomplish using the techniques described in
    https://aka.ms/azsdk/python/dpcodegen/python/customize
    """

2. Models Refactoring (azure/keyvault/secrets/models/)

The current _generated/ directory should be restructured:

Create models/__init__.py

• Follow the securitydomain pattern with TYPE_CHECKING and patch imports
• Import generated models from _models.py
• Import enums from _enums.py if any
• Import and extend with _patch
• Pattern:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from ._patch import *

from ._models import (  # type: ignore
    # Generated model imports
)

from ._patch import __all__ as _patch_all
from ._patch import *
from ._patch import patch_sdk as _patch_sdk

__all__ = [
    # List generated models
]
__all__.extend([p for p in _patch_all if p not in __all__])
_patch_sdk()

Create models/_patch.py

Move handwritten models from the current _models.py:

• SecretProperties - with all its properties and _from_secret_bundle, _from_secret_item class methods
• KeyVaultSecret - with _from_secret_bundle class method
• KeyVaultSecretIdentifier - parser for secret IDs
• DeletedSecret - with deletion-related properties

Export them:

__all__: List[str] = [
    "DeletedSecret",
    "KeyVaultSecret", 
    "KeyVaultSecretIdentifier",
    "SecretProperties",
]

def patch_sdk():
    """Do not remove from this file."""

Keep models/_models.py

This should contain generated model classes (currently in _generated/models/):

• SecretBundle
• DeletedSecretBundle
• SecretItem
• DeletedSecretItem
• SecretAttributes
• SecretSetParameters
• SecretUpdateParameters
• SecretRestoreParameters
• Etc.

These models are referenced by handwritten code but not directly exposed to users.

3. Internal Utilities (azure/keyvault/secrets/_internal/)

Create this directory to house handwritten utilities that `_patch....

This pull request was created as a result of the following prompt from Copilot chat.

Objective

Refactor azure-keyvault-secrets from the old handwritten-layer-on-generated-code structure to the new generation-focused structure with a _patch.py patching layer. Use azure-keyvault-securitydomain as the reference implementation for the target structure.

Background

Current Structure (Old Format):

• Handwritten code lives outside _generated/ directory
• Generated code is isolated in _generated/ subdirectory
• Handwritten _client.py wraps generated client
• Handwritten _models.py wraps generated models
• Manual exports in __init__.py

Target Structure (New Format - based on securitydomain):

• Generated code is in the main namespace (no _generated/ subdirectory)
• _patch.py files at various levels modify/extend generated code
• _internal/ directory for handwritten utilities that patches import
• __init__.py uses automatic patch-based imports
• models/ directory (not _generated/models/) with its own _patch.py

Detailed Refactoring Requirements

1. Root Package Structure (azure/keyvault/secrets/)

Update __init__.py

• Change from manual imports to patch-based pattern
• Add TYPE_CHECKING block for type hints
• Import from _patch with try/except pattern
• Call _patch_sdk() at module initialization
• Pattern to follow:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from ._patch import *

from ._client import SecretClient  # type: ignore
from ._version import VERSION

__version__ = VERSION

try:
    from ._patch import __all__ as _patch_all
    from ._patch import *
except ImportError:
    _patch_all = []
from ._patch import patch_sdk as _patch_sdk

__all__ = [
    "SecretClient",
]
__all__.extend([p for p in _patch_all if p not in __all__])

_patch_sdk()

Create _patch.py

This is the main handwritten customization file. It should include:

1. Imports:

   • Import generated SecretClient as base class
   • Import utilities from _internal/ (challenge auth, polling, etc.)
   • Import models from models/

2. ApiVersion enum:

   • Move from _shared/client_base.py
   • Include all supported API versions with V7_5 (or latest) as default

3. Enhanced SecretClient class:

   • Inherit from generated client
   • Override __init__ to add:
     • Challenge authentication policy
     • HTTP logging policy with Key Vault headers
     • API version handling
     • vault_url property
   • Keep existing method signatures but delegate to generated operations where possible
   • Add handwritten methods like send_request with API version formatting
   • Maintain polling customizations for delete/recover operations

4. Helper functions:

   • _format_api_version() function (similar to securitydomain)

5. Export list:

__all__: List[str] = [
    "SecretClient",
    "ApiVersion",
]

6. patch_sdk() function:

def patch_sdk():
    """Do not remove from this file.
    
    `patch_sdk` is a last resort escape hatch that allows you to do customizations
    you can't accomplish using the techniques described in
    https://aka.ms/azsdk/python/dpcodegen/python/customize
    """

2. Models Refactoring (azure/keyvault/secrets/models/)

The current _generated/ directory should be restructured:

Create models/__init__.py

• Follow the securitydomain pattern with TYPE_CHECKING and patch imports
• Import generated models from _models.py
• Import enums from _enums.py if any
• Import and extend with _patch
• Pattern:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from ._patch import *

from ._models import (  # type: ignore
    # Generated model imports
)

from ._patch import __all__ as _patch_all
from ._patch import *
from ._patch import patch_sdk as _patch_sdk

__all__ = [
    # List generated models
]
__all__.extend([p for p in _patch_all if p not in __all__])
_patch_sdk()

Create models/_patch.py

Move handwritten models from the current _models.py:

• SecretProperties - with all its properties and _from_secret_bundle, _from_secret_item class methods
• KeyVaultSecret - with _from_secret_bundle class method
• KeyVaultSecretIdentifier - parser for secret IDs
• DeletedSecret - with deletion-related properties

Export them:

__all__: List[str] = [
    "DeletedSecret",
    "KeyVaultSecret", 
    "KeyVaultSecretIdentifier",
    "SecretProperties",
]

def patch_sdk():
    """Do not remove from this file."""

Keep models/_models.py

This should contain generated model classes (currently in _generated/models/):

• SecretBundle
• DeletedSecretBundle
• SecretItem
• DeletedSecretItem
• SecretAttributes
• SecretSetParameters
• SecretUpdateParameters
• SecretRestoreParameters
• Etc.

These models are referenced by handwritten code but not directly exposed to users.

3. Internal Utilities (azure/keyvault/secrets/_internal/)

Create this directory to house handwritten utilities that _patch.py files import. Move relevant code from _shared/:

_internal/__init__.py

Export the key utilities:

from .challenge_auth_policy import ChallengeAuthPolicy
from .polling import DeleteRecoverPollingMethod, KeyVaultOperationPoller
from .helpers import parse_key_vault_id

__all__ = [
    "ChallengeAuthPolicy",
    "DeleteRecoverPollingMethod",
    "KeyVaultOperationPoller",
    "parse_key_vault_id",
]

_internal/challenge_auth_policy.py

Move challenge authentication logic from _shared/challenge_auth_policy.py

_internal/polling.py

Move polling utilities from _shared/_polling.py:

• DeleteRecoverPollingMethod
• KeyVaultOperationPoller

_internal/helpers.py

Move utilities from _shared/ like:

• parse_key_vault_id function

Note: Keep _shared/ directory if it contains code shared across multiple Key Vault SDKs (certificates, keys, etc.). Only move secrets-specific code to _internal/.

4. Operations (azure/keyvault/secrets/_operations/)

Generated operations should live here (currently in _generated/operations/).

_operations/__init__.py

Follow patch pattern to import generated operations.

_operations/_patch.py

If any operations need customization, add them here. Otherwise, can be minimal:

__all__: List[str] = []

def patch_sdk():
    """Do not remove from this file."""

5. Generated Client (_client.py, _configuration.py)

These should be the generated versions from codegen, not handwritten. The generated _client.py will be a base that _patch.py extends.

If these files need to be regenerated:

• The _client.py should contain basic generated SecretClient
• The _configuration.py should contain configuration class
• All customizations go in _patch.py

6. Async Support (azure/keyvault/secrets/aio/)

Apply the same pattern:

aio/__init__.py

• Patch-based imports
• Import from ._patch and .._patch (parent)

aio/_patch.py

• Async version of SecretClient customizations
• Import from ._client (generated async client)
• Import from .._internal for shared utilities
• Similar customizations as sync version

aio/_client.py, aio/_configuration.py

Generated async versions.

7. Keep Existing Files

These files should remain largely unchanged:

• _version.py - Version string
• _sdk_moniker.py - SDK moniker for telemetry
• py.typed - Type hint marker
• All test files
• All sample files
• Documentation files (README, CHANGELOG, etc.)

8. Remove Old Structure

After migration:

• Remove or deprecate old handwritten _client.py (if not converted to generated)
• Remove old handwritten _models.py (content moved to models/_patch.py)
• Remove _generated/ subdirectory structure
• Update imports throughout the codebase

Testing Requirements

• All existing tests should continue to pass
• Public API should remain unchanged
• Type hints should be preserved
• Documentation references should be updated if needed

Reference Implementation

Use sdk/keyvault/azure-keyvault-securitydomain as the reference for:

• File structure and organization
• _patch.py patterns and style
• Import patterns in __init__.py files
• How generated and handwritten code interact

Success Criteria

1. ✅ Generated code lives in main namespace (no _generated/ subdirectory)
2. ✅ _patch.py files properly extend generated code
3. ✅ _internal/ directory contains handwritten utilities
4. ✅ All public APIs remain unchanged and backward compatible
5. ✅ All existing tests pass
6. ✅ Structure matches azure-keyvault-securitydomain pattern
7. ✅ Type hints work correctly
8. ✅ Documentation builds successfully

Notes

• This is a proof-of-concept refactoring
• Start with the core structure and most important classes
• Can be done incrementally but ensure it's functional at each step
• Maintain backward compatibility throughout
• The PR should be opened as a DRAFT for review

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.