Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ type annotation issues to maintain type safety guarantees.
## Make Commands Reference

| Command | Purpose |
|---------|---------|
| --------- | --------- |
| `make sync` | Install all dependencies |
| `make install` | Alias for `make sync` |
| `make lint` | Lint Python + Markdown |
Expand Down
2 changes: 1 addition & 1 deletion docs/adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ provide alternative APIs tailored for specific use cases.
## Available Adapters

| Adapter | Description |
|---------|-------------|
| --------- | ------------- |
| [DataclassAdapter](#dataclassadapter) | Type-safe storage/retrieval of dataclass models with transparent serialization |
| [PydanticAdapter](#pydanticadapter) | Type-safe storage/retrieval of Pydantic models with transparent serialization |
| [RaiseOnMissingAdapter](#raiseonmissingadapter) | Optional raise-on-missing behavior for get operations |
Expand Down
6 changes: 3 additions & 3 deletions docs/stores.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ long-term storage, prefer stable stores.
Local stores are stored in memory or on disk, local to the application.

| Store | Stability | Async | Sync | Description |
|-------|:---------:|:-----:|:----:|:------------|
| ------- | :---------: | :-----: | :----: | :------------ |
| Memory | N/A | ✅ | ✅ | Fast in-memory storage for development and caching |
| Disk | Stable | ☑️ | ✅ | Persistent file-based storage in a single file |
| Disk (Per-Collection) | Stable | ☑️ | ✅ | Persistent storage with separate files per collection |
Expand Down Expand Up @@ -315,7 +315,7 @@ Secret stores provide secure storage for sensitive data, typically using
operating system secret management facilities.

| Store | Stability | Async | Sync | Description |
|-------|:---------:|:-----:|:----:|:------------|
| ------- | :---------: | :-----: | :----: | :------------ |
| Keyring | Stable | ✅ | ✅ | OS-level secure storage (Keychain, Credential Manager, etc.) |
| Vault | Unstable | ✅ | ✅ | HashiCorp Vault integration for enterprise secrets |

Expand Down Expand Up @@ -395,7 +395,7 @@ pip install py-key-value-aio[vault]
Distributed stores provide network-based storage for multi-node applications.

| Store | Stability | Async | Sync | Description |
|-------|:---------:|:-----:|:----:|:------------|
| ------- | :---------: | :-----: | :----: | :------------ |
| DynamoDB | Unstable | ✅ | ✖️ | AWS DynamoDB key-value storage |
| Elasticsearch | Unstable | ✅ | ✅ | Full-text search with key-value capabilities |
| Memcached | Unstable | ✅ | ✖️ | High-performance distributed memory cache |
Expand Down
2 changes: 1 addition & 1 deletion docs/wrappers.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ protocol, so they can be used anywhere a store can be used.
## Available Wrappers

| Wrapper | Description |
|---------|-------------|
| --------- | ------------- |
| [CompressionWrapper](#compressionwrapper) | Compress values before storing and decompress on retrieval |
| [FernetEncryptionWrapper](#fernetencryptionwrapper) | Encrypt values before storing and decrypt on retrieval |
| [FallbackWrapper](#fallbackwrapper) | Fallback to a secondary store when the primary store fails |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,54 +1,95 @@
from collections.abc import Sequence
from typing import TypeVar, get_origin
import contextlib
from collections.abc import Mapping
from dataclasses import is_dataclass
from typing import Annotated, Any, TypeVar, get_args, get_origin, is_typeddict

from key_value.shared.type_checking.bear_spray import bear_spray
from pydantic import BaseModel
from pydantic.type_adapter import TypeAdapter
from typing_extensions import TypeForm

from key_value.aio.adapters.pydantic.base import BasePydanticAdapter
from key_value.aio.protocols.key_value import AsyncKeyValue

T = TypeVar("T", bound=BaseModel | Sequence[BaseModel])
T = TypeVar("T")


class PydanticAdapter(BasePydanticAdapter[T]):
"""Adapter around a KVStore-compliant Store that allows type-safe persistence of Pydantic models."""
"""Adapter around a KVStore-compliant Store that allows type-safe persistence of any pydantic-serializable type.
# Beartype cannot handle the parameterized type annotation (type[T]) used here for this generic adapter.
# Using @bear_spray to bypass beartype's runtime checks for this specific method.
Supports:
- Pydantic models (BaseModel subclasses)
- Dataclasses and TypedDicts
- Dict types (dict[K, V])
- Sequences (list[T], tuple[T, ...], set[T])
- Primitives and other types (int, str, datetime, UUID, etc.)
Types that serialize to dicts (BaseModel, dataclass, TypedDict, dict) are stored directly.
All other types are wrapped in {"items": <value>} for consistent dict-based storage.
"""
Comment on lines +18 to +29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Clear docstring documenting expanded capabilities.

The docstring comprehensively lists supported types and explains the wrapping logic. One optional enhancement: consider mentioning that Annotated[T, ...] types are unwrapped to T before processing.

🤖 Prompt for AI Agents
In key-value/key-value-aio/src/key_value/aio/adapters/pydantic/adapter.py around
lines 18 to 29, update the module docstring to mention that
typing.Annotated[...] types are unwrapped to their underlying T before
processing; explicitly state that Annotated[T, ...] will be treated the same as
T for the purposes of the type checks and wrapping logic (i.e., treated as
model/dict types when T is such, or wrapped in {"items": value} when T is a
primitive/other), keeping the rest of the listed supported types and behavior
unchanged.


# Beartype cannot handle the parameterized type annotation used here.
# Using @bear_spray to bypass beartype's runtime checks for this method.
@bear_spray
def __init__(
self,
key_value: AsyncKeyValue,
pydantic_model: type[T],
pydantic_model: TypeForm[T],
default_collection: str | None = None,
raise_on_validation_error: bool = False,
) -> None:
"""Create a new PydanticAdapter.
Args:
key_value: The KVStore to use.
pydantic_model: The Pydantic model to use. Can be a single Pydantic model or list[Pydantic model].
pydantic_model: The type to use for serialization/deserialization. Can be any
pydantic-serializable type: BaseModel, dataclass, TypedDict, dict[K, V],
list[T], tuple[T, ...], set[T], or primitives like int, str, datetime, etc.
default_collection: The default collection to use.
raise_on_validation_error: Whether to raise a DeserializationError if validation fails during reads. Otherwise,
calls will return None if validation fails.
Raises:
TypeError: If pydantic_model is a sequence type other than list (e.g., tuple is not supported).
raise_on_validation_error: Whether to raise a DeserializationError if validation fails during reads.
Otherwise, calls will return None if validation fails.
"""
self._key_value = key_value
self._type_adapter = TypeAdapter[T](pydantic_model)
self._needs_wrapping = self._check_needs_wrapping(pydantic_model)
self._default_collection = default_collection
self._raise_on_validation_error = raise_on_validation_error

def _check_needs_wrapping(self, pydantic_model: Any) -> bool:
"""Determine if this type serializes to a non-dict and needs wrapping.
Types that serialize to dicts (BaseModel, dataclass, TypedDict, dict) don't need wrapping.
All other types (list, tuple, set, primitives, datetime, etc.) need to be wrapped in
{"items": <value>} to ensure consistent dict-based storage.
"""
origin = get_origin(pydantic_model)
self._is_list_model = origin is list

# Validate that if it's a generic type, it must be a list (not tuple, etc.)
if origin is not None and origin is not list:
msg = f"Only list[BaseModel] is supported for sequence types, got {pydantic_model}"
raise TypeError(msg)
# Handle Annotated[T, ...] by unwrapping to T
if origin is Annotated:
return self._check_needs_wrapping(get_args(pydantic_model)[0])

self._type_adapter = TypeAdapter[T](pydantic_model)
self._default_collection = default_collection
self._raise_on_validation_error = raise_on_validation_error
# Check if this type serializes to a dict naturally
return not self._serializes_to_dict(pydantic_model, origin)

def _serializes_to_dict(self, pydantic_model: Any, origin: type | None) -> bool:
"""Check if a type serializes to a dict naturally."""
# No generic origin - simple types like BaseModel, int, datetime, etc.
if origin is None:
if isinstance(pydantic_model, type):
if issubclass(pydantic_model, BaseModel):
return True
if is_dataclass(pydantic_model): # pyright: ignore[reportUnknownArgumentType]
return True
return is_typeddict(pydantic_model) # pyright: ignore[reportUnknownArgumentType]

# dict and Mapping subclasses serialize to dict
if origin is dict:
return True
with contextlib.suppress(TypeError):
if issubclass(origin, Mapping): # pyright: ignore[reportArgumentType]
return True

return False

def _get_model_type_name(self) -> str:
"""Return the model type name for error messages."""
Expand Down
21 changes: 11 additions & 10 deletions key-value/key-value-aio/src/key_value/aio/adapters/pydantic/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ class BasePydanticAdapter(Generic[T], ABC):
"""

_key_value: AsyncKeyValue
_is_list_model: bool
_needs_wrapping: bool
_type_adapter: TypeAdapter[T]
_default_collection: str | None
_raise_on_validation_error: bool
Expand All @@ -41,8 +41,9 @@ def _get_model_type_name(self) -> str:
def _validate_model(self, value: dict[str, Any]) -> T | None:
"""Validate and deserialize a dict into the configured model type.

This method handles both single models and list models. For list models, it expects the value
to contain an "items" key with the list data, following the convention used by `_serialize_model`.
This method handles both dict-serializing types (like BaseModel) and wrapped types
(like list, tuple, primitives). For wrapped types, it expects the value to contain
an "items" key with the data, following the convention used by `_serialize_model`.
If validation fails and `raise_on_validation_error` is False, returns None instead of raising.

Args:
Expand All @@ -55,15 +56,15 @@ def _validate_model(self, value: dict[str, Any]) -> T | None:
DeserializationError: If validation fails and `raise_on_validation_error` is True.
"""
try:
if self._is_list_model:
if self._needs_wrapping:
if "items" not in value:
if self._raise_on_validation_error:
msg = f"Invalid {self._get_model_type_name()} payload: missing 'items' wrapper"
raise DeserializationError(msg)

# Log the missing 'items' wrapper when not raising
logger.error(
"Missing 'items' wrapper for list %s",
"Missing 'items' wrapper for %s",
self._get_model_type_name(),
extra={
"model_type": self._get_model_type_name(),
Expand Down Expand Up @@ -98,10 +99,10 @@ def _validate_model(self, value: dict[str, Any]) -> T | None:
def _serialize_model(self, value: T) -> dict[str, Any]:
"""Serialize a model to a dict for storage.

This method handles both single models and list models. For list models, it wraps the serialized
list in a dict with an "items" key (e.g., {"items": [...]}) to ensure consistent dict-based storage
format across all value types. This wrapping convention is expected by `_validate_model` during
deserialization.
This method handles both dict-serializing types (like BaseModel) and types that serialize
to non-dict values (like list, tuple, primitives). For non-dict types, it wraps the serialized
value in a dict with an "items" key (e.g., {"items": [...]}) to ensure consistent dict-based
storage format. This wrapping convention is expected by `_validate_model` during deserialization.

Args:
value: The model instance to serialize.
Expand All @@ -113,7 +114,7 @@ def _serialize_model(self, value: T) -> dict[str, Any]:
SerializationError: If the model cannot be serialized.
"""
try:
if self._is_list_model:
if self._needs_wrapping:
return {"items": self._type_adapter.dump_python(value, mode="json")}

return self._type_adapter.dump_python(value, mode="json") # pyright: ignore[reportAny]
Expand Down
43 changes: 43 additions & 0 deletions key-value/key-value-aio/tests/adapters/test_pydantic.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,12 @@ async def test_simple_adapter_with_list(self, product_list_adapter: PydanticAdap
assert await product_list_adapter.delete(collection=TEST_COLLECTION, key=TEST_KEY)
assert await product_list_adapter.get(collection=TEST_COLLECTION, key=TEST_KEY) is None

async def test_adapter_with_list_int(self, store: MemoryStore):
adapter = PydanticAdapter[list[int]](key_value=store, pydantic_model=list[int])
await adapter.put(collection=TEST_COLLECTION, key=TEST_KEY, value=[1, 2, 3])
result: list[int] | None = await adapter.get(collection=TEST_COLLECTION, key=TEST_KEY)
assert result == [1, 2, 3]

async def test_simple_adapter_with_validation_error_ignore(
self, user_adapter: PydanticAdapter[User], updated_user_adapter: PydanticAdapter[UpdatedUser]
):
Expand Down Expand Up @@ -217,3 +223,40 @@ async def test_list_validation_error_logging(
assert model_type_from_log_record(record) == "Pydantic model"
error = error_from_log_record(record)
assert "missing 'items' wrapper" in str(error)

async def test_adapter_with_tuple(self, store: MemoryStore):
"""Test that tuple types are supported."""
adapter = PydanticAdapter[tuple[int, str, float]](key_value=store, pydantic_model=tuple[int, str, float])
await adapter.put(collection=TEST_COLLECTION, key=TEST_KEY, value=(1, "hello", 3.14))
result = await adapter.get(collection=TEST_COLLECTION, key=TEST_KEY)
assert result == (1, "hello", 3.14)

async def test_adapter_with_set(self, store: MemoryStore):
"""Test that set types are supported."""
adapter = PydanticAdapter[set[int]](key_value=store, pydantic_model=set[int])
await adapter.put(collection=TEST_COLLECTION, key=TEST_KEY, value={1, 2, 3})
result = await adapter.get(collection=TEST_COLLECTION, key=TEST_KEY)
assert result == {1, 2, 3}

async def test_adapter_with_datetime(self, store: MemoryStore):
"""Test that datetime types are supported."""
adapter = PydanticAdapter[datetime](key_value=store, pydantic_model=datetime)
test_dt = FIXED_CREATED_AT
await adapter.put(collection=TEST_COLLECTION, key=TEST_KEY, value=test_dt)
result = await adapter.get(collection=TEST_COLLECTION, key=TEST_KEY)
assert result == test_dt

async def test_adapter_with_dict(self, store: MemoryStore):
"""Test that dict types are supported and stored without wrapping."""
adapter = PydanticAdapter[dict[str, int]](key_value=store, pydantic_model=dict[str, int])
test_data = {"a": 1, "b": 2, "c": 3}
await adapter.put(collection=TEST_COLLECTION, key=TEST_KEY, value=test_data)
result = await adapter.get(collection=TEST_COLLECTION, key=TEST_KEY)
assert result == test_data

# Verify dict is stored directly (not wrapped)
raw_collection = store._cache.get(TEST_COLLECTION) # pyright: ignore[reportPrivateUsage]
assert raw_collection is not None
raw_entry = raw_collection.get(TEST_KEY)
assert raw_entry is not None
assert raw_entry.value == {"a": 1, "b": 2, "c": 3} # No "items" wrapper
Loading
Loading