feat: Multi-modal payload support, binary-safe serialization, and cross-platform file operations

[acere]

Summary

Addresses #20 by implementing binary-safe serialization for payloads and results containing images, and adds comprehensive multi-modal content support across endpoints. Also supersedes the approach in #21 with a more complete solution.

Related: #7 (serialization standardization)

Changes

Binary-safe image serialization (closes #20)

Payloads and results containing images were being double base64-encoded during serialization, causing significant memory overhead. Serialization now handles binary content natively via LLMeterBytesEncoder and InvocationResponseEncoder, avoiding the redundant encoding.

Multi-modal payload support

create_payload() on BedrockConverse, OpenAI, and SageMaker endpoints now accepts optional images, documents, videos, and audio keyword arguments (as file paths or raw bytes). Format detection uses puremagic (optional multimodal extra) with fallback to file extensions. Each endpoint maps formats to its expected convention automatically (Bedrock short names, OpenAI MIME types).

Text-only payloads are fully backward compatible — no changes needed for existing callers.

UPath standardization

All open() calls replaced with UPath.open() for consistent cross-platform and cloud storage support. Cost model save_to_file() now auto-creates parent directories.

JSON-safe Result stats

Result.to_dict() now converts datetime fields (start_time, end_time) via utc_datetime_serializer, so json.dumps(result.stats) works without a custom serializer.

Testing

• 575 unit tests passing (including property-based tests via Hypothesis)
• New test suites for multi-modal serialization, properties, and per-endpoint behavior
• Integration tests updated with multi-modal payload examples

Closes #20

[athewsey]

I think this str typing of path is wrong as something UPath-like should also be accepted right? Seems like there are multiple inconsistent path typings in our exposed APIs at the moment.

[athewsey]

This check only works when v is a local path - fails with e.g. UPath("s3://...")

[athewsey]

Import not at top level of file, and it's local import so not sure there's a good reason?

There are a couple of other instances of this in other files too

[athewsey]

This approach doesn't give users control over the ordering of the different types of block.

Might it be more flexible and still about-as-usable to instead export utility classes for the different types of content, and have this function take an ordered list of different types of content?

class ImageContent:
    @classmethod
    def from_path(cls, file_path):
        ...

class BedrockBase(Endpoint):
    @staticmethod
    def create_payload(
        user_message: str | list[str | AudioContent | DocumentContent | ImageContent | VideoContent]
        ...
    ):
        ...

BedrockBase.create_payload(
    [
        "What's the title of the graph?,
        ImageContent.from_path("my-cool-graph.png")
    ],
    ...
)

Same feedback applies to other connectors e.g. openai

[athewsey]

Not necessarily a problem to solve for the entire codebase here, but this is mixing absolute & local relative imports (see local dependencies section below).

As discussed I think we prefer to standardize on relative, so let's at least ensure we're not introducing new absolute imports in this PR.

[athewsey]

This can't work correctly because Bedrock endpoints don't want to receive __llmeter_bytes__ wrappers?

[athewsey]

Unless we have a good reason not to, I'd suggest to just expose a (json.dump default-compatible) function interface, like we did before?

1. My guess from StackOverflow/etc is that it's more common for developers to override default than the whole cls in json.dump - so people would be more familiar with that interface.
2. We're not currently using any of the added flexibility of providing a JSONEncoder class versus just a default function.
   1. It seems unlikely that we would want to get in to the complexities of re-implementing JSON {iter}encode from scratch.
   2. If our serialization methods were receiving an encoder object, rather than a class, then at least there'd be the potential benefit of users configuring params like indent only once and re-using the encoder - but if we pass it in as a class then that statefulness is lost and we still have duplication of indent/etc parameters in lots of places around the codebase.
3. We're already providing a function-based interface for decoding with llmeter_bytes_decoder below (instead of e.g. providing a JSONDecoder), so it seems inconsistent

[athewsey]

Should be ensure_path right?

[athewsey]

Should this also be ensure_path?

[athewsey]

Nullable yes, but I'm not sure there's any need for this argument to be optional? You'd never want to call ensure_path().

Unless there's some reason it doesn't work, I'd suggest the cleanest typing would be:

@overload
def ensure_path(path: ReadablePathLike | WritablePathLike) -> UPath: ...


@overload
def ensure_path(path: None) -> None: ...


def ensure_path(path):
    {implementation}

...Which I believe should propagate the fact that output is None if and only if input is None.