[Inference Providers] add image-to-image support for Replicate provider
#3188
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR introduces image-to-image support for the Replicate provider in Python by adding a new task class and updating both client and testing functionality.
- Added the ReplicateImageToImageTask class with custom payload preparation for image inputs.
- Extended tests to cover the image-to-image payload, including model version extraction.
- Updated client and documentation files to integrate and reflect the new feature.
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_inference_providers.py | Added tests to check the payload formation for image-to-image tasks. |
| src/huggingface_hub/inference/_providers/replicate.py | Introduced ReplicateImageToImageTask and corresponding payload logic. |
| src/huggingface_hub/inference/_providers/init.py | Updated imports and mapping for the new task. |
| src/huggingface_hub/inference/_generated/_async_client.py | Updated client to correctly process image-to-image responses. |
| src/huggingface_hub/inference/_client.py | Updated client to correctly process image-to-image responses. |
| docs/source/en/guides/inference.md | Revised documentation to reflect the new image-to-image support. |
Comments suppressed due to low confidence (1)
tests/test_inference_providers.py:1064
- Consider adding additional tests for other supported input types (e.g., file paths and BinaryIO) as well as tests for unsupported input types to ensure robust coverage of the _prepare_payload_as_dict behavior.
def test_image_to_image_payload(self):
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
This looks good to me, @hanouticelina. Thanks for implementing it!
💚 🐍 💚
| image_bytes: bytes | ||
| if isinstance(inputs, (str, Path)): | ||
| with open(inputs, "rb") as f: | ||
| image_bytes = f.read() | ||
| elif isinstance(inputs, bytes): | ||
| image_bytes = inputs | ||
| elif isinstance(inputs, BinaryIO): | ||
| image_bytes = inputs.read() | ||
| else: | ||
| raise TypeError(f"Unsupported input type for image: {type(inputs)}") |
There was a problem hiding this comment.
could use _open_as_binary here
There was a problem hiding this comment.
or even better def _b64_encode(content: ContentT) -> str:
There was a problem hiding this comment.
we could define a def _as_url(content: ContentT) -> str: helper that takes as input any str/url/path/binary and return either base64-url or a plain url. With a centralized logic we could add some logic to infer the mime-type (see image/jpeg below) if it really become necessary.
(a centralized helper would also take as input PIL.Image object when #3191 is addressed)
| def _prepare_payload_as_dict( | ||
| self, inputs: Any, parameters: Dict, provider_mapping_info: InferenceProviderMapping | ||
| ) -> Optional[Dict]: | ||
| image_url = _as_url(inputs, default_mime_type="image/jpeg") |
There was a problem hiding this comment.
nice having a default one 👍
Co-authored-by: Lucain <[email protected]>
|
Thanks for implementing this @hanouticelina! 🎉 |
…ider (huggingface#3188) * add image-to-image support for replicate * fixes * infer mime type * Update src/huggingface_hub/inference/_common.py Co-authored-by: Lucain <[email protected]> * Update src/huggingface_hub/inference/_common.py * fix --------- Co-authored-by: Lucain <[email protected]>
Same PR as: huggingface/huggingface.js#1564 but in Python.
Tested the PR with https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev with both local and URL images, it works as expected ✅
cc @zeke if you have some spare time to take a look at this PR 🤗