Create a light version huggingface partner libs with only hub to Avoid Unnecessary large pytorch Dependency #26027
Replies: 5 comments
-
similare issue in this discussion Refactor Import of HuggingFaceEndpointEmbeddings to Avoid Unnecessary large pytorch Dependency #24482 but different proposed solution |
Beta Was this translation helpful? Give feedback.
-
Completely agree with this suggestion, the core idea is to have an huggingface partner library oriented only for remote inference, meaning that this library will rely mostly on "http clients" that interact with inference servers (the huggingface hub or any custom deployment). This would remove all heavy dependencies. About the naming, it should reflect the "remote inference" aspect, clearly indicate that this module is designed to be used when the inference occurs on dedicated servers, which is mostly the case for most production projects. Maybe This new module could be introduce without making any breaking change, as you suggest :
|
Beta Was this translation helpful? Give feedback.
-
Hi! We are a group of 4 students from the university of Toronto, and we are looking into implementing this shortly! |
Beta Was this translation helpful? Give feedback.
-
Hello! Here is our current proposal. We will implement this soon, and any feedback is apprecaited in the meantime :)Outline of ChangesNew Lightweight Package Creation: Code Duplication and Refactoring: Dependency Minimization: Deprecation Notices: Testing Suite: CI/CD and Packaging Adjustments: Existing ArchitectureThe langchain-huggingface package in the LangChain repository provides integration with Hugging Face models and APIs, allowing users to leverage large language models (LLMs) and embedding functionalities within LangChain workflows. The architecture is structured to support both local and remote inference by interacting with Hugging Face’s Transformers library and API endpoints. Below is an overview of the existing structure and its main components: Core Components: Dual Support for Local and Remote Inference: Dependencies: Files to Modifylibs/partners/huggingface/langchain_huggingface/endpoints.py PseudocodeOutline of changes made to config.py
Outline of changes made to utils.py
Outline of changes made to embeddings.py
Outline of changes made to generation.py
And modify test file as wanted. |
Beta Was this translation helpful? Give feedback.
-
Hello, having the same issue with deploying huggingface_endpoint with docker, is there any walk around for this issue. |
Beta Was this translation helpful? Give feedback.
-
I checked:
Feature Request
Description
When importing
HuggingFaceEndpointEmbeddings
orHuggingFaceEndpoint
fromlangchain_huggingface.embeddings
andlangchain_huggingface.llms
, it is currently necessary to install the entirelangchain-huggingface
package. This package includes the PyTorch library as a dependency, which significantly increases the size of container images by up to 6GB. This is problematic for use cases that only require remote embedding or LLM API access and do not need the PyTorch library.This could be achieved by:
HuggingFaceEndpoint
andHuggingFaceEndpointEmbeddings
classes, containing only the necessary components for remote embedding API access.HuggingFaceEndpoint
andHuggingFaceEndpointEmbeddings
fromlangchain_huggingface.embeddings
andlangchain_huggingface.llms
.Additional Context
This change is particularly important for users who operate in environments with strict resource limitations or those who prioritize lightweight and efficient deployments.
Impact
This change will make the
langchain-huggingface
package more modular and user-friendly, especially for those who rely solely on remote services for embedding tasks.Thank you for considering this proposal. I believe it will greatly enhance the usability and efficiency of the
langchain-huggingface
package.Motivation
Proposal (If applicable)
Create a new lightweight library,
HuggingFaceLight
, containing only theHuggingFaceEndpoint
andHuggingFaceEndpointEmbeddings
classes for communication with the server. This new library should inherit from the LangChain base model so that it can be imported and used without requiring PyTorch or other heavy dependencies.Beta Was this translation helpful? Give feedback.
All reactions