Feature/realtimetalkingmvp - #9
Open
Deltaidiots wants to merge 6 commits into
Open
Conversation
…nimation, and enhance audio processing utilities - Added `tts.py` for a new TTS service implementation using the Piper TTS library, including error handling and logging. - Created `index.html` for a web interface that supports text-to-speech and audio file uploads for animation. - Enhanced `utils.py` with detailed type annotations, docstrings, and added a new function for real-time audio feature extraction. - Improved audio preprocessing and feature extraction methods to support real-time applications.
…face enhancement config
…eatures, and improve argument help descriptions
…cript for clarity
… flow - Introduced architecture.md detailing high-level overview, model architecture, and data flow. - Created data-flow.md outlining training and inference data pipelines. - Added README.md for diagrams directory with viewing instructions and diagram index. - Implemented various PlantUML diagrams including API architecture, audio processing, inference flow, and training pipeline. - Developed model-details.md to explain U-Net architecture and audio processing pipeline. - Compiled training-guide.md with prerequisites, data preparation, training configuration, and best practices.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a significant upgrade to the SyncTalk_2D project, adding a real-time web API and modularizing the codebase for maintainability and extensibility. The changes include a new FastAPI server, a centralized configuration system, enhanced documentation, and improvements to data processing and debugging features.
Major new features and architecture:
api.py) that supports real-time talking head animation via both text-to-speech and audio file upload, streaming results frame-by-frame. Includes endpoints for animation, avatar retrieval, health checks, TTS audio generation, and debug controls.README.md.Configuration and code organization:
config.py), organizing model, TTS, and debug settings for easier management and validation.README.md).Documentation and best practices:
README.mdto cover the new API, configuration, code structure, and best practices. Added references to detailed docs and PlantUML diagrams for visualizing system architecture and data flow.Data processing improvements:
data_utils/process.py) to avoid redundant work by checking for existing frames before extracting, and to robustly create output directories. [1] [2]