Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace onnx with ggml #22

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Replace onnx with ggml #22

wants to merge 4 commits into from

Conversation

abb128
Copy link
Owner

@abb128 abb128 commented Jan 27, 2025 •
edited
Loading

TODO:

  • don't leak memory (ggml context, allocated tensors)
  • fix other code todos
  • convert existing models to gguf
  • script/docs to convert models to gguf
  • session flushing
  • test bindings and live captions
  • speed benchmarking and comparison
  • re-add flakes
  • investigate live captions higher cpu usage (seems related to thread count)
  • possibly support vulkan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@abb128