You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This discussion is a follow up from #936 , that deviated from OP to talk about llama-server internal tools...
@aittalam wrotes:
Note that while tool calling is supported by llama.cpp's server, this feature is not taken advantage of in the TUI (which is why I provided the python code to test it). What the TUI should do is implement an agentic loop close to what I sketched above in python:
get the available tools from the llama.cpp server
add those tools' descriptions to the request you are sending to the server, together with your prompt
for any tool call the server returns you, execute it via the server's /tools endpoint (see the python code above) then return the output to the LLM
@devingfx wrotes:
But again, the whole point was to be able to quick test tools with random models using TUI and/or WebUI without having to code something :/
The reason of existence of this llamafile project is easyness, no config, no code right? So it would be cool to have tools working in TUI/WebUI, right ?
I understand though that this internal tools from llama-server.cpp are new features.
Are the TUI / WebUI specific to llamafile? Or are they external (pluggable) projects ?
What is the complexity to make internal tools working in these quick UIs ?
( My guess is that llama-server team would eventualy add some more basic tools, so getting them to work out of the box would be awesome ! )
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
This discussion is a follow up from #936 , that deviated from OP to talk about llama-server internal tools...
The reason of existence of this llamafile project is easyness, no config, no code right? So it would be cool to have tools working in TUI/WebUI, right ?
I understand though that this internal tools from llama-server.cpp are new features.
Are the TUI / WebUI specific to llamafile? Or are they external (pluggable) projects ?
What is the complexity to make internal tools working in these quick UIs ?
( My guess is that llama-server team would eventualy add some more basic tools, so getting them to work out of the box would be awesome ! )
All reactions