Local (Ollama) Deepseek and Theia AI Chatbot

[TheMatthew]

I tried a local Deepseek (1.5, 7, 8 and 14b), it has a particular structure where it wraps its answer with to give its reasoning before returning the value.

e.g.

Alright, let me figure out how to approach this. The user has been asking about how to handle exceptions in Java and mentioned something about a "page not found" error with the word "ds." I need to consider what each part means.

First, "404 404 page not found" is related to programming errors, specifically for issues when accessing non-existent resources. That doesn't seem directly relevant to handling exceptions in Java or any other topic.

Next, "ds" could stand for multiple things depending on context—like data structures (e.g., DSU stands for Disjoint Set Union), or something shorter like a variable name. Without more context, it's hard to tell what the user is asking about.

I should probably acknowledge that I can't help with a 404 error and then address the exception handling in Java if that's what they're referring to. It makes sense that after explaining exceptions, they might be looking for specific examples or solutions, so providing some code snippets would be useful.

Also, considering the user is likely someone new to programming, offering clear, concise answers with practical examples will help them learn more effectively. So, I'll structure my response to first state that I can't assist with 404 errors and then provide helpful information on exception handling in Java.

It seems like you're trying to access a "page not found" error, which is related to programming errors when accessing non-existent resources. Let me know if there's another topic or question you'd like to address!

This is good to get some additional info on the thought process... but it means when asking simple questions you may incur a 200-400x slowdown since you need 400 tokens to be generated for a simple yes/no answer. I am going to experiment again when the platform is back up, but for now, I would stick to llama, mistral or qwen for code completion.

[JonasHelming]

@TheMatthew Thank you for sharing this! You could experiment with the prompt to tell it to answer in a shorter way!

[xpomul]

I was experimenting with deepseek as well in the past days.
My results haven't been so bad, at least for some prompts.
I have used MFDoom/deepseek-r1-tool-calling:14b which has tool calling capabilities (which are now available in Theia 1.58.0 for ollama, thanks to #14623).

[TheMatthew]

I will try it out! I just wrote an article with the local llm experience I had. https://www.linkedin.com/pulse/comparative-analysis-typescript-code-generation-llm-models-khouzam-uypee/?trackingId=suvlwLkBSNGwyreiweCMbQ%3D%3D

[TheMatthew]

https://www.linkedin.com/pulse/llm-drag-racing-mfdoom-edition-matthew-khouzam-xd3ye/?trackingId=wl9jueHQ6MEMx1SaH5LjEA%3D%3D I just tested MFDoom/deepseek-r1-tool-calling:7/8b. Next up 14.

It is true, that MFDoom doesn't have nearly as much chain of thought though!