Ollama documentation needed for remote URL

[michela]

OllamaLLM remote server configuration should be documented - and this package should provide a method for overriding the base_url in code and not rely on undocumented environment variable OLLAMA_HOST

e.g. a parameter like this for clarity

llm = OllamaLLM(
    base_url='http://<YOUR IP>:11434',   # PROPOSAL - NOT SUPPORTED ATM
    model_name=LLM_MODEL, 
    model_params={
        "max_tokens": 2000,
        "response_format": {"type": "json_object"},
    }
)

Atm the only way to get remote OLLAMA server connection is to set OLLAMA_HOST env variable and this is only apparent if you do the above and you get this helpful error / clue!

  base_url=_parse_host(host or os.getenv('OLLAMA_HOST')),

[stellasia]

Hi @michela ,

The OllamaLLM class uses the ollama Python package and all parameters but the model_name and model_params will be passed to the ollama client. That means you should be able to use:

llm = OllamaLLM(
    host='http://<YOUR IP>:11434',
    model_name=LLM_MODEL, 
    model_params={
        "max_tokens": 2000,
        "response_format": {"type": "json_object"},
    }
)

We can work on making it clearer in the documentation.

[michela]

Ah great! Thanks