Ollama

Ollama is a command-line-first tool for downloading, running, and managing large language models locally, on your own hardware, rather than through a cloud API. It packages open-weight models into a simple format that can be pulled with a single command (ollama pull llama3.2) and run with minimal setup.

Local AIis beyond just chatting with models in the terminal, Ollama exposes a local API server that other applications can connect to — which is what makes it popular as a backend for tools like Obsidian plugins, coding assistants, and custom scripts, rather than a standalone chat app. It is lightweight, scriptable, and easy to automate,

It supports a wide range of quantized open models sized to fit modest hardware, making local, private inference practical even without a high-end GPU — running entirely offline once a model is downloaded, with no data leaving the machine.

Relevance

Ollama is my first choice of tooling to power retrieval-augmented search directly within this vault.It was the original inference engine I learned on, not necessarily better but a comfortable space to work.