Tools & software

v1.3.0 · Last verified: 25 September 2026

Independence: Independent educational resource. Not affiliated with Meta, Alibaba, Mistral, Google, Microsoft, OpenAI, Anthropic, xAI, NVIDIA, Apple, Ollama, LM Studio, or any model, tool or hardware maker. privatellm.app and other “PrivateLLM” products are unrelated. Accuracy: content is educational and may be incomplete or outdated — verify official docs and live prices before you buy or deploy. See disclaimer and affiliate disclosure.

A curated directory — not an exhaustive app store. Start with a runner, then add UI or RAG when you need them.

Model runners

Ollama

Simplest first path on Mac/Windows/Linux: install, pull a model from the live library, chat in terminal or API.

Recommended first runner. Copy names from ollama.com/library. Local tags stay on disk. :cloud tags and signed-in hosted models send the prompt to Ollama’s cloud (docs.ollama.com/cloud) — same window, different data flow. Leave cloud models alone if privacy is the point.

Official site →

LM Studio

Desktop GUI to browse, download, and chat with local models; good discoverability.

Friendly alternative when you want a GUI first.

Official site →

llama.cpp

High-performance GGUF inference core used under many UIs and tools.

Power users; also the engine behind a lot of the ecosystem.

Official site →

MLX / MLX-LM

Apple Silicon native path (Metal). Optional alongside Ollama on a Mac — not the Windows beginner default.

Use when you are already on a Mac and want a native stack. Windows users should stay on Ollama or LM Studio.

Official site →

Interfaces / UIs

Open WebUI

ChatGPT-like browser UI over local backends (often Ollama).

Great multi-user home-lab face for private models.

RAG / knowledge

AnythingLLM / private RAG apps

Desktop/self-host RAG wrappers that keep documents closer to your box.

Category — pick based on licence, deployment, and threat model.

PrivateGPT-class stacks

Document Q&A pipelines designed for local embeddings + local LLMs.

Expect assembly and maintenance; not zero-ops magic.

Embeddings (nomic-embed class and peers)

A separate small model that turns documents into vectors. RAG needs this plus a generator.

Not a chat model. Download an embedding tag your runner lists, keep files local, then point a RAG app at both.

Serving / multi-user

vLLM / TGI-class servers

Higher-throughput serving for multi-user or API-shaped internal tools.

Business path when one chat window is not enough.

Helpers

Hugging Face Hub

Model cards, weights, and community quant repos.

Always read the model card and licence before production use.

Official site →

GGUF / quantisation tooling

Convert and quantise weights for llama.cpp-class runtimes.

Most beginners should download existing GGUF rather than quantise day one.

Data flow, RAG, and coding tools

Ollama Cloud

Same product family, different data flow. A :cloud tag or signed-in hosted model sends the prompt off the machine. Official docs: docs.ollama.com/cloud. Privacy path: downloaded local tags only.

RAG pattern: a local embedding model, a local generator, and files that stay on disk. Apps in the RAG section wrap that. None of them are zero-ops. Coding agents: a local endpoint (Ollama, LM Studio, or a llama.cpp server) can sit under Cursor, Claude Code, Grok Build, or a workflow on vibecoding.au. That site is how you build; this site is where inference can live. Apple Intelligence is not Ollama.

Install path Choose models