Run LLMs Locally: AI Tools and Open-Weight Models That Keep Data on Your Machine
Local and self-hosted tools like Ollama, LM Studio, Open WebUI, and Dify, plus open-weight models such as Qwen, DeepSeek, and GLM—with notes on hardware requirements, licenses, and the limits of multi-machine inference.
In September 2026, NVIDIA open-sourced PAIR, which pools several home computers for inference; Perplexity open-sourced its local inference engine Lily; and GLM-5.3-Flash released its weights under MIT. Running models on your own machine has gone from a hobby to a practical choice for privacy and cost. The upside is that data stays on the device and there's no per-call bill; the catch is that model size is capped by your VRAM and memory. This collection layers the tools as runtime → interface → application, then lists open-weight models you can download.
Who is this for?
Recommended tools (8)
OllamaNew
Free and unlimited when run locally; tools like Open WebUI and OpenCode connect to it directly
LM StudioNew
Local server mode exposes an API for your code; free for personal use, closed source
Open WebUINew
Open-source and self-hosted with multi-user permissions; deployment needs Docker and basic ops
DifyNew
Turns local models into internal apps with knowledge bases and workflows; community edition is MIT-licensed
QwenNew
Covers text, code, and multimodal; weights are free to download and self-hosting costs only compute
DeepSeek
Noted for Chinese and coding ability, with a native 1M-token context; try it free on the web first
Zhipu AINew
GLM-5.3-Flash is released under MIT: 320B total parameters, 18B active, natively multimodal
OpenCode
MIT-licensed and can run local models through Ollama, so code doesn't have to leave your machine
How to choose a local deployment setup
| Tool / model | Type | Learning curve | Open source | Best for |
|---|---|---|---|---|
| Ollama | Runtime | Medium (CLI) | Yes | Running models locally and serving other programs |
| LM Studio | Desktop client | Low | No (free for personal use) | Getting a model running to see results |
| Open WebUI | Web interface | Medium (needs Docker) | Yes | Shared use on a team intranet |
| Dify | App platform | Medium | Community edition | Knowledge-base Q&A and internal workflows |
| Qwen / DeepSeek / GLM | Open-weight models | Depends on hardware | Open weights, licenses vary | Pick a size by license and VRAM |
FAQ
- Q: What hardware do I need to run an LLM locally?
- A: It depends on model size and quantization, so there's no single answer: smaller models run on an ordinary laptop, while large ones need lots of VRAM or memory. One concrete reference point: Perplexity's local Windows agent requires at least 24GB of VRAM, and its Mac hybrid compute requires at least 24GB of unified memory. Start with a small model in LM Studio or Ollama, then decide from the speed whether to change models or upgrade hardware.
- Q: Ollama or LM Studio?
- A: If you're comfortable with the command line and want to serve other programs, pick open-source Ollama; if you'd rather download and switch models through a graphical interface, pick LM Studio, which is free for personal use but closed source. Both can run a local API server, and for team use you can put Open WebUI in front of Ollama.
- Q: I have several computers at home. Can I combine them to run a bigger model?
- A: Current open-source options can't pool VRAM. NVIDIA's open-source PAIR (Personal AI Router, beta), released in September 2026, links machines running Ollama or LM Studio into one inference pool with Ollama- and OpenAI-compatible endpoints, but it routes requests—a model too big for one machine is still too big when they're linked.
- Q: Can open-weight models be used commercially?
- A: It depends on each license. GLM-5.3-Flash uses MIT and IFM's K2 Horizon family uses Apache-2.0, permissive licenses that allow commercial use; other models use custom licenses that restrict use cases or user counts. Check the license on each model card before deploying.