Private LLM model families — practical overview
Use this page to orient — then pick a small well-documented model from your runner’s library (Ollama, LM Studio, etc.). Exact model IDs change; we deliberately avoid hard-coding pull tags that go stale.
Landscape snapshot (late September 2026)
Open-weight local use is still a handful of families with frequent point releases. This is a map — not a scored ranking, and not a promise that last month’s “default 27B” is the tag you should pull tonight.
- Qwen 3.8-class ~27B dense is still widely discussed as a practical default on a 24 GB-class GPU or a high unified-memory Mac (Q4-class). Apache-2.0 is commonly reported on that dense card — verify the file you download. Qwen3.8-Flash-Next (late Aug) is an experimental / next-architecture preview, not a beginner default. Giant MoE / “Max” checkpoints are not consumer-GPU toys. There is no Qwen 4 27B as of 25 Sep 2026 — do not invent one.
- Gemma 4 sizes people actually discuss: E2B, E4B, 12B, 26B-A4B MoE, 31B dense. There is no Gemma 4 27B. Multimodal on the family. Licence: read the current card.
- gpt-oss: OpenAI open weights, commonly ~20B-class and ~120B-class MoE. 120B is not a first laptop model. Read the official card. This is not ChatGPT.
- DeepSeek V4-class plus distills. V4.1 Flash was reported around 10 Sep 2026. Full V4 / Pro / giant Flash builds are server-scale. If a runner does not list the checkpoint, do not assume llama.cpp or Ollama can run it yet.
- GLM / Z.ai: GLM-5.3 and Flash variants landed late Aug 2026. Flagships are not mini-PC defaults. Some runners lag new architectures — check the live library before you download.
- Llama 3.x / 4-class stays the tutorial ecosystem. Meta community terms, not “do anything forever.”
- Kimi and other giant MoE weights exist. They are not a first machine. Skip them until you have a server.
- RAG needs a second download: an embedding model (nomic-embed-class or whatever your runner lists) plus a generator. See Tools.
Family comparison (educational)
“Minimum hardware” means rough practical guidance for common quantised deployments — not a guarantee your exact file will fit or feel fast.
| Family | Size hint | Often used for | Hardware orientation | Licence (verify) | Notes |
|---|---|---|---|---|---|
| Qwen 3.x-class (3.5 / 3.6 / 3.8 and siblings) | Small dense (few-B) through ~27B dense and large MoE; exact IDs change by month | General chat, multilingual, coding, and multimodal experiments on consumer / workstation hardware | Small tags: laptop/16 GB class. ~27B Q4-class: often discussed around a 24 GB GPU or high unified-memory Mac. Giant MoE: server class. | Official dense checkpoints are often Apache-2.0 — still verify per file. Some large MoE / hosted variants use different Qwen terms. | As of late September 2026, Qwen3.8 ~27B is still the practical dense local default people discuss. Flash-Next is a preview, not night-one. Copy the live library tag. There is no Qwen 4 27B to pull. |
| Gemma 4-class | E2B / E4B, 12B, 26B-A4B MoE, 31B dense — not a 27B | Documented Google open-weight line; tiny first chats through single-GPU assistants; multimodal on the family | E2B/E4B and 12B-class for modest machines; 31B wants a serious single GPU or unified memory | Verify the current Gemma / Apache wording on the exact card — terms have changed between generations | Gemma 4 is the line people mean in late 2026. Still not a frontier cloud substitute. No Gemma 4 27B. |
| gpt-oss (OpenAI open weights) | Commonly discussed ~20B and ~120B-class open checkpoints (not ChatGPT Team) | People who want OpenAI-shaped open weights on their own box; reasoning-style local experiments | ~20B-class often discussed for 16 GB-class machines when quantised; 120B-class is a much larger step | Read the official gpt-oss card (Apache-style reporting is common — still verify) | This is open weights you download, not the ChatGPT product. Independence: we are not affiliated with OpenAI. |
| Llama 3.x / 4-class | Small (~8B-class) through 70B-class and newer Llama 4 variants; ecosystem still large | Tutorials, fine-tunes, tooling examples, and teams already standardised on Llama recipes | Small quants: consumer GPU or strong laptop. 70B-class: high VRAM or slow offload. | Llama community / Meta terms — not “do anything forever”; read current Meta terms for your use | Ecosystem familiarity remains high. Peak quality vs Qwen/Gemma/gpt-oss on a given date is a moving target. |
| Mistral / Ministral / Small-class | Ministral few-B through Small / larger MoE open drops | Snappy general use and Apache-friendly open drops when a local build exists | Small Ministral-class: modest hardware. Larger Small/MoE: check the card. | Many open Mistral drops use permissive licences — still verify per file | Licence cleanliness is often the business reason to look here; capability still depends on the checkpoint. |
| Phi-4 / compact Microsoft line | Compact / mid-small (exact sizes vary; Mini vs larger Phi-4-class) | Edge, education, and first experiments when you want a small footprint | Often aimed at more modest hardware than flagship open models — still verify the specific build | Check Microsoft’s licence for the exact release | Compact does not mean “matches frontier cloud reasoning.” |
| DeepSeek V4-class + distills | Full V4 MoE is server-scale; distill / flash / smaller checkpoints exist for local trials | Reasoning-style and coding experiments when you pick a size that actually fits | Start with the smallest official or well-supported distill your runner lists — not the full flagship | Check DeepSeek and host-repo licences for the exact files (MIT-style reporting is common on some drops) | Full V4 / Pro is not a 24 GB GPU toy. V4.1 Flash was reported mid-September 2026 — confirm the runner actually loads that arch before you treat it as a tonight pull. Distills and smaller Flash tags are the local path. |
| GLM / Z.ai open weights | Large MoE flagships plus smaller / flash variants that show up in local libraries | People following open-weight bench chatter; local use only via smaller/flash quants | Flagship GLM sizes are not consumer-GPU default. Prefer flash/small tags if you stay local. | Often reported MIT on some open GLM drops — verify the card | GLM-5.3 and Flash variants were discussed from late August 2026. Strong public scores on large checkpoints do not mean your mini-PC can host them, or that every GUI supports the arch on day one. |
| Coding specialists (Qwen Coder, DeepSeek Coder, Devstral-class, etc.) | Often 20–35B-class local coding builds; exact IDs churn | Local coding assistance on private repos when privacy matters more than peak cloud coding agents | Comfortable local coding usually needs more RAM/VRAM than toy chat demos | Varies by family — check each card | Pair with application-building guidance on vibecoding.au. Local coding help is a privacy win, not a guarantee of correct code. CodeLlama-class is legacy relative to current Qwen Coder / similar lines. |
How to read this page
- Prefer families with public weights and documented licences — still read the card for the file you download.
- Orient guidance to hobbyist and SMB hardware in Australia, not datacentre clusters.
- Treat VRAM/RAM notes as approximate learning bands for common quantised deployments, not lab certificates.
- Favour ecosystem support (popular runners/UIs) over paper-only releases.
- State limits plainly: private/open models often lag frontier cloud products on hard reasoning and multimodal polish.
- Re-check monthly/quarterly — names, sizes, and licences move fast. This table is a map, not a scoreboard.
Short family notes
Qwen 3.x-class (3.5 / 3.6 / 3.8 and siblings)
Often used for: General chat, multilingual, coding, and multimodal experiments on consumer / workstation hardware. Hardware orientation: Small tags: laptop/16 GB class. ~27B Q4-class: often discussed around a 24 GB GPU or high unified-memory Mac. Giant MoE: server class.. Licence: Official dense checkpoints are often Apache-2.0 — still verify per file. Some large MoE / hosted variants use different Qwen terms.. As of late September 2026, Qwen3.8 ~27B is still the practical dense local default people discuss. Flash-Next is a preview, not night-one. Copy the live library tag. There is no Qwen 4 27B to pull.
Gemma 4-class
Often used for: Documented Google open-weight line; tiny first chats through single-GPU assistants; multimodal on the family. Hardware orientation: E2B/E4B and 12B-class for modest machines; 31B wants a serious single GPU or unified memory. Licence: Verify the current Gemma / Apache wording on the exact card — terms have changed between generations. Gemma 4 is the line people mean in late 2026. Still not a frontier cloud substitute. No Gemma 4 27B.
gpt-oss (OpenAI open weights)
Often used for: People who want OpenAI-shaped open weights on their own box; reasoning-style local experiments. Hardware orientation: ~20B-class often discussed for 16 GB-class machines when quantised; 120B-class is a much larger step. Licence: Read the official gpt-oss card (Apache-style reporting is common — still verify). This is open weights you download, not the ChatGPT product. Independence: we are not affiliated with OpenAI.
Llama 3.x / 4-class
Often used for: Tutorials, fine-tunes, tooling examples, and teams already standardised on Llama recipes. Hardware orientation: Small quants: consumer GPU or strong laptop. 70B-class: high VRAM or slow offload.. Licence: Llama community / Meta terms — not “do anything forever”; read current Meta terms for your use. Ecosystem familiarity remains high. Peak quality vs Qwen/Gemma/gpt-oss on a given date is a moving target.
Mistral / Ministral / Small-class
Often used for: Snappy general use and Apache-friendly open drops when a local build exists. Hardware orientation: Small Ministral-class: modest hardware. Larger Small/MoE: check the card.. Licence: Many open Mistral drops use permissive licences — still verify per file. Licence cleanliness is often the business reason to look here; capability still depends on the checkpoint.
Phi-4 / compact Microsoft line
Often used for: Edge, education, and first experiments when you want a small footprint. Hardware orientation: Often aimed at more modest hardware than flagship open models — still verify the specific build. Licence: Check Microsoft’s licence for the exact release. Compact does not mean “matches frontier cloud reasoning.”
DeepSeek V4-class + distills
Often used for: Reasoning-style and coding experiments when you pick a size that actually fits. Hardware orientation: Start with the smallest official or well-supported distill your runner lists — not the full flagship. Licence: Check DeepSeek and host-repo licences for the exact files (MIT-style reporting is common on some drops). Full V4 / Pro is not a 24 GB GPU toy. V4.1 Flash was reported mid-September 2026 — confirm the runner actually loads that arch before you treat it as a tonight pull. Distills and smaller Flash tags are the local path.
GLM / Z.ai open weights
Often used for: People following open-weight bench chatter; local use only via smaller/flash quants. Hardware orientation: Flagship GLM sizes are not consumer-GPU default. Prefer flash/small tags if you stay local.. Licence: Often reported MIT on some open GLM drops — verify the card. GLM-5.3 and Flash variants were discussed from late August 2026. Strong public scores on large checkpoints do not mean your mini-PC can host them, or that every GUI supports the arch on day one.
Coding specialists (Qwen Coder, DeepSeek Coder, Devstral-class, etc.)
Often used for: Local coding assistance on private repos when privacy matters more than peak cloud coding agents. Hardware orientation: Comfortable local coding usually needs more RAM/VRAM than toy chat demos. Licence: Varies by family — check each card. Pair with application-building guidance on vibecoding.au. Local coding help is a privacy win, not a guarantee of correct code. CodeLlama-class is legacy relative to current Qwen Coder / similar lines.