Guide · 2026-08-14

Local LLM for Game Devs — NPCs and Procedural Storytelling

Last updated 2026-08-14

The models, VRAM budgets, and architecture that actually ship real-time NPC dialogue and procedural stories offline in 2026.

By Mohamed Meguedmi · 9 min read

Why run the model locally at all?

Cloud APIs are seductive for a prototype and a disaster in a shipped game. Every conversational NPC turn is a network round-trip you cannot control: it fails on a train, it adds per-token cost to a title you sold once for $20, and it hands your players' free-text chatter to a third party. For a game — an offline-first, latency-sensitive, cost-fixed product — local inference is the default, not the fallback.

The economics are stark. A modestly chatty RPG might generate 300–800 tokens per NPC exchange. At cloud rates that is fractions of a cent per turn, but multiplied across a 40-hour playthrough and thousands of players it becomes a recurring liability against a one-time purchase. A quantized 4B model running on the player's own GPU costs you nothing after ship. Run the numbers for your own turn budget in our cost calculator before you commit to an architecture.

The catch is that game dev has two very different LLM jobs, and conflating them is the most common mistake we see.

Two jobs: real-time dialogue vs. offline authoring

Real-time NPC dialogue runs inside the frame loop. The player says something, the shopkeeper answers now. Here latency dominates every other concern — a 900 ms pause breaks immersion harder than a slightly generic line. You want the smallest model that stays in character.

Procedural storytelling / authoring runs between frames, at a loading screen, or when a region first generates. The "game master" writes a side quest, seeds a rumor, or drafts a book found on a shelf. A 2–4 second generation is invisible here, so you can afford a larger, more coherent model.

Design rule: pick two models, not one. A fast 3B–4B for live dialogue, a slower 8B–12B for authoring. They can share the same runtime and be swapped by VRAM budget.

Model recommendations and benchmarks

The figures below are indicative throughput from our standardized harness (single request, greedy-ish sampling, 512-token generation) on consumer GPUs. Your mileage shifts with context length and sampler settings, but the ordering is stable. Full methodology lives on our benchmarks page, and the raw numbers are queryable through the BestLLMfor public API (CC BY 4.0) and the open-source MCP server if you want to script your own comparison.

Model (quant)	VRAM	Tok/s (RTX 4060 8GB)	Best job	Why
Llama 3.2 3B Instruct Q4_K_M	~2.5 GB	110–130	Live dialogue, barks	Fastest coherent option; tight instruction-following for short lines
Qwen3 4B Instruct Q4_K_M	~2.8 GB	85–105	Live dialogue, multilingual NPCs	Strong persona consistency, excellent non-English coverage
Gemma 3 4B Instruct Q4_K_M	~3.2 GB	75–95	Live dialogue with vision hooks	Good tone control; useful if NPCs react to on-screen imagery
Qwen3 8B Q4_K_M	~5.5 GB	45–60	Quest logic, branching	Better reasoning for multi-step quest infill
Mistral Nemo 12B Q4_K_M	~7.5 GB	30–42	Procedural storytelling	128K context, prose quality holds over long lore dumps

Our verdict for the live layer: Qwen3 4B Instruct is the best all-rounder for shipped NPCs because its persona adherence at 4B is unusually robust, and it degrades gracefully at Q4. If your title is English-only and you want maximum frames, Llama 3.2 3B is the speed pick. See the model cards for Llama 3.2 3B Instruct and Qwen3 4B Instruct for context lengths and license terms, and the Qwen3 library page for one-command pulls. Browse the full field in our catalog.

Hardware budgets that actually ship

Your minimum spec is the model your players can run, not what you author on. Plan for a two-model layout: a resident dialogue model plus an authoring model loaded on demand.

GPU tier	VRAM	Dialogue model	Authoring model	Notes
Entry (GTX 1660, RTX 3050)	6 GB	Llama 3.2 3B Q4	Same, larger prompts	Single-model only; keep context < 4K
Mainstream (RTX 4060, 3060 12GB)	8–12 GB	Qwen3 4B Q4	Qwen3 8B / Nemo 12B (swapped)	Sweet spot for indie ship targets
Enthusiast (RTX 4070 Ti, 4080)	16 GB	Qwen3 4B Q4 resident	Nemo 12B resident	Both models loaded simultaneously
CPU-only fallback	—	Llama 3.2 3B Q4	—	15–30 tok/s; acceptable for turn-based only

Two practical notes. First, quantization is your friend: Q4_K_M costs roughly 2–4 quality points versus FP16 on dialogue tasks while roughly halving VRAM — an easy trade for NPCs. Second, keep a CPU-only path. Not every player has a discrete GPU, and a turn-based game can tolerate 20 tok/s where an action title cannot.

Architecture: let the state machine drive

The single highest-leverage decision is refusing to let the LLM hold authoritative state. The pattern that ships is state-machine-augmented dialogue: your game logic owns the facts (who has what quest, what the shopkeeper's inventory is, whether the bridge is burned), and the model receives those facts as structured context and returns only phrasing.

# Pseudocode: the model phrases, it never decides
context = {
  "npc": "Bram the blacksmith",
  "mood": mood_fsm.current(),        # game owns this
  "known_facts": world.facts_for(npc),
  "player_line": sanitize(player_input),
}
line = llm.generate(persona_prompt, context, max_tokens=80)
# Quest state changes happen in code, gated on intent detection —
# never because the model "said so".

For procedural storytelling, use LLM in-fill: your generator lays down the skeleton (quest type, giver, reward tier, location) deterministically for balance and reproducibility, then the model fills prose into the gaps. This keeps quests solvable and rewards fair while making every telling feel handwritten. It also makes saves deterministic, because the structural seed is stored, not the generated text.

Integrating with Unity and Unreal

You do not embed model weights in your engine. You run a local inference server and talk to it over HTTP or a native binding.

The install path (Ollama)

1. Install the runtime. Download Ollama for the target OS and confirm it serves on localhost:11434.
2. Pull your two models. ollama pull qwen3:4b for dialogue and, say, ollama pull mistral-nemo:12b for authoring.
3. Warm the model at load. Send one throwaway request during your loading screen so the first real NPC line isn't cold-start slow.
4. Call from the engine. In Unity, hit the local endpoint with UnityWebRequest and stream tokens into a typewriter effect. In Unreal, use an HTTP module request or a community LLM plugin.
5. Stream, always. Streaming tokens to a typewriter UI hides latency — the player reads at ~200 wpm, so first-token time is what they feel, not total generation time.

Ship the runtime alongside your game or detect and prompt to install it on first run. Bundle the specific model tags you tested against; do not let players' latest drift silently under you.

Treat every player keystroke as hostile

The moment a player can type free text at an NPC, you have a prompt-injection surface. A player will try to make your paladin recite your system prompt, break character, or emit slurs your storefront will ban you for. Defend in layers: keep the system/persona prompt structurally separate from user text, wrap player input in clear delimiters, cap generation length, and run a cheap output filter before display. The broad landscape of these attacks is well catalogued in the OWASP Top 10 for LLM Applications — read it before you ship a talk-to-anything NPC. Because you own game state in code rather than in the model, a successful injection can make an NPC say something dumb, but it cannot grant the player a legendary sword.

The verdict

Ship two models. Use a fast 3B–4B for live dialogue and a 8B–12B for offline authoring, keep authoritative state in a deterministic layer, stream tokens to hide latency, and sanitize player input. Do that and a $300 mainstream GPU delivers NPCs that feel alive without a cloud bill or a privacy footnote.

Use case	Pick	Latency budget	Why it wins
Real-time NPC dialogue (multilingual)	Qwen3 4B Instruct Q4_K_M	< 150 ms first token	Best persona consistency at size; strong non-English
Real-time dialogue (English, max FPS)	Llama 3.2 3B Instruct Q4_K_M	< 120 ms first token	Fastest coherent 3B
Quest logic / branching infill	Qwen3 8B Q4_K_M	1–2 s	Reasoning headroom for multi-step quests
Procedural storytelling / lore prose	Mistral Nemo 12B Q4_K_M	2–4 s	128K context, prose quality over long lore
CPU-only / no discrete GPU	Llama 3.2 3B Q4_K_M	Turn-based only	Runnable everywhere at 15–30 tok/s