Guide · 2026-05-16

SillyTavern + Local LLM — Best Setup for Role-Play

The editorial team's tested 2026 stack for private, uncensored, low-latency role-play on consumer GPUs — no cloud, no logging, no compromises.

By Mohamed Meguedmi · 11 min read

Why this stack, in 2026

Role-play with local LLMs has matured fast. The 2024-era recipe of "Mythomax 13B + Oobabooga" is dead — modern 24B Mistral-Small derivatives now outperform the old 70B Llama 2 finetunes on character consistency, while running comfortably on a single RTX 4090 or even a 3090. SillyTavern itself reached version 1.13 in March 2026, adding native support for XTC and DRY samplers, MCP tool routing, and a rewritten group-chat orchestrator.

The bottleneck is no longer the model. It is the backend you plug into SillyTavern, the quantization you pick, and the sampler preset you load. Get those three right and a $400 used 3090 produces output that is genuinely indistinguishable from Claude Sonnet 4.6 for most fiction prompts — without sending a single token off the machine.

This guide is the editorial team's tested recipe, drawn from 600+ hours of role-play benchmarking across 14 GGUF and 9 EXL3 models in Q1–Q2 2026. Full methodology lives at /methodology/.

1. Pick the right backend (this matters more than the model)

SillyTavern is only a front-end. It does not run the model. Your real choice is which inference server lives behind it. Four candidates dominated 2025–2026 discussion threads on r/SillyTavernAI; here is how they compare on the criteria that actually affect RP quality.

Backend	Format	Samplers exposed	Context cap (default)	Tok/s on RTX 4090, 24B Q4_K_M	RP verdict
KoboldCpp 1.79	GGUF	All (XTC, DRY, smooth, mirostat, min-p)	User-set, no cap	42 t/s	Recommended. Best sampler control, single binary, no Python.
TabbyAPI 0.3	EXL3	All major + tensor-parallel	User-set	61 t/s	Fastest if you can fit fully in VRAM. Slight setup pain.
llama.cpp server	GGUF	Most (DRY added v0.4, XTC partial)	4096	40 t/s	Fine, but you re-implement what KoboldCpp gives for free.
Ollama 0.6	GGUF	Limited (no XTC, no DRY)	4096 (silent truncation)	38 t/s	Avoid for RP. Re-quantizes KV cache, breaks long context.

Ollama's convenience trade-off is the wrong one for role-play. The 4K default context silently drops your character card, world-info entries and chat history mid-scene — and the missing samplers mean repetition penalty is your only tool against the dreaded "shivers down her spine" loop. KoboldCpp solves both problems out of the box.

2. Hardware reality check

VRAM is the only number that matters. CPU and system RAM are near-irrelevant once the model fits in GPU memory. The table below maps real consumer GPUs to the largest model you can run at usable quality (Q4_K_M or better, 16K context, no KV offload).

GPU	VRAM	Largest viable model	Quant	Context (FP16 KV)	Used price (US, May 2026)
RTX 3060	12 GB	Lyra-Gutenberg-Nemo-12B	Q4_K_M	12K	$190
RTX 4070 Ti Super	16 GB	Cydonia-24B-v2.1	IQ3_M	16K	$520
RTX 3090 / 4090	24 GB	Magnum-Diamond-24B	Q4_K_M	32K	$650 / $1,400
RTX 5090	32 GB	Magnum-Diamond-24B	Q6_K	64K	$1,950
2× RTX 3090	48 GB	Behemoth-X-123B	IQ3_XXS	16K	$1,300
Apple M3 Max 64 GB	48 GB usable	Magnum-Diamond-72B	Q4_K_M	16K	$3,200 (used MBP)

The sweet spot in 2026 is unambiguous: a used RTX 3090 for $650. It runs the current best-in-class 24B RP model at full quality with 32K context, draws 350 W under load, and costs less than two months of a high-tier cloud RP subscription.

3. The ranked model shortlist

The team ran each candidate through a fixed 12-card battery (3 SFW slice-of-life, 3 long-form fantasy, 3 dark/morally complex, 3 multi-character group chats), scoring on character voice retention across 200 turns, prose variety (distinct-2 n-grams), and refusal rate. Scores below are mean of three runs at temperature 1.0, min-p 0.05, DRY 0.8.

#	Model	Params	Voice retention	Prose variety	Refusals	Best for
1	MS3.2-24B-Magnum-Diamond	24B	92%	0.71	0%	All-round flagship
2	Broken-Tutu-24B-Transgression-v2.0	24B	88%	0.69	0%	Darker themes, horror
3	TheDrummer/Cydonia-24B-v2.1	24B	89%	0.66	1%	16 GB GPUs (IQ3_M)
4	Lyra-Gutenberg-Mistral-Nemo-12B	12B	81%	0.63	2%	RTX 3060 / laptops
5	Behemoth-X-123B	123B	94%	0.74	0%	Dual-3090 enthusiasts

Magnum-Diamond wins the overall verdict because it nails the three failure modes most RP finetunes fall into: it stays in character under sampling pressure, it does not collapse into purple prose after turn 50, and it never refuses in-fiction violence or romance. It is the team's default recommendation for anyone with at least 20 GB of VRAM.

4. Step-by-step install (KoboldCpp + SillyTavern)

Total time on a clean Windows or Linux machine: about 15 minutes, assuming the model download has finished.

1. Download KoboldCpp. Grab the latest koboldcpp_cu12.exe (Windows) or koboldcpp-linux-x64-cuda1210 from the GitHub releases page. Single binary, no installer.
2. Download the model. Pull MS3.2-24B-Magnum-Diamond-Q4_K_M.gguf (~14.3 GB) from its HuggingFace repo. Use huggingface-cli download Doctor-Shotgun/MS3.2-24B-Magnum-Diamond-GGUF MS3.2-24B-Magnum-Diamond-Q4_K_M.gguf --local-dir ./models.
3. Launch KoboldCpp. Run ./koboldcpp --model ./models/MS3.2-24B-Magnum-Diamond-Q4_K_M.gguf --contextsize 32768 --gpulayers 99 --flashattention --port 5001. With a 24 GB GPU all 99 requested layers will offload; you should see ~22 GB VRAM in use.
4. Install SillyTavern. git clone https://github.com/SillyTavern/SillyTavern -b release, then run start.sh (Linux/macOS) or start.bat (Windows). It opens on http://127.0.0.1:8000.
5. Connect. In SillyTavern, choose API → Text Completion → KoboldCpp, URL http://127.0.0.1:5001, click Connect. The model name auto-populates.
6. Load the right preset. Import the official Mistral-V7-Tekken context template and set samplers: temperature 1.0, min-p 0.05, DRY multiplier 0.8, XTC threshold 0.1, repetition penalty 1.0 (off). This combination is what produced the benchmark scores in the table above.

5. Sampler recipe — the difference between great and unreadable

Default samplers from 2023 (high repetition penalty, temperature 0.7) actively damage modern 24B models. The 2026 consensus, validated across the text-generation-webui parameters wiki and our own A/B testing, is to let the model breathe and use DRY for anti-repetition instead.

• Temperature: 1.0 (Magnum-Diamond), 1.1 (Cydonia), 0.9 (Behemoth-123B).
• Min-p: 0.05. Acts as a quality floor — disables the bad tail without flattening creativity.
• DRY: multiplier 0.8, base 1.75, allowed length 2. Kills phrase loops without nuking valid repetition.
• XTC: threshold 0.1, probability 0.5. Removes the most-likely top tokens half the time, which is what gives modern RP its prose variety.
• Repetition penalty: off (1.0). With DRY active, classic rep pen degrades grammar.
• Disable Top-K, Top-P, Mirostat, TFS. They fight DRY/XTC.

6. Memory: lorebooks beat bigger context

Doubling context from 16K to 32K halves your tok/s and rarely improves long-session coherence. The better tool is SillyTavern's World Info (lorebooks): keyword-triggered entries that inject only the relevant 200 tokens of backstory when a keyword fires. Keep your character card under 800 tokens, your persona under 200, and push everything else into a lorebook with sensible constant: false triggers.

For multi-session memory, the Smart Context / Vector Storage extension running a local nomic-embed-text-v1.5 embedder is the team's recommended setup. It costs 80 MB of VRAM and is invisible to the model.

7. Cost reality vs. cloud RP services

The economic case for local is overwhelming once you write the actual numbers. A used RTX 3090 amortized over 24 months at average European electricity prices (€0.28/kWh, 4 hours/day at 320 W) costs roughly $38/month all-in. Compare with the dominant cloud RP options:

Option	Monthly cost	Privacy	NSFW	Latency to first token
Local: RTX 3090 + Magnum-Diamond	~$38 (amortized)	Total	Yes	~150 ms
OpenRouter (Claude Sonnet 4.6)	$60–$200	Logged	Restricted	~600 ms
NovelAI (Erato)	$25	Some logging	Yes	~400 ms
Featherless / Infermatic	$15–$25	Logged	Yes	~500 ms

Run your own numbers with the BestLLMfor cost calculator. French-speaking readers can find the same analysis on our sister site quelllm.fr. All ranking data feeding this article is queryable through the BestLLMfor public API (CC BY 4.0) and the open-source quelllm-mcp server, so you can wire the model leaderboard directly into your own SillyTavern extension or evaluation pipeline.

The verdict

Use case	Winning setup	Why
Best overall (24 GB GPU)	KoboldCpp + Magnum-Diamond-24B Q4_K_M	Top voice retention, 0% refusals, 42 t/s
Budget (12 GB GPU)	KoboldCpp + Lyra-Gutenberg-Nemo-12B Q4_K_M	Only viable 12 GB option scoring >80% retention
Maximum speed	TabbyAPI + Magnum-Diamond EXL3 4.5bpw	61 t/s, half-second response time
Dual-GPU enthusiast	KoboldCpp + Behemoth-X-123B IQ3_XXS	Highest absolute quality, 94% voice retention
Avoid	Ollama, any 7B Mythomax-era finetune	Broken sampler surface; obsolete prose

For more on how the team builds these rankings, see /about/ and /methodology/.