Guide · 2026-08-15

Local LLM for Data Scientists — A Notebook-First Workflow

Last updated 2026-08-15

The models, VRAM tiers, and Jupyter setup that let data scientists run a private coding assistant inside the notebook — no cloud, no data leaving the machine.

By Mohamed Meguedmi · 9 min read

Data scientists live in the notebook. The read–eval–plot loop is where hypotheses get tested, and a coding assistant is only useful if it lives in that same loop — aware of your columns, your dtypes, and the last traceback. A browser chat tab that you copy-paste into is friction. A local model wired into the kernel is a colleague.

This guide takes a position: for a notebook-first workflow in 2026, the Qwen2.5-Coder family is the default recommendation across every VRAM tier, with DeepSeek-Coder-V2 as the strongest alternative on mid-range cards. Below we cover model selection by hardware, the benchmarks that actually predict notebook quality, a ten-minute setup, and a repeatable pattern you can drop into any project.

Why notebook-first, not chat-first

The dominant pattern most people copy from tutorials is the “describe your data, paste code, run it, paste results back” loop. It works, but it leaks the two things that make a local model valuable: context and privacy. Every round trip is a manual re-description of state the kernel already holds.

A notebook-first workflow inverts this. The model runs behind a local API endpoint (Ollama exposes an OpenAI-compatible server on localhost:11434), and a thin helper cell feeds it live context — df.head(), df.dtypes, the schema, and the traceback from the failing cell. The academic Jupiter paper (arXiv:2509.09245) makes the same argument: data analysis is an interconnected, multi-step process, and tools that target isolated stages underperform ones that see the whole notebook state. You do not need their full agent to benefit — you just need the model in the kernel.

The rule of thumb: if you are copy-pasting DataFrame previews into a chat window, you have already lost the advantage of running locally. Pipe the context in programmatically.

Pick your model by VRAM

Model choice is dominated by how much VRAM you can dedicate to weights plus KV cache. A 4-bit quant (Q4_K_M) is the sweet spot for data-science coding: it keeps roughly 97–99% of the full-precision quality on code tasks while nearly halving the memory footprint versus 8-bit. The table below maps common cards to a confident recommendation.

VRAM	Recommended model	Quant	Approx. weights	What it's good for
8 GB (RTX 3060/4060, laptop)	Qwen2.5-Coder 7B	Q4_K_M	~4.7 GB	EDA boilerplate, plotting, docstrings, simple SQL
12 GB (RTX 3060 12G/4070)	Qwen2.5-Coder 14B	Q4_K_M	~9 GB	Feature engineering, multi-step pandas, joins
16 GB (RTX 4060 Ti 16G/4080)	DeepSeek-Coder-V2 16B Lite	Q4_K_M	~10.4 GB	SQL generation, wrangling, fast MoE inference
24 GB (RTX 3090/4090)	Qwen2.5-Coder 32B	Q4_K_M	~19.9 GB	Best overall: reasoning, refactors, tricky joins
48 GB+ (dual 3090, A6000)	Qwen2.5-Coder 32B	Q8_0	~35 GB	Long context (32K+), fewer quant artifacts

DeepSeek-Coder-V2 Lite is a mixture-of-experts model with 16B total but only ~2.4B active parameters, so it runs unusually fast on a 16GB card — a good choice if latency matters more to you than the last few points of reasoning. For the raw weights and license details, see the Qwen2.5-Coder-32B-Instruct model card and the DeepSeek-Coder-V2 page on ollama.com. Browse quant sizes for every tier in our model catalog.

The benchmarks that actually matter

Leaderboard HumanEval scores are a weak proxy for notebook work — they measure isolated function completion, not multi-cell data manipulation. Weight the following instead, in order: (1) pandas/dataframe task accuracy, (2) SQL correctness, (3) instruction-following on multi-step prompts, and (4) tokens/sec at your batch size of one. The table gives representative published figures for the recommended tier.

Model (Q4_K_M)	HumanEval (pass@1)	DS-1000 (data-sci)	Tokens/sec (24GB)	Practical verdict
Qwen2.5-Coder 32B	~92%	Strong	~35–45 t/s	Best local notebook assistant
Qwen2.5-Coder 14B	~89%	Good	~55–70 t/s	Best value on 12GB
DeepSeek-Coder-V2 16B Lite	~81%	Good (SQL)	~70–90 t/s	Fastest; SQL specialist
Qwen2.5-Coder 7B	~88%	Fair	~90–120 t/s	Best on 8GB / laptops

Figures are drawn from the models' published cards and the Qwen2.5-Coder technical report (arXiv:2409.12186); token rates are indicative for single-stream inference on a 24GB consumer GPU and vary with context length. For our reproducible methodology and the raw numbers, see the benchmarks hub and how we test. Every score is also queryable through the BestLLMfor public API (CC BY 4.0) and the companion open-source MCP server, so you can pull current numbers directly into a notebook cell.

The ten-minute setup: Ollama + Jupyter

The lightest stack that keeps everything local is Ollama for model serving plus a standard Jupyter kernel talking to it over HTTP. No API keys, no egress.

1. Install Ollama. Download from ollama.com/download and confirm the daemon is up with ollama --version.
2. Pull a model for your tier. For a 24GB card: ollama pull qwen2.5-coder:32b. For 8GB: ollama pull qwen2.5-coder:7b.
3. Install the client in your env. pip install openai jupyterlab — the OpenAI SDK speaks to Ollama's compatible endpoint unchanged.
4. Point the client at localhost. Set base_url="http://localhost:11434/v1" and any dummy api_key.
5. Verify from a cell. Send a one-line completion and confirm you get tokens back before wiring in context.

from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="local")

def ask(prompt, model="qwen2.5-coder:32b"):
    r = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.1,
    )
    return r.choices[0].message.content

print(ask("Return a pandas one-liner to drop rows where col 'age' is null."))

Keep temperature low (0.0–0.2) for code generation — you want determinism and correct syntax, not creativity.

A repeatable notebook pattern

The payoff comes from a small helper that injects live state. Instead of describing your data, hand the model the same view you would give a colleague: schema, a preview, and the error if there is one.

def ask_about(df, task):
    context = (
        f"Schema:\n{df.dtypes.to_string()}\n\n"
        f"Preview:\n{df.head(3).to_string()}\n\n"
        f"Task: {task}\n"
        "Return only runnable pandas code, no prose."
    )
    return ask(context)

# Usage
print(ask_about(df, "pivot monthly revenue by region, fill missing with 0"))

Three habits make this reliable in practice:

• Feed the traceback. On failure, pass traceback.format_exc() back with the offending code — local models fix their own errors well when they can see them.
• Cap the preview. Three to five rows is enough; more just burns context and slows the response.
• Ask for code only. The “no prose” instruction keeps output paste-ready and cuts token count roughly in half.

For deeper integrations — auto-inserting generated cells or streaming completions inline — see the tooling we track under the guides hub.

Cost and privacy: the real payoff

Two forces justify going local for data science specifically. First, data governance: notebooks routinely touch PII, financial records, and unreleased datasets. A local model means that data never crosses a network boundary, which sidesteps most vendor-review and DPA friction outright. Second, cost at volume: an iterative EDA session can fire hundreds of completions. On a metered API that adds up; locally the marginal cost is electricity.

Factor	Local (Qwen2.5-Coder 32B)	Hosted frontier API
Cost per 1M tokens	~$0 marginal (power only)	$3–$15
Data egress	None	Every request
Offline capable	Yes	No
Peak quality	High (notebook tasks)	Highest (hard reasoning)
Upfront cost	~$700–$1,800 GPU	$0

Break-even depends entirely on usage. Model your own numbers with the cost calculator — for teams running daily analysis, a 24GB card typically pays for itself within months versus a metered API.

Verdict

For a notebook-first workflow, stop treating a local LLM as a chat tab and put it in the kernel. The Qwen2.5-Coder family is the confident default at every hardware level; DeepSeek-Coder-V2 Lite is the pick when speed and SQL matter most. Match the model to your VRAM, wire in live context, and keep temperature low.

Your situation	Run this	Why
24GB GPU, want the best	Qwen2.5-Coder 32B Q4_K_M	Top open quality for notebook tasks
12GB GPU, best value	Qwen2.5-Coder 14B Q4_K_M	90% of the quality, half the VRAM
16GB, latency-sensitive / SQL-heavy	DeepSeek-Coder-V2 16B Lite	Fast MoE, strong SQL generation
8GB or laptop	Qwen2.5-Coder 7B Q4_K_M	Only viable strong option at this tier

Compare full specs in the best-of rankings, and pull current benchmark numbers straight into your notebook via the BestLLMfor public API or MCP server.