Temperature, top-p, top-k: the parameters

#Top-p, top-k, temperature: the definition in one table

For each word it produces, a model calculates a probability for every token in its vocabulary. Sampling parameters then decide which one to draw. Top-p and top-k reduce the list of candidates; temperature changes the gap between likely and unlikely ones. None adds knowledge to the model: they only adjust variety and risk.

These defaults come from the Ollama Modelfile reference. Many models provide their own values in their configuration file, which then override these defaults—hence the value of reading the model page, as we’ll see below.

#How an LLM chooses the next word

For each position, the model produces a probability distribution over its entire vocabulary. Always choosing the most probable token—called greedy decoding—produces flat, repetitive text that can sometimes get stuck in a loop. So we introduce random sampling, constrained by the sampling parameters. A fixed seed makes this sampling reproducible: the Ollama documentation specifies that a given seed produces the same text for the same prompt.

The order in which filters are applied matters. In llama.cpp, the GGUF inference engine also used by LM Studio, the default order is: penalties, DRY, top-n sigma, top-k, typical-p, top-p, min-p, XTC, then temperature last. In other words, top-k and top-p operate on the original probabilities, and temperature is used only to sample from what remains. Many tutorials state the opposite; the actual order may differ from one engine to another, and is configured in llama.cpp with the --samplers parameter.

#A numerical example: what each setting retains

Let's take seven candidates for the next word, with probabilities of 0.45; 0.25; 0.12; 0.08; 0.05; 0.03; 0.02. This table is an illustrative calculation, not a measurement on a real model.

The key takeaway: top-k always keeps the same number of candidates, regardless of the situation; top-p and min-p adapt. When the model is very confident (one token at 0.95), top-p may keep only one or two candidates; when it is choosing among ten words, it keeps more. That is why top-p is often preferred over top-k.

#1. Temperature: the randomness knob

Before sampling, temperature divides the model's logits. At low temperature, the gap between tokens widens: the most likely token overwhelms the others. At high temperature, the gap narrows. At 0, the model always chooses the most likely token and produces identical outputs every time, as the llama.cpp documentation confirms.

0 à 0,3

Extraction, classification, faithful summarization: you want the same answer every time.

0,4 à 0,7

Technical writing, translation, support: variety without drift.

0,8

Default value for Ollama and llama.cpp: general conversation.

1 and up

Creativity, brainstorming; greater risk of errors and digressions.

#2. Top-p: the adaptive filter

Top-p, or nucleus sampling, retains the most probable tokens until their cumulative probability reaches p. With 0.9, it discards the 10% probability tail; with 1.0, it filters nothing. Ollama's documentation notes that a higher value, such as 0.95, produces more varied text, while a lower one, such as 0.5, produces more focused and conservative text.

0,5 à 0,7

Very conservative: safe answers, sometimes monotonous.

0,9

Ollama default: cuts off the tail without stifling variety.

0,95

The default value for llama-server and the value recommended by several model cards.

1,0

Filter disabled: only the temperature remains active.

#3. Top-k: the fixed ceiling

Top-k keeps only the k most likely tokens. Ollama's documentation describes this parameter as a way to reduce the likelihood of producing nonsense: higher (100) means more diverse responses; lower (10) means more conservative ones. Its default is 40, as in llama.cpp, where 0 disables it.

For the “top p vs. top k” query: use top-p as the primary setting, keep top-k as a safeguard, and don’t adjust both in the same trial, or you won’t know which one caused the difference.

#Defaults and vendor recommendations

An engine's defaults are a generic compromise. Model publishers often provide values tailored to their models, and these may differ from the defaults. Here's what the Qwen3.5 model card recommends for its models, compared with the Ollama defaults (temperature 0.8, top-p 0.9, top-k 40).

These values apply to this model family, not to all models. Other vendors' model cards provide different values: look for the section on recommended sampling parameters in the Hugging Face model card or on the model's page in the Ollama library. The Qwen model card also notes that support for these parameters varies by inference engine.

#Penalties: repetition, frequency, presence

Local models, especially small or heavily quantized ones, sometimes loop over the same phrases. Penalties are the remedy, but their drawbacks are often poorly understood. In Ollama, repeat_penalty defaults to 1.0, meaning it is disabled; the documentation gives 1.5 as an example of a value that penalizes more strongly. The often-cited value of 1.1 is a choice to make, not a default.

repeat_penalty

Penalizes recently seen tokens; repeat_last_n (64 by default) sets the window examined.

frequency_penalty

Penalty proportional to the number of occurrences, for long generations.

presence_penalty

A penalty as soon as a token has appeared, to encourage a more varied vocabulary; Qwen recommends 1.5 in several of its modes, with the risk of language mixing if the value is too high.

#Apply these settings in Ollama

There are two methods: a Modelfile, which sets values for a named model, or the options field of an API request, which accepts the same parameters as the Modelfile. The Modelfile is suited to daily use; the API is suited to a program.

FROM llama3.2
PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER top_k 40

ollama create llama-precis -f ./Modelfile
ollama run llama-precis

curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Résume en une phrase : ...","stream":false,"options":{"temperature":0.3,"top_p":0.9}}'

For more on Modelfiles (system prompt, context, template), see the customization guide. For LM Studio and Jan, the same settings are configured in the model settings panel.

#Reasoning models: don't solve everything with temperature

Reasoning models first produce a chain of thought before their answer. On Ollama, these models expose a separate reasoning field, and the available commands vary by model: the documentation recommends querying the show API to find the accepted values and the default. For gpt-oss, for example, the levels are low, medium, and high, with medium as the default.

To reduce unnecessary overthinking on a simple question, adjust this reasoning level first, when available, rather than the temperature. Temperature should follow the model's specifications: reasoning that is too cold can freeze up, while reasoning that is too hot can wander.

#Starting presets by use case

These presets are starting points, not measurements. Change one parameter at a time, run the same prompt three times, and compare: if three attempts produce the same answer when you expect variety, the temperature is too low or the seed is fixed.

#Symptoms and settings to try

The model repeats the same sentence

Set repeat_penalty to around 1.1 or use presence_penalty, or shorten generation with num_predict.

Flat, generic responses

Increase the temperature by one step (0.7 to 0.9) or loosen top_p.

The model makes up facts

Lower the temperature, but be aware that this isn't enough: see the guide to hallucinations.

Broken JSON

Low temperature and an explicitly requested format; use Ollama's structured output mode when available.

Mixed languages or strange words

Lower presence_penalty or repeat_penalty, or slightly increase the model’s quantization.

• Ollama Modelfile: customize a model
• Prompting basics
• Master system prompts
• Understanding the context window
• Limit hallucinations in a local LLM

• Source: Ollama Modelfile reference
• Source: Qwen3.5-9B specification
• Source: llama.cpp, llama-server options
• Source: Ollama, reasoning models