Mistral Vibe CLI with Devstral locally: setup route

#Mistral Vibe CLI: what it is and what it isn't

Mistral Vibe CLI is a command-line program written in Python, released in December 2025 alongside Devstral 2 and Devstral Small 2. You launch it in a project directory, describe a task in natural language, and the agent reads your files, proposes changes, runs commands, and iterates until it gets the result. It follows the same principle as Claude Code, OpenCode, or Aider, but is designed around Devstral models, which were trained for this type of agentic loop.

The source code is published on GitHub in the mistralai/mistral-vibe repository, under the Apache 2.0 license according to the announcement from Mistral. You can read, modify, and use it in a business without royalties. The CLI can communicate with the Mistral API, but also with any server exposing an OpenAI-format API: Ollama, llama.cpp, LM Studio, or vLLM. This second mode, documented on the “offline models” page, is what interests us here.

Why use a local Vibe CLI instead of the API? Three reasons come up repeatedly. The code does not leave the machine, which matters for proprietary code or code covered by a confidentiality agreement. There is no usage-based billing or quota: you pay for the electricity and hardware you already have. Finally, the tool continues to work offline or behind a strict firewall. In exchange, you lose the 123-billion-parameter Devstral 2 model, which is too large for a workstation, and accept lower speed than a production server.

#Prerequisites: Python, Ollama, and enough memory for Devstral Small 2

Before installing anything, check these four points. They prevent three-quarters of the failures encountered with local coding agents.

System

Linux and macOS are the primary targets. On Windows, use WSL2 (Ubuntu): the CLI installs there as it does on Linux, and Ollama for Windows remains reachable from WSL at the localhost address.

Recent Python

The CLI is distributed as a Python package. The repository lists the minimum required version in its README; at the time of the announcement, this was Python 3.12 or later. Check with python3 --version before continuing.

Ollama installed and started

Ollama serves the model and exposes an OpenAI-compatible API at http://localhost:11434. If you have not done so already, our Ollama installation guide covers all three systems.

Memory for a 24B model

Devstral Small 2 has 24 billion parameters. In Q4_K_M quantization, the weights occupy about 14 GB, in addition to the context cache. Plan on a 24 GB GPU (RTX 3090, RTX 4090) or a Mac with 32 GB of unified memory for comfortable operation. The Mistral announcement also cites “a RTX 4090 or a Mac with 32 GB of RAM” as the target configuration.

With a 16 GB GPU (RTX 4080, RTX 5070 Ti desktop), the Q4 model fits, but the context quickly spills into system memory and speed drops. With 12 GB (RTX 3060, RTX 4070), Ollama splits the layers between the GPU and CPU: the agent works, but each loop iteration takes several dozen seconds. We did not measure these throughput figures on these cards; simply keep in mind that 24 GB is the comfort threshold for a 24B used as an agent, where the context grows with each file read.

#Install Mistral Vibe CLI

Mistral provides a one-line installation script that sets up the CLI in an isolated environment. This is the recommended method in the repository’s README.

curl -LsSf https://mistral.ai/vibe/install.sh | bash

If you prefer to control the installation, the package is published on PyPI under the name mistral-vibe. The uv tool installs it in a dedicated environment without affecting your other Python projects. Classic pip also works, preferably in a virtual environment.

# Installer uv si besoin : https://docs.astral.sh/uv/
uv tool install mistral-vibe

# Ou avec pip, dans un environnement virtuel
python3 -m venv ~/.venvs/vibe
source ~/.venvs/vibe/bin/activate
pip install mistral-vibe

Next, verify that the vibe command is available. If the terminal cannot find it, close and reopen it: the installation script adds a directory to your PATH, which takes effect the next time the shell starts.

vibe --version
vibe --help

#Prepare Devstral Small 2 in Ollama

Devstral Small 2 is published by Mistral under the Apache 2.0 license, with a 256,000-token context window. Ollama offers it in its library; the command below downloads the default version in 4-bit quantization. The download is approximately 14 GB.

ollama pull devstral-small-2
ollama list

The exact model name in the Ollama library may change (quantization variant, size suffix). If the command fails, search for “devstral” on ollama.com/library and use the displayed name. This is the name, exactly as it appears in ollama list, that you will copy into Vibe’s configuration.

One decisive setting remains: context length. Ollama loads models with a short default context, often 4,096 tokens. For a coding agent, that is far too little: Vibe’s system prompt, tool list, file contents, and session history exceed that size after just a few exchanges. The model then forgets the beginning of the conversation and the agent goes in circles. Set a context of at least 32,000 tokens, for example with a derived Modelfile.

cat > Modelfile.devstral <<'EOF'
FROM devstral-small-2
PARAMETER num_ctx 32768
EOF

ollama create devstral-small-2-32k -f Modelfile.devstral
ollama run devstral-small-2-32k "Réponds en un mot : prêt ?"

You can also set the context globally with the OLLAMA_CONTEXT_LENGTH environment variable when starting the service Ollama. The Modelfile has the advantage of applying only to this model and keeping an explicit name. A longer context consumes more memory: at 32k tokens on a 24B, plan for an additional 2 to 4 GB depending on the KV cache type. On 24 GB, that is the comfortable limit; beyond 64k, monitor memory with nvidia-smi.

#Connect the local model in config.toml

Mistral Vibe CLI reads its configuration from the ~/.vibe/config.toml file. The “offline models” documentation describes two tables there: providers, which declares a server and its address, and models, which maps a model name to a provider. An active_model key designates the model used by default. Here is a complete configuration for the Ollama prepared above.

# Modèle utilisé au lancement (doit correspondre à un alias ci-dessous)
active_model = "devstral-small-local"

# Fournisseur : votre Ollama local, API au format OpenAI
[[providers]]
name = "ollama"
api_base = "http://localhost:11434/v1"
api_key_env_var = ""
api_style = "openai"
backend = "generic"

# Modèle : le nom exact affiché par `ollama list`
[[models]]
name = "devstral-small-2-32k"
provider = "ollama"
alias = "devstral-small-local"
input_price = 0.0
output_price = 0.0

Three details need explanation. The address ends in /v1 because Vibe speaks the OpenAI dialect and Ollama exposes this format only under that prefix; the native API of Ollama, without /v1, will not work. The api_key_env_var key remains empty: Ollama requires no authentication locally. Prices are set to zero so the cost estimate displayed by the CLI remains consistent; they have no effect on operation.

For another local server, only the providers section changes. With llama.cpp (llama-server), the address is generally http://localhost:8080/v1; with LM Studio, http://localhost:1234/v1; with vLLM, http://localhost:8000/v1. In every case, the model's name field must be the identifier expected by that server, the one it returns on its /v1/models route.

curl -s http://localhost:11434/v1/models | python3 -m json.tool

You can declare multiple models, for example Devstral Small 2 locally and Devstral 2 through the Mistral API, then switch between them in the session. For the Mistral provider, the API key is read from an environment variable, typically MISTRAL_API_KEY; the key does not need to appear in plain text in the file. If your goal is 100% local operation, simply do not add any remote provider: nothing will leave the machine.

#First session: launch vibe and check the model

1. 01
   Open a project

   Open a terminal in a Git repository's directory, preferably a clean one (git status with no pending changes). The agent works on files in the current directory; a clean repository lets you review and undo its changes with git diff and git checkout.
2. 02
   Launch the CLI

   Type vibe. On the very first launch, the CLI may prompt you to enter an Mistral API key and create its ~/.vibe directory. If you wrote config.toml first, your local provider is already known. If the key prompt still appears, check /help: depending on the version, you can skip it or leave it blank when a local provider is configured.
3. 03
   Check the active model

   The interface displays the active model. It should be your devstral-small-local alias. The /help command lists the commands available in the session, including the one that switches models if you have declared several.
4. 04
   Give it a first bounded task

   Start small: “Read the README and summarize the project architecture in five lines,” then “Add a unit test for function X in file Y.” Watch the first response: if the model answers without using a tool or invents a file path, the problem is generally a context window that is too short or a model that has not been prepared for tool calling.
5. 05
   Review before accepting

   Vibe asks for confirmation before sensitive actions, such as writing files or executing commands. Review every proposed diff. A 24B model running locally makes mistakes more often than a 123B model served through the API, especially on large codebases: review is part of the workflow, not the exception.

During the session, keep an eye on the Ollama load: the ollama ps command shows the loaded model, the GPU and CPU share, and the effective context. If you see partial allocation on the CPU even though you thought the model would fit in video memory, the context is what overflowed the card.

ollama ps
nvidia-smi --query-gpu=memory.used,memory.total --format=csv

#Limitations of a 100% local Vibe CLI

A local coding agent works, but it cannot be compared directly with the API version. Here is what to expect, without invented figures: we did not run a benchmark for this guide, and the scores published by Mistral concern models served under reference conditions, not a desktop machine running a quantized model.

Devstral 2 remains out of reach

The large 123-billion-parameter model requires tens of gigabytes of memory even when heavily quantized. Locally, you work with Devstral Small 2, for which Mistral reports 68.0% on SWE-bench Verified, compared with 72.2% for Devstral 2. The gap is modest on this benchmark, but it is noticeable on long tasks and large repositories.

Quantization changes the model

The announced score applies to full-precision weights. Q4_K_M slightly reduces accuracy; Q8_0 preserves it better but requires about 26 GB for the weights alone, so you need a 32 GB card or a well-equipped Mac. Choosing a quantization is a trade-off that requires understanding the consequences.

The local context is much shorter than 256k

Devstral Small 2 supports 256,000 tokens, but you will not serve that many on 24 GB. Between 32k and 64k tokens, the agent can cover a module or a small service. For a monorepo, you need to scope the task file by file.

Speed depends on the hardware, not the CLI

A dense 24B generates more slowly than a similarly sized MoE model at the same memory capacity. Each agent loop iteration rereads the entire context: the longer the session, the longer each response takes. Plan short sessions and split tasks into smaller pieces.

Tool calling and formats

The agent relies on structured tool calls. A quantized model may occasionally produce a malformed call that the CLI rejects. Devstral was trained for this type of use and handles it much better than a general-purpose model, but the issue is not nonexistent.

A young tool

The CLI was released in December 2025 and is evolving quickly. The commands, configuration format, and editor integrations are changing. Update regularly (uv tool upgrade mistral-vibe, or rerun the installation script) and review the changelog on GitHub before reporting unexpected behavior.

The Mistral announcement also mentions editor integration through the ACP protocol, particularly with Zed. If you prefer an editor to the terminal, our Zed + Ollama guide shows how this editor already communicates with a local model; combining the two means letting Vibe control the agent while Zed displays the diffs.

#Troubleshooting

“vibe: command not found”

The executable directory is not in your PATH. Reopen the terminal. With uv, the path is usually ~/.local/bin; check with uv tool dir --bin and add it to your shell file if necessary.

Connection error to localhost:11434

Ollama is not running, or it is listening on another address. Test with curl http://localhost:11434/v1/models. Under WSL2, make sure Ollama is running properly on the Windows side and that the OLLAMA_HOST variable has not been restricted.

The server does not recognize the model

The name field in the models section does not match a model listed in ollama list. Copy the exact name, including the suffix, including that of your derived Modelfile if you created one.

The agent forgets what it just read

Context too short. Check the actual loaded context length with ollama ps; if it is 4,096, your Modelfile or your OLLAMA_CONTEXT_LENGTH variable was not applied.

Very slow responses, GPU barely used

Part of the model is running on the CPU. Reduce the context, switch to lighter quantization, or close the other models loaded by Ollama (OLLAMA_MAX_LOADED_MODELS).

The CLI requires an API key even though everything is local

Behavior depends on the version. Make sure config.toml is located at ~/.vibe/config.toml, that active_model points to an existing alias, then check the documentation’s “offline models” page for the current procedure.

#Go further

Vibe CLI is a good choice if you are committed to the Mistral ecosystem and Devstral. If you want to compare options before settling in, OpenCode is in the same category with more supported providers, and the question of which local coding model to use is independent of the agent you choose.