Beginner 4 minOllama

Install Ollama on Linux

Linux is the native OS for Ollama: it runs best there, updates most cleanly, and gives you the most control. This guide covers Ubuntu/Debian/Fedora with systemd, NVIDIA and AMD GPU configuration, and how to expose the server on your LAN without breaking everything.

By Mohamed Meguedmi·Update 2026-03-25·Tested on Ubuntu 24.04

#Why Linux for local AI?

Three pragmatic reasons: GPU drivers (CUDA, ROCm) are more stable on Linux than on Windows, CLI tools integrate naturally, and homelabs run Linux by default. Add systemd to have Ollama start cleanly at boot.

i
Tested distributions
This guide has been validated on Ubuntu 22.04 and 24.04, Debian 12, and Fedora 40. The steps are identical except for package-manager differences.

#Prerequisites

The Local AI Kit

Your private ChatGPT, free, on your own machine in an hour — LM Studio, Ollama, Open WebUI, your documents, no cloud.

  • Lifetime online access
  • PDF + files
  • Lifetime updates
The Local AI Kit →or all kits for life — $49 →
Kernel 5.10+
Any reasonable distribution in 2026. Use uname -r to check.
curl ou wget
To download the installation script.
sudo / root
The installation writes to /usr/local/bin and creates a systemd service.
10 GB of disk space
Models live in /usr/share/ollama/.ollama/models by default.

#1. One-line installation

Terminal
curl -fsSL https://ollama.com/install.sh | sh

The script detects your distribution, installs the binary in /usr/local/bin/ollama, creates an ollama system user, deploys a systemd unit, and starts the service. 30 seconds.

!
Curl | sh: check if needed
Piping a script into sh is convenient but opaque. In a professional context, download it first (curl -o install.sh https://ollama.com/install.sh), review it, then run it.
Verification
ollama --version
systemctl status ollama

#2. systemd service

The service is created at /etc/systemd/system/ollama.service. To customize it (e.g., change the model directory or expose it on the LAN):

Terminal
sudo systemctl edit ollama

Add the required environment variables in the [Service] section:

systemd override
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_MODELS=/mnt/ssd/ollama-models"
Environment="OLLAMA_KEEP_ALIVE=30m"
Environment="OLLAMA_MAX_LOADED_MODELS=2"
Apply
sudo systemctl daemon-reload
sudo systemctl restart ollama
sudo journalctl -u ollama -f  # logs live

#3. GPU NVIDIA

Ollama automatically detects CUDA if the drivers are installed. Check:

Terminal
nvidia-smi
# Si la commande n'existe pas :
sudo apt install nvidia-driver-550  # Ubuntu
sudo dnf install akmod-nvidia       # Fedora
→
Container Toolkit (optional)
If you run Ollama in Docker, install nvidia-container-toolkit so the GPU is exposed inside the container. Otherwise, Docker cannot access it.
Test with a model
ollama run mistral "Bonjour"
# Dans un autre terminal :
ollama ps  # colonne PROCESSOR doit dire 100% GPU

#4. AMD GPU (ROCm)

Radeon RX 6000, 7000, and Instinct cards are supported through ROCm. Installation is touchier than CUDA, but it works.

Ubuntu 22.04/24.04
wget https://repo.radeon.com/amdgpu-install/6.0/ubuntu/jammy/amdgpu-install_6.0.60000-1_all.deb
sudo apt install ./amdgpu-install_6.0.60000-1_all.deb
sudo amdgpu-install --usecase=rocm
sudo usermod -aG render,video $USER
# reboot
!
Cards without official support
If your GPU is not on AMD's official list (e.g., RX 6600, 6700), use HSA_OVERRIDE_GFX_VERSION=10.3.0 as an environment variable. This forces ROCm to treat the card as a gfx1030.

#5. Expose it on the local network

By default, Ollama listens only on localhost:11434. To let another computer or a Raspberry Pi query it:

In the systemd override
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_ORIGINS=http://192.168.1.*"
Firewall
sudo ufw allow from 192.168.1.0/24 to any port 11434
!
Never expose it to the Internet
Ollama has no native authentication. If you want to expose it beyond the LAN, put a reverse proxy (Caddy, nginx) with basic auth or JWT in front of it.

#Troubleshooting

systemctl status ollama → failed
Check journalctl -u ollama -n 50. Often: port 11434 is already in use, or there are permissions issues with the models directory.
GPU not detected
nvidia-smi and rocminfo must respond. If not: drivers. A reboot after installation is often required.
Slow model, 100% CPU
ollama ps ne montre pas GPU. Drivers ok, mais variable CUDA_VISIBLE_DEVICES mal réglée ? Vérifiez environnement du service.
Not enough space
Move OLLAMA_MODELS to a larger SSD. Don’t forget chown -R ollama:ollama on the new folder.
Did this guide help you?

Feedback, an error, or a clarification? Let us know—it improves the guide for everyone.