Air-gap your machine of inference

#When an air gap is justified—and when it costs too much

An air gap should be decided through a risk analysis, not as a matter of principle. Cases where it is justified include classified data or data covered by a high-level trade secret, contractual or sector-specific requirements that explicitly mandate an isolated network, environments that already use isolation and where the LLM must be integrated, and workstations dedicated to exceptionally sensitive files (a lawyer, a researcher). For ordinary personal data, including health or financial data, regulations call for measures proportionate to the risk rather than total isolation: a local LLM on an encrypted workstation, without a cloud account, already meets most needs. The guide to local AI in the enterprise and the privacy checklist cover this middle ground.

Before you get started, ask three questions. Who will handle the transfers, and how often? Which components does the stack require us to update (Ollama, llama.cpp, the interface, the models)? And what happens when a newer model becomes essential: the process must remain practical without exceptions, because an exception made once becomes the rule. If you cannot answer these three questions, start with an offline workstation per use case and a strict firewall, and move up a level only when the risk justifies it.

#Air gap, near-air gap, and firewall: don't confuse them

This guide covers all three: the firewall is the minimum layer every sensitive workstation should have, and a physical air gap is the top layer. On a Mac, macOS-specific settings belong in the guide to optimizing Macs with Apple chips; under Windows, the principles are the same, but control over components that call out is less granular than with a minimal Linux system, hence the preference here for Linux.

#Prepare the machine

Minimum system

Debian or Ubuntu Server without a graphical interface for an inference server. No browser, no package manager connected to an app store.

Automatic update services

Disable update timers, for example sudo systemctl disable --now apt-daily.timer apt-daily-upgrade.timer, so that no connection attempt is made silently.

Accounts

An account dedicated to LLM use, with sudo restricted to the administrator.

Radios and interfaces

Wi-Fi and Bluetooth disabled in the BIOS or firmware; better yet, physically absent. No network cable connected.

No double startup

A connected partition alongside it creates indirect paths (swap file, shared files).

Time

Set the clock manually or from an internal source: synchronization with a public time server is outbound traffic.

Encrypted disk

An isolated workstation can be stolen or seized: encrypt the disk (see the dedicated guide).

#LLM stack software that makes outbound calls by default

For a physically isolated machine, these calls fail harmlessly; for a machine protected only by a firewall, or one awaiting disconnection, they are the real risk. A typical local stack contains several components that contact the Internet unless prevented from doing so. Here are the ones the official documentation allows you to disable.

The Ollama documentation specifies that when using Ollama locally, the vendor sees neither the requests nor the data; disabling cloud features removes access to hosted models and web search, which is exactly the point. Be careful with Open WebUI: the documentation warns that if you have not downloaded an embedding model before enabling OFFLINE_MODE, RAG, web search, and document analysis features may not work. Download everything the machine will need before taking it offline.

#Strict firewall: deny by default, inbound and outbound

The firewall is the second line of defense, after physical disconnection, and the only protection on a workstation that remains on an internal network. The principle is deny by default in both directions, with a few explicit permissions to the isolated internal network. Write the allow rules before enabling the firewall; otherwise, you'll cut yourself off from an SSH session.

sudo ufw default deny incoming
sudo ufw default deny outgoing

# Réseau interne isolé (exemple), y compris l'administration SSH depuis un poste connu
sudo ufw allow in from 192.168.50.0/24 to any port 22 proto tcp
sudo ufw allow out to 192.168.50.0/24

sudo ufw enable
sudo ufw status verbose

Traffic on the local interface (loopback) is required for Ollama and Open WebUI, which communicate over 127.0.0.1; check with ufw status verbose that your configuration does not block it. If the machine is on an internal LAN, add protection at the router as well: a VLAN with no route to the Internet.

#Check what Ollama is listening on

sudo ss -tlnp | grep ollama
# attendu : 127.0.0.1:11434 et non 0.0.0.0:11434
# si 0.0.0.0 : régler OLLAMA_HOST=127.0.0.1:11434

#Transfer models and updates by USB drive

An isolated machine receives models (from a few GB to several dozen), updates (Ollama, llama.cpp, Open WebUI), and sometimes documents. The critical point is the USB drive: it’s the only door, and therefore the only entry point for malware. Always use the same dedicated drive, format it regularly, and never use a drive that has been used elsewhere.

1. 01
   Download on a connected machine

   Retrieve the model's GGUF file from Hugging Face or the update binary from the official website. Record the SHA-256 fingerprint displayed by the source when available.
2. 02
   Calculate the fingerprint and analyze

   Run sha256sum on the file, then pass it to an antivirus (such as ClamAV) on this transit machine, which is not the isolated machine.
3. 03
   Copy to the USB drive

   Copy the file and a text file containing its hash.
4. 04
   Check on the isolated side before use

   On the isolated machine, recalculate the hash: it must be identical. A difference means a file was modified or corrupted: do not use it.
5. 05
   Import into Ollama

   Create a Modelfile with FROM /chemin/vers/modele.gguf and then ollama create nom-du-modele -f Modelfile. Ollama does not quantize a GGUF during import: the file must already have the desired quantization.

# machine de transit
sha256sum modele-q4_k_m.gguf > modele.sha256

# machine isolée
sha256sum -c modele.sha256
printf 'FROM ./modele-q4_k_m.gguf\n' > Modelfile
ollama create mon-modele -f Modelfile
ollama run mon-modele

The alternative method is to copy the entire model directory from a connected machine (blobs and manifests) to the isolated machine. It works, but the directory is in a different location depending on the installation: the Ollama FAQ gives ~/.ollama/models on macOS, /usr/share/ollama/.ollama/models on Linux with the standard installer, and C:\Users\%username%\.ollama\models on Windows. Using an isolated GGUF file is easier to verify because there is only one fingerprint to compare. The guide on import GGUF provides details on the Modelfile.

#Harden the USB port

A USB key can masquerade as a keyboard or another device. On Linux, USBGuard can be described in one sentence as a USB device allowlisting tool: it defines which types of devices are authorized and how they can interact with the system. Also disable automatic media mounting so that nothing runs when connected. In highly sensitive contexts, a data diode (a device that allows data to pass in only one direction) replaces the USB key, at the cost of a more complex setup.

#Test isolation

A test proves airtightness only for the period it observed: repeat it after every update and let the capture run through a complete work session. The minimum protocol covers operation, no routing, no name resolution, and actual traffic.

Offline operation

Unplug the cable and turn off all radios, then run the full pipeline (model, RAG, interface). Everything should work; otherwise, a component depended on the Internet.

No route

ping -c 1 8.8.8.8 should fail (“Network is unreachable” or equivalent).

No DNS

host example.org must fail; check /etc/resolv.conf: no external server.

Established connections

sudo ss -tunp state established : aucune connexion vers une adresse hors du réseau interne.

Real-world traffic

A tcpdump capture on all interfaces, excluding the internal network, must remain empty throughout an entire usage session.

# rien ne doit apparaître (hors loopback et réseau interne)
sudo tcpdump -i any -n 'not (host 127.0.0.1 or host ::1) and not (net 192.168.50.0/24)'

#Regular audit

#Frequently asked questions about air-gapping an LLM machine

• Encrypt the model drive
• Privacy checklist
• Local AI in the enterprise: GDPR, sovereignty, and deployment
• Import a GGUF model from Hugging Face into Ollama
• Get the most out of a Mac Apple Silicon

• Source: Ollama FAQ, disableable cloud and locations
• Source: Ollama, import d's GGUF model
• Source: Open WebUI, environment variables
• Source: USBGuard