WebLLM: an AI model in the navigateur

#WebLLM: what the library does

With Ollama, a program installed on the machine loads the model and responds on http://localhost:11434. With WebLLM, everything goes through a web page: it loads the engine's JavaScript and WebAssembly code, fetches the model weights, then sends the computations to the GPU via WebGPU. There's no service to start and no port to open.

On the developer side, the API follows the shape of OpenAI's: you create an engine, then call engine.chat.completions.create with a list of messages. So an app written for this format adapts fairly quickly. However, the engine lives inside the page: it isn't an HTTP server that Open WebUI, a code editor, or a Python script could connect to.

WebGPU

Browser API that provides GPU access for compute. Without it, or without an available adapter, the engine can't run on the GPU.

WebAssembly

Format in which part of the engine is delivered. It's part of what the page has to load, just like the JavaScript.

The model weights

Large files, separate from the page code. They're downloaded from the URL specified in the configuration, then cached by the browser.

The MLC format

WebLLM uses models compiled for its engine, recognizable by the MLC suffix in their IDs. A GGUF file downloaded for Ollama or LM Studio won't load as is.

#WebLLM or Ollama: which tool for which need?

WebLLM doesn't replace a conventional local installation. Its value lies elsewhere: offering an AI feature in a web page to people who won't install anything.

WebLLM is a good fit for

A public demo, an internal web tool, an assistant built into an application, a quick test on a machine where you can't install anything.

Ollama or LM Studio are suitable for

Daily use, choosing from a large catalog, larger models, an API that other tools can reuse, operation that doesn't depend on the limits of a browser tab.

The trade-offs you accept

The catalog depends on models compiled in MLC format, available memory also depends on browser limits, and the cache can be cleared without warning.

We don't provide a speed comparison: no measurement with a documented protocol and hardware backs this guide. If speed matters for your project, measure on your target machines, with the same model and the same quantization, noting the browser and driver versions.

#WebGPU requirements and compatibility

MDN still listed WebGPU as limited availability, meaning not Baseline, on its page last modified September 2, 2026: the API doesn't work in some widely used browsers. Support depends on the combination of browser, OS, and graphics driver. So a browser may expose WebGPU on one system but not on another. Rather than a list of versions that quickly goes stale, check MDN's compatibility table, then test each target machine.

Secure context

According to MDN, the entire API is available only in a secure context: serve the page over HTTPS, or on localhost during development. A page opened over HTTP on a local network address won't work.

Browser and driver

Both up to date. A recent GPU isn't enough if the driver or browser blocks access.

GPU memory

QuelLLM reference figures in Q4 on the usual stack: 3B about 2 GB, 7B about 5 GB. These are ballpark figures, not WebLLM measurements.

Disk space and connection

Enough to store the model and download it at least once. Check the actual size in the loading progress.

Paste this test into the developer tools console on a page served over HTTPS or on localhost. It relies on the standard API and doesn't load any model. According to MDN, requestAdapter returns null when no suitable adapter is available, so the presence of navigator.gpu isn't enough to draw a conclusion.

if (!navigator.gpu) {
  console.log('WebGPU absent : navigateur non compatible, API désactivée ou contexte non sécurisé');
} else {
  const adapter = await navigator.gpu.requestAdapter();
  if (!adapter) {
    console.log('API présente, mais aucun adaptateur disponible');
  } else {
    console.log('shader-f16 :', adapter.features.has('shader-f16'));
    console.log('maxBufferSize :', adapter.limits.maxBufferSize);
    console.log('maxStorageBufferBindingSize :', adapter.limits.maxStorageBufferBindingSize);
  }
}

The shader-f16 line helps you pick the variant. WebLLM IDs generally distinguish q4f16 models, which require this feature, from q4f32 models, which don't. The two buffer limits tell you what the browser is willing to allocate; they can block a large model even if the GPU has enough memory.

#What the download involves

On first load, the page fetches the model files and the associated compiled code from the URLs in its configuration. In WebLLM's prebuilt list, these URLs point, to our knowledge, to external repositories such as Hugging Face and GitHub: verify this in the Network tab. Those hosts then see your IP address and the requested model, even if no message has been sent yet.

The files are then kept in browser storage, tied to the page's origin. MDN spells out the rules you need to know: by default, this storage is best-effort, and the browser can evict it under disk pressure without interrupting the user, starting with the least recently used origins. An evicted origin loses all its data at once. Safari can also delete script-created data for an origin that has had no interaction for seven days of browser use, when cross-site tracking prevention is enabled.

const { usage, quota } = await navigator.storage.estimate();
console.log('Utilisé :', usage, 'octets sur un quota estimé de', quota);

const persistant = await navigator.storage.persist();
console.log('Stockage persistant accordé :', persistant);

#Installing WebLLM in a page: the steps

This walkthrough assumes a JavaScript project with a build tool, such as Vite. Installing the package in your project, rather than importing it from a CDN, avoids adding a third-party domain to the page load and makes offline use easier.

npm install @mlc-ai/web-llm

1. 01
   Pin the version

   Note the installed version in package-lock.json and read the matching README. At that point, check the package license and the model license, which are two separate things.
2. 02
   List the available models

   Don't copy a model ID from an article: the list changes. Read it from the installed version, using the first snippet below. The memory field it shows is a starting point, not a guarantee.
3. 03
   Start small

   Start with the smallest useful model, in q4f32 if the previous test shows that shader-f16 is missing. A successful first attempt separates code problems from memory problems.
4. 04
   Show progress

   The download can take a while. Without visual feedback, the page looks stuck: initProgressCallback provides progress text you can display.
5. 05
   Move to a Web Worker

   Loading and generation can slow down the UI if they run on the main thread. For this, WebLLM provides CreateWebWorkerMLCEngine, with the WebWorkerMLCEngineHandler on the worker side. Plan for it before going live.
6. 06
   Test on target machines

   A result on a development workstation tells you nothing about a laptop with an integrated GPU. Show a clear message if WebGPU is missing, rather than a blank page.

import { prebuiltAppConfig } from '@mlc-ai/web-llm';

console.table(prebuiltAppConfig.model_list.map((m) => ({
  id: m.model_id,
  vramMo: m.vram_required_MB,
})));

import { CreateMLCEngine } from '@mlc-ai/web-llm';

// Remplacez par un identifiant relevé dans la liste ci-dessus
const modelId = 'IDENTIFIANT_DU_MODELE';

const engine = await CreateMLCEngine(modelId, {
  initProgressCallback: (p) => console.log(p.text),
});

const stream = await engine.chat.completions.create({
  messages: [{ role: 'user', content: 'Explique WebGPU en deux phrases.' }],
  stream: true,
});

let reponse = '';
for await (const chunk of stream) {
  reponse += chunk.choices[0]?.delta?.content || '';
}
console.log(reponse);

#Checking a WebLLM page: network, scripts, offline

At best, WebLLM guarantees where the computation runs. The rest depends on the page that embeds it: an analytics script, a widget, or a submit function can read the input field's contents and send them, since any script loaded in the page has access to it. This check applies to any site that advertises in-browser AI, including yours.

1. 01
   Set up a clean profile

   Open a browser profile with no extensions: an extension allowed on the page can read what's displayed there and skews the result. Open the Network tab and enable Preserve log.
2. 02
   Record the load

   Load the page, then note every domain contacted and its role: page, engine, model, font, analytics, advertising. Any domain without a clear role still needs to be explained.
3. 03
   Capture a message being sent

   Once the model is loaded, clear the log. Send a message containing a made-up marker, for example TEST-MARQUEUR-7Q2X, then search for it in the log using the Network tab's search. Also inspect every request sent during generation, including WebSocket messages.
4. 04
   Inspect the scripts and the security header

   In the Sources tab, list the domains scripts are loaded from. In the document's response headers, look for Content-Security-Policy: a strict connect-src directive narrows the possible destinations, but its absence proves nothing.
5. 05
   Check what's left on disk

   In the Application or Storage tab, open IndexedDB, Cache, and local storage. The model normally shows up there, but your conversations may be there too, in plain text for anyone using that session.
6. 06
   Actually cut the network

   Close the page, turn off Wi-Fi or unplug the cable, then reopen the URL and send a new message. The page should load and respond. Repeat after closing the browser.

These checks have their limits. A marker you can't find doesn't rule out data being sent encrypted or encoded by a script. WebRTC connections don't show up as ordinary requests in the Network tab: in Chrome, chrome://webrtc-internals lets you spot them. Finally, the result only holds for the version tested, since the site operator can change their scripts at any time.

You can say “works offline”

If reopening the page and sending a new message succeed with the network disconnected, on the tested browser, as of the test date. Note that the cache may be evicted.

You can say “no transmission observed”

If the log shows no request tied to the message, on the tested version and following the described protocol. This is a dated observation.

You can't say “completely private”

Unless you control all the code served and its updates, with no third-party scripts at all, and back this check up with a code review. Even then, the model host sees the download and extensions remain a risk.

#Serve everything from your own domain

For an internal tool, the easiest setup to control is serving the page, the engine, and the model from the same origin, with no third-party scripts at all. WebLLM accepts a custom configuration in which each model specifies the URL of its weights and of its compiled library. The paths below are examples to adapt.

import { CreateMLCEngine } from '@mlc-ai/web-llm';

const appConfig = {
  model_list: [{
    model: 'https://outil.example.fr/modeles/mon-modele/',
    model_id: 'mon-modele',
    model_lib: 'https://outil.example.fr/libs/mon-modele.wasm',
  }],
};

const engine = await CreateMLCEngine('mon-modele', { appConfig });

The engine builds the file URLs itself from this prefix. It may expect a directory structure similar to Hugging Face's: if requests fail with a 404, check the exact paths requested in the Network tab and replicate them on your server. The compiled library must match the model: don't pair it with an arbitrary one.

Once everything is self-hosted, a security policy can turn a promise into a technical constraint: according to MDN, connect-src restricts fetch, XMLHttpRequest, WebSocket, EventSource, and sendBeacon, among others, and falls back to default-src if it's absent. The following example hasn't been tested with WebLLM. Deploy it first in Content-Security-Policy-Report-Only mode and check the console.

Content-Security-Policy: default-src 'self'; script-src 'self' 'wasm-unsafe-eval'; worker-src 'self'; connect-src 'self'

This header also blocks downloads from Hugging Face: that's intentional, and the model must then be served from your domain. It doesn't cover every possible way out, such as navigating to another URL. For offline use, add a service worker that caches the page's files. WebLLM also provides an engine for this context, CreateServiceWorkerMLCEngine: see its README.

#Troubleshooting

navigator.gpu is undefined

First, check the address: it must be HTTPS or localhost. Then check MDN's compatibility table for your browser and operating system, and update both.

requestAdapter returns null

The API exists, but no suitable adapter is available. Update the driver and check that the browser's hardware acceleration is enabled. Try another browser to isolate the cause.

shader-f16 error

The selected model requires a feature your adapter lacks. Use its q4f32 variant if one exists.

QuotaExceededError

The origin's quota has been exceeded: MDN reports this error for IndexedDB, Cache, and OPFS. Free up space or use a smaller model, and catch the error in your code.

Crash at the end of loading or GPU lost

Probably insufficient memory or a buffer limit was reached. Close heavy tabs, drop down one model size, and check the exact message in the console.

The model downloads again

Private browsing, storage eviction, cleared site data, or a change of origin, since each domain has its own storage. To our knowledge, the package exposes hasModelInCache so you can check before starting the load.

The UI freezes during generation

The engine runs on the main thread. Switch to CreateWebWorkerMLCEngine.

Security policy error in the console

The message names the blocked directive and URL. Check whether that URL is legitimate before loosening the rule, especially if you advertise a page that sends nothing externally.

#Go further

If you're looking for an assistant for yourself rather than a feature built into a page, a standard local installation will be easier to control: the comparison of free solutions is the right place to start. For sensitive use, supplement the checks in this guide with the privacy checklist.