Updated September 2026

The Ollama API in Python

Verdict (September 2026): The cleanest path is the official ollama Python package talking to the local server on port 11434. Use client.chat() for conversational turns, pass stream=True for token-by-token output, and client.embed() for vectors. Wrap calls in try/except ollama.ResponseError and catch connection errors separately — a missing model or a stopped server are the two failures you will actually hit in practice.

Setup and the API surface on port 11434

Ollama installs a background HTTP server that listens on 127.0.0.1:11434. That single port is the entire API surface — the CLI, the desktop app, and your Python code all talk to it. If you want the mental model first, see what Ollama is; here we go straight to code.

Install the official client with pip install ollama. It is a thin wrapper over the HTTP endpoints, so anything you can do with curl against port 11434 you can do in Python. Confirm the server is up by opening http://localhost:11434 in a browser — it returns Ollama is running. If it does not, start it with ollama serve (the desktop app starts it for you). To bind a different host or port, set the OLLAMA_HOST environment variable before launching the server.

The endpoints you will use most:

Pull a model once before you call it — ollama pull <model-tag> — and swap <model-tag> below for something you have actually downloaded. Exact tags change often, so check the Ollama model picks or the official ollama-python repo rather than trusting a hard-coded name.

The chat endpoint: your first request

The chat endpoint takes a list of messages, each with a role and content, in the familiar OpenAI-style format. Here is a complete round-trip against port 11434:

from ollama import Client MODEL = "<model-tag>" client = Client(host="http://localhost:11434") resp = client.chat( model=MODEL, messages=[ {"role": "system", "content": "You are concise."}, {"role": "user", "content": "Explain port 11434 in one line."}, ], ) print(resp["message"]["content"])

The response is a dictionary; the text lives at resp["message"]["content"]. You can also read resp["eval_count"] and resp["eval_duration"] to compute tokens per second, which is the number that actually matters on consumer hardware. Ollama is stateless between calls, so to hold a conversation you keep appending assistant and user messages to the same list and resend it — you own the history, not the server. The endpoint also accepts an options dict for sampling controls like temperature and num_ctx.

Streaming responses token by token

For anything interactive, stream. Pass stream=True and iterate: each chunk carries a slice of the answer under the same message.content key, and a final chunk sets done to true along with timing stats.

stream = client.chat( model=MODEL, messages=[{"role": "user", "content": "Write a haiku about GPUs."}], stream=True, ) for chunk in stream: print(chunk["message"]["content"], end="", flush=True) print()

Streaming does not make generation faster, but it makes it feel faster: the first token shows up in tens of milliseconds instead of after the whole reply is done. Set flush=True on your prints so the terminal does not buffer output. If you accumulate the chunks into a string, remember to also capture the final chunk's stats if you want to log throughput.

Generating embeddings

Embeddings turn text into vectors for search, clustering, or retrieval-augmented generation. Use a dedicated embedding model — a chat model will not give you good vectors — and call client.embed():

EMBED_MODEL = "<embedding-model-tag>" result = client.embed( model=EMBED_MODEL, input=["first document", "second document"], ) vectors = result["embeddings"] print(len(vectors), "vectors of dim", len(vectors[0]))

The call returns an embeddings list; each vector's length is fixed by the model, commonly from a few hundred to a couple thousand dimensions. Batch by passing a list to input — one HTTP round-trip is far cheaper than looping a single string at a time. Store the vectors in any database that does cosine similarity and you have local semantic search with no data leaving port 11434. The raw endpoint is /api/embed if you prefer a plain requests.post call.

Sizing: will the model fit?

Before you pick a model tag, check it fits in VRAM. When it does not, Ollama offloads layers to the CPU, and that quietly tanks your tokens per second. Use the standard rules of thumb: FP16 weights cost about 2 GB per billion parameters, Q8_0 about 1.07, and Q4_K_M about 0.58, then add roughly 20% for the KV cache and overhead at an 8K context.

Those figures are weights only; add ~20% for real use. On a 16 GB card a 7–8B model at Q4_K_M leaves comfortable headroom, while FP16 does not fit at all. Run the numbers for your own card with the VRAM calculator, and if you are weighing runtimes, see Ollama vs llama.cpp.

Error handling that survives a cold start

Two failures dominate in practice: the model is not pulled, and the server is not running. The library signals the first with ollama.ResponseError (a 404 carrying a helpful message) and the second with a connection error from the underlying httpx client. Handle them separately so the message you log is actionable:

import ollama, httpx try: resp = client.chat(model=MODEL, messages=msgs) except ollama.ResponseError as e: print("API error:", e.error, "status", e.status_code) except httpx.ConnectError: print("Cannot reach Ollama on port 11434 - is 'ollama serve' running?")

Add a generous timeout when you load a large model for the first time. The initial call blocks while weights move into VRAM, which can take many seconds; pass timeout to a custom Client (it forwards to httpx) rather than letting a short default kill a legitimate cold start. For the exact request and response fields, keep the official Ollama API docs open — the shapes are stable, but new options land regularly, so verify against your installed version.

Frequently asked questions

What port does the Ollama API use?

By default the Ollama server listens on port 11434 at 127.0.0.1, and that single port serves every endpoint. You can change the bind address or port by setting the OLLAMA_HOST environment variable before starting the server. Opening http://localhost:11434 in a browser returns "Ollama is running" when it is up.

How do I call the Ollama API in Python?

Install the official client with pip install ollama, create a Client pointed at http://localhost:11434, and call client.chat() with a list of role/content messages. The response is a dictionary whose text lives at resp["message"]["content"]. Under the hood the library just makes HTTP calls to the local server, so you can also use plain requests if you prefer.

Does the Ollama Python library support streaming?

Yes. Pass stream=True to client.chat() or client.generate() and iterate over the result; each chunk carries a slice of the reply under message.content, and the final chunk sets done to true with timing stats. Streaming does not speed up generation but it delivers the first token almost immediately.

How do I fix a connection refused error on port 11434?

A connection refused or httpx.ConnectError almost always means the Ollama server is not running. Start it with ollama serve (or launch the desktop app, which starts it for you) and confirm http://localhost:11434 responds. If you changed OLLAMA_HOST, make sure your client points at the same host and port.

---

By Mohamed Meguedmi — independent comparator of locally-runnable LLMs, benchmarked on a real RTX 5070 Ti (data CC BY 4.0). See the local LLM leaderboard and the best Ollama models.