Hermes Agent With a Local Ollama Endpoint

What Hermes Agent actually is

Hermes Agent (repository NousResearch/hermes-agent) is Nous Research's self-improving agent framework: a terminal UI, a messaging gateway (Telegram, Discord, Slack, WhatsApp, Signal), persistent memory, and a skills system that creates and refines its own procedures from experience. It is distinct from the Hermes model family — Hermes Agent is the orchestration software, and any model, local or hosted, can sit behind it.

The project ships fast: tag v2026.9.24 (Hermes Agent v0.21.5), published September 24, 2026, rolls up several hundred merged pull requests since the prior tagged release. The repository is released under the MIT license by Nous Research.

The agent runs as a single gateway process that can serve several channels from the same memory and conversation state — terminal, Telegram, Discord, Slack, WhatsApp, Signal. You can start a task at your desk and keep following it from a phone without reconfiguring the model or losing context.

Connecting a local Ollama endpoint

Hermes Agent routes self-hosted OpenAI-compatible servers — Ollama, vLLM, llama.cpp — under their own provider names. The official configuration documentation is explicit about the expected shape: a base_url pointing at the server, and an empty API key acting as a placeholder.

auxiliary:
  compression:
    provider: ollama
    model: qwen3.6:27b-q4_k_m
    base_url: http://127.0.0.1:11434

A bare host:port base_url gets the OpenAI-compatible /v1 suffix appended automatically — no need to type it. The same mechanism applies to vLLM and llama.cpp by swapping the provider name and port.

Different roles can point at different providers in the same config: the main conversation model, the context-compression model, and the session-title model. A common pattern is a local Ollama model for the conversation itself, with a faster cloud model reserved for auxiliary tasks — keeping the bulk of the conversation private without letting a small local model slow down background jobs.

Which model actually calls tools

Hermes Agent exposes more than 40 tools — terminal, files, browser, memory, cron — that the model must invoke through structured calls, not describe in prose. An Ollama model that was never fine-tuned for tool calling, or the same model pushed to an aggressive quantization, frequently answers with an explanation instead of triggering the call.

• Main conversation model: prefer an Ollama variant explicitly listed as tool-capable (the tools tag on ollama.com/library) over a generic chat build.
• Auxiliary model (titles, compression): can stay light. On a custom local provider, Hermes Agent sends this call after the turn's reply has arrived, not concurrently — a single-slot local server otherwise risks answering the reply with the title request's JSON instead.
• Model id format: a named provider can prefix the model, e.g. ollama-local/qwen3.6:27b-q4_k_m; that prefix also applies when only the bare model id reaches the server.

That sequencing detail — title generation deferred until after the reply — is easy to miss and rarely documented elsewhere, but it is exactly what keeps a single-slot local Ollama instance from mixing up two concurrent requests.

Memory: what actually persists

Hermes Agent's memory is plain, editable text: MEMORY.md and USER.md under the profile directory. The model decides, through a dedicated tool, what is worth writing there — preferences, durable facts, learned procedures.

cat ~/.hermes/memories/MEMORY.md
cat ~/.hermes/memories/USER.md

The documented weak point: a small local model can say "saved" without the write tool ever firing. For a local model under roughly 30B parameters, or one with weak tool-calling, the official guidance is to ask explicitly for the memory tool and confirm the entry landed in the file — not to pile on more instructions in the prompt. Once the entries exist, a smaller model reads them fine, since they arrive directly in the system prompt.

Security: eight layers, not a wall

Nous Research documents an eight-layer security model: user authorization, human-in-the-loop approval for dangerous commands, file-write safeguards, container isolation (Docker, Singularity, Modal), MCP credential filtering, prompt-injection scanning in context files, cross-session isolation, and input sanitization on terminal working directories.

Dangerous command approval is set via approvals.mode in the config file, with three modes: fully automatic, manual, or the recommended default, smart, which auto-approves low-risk actions and flags the rest.

What a small local model doesn't do well here

An agent this rich in tools stresses a model's instruction-following hard: memory, planning, chained tool calls, and sometimes parallel subagents. On a 7B–14B model at Q4_K_M, expect missed tool calls, confirmations with no actual execution, and more frequent context compression on long sessions.

Feature	Typical failure with a small local model
Memory tool	Verbal confirmation with no actual file write
Chained tool use	The model narrates the next step instead of running it
Parallel subagents	Instructions get lost between agents, results need manual cross-checking
Long sessions	More frequent context compression, older details drop out

None of this is specific to Hermes Agent — it is the well-known ceiling of small quantized models against structured output formats. The project's own fix is not more prompting, but a stronger model at least for initial setup; once memory entries exist, a lighter model can read them without issue. Subagents and RPC-driven Python scripts — both used to parallelize work — rely on that same instruction-following capacity, so validate plain single-tool behavior before turning them on.

Setting it up in practice

1. Install Hermes Agent. On Linux, macOS or WSL2: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash, then reload the shell and run hermes.
2. Prepare the Ollama model. Confirm Ollama is listening on port 11434 and that the chosen model is advertised as tool-capable before wiring it up as a provider.
3. Declare the local provider. Use hermes config set to set provider: ollama, the model name, and base_url: http://127.0.0.1:11434 with an empty API key.
4. Test one tool call. Ask for a verifiable action (list a folder, read a file) and confirm in the terminal that the tool actually ran, not just that the agent described it.
5. Check memory for real. Ask the agent to remember something, then open MEMORY.md to confirm the entry is there before trusting it on this point.

Related reading and sources

For giving a local model tools outside Hermes Agent's own tool system, see Ollama + MCP: Give Your Local Model Tools and What Is MCP? Model Context Protocol Explained Simply. If tool calling itself is unreliable regardless of the agent, Claude Desktop + Local LLM via MCP covers a comparable setup with a different client.

Primary sources used in this guide: the Hermes Agent GitHub repository, the provider configuration docs, the security model docs, and the memory docs.