Dify, Self-Hosted: Building LLM Apps Without Writing the Plumbing

What Dify gives you

Everyone who builds a second LLM application writes the same scaffolding: a place to store prompts, a way to attach documents, a retrieval step, a conversation history, a log of what was asked and answered, and an HTTP endpoint the rest of the business can call. Dify is that scaffolding as a product, with a web interface in front of it.

Four things sit in the box:

• Apps — chat assistants, text generators, agents and workflows, each published with its own API key and a shareable web page.
• Workflow editor — a node graph where a request flows through retrieval, conditionals, model calls, code steps and tools.
• Knowledge base — upload documents, choose chunking and an embedding model, and attach the resulting index to any app. This is RAG with the moving parts exposed but not left to you.
• Observability — per-conversation logs, annotations and usage, which is what turns a demo into something you can debug.

Wiring it to a local model

Dify treats model providers as plugins, and Ollama is one of them. The configuration is short but has one recurring trap: inside Docker, localhost means the container, not your machine. The base URL must be the host address reachable from the container, and Ollama must be listening on the network interface rather than the loopback only — the Ollama service guide covers that setting.

You will configure two kinds of model, and forgetting the second is the usual reason a knowledge base returns nothing useful:

1. A generation model — the chat model that writes answers.
2. An embedding model — a separate, much smaller model that converts documents and questions into vectors. A chat model is not an embedding model; the knowledge base needs a real one.

For anything with multiple users or several workflow steps running at once, a server built for concurrency beats a desktop-oriented one. That is exactly the distinction drawn in Ollama vs vLLM in production.

What it costs to run

Component	Why it is there	Rough footprint
API + worker	Application logic, background jobs, document indexing	1–2 GB RAM
Database	Apps, prompts, conversations, users	Hundreds of MB, grows with logs
Vector store	Document embeddings for the knowledge base	Depends on corpus size
Cache / queue	Sessions and job queue	Small
Web front end + proxy	The interface you use	Small
The model	Served separately by Ollama or vLLM	The real cost — see the VRAM calculator

The official minimum for the platform is modest on paper — a couple of CPU cores and a few gigabytes of RAM — but that is for the containers only. On a single machine that also hosts a 27B model, plan the memory for the model first and give Dify what is left; document indexing is spiky and will contend with inference.

Dify, Flowise, Open WebUI or plain code

Your situation	Best fit
You want a chat interface over local models, nothing more	Open WebUI — see the GUI roundup
You want to build several internal apps with RAG and share them by API	Dify
You want a lighter visual flow builder	Flowise
You are shipping one application and own the codebase	Write it against the model API directly — the Ollama API in Python is a short read
You need agent teams with roles	A multi-agent framework such as CrewAI

Pitfalls worth knowing before you start

• No embedding model configured — the knowledge base indexes nothing useful and retrieval silently returns noise.
• localhost inside the container — the single most common Ollama connection failure.
• Context overflow — retrieved chunks plus system prompt plus history can exceed a local model's default window; raise it deliberately.
• Upgrades are real upgrades — a Compose stack with a database means reading migration notes rather than pulling the latest tag blindly. Back up before you do.
• It does not make a small model smarter — a good workflow around a 7B model still produces 7B answers.

Verdict

Dify is the right shape when several people need to build and run LLM applications on shared, private infrastructure and someone is willing to operate a Docker stack. It removes weeks of plumbing and gives non-developers a way to change a prompt without a deploy. For one person who wants to chat with local models, it is an order of magnitude more machinery than the job needs — install a front end instead, and come back to Dify when the second application appears.