What Is Hugging Face? A Practical Guide for Running Models Locally

What Hugging Face is

Hugging Face was founded in 2016 and started life as a chatbot app, which explains the emoji name. It pivoted to open-source tooling, and its transformers library became the standard way to load and run neural language models in Python. The company then built the Hugging Face Hub, a hosting platform built on Git where anyone can publish model weights, datasets and small web demos. As of September 20, 2026, it hosts well over a million public models. The official Hub documentation covers every feature; this page covers the part that matters for running models yourself.

For someone who wants to run AI on their own hardware, the Hub plays the role an app store plays for a phone, with two differences: almost everything is free, and nothing is curated for you. Knowing how to read it is the skill.

The three things on the Hub

Section	What it holds	Why a local-AI user cares
Models	Weight files plus configuration and documentation	This is where you get the file you will actually run
Datasets	Training and evaluation data	Only if you fine-tune or benchmark
Spaces	Small hosted web apps, usually demos of a model	Try a model in the browser before downloading 15 GB

The company also sells hosted inference, private repositories and enterprise features. None of it is needed to download open models.

Anatomy of a model page

Every model lives at huggingface.co/<organization>/<model-name>, for example huggingface.co/Qwen/Qwen3-8B. Four parts of the page are worth your attention.

• The organization name. Qwen, google, meta-llama, mistralai, openai, deepseek-ai are the labs themselves. Anything else is a third party: possibly excellent, but not the original.
• The model card. The README: what the model is, how it was trained, intended use, benchmark scores, prompt format and known limitations. Quality varies from thorough to empty.
• The license tag. Shown at the top of the card. More on this below.
• Files and versions. The actual downloads. This tab tells you immediately what kind of repository you are in.

Which repository do you need? Original weights vs GGUF

You see in "Files"	It is	Runs in	Size for an 8B model
model-00001-of-00004.safetensors …	Original weights, usually BF16, split into shards	transformers, vLLM, fine-tuning tools	≈ 16 GB
*-Q4_K_M.gguf, *-Q8_0.gguf …	Quantized single-file conversions	Ollama, LM Studio, llama.cpp, KoboldCpp, Jan	≈ 5 GB at Q4
*-AWQ, *-GPTQ, *-FP8 repositories	GPU-oriented quantization	vLLM and other server engines	≈ 5–9 GB
*-MLX-4bit repositories	Apple Silicon format	MLX, LM Studio on Mac	≈ 5 GB

The labs mostly publish the first kind. The GGUF files that desktop tools need are produced by converters: some labs publish their own, and community publishers such as bartowski, unsloth and ggml-org cover nearly everything else within hours of a release. To find them, open the original model page and look at the Quantizations link in the model tree on the right, or search the model name plus "GGUF." The format itself is explained in what is GGUF, and what serving engines do with the original weights in what is vLLM.

Picking the right file for your hardware

A GGUF repository typically lists ten or more files for the same model, one per quantization level. The file size is, to a close approximation, the VRAM the weights will occupy. Leave 1–3 GB free for context and overhead.

Model	Q4_K_M	Q5_K_M	Q8_0	BF16	Smallest comfortable GPU at Q4
Qwen 3 8B	5 GB	6 GB	9 GB	16 GB	8 GB
Gemma 4 12B	7 GB	9 GB	13 GB	24 GB	12 GB (8 GB with short context)
Qwen 3 14B	9 GB	11 GB	16 GB	28 GB	12 GB
gpt-oss 20B	13 GB	16 GB	23 GB	42 GB	16 GB
Mistral Small 3.2 24B	14 GB	17 GB	26 GB	48 GB	16 GB
Qwen 3.8 27B	16 GB	19 GB	29 GB	54 GB	24 GB
Llama 3.3 70B	40 GB	48 GB	75 GB	140 GB	2 × 24 GB

From the BestLLMfor model catalog, September 20, 2026. Sizes rounded up to the next GB.

Q4_K_M is the usual starting point: the largest drop in size for the smallest drop in quality. Move up to Q5 or Q8 if you have memory to spare; the quality differences are measured in Q4 vs Q5 vs Q8. For any other model and context length, the VRAM calculator does the arithmetic, and what is VRAM shows how to check what your card has.

Three ways to download

1. Let your runtime do it

The simplest path. LM Studio's search box queries Hugging Face directly and shows which files fit your machine. Ollama and llama.cpp can both pull a GGUF repository by name:

# Ollama: run any GGUF repository straight from the Hub
ollama run hf.co/bartowski/Qwen_Qwen3-8B-GGUF:Q4_K_M

# llama.cpp: download and serve
llama-server -hf bartowski/Qwen_Qwen3-8B-GGUF:Q4_K_M

2. The command-line tool

pip install -U huggingface_hub
hf download bartowski/Qwen_Qwen3-8B-GGUF --include "*Q4_K_M.gguf" --local-dir ./models

The --include filter matters: without it you download every quantization in the repository, often more than 100 GB.

3. The browser

Open "Files and versions" and click the download arrow next to one file. Fine for a single GGUF; impractical for sharded original weights.

Gated models. Some labs (Meta's Llama family, some Google and Mistral releases) require you to accept their terms first. Create a free account, click "Agree and access" on the model page, then authenticate the command-line tool once with hf auth login and an access token from your account settings. Community GGUF conversions of the same models are usually not gated, but the original license still applies to you.

Licenses: open does not always mean unrestricted

License	Examples in our catalog	Commercial use
Apache 2.0	Qwen 3 family, Gemma 4 12B, gpt-oss, Mistral Small 3.2	Yes, with attribution
MIT	DeepSeek R1 distills, GLM 4.7 Flash	Yes
Llama Community License	Llama 3.3 70B	Yes, with conditions: naming rules, acceptable-use policy, a user-count threshold
Non-commercial / research	Various research releases	No

The tag on the model card is the authoritative answer, and a fine-tune inherits the obligations of its base model. If you are deploying in a company, this is worth two minutes before you build on a model.

Is it safe to download from Hugging Face?

The platform is legitimate and scans uploads, but it is an open host, so judgment still applies.

• Prefer .safetensors and .gguf. Both are pure data formats (safetensors documentation). The older PyTorch .bin and .pt files use Python's pickle mechanism, which can run arbitrary code when loaded. The Hub flags these with a warning.
• Be wary of "trust remote code." Some repositories ship custom Python that the loader asks permission to execute. Only allow it for organizations you recognize.
• Check who published it. Download counts, the organization's other repositories and a link back from the lab's own page are good signals. Impersonation accounts exist.
• The model stays local. Once downloaded, a GGUF running in llama.cpp or Ollama makes no network calls. Your prompts do not go to Hugging Face.

Hugging Face vs the Ollama library vs LM Studio

They are layers of the same stack, not alternatives. Hugging Face is the warehouse. The Ollama library is a small, curated shelf of popular models repackaged with sensible defaults. LM Studio is a storefront that searches the warehouse for you. If a model is brand new, niche, uncensored, or a specific quantization, you will end up on the Hub. If you are just getting started, Ollama or the beginner's guide to running an LLM locally is the faster road, and the best Ollama models ranking tells you what to pull first.

The model sizes and licenses on this page come from the BestLLMfor catalog, which is open through our public API (CC BY 4.0) and MCP server.