How much does an LLM GPU server cost? Buying, renting, API

#The 3 options and their cost profiles

Before discussing numbers, you need to understand that the cost of a GPU server for LLMs falls into three very different economic models. The right choice depends not only on your budget, but especially on your usage volume and the sensitivity of your data.

Purchase (CAPEX)

You invest a significant amount upfront to own the machine. The marginal cost afterward is nearly zero (electricity). Profitable if you run it a lot and for a long time.

Rental (monthly OPEX)

A dedicated GPU server or a VM from a hosting provider, billed monthly or hourly. No upfront investment, but a recurring bill as long as the machine is running.

Cloud API (token-based OPEX)

You manage no machines; you pay based on consumption (per million tokens). Zero fixed cost, but the price rises linearly with usage.

#Buy a dedicated machine

The purchase is the classic self-hosting model: a machine at home or in your equipment room, amortized over 2 to 4 years. The cost is concentrated in the GPU, which often accounts for more than half of the machine’s total budget.

Entry-level GPU — RTX 3060 12 GB

About €300 used. Comfortably runs 7B–14B models in Q4. The entry ticket for a serious local LLM.

Versatile GPU — RTX 4070 / 4080 16 GB

€700 to €1,200. Smooth 14B models; 32B in Q4 with some headroom (19 GB of VRAM on the 4080… right at the limit).

High-end GPU — RTX 4090 24 GB

€1,600 to €2,000. Runs a 32B Q4 (≈19 GB) with ease and a 70B with aggressive quantization. The workstation reference.

Unified memory — Mac Studio / M4 Pro 48–128 GB

€2,000 to €6,000. Unified VRAM lets you load 70B models, or even larger ones, without multi-GPU. Quiet and energy-efficient.

Then there's the rest of the machine: motherboard, CPU, 32 to 64 GB of RAM, a substantial power supply (750 W+ for a 4090), and a well-ventilated case. Budget €500 to €900 for a decent PC foundation around the GPU. A complete GPU workstation capable of running 32B models therefore costs between €2,500 and €3,500 all-inclusive.

#Rent a GPU server from a hosting provider

Renting avoids the initial investment and the problem of obsolescence. There are two subcategories: a dedicated monthly GPU server (you reserve the machine continuously) and an hourly cloud GPU (you pay only for inference hours).

Monthly dedicated GPU — French host

At OVHcloud or Scaleway, a server with a dedicated GPU costs anywhere from a few hundred to more than a thousand euros per month, depending on the card (L4, L40S, H100). Data hosted in France, with sovereignty and GDPR as selling points.

GPU per hour — spot cloud

On platforms such as RunPod, Vast.ai, and Lambda, a GPU costs €0.20 to €3/hour to rent, depending on the card. Ideal for occasional batch jobs or fine-tuning, not for 24/7 use.

General-purpose cloud GPU VM

AWS, GCP, and Azure bill GPU instances by the hour, often at a higher price than specialists, but with the ecosystem and SLA of a major cloud provider.

Simple rule: for intermittent workloads (a few hours a day, batch jobs, experimentation), hourly rental wins. For a service that must respond continuously, a dedicated monthly GPU or purchasing takes the lead starting in the second or third month.

#Use a cloud API

The API is the serverless option: you neither buy nor rent a GPU; you pay for the tokens consumed. It’s the most rational starting point when volume is low or unpredictable, because the fixed cost is zero.

Billing model

Price per million input and output tokens. Output tokens generally cost more than input tokens.

Advantage

Zero maintenance, zero obsolescence, immediate access to 100B+ models that are impossible to run on a workstation.

Cost drawback

The price rises linearly with usage. A pipeline that processes 100,000 documents per day turns a “cheap” API into a four-figure monthly bill.

Privacy drawback

Your prompts leave your infrastructure. That is a deal-breaker for sensitive data (healthcare, legal, proprietary code) without a suitable dedicated cloud and contract.

#The self-hosting vs. API break-even point

This is the central calculation. A purchased GPU is a fixed cost; an API is a variable cost. There is therefore a token volume beyond which owning your own server costs less than paying as you go. Below that threshold, the API wins; above it, self-hosting pays for itself.

The intuition: a RTX 4090 costing 1,800 € and amortized over 3 years comes to about 50 €/month in hardware costs, plus electricity. If your API usage exceeds that amount each month, buying becomes cost-effective—often after just a few million tokens per day for recurring use. For occasional use of a few requests per day, the API remains unbeatable.

Low volume, non-sensitive data

API. The fixed cost of a server is not justified for a few thousand tokens per day.

High and recurring volume

Purchase. The GPU pays for itself in a few months, and the marginal cost per request drops to nearly zero.

Sensitive data, regardless of volume

Self-host (purchase or rental in France). Confidentiality decides the matter before you even calculate the cost.

One-off spike or fine-tuning

Hourly rental. You pay for the compute power only while the operation is running.

#Typical configuration by budget

Here are three self-hosted machine profiles corresponding to three levels of budget and ambition. The price ranges are rough 2026 estimates, including complete hardware.

1. 01
   GPU workstation — €2,500 to €3,500

   A 24 GB RTX 4090 (or a used RTX 3090 to halve the price) in a PC with 64 GB of RAM. Runs a 32B Q4 (≈19 GB of VRAM) with ease and a 70B with aggressive quantization. The benchmark choice for a developer or a small team.
2. 02
   Dedicated GPU server — €4,000 to €8,000 or monthly rental

   A professional card (L40S 48 GB, or 2× RTX 4090 in a tensor split) in a rack chassis designed to run 24/7 and serve multiple users through Open WebUI. Available for purchase or rental from a French hosting provider if you don’t want to manage the hardware.
3. 03
   Unified-memory mini PC — €700 to €6,000

   A Mac mini M4 (starting at ~€700) for a quiet, energy-efficient inference server, or a Mac Studio M5 Ultra (96 GB and up, 512 GB announced for late October 2026) to run 70B–123B models without multi-GPU. Negligible power consumption compared with a NVIDIA GPU.

#Hidden costs not to overlook

The purchase price is only the visible part. Three recurring costs are systematically underestimated when budgeting for an LLM GPU server.

Electricity

A RTX 4090 draws up to 450 W under load. With near-continuous inference, that amounts to several dozen euros per month depending on the kWh rate. A Mac with unified memory uses a fraction of that—a real argument for 24/7 operation.

Maintenance and availability

A server that must respond continuously requires monitoring, updates, and failure management. That is human time— invisible on the bill but very real.

Obsolescence

A GPU loses value and models evolve quickly. Amortizing it over 3 years is reasonable, but with a 2026 14B model surpassing a 2024 70B model, the hardware race is not always necessary—sometimes a newer, smaller model is enough.

Cooling and noise

A high-end GPU generates heat and noise. In a professional local setup, plan for room ventilation; at a workstation, acoustic comfort matters.

#Go further

Three related guides to refine your decision: the GPU selection guide details the VRAM/budget/card-by-card performance tradeoffs; the cost calculator precisely determines your self-hosting vs. API break-even point; and the 32 GB VRAM PC build guide lists the specific components for a machine capable of running 32B models.

Choose your GPU for local AI

The 2026 buying guide: RTX 4070 vs. 4090 vs. Mac M-Max, VRAM and budget tradeoffs.

LLM cost calculator

Calculate your self-hosted vs. API switching threshold based on your token volume.

AI PC build: 32 GB VRAM budget

Selecting components to build a machine capable of running 32B models locally.