Which LLM on GTX 1070 / 1080 (8 GB) ?

#GTX 1070, 1070 Ti, 1080: what sets them apart for an LLM

For an LLM, these three cards have the same capacity, 8 GB, and differ in memory bandwidth, which determines generation speed. The 1070 and 1070 Ti use GDDR5 on a 256-bit bus, for 256 GB/s; the 1080 reaches 320 GB/s with GDDR5X. Because an LLM rereads all its weights for every token, this 25% difference translates almost directly into speed, far more than the 1080's 2,560 cores versus the 1070's 1,920. None has Tensor Cores: Pascal remains limited to standard CUDA computation.

A GTX 1080 Ti, with 11 GB and 484 GB/s, belongs to another category: it has its own page. If your card is a 1080 Ti, read the guide dedicated to it instead.

• Which LLM on GTX 1080 Ti (11 GB)

#What 8 GB really enable

The key criterion is the ratio between the weights and the available memory, since the KV cache is added to them. The site's benchmark puts a 7–8B model in Q4 at around 5 GB of weights: on 8 GB, about 3 GB remains for context and the system, providing real headroom. A 9B model in Q4, at 6 GB, leaves 2 GB: it is possible with a short context. At 12 billion parameters, the weights reach 7 GB: almost nothing remains.

A theoretical ceiling is never reached. In the public benchmark, the GTX 1070 Ti reaches 37.82 tokens/s on a 3.56 GiB model, or about 56% of its 256 GB/s ceiling. Applying the same efficiency to the models above, a Granite 4.2 8B would run at around 30 tokens/s on a 1070 Ti: this is a proportional extrapolation, not a measurement.

#What public benchmarks say

QuelLLM does not test these cards. The figures come from the “Performance of llama.cpp on Nvidia CUDA” discussion in the llama.cpp repository, where each contributor runs llama-bench on Llama 2 7B in Q4_0 with all layers on the GPU. The GTX 1070 Ti reports 714.44 tokens/s for prompt processing and 37.82 tokens/s for generation.

For the cards on the same page that have a measurement, the comparison is useful: the RTX 2060 Super, an 8 GB, 256-bit Turing card, produces 60.04 tokens/s on the same test, versus 37.82 for the 1070 Ti. With similar memory bandwidth, the gain is 59%. The benchmark therefore explains the gap better than the core count or price: Turing makes better use of its memory.

These figures should be read with three caveats. Each benchmark row is an individual contribution: the driver, system, cooling, and card manufacturer vary from one contributor to another, explaining a few points of difference for the same chip. The test model, a Llama 2 7B, is older and smaller than current models: it serves as a common benchmark, not a prediction for any specific model. Finally, generation depends mainly on memory, while prompt processing depends on compute power: the two columns are measuring different things.

#Estimate GTX 1080 without measuring: method and limitations

The 1080 does not appear in the benchmark. To get an idea, we can apply the memory bandwidth ratio: 320 ÷ 256 = 1.25, or approximately 47 tokens/s (37.82 × 1.25) if efficiency remains identical to that of the 1070 Ti. This is a plausible assumption, not a result: drivers, the llama.cpp version, temperature, or card cooling can significantly affect throughput.

If you buy a used card for this purpose, ask the seller for a screenshot of a real test, or run llama-bench yourself on the benchmark model as soon as it arrives: the result can be compared directly with the public table.

#The context: where the 3 GB of headroom goes

On 8 GB, a Granite 4.2 8B in Q4 leaves about 3,4 GB. This reserve is shared by the KV cache, which grows with every conversation token, the computation buffers, and the space used by the driver and display if the card also drives your screen. Ollama starts with a 4 096-token window, adjustable through the OLLAMA_CONTEXT_LENGTH variable; doubling the window doubles the KV cache.

A simple rule of thumb: if you want to work with long documents (16,000 tokens or more), choose a model with 3 to 4 billion parameters, or an 8B model with a quantized KV cache. If you chat in short messages, the 8B in Q4 is the right choice. In both cases, use ollama ps to verify that the model is loaded 100% on the GPU: sharing with the processor indicates that you have run out of headroom.

• Quantize the KV cache to save VRAM
• Understanding the context window

#Flash Attention on Pascal: useful or not?

A common misconception is that Flash Attention is limited to cards with Tensor Cores. The llama.cpp benchmark disproves it: it runs every card with and without Flash Attention, and the GTX 1080 Ti, from the Pascal generation, appears in both tables. With Flash Attention, it goes from 1,084.41 to 1,138.14 tokens/s for prompt processing, about 5% better, and from 62.49 to 61.38 tokens/s for generation, 2% worse. Flash Attention's main benefit is not speed but memory: it reduces the context footprint, which matters on 8 GB.

Ollama automatically enables Flash Attention when the engine and GPU support it; the OLLAMA_FLASH_ATTENTION=1 variable forces it on, and 0 disables it. KV-cache quantization, which further reduces context memory usage, requires Flash Attention to be enabled.

OLLAMA_FLASH_ATTENTION=1 ollama serve

#What can you use an 8 GB Pascal for?

#Pascal: software support is the real limitation

The main risk comes not from speed but from the software. Ollama requires a NVIDIA 570 driver or newer for compute capability 5.0 to 6.2 cards, and the GTX 1070 through 1080 are included. But the CUDA 13 release notes state that Pascal has been removed, and Wikipedia notes that NVIDIA ended Game Ready support for this generation in December 2025, with security updates through October 2028.

Aujourd'hui

Ollama and llama.cpp work with driver 570 or later and CUDA 12 builds.

Tomorrow

New features and new engines will target CUDA 13: each update may drop Pascal.

Security

Driver security fixes are available through October 2028, but no new optimizations.

#Is a 1070 Ti better, or should I get something else?

For the same secondhand budget, the comparison comes down to three criteria: memory (8 GB minimum), architecture generation (Turing or newer to benefit from CUDA 13), and bandwidth. Without quoting prices, which change every week, here is how the options compare.

• Which LLM on RTX 2060 / 2060 Super
• Which LLM on RTX 3060 12 GB
• Which models for 8 GB of VRAM, all cards
• Compare GPU prices for AI

#Verdict: worth exploring, not worth investing in

If you already have a GTX 1070, 1070 Ti, or 1080, it’s an excellent entry point: 8 GB is enough for an 8B model in Q4, and throughput remains comfortable for chat. If you’re looking to buy, prioritize an 8 GB card with a Turing or newer architecture: at the same memory capacity, throughput is significantly better and software support lasts longer.

One final practical criterion: power consumption. These cards are 150 to 180 W models while gaming; during inference, they run cooler than in games, but a poorly ventilated case can reduce throughput. Check temperatures during a long generation before concluding that the card is slow.

• Source: public llama.cpp benchmark on CUDA
• Source: Ollama documentation, supported NVIDIA hardware
• Source: GeForce 10 series, specs
• Source: CUDA Toolkit release notes