Aya 23 8B

Overview

Cohere For AI's pre-Expanse 8B multilingual model covering 23 languages, now superseded by Aya Expanse 8B and restricted to non-commercial use.

When to pick this model

• Reproducing pre-Expanse multilingual baselines
• Research comparisons of Cohere multilingual lineage
• Non-commercial multilingual prototypes
• Teaching examples for multilingual instruction tuning

VRAM requirements by quantization

VRAM figures include model weights plus a typical 8k KV cache and ~600 MB runtime overhead (Ollama / llama.cpp baseline). Add headroom for higher context lengths.

In practice, Aya 23 8B fits an 8 GB consumer card at Q4_K_M (5 GB). Stepping up to Q8_0 nearly doubles the footprint to 9 GB, and unquantized FP16 weights take 16 GB — plan your GPU around the Q4 or Q5 figure unless you specifically need the higher fidelity.

Without a GPU, Aya 23 8B needs roughly 10 GB of system RAM to run on CPU via llama.cpp or Ollama — workable for background jobs, but far slower than GPU inference. Throughput estimates from our compatibility engine: around 10 tokens/sec on entry-level GPUs, on the order of 30 tokens/sec on a mid-range card, and up to 80 tokens/sec on high-end hardware — assuming the chosen quantization fully fits in VRAM.

What hardware do you need

The table below matches Aya 23 8B to common GPU memory tiers, using the highest-fidelity quantization that fully fits each card class. Spilling layers to system RAM works but costs most of the speed, so size your card to the quantization you actually want to run.

Which GPU should you buy to run Aya 23 8B?

To run Aya 23 8B locally at Q4, you need ~5 GB of VRAM. The best value for this is a RTX 5060 (8 GB VRAM).

Strengths

• Solid pre-Expanse multilingual coverage
• 23 languages including French, Arabic, and Chinese
• Compact 8B footprint for the breadth

Limitations

• CC-BY-NC 4.0 license blocks commercial use
• 8k context is restrictive by current standards
• Superseded by Aya Expanse 8B in every dimension

Typical workloads

In our catalog grid, Aya 23 8B is filed under Historical Multilingual — the use cases where its size/quality trade-off makes the most sense. Its tags translate to concrete workloads: multilingual workloads.

Note the 8k-token context window — fine for short interactions, limiting for long documents or big retrieval contexts. It ships under the CC-BY-NC 4.0 license — commercial use is generally possible but read the specific terms before embedding it in a product.

Architecture & training

Architecture: Dense 8B · IFT on Aya Collection

Training: 23 languages.

Quick start

ollama run aya:8b

Or use the open-source MCP server to query this model from Claude Desktop, Cursor, or any MCP-compatible client.

Similar models worth comparing

Frequently asked questions

How much VRAM does Aya 23 8B need?

At the recommended Q4_K_M quantization, Aya 23 8B needs about 5 GB of VRAM. Q8_0 takes 9 GB, and unquantized FP16 weights take 16 GB.

Can Aya 23 8B run without a GPU?

Yes — with roughly 10 GB of system RAM it runs CPU-only through llama.cpp or Ollama. Expect a fraction of GPU speed, which is fine for background or batch jobs but slow for interactive chat.

What context window does Aya 23 8B support?

Aya 23 8B supports a 8k-token context window (8,192 tokens).

Can I use Aya 23 8B commercially?

Aya 23 8B ships under the CC-BY-NC 4.0 license. Commercial use is generally permitted subject to its terms — review the license text before shipping a product.

How fast is Aya 23 8B on consumer hardware?

Our compatibility engine estimates on the order of 30 tokens/sec on a mid-range GPU and up to 80 tokens/sec on high-end cards, assuming the quantization fully fits in VRAM.

Which quantization of Aya 23 8B should I download first?

Start with Q4_K_M (5 GB) — the standard size/quality sweet spot. Step up to Q5_K_M or Q8_0 only if you have VRAM headroom. It already fits an 8 GB card at Q5_K_M.