Local fine-tuning on GB10: measurements and method

#The real-world case: reading invoices and producing JSON

The task is common in a small business: from the text of an invoice, quote, or credit note, the model must return JSON, meaning a structured record that software can read. This record has 31 scored fields: document type, number, dates, issuer, customer, product lines, discount, totals, VAT, deposit, IBAN. We chose this task because the answer can be checked field by field, without another model having to judge it.

The documents are fictional. A small program generated 12 layouts, meaning 12 invoice formats, with made-up company names. Its randomness is set by a fixed "seed," so it produces the same documents on every run. Training uses 2,000 documents across 10 layouts.

The evaluation uses 200 never-seen documents: 100 on the same 10 layouts, and 100 on 2 layouts absent from training. Those last 100 show whether the model understood the task or merely memorized patterns.

The model receives the document's text, not an image or a PDF. Here is an excerpt of the trained model's response on the first test document, a fictional four-line credit note.

{
  "type_document": "avoir",
  "numero": "AV-2025-0807",
  "date_emission": "2025-05-12",
  "emetteur": { "raison_sociale": "Menuiserie Vassouze SA", "code_postal": "38000", "ville": "Grenoble" },
  "lignes": [
    { "designation": "Ponçage et vitrification", "quantite": -58, "unite": "m²",
      "prix_unitaire_ht": 36.0, "taux_tva": 10, "montant_ht": -2088.0 },
    …
  ],
  "total_ht": -6356.0,
  "total_tva": -911.4,
  "total_ttc": -7267.4,
  "devise": "EUR"
}

#The method: LoRA, one epoch, 85 minutes

The machine is a GIGABYTE AI TOP ATOM, model ATAGB10-9000: NVIDIA GB10 chip, 128 GB of unified memory, a 4 TB PCIe 5.0 SSD, running DGX OS 7.5.0 (driver 580.159.03, CUDA 13.0). The software: the NVIDIA PyTorch container (version 26.03, with PyTorch 2.11), PEFT 0.21.2 for LoRA, Unsloth 2026.10.1 for QLoRA, and, for serving, the NVIDIA vLLM 26.09 container (vLLM 0.29.0). Our first guide in the series covers the machine and the speed of 13 models.

LoRA (Low-Rank Adaptation) leaves the original model intact and trains small matrices added to its layers. The result is an adapter, a small file you load with the model. Here, 43.6 million parameters are trained, compared with 8 billion for Qwen3-8B.

Three terms come up throughout this guide. A training step processes a batch of 4 documents. An epoch is one complete pass over the 2,000 documents. Loss measures the gap between the answer written by the model and the expected answer: the lower, the better.

Model

Qwen3-8B in bf16, a 16-bit number format, with no compression.

LoRA

Rank 16, alpha 32 (a scaling coefficient), dropout 0.05 (a small share of values is randomly ignored at each step to limit overfitting), on the 7 projections of each layer, meaning its 7 weight matrices: q, k, v, o, gate, up, down. The rank sets the size of the added matrices.

Duration

1 epoch: 500 steps of 4 documents.

Learning rate

1e-4, or 0.0001, with a gradual warmup over 30 steps followed by cosine decay.

Lots

We sort the 2,000 documents by length, form the batches, then shuffle their order with a fixed seed. Each batch contains documents of similar length, which means less unnecessary padding and therefore faster training.

Perte

Computed on the JSON response alone. The prompt and the document are masked: the model learns to write the response, not to copy the document.

Length

Up to 2,304 tokens. The longest document has 2,204 and the average is 1,143: none is truncated.

Memory

Gradient checkpointing is enabled: the model recomputes some results instead of keeping them all in memory.

These settings were fixed by an amendment to our protocol, dated October 7, 2026, before any evaluation on the 200 test documents. The protocol was frozen before the first measurement; the method, the dated list of amendments, and the data behind our tables are published on our methodology page. Memory figures in this guide are in GB, as Linux displays them: 1 GB = 1,024 MB.

Everything happens in the NVIDIA PyTorch container (version 26.03). A Python environment inside it gets the transformers and PEFT libraries. The paths below are simplified; the options are the ones from our test runs.

mkdir -p ~/venv-ft ~/resultats   # à créer d'abord : sinon Docker les crée au nom de root
docker run --rm --gpus all --ipc=host --user $(id -u):$(id -g) -e HOME=/tmp \
  -v ~/venv-ft:/venv nvcr.io/nvidia/pytorch:26.03-py3 \
  bash -lc "python -m venv --system-site-packages /venv; . /venv/bin/activate; pip install -q 'transformers>=4.56' peft==0.21.2 accelerate"

docker run --rm --init --gpus all --ipc=host --user $(id -u):$(id -g) -e HOME=/tmp \
  -v ~/venv-ft:/venv -v ~/models:/models:ro -v ~/donnees:/data:ro \
  -v ~/scripts:/s -v ~/resultats:/res \
  nvcr.io/nvidia/pytorch:26.03-py3 \
  bash -lc ". /venv/bin/activate; python /s/entrainer_factures.py /res/adapter"

model = AutoModelForCausalLM.from_pretrained(M, dtype=torch.bfloat16, device_map="cuda")
model.gradient_checkpointing_enable(); model.config.use_cache = False; model.enable_input_require_grads()
model = get_peft_model(model, LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, task_type="CAUSAL_LM",
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]))

# perte sur la réponse seule : les tokens de la consigne reçoivent l'étiquette -100 et sont ignorés
ids = (pi + ri)[:LMAX]; lab = ([-100] * len(pi) + ri)[:LMAX]

# lots de longueur voisine : tri par longueur, lots de 4, ordre des lots mélangé (graine fixe)
tri = sorted(range(len(exemples)), key=lambda i: len(exemples[i][0]))
lots = [tri[b:b + 4] for b in range(0, len(tri), 4)]; random.shuffle(lots)
sch = get_cosine_schedule_with_warmup(opt, 30, pas)

#Results: 95% perfect documents

The score compares, field by field, the value written by the model with the expected value, after normalization. A document is "perfect" when all its fields are correct. All three models receive the same prompt, at temperature 0 (the setting that makes responses most stable), through vLLM 26.09, 16 documents at a time. For Qwen3-8B, thinking is disabled. For gpt-oss 120B, reasoning effort is set to "low."

The trained model gets 95.0% of documents perfect, compared with 26.0% for the same model without training: 69 points higher, or 3.7 times as many error-free documents. It beats gpt-oss 120B (62.5%), a model with 117 billion parameters. It's also the fastest: 524 seconds for the 200 documents, compared with 991 seconds for gpt-oss 120B and 686 seconds for Qwen3-8B without training. Its responses are also shorter: 98,847 tokens written, compared with 160,702 for gpt-oss 120B.

On the 100 documents with never-before-seen layouts, the trained model maintains 94% perfect documents and 98.5% correct fields, compared with 96% and 99.9% on known layouts. The gap is 2 points on perfect documents (96% vs. 94%): the adapter isn't limited to the formats seen during training.

The per-field breakdown shows what training fixed. All 31 fields are exact in at least 97.5% of documents, and 18 of them in 100% of documents. Without training, 10 fields are exact between 49.5% and 61.5% of the time: the amounts (discount, totals, VAT, deposit, net amount due) and the line items (quantity, unit price, VAT rate, amount). After training, these 10 fields are exact 98% to 100% of the time.

All three models return valid JSON in 100% of cases and find the correct number of line items in 100% of documents. The difference therefore comes down to the field values, not the format of the response.

#What it costs: time, memory, electricity

These figures cover the GPU alone, as reported by the chip. According to NVIDIA, the power shown by nvidia-smi is for the GPU alone. The processor, memory, SSD, and power supply are not counted. The rate is EDF's Base option for 6 kVA, from the price schedule effective August 1, 2026; the subscription fee is not counted.

Each point on the curve comes from a single batch of 4 documents, so values vary from one reading to the next. The loss drops below 0.01 by step 30, after 5 minutes of training, and no later reading exceeds 0.0082. A single epoch was enough to reach 95% perfect documents in this case.

#QLoRA with Unsloth: a 70B and gpt-oss 120B on the same machine

QLoRA compresses the model's weights to 4 bits, which greatly reduces its memory footprint, then trains a LoRA adapter on top. It's the way to specialize a large model without changing machines. We used Unsloth, an open-source tool that, according to its publisher, makes fine-tuning lighter and faster.

NVIDIA publishes an official guide for Unsloth on DGX Spark, in a NVIDIA PyTorch container (version 25.11), and also offers an official PyTorch fine-tuning environment for models with 1 to 70 billion parameters. Unsloth states about 68 GB of memory for a 4-bit QLoRA of gpt-oss 120B; we measured a 64.4 GB peak, in the same range.

We measured speed and memory on two models: Llama 3.3 70B and gpt-oss 120B, both in the 4-bit format published by Unsloth. Each run has 40 steps, and the first 10, used for warmup, aren't counted in the average.

Sequences are 1,024 tokens, with a batch size of 1 and a LoRA of rank 16. These steps, run on our protocol's reference text, measure throughput and memory; they do not produce a useful model.

The 70B trains with a 41.3 GB peak, about a third of the 121.7 GB the system sees; gpt-oss 120B needs 64.4 GB. In both cases, there's room left for other services. Notably, gpt-oss 120B runs faster per step than the 70B (6.35 s vs. 9.39 s), even though it's larger.

It's an MoE model, a "mixture of experts": for each token, it uses only 5.1 of its 117 billion parameters. We train only its 4 attention matrices (q, k, v, o), or 11.9 million parameters. The 70B gets a LoRA on its 7 projections, or 207 million trained parameters.

For planning, here is an estimate based on our measurements. 1 000 steps of 1 024 tokens, or about one million tokens, would take 2 h 36 min with the 70B and 1 h 46 min with gpt-oss 120B. Check these durations on your own dataset. The 70B QLoRA run kept the GPU busy for 10.8 minutes, including model loading, for 8.9 Wh on the GPU side.

import os; os.environ.setdefault("UNSLOTH_SKIP_TORCHVISION_CHECK", "1")
import torch; torch.cuda.set_per_process_memory_fraction(0.8)
from unsloth import FastLanguageModel

model, tok = FastLanguageModel.from_pretrained(
    model_name="unsloth/Llama-3.3-70B-Instruct-bnb-4bit",   # ou unsloth/gpt-oss-120b-unsloth-bnb-4bit
    max_seq_length=1024, load_in_4bit=True, dtype=None)   # nos essais lisaient une copie locale du modèle
model = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32, lora_dropout=0.0,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],   # gpt-oss : q, k, v, o seulement
    use_gradient_checkpointing="unsloth", random_state=0)

Unsloth 2026.10.1 is installed in the same Python environment as above, in the same PyTorch container. For step-by-step installation, follow the official NVIDIA guide and the Unsloth documentation, linked at the end of the page.

#Serving the trained model with vLLM

vLLM is an open-source server designed to handle multiple users at once; we use it in the container provided by NVIDIA. It loads the base model, then adds the adapter to it. The API then exposes two names: Qwen3-8B, without the adapter, and factures, with the adapter. We evaluated both on the same server, which makes the comparison fair.

docker run -d --init --name vllm-factures --gpus all --ipc=host --ulimit memlock=-1 \
  -p 127.0.0.1:8002:8000 -v ~/models/hf:/m:ro -v ~/resultats:/res \
  nvcr.io/nvidia/vllm:26.09-py3 \
  vllm serve /m/Qwen/Qwen3-8B --served-model-name Qwen3-8B \
    --enable-lora --lora-modules factures=/res/adapter --max-lora-rank 16 \
    --max-model-len 6144 --gpu-memory-utilization 0.6

curl -s http://127.0.0.1:8002/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "factures",
  "messages": [
    {"role": "system", "content": "…consigne : rendre le JSON de la facture…"},
    {"role": "user", "content": "…texte de la facture…"}
  ],
  "temperature": 0, "max_tokens": 3000,
  "chat_template_kwargs": {"enable_thinking": false}
}'

With 16 documents sent at a time, the 200 invoices take 524 seconds. At that rate, 1,000 invoices take about 44 minutes: this figure is calculated from our measurement. The port is bound to the local address 127.0.0.1. To open the service to your network, add an API key; the guide on vLLM in production covers this step in detail.

#Reproduce this case yourself in five steps

1. 01
   Prepare the examples

   Gather pairs: the text of a document and the expected JSON response. Write a single prompt, the same one for training and evaluation. We used 2,000 examples; we did not measure a minimum threshold.
2. 02
   Keep a test set

   Set documents aside and never show them to the model. Add a few layouts that are absent from training, so you can tell whether the model generalizes.
3. 03
   Train for one epoch

   Sort the examples by length, form the batches, compute the loss on the response alone, and monitor the loss curve.
4. 04
   Score before and after

   Score the base model and the trained model with the same prompt, field by field. Also score a large model without training: that's the most useful comparison.
5. 05
   Serve with vLLM

   Load the adapter with the enable-lora option and connect your software to the API.

#Fine-tuning, RAG, or a better prompt: which should you choose?

Start with the prompt: it's free and immediate. If the model needs to find facts in your documents, RAG is a better fit: it's an assistant that searches your files first, before answering.

Fine-tuning is the right tool when the task is repetitive, the output format has to stay stable, and you have examples. Reading invoices meets all three conditions. The two approaches can be combined on the same machine; another guide in the series shows a complete RAG assistant.

#The limits of our measurements

Fictional documents

Our invoices are generated by a script, with 12 layouts. Real documents, scanned or photographed, first require text recognition (OCR) and may produce different results.

A single training run

We trained a single adapter and scored each model once, at temperature 0. We did not measure variation from one training run to another or compare with a longer training run.

gpt-oss 120B

It's scored with low reasoning effort. A different setting may change its score.

QLoRA

The QLoRA measurements cover speed and memory, over 40 steps. We did not evaluate the quality of trained 70B or 120B models.

Energy

It covers the GPU only.

Out of scope

We did not measure training all of a model's parameters, or using multiple connected machines.

Updates and corrections: this page will be revised if a new version of Unsloth, vLLM, or DGX OS changes these results; each correction will be dated.

#Who ATOM is the right choice for

An SMB or professional firm that wants to specialize a small model on its documents

85 minutes and 1.5 euro cents of electricity on the GPU side to go from 26% to 95% perfect invoices on our fictional invoice dataset, without sending a single document to a third party.

A developer or data team building prototypes

Qwen3-8B with LoRA, a 70B or gpt-oss 120B with QLoRA: all three fit on the same machine, with 33 to 64 GB of peak GPU memory.

A team that wants to train, evaluate, and serve in one place

The same machine trained the adapter, scored three models, and served the responses with vLLM.

An office without a server room

A one-liter chassis (150 × 150 × 50.5 mm, 1.2 kg) powered by a 240 W USB-C adapter.

Let's separate the platform from the manufacturer. The NVIDIA GB10 chip, its 128 GB of unified memory at 273 GB/s, CUDA, and DGX OS 7.5.0 determine most of the training speed. For inference, our measurements are within 4.3% at most of those published for the DGX Spark; the training runs in this guide were performed on the AI TOP ATOM.

GIGABYTE provides the chassis and its two-fan cooling. During the 87 minutes of logging for the training run, at an average of 53 W on the GPU side, the temperature stayed at 65 °C on average and 77 °C at most, with no throttling reported.

Prices checked on October 8, 2026 for the ATAGB10-9001 (128 GB, 4 TB PCIe 4.0 SSD): €6,346.27 including tax at the official AORUS store, where it was sold out; it was in stock on Amazon.fr. We measured the ATAGB10-9000 model, whose SSD is PCIe 5.0. GB10 prices change every week: check them before you buy.

#FAQ