# Reproducible pilot — 2026-09-12

One NVIDIA GeForce RTX 5070 Ti Laptop GPU, 12227 MiB reported. This is not a desktop 16 GB card, RTX 5090 or Mac measurement. Exact CPU, OS, driver and Ollama version: metadata.json. Exact model identities, digest, template and quantization: individual show files and summary.json.

One serial Ollama API request, num_ctx=4096, num_predict=256, temperature=0, seed=42, num_gpu=99, think=false. One excluded warmup then five generations per model. Every prompt contains a different run number; prompt-prefix caching can still affect prompt processing. Raw durations and counts remain separate. A maximum output token count is not a guaranteed full answer. GPU placement is archived after every run. Hermès GPU lock acquired nonblocking; ComfyUI running and pending queues empty before start. The previously warm Mistral Nemo context was restored after the run. No model download or runtime upgrade.

Run benchmark.py on the same prepared machine with the installed models; keep metadata and model show exports. The script uses the existing shared Ollama server and must run only in a reserved idle GPU window, under /tmp/hermes-gpu.lock; do not run on a busy machine. Runtime options not explicitly set retain the installed server defaults. Those model defaults and templates are retained in show.json. The measured generation times do not include loading, whole-application latency or energy.

French synthetic usage tasks: code must preserve insertion order, remove duplicates and leave input unchanged (including empty input); documents must return the four requested facts, with null for missing date, and exactly the requested keys. Inputs and full model outputs are retained. This small exercise is not HumanEval, a general quality ranking, or an English-language evaluation. Code snippets were inspected before executing functional assertions. Hermes3 fails deduplication/order and reaches the output token limit; its document facts pass but field names differ. Qwen3 and Mistral Nemo pass these limited checks. A faster model can fail a simple task.

License: CC BY 4.0 for our synthetic inputs, measurements, protocol and summaries. Model output is included as experimental evidence; model licenses remain applicable. No private user prompts or personal data. API documentation: https://docs.ollama.com/api/generate .