Sentence Transformers: embeddings in local

#What an embedding is and what the library does

An embedding model transforms text into a list of numbers—a vector—so that two texts with similar meanings have similar vectors. The Sentence Transformers documentation describes these models, known as bi-encoders, as computing a fixed-size representation for a text, with embedding computation that is often efficient and similarity computation that is very fast. This is the foundation of RAG: encode the question, find the passages whose vectors are closest, and give them to the language model. Two consequences to remember: the model that encodes the documents must be the one that encodes the questions, and its notion of “close” comes from its training. A model trained primarily on English is an unreliable judge of French.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)  # [3, 384]
similarities = model.similarity(embeddings, embeddings)

This model produces 384-dimensional vectors and serves as a starting example, but it is designed for English: for a French corpus, replace it as indicated below. The library automatically places the model on the best available device (cuda, mps, or cpu), and you can force the choice with the device parameter.

#Install and encode a French corpus

1. 01
   Install the library

   Use pip install -U sentence-transformers in a Python environment. Version 6.1.0 was released on September 18, 2026, and requires Python 3.10 or later, according to PyPI.
2. 02
   Choose a multilingual model

   Choose a model advertised as multilingual (see the table below), not an all-* model trained for English.
3. 03
   Batch-encode documents

   Pass a list of texts to encode: the library processes them in batches, making much better use of the hardware than one call per text.
4. 04
   Encode the question with the same model

   Use the same model and prefixes as for the documents, then compare with similarity.
5. 05
   Keep the model name with the index

   Note it in the metadata: if you change it, you'll need to re-encode the entire corpus.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("intfloat/multilingual-e5-large")

docs = [
    "Le contrat peut être résilié avec un préavis de trois mois.",
    "La facture est payable à trente jours fin de mois.",
]
question = "Quel est le délai pour mettre fin au contrat ?"

doc_emb = model.encode(docs, prompt="passage: ", normalize_embeddings=True)
q_emb = model.encode(question, prompt="query: ", normalize_embeddings=True)
print(model.similarity(q_emb, doc_emb))

The query: prefix for questions and passage: prefix for documents is the one cited in the Sentence Transformers documentation for this model. Without it, retrieval degrades without an error message. The encode prompt parameter applies the prefix to each text.

#Choose a model that understands French

The documentation suggests original models and recommends consulting the MTEB leaderboard for inspiration, with two caveats: filter out models that are too large for your hardware, and experiment, because highly ranked models may not perform well on your own tasks. The following table summarizes what the documentation says about the cited models.

#Errors that ruin a corpus without an error message

An English-language model on French

Everything works, yet retrieval is still skewed. This is the most common error.

Passages longer than the model limit

The documentation specifies that longer texts are truncated to the first tokens up to max_seq_length: the end of each long passage becomes invisible.

A different model for documents and questions

The vectors are no longer talking about the same thing: bizarre results, generally introduced by updating only part of the pipeline.

Forgotten prefixes

Some models require a different prefix for questions and documents; omitting it degrades retrieval.

Unnormalized vectors with a poor score

If the model expects cosine similarity, normalize the vectors or use the corresponding metric; otherwise, ranking quality degrades.

#Measuring similarity: normalization and cosine similarity

Two vectors are compared using a similarity measure, most often cosine similarity. The Ollama documentation states that its endpoint returns normalized vectors and recommends cosine for most semantic searches. With normalized vectors, cosine and dot product produce the same ranking, allowing you to use an index optimized for dot product. The point to watch is consistency: the vector database's metric must match the one the model expects, and it is listed in its documentation.

For length, the documentation gives an order of magnitude: 512 tokens for many BERT-type models, or 300 to 400 words in English, and 256 tokens for all-MiniLM-L6-v2. French uses more tokens per word: split your passages below the limit with some headroom, and check model.max_seq_length. You can reduce it, but not increase it beyond what the model supports. The documentation adds that a model trained on short texts represents long texts less effectively.

#Index quickly on your machine

CPU or GPU

The model runs on the best available device. A GPU speeds up bulk indexing but makes little difference for a single query.

Reduced precision

On a GPU, switching to float16 or bfloat16 speeds up inference with minimal precision loss, according to the documentation.

ONNX and OpenVINO backends

The documentation claims possible speedups of up to 2 to 3 times depending on the hardware and backend; test them on your machine.

Multiple GPUs or processes

encode accepts a list of devices: useful for large corpora, less so for small ones because of startup overhead.

More compact vectors

Binary or integer quantization of vectors and models with truncatable dimensions reduce storage and search costs.

Caching and indexing

Don't re-encode unchanged documents: the cache key is based on the content, not the filename. On a machine that also serves a language model, index when no one is using it.

#How much a vector index weighs

The size of an index is calculated by multiplying the number of vectors by their dimensionality and by four bytes if the numbers are float32. A 384-dimensional vector takes 1,536 bytes: one million passages represent about 1.5 GB. At 1,024 dimensions, the same million takes about 4 GB. These figures exclude metadata and the vector database's index structures. The model choice therefore directly affects the RAM required for search, and vector quantization mentioned above can reduce this footprint.

#Sentence Transformers or Ollama's embeddings API

Ollama also exposes embeddings: the documentation states that the ollama run peut command produces vectors and that the api/embed endpoint returns L2-normalized vectors. It recommends the embeddinggemma, qwen3-embedding, and all-minilm models, with vectors typically ranging from 384 to 1 024 dimensions, and reiterates two rules: cosine similarity for most searches, and the same model for indexing and querying.

#Embedding is only a first-pass filter

Sentence Transformers documentation describes the bi-encoder as the first step in a two-stage search, where a cross-encoder, or reranker, re-ranks the best results. The cross-encoder reads the question and each passage together: it is slower but more accurate on a handful of candidates. Retrieving around twenty passages by similarity, then re-ranking them to keep only a few, is one of the least expensive improvements to an RAG pipeline. The dedicated reranker guide explains how to set it up; measure the gain on your twenty questions before keeping it, because it adds latency to every query, a second model to maintain, and additional memory usage.

A good first step, even before comparing models, is to look at what your corpus contains: short, self-contained sentences or long technical paragraphs? Questions phrased as keywords or complete sentences? The answer guides your choice of length limit, chunking strategy, and model. A dense legal corpus calls for different settings than a short question-and-answer database, and no public ranking can make that decision for you.

#Evaluate the model choice before indexing

1. 01
   Write twenty real-world questions

   Use questions your users will ask, phrased in their words, not the document's.
2. 02
   Record the expected transition

   For each question, identify the passage or passages that contain the answer.
3. 03
   Compare two or three models

   Encode the same corpus with each model and check whether the correct passage appears among the top five results.
4. 04
   Replay on every change

   Model, chunking, or prefix changed: rerun this short test before reindexing.

#FAQ

#Go further

• The best embedding models for French
• BGE-M3: embeddings that really speak French
• Add a reranker to your pipeline
• Chunking strategies
• Qdrant: the vector database for local RAG
• Local RAG: introduction

• Source: Sentence Transformers, quickstart
• Source: Sentence Transformers, embedding computation
• Source: Sentence Transformers, pretrained models
• Source: Ollama, embeddings