← Stackzilla Blog

Changing an Embedding Model Is a Data Migration

Published November 17, 2026 · 3 min read · embeddings, search, AI engineering, data engineering

Version document and query embeddings together, rebuild safely, and compare retrieval quality before switching a production search index.

An embedding model turns content into vectors used for similarity search. Switching models can improve retrieval, but it changes the representation of the data. It is not merely a new configuration value on the query side. Vectors from different embedding models generally should not be treated as points in one shared semantic space. Even when their dimensions match, their coordinates may mean different things. A query embedded with one model cannot be assumed compatible with documents embedded by another. ## Version the entire retrieval pipeline Record the embedding model, supported dimensions, chunking approach, preprocessing, and relevant index settings. These choices affect what the search system retrieves. A chunking change can be as consequential as a model change. Splitting a policy paragraph from its exception may produce misleading retrieval even when every individual vector is valid. Keep a durable reference from each chunk to its original document, version, and access scope. Similarity does not establish authorization. Tenant and permission filters must continue to apply during and after migration. ## Build a new index alongside the old one Create a separate versioned collection or index rather than mixing incompatible vectors into the live one. Backfill documents in resumable batches and track failures explicitly. Handle documents that change during the rebuild. A one-time snapshot can become stale before completion. Use a watermark, change stream, or another reconciliation process appropriate to the system. Check for missing documents, duplicate chunks, and indexing errors before switching traffic. A completed batch count does not prove that every required document is searchable. ## Evaluate retrieval before cutover Run representative queries with known relevant sources against both versions. Compare retrieval quality, latency, storage, and embedding cost. Include queries with exact identifiers, ambiguous language, and no relevant answer. Evaluate the downstream answer separately when the index supports a retrieval-augmented assistant. Better similarity scores do not necessarily mean more accurate final responses. Use a controlled cutover and retain the previous index long enough for rollback. Ensure the application switches its query embedding model and target index together. A partial rollout that updates only one side can quietly degrade results. Re-embedding may involve sending documents to a service, so confirm data-handling requirements and permissions before the rebuild. Do not assume an existing integration authorizes every new use of the data. The useful mental model is a versioned data migration with a quality experiment attached. That framing makes compatibility, completeness, privacy, and rollback visible before users discover the consequences. ## Further reading - [OpenAI: Embeddings guide](https://platform.openai.com/docs/guides/embeddings) - [Pinecone: Data modeling](https://docs.pinecone.io/guides/index-data/data-modeling)

Read the full article on Stackzilla →