A dense 12B beat a sparse 35B MoE at writing search context
I run semantic search over a few thousand recorded meetings. The single thing that makes it work is not the embedding model or the vector store — it is a paragraph of context an LLM writes for every chunk before I embed it. That enrichment step roughly doubled retrieval quality. It is also the slowest, most expensive part of the pipeline, so this summer I tried to make it cheaper by swapping the model that writes the context.
Read entry