Query rewriting
Overview
Query rewriting is a prompt engineering technique that transforms an input query into one or more alternative formulations designed to improve the quality and coverage of retrieved documents from a vector database or search index. Rather than directly passing a user's original query to a retrieval system, query rewriting anticipates vocabulary mismatches, implicit intent gaps, and coverage blind spots by generating semantically equivalent or complementary query variants.
The core motivation addresses a fundamental challenge in RAG systems: users often phrase questions using colloquial language, domain-specific terminology, or implicit context that may not align well with the embedding space or indexing strategy of the knowledge corpus. A query about "car accidents" may fail to retrieve relevant content indexed under "motor vehicle collisions" or "traffic incidents." Query rewriting bridges this gap by generating alternative forms before retrieval, thereby improving both recall and precision.
Query rewriting is typically performed by the LLM itself or a specialized rewriting component, and may involve multiple passes or parallel rewrites. This contrasts with post-retrieval reranking, which operates on already-retrieved results. Query rewriting occurs upstream in the pipeline, directly influencing which documents are surfaced for grounding.
How it works
Query rewriting operates through several mechanisms:
Single-pass rewriting: The LLM receives the original query and an instruction to rephrase it for clarity, specificity, or alignment with document indexing conventions. Example: "Rewrite this query for a medical knowledge base: 'Why do I feel tired?'" → "What are the primary causes of chronic fatigue and associated diagnostic criteria?"
Multi-variant generation: The system generates multiple query rewrites in parallel, each capturing different facets of the user's intent. All variants are executed against the retrieval system, and results are deduplicated and ranked. This approach, sometimes called query fan-out, increases coverage at the cost of additional retrieval calls.
Iterative refinement: Query rewrites are generated, documents retrieved, and their relevance assessed via LLM-as-judge or explicit feedback signals. Poor-performing rewrites inform the next generation cycle, allowing the system to converge on effective query formulations.
Semantic expansion: Rewrites introduce synonyms, hypernyms, hyponyms, or domain-specific terminology without changing core intent. A query about "photosynthesis" might be rewritten to include "light-dependent reactions," "chlorophyll," and "ATP production" to match diverse document vocabularies.
Query rewriting is distinct from Hypothetical Document Embeddings (HyDE), which generates hypothetical documents rather than query variants. However, both techniques aim to improve the alignment between query embeddings and document embeddings in a vector database.
| Term | Distinction |
|---|---|
| RAG | RAG is the broader pipeline combining retrieval and generation. Query rewriting is a specific optimization technique applied within the retrieval phase of RAG. |
| Hypothetical Document Embeddings | HyDE generates synthetic documents matching the query intent, then embeds and retrieves against them. Query rewriting reformulates the query itself, not the target documents. |
| Reranking | Reranking operates on already-retrieved documents to reorder them by relevance. Query rewriting occurs before retrieval to improve the initial result set, not after. |
| Hybrid search | Hybrid search combines multiple retrieval methods (e.g., BM25 and semantic search). Query rewriting optimizes a single query for better performance across any retrieval method. |
| Prompt engineering | Prompt engineering is the broader discipline of crafting inputs to LLMs. Query rewriting is a specific prompt engineering application focused on query reformulation for retrieval. |
Examples
E-commerce product discovery: A user queries "durable bags for hiking." A query rewriting system generates variants: "water-resistant backpacks for outdoor trekking," "heavy-duty outdoor bags," and "weather-proof camping gear." These variants are retrieved separately and results merged, improving recall over a single query.
Medical information retrieval: A patient asks "my joints hurt when it rains." A rewriting system reformulates to clinical terms: "joint pain exacerbation during weather changes," "barometric pressure and arthralgia," and "inflammatory joint symptoms." This alignment with medical indexing improves retrieval of evidence-based content.
Legal document search: A lawyer searches "what happens if you break a contract?" Query rewriting generates "breach of contract remedies," "contractual liability and damages," and "enforcement of contractual obligations." These professionally-indexed terms retrieve relevant case law and statutes that casual language would miss.
See also
- Retrieval-augmented generation (RAG)
- Prompt engineering
- Hypothetical Document Embeddings
- Query fan-out
- Retrieval precision and recall
- Embeddings
- Vector database
- Reranker
