Cross encoders and bi encoders are two types of encoding techniques used in natural language processing (NLP) to understand the relationship between two pieces of text.
- Bi-encoder → Two inputs (query + document) are encoded separately into vector embeddings. A similarity metric (cosine, dot product) compares them.
- Cross-encoder → Both inputs are processed together, allowing the model to attend to every token jointly. It outputs a single relevance score.
- Bi-encoder (Retriever) → Encode millions of docs once, store in a vector DB, retrieve top-100 candidates quickly.
- Cross-encoder (Reranker) → Take those top-100 and rescore with high precision, returning the best 5–10.
Encoder Architectures
Bi-Encoders (Dual Encoders)
Architecture: Processes queries and documents separately through identical encoders
Process:
Query: "What is machine learning?"
↓
[BERT Encoder] → Query Embedding (768 dims)
Document: "Machine learning is a type of AI..."
↓
[BERT Encoder] → Document Embedding (768 dims)
Similarity Score = Cosine Similarity(Query Emb, Doc Emb)
- Fast: Can precompute all document embeddings once, reuse for all queries
- Scalable: Efficient for large document collections
- Flexible: Different loss functions for training
- Trade-off: Potentially less accurate than cross-encoders
Example
- DPR (Dense Passage Retrieval)
- ANCE (Approximate Nearest Neighbor Negative Contrastive Learning)
- SBERT (Sentence-BERT)
Cross-Encoders
Processes query and document together in single pass
[Query, Document] → [BERT Encoder with Cross-Attention] → Relevance Score
- Accurate: Full cross-attention between query and document tokens
- Slow: Requires encoding for each query-document pair
- O(n²) complexity: For D documents and Q queries: D × Q encodings needed
- Purpose: Reranking top candidates from bi-encoder
Used in reranking pipelines with models like BERT-base