BERT
Encoder-only Transformer pre-trained with masked language modelling.
Обзор
BERT is an encoder-only Transformer pre-trained by masking words in a sentence and learning to recover them, so it builds a rich representation of text rather than generating new text. It solves the problem that earlier models required a separate architecture for every downstream understanding task. It exists because bidirectional context matters for understanding: knowing the words on both sides of a gap constrains its meaning far more than knowing only the left side. It remains the reference point for encoder-style language models.
Граф технологий
Как BERT связана с остальным стеком.
Использует и зависит от
- Transformer architecture Algorithms BERT uses Transformer High ✓ source
- BGE model Embedding Models BGE based on BERT High ✓ source
- Cross-Encoder concept Techniques Cross-Encoder based on BERT High ✓ source
- E5 model Embedding Models E5 based on BERT High ✓ source
- GTE model Embedding Models GTE based on BERT High ✓ source
- Instructor model Embedding Models Instructor based on BERT High ✓ source
- Reranking technique Techniques Reranking based on BERT High ✓ source
- Sentence Transformers framework Frameworks Sentence Transformers based on BERT High ✓ source
- Transformer architecture Algorithms BERT based on Transformer High ✓ source
Применяется к
- Distillation technique Techniques Distillation used for BERT High ✓ source