Colossal Clean Crawled Corpus
Heavily filtered Common Crawl used to train T5.
Обзор
C4 is a cleaned and filtered derivative of a single Common Crawl snapshot, built for training and released with the T5 paper. It solves the problem that raw crawl data contains boilerplate, navigation and machine-generated text at a volume that damages training. It exists as the corpus whose filtering rules became the baseline every later web dataset is measured against.
Граф технологий
Как Colossal Clean Crawled Corpus связана с остальным стеком.
Использует и зависит от
- Common Crawl dataset Datasets Colossal Clean Crawled Corpus based on Common Crawl High ✓ source
Применяется к
- Pretraining technique Techniques Colossal Clean Crawled Corpus used for Pretraining High ✓ source
Альтернативы
Сравнения
Подробные сопоставления с альтернативами выше.