Colossal Clean Crawled Corpus
Heavily filtered Common Crawl used to train T5.
Resumen
C4 is a cleaned and filtered derivative of a single Common Crawl snapshot, built for training and released with the T5 paper. It solves the problem that raw crawl data contains boilerplate, navigation and machine-generated text at a volume that damages training. It exists as the corpus whose filtering rules became the baseline every later web dataset is measured against.
Grafo de tecnologías
Cómo se conecta Colossal Clean Crawled Corpus con el resto del stack.
Usa y depende de
- Common Crawl dataset Datasets Colossal Clean Crawled Corpus based on Common Crawl High ✓ source
Aplicado a
- Pretraining technique Techniques Colossal Clean Crawled Corpus used for Pretraining High ✓ source
Alternativas
Comparativas
Comparativas detalladas con las alternativas anteriores.