Colossal Clean Crawled Corpus
Heavily filtered Common Crawl used to train T5.
Aperçu
C4 is a cleaned and filtered derivative of a single Common Crawl snapshot, built for training and released with the T5 paper. It solves the problem that raw crawl data contains boilerplate, navigation and machine-generated text at a volume that damages training. It exists as the corpus whose filtering rules became the baseline every later web dataset is measured against.
Graphe des technologies
Comment Colossal Clean Crawled Corpus se rattache au reste de la pile.
Utilise et dépend de
- Common Crawl dataset Datasets Colossal Clean Crawled Corpus based on Common Crawl High ✓ source
Appliqué à
- Pretraining technique Techniques Colossal Clean Crawled Corpus used for Pretraining High ✓ source
Alternatives
Comparaisons
Comparaisons détaillées avec les alternatives ci-dessus.