Colossal Clean Crawled Corpus
Heavily filtered Common Crawl used to train T5.
Visão geral
C4 is a cleaned and filtered derivative of a single Common Crawl snapshot, built for training and released with the T5 paper. It solves the problem that raw crawl data contains boilerplate, navigation and machine-generated text at a volume that damages training. It exists as the corpus whose filtering rules became the baseline every later web dataset is measured against.
Grafo de tecnologias
Como Colossal Clean Crawled Corpus se liga ao resto do stack.
Usa e depende de
- Common Crawl dataset Datasets Colossal Clean Crawled Corpus based on Common Crawl High ✓ source
Aplicado a
- Pretraining technique Techniques Colossal Clean Crawled Corpus used for Pretraining High ✓ source
Alternativas
Comparações
Comparações detalhadas com as alternativas acima.