Heavily filtered Common Crawl used to train T5.
Для этих двух технологий пока не зафиксированы сравнимые характеристики.
Сформировано из различий выше по фиксированным правилам, а не моделью.
Choose Colossal Clean Crawled Corpus if…
Ни одна из зафиксированных характеристик не говорит в его пользу.
Choose FineWeb if…
Ни одна из зафиксированных характеристик не говорит в его пользу.
Страница остаётся доступной, но исключена из поисковых индексов, пока не наберёт достаточно подробностей со ссылками на источники (PRD SEO-08).
dataset · Datasets
C4 is a cleaned and filtered derivative of a single Common Crawl snapshot, built for training and released with the T5 paper. It solves the problem that raw crawl data contains boilerplate, navigation and machine-generated text at a volume that damages training. It exists as the corpus whose filtering rules became the baseline every later web dataset is measured against.
dataset · Datasets
FineWeb is a 15-trillion-token web corpus derived from Common Crawl, released together with the exact filtering and deduplication steps used to produce it. It solves the problem that most published corpora document their contents but not their construction, leaving any difference in results unattributable. It exists as a corpus built to be reproducible, which makes it usable as a baseline rather than just as data.
