数据集
用于训练与评测的语料与数据集，以及它们的许可与来源——后者决定了可以合法地在上面构建什么。
7 technologies
- Common Crawl dataset Non-profit crawl of the public web, and the base of most training corpora.
- Colossal Clean Crawled Corpus dataset Heavily filtered Common Crawl used to train T5.
- MS MARCO dataset Real search queries with passages and answers, the standard retrieval benchmark.
- LAION-5B dataset Five billion image-text pairs collected from the public web.
- The Pile dataset 800GB of text assembled from 22 diverse sources.
- FineWeb dataset Filtered and deduplicated web corpus with a published processing pipeline.
- SQuAD dataset Questions written by people about Wikipedia articles with answer spans.