GPT
Family of decoder-only Transformer language models trained with a generative objective.
Overview
GPT is a family of decoder-only Transformer language models trained to predict the next token on large text corpora. It solves the problem of producing fluent, contextually appropriate text without hand-built linguistic rules. It exists because the generative pre-training objective turned out to transfer well to tasks the model was never explicitly trained for, which removed the need for a labelled dataset per task. Later members of the family are the models most people mean when they say "large language model".
Technology Graph
How GPT connects to the rest of the stack.
Uses and depends on
- Transformer architecture Algorithms GPT uses Transformer High ✓ source
- Transformer architecture Algorithms GPT based on Transformer High ✓ source
Tools
Software that implements or integrates with GPT.