Softmax
Turns a vector of scores into a probability distribution.
Überblick
Softmax exponentiates each score and divides by the sum, producing values between zero and one that add up to one. It solves the problem that raw scores are unbounded and unnormalised, so they cannot be compared as a weighting. It exists wherever a choice has to be made from scores, which in a transformer is the attention weights and the output distribution over the vocabulary.
Technologiegraph
Wie Softmax mit dem übrigen Stack zusammenhängt.