Softmax
Turns a vector of scores into a probability distribution.
Обзор
Softmax exponentiates each score and divides by the sum, producing values between zero and one that add up to one. It solves the problem that raw scores are unbounded and unnormalised, so they cannot be compared as a weighting. It exists wherever a choice has to be made from scores, which in a transformer is the attention weights and the output distribution over the vocabulary.
Граф технологий
Как Softmax связана с остальным стеком.