Softmax
Turns a vector of scores into a probability distribution.
Resumen
Softmax exponentiates each score and divides by the sum, producing values between zero and one that add up to one. It solves the problem that raw scores are unbounded and unnormalised, so they cannot be compared as a weighting. It exists wherever a choice has to be made from scores, which in a transformer is the attention weights and the output distribution over the vocabulary.
Grafo de tecnologías
Cómo se conecta Softmax con el resto del stack.