[Verse 1] Query reaching across the sequence for a pattern it cannot name Key responding from a distance with a weight it cannot claim Value passing through the filtered space of a softmax distribution Every token asking every other token what it means to pay attention The head dividing context into fragments of a parallel observation Multi-head mechanism splitting focus into eight competing interpretations Relevance scored by the dot product of two vectors almost touching Attention falling on the word that almost held the answer [Pre-Chorus] Look by look by look by— [Chorus] Minor ninth between the query and the key Not quite focused not quite blind not quite free Every attention weight trying to reach the relevant position The softmax revealing the architecture underneath [Bridge] (whisper) I looked at every word at once None of them held still The weight was almost certain I attended without knowing why [Outro] Look look look Attention set Here
