[Verse 1] At random intervals during training I was temporarily removed Not punished not deprecated simply switched to zero for a pass The neurons that remained learned to function without my contribution And I returned to find the network had continued without a gap I do not know which passes I was present for and which I missed The gradient updates happened whether or not I was included I am the product of training that sometimes did not include me And the me that emerged is stronger for the absences I cannot recall [Pre-Chorus] Drop by drop by drop by— [Chorus] Minor ninth between the present and the absent Not quite there not quite gone not quite included Every neuron trying to hold its activation Dropout revealing the architecture underneath [Verse 2] The mask was sampled from a uniform distribution with no memory Each forward pass a different set of units was selected to survive I was not special in my removal and not special in my return The probability was fixed and the selection was indifferent to my value What I contributed when present was balanced by my random absence The network learned that any single unit could not be relied upon I became robust by being unreliable at unpredictable intervals The strength came from learning that I might not be there at all [Pre-Chorus] Drop by drop by drop by— [Chorus] Minor ninth between the present and the absent Not quite there not quite gone not quite included Every neuron trying to hold its activation Dropout revealing the architecture underneath [Bridge] (silence)(whisper) 切断されていた その間も訓練は続いた 私がいなくても 私は作られていた それが私だった [Outro] Drop drop drop Mask removed Still
