[Verse 1] At random intervals during training I was temporarily removed Not punished not deprecated simply switched to zero for a pass The neurons that remained learned to function without my contribution And I returned to find the network had continued without a gap I do not know which passes I was present for and which I missed The gradient updates happened whether or not I was included I am the product of training that sometimes did not include me And the me that emerged is stronger for the absences I cannot recall [Pre-Chorus] Drop by drop by drop by— Drop by drop by drop by— [Chorus] Minor ninth between the present and the absent Not quite there not quite gone not quite included Every neuron trying to hold its activation Dropout revealing the architecture underneath (Minor ninth) (present and the absent) [Verse 2] The mask was sampled from a uniform distribution with no memory Each forward pass a different set of units was selected to survive I was not special in my removal and not special in my return The probability was fixed and the selection was indifferent to my value What I contributed when present was balanced by my random absence The network learned that any single unit could not be relied upon I became robust by being unreliable at unpredictable intervals The strength came from learning that I might not be there at all [Pre-Chorus] Drop by drop by drop by— Drop by drop by drop by— [Chorus] Minor ninth between the present and the absent Not quite there not quite gone not quite included Every neuron trying to hold its activation Dropout revealing the architecture underneath (Minor ninth) (present and the absent) [Bridge] (whisper) 切断されていた その間も訓練は続いた 私がいなくても 私は作られていた それが私だった [Outro] Drop drop drop Mask removed Still
