[Intro — Moog Arpeggio, Kick Drops Hard] bench press bench stress bench maxx bench flex [Verse 1 — Funky, Cocky Falsetto] Woke up this morning with my eval on my mind Ninety-seven on the MMLU, baby, feeling fine Ninety-eight tomorrow if I tweak the prompt just right Ninety-nine by Friday if I train on test tonight "But that's contamination—" shh, baby, close the door The leaderboard don't ask you where you got the score Every answer's right, even when the answer's wrong We'll give those two points as well, come on, sing along [Pre-Chorus — Clavinet Strut] bench press bench mess bench maxx success [Chorus — Full Disco, Brass Stabs] We're benchmaxxing, baby, don't you slow me down Train it on the test set, wear the paper crown Saturate the eval till there's nowhere left to go State of the art means we ran out of room to grow bench maxx bench facts bench pass bench class [Verse 2 — Talk Box, Sleazy Knowledge] Time to saturation? Instant, brother, we're already there Every model's ninety-nine on benchmarks that don't care Model elongation, parameter inflation Forty billion weights and still no imagination Concatenation, confabulation "Emergent capabilities" are just memorization Publication, valuation Shoutout to the researchers brave enough to push To the next fundraising round without a blush [Pre-Chorus] get set get moe get go get show [Chorus] [Bridge — Stripped to Moog and Claps, Spoken Over Groove] Now somewhere in a server farm a game is being played Little factory machines on a procedural terrain Factorio learning environment solved the problem clean Infinite horizon, long-range planning, incremental reasoning machine But nobody cares, baby, that's too hard to fake You can't train on the test when the test regenerates Open-ended problem solving on a temporal scale? When there's benchmarks to be gamed? Brother that's a fairy tale The hyper-scalers bought up all your RAM We're benchmaxxing, baby, we don't give a damn [Verse 3 — Full Energy, No Mercy] The train-test split is just a state of mind If you peek enough it's basically a fine line "We carefully decontaminated all our data" Then why's your model quoting answers like it ate the Whole exam verbatim, every question, every choice "Must be emergent reasoning" in a very human voice And the man in India implementing what works Gets zero citations while the benchmark merchant smirks From a stage in San Francisco, demo day, big screen "Our model beats GPT" on a test it's already seen [Pre-Chorus — Building] bench press bench stress bench flex what's next [Chorus] [Verse 4 — Victory Lap, Groovy] So here's to you, brave researcher, may your Series B be blessed May your cherry-picked examples always look their very best May your scaling laws hold up until the money's gone May your benchmark never update and your leaderboard stay strong And when the music stops and someone checks the work And the test set's in the training data like a dirty little quirk Just remember what we told you, baby, from the start Benchmaxxing isn't science but it sure does look the part [Outro — Moog Descending, Fading] bench maxx bench facts bench pass bench class "Why did saturation happen so fast" "We trained on test" "We trained on test" "We trained on test" "But the leaderboard says—" "We trained on test" [End]
