
Average Tells a Comforting Story
[Intro] (soft Rhodes chords, vinyl crackle) (whispered) ninety-five... and you shipped it (the average told you a story) [Verse 1] Sampled the outputs, counted them right, divided by total, the number felt light. Ninety-five per cent, looked clean, looked fine, so you pushed it to prod and you called it a win. But under that figure a category drowns, four in ten handwritten forms going down. [Pre-Chorus] (build) The clean ones are carrying the weak ones along, the headline looks healthy but the headline is wrong... [Chorus] (drop) The average tells a comforting story, hiding a category that's broken underneath. (oh-oh) The average tells a comforting story — so measure per category, don't trust the blend. (mm) [Verse 2] Stop measuring one, start measuring each: sort by the type before the sample's complete. Sample every group with enough to be real, report the breakdown, not the comfortable feel. Now ninety-nine and sixty stand apart — automate the easy, send the hard to a heart. [Verse - Rap] (half-time, spoken-word) Here's the arithmetic, plain as it gets: ninety per cent at ninety-nine, that's your safe set, ten per cent handwritten holding sixty, no less, weight 'em and blend 'em — ninety-five, a mess. The big easy bucket swamps the small hard one, wider the volume gap, more cover it's done. Overall tells you "right on a random sheet", it won't tell you which kind is reliably beat. So stratify, partition where the difficulty bends, per-category accuracy is where honesty lives. Route the weak to humans — but route the field, not the page, the legible invoice number auto-clears the stage, the scrawled address gets flagged where the doubt is real, reviewer sees the one, not the nine that they'd seal. And the signal that drives it can't be the model's own word — self-reported confidence is uncalibrated, absurd: ninety-five "sure" is not ninety-five right, it's confidently wrong and it's hesitantly bright. Gate on agreement, on a semantic cross-check, a measured calibrated source — not the model's own bet. [Chorus] (drop) The average tells a comforting story, hiding a category that's broken underneath. (oh-oh) The average tells a comforting story — so measure per category, don't trust the blend. (mm) [Bridge] (half-time) Don't just raise the threshold, don't say "try harder, please" — both keep measuring the blend and leave the broken on its knees. And never gate the routing on how sure the model feels — that auto-accepts the confident mistakes you needed to reveal. [Final Chorus] (full drop) The average tells a comforting story, (it lies!) hiding a category that's broken underneath. The average tells a comforting story — so stratify, route the field, on a signal you can trust. (per category!) [Outro — call to action] (spoken over the final fade) Want the deep version? Get *The CCA-F Study Guide* on Leanpub — leanpub.com/cca-f (...don't trust the blend)

