It's Not DNS

[Verse 1] Monday morning, helpdesk lighting up the phone Half the building locked out, nobody getting work done Authentication failing, tickets coming in Senior engineer on the case, where to begin [Pre-Chorus] There's a graybeard in the corner shaking his head "Check your DNS" is all he said [Chorus] It's not DNS There's no way it's DNS It was DNS [Verse 2] Active Directory healthy, LDAP alive User accounts are fine and the services survive Network's fine, routing clean, the firewall opens wide Clock skew too great, that's what Kerberos replied NTP's not syncing, clock is running wrong Drifting by six minutes, drifting all day long Pool.ntp.org gets queried, nothing coming back Time to run a dig command and follow up the stack [Pre-Chorus] There's a graybeard in the corner shaking his head "Check your DNS" is all he said [Chorus] It's not DNS There's no way it's DNS It was DNS [Verse 3] SERVFAIL in the dig, right there in the trace DNS can't reach the time server. that's the case Four layers down from login, that's where this began One config fix and the whole chain runs as planned NTP syncs up and the clock comes back in line Kerberos is happy, authentication's fine Whole building's back online, the work begins The graybeard in the corner slowly grins [Pre-Chorus] There's a graybeard in the corner shaking his head "Check your DNS" is all he said [Chorus] It's not DNS There's no way it's DNS It was DNS [Outro] Four layers deep... It was DNS... Always DNS...