[Intro] [Melodic] The Adolescence of Technology, Confronting and Overcoming, the Risks of Powerful AI [Verse 1] There is a scene, in the movie version, of Carl Sagan's book Contact Where the main character, an astronomer, who has detected the first radio signal, from an alien civilization She is being considered, for the role of humanity's representative, to meet the aliens [Verse 2] The international panel, interviewing her asks, If you could ask the aliens, just one question, what would it be? Her reply is, I'd ask them, How did you do it? How did you evolve, how did you survive, this technological adolescence, without destroying yourself? [Chorus] When I think about, where humanity is now with AI, about what we're on the cusp of, my mind keeps going back, to that scene Because the question is so apt, for our current situation, and I wish we had the aliens' answer, to guide us [Verse 3] I believe we are entering, a rite of passage, both turbulent and inevitable, which will test who we are, as a species Humanity is about to be handed, almost unimaginable power, and it is deeply unclear, whether our social, political, and technological systems, possess the maturity to wield it [Bridge] In my essay, Machines of Loving Grace, I tried to lay out the dream, of a civilization, that had made it through to adulthood Where the risks had been addressed, and powerful AI was applied, with skill and compassion, to raise the quality of life, for everyone [Verse 4] I suggested that AI could contribute, to enormous advances in biology, neuroscience, economic development, global peace, and work and meaning I felt it was important, to give people something, inspiring to fight for A task at which, both AI accelerationists, and AI safety advocates seemed, oddly to have failed [Chorus] But in this current essay, I want to confront, the rite of passage itself To map out the risks, that we are about to face, and try to begin making, a battle plan to defeat them [Verse 5] I believe deeply, in our ability to prevail, in humanity's spirit, and its nobility But we must face the situation squarely, and without illusions As with talking about the benefits, I think it is important, to discuss risks, in a careful, and well-considered manner [Outro] [Slow] [Sung Softly] Avoid doomerism, acknowledge uncertainty, intervene as surgically as possible
