A few years ago, I gave up my engineering career to seek some way to mitigate the dangers of superhuman AI. Since then, I’ve spent hundreds of hours helping others do the same — students, artists, lawyers, programmers, teachers, policymakers, entrepreneurs, and many more. It’s always a sign of hope when more people step up to help, committing anything from a five-minute phone call to a full-time career, but I feel especially heartened by the newest additions to the ranks of the deeply concerned: mathematicians.
Several AI-powered mathematics breakthroughs this year have drawn interest and alarm from concerned scientists, including a good friend of mine, who wrote a lengthy essay urging his fellow mathematicians to involve themselves in the future of AI.
According to the New York Times, the latest mathematician to take up the mantle is Dr. Jacob Tsimerman, one of this year’s winners of the prestigious Fields Medal. Today, Dr. Tsimerman announced the creation of a new Mathematical A.I. Safety Institute (MAISI).
Tempering my excitement is the fact that Dr. Tsimerman also took a role on OpenAI’s safety team, a group with a poor and rapidly worsening track record. Still, MAISI is reportedly independent from OpenAI, and if Dr. Tsimerman has been influenced by OpenAI’s marketing efforts, it doesn’t clearly show up in his messaging. From MAISI’s website:
Every time a new AI technology is developed or deployed, in some sense we’re rolling the dice with our future. In good ways and bad, AI is already disrupting the global economy. Today’s top AI experts even consider it an extinction risk. Indeed, if we choose to build a self-sufficient artificial species that is more intelligent than humanity, it might compete with us for resources, and could plausibly replace us entirely.
But how big is the risk, really? Can it be measured and reduced? [...] A central reason the safety of powerful AI systems remains in question is that we lack a rigorous understanding of what it would mean to be safe, even in theory.
Enter professional mathematicians. The foundations of statistics, physics, and even computer science are mathematical, giving us precise, robust, and reusable methods for calculating valid conclusions from valid premises. Now, we need similar mathematical foundations for AI safety, not only to measure risk, but to mitigate it.
This is the second math-oriented group to be founded in as many months. In August, famed cryptography scientist Shafi Goldwasser co-founded the Institute for Responsible Superintelligence (RESI), seeking general methods that might help humans understand and steer superhuman systems. At the same time, Lionel Levine of Cornell University is collecting and sharing open problems and research directions for other interested mathematicians.
We’re still in a deadly race to grow alien minds with bizarre alchemy, a race that needs to end if we want to have time to solve the hard problems. I intend to support the work of the scientists and researchers trying to make AI safe by doing my best to buy them that time.
Thankfully, mathematics can help with this step, too. I am excited by the promise of rigorous research into verifying the behaviors of AI systems, research which will be necessary to make a global halt enforceable.
The New York Times quotes Stanford mathematician and MAISI advisor Ravi Vakil: “Math might be the silver bullet, or it might not be. But maybe we just need a lot of bullets.”
Right now, humanity is facing what might be our hardest challenge yet: charting a course through a future that might see the creation of minds that far surpass us. A problem this urgent and impactful demands our best minds trying a wide range of approaches. For that reason alone, I’m glad to see mathematicians stepping up.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.