Former Google engineer stops retirement savings over AI extinction risks

A personal financial gamble

Nate Soares, a machine learning researcher and president of the Machine Intelligence Research Institute (MIRI), has made a drastic personal choice based on his views of technology. He revealed in a podcast with Nicholas Thompson that he stopped contributing to his retirement savings roughly ten years ago. This decision stems from his belief that humanity may not survive long enough to actually use those funds.

Soares currently holds a 401K from his previous tenure at Google, but he has not added to it in a decade. He explains that his conviction regarding the risks of artificial intelligence has fundamentally altered his long-term financial planning. For Soares, the threat of AI is not a distant hypothetical but a pressing reality that dictates his current lifestyle.

His stance is rooted in the belief that we are approaching a critical turning point in human history. He suggests that the window of time available to address these risks is closing much faster than many people realize. By choosing not to save, he is essentially betting on a future where the current trajectory of AI development reaches a terminal conclusion for the human species.

The math of extinction

Soares provides a specific and alarming estimate regarding the timeline for these risks. He believes there is a greater than 50% chance that advanced AI systems will cause human extinction within the next 10 years. This perspective places him in the camp of "AI doomerism," a movement that views the creation of superintelligent systems as a primary existential threat.

During his conversation with Thompson, Soares noted that the clock is ticking faster than the public might assume. He argues that the rapid advancement of machine learning means we may not have a decade left to establish proper safeguards. His concern is not about the immediate effects of AI on the economy, but rather the ultimate survival of the human race.

To put this in perspective, Soares distinguishes his fears from the common anxiety surrounding job automation. He joked that if AI were to kill all humans, everyone would technically be at full employment the moment before it happened. His focus is strictly on the possibility of advanced systems becoming so capable that they become impossible for humans to control.

The horse and the car

When challenged on the current limitations of AI, Soares uses a historical analogy involving transportation. He compares the early development of automobiles to the eventual replacement of horses by cars. He suggests that horses might have looked at early cars and dismissed them because they could not yet perform every task a horse could do.

Soares argues that we should not mistake current technical hurdles for permanent boundaries. He observes that AI systems are becoming increasingly proficient at handling longer and more complex tasks every day. Just as the car eventually surpassed the horse despite early flaws, he believes AI will continue to scale until it reaches unprecedented capabilities.

This perspective serves as a warning against complacency in the tech industry. He suggests that the fact that today's AI has shortcomings does not mean future iterations will remain limited. Instead, he views the current progress as a steady climb toward systems that could eventually surpass human cognitive abilities.

A path toward safety

Despite his dire warnings, Soares does not believe that human extinction is an unavoidable destiny. He maintains that there are still actions that can be taken to prevent the worst-case scenarios. His primary recommendation is to intentionally limit the scaling of AI systems that are specifically designed to mimic or exceed human intelligence.

He makes a clear distinction between using AI for narrow, beneficial applications and building general intelligence. For example, he supports using AI to find new treatments for diseases like cancer. However, he argues that we should not be