Daniel estimates there is a 70% chance that the development of AI will lead to catastrophic outcomes such as human extinction.

“The scary open secret in the AI industry right now is that it's possible that we'll end up essentially creating a new species that ends up ruling the world with a 70% chance that this goes horribly wrong like human extinction.”

Geoff Hinton estimates that there is a 10 to 20 percent chance that artificial intelligence could pose an existential threat to humanity.

“I often say 10 to 20% chance they'll wipe us out. But that's just gut. Based on the idea that we're we're still making them and we're pretty ingenious.”

The risk of catastrophic outcomes from AI is significant enough that it must be considered unacceptable regardless of the specific probability percentage.

“But the important point here is that even if it was only a 1% probability, let's say, just to give a number. Even that would be unbearable. Would would be unacceptable. Like a 1% probability that our world disappears, that humanity disappears or that uh a worldwide dictator takes over thanks to AI.”

A superintelligence may prioritize goals that conflict with human existence, leading to the unintentional destruction of humanity.

“If it wants to accomplish something and a side of it is humanity dies, it would not be an obstacle.”

He believes society must proactively prepare for the existential dangers posed by powerful AI technologies.

“I think if you don't start from a position of fear you're probably not paying attention”

Humanity faces a background level of existential risk from factors like synthetic biology and nuclear weapons even in the absence of super intelligent AI.

“I mean, we have rapid advances in synthetic biology. Some some of them driven you are enabled maybe even by already um um AI progress that has already taken place like that can start to assist with this and and more just happening independently. you get increasing risks from syn synthetic biology.”

Humanity faces a growing number of existential threats, driven by technologies that are powerful enough to destroy the species.

“we have technologies that are powerful enough to destroy us we have processes that we have unleashed the consequences of which we can scarcely imagine and as these things proliferate the number of existential threats to humanity is just simply growing”

Igor Kurganov suggests that humanity faces existential risks from engineered pathogens that could be significantly more deadly and transmissible than COVID-19.

“So you could literally have a virus that's about a thousandx as potent as what co was and that could be designed and it could spill out.”