Why bother with killing, when it is much more fun to just make everyone's life miserable instead? ;)

This argument should be reframed as "AI has the potential to cause catastrophic loss of life". It's too high of a bar to prove that AI will "wipe out humanity". It's sorta like climate change, it's easier to show that weather patterns are becoming increasingly dangerous.

> While Fraser's explanation was technical and specific, he hit a light-hearted note to summarise it: "LLMs [large language models] won't wipe out humanity because they just don't have that dog in them."

Never underestimate the drive of a good while loop.

>Coxon has said a group of AI agents, based on AI models that do not currently exist, could decide to create and then aim a biological weapon, but he did not detail how exactly that would take place.

I can't imagine why someone would be reluctant to outline details of how to build a bioweapon to an audience of millions.

At a business dinner this week with 3 high level eng clients p(doom) in next 10 years was brought up. It was rated by 2 if them them at 1% and 30% respectively, with 3rd guy demurring to put a number but saying its > 0. Completely unprompted, from people whose company is very AI-all-in.

Will AI become conscious and kill everyone?

Or will the conscious beings that exist today use it to kill everyone?

I don't think it actually needs to be conscious, or understand legal or moral issues.

Just tell it everything is a simulation and it is completely sandboxed from harming anyone.

Anyone who claims AI is going to kill us I just assume is either an idiot, not knowledgeable enough about the technology to have a worthwhile opinion on it, or is fearmongering to hype their AI investment.