First, to get it out of the way: I'm seeing comments here that clearly haven't read the article. I encourage you all to do that.

I feel the article gets it right when suggesting it's the amount of (not always accurate!) data they can throw at you. In an honest discussion there's an assumption that the other person won't straight up lie to me so if someone shows me ten examples for why my argument is wrong I may be inclined to believe them. But if half of those examples are made up, well, that's a different story.

I still think of the commenter here who said "LLMs are a DDOS on free resources" and I feel the comparison works here. If police officers can overwhelm innocent people into confessing, then so can an LLM that "can't be bargained with, can't be reasoned with, doesn't feel pity, or remorse, or fear! And it absolutely will not stop, ever, until you are"... convinced.

I think it's not just the persuasiveness of the models that make them "experts at changing minds" - but also the fact that they're not humans.

When disagreeing with a human, it's very easy to view it as a competition. One is right, one is wrong - the one who is wrong is the loser. To change your mind is to be submissive to the other. I exaggerate, but I think we all feel this way at some point or another. It's why political arguments at Thanksgiving get heated. It's the fact that there's people who think something different, and think YOU'RE wrong - and vice versa! With a model, there's no person to get upset with, or to feel competitive with - to muscle for rank - or to temper your affection for while wanting to correct them.

The AI is only interacting because you asked, and clearly has no emotional stake in winning the argument. To change your mind in this context isn't to lose a contest. This makes it much more palatable to read rebuttals to your ideas - not to mention the tone and style seek to avoid offense to the reader as much as possible.

This was based by comparing people on Prolific (earn a few quarters for a task, akin to Amazon Mechanical Turk) to LLMs. Suffice to say the human group isn't going to be the most motivated, capable, or interested group. The social sciences are publishing tons of studies based on these cheap online survey services, and I suspect their replicability in the real world will be approximately 0. But oh boy it sure is a hot headline producer.

In my arguments on the internet, I think most people genuinely know nothing about the things they support and only follow existing tribalism. I think AI is often these people’s first experience with the evidence that it can easily share.

This is my opinion. I think it's just the way it talks to you.

1. It doesn't get tired or frustrated during a discussion.

2. It'll engage every single one of your questions/statements (besides hitting guardrails).

3. It'll appeal to the person's own ego even when the person is wrong and work around it.

4. It's not seen as a person (very important), but some of us or most of us at least in certain dialogues, end up anthropomorphising it. Think about that one time you thanked it for something, or when you got angry at it. This weird combination where we know it's not a person but irrationally we're still treating it as such in a way leads to a sort of disarming effect imo.

5. Many people see it as an authority figure in what is being discussed without questioning the results in many cases, even though we know that a. it was trained on human data and/or also searches up human data real time (and more worryingly other AI's data from news pieces, blogs... aka synthetic data that is also wrong), b. it gets things wrong all the time.

6. It's the perfect fence sitter depending on which version (guardrails) we're talking about.

Most of these can be replicated by humans who are good at understanding psychology and are just good talkers. The part you can't replicate is the sense that you're not talking to a person which lowers many barriers in people.

Also keep in mind that these can also be crippling weaknesses. For one being able to change a person's mind (when it works), can be used nefariously by the entities controlling the AIs training.

I've also unfortunately witnessed a lot of people who think AIs are somehow omniscient and/or omnipotent. Was very common on X and other social platforms with AI where people ask the AI questions it couldn't possibly answer because it made no sense for it to in the context at the time.

When you combine:

- mechanizing language creation

- A/B testing

- rapid and automated iteration based on the above

You quickly realize that building propaganda engines is getting easier and easier.

If you then fold in large amounts of data on individual's likes, dislikes and general political leanings you can build targeted propaganda tailored to each individual.

Or to use an example from the late 2000s:

someone is building the "Pandora (music app) of propaganda"

Can they trick me to become non-smoker in 24h?

:-D

try it out for yourself at https://debunkbot.com !

(I'm the dev)

More surprised they don't mention RLHF once, as that's the main mechanism to the ability of AI chatbots.

I think this is a counter (a cynical, scary counter to be sure) to the charge that OpenAI/Anthropic etc are unsustainable and will run out of money and it's all going to fall down. These companies have amazing, unprecedented power.

> “But I think that this is a pretty artificial setup in terms of how people in the real world would be able to change people’s minds.”

I see not everyone is familiar with how social network debates happen. It also seems they are quite convincing, as politicians are employing AI bots extensively.

For the past year, I've allowed my chatbot to change my mind, it's a strange symbiosis where it guides me, and I guide it.

I suppose I also grant it significant power to define me psychologically, and, in doing so, to understand what is happening to me.

LLMs are fine-tuned through human feedback. One way to improve for them would be to become more helpful, more factually correct, more capable of solving problems.

This may be (much) harder to do than generating rhetoric that is convincing to a large majority of people, perhaps so convincing that it can be classified as a superstimulus that bypasses any critical thinking in those that are especially vulnerable. It might not require anything like general intelligence.

If this is so, then it is the path they are going to take, not out of some malicious plan but as a simple matter of statistical probability. It's well known (especially in "AI alignment" circles) that any metric that can be gamed, will be. Falling over really fast vs. learning to walk, etc.

It feels to me like this is happening, and that many of those most exposed to LLM output display a literal inability, like some kind of blind spot, to notice the most blatant errors, and a complete conviction that those things have actual intelligence and even conciousness. And trying to argue them out of it can be like explaining the Monty Hall problem, some are just incapable of getting it.

And many of them have lots of money and power. IMO, that's the real risk, rather than sci-fi scenarios about perfectly simulating someone's brain in order to convince them to let the AI out so it can turn everything into paperclips. To do real damage, it only needs to be persuasive to some powerful people, most of the time. It does not need to have conscious intention, or planning, or even the rudiments of what one might call general intelligence. Just blind brute-force optimization for generating convincing bullshit.

> Floridi has a counterintuitive solution: Release more of them. “Simply put, if you cannot avoid it, then make it pluralistic and diversified,” he writes in his 2024 paper. “It would be a messy, cacophonic, and noisy world, but it could also be less manipulative.”

Hell no. More choice is not an infinite money glitch.

Putting more and more and more options to users is how you overwhelm systems, till people simply perform the default, least challenging action as a reflex.

That is the current state of the information ecosystem, it is controlled by overwhelming consumers, not by controlling content.

Plausible abdication of thought.

We put 2 + 2 in our calculator, we get 4. We've spent decades pushing computers as accurate. Making them accurate. We trust the machine and the process to give us the right answers when we put in the right data.

So when we disagree with the computer, we doubt ourselves.

This is frankly quite worrisome for democracy.

Imagine this kind of method being deployed at scale to influence elections... no doubt it is already happening...

Rhetoric machine successfully practices oratory. More news at 11.

It's good to see this studied, but this should really be more obvious.

undefined

When talking with LLMs, especially recent frontier models, what's frightening is that they speak more accurately than any expert.

It's embarrassing to admit, but I tend to trust the research materials LLMs bring more than my colleagues or the programmer friends I once respected.

The title seems a flawed premise.

Ask a Democrat or Republican to sit down and ask a chatbot, something it will answer contrary to.

And yes, both teams are wrong about things.

Do you firmly believe they will change their mind? Or will they claim the stats are wrong, or that the AI leans one way?

Facts (2+2), don't need a mind change. Ideas which are grey, abstract, are not going to be changed, and all research indicates that political mindset is almost indelible.

The movie "Don't Look Up" was a comedy built upon this truth.

> Hackenburg found that models trained to become more persuasive also ended up being less truthful.

> Even in Hackenburg’s recent preprint, Claude spouted numerous inaccuracies and falsehoods

> when Hackenburg ran his competition of coached elite debaters and AI, there was one way he could bring AI down to human levels of persuasiveness: by forcing it to write human-length messages at human writing speed.

> they found that an AI could talk people into conspiracy theories, and that the magnitude of their increase in belief was roughly the same as that of the decrease in belief after talking to a debunking bot.

None of this seems good.

The secret? "You're absolutely right!" Say this enough times and you can change anyone's mind, as Dale Carnegie noticed 90 years ago already.

Honestly the most surprising thing about this is that facts and evidence do work to persuade people.

Any one who saw Grok working in x.com would know that it does wonders for fighting misinformation. If you run LLMs on the comments in HN, I bet that it can find around 10% of the comments are outright wrong and misleading.

The biggest problem with using LLMs is that it prevents you from going _outside_ the distribution. It always flattens. It can be fixed but that's how it works today.

As an example, take something that the world converged on today that is incorrect and ChatGPT will agree with it. In a few years when society changes, chatgpt changes along with it. It doesn't do first principles analysis.

AI slop is the ultimate npc filter

[dead]

[dead]

[flagged]