Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Thomas Nagel says that it's criminal to build AI that feigns consciousness (https://www.wired.com)

19 points by eph0m0ral 1 day ago | 33 comments | View on ycombinator

tim333 about 14 hours ago |

I don't buy it being down to the substrate. I haven't noticed people with cochlear implants suddenly lose true consciousness because there are some chips in there.

Hinton was quite interesting discussing it https://www.youtube.com/watch?v=hDyUuc4utmk&t=85s

His take: If you give a multimodal LLM a camera and a pointer arm and say point at some objects, say an apple, it can do it. If you put a mirror in front of the camera it would point the wrong way and if you explained that to the LLM it would probably say - ah I see - the mirror makes it look to me that the apple is in a different location to where it is, which implies it has a subjective experience of where the apple is.

My take is human consciousness got there through evolution as a practical thing for survival and success. The brain is wired to bring together relevant information like there's a bear over there, it seems scary - to both the decision making bit - run or not - and the language and memory bits which are probably a bit LLM like - they can churn out words saying "wow, I saw a bear".

The consciousness bits are what the language like bit is fed, as opposed say to the gut muscles pushing around yesterday's food which are not wired up to that so you don't have conscious awareness of that, usually.

nextaccountic 1 day ago |

> Like everyone, I’m tremendously impressed by the lifelikeness of these systems compared to the history of earlier AI. But the disposition of some philosophers to take seriously the question whether these things are conscious or might soon become conscious is just ridiculous. They are assuming that the physical substrate is irrelevant. They all think that the material conditions for consciousness can be given in functionalist terms.

> (...)

> Functionalism is substrate-independent. It’s more than likely that the truth is one which links physical processes and systems of certain kinds to mental phenomena. Complicated data centers have absolutely nothing in common with the physical character of a nervous system. We have strong reasons to believe that while there may be consciousness elsewhere in the universe, it would require the appearance of some kind of organism with something analogous to a nervous system.

Why should we assume our substrate is the "right" substrate for truly conscious beings?

Perhaps we have fake consciousness too (just like programs running on silicon), and some advanced alien being could look at our biological neurons stuff and immediately recognize our fakeness. Perhaps those aliens could proceed to assert that their own substrate is the only one that could possibly be conscious - all the other substrates (like ours) being fake.

We couldn't possibly know, because we don't know what consciousness is.

gnabgib 1 day ago |

Please use the original title (A Famed Philosopher Takes on AI Consciousness)

> https://news.ycombinator.com/newsguidelines.html

undefined 1 day ago |

undefined

calf 1 day ago |

Ok I'm on the total opposite of this guy, I am almost tempted to write a short letter telling him why Thomas Nagel, famous American philosopher, is wrong on this.

Basically his opinion is derivative of stochastic parrot theory. He admits as much in the interview. (Since stochastic parrot theory is wrong, I could stop right here, and so should he if he had a truly sufficient argument in the first place...)

He is committing the mistake as Scott Aaronson says, justa-ism. It's "just a language model" therefore it cannot possibly really be intelligent/alive/conscious/goalpostmoving. Followed by some tricky rhetorical move appealing to human exceptionalism.

The fact is that philosophers would do well to take computational complexity theory, the one involving actual math homework. All Nagel's woo-woo talk about a substrate and a need for an "analogous nervous system" shows yet again how CS as a nascent field, and theoretical CS in particular, has not fully been internalized by other disciplines.

Nagel says a bat is conscious. But an LLM with trillion dimensional neuron parameters, obtained through stochastic gradient descent, cannot possibly conscious because it is on the wrong substrate? Really?

In fact it is the other way around. The true "substrate" is the neural network model, of which humans (and bats) are merely one example of. That is the correct way to understand what LLM computation is. The abstract LLM neural net is what Alan Turing himself envisioned as the alternative to classical computation.

cajal about 23 hours ago |

lmao nothing makes the ai bros on this site more mad than suggesting claude isn’t a sentient being