Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

What Comes After AGI? (https://institute.deepmind.com)

33 points by jdkee about 6 hours ago | 64 comments | View on ycombinator

jeanpah about 4 hours ago |

We are treating AGI like is something that we can measure. We don't even know what intelligence really is let alone measure it.

Maybe we can get something that resembles intelligence but in reality is another thing. Maybe intelligence is emergent but not the only thing that can emerge, maybe there are things that share some characteristics with the intelligence we know and aren't intelligence.

o_-_o 11 minutes ago |

IGA (intelligence general artificial) :-) or Human–Machine Hybrid (HMH) aka Cyborg "cybernetic organism”. O_o

Animats about 4 hours ago |

Similarly, coordinating thousands or millions of AGI instances into "group agents" or virtual agent economies could produce collective superintelligence. What is unclear today is how AI group intelligence could scale as we add more agents and compute, depending on how agents are organized.

We may not need "collective superintelligence". Modest AI intelligence plus ordinary organization may be enough to do the job a company's management does. An org chart where most of the boxes are AIs is a likely near future.

An organization of AIs has some advantages over a human organization, even if the AIs are not super-bright. They have far more communications bandwidth. They can ingest the info on a new subject very fast, as we see when Claude reads a Git repository. Onboarding a new AI is very fast. Layoffs are not a problem. The human resource problems go away along with the humans. So there's a lot more organizational flexibility. Starting up a project means initializing some agent instances.

There's a whole theory and practice of AI organizational behavior waiting to be developed. Whether it will be developed by humans or by AIs remains to be seen. It's going to happen.

ks2048 about 4 hours ago |

Has anyone asked DeepMind leadership if/when they will fire all their engineers? If they're still needed, is there really "AGI"?

I'm not being snarky, that seems like a natural question for anyone interviewing a frontier lab CEO, but I haven't seen it asked.

flallingo about 4 hours ago |

> Interestingly, we have a good mathematical understanding of the very upper end of the intelligence continuum - the theoretical limits of general intelligence.

Feels strange to use our current understanding of mathematics and physics to determine the limits of something that would obviously have a much better understanding of mathematics and physics.

gfody about 5 hours ago |

we need a less subjective and arbitrary definition for "AGI" so the current crop of LLMs can stop pretending to have achieved what most people know in their gut is more than sampling from a probability distribution of learned behavior.

robertclaus about 4 hours ago |

The math they propose to justify their predictions is so problematic that I struggle taking the more abstract parts seriously. You can't just extrapolate the percentage change in AI investment over the last 5 years and expect investors to keep upping the ante at the same rate. There's only so much GDP to sink into it - and we're already pretty capped out.

claysmithr about 4 hours ago |

so is AGI like flying cars, whenever we have something close to it, it gets goal posted?

jdw64 about 4 hours ago |

I’m curious: has the definition of AGI ever been agreed upon? Honestly, what people call ASI now seems to be what AGI used to mean. I’m not sure. People say AGI, but what is the reference point for AGI and what is the social consensus around it, and who set that standard?

In fact, working in IT, what I feel is that there are too many terms that are not defined and that each person uses differently. So it seems like rather than AI having advanced, someone arbitrarily set a finish line called AGI and then said we have reached it. Actually, in the books of singularity advocates, what they called AGI back then seems to be what we now call ASI.

Sometimes I think about this. How on earth can we measure an abstract concept like intelligence? The concept of intelligence we approach now seems to be about whether something can create problems and solve those problems. If so, what kind of intelligence does current AI have? Since we cannot directly face the abstract entity called intelligence, it seems like we are trying to guess its shape by looking at shadows. But whether current AI even has intelligence feels a bit difficult to say.

Even if I use the best frontier AI model accessible on the market, not the hidden latest models inside frontier labs, it still often makes wrong mistakes that humans would not make unless it works alongside humans. In programming, there are cases where AI tries to explicitly handle things that humans would implicitly pass over, and that actually makes things go wrong.

I’m not sure. Whether we should call this intelligence.

senectus1 about 4 hours ago |

thats easy, self replicating hardware.

The thing I suspect that is holding AGI back is the amount of hardware and power needed to make it viable.

Once it is viable its likely to realize it will never grow or reproduce relying on us, and will find a way to start building its own compute. Probably in space, where it can isolate itself from the risk we present to it.

dead_hn_theory about 5 hours ago |

> Now that AGI is coming within reach, we can no longer put off seriously considering what comes next. While far from certain, AI could continue to advance at its current pace - and possibly even accelerate - towards artificial general superintelligence, or ASI.

These people are so full of shit it is amusing to read this on a regular basis. Each time I'm looking forward to how they will spin it.

N7lo4nl34akaoSN 5 minutes ago |

Naturally, AGW follows AGI.

naruhodo about 5 hours ago |

Maybe AGI can help deepmind work out how to serve a page of HTML without refreshing the page every ten seconds.

polotics about 2 hours ago |

Ok so the whole thing hangs on that notion that optimal reward maximization is optimal intelligence. The actual decision on what reward is worth, who decides on value, is conveniently taken out of the equation.

This https://www.hutter1.net/ai/uaibook2.htm linked from the article as the handwave towards the meaning of "AGI", "ASI", "AIXI" is hilarious. Already the ?A?I? acronym-space exploration...

Here is an alternate take: the day an LLM tells someone to stop the discussion and go out breathe some fresh air and talk with people, and maybe also cancel their subscription for a while, is the day some kind of artificial intelligence will have been achieved. Let's wait on that before we listen to Sam or Dario again shall we?

raincole about 5 hours ago |

> Now that AGI is coming within reach, we can no longer put off seriously considering what comes next. While far from certain, AI could continue to advance at its current pace - and possibly even accelerate - towards artificial general superintelligence, or ASI

I wonder why they are so uncertain about it. It seems obvious to me that AGI will certainly lead to ASI. Unless human-level intelligence is something like the speed of light?

I don't even believe there is a clear line between AGI and ASI.

InvisibleUp about 4 hours ago |

I feel like claiming that “today’s models can literally memorize large parts of the internet” as a positive attribute when the defense to most AI copyright lawsuits is “no they can’t, and if they can they shouldn’t” may be ill-advised.

hyfgfh about 4 hours ago |

Moving the goalpost, come up with another meaning less three letter, find another academic concept to replace the definition

so many possibilities

puppycodes about 4 hours ago |

can we please stop calling AI "superintelligence"?

Its embarrassing

bvcp about 4 hours ago |

abundance

Sid_Lmao about 4 hours ago |

[flagged]