Keith Schacht’s Weblog

Dario from Anthropic is the most articulate and thoughtful person I’ve found on the subject of AI. In this interview he made a fantastic point I had not heard:

Let’s say it was 1995 and Moore’s Law was making computers faster and everyone was saying, “some day we’ll have super computers and we’ll be able to sequence the genome and do all these great things.” But there is no discrete point where we pass a threshold and then have super computers.

Super computers is a term we use but it’s a vague term we use to describe computers which are a lot faster than what we have today. I feel the same way about AGI. There is a smooth exponential. And if, by AGI, you mean AI is getting better and better and will do more and more of what humans do until it’s smarter than humans and it will continue to get smarter from there, then I believe in AGI. But if AGI is some discrete thing, which is how many people talk about it, then it’s just a meaningless buzzword.

But he then goes on to reference his essay in which he gives one of the best descriptions of an almost discrete point I’ve read! He doesn’t call this AGI but instead uses a great descriptor:a country of geniuses in a datacenter:

In terms of pure intelligence4, it is smarter than a Nobel Prize winner across most relevant fields – biology, programming, math, engineering, writing, etc. This means it can prove unsolved mathematical theorems, write extremely good novels, write difficult codebases from scratch, etc.

In addition to just being a “smart thing you talk to”, it has all the “interfaces” available to a human working virtually, including text, audio, video, mouse and keyboard control, and internet access. It can engage in any actions, communications, or remote operations enabled by this interface, including taking actions on the internet, taking or giving directions to humans, ordering materials, directing experiments, watching videos, making videos, and so on. It does all of these tasks with, again, a skill exceeding that of the most capable humans in the world.

It does not just passively answer questions; instead, it can be given tasks that take hours, days, or weeks to complete, and then goes off and does those tasks autonomously, in the way a smart employee would, asking for clarification as necessary.

It does not have a physical embodiment (other than living on a computer screen), but it can control existing physical tools, robots, or laboratory equipment through a computer; in theory it could even design robots or equipment for itself to use. The resources used to train the model can be repurposed to run millions of instances of it (this matches projected cluster sizes by ~2027), and the model can absorb information and generate actions at roughly 10x-100x human speed5. It may however be limited by the response time of the physical world or of software it interacts with.

Each of these million copies can act independently on unrelated tasks, or if needed can all work together in the same way humans would collaborate, perhaps with different subpopulations fine-tuned to be especially good at particular tasks.

— Dario’s interview with Lex Friedman