Hype, yes, and "...its just Claude with N tools and X and Y..." and okay, but if that's all it takes to run our lives, then, yeah, we're pretty close to being cooked.
We're not all that complicated really. Most people really *could* be run inside a Very Small Shell Script.
Or as Anthony Hopkins in "Westworld" put it:
"Intelligence, it seems, is just like a Peacock really..."
"Yes, in two years AI agents are going to be absurdly powerful. Wait for it."
With every release of an agent that doesn't really work I update a little against this, actually. Not that I expected them to Really Work yet - but I'm kind of convinced by Colin Fraser's point that you can't really get goal-oriented behaviors out of LLMs yet: if they have goals at all, it's to simulate text, and simulating text that looks like solving problems doesn't suffice to solve lots of hard problems. It won't be critical enough of its own work, it won't have high enough standards.
Specifically I update against something like "a 10x scaled up GPT-4.5 with CoT and great scaffolding with aggressive agent-flavored RL" could produce good-enough agents. In that world, the gap probably requires a transformers-sized breakthrough, so to speak (though given the manpower thrown at this problem and the crazy-high benefits we might get that).
Yeah, I'm not really seeing why an agent _wrapper_ is that big a deal. As you said:
"simulating text that looks like solving problems doesn't suffice to solve lots of hard problems" which I view at: LLMs + CoT still has a substantial way to go (maybe another breakthrough, like LLMs themselves) before it has the core ability to _reliably_ solve most problems that humans can solve. _Given_ that capability, putting an agentic wrapper around it sounds like, as Zvi said, a month of work.
Many Thanks! I don't see a sharp boundary between goal-oriented behaviors and reliable question answering. One can always _ask_ a chatbot something like: "If I want to get from New York to Boston with a minimum of expense, hassle, and discomfort, what should I do?". And a reliable answer to this question would be a plan for travel from New York to Boston. Now, how far we are from _getting_ this and similar answers reliably is another question...
I know you wrote recently on On Writing that you don't write the headline first, because that's not dissimilar from The Bottom Line, but honestly you have consistently great headlines when they're not purely functional. Never trust a product named after a DLC Dark Souls boss...
Also want to note that I feel like reading enough of these "incept via hype" blow-by-blows in AI has given me a useful general Inception lens for the world, and it's sorta depressing how often one finds that pattern in modern adam kadmon. Maybe I don't need to make myself finally watch actual Inception, after all.
Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/the-manus-marketing-madness?r=67y1h&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true
Hype, yes, and "...its just Claude with N tools and X and Y..." and okay, but if that's all it takes to run our lives, then, yeah, we're pretty close to being cooked.
We're not all that complicated really. Most people really *could* be run inside a Very Small Shell Script.
Or as Anthony Hopkins in "Westworld" put it:
"Intelligence, it seems, is just like a Peacock really..."
https://youtu.be/OjsGJOXhPGU
Yes, this is correct. Once you have the thinking piece, the rest is just a matter of time.
"Yes, in two years AI agents are going to be absurdly powerful. Wait for it."
With every release of an agent that doesn't really work I update a little against this, actually. Not that I expected them to Really Work yet - but I'm kind of convinced by Colin Fraser's point that you can't really get goal-oriented behaviors out of LLMs yet: if they have goals at all, it's to simulate text, and simulating text that looks like solving problems doesn't suffice to solve lots of hard problems. It won't be critical enough of its own work, it won't have high enough standards.
Specifically I update against something like "a 10x scaled up GPT-4.5 with CoT and great scaffolding with aggressive agent-flavored RL" could produce good-enough agents. In that world, the gap probably requires a transformers-sized breakthrough, so to speak (though given the manpower thrown at this problem and the crazy-high benefits we might get that).
Yeah, I'm not really seeing why an agent _wrapper_ is that big a deal. As you said:
"simulating text that looks like solving problems doesn't suffice to solve lots of hard problems" which I view at: LLMs + CoT still has a substantial way to go (maybe another breakthrough, like LLMs themselves) before it has the core ability to _reliably_ solve most problems that humans can solve. _Given_ that capability, putting an agentic wrapper around it sounds like, as Zvi said, a month of work.
> but I'm kind of convinced by Colin Fraser's point that you can't really get goal-oriented behaviors out of LLMs yet
Honestly, I don't expect to get goal-oriented behavior by scaling LLMs. But I do think we're getting dangerously close anyways.
I could argue exactly why I fear that, but I doing so seems like a bad idea at this point.
Many Thanks! I don't see a sharp boundary between goal-oriented behaviors and reliable question answering. One can always _ask_ a chatbot something like: "If I want to get from New York to Boston with a minimum of expense, hassle, and discomfort, what should I do?". And a reliable answer to this question would be a plan for travel from New York to Boston. Now, how far we are from _getting_ this and similar answers reliably is another question...
I know you wrote recently on On Writing that you don't write the headline first, because that's not dissimilar from The Bottom Line, but honestly you have consistently great headlines when they're not purely functional. Never trust a product named after a DLC Dark Souls boss...
Also want to note that I feel like reading enough of these "incept via hype" blow-by-blows in AI has given me a useful general Inception lens for the world, and it's sorta depressing how often one finds that pattern in modern adam kadmon. Maybe I don't need to make myself finally watch actual Inception, after all.
Inception (the movie) Is Not About Inception.
(and didn't hold up as well on 2nd viewing but is still very good, 4/5 stars.)