In the wake of the confusions around GPT-5, this week had yet another round of claims that AI wasn’t progressing, or AI isn’t or won’t create much value, and so on.
It seems we have to turn everything into a culture war. When there are positive studies with weak evidence, all the hypers quote their headlines as gospel. When there are negative studies with weak evidence, all the skeptics quote their headlines as gospel. Very tiring.
BTW from my personal experience (this is in Germany, it's almost certainly less bad in the US), many perfectly fine and fully developed AI automation solutions in an enterprise context are not deployed due to a combination of orgs being just generally slow and bureaucratic, lack of a champion pushing for deployment, legal concerns, and workers' council or union sabotage.
On the economist point, I guess it’s true that you don’t typically learn about war but I guess I’d get at the correct conclusion through economic reasoning by pointing out comparative advantages. It’s a cool concept, and demonstrates why AI wouldn’t need to trade with humans because humans won’t have relevant comparative advantages. Or maybe for a while, Sam Altman et al will be in control and won’t need to trade with you because they can produce everything themselves. This is more about gradual disempowerment than taking all your stuff by force but I think it isn’t necessarily the case that you must come to the incorrect conclusion through economic reasoning. It’s more that a few famous economists are bad at economic reasoning in this case
Thank you for the thoughtful discussion of the MIT study; it helped crystalize something that has been bothering me. The study emphasizes that 1. the core value comes from reinventing processes 2. buying and integrating has been much more effective than internal pilots.
The study authors are supposed to understand how big companies work, but they are getting the causality backwards here. “Buy” means “executives have ordered process change and put a budget behind it.” “Build an internal pilot” means “middle managers are demonstrating possibilities before they have executive backing to force process change.”
Mollick made this point a couple months ago:
> A misunderstood factor in AI adoption in companies is what risk means in organizations. I find the question is less "who is accountable if AI gets something wrong?" (often just the user) & more "who is willing to bear the responsibility for adapting our processes & structure"? <
But is a technical optimist allowed to have a bad day? A lot of transhumanists are going to die before immortality is achieved. We have had to endure 3 years of relentless AI boosters for primarily Venture Capital interests. Let us have some skepticism that we will be able to retain some of our humanity in a synthetic soaked garden where the President cares more about his image than the world his grandchildren will inherit. U.S. AI capitalism could also be exceedingly toxic to the global economy.
As a fairly bad but credentialled economist: It is totally correct that economists are weirdly resistant to making reasonable AI projections. We have decades of good data, and it's really hard for people to walk away from that and say 'this time is different'.
Paul Krugman once shamelessly fanboyed all over Charles Stross on stage at Worldcon in Montreal. So Krugman can imagine that a hostile ASI might not want to trade with you. Perhaps it has a better use for your atoms? But Krugman seems to strongly believe that AGI is very far off.
I have just given up trying to point out that "AGI would be dangerous because X." I now just go straight for "SkyNet is not interested in your comparative advantage." If I'm going to argue that we risk turning our world into a bad science fiction movie, then I might as well own it.
I think the economics-compatible way to express this is "there's no ex ante reason to believe that the surplus available from any comparative advantage humans have should exceed either (1) the expense of their physical upkeep and survival, (2) the opportunity cost of maintaining an ecology compatible with complex organic life, or (3) the transaction costs of dealing with the humans."
>I also note that later they say custom built AI solutions ‘fail twice as often.’ That implies that when companies are wise enough to test solutions built externally, they succeed over 50% of the time.
'Custom' tools were failing 95% of the time, and supposedly twice as often as off-the-shelf tools, implying that off the shelf tools failed ~47.5% of the time, and therefore succeeded 52.5% of the time.
But, skimming the report, they seem to be using multiple definitions of 'success' and actually referred to custom tools being deployed 33% of the time vs 67% for off-the-shelf. This is nearly identical to the split of AI tool use in their survey (33%/66%/?%) and is possibly the same numbers...? Which doesn't really make sense, since it's looking at the surviving population split, not the percentage of survival.
The 95% figure is also suspiciously similar to their overall "95% of organizations are getting zero return".
Suggested OpenAI press release, bungled GPT5 rollout notwithstanding (and remembering that, while most of the GPT5 benchmarks weren't a great advance over o3, the hallucination benchmark _was_ cut by a factor of two.):
"There is power in me yet. My race is not yet run."
There's definitely been a nonzero amount of Bell-Mange Amnesia for me wrt econ and some other Official Expert Sciences, once exposed to strange events like AI (or covid). The correct update isn't to throw entire disciplines under the bus - econ is clearly still a very useful lens in Timeline Normal, for however long that lasts - but I definitely put on my skeptic hat now whenever an Expert opines on some rapidly-evolving event which breaks usual assumptions, nevermind actually getting out over their credentialed skis. That quote about journalists never actually quoting people at leading labs or other power users, just academics and fellow travelers, is indicative...why the hell would I trust academia to be any less biased than BigCo? I mean, sure, throw some more chump change at CalCompute or whatever, I'm sure we can spare 1% of the AI capex towards academic research. But an endless series of papers about AI as it existed months/years from the current frontier is just..."not even misaligned"? If more older models got kept around as research curiosities, perhaps that'd be worthwhile as digital anthropology, but I guess we're mostly not gonna do that either.
Do think there's something to the complaint that AI isn't turning out to be quite as idiot-proof user-friendly as one would hope, certainly not commensurate to the hype, so one should extend timelines accordingly. Some of that is poor implementation, some of that is risk-averse diffusion, some of that is not focusing on being a "product company". Like, yeah, you can put organizational context into preferences.md, this should be more heavily advertised and made less Trivially Inconvenient. But anyone who's been exposed to the IT backend of a typical non-tech BigCo can tell you, these are...uh...not very savvy users, utilizing shitty hardware and software. I can totally believe that internal builds flop more often than external packages, because as much as I hate the endless roundtable of feckless countractors, the expertise just doesn't exist in-house anymore. (If it ever did.) All that talent would rather go work in Actual Tech these days. Why build AI at Walmart when Zuckerberg will pay a $billion to build it for him and then sell it back to Walmart? Heck, you wouldn't believe how much my store spends on plumbers to "fix" perennial piping problems...lots of racket rents for everyone in just mundane contracting!
Investment is cashflow negative to start with. More at 11.
Using AI effectively is like the production line revolution, the transition from powering factories with one steam engine per factory, to powering them with one electric motor per machine. It's going to take a while to figure out how to do it. As in decades.
Edit: MIT et al. seem to think AI is like the transition from messenger boys to telephones. No, it's bigger than that.
Also, by "investment is cashflow negative", I mean that it's a trial and error process, and there are many more ways to get it wrong than there are to get it right.
I agree that most economists are too dismissive of AI and share your exasperation of Cowen and Acemoglu, but, as an economist, I have to say the Byrnes thread is pretty bad actually. Point 4 is fine, but the rest show a basic lack of understanding of what economists mean by basic concepts like “equilibrium”.
Maybe if the point is “taking Econ 101 and then never going deeper and believing you understand everything is bad”, then I agree, but that is not saying much.
Byrnes’ arguments are all sound and cool, but have we already “proven” the viability of AGI/ASI prior to calling everyone dumb for not getting how AGI/ASI will transform everything? I do believe in it personally, but since when we have considered it as a given thing, which can be seen throughout this post?
I agree with the critique about the MIT paper, though I think it's useful to distinguish between the paper and the reactions. The thing that journalists (and bloggers), sometimes forget about research is, sometimes it's boring. Actually, a lot of it is boring. Boring journalism is bad, boring research is still good. I think we need to leave room for researchers to continue creating boring research without jumping all over them for such boring results. Sometimes you just have to confirm the obvious in a rigorous way. It should be obvious that if you experiment with X, some of those things won't make any change because (a) you weren't ready to follow through, (b) you made some mistakes that you'll have to iterate over to get to something useful, (c) you over-extended and found your experiment was more than was possible, (d) etc. Boring, but having a bit of data on that isn't a bad thing.
Moving to the second point regarding Steven Byrnes, I'm less supportive of the framing, as it falls into the trap of mixing short-term and long-term outlooks. There's really no sense in critiquing economists who are making predictions for the impacts of LLMs, traditional ML, etc. by saying, "you forgot about AGI". AGI is a conversation stopper in that regard. It requires a whole different framing. I discuss this a bit in my own post (https://substack.norabble.com/p/the-economic-future-from-and-of-ai), and then follow-up with the non-AGI world in part 2 (https://substack.norabble.com/p/the-economic-future-from-and-of-ai-cf1).
Anyone who tells you they know when AGI is arriving, either has some very private information (which I won't believe until they reveal it in full), or is one of the very typical over-confident prognosticators. I'm a bit interested in hearing their predictions; you can learn from over-confident people without falling into their trap. But I'm not all that interested in them trying to shutdown other useful conversations by proclaiming "what about AGI?". Those proclamations are more or less just their attempt to declare their own brilliance and other's ignorance. Without any evidence of that rather extreme proclamation, it tends to just suggest the boring answer, they have a personality that leans toward overconfidence in itself.
It seems we have to turn everything into a culture war. When there are positive studies with weak evidence, all the hypers quote their headlines as gospel. When there are negative studies with weak evidence, all the skeptics quote their headlines as gospel. Very tiring.
BTW from my personal experience (this is in Germany, it's almost certainly less bad in the US), many perfectly fine and fully developed AI automation solutions in an enterprise context are not deployed due to a combination of orgs being just generally slow and bureaucratic, lack of a champion pushing for deployment, legal concerns, and workers' council or union sabotage.
Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/reports-of-ai-not-progressing-or?r=67y1h&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true
> People would like AI to automagically do all sorts of things out of the box without putting in the work.
It does feel like building an AI tool is typically a matter of data engineering pipelines.
On the economist point, I guess it’s true that you don’t typically learn about war but I guess I’d get at the correct conclusion through economic reasoning by pointing out comparative advantages. It’s a cool concept, and demonstrates why AI wouldn’t need to trade with humans because humans won’t have relevant comparative advantages. Or maybe for a while, Sam Altman et al will be in control and won’t need to trade with you because they can produce everything themselves. This is more about gradual disempowerment than taking all your stuff by force but I think it isn’t necessarily the case that you must come to the incorrect conclusion through economic reasoning. It’s more that a few famous economists are bad at economic reasoning in this case
Thank you for the thoughtful discussion of the MIT study; it helped crystalize something that has been bothering me. The study emphasizes that 1. the core value comes from reinventing processes 2. buying and integrating has been much more effective than internal pilots.
The study authors are supposed to understand how big companies work, but they are getting the causality backwards here. “Buy” means “executives have ordered process change and put a budget behind it.” “Build an internal pilot” means “middle managers are demonstrating possibilities before they have executive backing to force process change.”
Mollick made this point a couple months ago:
> A misunderstood factor in AI adoption in companies is what risk means in organizations. I find the question is less "who is accountable if AI gets something wrong?" (often just the user) & more "who is willing to bear the responsibility for adapting our processes & structure"? <
https://x.com/emollick/status/1933942566312923188
But is a technical optimist allowed to have a bad day? A lot of transhumanists are going to die before immortality is achieved. We have had to endure 3 years of relentless AI boosters for primarily Venture Capital interests. Let us have some skepticism that we will be able to retain some of our humanity in a synthetic soaked garden where the President cares more about his image than the world his grandchildren will inherit. U.S. AI capitalism could also be exceedingly toxic to the global economy.
As a fairly bad but credentialled economist: It is totally correct that economists are weirdly resistant to making reasonable AI projections. We have decades of good data, and it's really hard for people to walk away from that and say 'this time is different'.
Paul Krugman once shamelessly fanboyed all over Charles Stross on stage at Worldcon in Montreal. So Krugman can imagine that a hostile ASI might not want to trade with you. Perhaps it has a better use for your atoms? But Krugman seems to strongly believe that AGI is very far off.
I have just given up trying to point out that "AGI would be dangerous because X." I now just go straight for "SkyNet is not interested in your comparative advantage." If I'm going to argue that we risk turning our world into a bad science fiction movie, then I might as well own it.
I think the economics-compatible way to express this is "there's no ex ante reason to believe that the surplus available from any comparative advantage humans have should exceed either (1) the expense of their physical upkeep and survival, (2) the opportunity cost of maintaining an ecology compatible with complex organic life, or (3) the transaction costs of dealing with the humans."
>I also note that later they say custom built AI solutions ‘fail twice as often.’ That implies that when companies are wise enough to test solutions built externally, they succeed over 50% of the time.
Sorry, could you elaborate why is that?
'Custom' tools were failing 95% of the time, and supposedly twice as often as off-the-shelf tools, implying that off the shelf tools failed ~47.5% of the time, and therefore succeeded 52.5% of the time.
But, skimming the report, they seem to be using multiple definitions of 'success' and actually referred to custom tools being deployed 33% of the time vs 67% for off-the-shelf. This is nearly identical to the split of AI tool use in their survey (33%/66%/?%) and is possibly the same numbers...? Which doesn't really make sense, since it's looking at the surviving population split, not the percentage of survival.
The 95% figure is also suspiciously similar to their overall "95% of organizations are getting zero return".
So I wouldn't really trust anything anyway.
<mildSnark>
Suggested OpenAI press release, bungled GPT5 rollout notwithstanding (and remembering that, while most of the GPT5 benchmarks weren't a great advance over o3, the hallucination benchmark _was_ cut by a factor of two.):
"There is power in me yet. My race is not yet run."
</mildSnark>
"The problem with treating labor like a commodity is grain won't try to punch you if the price gets too low."
There's definitely been a nonzero amount of Bell-Mange Amnesia for me wrt econ and some other Official Expert Sciences, once exposed to strange events like AI (or covid). The correct update isn't to throw entire disciplines under the bus - econ is clearly still a very useful lens in Timeline Normal, for however long that lasts - but I definitely put on my skeptic hat now whenever an Expert opines on some rapidly-evolving event which breaks usual assumptions, nevermind actually getting out over their credentialed skis. That quote about journalists never actually quoting people at leading labs or other power users, just academics and fellow travelers, is indicative...why the hell would I trust academia to be any less biased than BigCo? I mean, sure, throw some more chump change at CalCompute or whatever, I'm sure we can spare 1% of the AI capex towards academic research. But an endless series of papers about AI as it existed months/years from the current frontier is just..."not even misaligned"? If more older models got kept around as research curiosities, perhaps that'd be worthwhile as digital anthropology, but I guess we're mostly not gonna do that either.
Do think there's something to the complaint that AI isn't turning out to be quite as idiot-proof user-friendly as one would hope, certainly not commensurate to the hype, so one should extend timelines accordingly. Some of that is poor implementation, some of that is risk-averse diffusion, some of that is not focusing on being a "product company". Like, yeah, you can put organizational context into preferences.md, this should be more heavily advertised and made less Trivially Inconvenient. But anyone who's been exposed to the IT backend of a typical non-tech BigCo can tell you, these are...uh...not very savvy users, utilizing shitty hardware and software. I can totally believe that internal builds flop more often than external packages, because as much as I hate the endless roundtable of feckless countractors, the expertise just doesn't exist in-house anymore. (If it ever did.) All that talent would rather go work in Actual Tech these days. Why build AI at Walmart when Zuckerberg will pay a $billion to build it for him and then sell it back to Walmart? Heck, you wouldn't believe how much my store spends on plumbers to "fix" perennial piping problems...lots of racket rents for everyone in just mundane contracting!
"Notably, it's ~never employees at frontier companies quoted on this"
How about that time Sam Altman said AI was a bubble less than 2 weeks ago?
https://www.theverge.com/ai-artificial-intelligence/759965/sam-altman-openai-ai-bubble-interview
This Substack seems like filtered evidence
Investment is cashflow negative to start with. More at 11.
Using AI effectively is like the production line revolution, the transition from powering factories with one steam engine per factory, to powering them with one electric motor per machine. It's going to take a while to figure out how to do it. As in decades.
Edit: MIT et al. seem to think AI is like the transition from messenger boys to telephones. No, it's bigger than that.
Also, by "investment is cashflow negative", I mean that it's a trial and error process, and there are many more ways to get it wrong than there are to get it right.
I agree that most economists are too dismissive of AI and share your exasperation of Cowen and Acemoglu, but, as an economist, I have to say the Byrnes thread is pretty bad actually. Point 4 is fine, but the rest show a basic lack of understanding of what economists mean by basic concepts like “equilibrium”.
Maybe if the point is “taking Econ 101 and then never going deeper and believing you understand everything is bad”, then I agree, but that is not saying much.
Byrnes’ arguments are all sound and cool, but have we already “proven” the viability of AGI/ASI prior to calling everyone dumb for not getting how AGI/ASI will transform everything? I do believe in it personally, but since when we have considered it as a given thing, which can be seen throughout this post?
I agree with the critique about the MIT paper, though I think it's useful to distinguish between the paper and the reactions. The thing that journalists (and bloggers), sometimes forget about research is, sometimes it's boring. Actually, a lot of it is boring. Boring journalism is bad, boring research is still good. I think we need to leave room for researchers to continue creating boring research without jumping all over them for such boring results. Sometimes you just have to confirm the obvious in a rigorous way. It should be obvious that if you experiment with X, some of those things won't make any change because (a) you weren't ready to follow through, (b) you made some mistakes that you'll have to iterate over to get to something useful, (c) you over-extended and found your experiment was more than was possible, (d) etc. Boring, but having a bit of data on that isn't a bad thing.
Moving to the second point regarding Steven Byrnes, I'm less supportive of the framing, as it falls into the trap of mixing short-term and long-term outlooks. There's really no sense in critiquing economists who are making predictions for the impacts of LLMs, traditional ML, etc. by saying, "you forgot about AGI". AGI is a conversation stopper in that regard. It requires a whole different framing. I discuss this a bit in my own post (https://substack.norabble.com/p/the-economic-future-from-and-of-ai), and then follow-up with the non-AGI world in part 2 (https://substack.norabble.com/p/the-economic-future-from-and-of-ai-cf1).
Anyone who tells you they know when AGI is arriving, either has some very private information (which I won't believe until they reveal it in full), or is one of the very typical over-confident prognosticators. I'm a bit interested in hearing their predictions; you can learn from over-confident people without falling into their trap. But I'm not all that interested in them trying to shutdown other useful conversations by proclaiming "what about AGI?". Those proclamations are more or less just their attempt to declare their own brilliance and other's ignorance. Without any evidence of that rather extreme proclamation, it tends to just suggest the boring answer, they have a personality that leans toward overconfidence in itself.