If I may steelman the tech stack & flywheel argument:
1. Model architecture and chip design co-evolved. New chips were developed for DL, for CNNs, and for transformers. And both DL and transformers had problems breaking through because they were not as efficient to run as they could be. New architectures such as Mamba are constrained by needing to work well with chips optimized for transformers.
2. If American organizations or companies published open SOTA models regularly there wouldn't be economic incentive for competitors to train their own foundation models. This happened with Whisper for example, for like 2 years all TTS was fine-tunes of Whisper.
2a. If new weights were published once or twice a year, nobody else would pay the cost to build training clusters. And everybody would use American-trained foundation models.
3. If in addition the new models used different architectures that have new requirements of the accelerator chips, the competitor hardware companies would have problem keeping track and could never efficiently run the newest class of models.
There are obvious problems in this argument (architectures haven't changed significantly for years, only training methods; it's quite risky to publish open-weight models; there's a lot more money in closed SOTA models; this whole thing assumes the "normal technology" proposition). But it exists.
"No, seriously, check the tape if you’re not sure. He keeps saying ‘we need more base load power’ and this isn’t base load power, so we should destroy it. And call that ‘national security.’
This is madness. This is straight up sabotage of America. Will no one stop this?
Meanwhile, it seems it’s happening, the H20 is banned in China, all related work by Nvidia has been suspended, and for now procurement of any other downgraded chips (e.g. the B20A) has been banned as well."
So, it sounds like the way to visualize this is as a Great Power duel, with the USA and PRC drawing out their pistols, taking careful aim, and firing - with the careful aim being each at their own feet...
I have no idea what's going on with the PRC suddenly refusing the chips they've been clamoring for. My best guess (totally uninformed), is that a domestic chip maker like Huawei lobbied the government to block the foreign competition and give them a monopoly on the local market, same way American companies are lobbying Trump.
Arguably, in fact, the recent wind power refusal thing you mentioned, could be an example of exactly that. What exactly is the 'national security issue' with an already completed wind farm opening for business? Well, it would offer competition to coal and natural gas. Coal mines and natural gas fields would make less money. Can't be having that. Same way we can't be having dirty foreign ships touching our precious Jones Act protected ports, and taking business from our precious Jones Act protected fleet.
(Though, I suspect a large part of what's going on is some sort of "Moral Mazes" dynamic, where subordinates compete to be as stupid and aggressive as possible to show how loyal they are to the boss, how enthusiastic they are to get with the program. Same thing happened with China and its "Wolf Warrior" diplomacy, same thing happens every day on Twitter/Tumblr/Bluesky, and the same thing happens all throughout history when young men hang out together and compete to show how stupid & reckless/"macho" they can be. Nothing shows loyalty, quite like aggressively taking the fight to the enemy...)
I am glad Some Things You Miss was not intended to not be confusing, because it was. Also glad it's simple to coexist with ASI, we just need to do the equivalent of building roofs to keep out rain! Perhaps it will involve onion futures? (https://astralcodexten.substack.com/p/the-onion-knight)
Even if one grants all the Sacks premises, of no AGI, meaningful moat, highly codependent hardware and software design, etc...the whole argument feels like a rehash of the PNTR debates of the 90s. Surely if we can get China addicted to sweet, sweet American <s>dollars</s> tech stack, they'll liberalize, and it will increase our geopolitical leverage, and we'll hold onto our world-class manufacturing advantage anyway so it's fine? And like, my family fled China to get away from grinding poverty and repression, so obviously I'm more than sympathetic to a billion of my former countrymen being lifted out of poverty! But we Ran The Experiment over the last 30ish years, and mostly none of those aspirational hopes came to pass. What will make things different this time? Nevermind that <s>reverse engineering and theft of intellectual property</s> fast-following for just a couple industries is not exactly a heavy lift, given already-evidenced expertise across so many other domains to date. Right now there's no real reason to, since the export controls were always porous...but if a genuine moat came about? I'd hardly expect that to last, versus an actual dedicated Adversary who's demonstrably not cool with permanently playing second fiddle to our hegemon. Certainly not under this administration, which has not exactly inspired me with its robustness to bribery "jailbreaks"!
Do we have *any* idea what’s going on with the “smarter” models companies claim to have developed but not released because they require far too much compute, beyond the occasional unverifiable claim about performance on one benchmark or another? I fear the safety research community is going to end up effectively stumbling around blindfolded in coming years as advances outstrip sufficient compute, resulting in companies keeping their most advanced models inaccessible to the public.
I cancelled my stratechery subscription after I read that article. The degree to which Ben will torture arguments to defend Trump is too much. And Ben’s takes on AI are tired and uninformed.
Sometimes you need to actually work in an industry to form useful opinions and Ben hasn’t worked in any industry in 20 years.
More seriously, one of my friends is a mental health councilor for a middle school and I actually don't know if AI could do the most important part of his job yet, namely calling the police on abusive parents. Obviously claude could do that and tries under the right circumstances, I'm just not clear legally what happens when claude calls 911
Re: Intel, I have been wondering for a long time now: why aren't they making AI chips for NVIDIA...? A B100 (or whatever) Intel-fabbed variant would have worse power usage but fully work with CUDA and the rest of the software stack used for frontier training, would have no problem selling, would likely have any export controls waived... Sure, there would be 12-18 months of engineering time to migrate to their process node, but they could have started that last year and have chips for sale next year.
It seems like free money, just sitting there on the table, coming only at the cost of their pride.
There's rumours of moving the gaming chips to Intel fabs, which would free up TSMC capacity, but that still seems like small-time thinking when their company is in a death spiral.
About the administration's "deal"... I was far less surprised to hear about than (apparently) many others. It seems like an on-brand effort to "save" the company while also controlling a strategic asset. Extortion isn't the point, just the means.
Weird to also free them from CHIPS obligations, though I haven't looked at the details. Assuming a goal of 'AI dominance', or even the distorted market share version, they should want the fabs built as that's one of the harder parts to accelerate.
I think they are best modelled as consistently short-sighted and deal-driven rather than random/malicious/???. The Intel stake should be a big update if it surprised you and a small update otherwise.
I suspect the hassle is just too big - witness Samsung's similar issues (for a while, their yields were abysmal so fabbing big chips was probably out of question there, not sure what the current state is).
Plus I think even Intel now fabs some of their chips at TSMC, which says a lot.
prediction: it'll come out by EOY or early next that all the major labs are going hard on latent space reasoning, and then people's timelines will compress again.
Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/ai-131-part-2-various-misaligned
At least on Pocket Cast, the audio version of this article cuts off after the first couple sentences.
If I may steelman the tech stack & flywheel argument:
1. Model architecture and chip design co-evolved. New chips were developed for DL, for CNNs, and for transformers. And both DL and transformers had problems breaking through because they were not as efficient to run as they could be. New architectures such as Mamba are constrained by needing to work well with chips optimized for transformers.
2. If American organizations or companies published open SOTA models regularly there wouldn't be economic incentive for competitors to train their own foundation models. This happened with Whisper for example, for like 2 years all TTS was fine-tunes of Whisper.
2a. If new weights were published once or twice a year, nobody else would pay the cost to build training clusters. And everybody would use American-trained foundation models.
3. If in addition the new models used different architectures that have new requirements of the accelerator chips, the competitor hardware companies would have problem keeping track and could never efficiently run the newest class of models.
There are obvious problems in this argument (architectures haven't changed significantly for years, only training methods; it's quite risky to publish open-weight models; there's a lot more money in closed SOTA models; this whole thing assumes the "normal technology" proposition). But it exists.
"No, seriously, check the tape if you’re not sure. He keeps saying ‘we need more base load power’ and this isn’t base load power, so we should destroy it. And call that ‘national security.’
This is madness. This is straight up sabotage of America. Will no one stop this?
Meanwhile, it seems it’s happening, the H20 is banned in China, all related work by Nvidia has been suspended, and for now procurement of any other downgraded chips (e.g. the B20A) has been banned as well."
So, it sounds like the way to visualize this is as a Great Power duel, with the USA and PRC drawing out their pistols, taking careful aim, and firing - with the careful aim being each at their own feet...
I have no idea what's going on with the PRC suddenly refusing the chips they've been clamoring for. My best guess (totally uninformed), is that a domestic chip maker like Huawei lobbied the government to block the foreign competition and give them a monopoly on the local market, same way American companies are lobbying Trump.
Arguably, in fact, the recent wind power refusal thing you mentioned, could be an example of exactly that. What exactly is the 'national security issue' with an already completed wind farm opening for business? Well, it would offer competition to coal and natural gas. Coal mines and natural gas fields would make less money. Can't be having that. Same way we can't be having dirty foreign ships touching our precious Jones Act protected ports, and taking business from our precious Jones Act protected fleet.
(Though, I suspect a large part of what's going on is some sort of "Moral Mazes" dynamic, where subordinates compete to be as stupid and aggressive as possible to show how loyal they are to the boss, how enthusiastic they are to get with the program. Same thing happened with China and its "Wolf Warrior" diplomacy, same thing happens every day on Twitter/Tumblr/Bluesky, and the same thing happens all throughout history when young men hang out together and compete to show how stupid & reckless/"macho" they can be. Nothing shows loyalty, quite like aggressively taking the fight to the enemy...)
> a domestic chip maker like Huawei lobbied the government to block the foreign competition and give them a monopoly on the local market
This is quite different from my model of how governance works in China.
I am glad Some Things You Miss was not intended to not be confusing, because it was. Also glad it's simple to coexist with ASI, we just need to do the equivalent of building roofs to keep out rain! Perhaps it will involve onion futures? (https://astralcodexten.substack.com/p/the-onion-knight)
Even if one grants all the Sacks premises, of no AGI, meaningful moat, highly codependent hardware and software design, etc...the whole argument feels like a rehash of the PNTR debates of the 90s. Surely if we can get China addicted to sweet, sweet American <s>dollars</s> tech stack, they'll liberalize, and it will increase our geopolitical leverage, and we'll hold onto our world-class manufacturing advantage anyway so it's fine? And like, my family fled China to get away from grinding poverty and repression, so obviously I'm more than sympathetic to a billion of my former countrymen being lifted out of poverty! But we Ran The Experiment over the last 30ish years, and mostly none of those aspirational hopes came to pass. What will make things different this time? Nevermind that <s>reverse engineering and theft of intellectual property</s> fast-following for just a couple industries is not exactly a heavy lift, given already-evidenced expertise across so many other domains to date. Right now there's no real reason to, since the export controls were always porous...but if a genuine moat came about? I'd hardly expect that to last, versus an actual dedicated Adversary who's demonstrably not cool with permanently playing second fiddle to our hegemon. Certainly not under this administration, which has not exactly inspired me with its robustness to bribery "jailbreaks"!
Do we have *any* idea what’s going on with the “smarter” models companies claim to have developed but not released because they require far too much compute, beyond the occasional unverifiable claim about performance on one benchmark or another? I fear the safety research community is going to end up effectively stumbling around blindfolded in coming years as advances outstrip sufficient compute, resulting in companies keeping their most advanced models inaccessible to the public.
I cancelled my stratechery subscription after I read that article. The degree to which Ben will torture arguments to defend Trump is too much. And Ben’s takes on AI are tired and uninformed.
Sometimes you need to actually work in an industry to form useful opinions and Ben hasn’t worked in any industry in 20 years.
Re: mental health care
Listen, this isn't the fault of the AI, we need to start jailing anxious people when they ask AI for help.
More seriously, one of my friends is a mental health councilor for a middle school and I actually don't know if AI could do the most important part of his job yet, namely calling the police on abusive parents. Obviously claude could do that and tries under the right circumstances, I'm just not clear legally what happens when claude calls 911
Re: Intel, I have been wondering for a long time now: why aren't they making AI chips for NVIDIA...? A B100 (or whatever) Intel-fabbed variant would have worse power usage but fully work with CUDA and the rest of the software stack used for frontier training, would have no problem selling, would likely have any export controls waived... Sure, there would be 12-18 months of engineering time to migrate to their process node, but they could have started that last year and have chips for sale next year.
It seems like free money, just sitting there on the table, coming only at the cost of their pride.
There's rumours of moving the gaming chips to Intel fabs, which would free up TSMC capacity, but that still seems like small-time thinking when their company is in a death spiral.
About the administration's "deal"... I was far less surprised to hear about than (apparently) many others. It seems like an on-brand effort to "save" the company while also controlling a strategic asset. Extortion isn't the point, just the means.
Weird to also free them from CHIPS obligations, though I haven't looked at the details. Assuming a goal of 'AI dominance', or even the distorted market share version, they should want the fabs built as that's one of the harder parts to accelerate.
I think they are best modelled as consistently short-sighted and deal-driven rather than random/malicious/???. The Intel stake should be a big update if it surprised you and a small update otherwise.
I suspect the hassle is just too big - witness Samsung's similar issues (for a while, their yields were abysmal so fabbing big chips was probably out of question there, not sure what the current state is).
Plus I think even Intel now fabs some of their chips at TSMC, which says a lot.
prediction: it'll come out by EOY or early next that all the major labs are going hard on latent space reasoning, and then people's timelines will compress again.