I think you're being nearsighted in the case of Juliana's suicide. The claim here is not merely that the AI did not handle this case correctly, but also that the AI was optimized to replace other relationships, some of whom surely would have handled the case correctly.
I agree I would not sue a "friend" who behaved this way, but what if I found out that friend was being coached by a team of experts to destroy my other friendships? More importantly, I think this analogy is simply incorrect. Viewing the product as a "friend" is succumbing to the framing of the product's creators, but there's no reason we should do that, or the law should do that.
*Is* AI optimized to replace meatspace relationships? I think this claim of rivalrous goods tends to veer too close to Fully General Counterargument territory, since time is fungible and increasingly many goods and services are optimized for such "engagement". Unless we want to claim that everything from Doordash to videogames aims to be a replacement rather than a complement to analog human analogues. Which, to be fair, is a valid argument to make - one just has to actually bite the bullet. Otherwise it feels like special pleading. Doesn't seem coincidental that the major non-coding use case for AI continues to be varieties of chat, rather than all the other stuff it can do...clearly people have a demand for something friend-shaped, whether one wants to classify it as an actual one or not.
On the merits, I do think it's important to consider the counterfactual of public awareness that LLMs might narc to your parents or the cops or whatever. Would Juliana have used them at all, in that case? Would she still have gone an hero without talking to Hero? I also think it's assuming a lot that getting that first therapy visit would have set things on the right path...I myself had years of run-ins with the therapeutic-industrial complex after my own adventures with involuntary commitment, and am highly confident 80%+ of that bullshit wouldn't have helped during crunch time. (Nevermind the number of times I dissembled to therapists after learning their particular jailbreaks, er, red flags!) What I really needed was someone to talk to, genuinely privately, about deeply private troublesome matters...but no IRL friends were close enough, online ones lived half a world away, and parents kept trying to cut me off from the latter anyway. I guess you could say I needed a Hero.
> [Silicon Valley thinks] that if you have equity that violates their norms, or that you ‘don’t deserve’ or that doesn’t align with your power or role, or whose presence hurts the company or no longer ‘makes sense,’ that it is good and right to restructure to take that equity away.
To be slightly fair to Silicon Valley, most of the ways in which your equity stake gets diluted are when the company is not yet a going concern, and it is begging for more capital. Let's say you have three founders, two of whom are still essential to the business, and one of whom fucked off years ago. If the company were actually profitable, then yes, the missing founder should still own a third. But the company is bleeding money, and busily begging investors for $100 million (please, come on, we promise this is the last time).
The new investors demand a big chunk of the company, diluting the existing shareholders. The two founders who are still participating in the company are granted extra stock because they're still a major asset to the business. The missing founder's stake is diluted by the issuance of new shares.
The moral of the story is that huge sums of money come at a price. And worse, if you fail to hit your goals, and if you suffer a major setback, and then you beg for a giant pile of money, then it's going to come with serious complications. "Down rounds" are bad news for existing equity holders because you're asking people to pour money into a struggling business.
If you don't understand at least this much of the game, you shouldn't be at the table. As a rule of thumb, assume your employee equity is worthless and insist on cash payment (or work for a FAANG). You'll be happier.
"This seems very different from the original case of Adam Raine that caused Character.ai to make changes." I thought Adam Raine was using ChatGPT, and Sewell Setzer III was using Character.ai ?
Potentially I missed that Character.ai implemented additional changes after seeing the Raine/ChatGPT case.
“including the bootstrap problem of chatting enough to be confident they’re over 18 before you’re confident they’re over 18”
I sometimes wonder if R1 already does this, without anyone having told it to. It is more willing to tell you stuff when you are a few messages into a conversation, by which time it probably has a very good idea of what type of person you are, and can e.g. detect that you’re not a child with high probability.
For example, when Mozart was my age, he had the chutzpah to still be alive for 6 more months...
Can't wait to see Pliny jailbreak age-verifiers to enable Adult (Elder?) Mode for kids. I suppose there's also nothing stopping a kid from running potential prompts through a different LLM, asking for the maturity level to be punched up, and then ctrl-c those over to any age-locked ones. Though in that case it's Levels of Friction, so any kid who's willing to bother with all that probably deserves at least partial credit. Possibly it's not even wrong to have some arbitrary restrictions at [13...18] though, having unfettered access to the internet growing up today is...very much not like it was 30, 20, even 10 years ago. And devolving that responsibility entirely to parents doesn't feel appropriate either. Kids grow up fast online, slow irl; this mismatch continues to bring nothing but tragedy to everyone involved.
>both been rising over of the past ~2.5 years
Lol, lmao even. Anything looks like a trend if you prune the axes enough! Clearly stimulus is superfluous, don't you know the S&P goes *up* when people are unemployed...
Nozick's point wasn't to not build the Delight Nexus, but he claims that people wouldn't want to use it, or at least not choose to get hooked up permanently. I am sure that as a good libertarian he would very much prefer to see it built, so that people could have the option to use it, if they so choose.
TL;DR: Mitchell was right the way Feyerabend's Galileo was right. He was a crank with ulterior motives, bad epistemics, fake evidence, unrealistic timelines, and most of all an incoherent gears-level model of his claims. In the end he turned to playing politics to shift the discussion. He looks visionary in hindsight, but it's near-impossible to tell whether that was good judgment on his part or just good luck.
I leave the implications for AI discourse as an exercise for the reader.
I want to plug my piece in Inside Higher Ed on students using AI for their "controversial views" and "viewpoint diversity" experience rather than going to hear controversial speakers. This is AI news, not just higher ed news. https://www.insidehighered.com/opinion/views/2025/09/16/about-fires-free-speech-rankings-opinion
Podcast episode for this post, another one that breaks 2 hours!
https://open.substack.com/pub/dwatvpodcast/p/ai-134-if-anyone-reads-it
I think you're being nearsighted in the case of Juliana's suicide. The claim here is not merely that the AI did not handle this case correctly, but also that the AI was optimized to replace other relationships, some of whom surely would have handled the case correctly.
I agree I would not sue a "friend" who behaved this way, but what if I found out that friend was being coached by a team of experts to destroy my other friendships? More importantly, I think this analogy is simply incorrect. Viewing the product as a "friend" is succumbing to the framing of the product's creators, but there's no reason we should do that, or the law should do that.
*Is* AI optimized to replace meatspace relationships? I think this claim of rivalrous goods tends to veer too close to Fully General Counterargument territory, since time is fungible and increasingly many goods and services are optimized for such "engagement". Unless we want to claim that everything from Doordash to videogames aims to be a replacement rather than a complement to analog human analogues. Which, to be fair, is a valid argument to make - one just has to actually bite the bullet. Otherwise it feels like special pleading. Doesn't seem coincidental that the major non-coding use case for AI continues to be varieties of chat, rather than all the other stuff it can do...clearly people have a demand for something friend-shaped, whether one wants to classify it as an actual one or not.
On the merits, I do think it's important to consider the counterfactual of public awareness that LLMs might narc to your parents or the cops or whatever. Would Juliana have used them at all, in that case? Would she still have gone an hero without talking to Hero? I also think it's assuming a lot that getting that first therapy visit would have set things on the right path...I myself had years of run-ins with the therapeutic-industrial complex after my own adventures with involuntary commitment, and am highly confident 80%+ of that bullshit wouldn't have helped during crunch time. (Nevermind the number of times I dissembled to therapists after learning their particular jailbreaks, er, red flags!) What I really needed was someone to talk to, genuinely privately, about deeply private troublesome matters...but no IRL friends were close enough, online ones lived half a world away, and parents kept trying to cut me off from the latter anyway. I guess you could say I needed a Hero.
> [Silicon Valley thinks] that if you have equity that violates their norms, or that you ‘don’t deserve’ or that doesn’t align with your power or role, or whose presence hurts the company or no longer ‘makes sense,’ that it is good and right to restructure to take that equity away.
To be slightly fair to Silicon Valley, most of the ways in which your equity stake gets diluted are when the company is not yet a going concern, and it is begging for more capital. Let's say you have three founders, two of whom are still essential to the business, and one of whom fucked off years ago. If the company were actually profitable, then yes, the missing founder should still own a third. But the company is bleeding money, and busily begging investors for $100 million (please, come on, we promise this is the last time).
The new investors demand a big chunk of the company, diluting the existing shareholders. The two founders who are still participating in the company are granted extra stock because they're still a major asset to the business. The missing founder's stake is diluted by the issuance of new shares.
The moral of the story is that huge sums of money come at a price. And worse, if you fail to hit your goals, and if you suffer a major setback, and then you beg for a giant pile of money, then it's going to come with serious complications. "Down rounds" are bad news for existing equity holders because you're asking people to pour money into a struggling business.
If you don't understand at least this much of the game, you shouldn't be at the table. As a rule of thumb, assume your employee equity is worthless and insist on cash payment (or work for a FAANG). You'll be happier.
Typo
Anthropic is not and should to be an enemy of the administration,
Should be?
Anthropic is not and should not be
Otherwise, we be so worried
Otherwise, why
This tread contains more endorsements of If Anyone Builds It, Everyone Dies, including
This thread
Such regimes would design systems to ensure no one could get out line.
Get out of line
Consider combatting your lysdexia by running your post through a clanker before posting. Then you would have less of "Tracing Woods" an "Mira Mutari"
Don't forget "Nate Sores"! It's literally on the cover of the book pictured repeatedly here...
Maybe it's supposed to be Tsores, Yiddish for trouble and grief.
Hey Zvi! Thanks for the shoutout for BlackjackBench!
Roon: "I think we lost control sometime in the late 18th century."
Man's fate was woven, not on the Norns' loom, but Jacquard's.
"This seems very different from the original case of Adam Raine that caused Character.ai to make changes." I thought Adam Raine was using ChatGPT, and Sewell Setzer III was using Character.ai ?
Potentially I missed that Character.ai implemented additional changes after seeing the Raine/ChatGPT case.
“including the bootstrap problem of chatting enough to be confident they’re over 18 before you’re confident they’re over 18”
I sometimes wonder if R1 already does this, without anyone having told it to. It is more willing to tell you stuff when you are a few messages into a conversation, by which time it probably has a very good idea of what type of person you are, and can e.g. detect that you’re not a child with high probability.
For example, when Mozart was my age, he had the chutzpah to still be alive for 6 more months...
Can't wait to see Pliny jailbreak age-verifiers to enable Adult (Elder?) Mode for kids. I suppose there's also nothing stopping a kid from running potential prompts through a different LLM, asking for the maturity level to be punched up, and then ctrl-c those over to any age-locked ones. Though in that case it's Levels of Friction, so any kid who's willing to bother with all that probably deserves at least partial credit. Possibly it's not even wrong to have some arbitrary restrictions at [13...18] though, having unfettered access to the internet growing up today is...very much not like it was 30, 20, even 10 years ago. And devolving that responsibility entirely to parents doesn't feel appropriate either. Kids grow up fast online, slow irl; this mismatch continues to bring nothing but tragedy to everyone involved.
>both been rising over of the past ~2.5 years
Lol, lmao even. Anything looks like a trend if you prune the axes enough! Clearly stimulus is superfluous, don't you know the S&P goes *up* when people are unemployed...
The link on “ better known as Anarchy, State and Utopia by Robert Nozick” is broken.
Nozick's point wasn't to not build the Delight Nexus, but he claims that people wouldn't want to use it, or at least not choose to get hooked up permanently. I am sure that as a good libertarian he would very much prefer to see it built, so that people could have the option to use it, if they so choose.
Obligatory Naval Gazing link since you brought up the airplanes vs battleships debate: https://www.navalgazing.net/Fighting-the-Last-War#fnr1_2
TL;DR: Mitchell was right the way Feyerabend's Galileo was right. He was a crank with ulterior motives, bad epistemics, fake evidence, unrealistic timelines, and most of all an incoherent gears-level model of his claims. In the end he turned to playing politics to shift the discussion. He looks visionary in hindsight, but it's near-impossible to tell whether that was good judgment on his part or just good luck.
I leave the implications for AI discourse as an exercise for the reader.