So you’re saying GPT knows it’s Dobby? Not exactly a goblin but given the way that universe handles race, racism, and slavery (badly) it’s close enough to accurate huh
Google's decision makes me a lot more frigid on the company's prospects given how publicly unpopular this is among senior DeepMind employees. I expect people to leave.
The Trump administration definitely has some very smart people working in it, and occasionally you read something they are putting out, the overall feel is... very "Trumpy", for lack of a better word, and then all of a sudden there is a very calm and insightful paragraph.
If we tell a billion people in China that they are not allowed to use frontier AI, then how will that play out over the next ten or twenty years?
I don't see a billion people going quietly if they are told they don't get frontier AI, their children don't get frontier AI, and in order to stop them from frontier AI, their computer chips will be handicapped.
I understand being cautious when it comes to international security, but historically, the road to peace has been to share, collaborate, travel, and trade.
Senator Can't Well doesn't want AI to improve efficiency in medical spending? Nominative determinism remains an incurable disease. For the rebuttal position, we now turn to AI researcher Max Harms...
Gemini seems to be in a rut, not able to easily follow either OAI's "number go up, it's a good multimodal model, sir" or Anthropic's "very align, such welfare, wow coding" development paths. So the government getting all lawful usage of the 3rd? 4th? best AI is not the worst thing (still an improvement over Grok, obviously). Makes me worry about the parent company though. I mean, look at Facebook continuing to flounder and set money on fire, and Google has a lot more ruin in it. At least they still have Waymo?
On AI and work, in nothing-ever-happens worlds where AI doesn't take over: yes yes, elasticity of demand, new jobs, etc.
But also: AI will allow governments and industries to complexify regulations and protocols and contracts and policies to such a degree that everyone will be forced to employ AI agents to navigate the system. Humans will be completely unable to compete unaided.
"It’s also true that there is a baffling amount of ‘sure this kills everyone 10%+ or even 25%+ of the time, but mainly I worry about the jobs.’"
This can be a coherent position. Your worries about a danger should not be a strict function of its magnitude, it should also incorporate your personal ability to avert that danger.
Suppose you have a really good telescope and know there's a 20% chance that an asteroid will obliterate your planet in 1 month but you have no ability to divert the asteroid because your planet still has very early 20th century technology. It might technically be possible if everyone worked together in a perfectly coordinated way, but half the population doesn't even believe in asteroids, they have seen your pictures proving there's a moving shiny potato in the sky, but they either don't believe you could possibly know what it is, or they think you faked it with a literal potato. Do you spend all your energy trying to figure out how to do rocketry or do you try to compartmentalize your fear, continue lobbying for better telescopes, and spend some quality time with your family?
Suppose live in a capitalist society where financial incentives are effectively unbounded, under a government where regulation can be avoided almost entirely by just throwing around money. And then you notice that there are three or four gargantuan companies and a bunch of merely massive ones trying to build a machine god and successfully avoiding regulation by throwing around money. In principal this could be stopped until we figure out how to do it safely. We merely need to convince most of the population both that they may succeed and that it would be hard for any humans to survive much longer if they do, and then get them all on the same page so they push their miniscule democracy levers and post content into the communication networks run by those very companies in just the right way and at the right time to get the government to change its position. But half the population thinks AI is a parlor trick, or that it has already reached its peak capability because it ran out of data, or that because it's non corporeal it can't have anything like true intelligence or will, or that if it gets intelligent enough it will automatically be benevolent, or that we can just pull the plug right? Or they just aren't paying any attention, or they have too much trust in the government, or too much trust in the giant corporations. Do you spend all your energy worrying about alignment or do you continue lobbying for policies that address things most of them are starting to believe in and spend some quality time with your family?
I'm not arguing that this is for sure a correct position, but it is coherent, maybe even reasonable.
The data from PauseAI and ControlAI is that most people haven't heard about the machine god or even the automatize most jobs thing. Once they hear about it they think it's insane and that it should be stopped.
>half the population thinks AI is a parlor trick, or that it has already reached its peak capability because it ran out of data ....
From speaking to around 300 random people it's mostly not paying attention but when they hear about what's going on they think: "WTF, this is insane".
The bubble we are in overselects hard for people who are difficult to convince/have given up.
Yeah they might react that way when you tell them, then what? What about when they talk to the next guy who is some Ed Zitron type? There was a time when single payer was obscenely popular in America, 60% or more, and that's an actual fairly concrete policy plan, nothing happened on that and it's hard to see how it even could have if, say, Bernie Sanders had been elected. Again, I'm not saying it's hopeless, I still bring up these concerns to people, but the mundane and obvious is easier to convey, and even then it's hard to get people concerned let alone agreeing on what to do about it.
Yea, I think it's good to discuss these concerns because I don't think things are as dark (except for Gradual Disempowerment).
* Ed Zitron types will become less relevant as AI keeps going up and getting more powerful
* Single payer is way more complicated than AI I think.
> I still bring up these concerns to people, but the mundane and obvious is easier to convey, and even then it's hard to get people concerned let alone agreeing on what to do about it.
As far as talking to people: Demonstrating stuff/showing a claude code demo, some statements by AI CEOs, Scientists can help. Grown not built (no one understands the systems).
Getting people to become active seems like the main difficulty to me and we are trying hard to optimize it. One: The success rate is low so it's hard to stay motivated. However, one person who themselves becomes actually active can then do a lot. The second and third order effects of getting a person active are huge.
PauseAI/TBC/ControlAI/Humanity in Control/...? Are all trying to optimize this. It depends. There is micro-commit, becoming a member somewhere, sharing a post, thinking about it, getting one step closer to doing something.
The hardest part is getting people to understand, and not doubt, RSI driven exponential growth. It’s made tougher because the intelligence explosion could feasibly not happen and people (very reasonably!) hate the SV people and disbelieve what they say.
Mode collapse often lands on nerdy/fantasy tropes. LLMs love names like "Elara" and "Aeris" and "Varis", which sound like fantasy characters. In 2022, ChatGPT was obsessed with saying "delve".
To make up a just-so story, goblins are playful and nonthreatening, don't have intense/sexual/culturally-freighted connotations like demons/witches, and are well-entrenched in pop cultural usage (use of "gremlin" for a minor but perplexing technical issue goes back at least to WW2). So they're probably a fairly big target in the RL landscape.
I'm surprised LLMs haven't fixated on monkeys, pirates and ninjas.
> If ‘I used a real LLM instead’ is a valid response, then This Is Fine, and if you can just type ‘good morning, copilot’ then that works, too. Maybe you do both?
People, Simon Willison's prominent blog called GPT-5 Thinking his "Research Goblin" in Sept. 2025. https://simonwillison.net/2025/Sep/6/research-goblin/ Can it really be a coincidence that the next model release is the one that started obsessing over goblins?
I note with embarrassed chagrin that I didn't even realize EY had two different twitter accounts. Thought he just switched handles. The "Relevant People" sidebar feature is honestly pretty neat, and it'd be cool to have a similar widget on other fora-type wobsites. (Baffled at Substack still leaving huge whitespace to either side of the content, it's so many wasted pixels?) Especially for topics like AI, which have a spiraling cast of characters that it's easy to lose track of. This is also why I'm pleased at the ongoing blog practice of putting (organization and/or context of post) after peoples' names for block quotes and tweets, it's quite helpful.
yup
So you’re saying GPT knows it’s Dobby? Not exactly a goblin but given the way that universe handles race, racism, and slavery (badly) it’s close enough to accurate huh
It has more like Civil Engineer Dobby than Jeeves Dobby, but very much so.
Podcast episode for this post:
https://dwatvpodcast.substack.com/p/ai-166-google-sells-out
Google's decision makes me a lot more frigid on the company's prospects given how publicly unpopular this is among senior DeepMind employees. I expect people to leave.
The Trump administration definitely has some very smart people working in it, and occasionally you read something they are putting out, the overall feel is... very "Trumpy", for lack of a better word, and then all of a sudden there is a very calm and insightful paragraph.
If we tell a billion people in China that they are not allowed to use frontier AI, then how will that play out over the next ten or twenty years?
I don't see a billion people going quietly if they are told they don't get frontier AI, their children don't get frontier AI, and in order to stop them from frontier AI, their computer chips will be handicapped.
I understand being cautious when it comes to international security, but historically, the road to peace has been to share, collaborate, travel, and trade.
Better not to build it in the first place.
Senator Can't Well doesn't want AI to improve efficiency in medical spending? Nominative determinism remains an incurable disease. For the rebuttal position, we now turn to AI researcher Max Harms...
Gemini seems to be in a rut, not able to easily follow either OAI's "number go up, it's a good multimodal model, sir" or Anthropic's "very align, such welfare, wow coding" development paths. So the government getting all lawful usage of the 3rd? 4th? best AI is not the worst thing (still an improvement over Grok, obviously). Makes me worry about the parent company though. I mean, look at Facebook continuing to flounder and set money on fire, and Google has a lot more ruin in it. At least they still have Waymo?
On AI and work, in nothing-ever-happens worlds where AI doesn't take over: yes yes, elasticity of demand, new jobs, etc.
But also: AI will allow governments and industries to complexify regulations and protocols and contracts and policies to such a degree that everyone will be forced to employ AI agents to navigate the system. Humans will be completely unable to compete unaided.
AI will mainly create jobs for AI.
I felt some trepidation before I clicked on the link for "Eliezer successfully pegged" but I did it
"It’s also true that there is a baffling amount of ‘sure this kills everyone 10%+ or even 25%+ of the time, but mainly I worry about the jobs.’"
This can be a coherent position. Your worries about a danger should not be a strict function of its magnitude, it should also incorporate your personal ability to avert that danger.
Suppose you have a really good telescope and know there's a 20% chance that an asteroid will obliterate your planet in 1 month but you have no ability to divert the asteroid because your planet still has very early 20th century technology. It might technically be possible if everyone worked together in a perfectly coordinated way, but half the population doesn't even believe in asteroids, they have seen your pictures proving there's a moving shiny potato in the sky, but they either don't believe you could possibly know what it is, or they think you faked it with a literal potato. Do you spend all your energy trying to figure out how to do rocketry or do you try to compartmentalize your fear, continue lobbying for better telescopes, and spend some quality time with your family?
Suppose live in a capitalist society where financial incentives are effectively unbounded, under a government where regulation can be avoided almost entirely by just throwing around money. And then you notice that there are three or four gargantuan companies and a bunch of merely massive ones trying to build a machine god and successfully avoiding regulation by throwing around money. In principal this could be stopped until we figure out how to do it safely. We merely need to convince most of the population both that they may succeed and that it would be hard for any humans to survive much longer if they do, and then get them all on the same page so they push their miniscule democracy levers and post content into the communication networks run by those very companies in just the right way and at the right time to get the government to change its position. But half the population thinks AI is a parlor trick, or that it has already reached its peak capability because it ran out of data, or that because it's non corporeal it can't have anything like true intelligence or will, or that if it gets intelligent enough it will automatically be benevolent, or that we can just pull the plug right? Or they just aren't paying any attention, or they have too much trust in the government, or too much trust in the giant corporations. Do you spend all your energy worrying about alignment or do you continue lobbying for policies that address things most of them are starting to believe in and spend some quality time with your family?
I'm not arguing that this is for sure a correct position, but it is coherent, maybe even reasonable.
The data from PauseAI and ControlAI is that most people haven't heard about the machine god or even the automatize most jobs thing. Once they hear about it they think it's insane and that it should be stopped.
>half the population thinks AI is a parlor trick, or that it has already reached its peak capability because it ran out of data ....
From speaking to around 300 random people it's mostly not paying attention but when they hear about what's going on they think: "WTF, this is insane".
The bubble we are in overselects hard for people who are difficult to convince/have given up.
Yeah they might react that way when you tell them, then what? What about when they talk to the next guy who is some Ed Zitron type? There was a time when single payer was obscenely popular in America, 60% or more, and that's an actual fairly concrete policy plan, nothing happened on that and it's hard to see how it even could have if, say, Bernie Sanders had been elected. Again, I'm not saying it's hopeless, I still bring up these concerns to people, but the mundane and obvious is easier to convey, and even then it's hard to get people concerned let alone agreeing on what to do about it.
Yea, I think it's good to discuss these concerns because I don't think things are as dark (except for Gradual Disempowerment).
* Ed Zitron types will become less relevant as AI keeps going up and getting more powerful
* Single payer is way more complicated than AI I think.
> I still bring up these concerns to people, but the mundane and obvious is easier to convey, and even then it's hard to get people concerned let alone agreeing on what to do about it.
As far as talking to people: Demonstrating stuff/showing a claude code demo, some statements by AI CEOs, Scientists can help. Grown not built (no one understands the systems).
Getting people to become active seems like the main difficulty to me and we are trying hard to optimize it. One: The success rate is low so it's hard to stay motivated. However, one person who themselves becomes actually active can then do a lot. The second and third order effects of getting a person active are huge.
PauseAI/TBC/ControlAI/Humanity in Control/...? Are all trying to optimize this. It depends. There is micro-commit, becoming a member somewhere, sharing a post, thinking about it, getting one step closer to doing something.
I can recommend https://torchbearer.community/ if you want to become more active. There are some smart and fun people like Connor trying to solve this. Micro-commit to just work on the problem for a few minutes per week: https://microcommit.io/. If you haven't you could contact your lawmakers right now: https://controlai.com/take-action
The hardest part is getting people to understand, and not doubt, RSI driven exponential growth. It’s made tougher because the intelligence explosion could feasibly not happen and people (very reasonably!) hate the SV people and disbelieve what they say.
>Okay, but why is it happening?
Mode collapse often lands on nerdy/fantasy tropes. LLMs love names like "Elara" and "Aeris" and "Varis", which sound like fantasy characters. In 2022, ChatGPT was obsessed with saying "delve".
To make up a just-so story, goblins are playful and nonthreatening, don't have intense/sexual/culturally-freighted connotations like demons/witches, and are well-entrenched in pop cultural usage (use of "gremlin" for a minor but perplexing technical issue goes back at least to WW2). So they're probably a fairly big target in the RL landscape.
I'm surprised LLMs haven't fixated on monkeys, pirates and ninjas.
> If ‘I used a real LLM instead’ is a valid response, then This Is Fine, and if you can just type ‘good morning, copilot’ then that works, too. Maybe you do both?
Have Claude Code open Copilot and say hello.
People, Simon Willison's prominent blog called GPT-5 Thinking his "Research Goblin" in Sept. 2025. https://simonwillison.net/2025/Sep/6/research-goblin/ Can it really be a coincidence that the next model release is the one that started obsessing over goblins?
The OpenAI post links a Reddit thread from April 2025 suggesting that some people were noticing "fitness goblin" and "chaos goblin" coming out a bit too much: https://www.reddit.com/r/ChatGPT/comments/1k5hg5c/does_anyone_elses_chatgpt_refer_to_people_as/
I suspect that Willison came up with the term "Research Goblin" in conversation with ChatGPT, which was already secretly obsessed.
Ok, the timing doesn’t fit then. Thank you!
I note with embarrassed chagrin that I didn't even realize EY had two different twitter accounts. Thought he just switched handles. The "Relevant People" sidebar feature is honestly pretty neat, and it'd be cool to have a similar widget on other fora-type wobsites. (Baffled at Substack still leaving huge whitespace to either side of the content, it's so many wasted pixels?) Especially for topics like AI, which have a spiraling cast of characters that it's easy to lose track of. This is also why I'm pleased at the ongoing blog practice of putting (organization and/or context of post) after peoples' names for block quotes and tweets, it's quite helpful.