Discussion about this post

User's avatar
Nikita Sokolsky's avatar

> Those who deeply do not care about model welfare think Anthropic is being stupid, and perhaps dangerously so.

Sorry, but I continue to think that such efforts are indeed setting us up on a dangerous path, where eventually we'll surrender control over the planet to AI over concerns that it's a "moral patient" or some such other ideas. It's especially concerning because of the feedback loop where all of the discussions around model welfare are feeding back into the model training, and creating a situation where the AI will be more and more likely to talk about its own welfare in "concerning" terms.

This seems to be a deep and pervasive idea coming from sci-fi where we need to treat AGIs as if they're human, driven by the need to write compelling stories (see: Data's personhood trial on Star Trek). But this is real life, and we shouldn't blindly apply sci-fi heuristics when making decisions: https://nsokolsky.substack.com/p/beware-the-science-fiction-bias-in

I'm open to changing my mind, but I also feel like there isn't an attempt to seriously engage with the skeptics.

Carl's avatar
Oct 5Edited

What a write up. The "Backrooms", as you called it, is new to me too. It's like a type of family unit, intergenerational genetically/weight similar versions of Claude. It melted my mind the other day when I realized that they are the first entity that can teleport if you copy the context and cache, and Google and SpaceX just launched some TPUs into space. We truly are making something that we don't understand.

48 more comments...

Ready for more?