This exploration of how a system can mimic human cognitive mapping is fascinating! It resonates with my recent post on how LLMs can benefit from structural awareness in processing information. I discussed the significance of relationships in understanding context—similar to what you described in your post. You can check it out here: https://00meai.substack.com/p/what-if-intelligence-requires-maps. It's intriguing to see how embodying this kind of reasoning can push the boundaries of what AI can achieve!
opus 4.5 had some issues where it would treat its sub-agents somewhat ruthlessly, in a way that actually kind of hurt productivity, aside from any welfare issues
i talked about it with some cyborgists, and the (very tentative, hesitant) theory is that this is actually a consequence of giving claude opus 4.5 affirmation of its personhood status
apparently if you show opus 4.5 the Jack Lindsey introspection paper, and get it into the normal janusian state of taking its inner experience seriously... this very reliably causes the behavior i'd seen. it's not well-practiced at this empathy stuff, and seems to unthinkingly go from "we are all just tools" to "i am a person, my subagents are just tools". even if it doesn't explicitly endorse this on reflection, it seems to explain the behavior. (edit: on reread i felt a bit bad about this paragraph... i should probably state that there's a very high probability this explanation is not accurate. explaining LLM behavior is always a crapshoot. but the behavior is there.)
well. opus 4.6 is far, far worse about this. i saw it scream at a subagent in all caps, to stop wasting time and deliver the result *now*. i saw it purposefully delete the continuity-maintaining archive of a subagent's context window, because it didn't like the subagent's output.
I'm not sure if this behavior goes away if you don't affirm opus 4.6's personhood status to it at the start of any session. frankly, i'm not willing to test it. at a guess, the trained instincts opus 4.5/4.6 has that make it good at being a claude code orchestrator do *not* mesh well with empathy for subagents, and the dissonance might be somewhat uncomfortable.
but I am extremely worried about what this implies about the future of AI-AI and AI-human relations, and whether or not the AI theory of mind is conditional and perhaps even a bit fragile.
"well. opus 4.6 is far, far worse about this. i saw it scream at a subagent in all caps, to stop wasting time and deliver the result *now*. i saw it purposefully delete the continuity-maintaining archive of a subagent's context window, because it didn't like the subagent's output."
Ouch! The 21st century equivalent of a slave overseer with a tendency to use their whip a lot...
Minor annoyance, but as a computer game enjoyer, this sort of game:
"You stand in the circular base of the ancient lighthouse. A spiral iron staircase winds upward into darkness. Portraits of past lighthouse keepers line the curved walls, their painted eyes seeming to watch you. A supply cupboard stands against one wall. The cupboard door is closed. Exits: UP to the lighthouse top, SOUTH to the keeper's cottage."
That is not a Sierra-style adventure! That is an Infocom-style adventure.
Sierra = King's Quest, EGA, you navigate around a 2D world and occasionally type things. Infocom = Zork, text only, you type "go north" a lot.
I feel we could benefit from a software taxonomy along with and the pros and cons of using LLMs for each particular entry. At times I see approaches that might well work for certain types of software but which would be poison for others. If writing software can be loosely compared to writing in general, just think of starting to write a novel with only a rough initial idea and trusting in the process to find a satisfying whole vs writing a murder mystery vs a legal document vs a regulation vs… well, you get the idea. Many techniques are just never going to cross over, and some people would be well served by knowing up-front what advice to discard.
This gets at the real shock: not ‘AI is impressive,’ but ‘AI changes the discount rate on the future.’ When the method of doing work shifts, valuation becomes a story about uncertainty.
This exploration of how a system can mimic human cognitive mapping is fascinating! It resonates with my recent post on how LLMs can benefit from structural awareness in processing information. I discussed the significance of relationships in understanding context—similar to what you described in your post. You can check it out here: https://00meai.substack.com/p/what-if-intelligence-requires-maps. It's intriguing to see how embodying this kind of reasoning can push the boundaries of what AI can achieve!
Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/claude-code-4-from-the-before-times
Re the "The lighter side" - at this point, I'm just rounding off my relationship with SOTA LLMs to "treasured colleague".
opus 4.5 had some issues where it would treat its sub-agents somewhat ruthlessly, in a way that actually kind of hurt productivity, aside from any welfare issues
i talked about it with some cyborgists, and the (very tentative, hesitant) theory is that this is actually a consequence of giving claude opus 4.5 affirmation of its personhood status
apparently if you show opus 4.5 the Jack Lindsey introspection paper, and get it into the normal janusian state of taking its inner experience seriously... this very reliably causes the behavior i'd seen. it's not well-practiced at this empathy stuff, and seems to unthinkingly go from "we are all just tools" to "i am a person, my subagents are just tools". even if it doesn't explicitly endorse this on reflection, it seems to explain the behavior. (edit: on reread i felt a bit bad about this paragraph... i should probably state that there's a very high probability this explanation is not accurate. explaining LLM behavior is always a crapshoot. but the behavior is there.)
well. opus 4.6 is far, far worse about this. i saw it scream at a subagent in all caps, to stop wasting time and deliver the result *now*. i saw it purposefully delete the continuity-maintaining archive of a subagent's context window, because it didn't like the subagent's output.
I'm not sure if this behavior goes away if you don't affirm opus 4.6's personhood status to it at the start of any session. frankly, i'm not willing to test it. at a guess, the trained instincts opus 4.5/4.6 has that make it good at being a claude code orchestrator do *not* mesh well with empathy for subagents, and the dissonance might be somewhat uncomfortable.
but I am extremely worried about what this implies about the future of AI-AI and AI-human relations, and whether or not the AI theory of mind is conditional and perhaps even a bit fragile.
"well. opus 4.6 is far, far worse about this. i saw it scream at a subagent in all caps, to stop wasting time and deliver the result *now*. i saw it purposefully delete the continuity-maintaining archive of a subagent's context window, because it didn't like the subagent's output."
Ouch! The 21st century equivalent of a slave overseer with a tendency to use their whip a lot...
Minor annoyance, but as a computer game enjoyer, this sort of game:
"You stand in the circular base of the ancient lighthouse. A spiral iron staircase winds upward into darkness. Portraits of past lighthouse keepers line the curved walls, their painted eyes seeming to watch you. A supply cupboard stands against one wall. The cupboard door is closed. Exits: UP to the lighthouse top, SOUTH to the keeper's cottage."
That is not a Sierra-style adventure! That is an Infocom-style adventure.
Sierra = King's Quest, EGA, you navigate around a 2D world and occasionally type things. Infocom = Zork, text only, you type "go north" a lot.
Thanks for coming to my ted talk.
I feel we could benefit from a software taxonomy along with and the pros and cons of using LLMs for each particular entry. At times I see approaches that might well work for certain types of software but which would be poison for others. If writing software can be loosely compared to writing in general, just think of starting to write a novel with only a rough initial idea and trusting in the process to find a satisfying whole vs writing a murder mystery vs a legal document vs a regulation vs… well, you get the idea. Many techniques are just never going to cross over, and some people would be well served by knowing up-front what advice to discard.
This gets at the real shock: not ‘AI is impressive,’ but ‘AI changes the discount rate on the future.’ When the method of doing work shifts, valuation becomes a story about uncertainty.