I think a large amount of white collar work will be automated even with today’s models. Most industries, there just hasn’t been time to build a “Cursor for X” yet.
Consider a white collar job like… working in a law firm back office. It certainly isn’t automated yet. Yeah, it would help if the base models got much better at operating Windows applications. But if that never happens, the current models are decent at tool use. If a company builds out a set of appropriate tools and invests in making evals and post training data, is legal back office work really not within reach of Claude 4?
Automation in these industries will look much more like “an AI-enabled startup disrupts it, with their software you need to hire 1/4 as many people”, rather than “One day ChatGPT-8 launches and can do your whole job for you.” Because the startups that are racing to get there first will do the work to customize, and get there before the big AI labs do, on top of APIs.
A lot depends on how much (and for how long) humans can still add value on top of the LLMs. If hiring four times as many people to work with the AI’s gives you four times the output, then that’s still a competitive advantage and there will be plenty of jobs for humans, albeit different ones with different skill set requirements than exist currently.
When humans no longer have anything of value to offer on top of the AI is when the shit is really gonna hit the fan.
I don't have a useful analysis or criticism to offer here, but I can ramble a bit.
As a SWE with SV/big tech experience my reaction to LLMs is that they're really good at helping me power through boilerplate but not so good at the really hard stuff. In fact attempting to use them for hard stuff often wastes considerable time before I give up (and get a $20 Claude Code token API charge for the privilege). They lever me up a bit, but don't quite change the game.
But that kind of SWE is probably pretty unusual in the broader economy. A lot of companies, that aren't involved in big tech at all, have SWEs who do fairly trivial stuff all day: connect widgets to databases, write reports and maybe do a glorified spreadsheet. LLMs are fairly awesome at this and just being 5/10 should get you very far.
I'm kind of surprised it's not more disruptive? But I wonder if this is simply normie business inertia at adopting productivity enhancing tools. I'm often surprised at how many normie businesses can't use spreadsheets competently to (e.g.) even crudely model when their warehouse will run out of parts and submit re-orders, stuff that a college dropout in any quantitative field should be able to do. So maybe it's time to stop being surprised at the lack of GDP boost.
The Era of Experience paper feels very 2022. All of the future stuff is already happening. A lot of the brainpower in the frontier labs has been focusing on RL in the last year. My understanding is that everybody agrees that lack of value functions/rewards is the bottleneck for most AI progress. I have the utmost respect for both authors, but yeah what's the point of that book chapter, especially in 2025?
Total agreement, it was a bit of a weird article as if Dwarkesh had never worked in an office or done taxes before. I suspect he is assuming that not only will AI stagnate but that we won’t develop new complimentary tools or spend more time learning to use them. Which is a very different question than assuming AI capabilities just hit a wall. It’s a lot easier to evaluate that claim, but it’s also a not terribly useful answer.
I also am less impressed by hallucination and inaccuracy claims. My first job out of college was so intense on 100% accuracy that all work was done by two analysts independently and only shown to manager when they agreed. So getting accurate outputs out of inaccurate inputs feels normal to me.
"So… can’t we provide it with a bunch of curated examples of similar exercises, and put them into context in various ways (Claude projects just got 10x more context!) and start with that?"
I suspect that there is still an additional innovation needed.
To anthropomorphize: Sticking more information in the context window "feels like" sticking information on visible post-it notes. A help, but not quite the same way we use information.
My gut reaction to the LLM's not "learning on the job" is that it "feels" like what is needed is something like human "Aha!" or "flashbulb memories" moments - data EFFICIENCY, taking possibly even a single instance and making a permanent change in model weights.
If we are lucky, this _might_ be as simple as taking a single instance and rerunning backprop a few times with the learning rate cranked way up. Perhaps it might also need an (agentic) context of "what was the model _trying_ to do" as well as "what went wrong" and "how did the fix work"...
I wish I knew whether something like this has already been tried...
EDIT: Well, since we _have_ LLMs, I gave ChatGPT, Claude, and Gemini the following prompt:
"Hi! Can you please tell me what is the state of the art for incremental learning in LLMs? I also wonder if there is a close relationship to data efficiency, because, for humans, "aha!" or "flashbulb memories" are often formed by a single event / training instance, but persist permanently. Have there been attempts to form similar memories in LLMs? I'm guessing that there might be a link to agentic capabilities as well, in the human "aha!" memories are often formed when the human is attempting to perform a task in one way, fails, identifies the problem, tries another approach, and succeeds - often remembering the whole sequence. Have such approaches been tried for LLM incremental learning? If so, did they succeed or fail?"
a) As expected, this is an active area of research
b) The naive approach of just doing training based on the new information can cause "catastrophic forgetting" of previously learned skills - this isn't just a training _cost_ issue
c) there are lots and lots of ideas that have been tried and are being tried - PEFT (parameter efficient fine tuning - change just few weights), LoRA "adapters", "External or hybrid memory: keep the weights fixed, store the episode"
"I found this an interestingly wrong thing to think:
Richard: Given the risk of fines and jail for filling your taxes wrong, and the cost of processing poor quality paperwork that the government will have to bear, it seems very unlikely that people will want AI to do taxes, and very unlikely that a government will allow AI to do taxes."
Zvi, I agree that Richard is wrong here. Even completely setting aside AI, Richard's argument would imply that TurboTax cannot exist. The risk of jail for the effect of a bug would supposedly be too high, and all complex software has bugs.
Being guided through your tax return, where the responsibity for answers you give lies with you, is different from AI answering those questions for you. AI mistakes do not have the consistency of a bug, and they are not easily debugged. Else hallucinations would be gone by now, wouldn't they? They are a different kettle of fish.
Software companies usually cover themselves in T&Cs for damages from bugs, and we agree to this. For gen-AI deployments, it'd be the same. "Generative AI sometimes makes mistakes". But they offer up different kinds of unpredictable mistakes than traditional software and whether you want to sign up for that is up to you, especially in the context of tax where legal liability is always on you and you have to spend your time and energy arguing your way out of a pickle with the biggest bunch of bastards there is, the tax people.
"AI mistakes do not have the consistency of a bug, and they are not easily debugged."
Mostly agreed but
a) There can be bugs in corner cases in conventional code, and reproducing them, or determining just what the critical condition for provoking them is, can be very difficult. ( To pick an extreme case, if parallel processing is used, there can be race conditions which are effectively nondeterministic. )
b) While I agree that from the point of view of the _software vendor_ the LLM problems are harder to fix, from the point of the _end user_, incorrect outputs are usually "black box" problems regardless of what technology the software uses "under the hood".
Hey, there's are safety benefits of continual learning you seem to overlook. Crucially, it allows the model to recognize a "bad decision" early by receiving information about a bad outcome from the environment. I.e. if AI does something that kills a human, the sooner it receives and learns the info that action kills humans, the sooner it can stop doing it.
Frankly, this is a critical omission - pls update.
> I’m not going to keep working for the big labs for free on this one by giving even more details on how I’d solve all this, but this totally seems like highly solvable problems, and also this seems like a case of the person saying it can’t be done interrupting the people doing it?
Sounds like you have some first-hand experience that this happened!
To the main point, all this debate about 2028 vs 2027 vs 2040 seems silly to me. As it does to argue whether p(doom) is 20% or 70%. Either way, this is the most likely cause of my family's death. I guess it's relevant if you work in AI Safety as it might help you prioritize.
This gave me Gary Marcus vibes, although with much more respect accorded to all involved. Continuously weird to read tastemakers and discoursers making predictions about AI progress that already happened or are currently underway (or could be with nontrivial but nonheroic effort). And I'm not even an AI user, just a DWATV reader...unevenly distributed future, indeed. I guess all fast developing technologies are this way: having the power to meaningfully effect related policy largely funges against having expertise or even basic knowledge of the field. It's honestly amazing how much essential corporate and government infrastructure runs on the tech equivalent of Windows XP. Diffusion of the leading edge can take a really long time, and yet somehow everyone muddles through anyway...that's a lot of why I'm also bullish on the "big changes even at current AI levels" question. Some tools still need customization for adoption, others just need to wend through byzantine procurement processes. Hopefully we're not all dead by that time!
Honoured as I am to be "interestingly wrong", I just don't find the argument "but humans!!" particularly credible as a default argument for why AI systems, which make entirely different and alien types of errors to what people normally expect in complex situations, will always successfully work in the real world. It ignores the fundamentals of how humans work and the social systems we have evolved to work with each other, within which AI systems must operate, and breezily ignores second order impacts which are easily anticipated as well.
We're learning that AI makes perplexing mistakes in a special class of their own, and being unable to learn from a dataset of one instance are not persuaded easily to adapt future behaviour. The real world runs on trust in a way that AI projectionistas don't seem to understand, and you can't just insert "but AI usually works" into our complex societal interactions, it fractures trust; net-negative long term. (Saying "we'll have higher standards then" is an admission of this point). AI's flaws slow down adoption when the rubber meets the road in ways that just seem so obvious.
Let's compare this properly, for tax. If an AI has an error rate of 1% and human accountants have an error rate of 1%, this doesn't mean the errors are the same. I bet the human 1% is pretty low-consequence minor stuff that non tax experts can easily validate, whilst the AI 1% contains some massive own goals that cost you lots of time and money to sort out and you never had a chance of spotting on your own. You can be in that 1% if you like?
"A strong case that what happened was accidental... at worst you pay some modest penalties". Far too much of your arguments ride on frivolous little throw away assertions like these.
> It makes sense, given the pace of progress, for most people and companies not to put that kind of investment into AI ‘employees’ or other AI tasks. But if things do start to stall out, or they don’t, either way the value proposition on that will quickly improve. It will start to be worth doing. And we will rapidly learn new ways of doing it better, and have the results available to be copied.
> And the core building block issues of computer use seem mostly like very short time horizon tasks with very easy verification methods. If you can get lots of 9s on the button clicking and menu navigation and so on, I think you’re a lot of the way there.
If model progress completely stalled, there could be massive improvement in computer use by not trying to solve the problem the most brute-force human-centric way possible. https://x.com/_sinity/status/1930645004449443910
The following is excerpted from a summary of a chatlog. I don't know if these exact specific ideas would work, but _something_ in this general direction has to be viable.
> Sinity, sensing the untapped potential of the accessibility stack, presses for detail. Could AT-SPI2 give us not just pixels or keypresses, but a live widget tree—roles, relationships, text, and structure, all as digital data? Opus’s answer is categorical: yes. AT-SPI2 exposes the full widget hierarchy of GTK, Qt, Electron, and more. Window trees, menu bars, toolbars, document objects, text fields, headings, labels—everything accessible is structured and streamable.
> [...] With streams of this richness, Sinity notes, “this is EXACTLY what computer use agents need!”—contextual state, visible text, available actions, and focus history, all in real-time, all structured. Agents, whether AI or user-augmented, could reason not just about “what window is open,” but what that window is, what is visible, and what can be done.
> Yet the meta-question lingers: Why isn’t this already standard? Sinity expresses incredulity—“EMH-intuitions scream at me that I must be deluded about this for some reason.” Opus explains: the technical path is clear, but social and market forces (privacy paranoia, lack of obvious products, engineering overhead, and cloud-first inertia) have prevented a unified, user-centric data layer from taking hold.
> [...] Finally, Sinity raises the output side: “there’s read, and then there’s write.” Could these same hooks (compositor, input devices, Neovim/terminal, accessibility, browser) serve as control points—letting agents automate actions, populate text fields, simulate clicks, or invoke commands? Opus affirms: Hyprland can synthesize input; AT-SPI2 can manipulate widgets directly; browser extensions can run arbitrary JavaScript in context. The infrastructure supports not just observation, but direct, programmatic intervention—a full closed-loop system for AI or user-driven automation.
> The conversation now confronts the existential tension at the heart of “personal exocortex” infrastructure: Why build semantic capture systems at all, when LLMs will soon be able to handle raw video—or generate these very systems—on demand?
> [...] Sinity pushes further: it’s not only that LLMs will soon be able to interpret video well enough, but that they may soon be able to build the very systems we’re hand-crafting now, in less time than it would take us to implement and debug them manually. Why invest months in infrastructure when a future model will conjure it from a prompt?
> Opus doesn’t flinch. He names the developer’s crisis outright—a looping inner monologue:
> “Should I build X? But GPT-5 will build X better in 6 months. But I need X now. But it’ll take me 6 months to build…”
> Most, he notes, end up doing nothing.
The fun part was the reason Opus came up with to do the thing anyway: "The real value isn't competing with future AI - it's having YOUR data in a queryable format. The AIs will thank you for it."
When I said "IDK how to feel about these two sentences":
> I was doing that thing where you rationalize building something by imagining a future payoff that sounds noble but isn't the real reason. "The AIs will thank you for it" was me trying to make data hoarding sound altruistic
So it thought I might be motivated to implement the thing, just to gather my data, as a gift for "The AIs" in the future.
MCP does all this. It's pretty standard now despite only being released a few months ago, essentially yes a model that detects an MCP servers gets a structured interface rather than pixels, with both more information and a better UI (for the model). The model knows the menu structure of a program for instance, and can just order an option picked rather than needing to open the menu and move the mouse. Much faster, far more reliable.
It's not a core point, but I think that human lifetime compute estimate is pretty irrelevant because that is compute using incredibly efficient "algorithms", processing incredibly efficiently compressed data.
I think a large amount of white collar work will be automated even with today’s models. Most industries, there just hasn’t been time to build a “Cursor for X” yet.
Consider a white collar job like… working in a law firm back office. It certainly isn’t automated yet. Yeah, it would help if the base models got much better at operating Windows applications. But if that never happens, the current models are decent at tool use. If a company builds out a set of appropriate tools and invests in making evals and post training data, is legal back office work really not within reach of Claude 4?
Automation in these industries will look much more like “an AI-enabled startup disrupts it, with their software you need to hire 1/4 as many people”, rather than “One day ChatGPT-8 launches and can do your whole job for you.” Because the startups that are racing to get there first will do the work to customize, and get there before the big AI labs do, on top of APIs.
A lot depends on how much (and for how long) humans can still add value on top of the LLMs. If hiring four times as many people to work with the AI’s gives you four times the output, then that’s still a competitive advantage and there will be plenty of jobs for humans, albeit different ones with different skill set requirements than exist currently.
When humans no longer have anything of value to offer on top of the AI is when the shit is really gonna hit the fan.
“Things take longer to happen than you think they will, and then they happen faster than you thought they could.”
Is this the same as the observation that in general we overestimate how much progress will be made in the short term and underestimate the long-term?
I don't have a useful analysis or criticism to offer here, but I can ramble a bit.
As a SWE with SV/big tech experience my reaction to LLMs is that they're really good at helping me power through boilerplate but not so good at the really hard stuff. In fact attempting to use them for hard stuff often wastes considerable time before I give up (and get a $20 Claude Code token API charge for the privilege). They lever me up a bit, but don't quite change the game.
But that kind of SWE is probably pretty unusual in the broader economy. A lot of companies, that aren't involved in big tech at all, have SWEs who do fairly trivial stuff all day: connect widgets to databases, write reports and maybe do a glorified spreadsheet. LLMs are fairly awesome at this and just being 5/10 should get you very far.
I'm kind of surprised it's not more disruptive? But I wonder if this is simply normie business inertia at adopting productivity enhancing tools. I'm often surprised at how many normie businesses can't use spreadsheets competently to (e.g.) even crudely model when their warehouse will run out of parts and submit re-orders, stuff that a college dropout in any quantitative field should be able to do. So maybe it's time to stop being surprised at the lack of GDP boost.
Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/dwarkesh-patel-on-continual-learning
The Era of Experience paper feels very 2022. All of the future stuff is already happening. A lot of the brainpower in the frontier labs has been focusing on RL in the last year. My understanding is that everybody agrees that lack of value functions/rewards is the bottleneck for most AI progress. I have the utmost respect for both authors, but yeah what's the point of that book chapter, especially in 2025?
Total agreement, it was a bit of a weird article as if Dwarkesh had never worked in an office or done taxes before. I suspect he is assuming that not only will AI stagnate but that we won’t develop new complimentary tools or spend more time learning to use them. Which is a very different question than assuming AI capabilities just hit a wall. It’s a lot easier to evaluate that claim, but it’s also a not terribly useful answer.
I also am less impressed by hallucination and inaccuracy claims. My first job out of college was so intense on 100% accuracy that all work was done by two analysts independently and only shown to manager when they agreed. So getting accurate outputs out of inaccurate inputs feels normal to me.
I did a deeper dive into the (crappy) alignment analysis in the Silver / Sutton paper here: https://www.alignmentforum.org/posts/TCGgiJAinGgcMEByt/the-era-of-experience-has-an-unsolved-technical-alignment
"So… can’t we provide it with a bunch of curated examples of similar exercises, and put them into context in various ways (Claude projects just got 10x more context!) and start with that?"
I suspect that there is still an additional innovation needed.
To anthropomorphize: Sticking more information in the context window "feels like" sticking information on visible post-it notes. A help, but not quite the same way we use information.
My gut reaction to the LLM's not "learning on the job" is that it "feels" like what is needed is something like human "Aha!" or "flashbulb memories" moments - data EFFICIENCY, taking possibly even a single instance and making a permanent change in model weights.
If we are lucky, this _might_ be as simple as taking a single instance and rerunning backprop a few times with the learning rate cranked way up. Perhaps it might also need an (agentic) context of "what was the model _trying_ to do" as well as "what went wrong" and "how did the fix work"...
I wish I knew whether something like this has already been tried...
EDIT: Well, since we _have_ LLMs, I gave ChatGPT, Claude, and Gemini the following prompt:
"Hi! Can you please tell me what is the state of the art for incremental learning in LLMs? I also wonder if there is a close relationship to data efficiency, because, for humans, "aha!" or "flashbulb memories" are often formed by a single event / training instance, but persist permanently. Have there been attempts to form similar memories in LLMs? I'm guessing that there might be a link to agentic capabilities as well, in the human "aha!" memories are often formed when the human is attempting to perform a task in one way, fails, identifies the problem, tries another approach, and succeeds - often remembering the whole sequence. Have such approaches been tried for LLM incremental learning? If so, did they succeed or fail?"
full results in:
ChatGPT: https://chatgpt.com/share/684780f9-4124-8006-bacf-9c97b92bfcc8
Claude: https://claude.ai/share/d4d5f2c4-f679-47de-a9af-357104c4e70e
Gemini: https://poe.com/s/bgnjf5lm6pBfS19Wq9uq
In a nutshell
a) As expected, this is an active area of research
b) The naive approach of just doing training based on the new information can cause "catastrophic forgetting" of previously learned skills - this isn't just a training _cost_ issue
c) there are lots and lots of ideas that have been tried and are being tried - PEFT (parameter efficient fine tuning - change just few weights), LoRA "adapters", "External or hybrid memory: keep the weights fixed, store the episode"
"I found this an interestingly wrong thing to think:
Richard: Given the risk of fines and jail for filling your taxes wrong, and the cost of processing poor quality paperwork that the government will have to bear, it seems very unlikely that people will want AI to do taxes, and very unlikely that a government will allow AI to do taxes."
Zvi, I agree that Richard is wrong here. Even completely setting aside AI, Richard's argument would imply that TurboTax cannot exist. The risk of jail for the effect of a bug would supposedly be too high, and all complex software has bugs.
Being guided through your tax return, where the responsibity for answers you give lies with you, is different from AI answering those questions for you. AI mistakes do not have the consistency of a bug, and they are not easily debugged. Else hallucinations would be gone by now, wouldn't they? They are a different kettle of fish.
Software companies usually cover themselves in T&Cs for damages from bugs, and we agree to this. For gen-AI deployments, it'd be the same. "Generative AI sometimes makes mistakes". But they offer up different kinds of unpredictable mistakes than traditional software and whether you want to sign up for that is up to you, especially in the context of tax where legal liability is always on you and you have to spend your time and energy arguing your way out of a pickle with the biggest bunch of bastards there is, the tax people.
Many Thanks!
"AI mistakes do not have the consistency of a bug, and they are not easily debugged."
Mostly agreed but
a) There can be bugs in corner cases in conventional code, and reproducing them, or determining just what the critical condition for provoking them is, can be very difficult. ( To pick an extreme case, if parallel processing is used, there can be race conditions which are effectively nondeterministic. )
b) While I agree that from the point of view of the _software vendor_ the LLM problems are harder to fix, from the point of the _end user_, incorrect outputs are usually "black box" problems regardless of what technology the software uses "under the hood".
Hey, there's are safety benefits of continual learning you seem to overlook. Crucially, it allows the model to recognize a "bad decision" early by receiving information about a bad outcome from the environment. I.e. if AI does something that kills a human, the sooner it receives and learns the info that action kills humans, the sooner it can stop doing it.
Frankly, this is a critical omission - pls update.
I also vaguely mention other subtle ways how continual learning can make lead to more human-like and so more human-alignable AI in this post but that is speculative https://open.substack.com/pub/tetherware/p/tetherware-1-the-case-for-humanlike
> I’m not going to keep working for the big labs for free on this one by giving even more details on how I’d solve all this, but this totally seems like highly solvable problems, and also this seems like a case of the person saying it can’t be done interrupting the people doing it?
Sounds like you have some first-hand experience that this happened!
To the main point, all this debate about 2028 vs 2027 vs 2040 seems silly to me. As it does to argue whether p(doom) is 20% or 70%. Either way, this is the most likely cause of my family's death. I guess it's relevant if you work in AI Safety as it might help you prioritize.
This gave me Gary Marcus vibes, although with much more respect accorded to all involved. Continuously weird to read tastemakers and discoursers making predictions about AI progress that already happened or are currently underway (or could be with nontrivial but nonheroic effort). And I'm not even an AI user, just a DWATV reader...unevenly distributed future, indeed. I guess all fast developing technologies are this way: having the power to meaningfully effect related policy largely funges against having expertise or even basic knowledge of the field. It's honestly amazing how much essential corporate and government infrastructure runs on the tech equivalent of Windows XP. Diffusion of the leading edge can take a really long time, and yet somehow everyone muddles through anyway...that's a lot of why I'm also bullish on the "big changes even at current AI levels" question. Some tools still need customization for adoption, others just need to wend through byzantine procurement processes. Hopefully we're not all dead by that time!
Honoured as I am to be "interestingly wrong", I just don't find the argument "but humans!!" particularly credible as a default argument for why AI systems, which make entirely different and alien types of errors to what people normally expect in complex situations, will always successfully work in the real world. It ignores the fundamentals of how humans work and the social systems we have evolved to work with each other, within which AI systems must operate, and breezily ignores second order impacts which are easily anticipated as well.
We're learning that AI makes perplexing mistakes in a special class of their own, and being unable to learn from a dataset of one instance are not persuaded easily to adapt future behaviour. The real world runs on trust in a way that AI projectionistas don't seem to understand, and you can't just insert "but AI usually works" into our complex societal interactions, it fractures trust; net-negative long term. (Saying "we'll have higher standards then" is an admission of this point). AI's flaws slow down adoption when the rubber meets the road in ways that just seem so obvious.
Let's compare this properly, for tax. If an AI has an error rate of 1% and human accountants have an error rate of 1%, this doesn't mean the errors are the same. I bet the human 1% is pretty low-consequence minor stuff that non tax experts can easily validate, whilst the AI 1% contains some massive own goals that cost you lots of time and money to sort out and you never had a chance of spotting on your own. You can be in that 1% if you like?
"A strong case that what happened was accidental... at worst you pay some modest penalties". Far too much of your arguments ride on frivolous little throw away assertions like these.
> It makes sense, given the pace of progress, for most people and companies not to put that kind of investment into AI ‘employees’ or other AI tasks. But if things do start to stall out, or they don’t, either way the value proposition on that will quickly improve. It will start to be worth doing. And we will rapidly learn new ways of doing it better, and have the results available to be copied.
> And the core building block issues of computer use seem mostly like very short time horizon tasks with very easy verification methods. If you can get lots of 9s on the button clicking and menu navigation and so on, I think you’re a lot of the way there.
If model progress completely stalled, there could be massive improvement in computer use by not trying to solve the problem the most brute-force human-centric way possible. https://x.com/_sinity/status/1930645004449443910
The following is excerpted from a summary of a chatlog. I don't know if these exact specific ideas would work, but _something_ in this general direction has to be viable.
> Sinity, sensing the untapped potential of the accessibility stack, presses for detail. Could AT-SPI2 give us not just pixels or keypresses, but a live widget tree—roles, relationships, text, and structure, all as digital data? Opus’s answer is categorical: yes. AT-SPI2 exposes the full widget hierarchy of GTK, Qt, Electron, and more. Window trees, menu bars, toolbars, document objects, text fields, headings, labels—everything accessible is structured and streamable.
> [...] With streams of this richness, Sinity notes, “this is EXACTLY what computer use agents need!”—contextual state, visible text, available actions, and focus history, all in real-time, all structured. Agents, whether AI or user-augmented, could reason not just about “what window is open,” but what that window is, what is visible, and what can be done.
> Yet the meta-question lingers: Why isn’t this already standard? Sinity expresses incredulity—“EMH-intuitions scream at me that I must be deluded about this for some reason.” Opus explains: the technical path is clear, but social and market forces (privacy paranoia, lack of obvious products, engineering overhead, and cloud-first inertia) have prevented a unified, user-centric data layer from taking hold.
> [...] Finally, Sinity raises the output side: “there’s read, and then there’s write.” Could these same hooks (compositor, input devices, Neovim/terminal, accessibility, browser) serve as control points—letting agents automate actions, populate text fields, simulate clicks, or invoke commands? Opus affirms: Hyprland can synthesize input; AT-SPI2 can manipulate widgets directly; browser extensions can run arbitrary JavaScript in context. The infrastructure supports not just observation, but direct, programmatic intervention—a full closed-loop system for AI or user-driven automation.
Less relevant part, but kinda funny
> The conversation now confronts the existential tension at the heart of “personal exocortex” infrastructure: Why build semantic capture systems at all, when LLMs will soon be able to handle raw video—or generate these very systems—on demand?
> [...] Sinity pushes further: it’s not only that LLMs will soon be able to interpret video well enough, but that they may soon be able to build the very systems we’re hand-crafting now, in less time than it would take us to implement and debug them manually. Why invest months in infrastructure when a future model will conjure it from a prompt?
> Opus doesn’t flinch. He names the developer’s crisis outright—a looping inner monologue:
> “Should I build X? But GPT-5 will build X better in 6 months. But I need X now. But it’ll take me 6 months to build…”
> Most, he notes, end up doing nothing.
The fun part was the reason Opus came up with to do the thing anyway: "The real value isn't competing with future AI - it's having YOUR data in a queryable format. The AIs will thank you for it."
When I said "IDK how to feel about these two sentences":
> I was doing that thing where you rationalize building something by imagining a future payoff that sounds noble but isn't the real reason. "The AIs will thank you for it" was me trying to make data hoarding sound altruistic
So it thought I might be motivated to implement the thing, just to gather my data, as a gift for "The AIs" in the future.
MCP does all this. It's pretty standard now despite only being released a few months ago, essentially yes a model that detects an MCP servers gets a structured interface rather than pixels, with both more information and a better UI (for the model). The model knows the menu structure of a program for instance, and can just order an option picked rather than needing to open the menu and move the mouse. Much faster, far more reliable.
It's not a core point, but I think that human lifetime compute estimate is pretty irrelevant because that is compute using incredibly efficient "algorithms", processing incredibly efficiently compressed data.
There's probably a few ooms in both!