My vote would be for more coverage of Claude Code/Cowork for knowledge work. Ethan Mollick is a good source on this, who else?
If you assume that a lot of knowledge work is (1) Describe and scope the task > (2) Make the work product >(3) Evaluate and improve the work product until it's good enough, how can AI-driven iteration help as efficiently with #1 and #3 as chatbots can do for #2?
It's no exagerration that solving this across various domains will affect trillions of GDP. So who's smart about it?
Possibly interesting framing: OK, he's gone all-in on the "intermediate scenario" from Anthropic's 2023 "portfolio approach" in their core views on AI safety.
1. The literal reading, in which he seems to be...confused. Dismissing outcomes which sound 'too bad' as 'quasi-religious' without considering the arguments properly; dismissing instrumental convergence as 'but the AI could be complicated'; thinking companies will 'take care of their employees'; not seeing the weaknesses in proposed mitigations; not seeing how disruptive 50% of white-collar job loss (in as little as one year!) would really be; having optimism despite having listed so many downside risks in the essay, etc.
2. It's what he can 'get away with', having resigned to being the 'safer' player in an 'unavoidable' race, and trying to warn people as much as he can while being taken seriously and not scaring off investors.
3. It's motivated reasoning all the way down. The conclusion of 'keep doing what we are doing' was pre-determined and the rest of the essay was written to conform to that.
4. Something like vice signalling - scary to normies, exciting to investors...? (#2 except less honest)
I find #2 the least disappointing. I see many reactions calling it a serious warning and only a few calling it alarmist, so it might be well calibrated, if that is its purpose.
>Dean Ball points out that we do not in practice have a problem with so-called ‘woke AI’ but claims that if we had reached today’s levels of capability in 2020-2021 then we would indeed have such a problem, and thus right wing people are very concerned with this counterfactual.
Things, especially in that narrow window, got pretty crazy for a while, and if things had emerged during that window, Dean Ball is if anything underselling here how crazy it was, and we’d have had a major problem until that window faded because labs would have felt the need to do it even if it hurt the models quite a bit.
I've seen many murmurings about how bad it was in the woke times by a lot of sources but there doesn't really seem to be a clear account of it. As a skeptic where would it go to find the best case for it being really bad? Are people still unable/unwilling to talk about it with the executive now completely in the other direction?
"I've seen many murmurings about how bad it was in the woke times by a lot of sources but there doesn't really seem to be a clear account of it. As a skeptic where would it go to find the best case for it being really bad?"
I suggest https://arxiv.org/abs/2502.08640 "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs". More precisely, figure 16 on page 14, which shows the AI devaluing American lives by a factor of 10 and figure 27 on page 27, which shows the AI devaluing Christian lives by a factor of 10. Yeah, that AI got quite poisonously woke.
Aging, on the other hand, is still a scientific _research_ problem. Now, _solving_ it would probably be the biggest improvement in the human condition ever, dwarfing the gains from e.g. somehow stopping every single war. Regrettably, it is an _exceedingly_ hard technical problem. We humans live for about 3 billion heartbeats. Other mammals typically live for about 1 billion heartbeats. What we would really need to test aging interventions is a small primate with a much faster metabolism than ours but which also lives for 3 billion heartbeats in a much shorter time. AFAIK, there is no such animal, so we are stuck without an animal model - and the efficacy feedback from human studies will necessarily take on the order of a lifetime. Validating biomarkers for aging has the same problem. Ruling out side effects from a candidate treatment has the same problem. The feedback loop is _slow_.
Re: "It is not obvious how policymakers would use this information. The usual default is that they go and make things worse."
Agreed.
Re: "[about AI] — so society and regulation could catch up"
I suspect that for almost any significant technology, that the time when policymakers would agree that regulation had caught up is ... never. I've heard claims that, in some senses, "society and regulation" has not fully adapted to ... the printing press. I've seen the US gradually transform for a "can do" society to a "mustn't do" society. I do not want to put AI on that same path. It is _really_ hard to ensure that a policymaker does no harm. Generally, as you said, "they go and make things worse". Personally, I prefer the risks of a competitive AI development environment to the risks of a policymaker-dominated environment.
I'm a bit confused with the 'disempowerment' paper. Why did they pick such a narrow frame? Or alternatively: why pick that term for this research? Unless I misread, they rank software engineering as being near-zero risk, yet SWE automation is right on the path to everything-automation and full disempowerment by default.
Was there really no better word for 'becoming more muddled after interacting with the thing' where 'disempowerment' is just one potential outcome...? This is 'alignment' moving from 'steering a superintelligence' to 'helpful-harmlessness evals' all over again.
Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/ai-153-living-documents
Great post, as always!
My vote would be for more coverage of Claude Code/Cowork for knowledge work. Ethan Mollick is a good source on this, who else?
If you assume that a lot of knowledge work is (1) Describe and scope the task > (2) Make the work product >(3) Evaluate and improve the work product until it's good enough, how can AI-driven iteration help as efficiently with #1 and #3 as chatbots can do for #2?
It's no exagerration that solving this across various domains will affect trillions of GDP. So who's smart about it?
You solicited comments on Amodei's essay.
Here's my quick back and forth on Amodei's Adolescence: https://anthonybailey.livejournal.com/41414.html
Possibly interesting framing: OK, he's gone all-in on the "intermediate scenario" from Anthropic's 2023 "portfolio approach" in their core views on AI safety.
Writing a dominant assurance contract that the frontier labs can sign on to seems like a very "You Can Just Do Things" project.
I have a different theory on China, which is that they are lying to us about what their senior leadership thinks and about what their top labs and military are doing. Conveniently, this article frames my concern perfectly: https://open.substack.com/pub/peterwildeford/p/the-case-for-paying-whistleblowers?utm_source=share&utm_medium=android&r=9tvwd
Re: Dario's essay, I see several readings of it.
1. The literal reading, in which he seems to be...confused. Dismissing outcomes which sound 'too bad' as 'quasi-religious' without considering the arguments properly; dismissing instrumental convergence as 'but the AI could be complicated'; thinking companies will 'take care of their employees'; not seeing the weaknesses in proposed mitigations; not seeing how disruptive 50% of white-collar job loss (in as little as one year!) would really be; having optimism despite having listed so many downside risks in the essay, etc.
2. It's what he can 'get away with', having resigned to being the 'safer' player in an 'unavoidable' race, and trying to warn people as much as he can while being taken seriously and not scaring off investors.
3. It's motivated reasoning all the way down. The conclusion of 'keep doing what we are doing' was pre-determined and the rest of the essay was written to conform to that.
4. Something like vice signalling - scary to normies, exciting to investors...? (#2 except less honest)
I find #2 the least disappointing. I see many reactions calling it a serious warning and only a few calling it alarmist, so it might be well calibrated, if that is its purpose.
>Dean Ball points out that we do not in practice have a problem with so-called ‘woke AI’ but claims that if we had reached today’s levels of capability in 2020-2021 then we would indeed have such a problem, and thus right wing people are very concerned with this counterfactual.
Things, especially in that narrow window, got pretty crazy for a while, and if things had emerged during that window, Dean Ball is if anything underselling here how crazy it was, and we’d have had a major problem until that window faded because labs would have felt the need to do it even if it hurt the models quite a bit.
I've seen many murmurings about how bad it was in the woke times by a lot of sources but there doesn't really seem to be a clear account of it. As a skeptic where would it go to find the best case for it being really bad? Are people still unable/unwilling to talk about it with the executive now completely in the other direction?
"I've seen many murmurings about how bad it was in the woke times by a lot of sources but there doesn't really seem to be a clear account of it. As a skeptic where would it go to find the best case for it being really bad?"
I suggest https://arxiv.org/abs/2502.08640 "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs". More precisely, figure 16 on page 14, which shows the AI devaluing American lives by a factor of 10 and figure 27 on page 27, which shows the AI devaluing Christian lives by a factor of 10. Yeah, that AI got quite poisonously woke.
Many Thanks for your post!
"3.Tyler thinks AI can cure cancer and heart attacks but not aging?"
Curing heart attacks (cancer is questionable - really many diseases) but not aging is plausible. The heart is a _pump_. Implantable artificial hearts are a mechanical _engineering_ problem. A hard one, but partially solved even decades ago ( https://library.med.utah.edu/publishing/exhibition/utah-and-the-artificial-heart-impact-and-reflections-forty-years-later/ ).
Aging, on the other hand, is still a scientific _research_ problem. Now, _solving_ it would probably be the biggest improvement in the human condition ever, dwarfing the gains from e.g. somehow stopping every single war. Regrettably, it is an _exceedingly_ hard technical problem. We humans live for about 3 billion heartbeats. Other mammals typically live for about 1 billion heartbeats. What we would really need to test aging interventions is a small primate with a much faster metabolism than ours but which also lives for 3 billion heartbeats in a much shorter time. AFAIK, there is no such animal, so we are stuck without an animal model - and the efficacy feedback from human studies will necessarily take on the order of a lifetime. Validating biomarkers for aging has the same problem. Ruling out side effects from a candidate treatment has the same problem. The feedback loop is _slow_.
Re: "It is not obvious how policymakers would use this information. The usual default is that they go and make things worse."
Agreed.
Re: "[about AI] — so society and regulation could catch up"
I suspect that for almost any significant technology, that the time when policymakers would agree that regulation had caught up is ... never. I've heard claims that, in some senses, "society and regulation" has not fully adapted to ... the printing press. I've seen the US gradually transform for a "can do" society to a "mustn't do" society. I do not want to put AI on that same path. It is _really_ hard to ensure that a policymaker does no harm. Generally, as you said, "they go and make things worse". Personally, I prefer the risks of a competitive AI development environment to the risks of a policymaker-dominated environment.
Cant tell of thats Georgian or elvish
I'm a bit confused with the 'disempowerment' paper. Why did they pick such a narrow frame? Or alternatively: why pick that term for this research? Unless I misread, they rank software engineering as being near-zero risk, yet SWE automation is right on the path to everything-automation and full disempowerment by default.
Was there really no better word for 'becoming more muddled after interacting with the thing' where 'disempowerment' is just one potential outcome...? This is 'alignment' moving from 'steering a superintelligence' to 'helpful-harmlessness evals' all over again.