One result of going on vacation was that I wasn’t able to spin events off into focused posts this week, so I’m going to fall back on splitting the weekly instead, plus some reserving a few subtopics for later posts, including AI craziness (the Tim Hua post on this is excellent
On the education front, it really seems like there’s a wide-open opportunity to train models *on how to teach things to humans*, since the student’s pre- and post-instruction performance constitutes a good objective evaluation of the model’s teaching. Could be a major PR win for a frontier lab that does this, too, since a lot of anti-ai arguments have “it makes you dumber” at their core.
Claude Code is good at deleting code, but you have to be fairly specific about it. It's very human in this way - software engineers also rarely delete code, and as they work on a project, they tend to add more and more code over time. It helps to do the high-level planning yourself, like figuring out how to test your core functionality, and which functionality should be deleted. When you give the LLM smaller tasks, the sort you would give a junior engineer, it is more effective.
I have no idea where intrinsic motivation to learn comes from. I have it, my friends have it, all high-achieving people I know have it. And others... don't. Genetics? Upbringing? Both? I just know there were people for whom learning more was not in any way motivating.
I'm just going to say, right out of the gate, it sure was not school. I think school can kill the natural proclivity to learn, or it can support it. But it can't create it where it doesn't exist.
A friend of mine, teaching seminars at Oxford, noted her frustration with the undergraduates students coming back from summer vacations: many had no read no books at all (in answer to her opening of class icebreaker), others expressed no interest in the questions being asked. A mutual friend pointed out to her that we - all former graduate students - had been the top 1% of our high schools, then the top 1% of our undergraduate programs. Most people are the 99%, even in elite institutions.
So maybe what we need is better external motivation to learn? Maybe not "beatings shall continue until moral improves" but some genuine 'You do actually need this.' Rhetoric and projections about AI coming for all the jobs sooner rather than later may be a killer of motivation, the same way telling young people they're going to die in a nuclear war isn't great for mental health.*
*AI is coming for all jobs sooner rather than later can still be true, it's just obviously demotivating for most minds.
My history is poor academic performance relative to IQ, thanks to barely having motivation to learn in higher education, followed by learning being pretty much my #1 hobby from my 30s onward. I've often wondered why this has been the case - one obvious answer is simply not finding much interesting in school, but this doesn't fully explain it as I now find some of the formerly-boring topics very interesting (maths, physics, languages). Another is having been distracted by videogames - but I still am! Maturity has to have played a role for me, I suppose. It's weird, I actually can't comprehend how I was in my teens and early 20s, I cannot empathise with that past self at all now.
1% false positive rate is still way too high for purposes of using AI detectors in schools for enforcement.
At that rate if a single student submits a hundred non-AI essays over 4 years in school, they have a better than even chance of getting flagged for cheating at least once.
A majority of students would at some point experience an extremely harsh and completely wrongful punishment. What you could do is make the punishment less harsh, like just say we won't grade anything that doesn't pass the AI detector.
At that point, figuring out how to write your way past the AI detector becomes a necessary skill even for non-cheating students, and you're going to get some new equilibrium with worse false negative rates, although perhaps they could be kept low enough that it still deters.
I can confirm that ElevenLabs still leads the pack in solid speech generation for narration, which is a bit of a shame as it remains shockingly expensive.
Apropos, here is the ElevenLabs-generated podcast episode for this post:
"As in, Trump wants TSMC, a Taiwanese company that is not American, to pay 15% profit-sharing on AI chips sold to China, which is also not America, but is otherwise fine with continuing to let China buy the chips."
Isn't such a ... proposal supposed to be delivered with Trump wearing an eyepatch, with a parrot on his shoulder, and (optionally) a peg leg?
At least the top two think tanks (Urban / Brookings) seem correct to me. But they're unusual in the extent they prioritize goals included in the ranking criteria (e.g., methodological rigor) vs goals not counted or negatively counted (e.g., articulating a consistent political viewpoint). Choosing those ranking criteria is sort of assuming the conclusion.
Further down the list I do think I see some bias in favor of leftist over center right? But that might be colored by my own politics too. The hard rightists are playing a different epistemic game entirely and aren't actually trying to compete on these criteria, no surprise they're at the bottom. Also AEI are apparently the last ones to figure out (or admit?) that the right wing has moved away from them? If anyone else had done the study, they'd be coded center right.
Ad Twitter feed filtering: I recently published a Chrome extension which does something similar - it hides posts which do not match the user preferences, as judged by an LLM. It only considers the text of the post, though.
> Notice the parallel to radiologists. Not only has demand not fallen yet, but for now pay there is very high, exactly because future demand is anticipated to be lower, and thus less doctors chose radiology. You need to pay a premium to attract talent and compensate for the lack of long term prospects.
Does anyone have any source for this? As a (very junior) doctor, I never have the impression that people are considering this when thinking about radiology as a career. Would be interested to see evidence that there's a shortage of radiologists because people are opting out for fear of AI replacement.
Thanks for the link- useful context. But as far as I can tell it just cites a source showing that radiologists are in demand and therefore can earn a lot of money. Is there a particular reason to believe that this shortage is best explained by doctors choosing not to do it for fear of AI?
AI tools have definitely caused me to raise my hiring bar for junior software engineers. Historically, my teams have hired junior engineers to meet two related needs:
1. Short-term: Increase the team's velocity by allowing senior engineers to delegate well-specified tasks. As soon as a senior engineer has a good idea of how they want something implemented, they can hand it off to a junior and just do code reviews.
2. Medium-term: Develop more senior engineers, who can develop their own designs based on a deep understanding of the product and technical options.
With AI coding tools, we basically no longer need #1. If I am able to specify the problem in enough details for a recent college grad to implement it, then AI tools can probably do it almost as well in much less time.
Because we no longer have urgent short-term needs to fill for team velocity, we can afford to be more patient hunting for great candidates to fulfill #2. My bar for junior hiring has essentially moved up to "does this person seem like a potential superstar"? A lot of otherwise productive candidates don't meet this bar.
“My guess is that a lot of why NSFW is doing relatively well is that the threshold for ‘good enough’ in NSFW is a lot lower than the threshold in many other places. “
At this point, Trekkie Monster from the Avenue Q starts singing “The Internet is for Porn”.
I'm kinda surprised/dubious as "pure LLMs" having these medical success rates. Am I missing something?
And/or: they talk about it being impossible to point to sources, but don't some LLMs do that already? Even ChatGPT5? Because I can't help but wonder if the "vignettes" mimicked the literature where diseases were discussed/identified. So "prompt engineering" becomes "vignette engineering". Maybe most doctors can do/learn that, but it needs to be shown?
Chatting to a (UK) radiologist lately they’re less concerned about AI and more about outsourcing to other countries (which an aside has complicated medicolegal implications). Curious if the locals or the outsourcers will go hard on AI first.
On the education front, it really seems like there’s a wide-open opportunity to train models *on how to teach things to humans*, since the student’s pre- and post-instruction performance constitutes a good objective evaluation of the model’s teaching. Could be a major PR win for a frontier lab that does this, too, since a lot of anti-ai arguments have “it makes you dumber” at their core.
Claude Code is good at deleting code, but you have to be fairly specific about it. It's very human in this way - software engineers also rarely delete code, and as they work on a project, they tend to add more and more code over time. It helps to do the high-level planning yourself, like figuring out how to test your core functionality, and which functionality should be deleted. When you give the LLM smaller tasks, the sort you would give a junior engineer, it is more effective.
On "School Daze":
I have no idea where intrinsic motivation to learn comes from. I have it, my friends have it, all high-achieving people I know have it. And others... don't. Genetics? Upbringing? Both? I just know there were people for whom learning more was not in any way motivating.
I'm just going to say, right out of the gate, it sure was not school. I think school can kill the natural proclivity to learn, or it can support it. But it can't create it where it doesn't exist.
A friend of mine, teaching seminars at Oxford, noted her frustration with the undergraduates students coming back from summer vacations: many had no read no books at all (in answer to her opening of class icebreaker), others expressed no interest in the questions being asked. A mutual friend pointed out to her that we - all former graduate students - had been the top 1% of our high schools, then the top 1% of our undergraduate programs. Most people are the 99%, even in elite institutions.
So maybe what we need is better external motivation to learn? Maybe not "beatings shall continue until moral improves" but some genuine 'You do actually need this.' Rhetoric and projections about AI coming for all the jobs sooner rather than later may be a killer of motivation, the same way telling young people they're going to die in a nuclear war isn't great for mental health.*
*AI is coming for all jobs sooner rather than later can still be true, it's just obviously demotivating for most minds.
My history is poor academic performance relative to IQ, thanks to barely having motivation to learn in higher education, followed by learning being pretty much my #1 hobby from my 30s onward. I've often wondered why this has been the case - one obvious answer is simply not finding much interesting in school, but this doesn't fully explain it as I now find some of the formerly-boring topics very interesting (maths, physics, languages). Another is having been distracted by videogames - but I still am! Maturity has to have played a role for me, I suppose. It's weird, I actually can't comprehend how I was in my teens and early 20s, I cannot empathise with that past self at all now.
1% false positive rate is still way too high for purposes of using AI detectors in schools for enforcement.
At that rate if a single student submits a hundred non-AI essays over 4 years in school, they have a better than even chance of getting flagged for cheating at least once.
A majority of students would at some point experience an extremely harsh and completely wrongful punishment. What you could do is make the punishment less harsh, like just say we won't grade anything that doesn't pass the AI detector.
At that point, figuring out how to write your way past the AI detector becomes a necessary skill even for non-cheating students, and you're going to get some new equilibrium with worse false negative rates, although perhaps they could be kept low enough that it still deters.
> Men use LLMs more than women, although the gap is not that large, with women being 42% of ChatGPT, 42% of Perplexity and 31% of Claude
Quibble: this is not a small gap, ratio-wise it’s about 3:2 for GPT and Ppl. and 2:1 for Claude
I can confirm that ElevenLabs still leads the pack in solid speech generation for narration, which is a bit of a shame as it remains shockingly expensive.
Apropos, here is the ElevenLabs-generated podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/ai-132-part-1-improved-ai-detection
"As in, Trump wants TSMC, a Taiwanese company that is not American, to pay 15% profit-sharing on AI chips sold to China, which is also not America, but is otherwise fine with continuing to let China buy the chips."
Isn't such a ... proposal supposed to be delivered with Trump wearing an eyepatch, with a parrot on his shoulder, and (optionally) a peg leg?
At least the top two think tanks (Urban / Brookings) seem correct to me. But they're unusual in the extent they prioritize goals included in the ranking criteria (e.g., methodological rigor) vs goals not counted or negatively counted (e.g., articulating a consistent political viewpoint). Choosing those ranking criteria is sort of assuming the conclusion.
Further down the list I do think I see some bias in favor of leftist over center right? But that might be colored by my own politics too. The hard rightists are playing a different epistemic game entirely and aren't actually trying to compete on these criteria, no surprise they're at the bottom. Also AEI are apparently the last ones to figure out (or admit?) that the right wing has moved away from them? If anyone else had done the study, they'd be coded center right.
Ad Twitter feed filtering: I recently published a Chrome extension which does something similar - it hides posts which do not match the user preferences, as judged by an LLM. It only considers the text of the post, though.
Link: https://chromewebstore.google.com/detail/great-filter/mbifgfgfbnemojmfkckodkikibihcgaj
Who else thinks that that the models all trend to center-left because center-left is broadly correct?
> Notice the parallel to radiologists. Not only has demand not fallen yet, but for now pay there is very high, exactly because future demand is anticipated to be lower, and thus less doctors chose radiology. You need to pay a premium to attract talent and compensate for the lack of long term prospects.
Does anyone have any source for this? As a (very junior) doctor, I never have the impression that people are considering this when thinking about radiology as a career. Would be interested to see evidence that there's a shortage of radiologists because people are opting out for fear of AI replacement.
Zvi mentioned radiologists in a previous AI post. See the "They Took Our Jobs" section: https://thezvi.substack.com/p/ai-128-four-hours-until-probably
I dont think it's the cite or data you're looking for but it gives a bit more context to where Zvi's head is at i think.
Thanks for the link- useful context. But as far as I can tell it just cites a source showing that radiologists are in demand and therefore can earn a lot of money. Is there a particular reason to believe that this shortage is best explained by doctors choosing not to do it for fear of AI?
It just suits the narrative, I guess.
AI tools have definitely caused me to raise my hiring bar for junior software engineers. Historically, my teams have hired junior engineers to meet two related needs:
1. Short-term: Increase the team's velocity by allowing senior engineers to delegate well-specified tasks. As soon as a senior engineer has a good idea of how they want something implemented, they can hand it off to a junior and just do code reviews.
2. Medium-term: Develop more senior engineers, who can develop their own designs based on a deep understanding of the product and technical options.
With AI coding tools, we basically no longer need #1. If I am able to specify the problem in enough details for a recent college grad to implement it, then AI tools can probably do it almost as well in much less time.
Because we no longer have urgent short-term needs to fill for team velocity, we can afford to be more patient hunting for great candidates to fulfill #2. My bar for junior hiring has essentially moved up to "does this person seem like a potential superstar"? A lot of otherwise productive candidates don't meet this bar.
“My guess is that a lot of why NSFW is doing relatively well is that the threshold for ‘good enough’ in NSFW is a lot lower than the threshold in many other places. “
At this point, Trekkie Monster from the Avenue Q starts singing “The Internet is for Porn”.
I'm kinda surprised/dubious as "pure LLMs" having these medical success rates. Am I missing something?
And/or: they talk about it being impossible to point to sources, but don't some LLMs do that already? Even ChatGPT5? Because I can't help but wonder if the "vignettes" mimicked the literature where diseases were discussed/identified. So "prompt engineering" becomes "vignette engineering". Maybe most doctors can do/learn that, but it needs to be shown?
and aren't the radiology-image AIs more CNN than LLM?
Chatting to a (UK) radiologist lately they’re less concerned about AI and more about outsourcing to other countries (which an aside has complicated medicolegal implications). Curious if the locals or the outsourcers will go hard on AI first.