On the lying and hallucinating front I've recalibrated recently as I'm coming to understand how bad things are.
The "1 kilogram of feathers" stuff is a classic trick and you can see why it'd fail, but the most glaring example is I've been discussing running with o3 and mentioned sprinting 100m every km on a 10k run , and how that may affect preparedness, recovery etc.
Not only could it not keep track over several messages of my pace, but it simply kept making up how many sprints that'd be "10 to 12" at another time mentioning "about 25 dashes"
I have the strong suspicion that openAI is outright degrading models shortly after their release, because it didn't feel like it could do such massive errors when I first started using it.
The naming "conventions" by AI companies reminds me of buying a mattress. Deliberate obfuscation preventing consumers/users from making informed choices and actually impairing acceptance (in general) of AI potential value.
The X thread on "how we use Claude Code" isn't very useful. Sure, you can use it to do things. What I want to know is what techniques create more leverage. The analogy is factory management. "We use factories to build nails and another factory to build steel rods!" Great! But how are they managed? Are there common patterns between them?
Things I've tried:
- claude manages an objective.md and status.md. It keeps these up to date to track its status and claude.md directs it on how to use it.
- claude manages a code_atlas.md that gives it a quick way to understand the codebase.
- claude has an AGENT_IDENTITY.md that gives this specific agent awareness of its own working style and cognitive modes, allowing me to try different agents in different directories with different parameters.
- claude has a /cognitivemode command with some options like analytical, lateral, minimalist with instructions on changing its own mode if it thinks it needs to
But I just try things I think of and the whole thing feels to me like the very early days of "factories." "What the heck is Ford doing over there anyway? If only we knew how to do it here?" We need deeper, faster diffusion of process techniques.
It's fuzzy. Best to break things down. Check each tiny step. Claude can check, too. If instructions are fuzzy output is even more. Keep refactoring. See also Ford automation.
Yesterday I realized that it is probably worth my time to experiment with using LLMs as teachers. For now, I've settled on Gemini because of the huge context window.
I'm experimenting with the system prompt, and am also adding (at the beginning of each chat) a few model interactions that teach Gemini how to act.
Has anyone experimented with this? I feel like I'm probably limited because of a skill isue.
Second, you'll be shocked to hear I have opinions on the humanlike AI voices from the "Fun With Media Generation" section. For some reason all of the development effort is going into "conversational" voice models, with ums and ahs and verbal tics. It's clear people are driving towards voices that can either explicitly trick you, or fool your subconscious into believing you're talking to a human conversation partner. It is really frustrating because, in my opinion, AI narrators with solid, clear, narrative delivery are much more valuable, and development in this area towards quality at a much cheaper price would do worlds of good for accessibility etc.
I have a bit of experience with the "catching bad actors" sort of work at large internet companies. The main difficulty with spam filtering is that it isn't just a classification problem. It's an adversarial system and the spammers get to test against your filter. So catching 99.99% of the spam in your test set can easily mean you miss millions of spam messages once the spammers test and see which 0.01% you miss.
So the problem isn't just classification, it's operational, like how can you delay the testing procedures of the spammers, or how quickly can you deploy new filters. To fix your spam you also have to fix mundane software engineering issues like "it's hard for us to continually and quickly do an automatic deploy to production" and often the "speeding up your infrastructure" tasks end up being the majority of the actual work of the spam-fighting teams.
Sorry for multiple comments I’m listening and commenting as I go. I’ll halt now and just listen but I have inside info behind my comments that I don’t think is public so just wanted to signal. I’ll go quiet on commenting now. Thank you for your work Zvi 🙊🙉🙈🤍🦋
Back during the years of MOOC madness, I quipped that what we actually needed was "Extremely Individualized Emotionally Intelligent Offline" education: Old MacDonald's School.
I too hope we get that with AIs, Zvi. That would be a nice thing.
"I want to love AI tutors, because it means everyone can learn all the things. We should all root for them to be great, and for education to radically improve."
Fair. The broader question is, if we are in a time line remotely like the AI 2027 one, what is it sensible for kids to learn?
Many Thanks! Those are useful and certainly Harris's "What can be, unburdened by what has been" sounded very much like a twisted version of "Those who cannot learn from history are doomed to repeat it." - George Santayana.
Still, there are other facts that are also useful, even on a timescale too short to ask even the friendliest ASI a question - e.g. that 120 volts is more than sufficient to give a nasty shock.
Are you saying we need the government to be teaching children basic everyday safety or that we should be giving them such a vast breadth of different topics, the way we try to do now, that electrics are part of the pedagogy?
Many Thanks! I honestly don't know. Given the assumption of a friendly ASI fairly ubiquitously available (e.g. through smart phones), a _lot_ of information will be available through it, just as it is now through web searches and (albeit with hallucinations today) through today's LLM chatbots.
To pick your history example: Given a sufficiently intelligent and reliable and trustworthy (not ideological!) chatbot, one could reasonably ask it "What are parallels to this last week's conflict between Israel and Iran, and what ultimately happened in each of them?" But the child at least has to know to ask the question! But this, when and if it exists, would substitute for a lot of memorization of the parallel situations ahead of time.
But a lot of information needs background information to make sense of it. How much, what kind, with what priority? It is very hard to say.
And, as in the 120 volt case, quite a large amount of information is needed too fast to allow even a 5 minute verbal inquiry even with an arbitrarily intelligent assistant accessible through one's phone.
On the lying and hallucinating front I've recalibrated recently as I'm coming to understand how bad things are.
The "1 kilogram of feathers" stuff is a classic trick and you can see why it'd fail, but the most glaring example is I've been discussing running with o3 and mentioned sprinting 100m every km on a 10k run , and how that may affect preparedness, recovery etc.
Not only could it not keep track over several messages of my pace, but it simply kept making up how many sprints that'd be "10 to 12" at another time mentioning "about 25 dashes"
I have the strong suspicion that openAI is outright degrading models shortly after their release, because it didn't feel like it could do such massive errors when I first started using it.
The naming "conventions" by AI companies reminds me of buying a mattress. Deliberate obfuscation preventing consumers/users from making informed choices and actually impairing acceptance (in general) of AI potential value.
The X thread on "how we use Claude Code" isn't very useful. Sure, you can use it to do things. What I want to know is what techniques create more leverage. The analogy is factory management. "We use factories to build nails and another factory to build steel rods!" Great! But how are they managed? Are there common patterns between them?
Things I've tried:
- claude manages an objective.md and status.md. It keeps these up to date to track its status and claude.md directs it on how to use it.
- claude manages a code_atlas.md that gives it a quick way to understand the codebase.
- claude has an AGENT_IDENTITY.md that gives this specific agent awareness of its own working style and cognitive modes, allowing me to try different agents in different directories with different parameters.
- claude has a /cognitivemode command with some options like analytical, lateral, minimalist with instructions on changing its own mode if it thinks it needs to
But I just try things I think of and the whole thing feels to me like the very early days of "factories." "What the heck is Ford doing over there anyway? If only we knew how to do it here?" We need deeper, faster diffusion of process techniques.
It's fuzzy. Best to break things down. Check each tiny step. Claude can check, too. If instructions are fuzzy output is even more. Keep refactoring. See also Ford automation.
On AI tutoring:
Yesterday I realized that it is probably worth my time to experiment with using LLMs as teachers. For now, I've settled on Gemini because of the huge context window.
I'm experimenting with the system prompt, and am also adding (at the beginning of each chat) a few model interactions that teach Gemini how to act.
Has anyone experimented with this? I feel like I'm probably limited because of a skill isue.
First, Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/ai-120-while-o3-turned-pro
Second, you'll be shocked to hear I have opinions on the humanlike AI voices from the "Fun With Media Generation" section. For some reason all of the development effort is going into "conversational" voice models, with ums and ahs and verbal tics. It's clear people are driving towards voices that can either explicitly trick you, or fool your subconscious into believing you're talking to a human conversation partner. It is really frustrating because, in my opinion, AI narrators with solid, clear, narrative delivery are much more valuable, and development in this area towards quality at a much cheaper price would do worlds of good for accessibility etc.
I have a bit of experience with the "catching bad actors" sort of work at large internet companies. The main difficulty with spam filtering is that it isn't just a classification problem. It's an adversarial system and the spammers get to test against your filter. So catching 99.99% of the spam in your test set can easily mean you miss millions of spam messages once the spammers test and see which 0.01% you miss.
So the problem isn't just classification, it's operational, like how can you delay the testing procedures of the spammers, or how quickly can you deploy new filters. To fix your spam you also have to fix mundane software engineering issues like "it's hard for us to continually and quickly do an automatic deploy to production" and often the "speeding up your infrastructure" tasks end up being the majority of the actual work of the spam-fighting teams.
If you wrote up some stories from the field i would love to read them
Are the bots the bug the feature … or the means
All the models are liar liars when push comes to shove
"But superintelligence is probably coming soon so this won’t matter much" is the new "Carthago delenda est".
Every game is won in the meta.
Sorry for multiple comments I’m listening and commenting as I go. I’ll halt now and just listen but I have inside info behind my comments that I don’t think is public so just wanted to signal. I’ll go quiet on commenting now. Thank you for your work Zvi 🙊🙉🙈🤍🦋
Back during the years of MOOC madness, I quipped that what we actually needed was "Extremely Individualized Emotionally Intelligent Offline" education: Old MacDonald's School.
I too hope we get that with AIs, Zvi. That would be a nice thing.
> Elizabeth Warren is sufficiently in opposition to retweet Marjorie Taylor Greene
I don't see her retweeting it? That's just Tegmark retweeting Greene while adding a screenshot of a similar tweet by Warren.
Factual error alert : the BBB temporarily fixes the section 174 tax treatment :
https://www.crowell.com/en/insights/client-alerts/house-committee-passes-part-of-big-beautiful-bill-containing-noteworthy-improvements-to-research-and-development-incentives-for-companies
"I want to love AI tutors, because it means everyone can learn all the things. We should all root for them to be great, and for education to radically improve."
Fair. The broader question is, if we are in a time line remotely like the AI 2027 one, what is it sensible for kids to learn?
Reading #1, then history, arithmetic, history, and finally history.
Many Thanks! Those are useful and certainly Harris's "What can be, unburdened by what has been" sounded very much like a twisted version of "Those who cannot learn from history are doomed to repeat it." - George Santayana.
Still, there are other facts that are also useful, even on a timescale too short to ask even the friendliest ASI a question - e.g. that 120 volts is more than sufficient to give a nasty shock.
Are you saying we need the government to be teaching children basic everyday safety or that we should be giving them such a vast breadth of different topics, the way we try to do now, that electrics are part of the pedagogy?
Many Thanks! I honestly don't know. Given the assumption of a friendly ASI fairly ubiquitously available (e.g. through smart phones), a _lot_ of information will be available through it, just as it is now through web searches and (albeit with hallucinations today) through today's LLM chatbots.
To pick your history example: Given a sufficiently intelligent and reliable and trustworthy (not ideological!) chatbot, one could reasonably ask it "What are parallels to this last week's conflict between Israel and Iran, and what ultimately happened in each of them?" But the child at least has to know to ask the question! But this, when and if it exists, would substitute for a lot of memorization of the parallel situations ahead of time.
But a lot of information needs background information to make sense of it. How much, what kind, with what priority? It is very hard to say.
And, as in the 120 volt case, quite a large amount of information is needed too fast to allow even a 5 minute verbal inquiry even with an arbitrarily intelligent assistant accessible through one's phone.
28. I was just following purchase orders:
By Democratic Law everyone is entitled to their own beliefs
As a matter of Nature’s law, you are not entitled to accuracy.
> to stabilize the debt levels we would have to grow 0.5% per year faster than CBO projections
This of course assumes that borrowing does not increase proportionate to any increase in GDP growth.