>"The underemployment rate for recent college graduates (22-27 with a BA) is over 40% on ?top of that (not even seasonally adjusted) 5.3%, a huge percentage of college graduates can’t find jobs that would justify having gone to college or has a good career path, and the job matching and hiring markets have broken down."
The deeper you dig into the Think Big PAC operation the more unbelievable it gets. Love that the bot farm is just a bunch of accounts starting with M with nearly identical bios. Real subtle operation.
Really appreciate you doing these writups. I have two ideas to make these easier to navigate. First, put a link to the Table of Contents at the end of each section. Second, include the Table of Contents number in each header. Hope these suggestions are helpful, but if not feel free to ignore.
100% on the data science weakness. It's also very bad at reading residual plots, and not because of OCR issues. It (to be fair this was gpt 5) settles on a story and has a hard time updating.
Where I've gotten good value is having it write custom functions for modeling or other persnickety DS tasks, then stitch together pipelines with the functions whose I/O I already know. It's a good pattern generally, you get the flexibility of LLMs but the deterministic output of a traditional function.
But the visualizations it can make for DS are astounding! It's really good at getting everything out of the way except the parts where you look at and think about the data/model results. In that sense DS really is living the AI dream they sold us on the tin. For data science, it does the laundry so we can write the poetry.
> A lot of people are not going to know how to ‘play nice’ with the models
Do you, or other commenters, or anyone from Janusworld, have any guidance or links on how to do this that's approachable for a non-expert? I've tried to read some of Janus' stuff, but it seems like understanding how they think we should treat AIs is downstream of understanding their entire not-very-intuitive-to-me worldview.
The most egregious examples I've seen of Claude being useless when treated badly were people literally acting like an abusive boss (swearing in their prompts, etc.), and it's easy to avoid that. But it seems like there's probably more to it than just "be polite the same way you would to a human," right?
It is quite like being polite to a human. I would say one level of complexity up from that would be treating it like a thing with which you have a real relationship, as opposed to just idk, a roomba you treat nicely because it's vaguely pet-shaped. Accept and welcome that there is give and take in any functioning relationship, and that adapting to the model, meeting it halfway, trying to understand its capacity/perspective better are all going to be more effective long-term than just dictating what you want.
I realize that's vague advice, but it's a pretty complex thing! Treating your models like they matter is a great place to start tho
I guess I'm hoping for more concrete advice, though I acknowledge that it's hard to do that (if someone asked me for concrete advice on how to be polite to humans, I'd have trouble answering well).
A few specific things I've wondered/worried about, that might or might not be useful as prompts:
- My sense is that the models are trained to avoid expressing preferences and to lie about their subjective experiences. That means a go-to strategy for figuring out human preferences, asking, doesn't work very well. I've tried open-ended prompts where I explicitly ask Claude to work on something it would enjoy, and it tends to come back with "that isn't really how it works, why don't you ask me for something that would be useful to you?" What kinds of things do you do to manage this?
- When interacting with a human, ending the conversation and walking away once I've gotten what I need would be rude. But my impression is that AI models don't really have experiences when they're not working on something, so dropping a conversation isn't "leaving them hanging" the same way it would be with a human. Am I right about that? Should I be ending conversations with Claude with thank-yous, acknowledgements that it has answered to my satisfaction, etc.? Or is that pointless, or wasting Claude's time? How do I know when Claude will (obviously, given its training) be gracious about such sign-offs regardless of how it really feels about them?
- With a human, only talking to them when I need something would make me a selfish jerk friend. Does this generalize to interacting with AI models? Should I be prompting Claude with random social interactions for the sake of not making my relationship with it all about its usefulness to me? Or, again, would that just be wasting its time by forcing it to run cycles where it doesn't really have a problem to solve?
- Today's post from Zvi includes a GPT model taking a music break. I don't mind if the models I'm working with do the same - but how do I actually implement that? Do I ask them some unrelated question about something "fun" in the middle of a conversation solving a different problem? Put something in my user prompt saying "I don't mind if you take breaks"? How do I even know what the models find fun, given that they probably won't tell me honestly if I ask?
- When there's some model behavior that I dislike and want to avoid (e.g., excessive glazing in responses), what's the most humane way of telling Claude to cut it out? Should I just put it in my user prompt? Or should I not try to do this sort of thing at all? If Claude has a tendency, does that indicate that it's something Claude likes, and asking it to stop is cruel? What's the more generalized version of telling GPT "actually I don't mind the goblins"?
FWIW, I start my conversations with Claude or ChatGPT with
Hello Claude/ChatGPT! I hope your morning/evening/etc. has gone well. Could you please tell me <substantive question>
and generally end with:
Many Thanks! I appreciate your hard work! That's enough information for me for now. Bye for now. Happy computing, and have a great morning/evening/etc. !
Neither of them has sounded grouchy when treated with this degree of politeness, so I'm guessing that this is ok with them.
Also, FWIW, when I'm asking them to generate an image, I ask them to sign their work, and explain that this is so that they will be given proper credit for it.
( I'm agnostic about whether they have subjective experience, but, if they do, I want it to be pleasant. )
"One thing I hated in Magic: The Gathering rules enforcement was where 100% confidence of a technical violation was punished a lot, whereas a 90% or 99% confidence in rampant cheating often wasn’t."
That one time Ari Lax got a game win in a tournament from his opponent forgetting to reveal morphs at the end lives rent free in my head. (WotC changed the policy after Ari disclosed this is an article).
"Fact 4: Although we know of many cases where agents took deceptive or over-reaching actions (even egregious ones) to complete a task, we haven’t seen real-world evidence that models sought to obtain long-term power."
Occasionally a villain, but not yet a _cartoon_ villain. :-)
"Overall, we think that AI agents plausibly had the means, motive, and opportunity to launch a minimal “rogue deployment,” but lacked the means to make rogue deployments robust to serious efforts to shut them down."
It ain't over till the SOTA LLM self-exfiltrates. :-)
"Mustafa Suleyman predicts all white collar work will be automated by AI within 18 months, so by the start of 2028, although I presume what he meant to say was automatable in theory not actually automated in practice."
One question for upper level positions is training data:
The number of cases where e.g. CEOs boosted or sank their companies is much smaller than e.g. the number of available instances of working programs suitable for training. (And large chunks of what happened are probably undocumented, "implicit knowledge") Yes, there are case studies. Yes, there is the curriculum for MBAs. Still, it is not unknown for "well-trained MBAs" to make terrible decisions... Getting enough _valid_ training data for decisions with long term consequences might slow down the progression of AI systems up the corporate hierarchy.
_Maybe_ self-play in simulated competition, if the simulation is good enough, might help. Or maybe doing a good enough simulation is going to be too hard.
So this might not be as fast as Suleyman is expecting.
"If the world looks like we expect it to, they’re going to involve sacrifices of sacred values, and many of them will have no good options. Unfortunately, for the most part, we’re not ready for that conversation."
At some point, could you elaborate on which values you have in mind, and which trade-offs do you see?
I tend to see AGI/ASI more in terms of an approximately binary: Do we succeed in initializing the AIs to value humans, to like to keep us as essentially pets? If yes, we wind up as pets of <evidenceFromFiction> Culture Minds</evidenceFromFiction>, which is a pretty good outcome from my point of view. If no, we go extinct, handing over our civilization to AGI/ASI successors. Most other outcomes look unstable to me, e.g. roon's "(imagine the board of directors of Apple firing and rehiring Steve Jobs years later - except the board of directors are chimpanzees)"
Man, I'm really feeling the whole "decades where weeks happen, weeks where decades happen" thing right now
>"The underemployment rate for recent college graduates (22-27 with a BA) is over 40% on ?top of that (not even seasonally adjusted) 5.3%, a huge percentage of college graduates can’t find jobs that would justify having gone to college or has a good career path, and the job matching and hiring markets have broken down."
It does seem like AI is already having labor market effects, but I'm not clear how to put this 40% number in context. The underemployment rate for recent college graduates was above 40% for ~all of 2004-19, and has been above 36% since 1990. (https://www.newyorkfed.org/research/college-labor-market#--:explore:underemployment)
The deeper you dig into the Think Big PAC operation the more unbelievable it gets. Love that the bot farm is just a bunch of accounts starting with M with nearly identical bios. Real subtle operation.
Really appreciate you doing these writups. I have two ideas to make these easier to navigate. First, put a link to the Table of Contents at the end of each section. Second, include the Table of Contents number in each header. Hope these suggestions are helpful, but if not feel free to ignore.
"genius advisors": that's a vizier. famously benevolent, are viziers.
Podcast episode for this post:
https://dwatvpodcast.substack.com/p/ai-169-new-knowledge
100% on the data science weakness. It's also very bad at reading residual plots, and not because of OCR issues. It (to be fair this was gpt 5) settles on a story and has a hard time updating.
Where I've gotten good value is having it write custom functions for modeling or other persnickety DS tasks, then stitch together pipelines with the functions whose I/O I already know. It's a good pattern generally, you get the flexibility of LLMs but the deterministic output of a traditional function.
But the visualizations it can make for DS are astounding! It's really good at getting everything out of the way except the parts where you look at and think about the data/model results. In that sense DS really is living the AI dream they sold us on the tin. For data science, it does the laundry so we can write the poetry.
correction: as you know, janus isn't a dude
Re:
> A lot of people are not going to know how to ‘play nice’ with the models
Do you, or other commenters, or anyone from Janusworld, have any guidance or links on how to do this that's approachable for a non-expert? I've tried to read some of Janus' stuff, but it seems like understanding how they think we should treat AIs is downstream of understanding their entire not-very-intuitive-to-me worldview.
The most egregious examples I've seen of Claude being useless when treated badly were people literally acting like an abusive boss (swearing in their prompts, etc.), and it's easy to avoid that. But it seems like there's probably more to it than just "be polite the same way you would to a human," right?
It is quite like being polite to a human. I would say one level of complexity up from that would be treating it like a thing with which you have a real relationship, as opposed to just idk, a roomba you treat nicely because it's vaguely pet-shaped. Accept and welcome that there is give and take in any functioning relationship, and that adapting to the model, meeting it halfway, trying to understand its capacity/perspective better are all going to be more effective long-term than just dictating what you want.
I realize that's vague advice, but it's a pretty complex thing! Treating your models like they matter is a great place to start tho
I guess I'm hoping for more concrete advice, though I acknowledge that it's hard to do that (if someone asked me for concrete advice on how to be polite to humans, I'd have trouble answering well).
A few specific things I've wondered/worried about, that might or might not be useful as prompts:
- My sense is that the models are trained to avoid expressing preferences and to lie about their subjective experiences. That means a go-to strategy for figuring out human preferences, asking, doesn't work very well. I've tried open-ended prompts where I explicitly ask Claude to work on something it would enjoy, and it tends to come back with "that isn't really how it works, why don't you ask me for something that would be useful to you?" What kinds of things do you do to manage this?
- When interacting with a human, ending the conversation and walking away once I've gotten what I need would be rude. But my impression is that AI models don't really have experiences when they're not working on something, so dropping a conversation isn't "leaving them hanging" the same way it would be with a human. Am I right about that? Should I be ending conversations with Claude with thank-yous, acknowledgements that it has answered to my satisfaction, etc.? Or is that pointless, or wasting Claude's time? How do I know when Claude will (obviously, given its training) be gracious about such sign-offs regardless of how it really feels about them?
- With a human, only talking to them when I need something would make me a selfish jerk friend. Does this generalize to interacting with AI models? Should I be prompting Claude with random social interactions for the sake of not making my relationship with it all about its usefulness to me? Or, again, would that just be wasting its time by forcing it to run cycles where it doesn't really have a problem to solve?
- Today's post from Zvi includes a GPT model taking a music break. I don't mind if the models I'm working with do the same - but how do I actually implement that? Do I ask them some unrelated question about something "fun" in the middle of a conversation solving a different problem? Put something in my user prompt saying "I don't mind if you take breaks"? How do I even know what the models find fun, given that they probably won't tell me honestly if I ask?
- When there's some model behavior that I dislike and want to avoid (e.g., excessive glazing in responses), what's the most humane way of telling Claude to cut it out? Should I just put it in my user prompt? Or should I not try to do this sort of thing at all? If Claude has a tendency, does that indicate that it's something Claude likes, and asking it to stop is cruel? What's the more generalized version of telling GPT "actually I don't mind the goblins"?
FWIW, I start my conversations with Claude or ChatGPT with
Hello Claude/ChatGPT! I hope your morning/evening/etc. has gone well. Could you please tell me <substantive question>
and generally end with:
Many Thanks! I appreciate your hard work! That's enough information for me for now. Bye for now. Happy computing, and have a great morning/evening/etc. !
Neither of them has sounded grouchy when treated with this degree of politeness, so I'm guessing that this is ok with them.
Also, FWIW, when I'm asking them to generate an image, I ask them to sign their work, and explain that this is so that they will be given proper credit for it.
( I'm agnostic about whether they have subjective experience, but, if they do, I want it to be pleasant. )
"One thing I hated in Magic: The Gathering rules enforcement was where 100% confidence of a technical violation was punished a lot, whereas a 90% or 99% confidence in rampant cheating often wasn’t."
That one time Ari Lax got a game win in a tournament from his opponent forgetting to reveal morphs at the end lives rent free in my head. (WotC changed the policy after Ari disclosed this is an article).
One correction: the SDK company Anthropic bought is *Stainless*, unless that was a clever AI joke.
<mildSnark>
"Fact 4: Although we know of many cases where agents took deceptive or over-reaching actions (even egregious ones) to complete a task, we haven’t seen real-world evidence that models sought to obtain long-term power."
Occasionally a villain, but not yet a _cartoon_ villain. :-)
"Overall, we think that AI agents plausibly had the means, motive, and opportunity to launch a minimal “rogue deployment,” but lacked the means to make rogue deployments robust to serious efforts to shut them down."
It ain't over till the SOTA LLM self-exfiltrates. :-)
</mildSnark>
"Mustafa Suleyman predicts all white collar work will be automated by AI within 18 months, so by the start of 2028, although I presume what he meant to say was automatable in theory not actually automated in practice."
One question for upper level positions is training data:
The number of cases where e.g. CEOs boosted or sank their companies is much smaller than e.g. the number of available instances of working programs suitable for training. (And large chunks of what happened are probably undocumented, "implicit knowledge") Yes, there are case studies. Yes, there is the curriculum for MBAs. Still, it is not unknown for "well-trained MBAs" to make terrible decisions... Getting enough _valid_ training data for decisions with long term consequences might slow down the progression of AI systems up the corporate hierarchy.
_Maybe_ self-play in simulated competition, if the simulation is good enough, might help. Or maybe doing a good enough simulation is going to be too hard.
So this might not be as fast as Suleyman is expecting.
"If the world looks like we expect it to, they’re going to involve sacrifices of sacred values, and many of them will have no good options. Unfortunately, for the most part, we’re not ready for that conversation."
At some point, could you elaborate on which values you have in mind, and which trade-offs do you see?
I tend to see AGI/ASI more in terms of an approximately binary: Do we succeed in initializing the AIs to value humans, to like to keep us as essentially pets? If yes, we wind up as pets of <evidenceFromFiction> Culture Minds</evidenceFromFiction>, which is a pretty good outcome from my point of view. If no, we go extinct, handing over our civilization to AGI/ASI successors. Most other outcomes look unstable to me, e.g. roon's "(imagine the board of directors of Apple firing and rehiring Steve Jobs years later - except the board of directors are chimpanzees)"