The main event continues to be the fallout from The Gemini Incident. Everyone is focusing there now, and few are liking what they see. That does not mean other things stop. There were two interviews with Demis Hassabis, with Dwarkesh Patel’s being predictably excellent. We got introduced to another set of potentially highly useful AI products. Mistral partnered up with Microsoft the moment Mistral got France to pressure the EU to agree to cripple the regulations that Microsoft wanted crippled. You know. The usual stuff.
A couple of thoughts about OpenAI and its sales efforts. My understanding is that the sales team is trying to sign up enterprise customers. For ex., think Ford or Disney or Coca-Cola wanting to train an LLM on corporate data. Useful for the company and its employees, I think, and, sure, OpenAI has their API and other tooling to get it done, I guess. But OpenAI seems to want to be *both* a research org and an enterprise sales org. That...doesn't really make sense to me?
I've thought for a while now that OpenAI should separate itself into a research org, call it OpenAI_Labs, and an enterprise sales org, call it OpenAI_Sales. OpenAI_Sales licenses OpenAI_Labs software on a preferential basis, and builds out enterprise sales capability and hires a CEO who knows the enterprise sales playbook cold. Else OpenAI is just going to get curb-stomped by all the enterprise-focused companies out there that *do* know how to run enterprise sales playbooks.
> Dan Shipper spent a week with Gemini 1.5 Pro and reports it is fantastic
Aside from the part where he was fooled by a large Gemini confabulation, however - where, per Murphy's law on irony, he was using the confabulation as an example of why you should trust Gemini and 'delegate' and not worry about problems like confabulations or check the outputs. (I assume you read the pre-correction version.)
The Wonka experience failure was an AI-alignment issue, not because of the poor quality, but because of the script. The script for it was wholly AI-generated, as the lead actor has noted on TikTok, and the AI author inserted a novel, menacing non-Roald-Dahl character into the event script, who frightened the children to tears. That creepiness, I believe, is why parents called the police, not just because of the low-effort swindle, which could have been addressed without law enforcement.
"even if we do successfully get AIs to reflect the preferences expressed by the feedback they get, and even if everyone involved is well-intentioned, the hard parts of getting an AI that does things that end well would be far from over. We don’t know what we value, what we value changes," there is an assumption that we need to teach AI values, our values so that it acts on its own for our benefit as a kind of AI governor. But AI doesn't need to have any values to be able to answer queries like "Prepare detailed plan for achieving X which doesn't do harm to humans, tell me of any potential controversial consequences". Even current LLM can do that and they understand concept of "harm to humans" well enough. LLMs are amoral, they don't care about what we do with the output even if they can predict it. We can use AI as a tool even if it is way smarter than us. More about this and other common assumptions about AGI in https://medium.com/@jan.matusiewicz/agi-safety-discourse-clarification-7b94602691d8
This quote from the Dwarkesh Patel interview stood out to me:
"We’re working on things like AlphaZero-like planning mechanisms on top that make use of that model in order to make concrete plans to achieve certain goals in the world, and perhaps sort of chain thought together, or lines of reasoning together, and maybe use search to kind of explore massive spaces of possibility."
Pretty clearly refutes the arguments that LLMs won't have goals and we won't have to worry about the instrumental goal issues RL has.
"First note is that this says ‘text and images’ rather than images. Good.
However it also identifies the problem as ‘offended our users’ and ‘shown bias.’ That does not show an appreciation for the issues in play."
The charitable reading of Pichai's statement is that he knows perfectly well the underlying issues with Geminipocalypse, but deliberately chooses to frame the problems with Gemini using the same kind of language the people responsible for Geminipocalypse often push in their own justifications. Kind of a cheeky "hoist by your own petard" and "if you can use therapeutic appeals to harm and bias to advance your own purposes, then we can use those same principles to argue why you are wrong" thing.
Then again I may just be inhaling the copium and giving Google's top brass too much credit.
A pedantic reminder that that Minus comic doesn't mean what most think it does. Within the story of the long running comic, the child is a godlike being who has summoned the meteorite for her own amusement, so she can easily bat it away.
I think the difference in jobs-we-mourn and jobs-we-do-not is concentration.
There were never entire towns or communities whose existence depended on blockbuster. There are/were for coal mining. So when coal mining goes away, entire communities disappear. When blockbuster disappeared, the 0.3% of employees in the town found a new job without having to move or otherwise disrupt their life.
I think this will continue to be true.
The jobs-we-will-mourn in the future will either be hollywood jobs or silicon valley jobs, when/if those get en-masse replaced (those are the two most concentrated classes of jobs I can think off the top of my head, but I'm sure there are others).
With regards to khanmigo, I'm very curious if this is just now launching or if it just now moving out of beta, since we first heard about it almost a year ago I think. If it's the former, then hopefully very soon we will get the results of the attempt to replicate the results in Blooms 2-sigma problem paper [0]. If it's the latter, then hopefully they will share the results of the beta testing and we will have it even sooner.
In other words, I very, very much want to know if Khanmigo comes even close to the dramatic results we see with human tutors.
re: Blockbuster, I was (I guess) technically one of those jobs lost. At the time I worked at a third party company that managed their customer database and marketing. When they died - and hilariously, dumped all their debt on Blockbuster Canada - I got reassigned to the Harley-Davidson client team. As a microcosm, I think it explains why there wasn't a big uproar: almost everyone at the company had relatively marketable skills and moved onto to doing something similar with a place with a different logo - even if you were the blue shirt at the counter, there are lots of jobs that need the clerk skillset, or you were planning to upgrade to a more prestigious job anyway.
> > we have an incredible springboard for the Al wave
This makes me want to channel Bernard from "Yes, Minister", except I can't do it justice: "Er, don't you mean a surfboard? Springboards are usually found in pools, not the ocean. And if you dive into a wave, you'll wind up behind it, not in front like a surfer."
> > Our analysis found that the individual GPT-3.5 output with the highest percentage of plagiarism was in Physics
Yeah, I bet they all plagiarize "F = ma". Seriously, though, before looking closely, I register a prediction that there'll be an effect where the more hard-science the field, the more we'll see identical phrasing used to represent identical ideas, because of the number of concepts that have precise technical definitions. After looking, this seems somewhat borne out by the data, although I wonder what Psychology is doing up there.
> [filtered water metaphor]
5. If you find out that the maker of your water filter has been quietly updating it to pass through chemicals that produce effects which they view as politically desirable.
> > Gamers worldwide left confused after trying Google's new chess app.
Oh, this is just like stuff we did in elementary school. Player with their king on the right goes first, and then you have to remember whose pieces are whose. Touch a wrong piece and you forfeit your turn.
> Flagging stuff as important when it isn’t is fine, but not the other way around.
This reminds me of the short story "Huddling Place" by Clifford D. Simak. A doctor becomes intensely agoraphobic, in part due to never having to leave home because of his robot servants. He is asked to travel to another planet, to save what might be the most valuable life in all civilization, but finds it difficult. With effort he manages to pack, but then in the end, discovers that his robot butler had declined on his behalf. (Who are we, if not our patterns?)
It was Jenkins, the light from the fireplace flickering on his shining metal hide.
"Had you called earlier, sir?" he asked.
Webster shook his head.
"I was afraid you might have," Jenkins explained, "and wondered why I didn't come. There was a most extraordinary occurrence, sir. Two men came with a ship and said they wanted you to go to Mars."
"They are here," said Webster. "Why didn't you call me?"
He struggled to his feet.
"I didn't think, sir," said Jenkins, "that you would want to be bothered. It was so preposterous. I finally made them understand you could not possibly want to go to Mars."
Webster stiffened, felt chill fear gripping at his heart. Hands groping for the edge of the desk, he sat down in the chair, sensed the walls of the room closing in about him, a trap that would never let him go.
A couple of thoughts about OpenAI and its sales efforts. My understanding is that the sales team is trying to sign up enterprise customers. For ex., think Ford or Disney or Coca-Cola wanting to train an LLM on corporate data. Useful for the company and its employees, I think, and, sure, OpenAI has their API and other tooling to get it done, I guess. But OpenAI seems to want to be *both* a research org and an enterprise sales org. That...doesn't really make sense to me?
I've thought for a while now that OpenAI should separate itself into a research org, call it OpenAI_Labs, and an enterprise sales org, call it OpenAI_Sales. OpenAI_Sales licenses OpenAI_Labs software on a preferential basis, and builds out enterprise sales capability and hires a CEO who knows the enterprise sales playbook cold. Else OpenAI is just going to get curb-stomped by all the enterprise-focused companies out there that *do* know how to run enterprise sales playbooks.
> Dan Shipper spent a week with Gemini 1.5 Pro and reports it is fantastic
Aside from the part where he was fooled by a large Gemini confabulation, however - where, per Murphy's law on irony, he was using the confabulation as an example of why you should trust Gemini and 'delegate' and not worry about problems like confabulations or check the outputs. (I assume you read the pre-correction version.)
The Wonka experience failure was an AI-alignment issue, not because of the poor quality, but because of the script. The script for it was wholly AI-generated, as the lead actor has noted on TikTok, and the AI author inserted a novel, menacing non-Roald-Dahl character into the event script, who frightened the children to tears. That creepiness, I believe, is why parents called the police, not just because of the low-effort swindle, which could have been addressed without law enforcement.
https://x.com/mttpgn/status/1763233980755714251?s=46
"even if we do successfully get AIs to reflect the preferences expressed by the feedback they get, and even if everyone involved is well-intentioned, the hard parts of getting an AI that does things that end well would be far from over. We don’t know what we value, what we value changes," there is an assumption that we need to teach AI values, our values so that it acts on its own for our benefit as a kind of AI governor. But AI doesn't need to have any values to be able to answer queries like "Prepare detailed plan for achieving X which doesn't do harm to humans, tell me of any potential controversial consequences". Even current LLM can do that and they understand concept of "harm to humans" well enough. LLMs are amoral, they don't care about what we do with the output even if they can predict it. We can use AI as a tool even if it is way smarter than us. More about this and other common assumptions about AGI in https://medium.com/@jan.matusiewicz/agi-safety-discourse-clarification-7b94602691d8
This quote from the Dwarkesh Patel interview stood out to me:
"We’re working on things like AlphaZero-like planning mechanisms on top that make use of that model in order to make concrete plans to achieve certain goals in the world, and perhaps sort of chain thought together, or lines of reasoning together, and maybe use search to kind of explore massive spaces of possibility."
Pretty clearly refutes the arguments that LLMs won't have goals and we won't have to worry about the instrumental goal issues RL has.
"First note is that this says ‘text and images’ rather than images. Good.
However it also identifies the problem as ‘offended our users’ and ‘shown bias.’ That does not show an appreciation for the issues in play."
The charitable reading of Pichai's statement is that he knows perfectly well the underlying issues with Geminipocalypse, but deliberately chooses to frame the problems with Gemini using the same kind of language the people responsible for Geminipocalypse often push in their own justifications. Kind of a cheeky "hoist by your own petard" and "if you can use therapeutic appeals to harm and bias to advance your own purposes, then we can use those same principles to argue why you are wrong" thing.
Then again I may just be inhaling the copium and giving Google's top brass too much credit.
A pedantic reminder that that Minus comic doesn't mean what most think it does. Within the story of the long running comic, the child is a godlike being who has summoned the meteorite for her own amusement, so she can easily bat it away.
I think the difference in jobs-we-mourn and jobs-we-do-not is concentration.
There were never entire towns or communities whose existence depended on blockbuster. There are/were for coal mining. So when coal mining goes away, entire communities disappear. When blockbuster disappeared, the 0.3% of employees in the town found a new job without having to move or otherwise disrupt their life.
I think this will continue to be true.
The jobs-we-will-mourn in the future will either be hollywood jobs or silicon valley jobs, when/if those get en-masse replaced (those are the two most concentrated classes of jobs I can think off the top of my head, but I'm sure there are others).
With regards to khanmigo, I'm very curious if this is just now launching or if it just now moving out of beta, since we first heard about it almost a year ago I think. If it's the former, then hopefully very soon we will get the results of the attempt to replicate the results in Blooms 2-sigma problem paper [0]. If it's the latter, then hopefully they will share the results of the beta testing and we will have it even sooner.
In other words, I very, very much want to know if Khanmigo comes even close to the dramatic results we see with human tutors.
[0] https://en.wikipedia.org/wiki/Bloom%27s_2_sigma_problem
re: Blockbuster, I was (I guess) technically one of those jobs lost. At the time I worked at a third party company that managed their customer database and marketing. When they died - and hilariously, dumped all their debt on Blockbuster Canada - I got reassigned to the Harley-Davidson client team. As a microcosm, I think it explains why there wasn't a big uproar: almost everyone at the company had relatively marketable skills and moved onto to doing something similar with a place with a different logo - even if you were the blue shirt at the counter, there are lots of jobs that need the clerk skillset, or you were planning to upgrade to a more prestigious job anyway.
> > we have an incredible springboard for the Al wave
This makes me want to channel Bernard from "Yes, Minister", except I can't do it justice: "Er, don't you mean a surfboard? Springboards are usually found in pools, not the ocean. And if you dive into a wave, you'll wind up behind it, not in front like a surfer."
> > Our analysis found that the individual GPT-3.5 output with the highest percentage of plagiarism was in Physics
Yeah, I bet they all plagiarize "F = ma". Seriously, though, before looking closely, I register a prediction that there'll be an effect where the more hard-science the field, the more we'll see identical phrasing used to represent identical ideas, because of the number of concepts that have precise technical definitions. After looking, this seems somewhat borne out by the data, although I wonder what Psychology is doing up there.
> [filtered water metaphor]
5. If you find out that the maker of your water filter has been quietly updating it to pass through chemicals that produce effects which they view as politically desirable.
> > Gamers worldwide left confused after trying Google's new chess app.
Oh, this is just like stuff we did in elementary school. Player with their king on the right goes first, and then you have to remember whose pieces are whose. Touch a wrong piece and you forfeit your turn.
> Flagging stuff as important when it isn’t is fine, but not the other way around.
This reminds me of the short story "Huddling Place" by Clifford D. Simak. A doctor becomes intensely agoraphobic, in part due to never having to leave home because of his robot servants. He is asked to travel to another planet, to save what might be the most valuable life in all civilization, but finds it difficult. With effort he manages to pack, but then in the end, discovers that his robot butler had declined on his behalf. (Who are we, if not our patterns?)
http://www.pierssen.com/cfile/huddling_place.html
----
A tap came on the door.
"Come in," Webster called.
It was Jenkins, the light from the fireplace flickering on his shining metal hide.
"Had you called earlier, sir?" he asked.
Webster shook his head.
"I was afraid you might have," Jenkins explained, "and wondered why I didn't come. There was a most extraordinary occurrence, sir. Two men came with a ship and said they wanted you to go to Mars."
"They are here," said Webster. "Why didn't you call me?"
He struggled to his feet.
"I didn't think, sir," said Jenkins, "that you would want to be bothered. It was so preposterous. I finally made them understand you could not possibly want to go to Mars."
Webster stiffened, felt chill fear gripping at his heart. Hands groping for the edge of the desk, he sat down in the chair, sensed the walls of the room closing in about him, a trap that would never let him go.
Dude you've got to write less. I think this is the first one of your posts I've made it to the end of, and it took me over an hour.