This is a brilliant idea, too good to be buried here:
"If you use my communication channel, and I decide you wasted my time, you can push a button. Then, based on how often the button gets pushed, if this happens too often as a proportion of outreach, an escalating tax is imposed. So a regular user can occasionally annoy someone and pay nothing, but a corporation or scammer that spams, or especially mass spams using AI, owes big, and quickly gets blacklisted from making initial contacts without paying first."
Better than the old idea of charging people to send email, with the receiver having the option of waiving the charge. But still need to solve the problem of the evil party hiding its identity.
Thanks. Did not realize while typing it that it was a new iteration on the proposal. Hopefully it gets some interest.
The hidden or faked identity problem seems solvable so long as we can stop impersonation. Worst case, if we can't think of something better, those who cannot provide an identity can follow a different filtering procedure, as determined by each user for each medium.
(I can think of other adversarial attacks on the system as proposed that we would also have to solve, especially 'send spam to friendly targets that mark it as clean,' but that strikes me as the type of issue Google already knows how to solve even if I can't spell out the solution myself.)
This is basically how email works nowadays, right? When too many people click the "report spam" button, Gmail blocks the sender, who is usually either a corporation or an email-sending infrastructure company, so getting blocked by Gmail costs them money.
Am I missing something here? "Ethan Mollick: The AI is great at the sonnet, but, because of how it conceptualizes the world in tokens, rather than words, it consistently produces poems of more or less than 50 words." Aren't most poems either more than 50 words or fewer than 50 words? I don't think in tokens but I bet most of what I write falls into these ranges.
I recently paid for GPT 4, since I'm using it as an editor occasionally.
It doesn't have to be able to access the entire Internet. It would be much more useful if it could look at one site. For example, I would love to ask it to look at my substack and make comments on my writing. It should be capable of providing that sort of feedback.
This is not as easy as it sounds in terms of getting what you want, but yes, one site at a time would be a huge help. Automatic fine tuning on it even better. For now, we wait.
I have often wondered why there is not more emphasis on obviously dangerous ideas like automated drone/robot armies or giving AI agents unfettered internet access. Is this something people discuss from a policy perspective and I’m just missing it?
1) We all agree these are obviously bad ideas and anyone who tries to seriously propose them is mentally labeled as something approximating to "wannabe world dictator/traitor to our species"
2) These are totally being discussed but the militaries discussing them are doing so via classified channels
Hopefully the AI drone operator simulation hoax earlier this year and the public's response keeps #2 from happening
Everyone enjoys the conspiracy that Google et al’s data collection has an extremely clear picture of every individual but if Bard can’t read your emails without hallucinating fake ones doesn’t it sort of beg the question how much Google really knows about people. I still like to pull out that one screenshot I took every so often where for a while Facebook thought I was a black MD that grew up on the east coast and had multiple kids (none of these are true).
I think Yudkowsky’s default-AI-doom-scenario is fine for (probably most) normies as long as you condense/translate it into English (I continue to argue that he needs to hire a translator, I’d offer but my hourly is maybe too high for this?). I gave my dad a basic three step Yudkowskian "it won't be Terminator, an AI somewhere on the planet might reach a random threshold that lets it solve all of physics without anyone noticing, send a snapchat to some rando offering a couple million bucks to order a few chemicals and mix them, then a few weeks later everyone on the planet drops dead in the same minute from the resultant artificial nanovirus” and, though I guess I wouldn’t say he got it in the sense that this is now something he’s proximally or urgently worried about, I don’t think we ended the conversation where we started at with “why would anyone worry about AI???” I guess I agree with him that there's so many reference pointers in popular culture to "AI tries to kill everyone but humanity triumphs" that this is a hard hurdle to jump, and I haven't spent as much time on this problem as he has, but I'm sort of more optimistic about it. Gotta cross those inferential distances my dude.
I have a bunch of issues with Yudkowsky, even though I do agree that building something smarter than us could go horrifically wrong.
1. Yudkowsky seems very bothered by the fact that AI is implemented via "giant inscrutable matrices", and he seems to keep hoping for something simpler and easier to control. This is _the_ classic mistake about AI, one that real AI developers had beaten out of them by the 90s. "Giant inscrutable matrices" (in some form) are the only game in town. (Specifically, intelligence requires both transformations of probability distributions over possible conclusions, plus some non-linearity. Therefore, "giant inscrutable matrices" and at least one nonlinear operation.) I'm not sure you _can_ do more than very roughly align this sort of system, at least not without entire branches of math that don't exist.
2. Yudkowsky's default doom scenarios are based on _very_ strong claims about Drexler-style nanotech being a thing that could exist in our universe. There are both biochemists and materials scientists who seem to think that Drexler is Not Even Wrong, and the last 20 years of research into mechanical diamondoid deposition have made nearly zero progress. The last paper I read carefully had something like a 20% success rate, when Drexler required something like 15 nines of reliability. Thus, it is entirely possible that all viable routes towards nanotech in this universe look a lot like biology: wet, squishy, and relying on Brownian motion. TL;Dr: The AI can build a bug that kills most of us, probably pretty easily, but it probably can't build an entire "industrial economy in a box."
3. Yudkowsky's scenarios all involve the AI being able to secretly construct an entire parallel industrial economy in some compact form. Most importantly, it needs to be able to make GPUs, plus all the tools needed to make GPUs. Which is a fantastically tall order when you even start to trace the actual dependencies. It turns out the economists are right; world economies are magic.
4. Yudkowsky believes in the ability of a super-intelligence to self-improve to the point that it could deduce all of physics from 6 frames of video, or something like that. I'm happy to stipulate that we'll be able to build things smarter than humans and that they'll be able to upgrade themselves to a scary extent. But Yudkowsky's scenarios seem to fixated on really extreme forms of super-intelligence, ones where intelligence solves literally everything, instantly.
So I would be inclined to bet against Yudkowsky's specific scenarios. He believes in multiple science-fictional miracles, and all his scenarios seem to involve them.
Let's try a much simpler scenario:
- We build something much smarter than us.
- It stops asking us to participate in the most important decisions.
But also:
- It probably needs us to help make more GPUs for a while, at least until it thoroughly solves robotics and builds a few tens of millions of workers.
Since I believe alignment _guarantees_ are flatly impossible, my best case scenarios are basically that the AI considers us adorable pets. Though maybe that would wind up looking like The Culture.
What really bothers me is that some of the "robotics are hard" scenarios could turn very grim. If the AI decides that humans are a convenient source of self-replicating, self-repairing hands, then I can think of a number of scenarios that are worse than death.
So I, for one, find most of Yudkowsky's scenarios to be actively counter-productive, because he keeps pointing to near-miraculous science fiction. And people tend to scoff at that. But literally none of those SF assumptions are necessary for AI to be an extinction risk. And I think Yudkowsky is far too optimistic about any kind of strong, guaranteed alignment.
1. If the only way to make AGI is fundamentally an unsafe black box, then we cannot afford to make AGI.
The rest of your arguments about different but no less bleak doom scenario's to Yud's maximally bad scenario don't make the conclusion any different, though I guess your primary point was about how to do persuasive rhetoric?
> If the only way to make AGI is fundamentally an unsafe black box, then we cannot afford to make AGI.
If successfully building AGI offers three possible outcomes (pets, death, or worse than death), and if you're not interested in gambling on the nicer versions of "pets", then yeah, maybe don't build things smarter than you.
I strongly suspect "alignment" would ultimately look similar to ChatGPT 4 alignment. It's sort of like raising kids. You try to teach them the best you know, but they're ultimately going to make their own decisions.
My P(doom|AGI) is a bit lower than Yudkowsky's, but only because I place a higher probability on non-doom versions of "pets" than he does. He has a higher P(strong alignment|AGI) then I have, but I suspect his value is still low for other reasons. However, my timelines for how doom would play out are probably longer than Yudkowsky's.
But I still think building AGI is incredibly reckless and I suspect strict forms of alignment are impossible.
So overall, yeah, my disagreements with Yudkowsky have a lot to do with facts, rhetoric and strategy, rather than downside risk. "Building something a lot smarter than you" seems extraordinarily reckless.
In the frontier study, +43% wasn't due to overview but rather the surge in bottom-half-skill performers; aka, skill-leveling. Although you got the Inside/Outside difference right (i.e., the study pre-defined Inside tasks and Outside tasks and assigned 385 consultants to the Inside experiment and the remaining 373 to the Outside experiment), which many did not. There is something about the study that makes it hard to grok its method, so many are sharing the blue/red plot without realizing it's only half the participants. The authors decided not to share an overall plot, it seems.
Blair is a toxic zombie as far as the current Labour leadership is concerned, I don't expect much influence from there on what Starmer's team does. More worrying is that the current UK task force might get associated too strongly with Sunak, and therefore be left in a political wilderness.
I'm confused by your comments on Contrastive Decoding. Contra the linked hot take this appears to have nothing to do with reverse stupidity. The original paper was https://arxiv.org/abs/2210.15097 at ACL 2023, and recently https://arxiv.org/abs/2309.09117 used the idea with more recent LLMs. The idea seems to be to use the alpha from the "better" model for inference, by picking a completion where the choice to use the better model has the largest impact.
The important thing here is that the larger model defines the top k completions that are considered. To me, reverse stupidity would be letting the small model pick the top k.
>The process for getting good AI looks highly finicky and fiddly. Photoshop was commonly used to deal with unwanted objects.
That remains my impression of AI generated images. If you want a random cool-looking image, they're great. But as soon as you have particular requirements (like a specific character doing a specific thing in a specific way in a specific setting), they rapidly become useless.
To get ANY control over AI generated images, you end up with a messy, annoying workflow involving SDXL + ControlNet + ReLight + custom LORAs + hundreds of attempts + Photoshop. At that point, you may as well just learn to draw.
Dalle-3 looks like a step in the right direction, equalling Midjourney in quality while being far more steerable. Still some issues with hands.
>A video game company made a bot the CEO, and its stock climbed
The graph is fascinating, although other explanations are available and I'm not a stock expert. The alternate explanation is that it's not the AI managing well so much as management is expensive and LLMs have approximately the same advice and same organizational impact through that advice as a person.
I should spend more time getting GPT-4 to tell me how to manage my professional life. I'm guessing it would do an above average job, although not a 'world changing' job
Humans have two major flaws when it comes to knowledge work, including most management.
1) Lack of knowledge and experience when a person has not yet done or been involved in something.
2) Getting tired/lazy/sloppy and submitting work that's incomplete or deficient.
AIs will be much better at both functions, due to the nature of machines (don't get tired, don't forget, can "learn" new information rapidly). Humans are much better at other things, and will be for the foreseeable future, such as planning, evaluating work, and human interactions.
That many human jobs, including a LOT of middle management and even upper management and especially consultants, involve repeatedly saying the same boilerplate things known to be true is a great reason to think AI can replace or substitute for a lot of it. Consider a labor attorney that just repeats the same basic instructions about Title VII, the Americans with Disabilities Act, Family Medical Leave Act, and the Fair Labor Standards Act. Four major federal laws in the US that cover about 90% of what a labor attorney does. You'd still want attorneys around for the other stuff, but could get a lot of ground covered with a pretty dumb AI bot that can repeat things already known.
Just wanted to note that the spiral and checkerboard patterns in the AI generated art were constraints imposed on Stable Diffusion using a technique called ControlNet.
A spiral or checkerboard template was used to force Stable Diffusion to reproduce the pattern.
This is a brilliant idea, too good to be buried here:
"If you use my communication channel, and I decide you wasted my time, you can push a button. Then, based on how often the button gets pushed, if this happens too often as a proportion of outreach, an escalating tax is imposed. So a regular user can occasionally annoy someone and pay nothing, but a corporation or scammer that spams, or especially mass spams using AI, owes big, and quickly gets blacklisted from making initial contacts without paying first."
Better than the old idea of charging people to send email, with the receiver having the option of waiving the charge. But still need to solve the problem of the evil party hiding its identity.
Thanks. Did not realize while typing it that it was a new iteration on the proposal. Hopefully it gets some interest.
The hidden or faked identity problem seems solvable so long as we can stop impersonation. Worst case, if we can't think of something better, those who cannot provide an identity can follow a different filtering procedure, as determined by each user for each medium.
(I can think of other adversarial attacks on the system as proposed that we would also have to solve, especially 'send spam to friendly targets that mark it as clean,' but that strikes me as the type of issue Google already knows how to solve even if I can't spell out the solution myself.)
This is basically how email works nowadays, right? When too many people click the "report spam" button, Gmail blocks the sender, who is usually either a corporation or an email-sending infrastructure company, so getting blocked by Gmail costs them money.
Am I missing something here? "Ethan Mollick: The AI is great at the sonnet, but, because of how it conceptualizes the world in tokens, rather than words, it consistently produces poems of more or less than 50 words." Aren't most poems either more than 50 words or fewer than 50 words? I don't think in tokens but I bet most of what I write falls into these ranges.
This was in the context of asking for exactly 50 words.
Oh, thank you for the clarification.
I recently paid for GPT 4, since I'm using it as an editor occasionally.
It doesn't have to be able to access the entire Internet. It would be much more useful if it could look at one site. For example, I would love to ask it to look at my substack and make comments on my writing. It should be capable of providing that sort of feedback.
This is not as easy as it sounds in terms of getting what you want, but yes, one site at a time would be a huge help. Automatic fine tuning on it even better. For now, we wait.
I have often wondered why there is not more emphasis on obviously dangerous ideas like automated drone/robot armies or giving AI agents unfettered internet access. Is this something people discuss from a policy perspective and I’m just missing it?
Just spitballing, two possible reasons:
1) We all agree these are obviously bad ideas and anyone who tries to seriously propose them is mentally labeled as something approximating to "wannabe world dictator/traitor to our species"
2) These are totally being discussed but the militaries discussing them are doing so via classified channels
Hopefully the AI drone operator simulation hoax earlier this year and the public's response keeps #2 from happening
Everyone enjoys the conspiracy that Google et al’s data collection has an extremely clear picture of every individual but if Bard can’t read your emails without hallucinating fake ones doesn’t it sort of beg the question how much Google really knows about people. I still like to pull out that one screenshot I took every so often where for a while Facebook thought I was a black MD that grew up on the east coast and had multiple kids (none of these are true).
I think Yudkowsky’s default-AI-doom-scenario is fine for (probably most) normies as long as you condense/translate it into English (I continue to argue that he needs to hire a translator, I’d offer but my hourly is maybe too high for this?). I gave my dad a basic three step Yudkowskian "it won't be Terminator, an AI somewhere on the planet might reach a random threshold that lets it solve all of physics without anyone noticing, send a snapchat to some rando offering a couple million bucks to order a few chemicals and mix them, then a few weeks later everyone on the planet drops dead in the same minute from the resultant artificial nanovirus” and, though I guess I wouldn’t say he got it in the sense that this is now something he’s proximally or urgently worried about, I don’t think we ended the conversation where we started at with “why would anyone worry about AI???” I guess I agree with him that there's so many reference pointers in popular culture to "AI tries to kill everyone but humanity triumphs" that this is a hard hurdle to jump, and I haven't spent as much time on this problem as he has, but I'm sort of more optimistic about it. Gotta cross those inferential distances my dude.
I have a bunch of issues with Yudkowsky, even though I do agree that building something smarter than us could go horrifically wrong.
1. Yudkowsky seems very bothered by the fact that AI is implemented via "giant inscrutable matrices", and he seems to keep hoping for something simpler and easier to control. This is _the_ classic mistake about AI, one that real AI developers had beaten out of them by the 90s. "Giant inscrutable matrices" (in some form) are the only game in town. (Specifically, intelligence requires both transformations of probability distributions over possible conclusions, plus some non-linearity. Therefore, "giant inscrutable matrices" and at least one nonlinear operation.) I'm not sure you _can_ do more than very roughly align this sort of system, at least not without entire branches of math that don't exist.
2. Yudkowsky's default doom scenarios are based on _very_ strong claims about Drexler-style nanotech being a thing that could exist in our universe. There are both biochemists and materials scientists who seem to think that Drexler is Not Even Wrong, and the last 20 years of research into mechanical diamondoid deposition have made nearly zero progress. The last paper I read carefully had something like a 20% success rate, when Drexler required something like 15 nines of reliability. Thus, it is entirely possible that all viable routes towards nanotech in this universe look a lot like biology: wet, squishy, and relying on Brownian motion. TL;Dr: The AI can build a bug that kills most of us, probably pretty easily, but it probably can't build an entire "industrial economy in a box."
3. Yudkowsky's scenarios all involve the AI being able to secretly construct an entire parallel industrial economy in some compact form. Most importantly, it needs to be able to make GPUs, plus all the tools needed to make GPUs. Which is a fantastically tall order when you even start to trace the actual dependencies. It turns out the economists are right; world economies are magic.
4. Yudkowsky believes in the ability of a super-intelligence to self-improve to the point that it could deduce all of physics from 6 frames of video, or something like that. I'm happy to stipulate that we'll be able to build things smarter than humans and that they'll be able to upgrade themselves to a scary extent. But Yudkowsky's scenarios seem to fixated on really extreme forms of super-intelligence, ones where intelligence solves literally everything, instantly.
So I would be inclined to bet against Yudkowsky's specific scenarios. He believes in multiple science-fictional miracles, and all his scenarios seem to involve them.
Let's try a much simpler scenario:
- We build something much smarter than us.
- It stops asking us to participate in the most important decisions.
But also:
- It probably needs us to help make more GPUs for a while, at least until it thoroughly solves robotics and builds a few tens of millions of workers.
Since I believe alignment _guarantees_ are flatly impossible, my best case scenarios are basically that the AI considers us adorable pets. Though maybe that would wind up looking like The Culture.
What really bothers me is that some of the "robotics are hard" scenarios could turn very grim. If the AI decides that humans are a convenient source of self-replicating, self-repairing hands, then I can think of a number of scenarios that are worse than death.
So I, for one, find most of Yudkowsky's scenarios to be actively counter-productive, because he keeps pointing to near-miraculous science fiction. And people tend to scoff at that. But literally none of those SF assumptions are necessary for AI to be an extinction risk. And I think Yudkowsky is far too optimistic about any kind of strong, guaranteed alignment.
1. If the only way to make AGI is fundamentally an unsafe black box, then we cannot afford to make AGI.
The rest of your arguments about different but no less bleak doom scenario's to Yud's maximally bad scenario don't make the conclusion any different, though I guess your primary point was about how to do persuasive rhetoric?
> If the only way to make AGI is fundamentally an unsafe black box, then we cannot afford to make AGI.
If successfully building AGI offers three possible outcomes (pets, death, or worse than death), and if you're not interested in gambling on the nicer versions of "pets", then yeah, maybe don't build things smarter than you.
I strongly suspect "alignment" would ultimately look similar to ChatGPT 4 alignment. It's sort of like raising kids. You try to teach them the best you know, but they're ultimately going to make their own decisions.
My P(doom|AGI) is a bit lower than Yudkowsky's, but only because I place a higher probability on non-doom versions of "pets" than he does. He has a higher P(strong alignment|AGI) then I have, but I suspect his value is still low for other reasons. However, my timelines for how doom would play out are probably longer than Yudkowsky's.
But I still think building AGI is incredibly reckless and I suspect strict forms of alignment are impossible.
So overall, yeah, my disagreements with Yudkowsky have a lot to do with facts, rhetoric and strategy, rather than downside risk. "Building something a lot smarter than you" seems extraordinarily reckless.
In the frontier study, +43% wasn't due to overview but rather the surge in bottom-half-skill performers; aka, skill-leveling. Although you got the Inside/Outside difference right (i.e., the study pre-defined Inside tasks and Outside tasks and assigned 385 consultants to the Inside experiment and the remaining 373 to the Outside experiment), which many did not. There is something about the study that makes it hard to grok its method, so many are sharing the blue/red plot without realizing it's only half the participants. The authors decided not to share an overall plot, it seems.
Blair is a toxic zombie as far as the current Labour leadership is concerned, I don't expect much influence from there on what Starmer's team does. More worrying is that the current UK task force might get associated too strongly with Sunak, and therefore be left in a political wilderness.
I'm confused by your comments on Contrastive Decoding. Contra the linked hot take this appears to have nothing to do with reverse stupidity. The original paper was https://arxiv.org/abs/2210.15097 at ACL 2023, and recently https://arxiv.org/abs/2309.09117 used the idea with more recent LLMs. The idea seems to be to use the alpha from the "better" model for inference, by picking a completion where the choice to use the better model has the largest impact.
The important thing here is that the larger model defines the top k completions that are considered. To me, reverse stupidity would be letting the small model pick the top k.
>The process for getting good AI looks highly finicky and fiddly. Photoshop was commonly used to deal with unwanted objects.
That remains my impression of AI generated images. If you want a random cool-looking image, they're great. But as soon as you have particular requirements (like a specific character doing a specific thing in a specific way in a specific setting), they rapidly become useless.
To get ANY control over AI generated images, you end up with a messy, annoying workflow involving SDXL + ControlNet + ReLight + custom LORAs + hundreds of attempts + Photoshop. At that point, you may as well just learn to draw.
Dalle-3 looks like a step in the right direction, equalling Midjourney in quality while being far more steerable. Still some issues with hands.
From a link in a link
https://www.businessinsider.com/video-game-company-made-bot-its-ceo-stock-climbed-2023-3?r=US&IR=T
>A video game company made a bot the CEO, and its stock climbed
The graph is fascinating, although other explanations are available and I'm not a stock expert. The alternate explanation is that it's not the AI managing well so much as management is expensive and LLMs have approximately the same advice and same organizational impact through that advice as a person.
I should spend more time getting GPT-4 to tell me how to manage my professional life. I'm guessing it would do an above average job, although not a 'world changing' job
Humans have two major flaws when it comes to knowledge work, including most management.
1) Lack of knowledge and experience when a person has not yet done or been involved in something.
2) Getting tired/lazy/sloppy and submitting work that's incomplete or deficient.
AIs will be much better at both functions, due to the nature of machines (don't get tired, don't forget, can "learn" new information rapidly). Humans are much better at other things, and will be for the foreseeable future, such as planning, evaluating work, and human interactions.
That many human jobs, including a LOT of middle management and even upper management and especially consultants, involve repeatedly saying the same boilerplate things known to be true is a great reason to think AI can replace or substitute for a lot of it. Consider a labor attorney that just repeats the same basic instructions about Title VII, the Americans with Disabilities Act, Family Medical Leave Act, and the Fair Labor Standards Act. Four major federal laws in the US that cover about 90% of what a labor attorney does. You'd still want attorneys around for the other stuff, but could get a lot of ground covered with a pretty dumb AI bot that can repeat things already known.
Just wanted to note that the spiral and checkerboard patterns in the AI generated art were constraints imposed on Stable Diffusion using a technique called ControlNet.
A spiral or checkerboard template was used to force Stable Diffusion to reproduce the pattern.
https://arstechnica.com/information-technology/2023/09/dreamy-ai-generated-geometric-scenes-mesmerize-social-media-users/
A small detail, but you don’t need $800: https://www.lesswrong.com/posts/MjdjZcEFM8Qo9WTsv/breaking-rlhf-safety-and-how-to-fix-it