> If the only way to make AGI is fundamentally an unsafe black box, then we cannot afford to make AGI.
If successfully building AGI offers three possible outcomes (pets, death, or worse than death), and if you're not interested in gambling on the nicer versions of "pets", then yeah, maybe don't build things smarter than you.
> If the only way to make AGI is fundamentally an unsafe black box, then we cannot afford to make AGI.
If successfully building AGI offers three possible outcomes (pets, death, or worse than death), and if you're not interested in gambling on the nicer versions of "pets", then yeah, maybe don't build things smarter than you.
I strongly suspect "alignment" would ultimately look similar to ChatGPT 4 alignment. It's sort of like raising kids. You try to teach them the best you know, but they're ultimately going to make their own decisions.
My P(doom|AGI) is a bit lower than Yudkowsky's, but only because I place a higher probability on non-doom versions of "pets" than he does. He has a higher P(strong alignment|AGI) then I have, but I suspect his value is still low for other reasons. However, my timelines for how doom would play out are probably longer than Yudkowsky's.
But I still think building AGI is incredibly reckless and I suspect strict forms of alignment are impossible.
So overall, yeah, my disagreements with Yudkowsky have a lot to do with facts, rhetoric and strategy, rather than downside risk. "Building something a lot smarter than you" seems extraordinarily reckless.
> If the only way to make AGI is fundamentally an unsafe black box, then we cannot afford to make AGI.
If successfully building AGI offers three possible outcomes (pets, death, or worse than death), and if you're not interested in gambling on the nicer versions of "pets", then yeah, maybe don't build things smarter than you.
I strongly suspect "alignment" would ultimately look similar to ChatGPT 4 alignment. It's sort of like raising kids. You try to teach them the best you know, but they're ultimately going to make their own decisions.
My P(doom|AGI) is a bit lower than Yudkowsky's, but only because I place a higher probability on non-doom versions of "pets" than he does. He has a higher P(strong alignment|AGI) then I have, but I suspect his value is still low for other reasons. However, my timelines for how doom would play out are probably longer than Yudkowsky's.
But I still think building AGI is incredibly reckless and I suspect strict forms of alignment are impossible.
So overall, yeah, my disagreements with Yudkowsky have a lot to do with facts, rhetoric and strategy, rather than downside risk. "Building something a lot smarter than you" seems extraordinarily reckless.