I'll also call out the scenario where something like 5 to 6 ASI are instantiated in fairly close order to each other (Perhaps a a mix of private corporate ASIs and national government ASIs) and the world state veers towards a conflict state of each of these ASIs making their plays mainly against each other with humans mostly playing the …
I'll also call out the scenario where something like 5 to 6 ASI are instantiated in fairly close order to each other (Perhaps a a mix of private corporate ASIs and national government ASIs) and the world state veers towards a conflict state of each of these ASIs making their plays mainly against each other with humans mostly playing the role of manipulable pawns and resource sources.
It's a bit pedantic towards the odds, but if the first ASI (call it Alpha) gets defeated by a different ASI Beta who then goes on to disempower humans overall, that is technically a failure in steps 4/5 even though humans don't survive in the end.
I'll also suggest to Zvi that a future poll include cases like "Humanity is reduced to fewer than a million individuals" to cover the "hiding in caves" state and possibly also the lines where a social media AI is compelled to keep a supply of tame humans as bio-trophies for it's score maximization.
I'll also call out the scenario where something like 5 to 6 ASI are instantiated in fairly close order to each other (Perhaps a a mix of private corporate ASIs and national government ASIs) and the world state veers towards a conflict state of each of these ASIs making their plays mainly against each other with humans mostly playing the role of manipulable pawns and resource sources.
It's a bit pedantic towards the odds, but if the first ASI (call it Alpha) gets defeated by a different ASI Beta who then goes on to disempower humans overall, that is technically a failure in steps 4/5 even though humans don't survive in the end.
I'll also suggest to Zvi that a future poll include cases like "Humanity is reduced to fewer than a million individuals" to cover the "hiding in caves" state and possibly also the lines where a social media AI is compelled to keep a supply of tame humans as bio-trophies for it's score maximization.