NOTE: I had another draft that was much longer and went into my view of various regulatory options and details and other things like that. as well as the struggles ahead and various ongoing battles and lots of other stuff. I decided that it was much better to keep this short and to the point instead, and to indeed keep it short I am putting the explanation of this in a comment rather than in the main body. I do plan to post about that stuff in the future.
I'd really like to see an attempt to flesh out the argument for AI risk in greater clarify/rigor in the way it might be done in a well-functioning academic discipline. Even if you are already convinced, I feel like the process of fleshing that argument out and dealing with objections would help illuminate exactly what's necessary for the problem to arise and hence suggest what will and won't help solve it.
For instance, merely having a good grip on exactly how much risk varies with ability/intelligence or what assumptions must be made about AI motivation would suggest some solutions and rule out others.
I mean, I've been impressed with the quality and professionalism of work that's trying to solve the alignment problem and the discussions going on there. However, the argument for the danger doesn't seem to get a similar treatment (Bostrom was a good start but kinda seems like the high water mark in terms of careful treatment).
I feel like the problem is that people who are convinced want to find solutions and the doubters just aren't motivated (and it's not mainstream in any academic discipline) so it doesn't get a professional treatment.
Definitely. Everyone agrees that IF (big if) we assume AIs will kill us, then people/labs that don't believe that will continue forging ahead without taking necessary precautions and therefore cause the problem. While this Taskforce is a good idea, before you can really get into successfully doing Step 1 of "Find a solution" it will need to do Step 0 of "persuade most everyone there is a problem to be fixed." (Thankfully step 0 is complementary to, and makes step 1 easier).
Nuclear non-proliferation had Hiroshima and Nagasaki (plus the relatively low cost/knowledge to create them), the most compelling arguments ever created in favor of the need for some kind of system. AI is complex software. Create somewhat complex software worlds and unleash sufficiently complex AI on them. If you see mushroom clouds, broadcast that fact to the world. People understand how virtual worlds work, and how the mirror the real world. You have to convince everyone capable of training new AIs first, or else even if you create (and to be clear, you should!) a brilliant method of alignment, the un-convinced people (and there are just too many hypothetical counter-arguments to convince everyone, you need proof) won't implement it.
I filled out one of the google forms to this effect.
NOTE: I had another draft that was much longer and went into my view of various regulatory options and details and other things like that. as well as the struggles ahead and various ongoing battles and lots of other stuff. I decided that it was much better to keep this short and to the point instead, and to indeed keep it short I am putting the explanation of this in a comment rather than in the main body. I do plan to post about that stuff in the future.
I'd really like to see an attempt to flesh out the argument for AI risk in greater clarify/rigor in the way it might be done in a well-functioning academic discipline. Even if you are already convinced, I feel like the process of fleshing that argument out and dealing with objections would help illuminate exactly what's necessary for the problem to arise and hence suggest what will and won't help solve it.
For instance, merely having a good grip on exactly how much risk varies with ability/intelligence or what assumptions must be made about AI motivation would suggest some solutions and rule out others.
I mean, I've been impressed with the quality and professionalism of work that's trying to solve the alignment problem and the discussions going on there. However, the argument for the danger doesn't seem to get a similar treatment (Bostrom was a good start but kinda seems like the high water mark in terms of careful treatment).
I feel like the problem is that people who are convinced want to find solutions and the doubters just aren't motivated (and it's not mainstream in any academic discipline) so it doesn't get a professional treatment.
Definitely. Everyone agrees that IF (big if) we assume AIs will kill us, then people/labs that don't believe that will continue forging ahead without taking necessary precautions and therefore cause the problem. While this Taskforce is a good idea, before you can really get into successfully doing Step 1 of "Find a solution" it will need to do Step 0 of "persuade most everyone there is a problem to be fixed." (Thankfully step 0 is complementary to, and makes step 1 easier).
Nuclear non-proliferation had Hiroshima and Nagasaki (plus the relatively low cost/knowledge to create them), the most compelling arguments ever created in favor of the need for some kind of system. AI is complex software. Create somewhat complex software worlds and unleash sufficiently complex AI on them. If you see mushroom clouds, broadcast that fact to the world. People understand how virtual worlds work, and how the mirror the real world. You have to convince everyone capable of training new AIs first, or else even if you create (and to be clear, you should!) a brilliant method of alignment, the un-convinced people (and there are just too many hypothetical counter-arguments to convince everyone, you need proof) won't implement it.
I filled out one of the google forms to this effect.