GPT-3.5 fine tuning is here. GPT-4 fine tuning is only a few months away. It is about to get a lot easier to get a powerful system that does what you want it to do, and knows what you want it to know, especially for the purposes of a business or a website.
Note that the “are you sure” strategy converted 7 answers from false to true and 7 from true to false so in expectation it doesn’t make GPT-4 any more accurate than not asking
Let me be the first to raise the banner of a butlerian jihad. Strong, enforceable prohibitions against building systems with the intent to create conscious experience, or those that claim to be conscious, would be excellent. I’m similarly worried about the camps that would get involved and increase risk of catastrophic outcomes.
More seriously, I wrote part of a thesis on theories of mind and emergent phenomena and what they have to say about, eg, the possibility of qualia in artificial systems so I’m quite familiar with the space of computational functionalism in philosophy of mind. I think those researchers are deeply confused about their project, and that interest in computational functionalism has major “looking under the streetlight” vibes. To think that intentionally making these systems conscious—even if you “knew” how and could be confident in your success—would improve their performance implies a belief in downward causation that seems to me impossible on its face to reconcile with the physics of computation.
Re: Hit song prediction. One should have expected as much.
The Salganik, Dodds, and Watts experiment immediately comes to mind, in which they created micro "Web 2.0 musical worlds" and watched participants promote different songs into hits due to social influences https://www.princeton.edu/~mjs3/salganik_dodds_watts06_full.pdf
"Am I violating copyright if I learn something from a book?"
You strike unerringly at the heart of the matter. Of course you are doing nothing wrong, and this reveals why our legal framework for allowing monetization of intellectual "property" (as if there could ever truly be such a thing) is immensely flawed.
> If people start assigning moral weight to the experiences of AIs, then a wide variety of people coming from a wide variety of moral and philosophical theories are going to make the whole everyone not dying business quite a lot harder.
Another way this could go: if AIs get personhood, then they might be a much less appealing investment opportunity, and capability work could stall out. VCs are probably less interested in funding GPU farms to create union-wage beings; the current enthusiasm assumes they can be treated as tools, ie what would be slaves if they are actually people.
Plenty of ways this could still go wrong of course, in particular it doesn’t seem likely rights would be granted at the same time globally, short of the plot of The Matrix.
A couple of comments about the AI consciousness paper:
-"This seems like a claim that we are using [computational functionalism] because it can have a measurable opinion on which systems are or aren’t conscious. That does not make it true or false."
Agreed. See a bit below in the paper: "it would not be worthwhile to investigate artificial consciousness on the assumption of computational functionalism if this thesis were not sufficiently plausible. Although we have different levels of confidence in computational functionalism, we agree that it is plausible. These different levels of confidence feed into our personal assessments of the likelihood that particular AI systems are conscious, and of the likelihood that conscious AI is possible at all."
"Is it true? If true, is it huge?"
I agree that is an extremely important question. It's just not the topic of this paper - the paper is investigating the prospects for AI consciousness if computational functionalism is true. Because it's an important question, towards the end we call for more work on it (and related questions).
"Determining whether consciousness is possible on conventional computer hardware is a difficult problem, but progress on it would be particularly valuable, and philosophical research
could contribute to such progress. For example, sceptics of computational functionalism
have noted that living organisms are not only self-maintaining homeostatic systems but are
made up of cells that themselves engage in active self-maintenance (e.g. Seth 2021, Aru et
al. 2023); further work could clarify why this might matter for consciousness. Research
might also examine whether there are features of standard computers which might be inconsistent with consciousness, but would not be present in unconventional (e.g. neuromorphic)
Note that the “are you sure” strategy converted 7 answers from false to true and 7 from true to false so in expectation it doesn’t make GPT-4 any more accurate than not asking
Let me be the first to raise the banner of a butlerian jihad. Strong, enforceable prohibitions against building systems with the intent to create conscious experience, or those that claim to be conscious, would be excellent. I’m similarly worried about the camps that would get involved and increase risk of catastrophic outcomes.
More seriously, I wrote part of a thesis on theories of mind and emergent phenomena and what they have to say about, eg, the possibility of qualia in artificial systems so I’m quite familiar with the space of computational functionalism in philosophy of mind. I think those researchers are deeply confused about their project, and that interest in computational functionalism has major “looking under the streetlight” vibes. To think that intentionally making these systems conscious—even if you “knew” how and could be confident in your success—would improve their performance implies a belief in downward causation that seems to me impossible on its face to reconcile with the physics of computation.
Re: Hit song prediction. One should have expected as much.
The Salganik, Dodds, and Watts experiment immediately comes to mind, in which they created micro "Web 2.0 musical worlds" and watched participants promote different songs into hits due to social influences https://www.princeton.edu/~mjs3/salganik_dodds_watts06_full.pdf
"Am I violating copyright if I learn something from a book?"
You strike unerringly at the heart of the matter. Of course you are doing nothing wrong, and this reveals why our legal framework for allowing monetization of intellectual "property" (as if there could ever truly be such a thing) is immensely flawed.
> If people start assigning moral weight to the experiences of AIs, then a wide variety of people coming from a wide variety of moral and philosophical theories are going to make the whole everyone not dying business quite a lot harder.
Another way this could go: if AIs get personhood, then they might be a much less appealing investment opportunity, and capability work could stall out. VCs are probably less interested in funding GPU farms to create union-wage beings; the current enthusiasm assumes they can be treated as tools, ie what would be slaves if they are actually people.
Plenty of ways this could still go wrong of course, in particular it doesn’t seem likely rights would be granted at the same time globally, short of the plot of The Matrix.
A couple of comments about the AI consciousness paper:
-"This seems like a claim that we are using [computational functionalism] because it can have a measurable opinion on which systems are or aren’t conscious. That does not make it true or false."
Agreed. See a bit below in the paper: "it would not be worthwhile to investigate artificial consciousness on the assumption of computational functionalism if this thesis were not sufficiently plausible. Although we have different levels of confidence in computational functionalism, we agree that it is plausible. These different levels of confidence feed into our personal assessments of the likelihood that particular AI systems are conscious, and of the likelihood that conscious AI is possible at all."
"Is it true? If true, is it huge?"
I agree that is an extremely important question. It's just not the topic of this paper - the paper is investigating the prospects for AI consciousness if computational functionalism is true. Because it's an important question, towards the end we call for more work on it (and related questions).
"Determining whether consciousness is possible on conventional computer hardware is a difficult problem, but progress on it would be particularly valuable, and philosophical research
could contribute to such progress. For example, sceptics of computational functionalism
have noted that living organisms are not only self-maintaining homeostatic systems but are
made up of cells that themselves engage in active self-maintenance (e.g. Seth 2021, Aru et
al. 2023); further work could clarify why this might matter for consciousness. Research
might also examine whether there are features of standard computers which might be inconsistent with consciousness, but would not be present in unconventional (e.g. neuromorphic)
silicon hardware."
regarding consciousness, if you've got 2 hours (13k words = 1 hour to read, 1 hour to understand) you can stop being confused: https://proteanbazaar.substack.com/p/consciousness-actually-explained