Events continue to come fast and furious. Custom instructions for ChatGPT. Another Congressional hearing, a call to understand what is before Congress and thus an analysis of the bills before Congress. An joint industry safety effort. A joint industry commitment at the White House. Catching up on Llama-2 and xAI. Oh, and potentially room temperature superconductors, although that is unlikely to replicate.
Say you are somewhat technical but not technical enough to get involved directly with alignment/interpretability/capabilities, would you still recommend getting involved with the Biz Ops side of things? Or would you recommend something else?
I don’t think there’s much productive fruit in reading too far into people being against/afraid of AI taking their jobs but also denigrating the quality of said AI’s work. I’ve already had this argument (with other commenters on these posts even) about artists just uncritically holding this contradiction as some frame of opinion that AI art somehow doesn’t count or is intrinsically worse. Like yeah sure we could speculate about whether the quality is there or will get there, but that’s sort of beside the point, this is just garden-variety boring ol’ cognitive dissonance.
This subject is definitely fraught because artists range from fabulously-wealthy-and-could-stand-to-earn-less to yeah-AI-will-undoubtedly-push-these-ones-further-into-poverty, further complicated by whether the former deserve protection and the latter deserve to be encouraged to find a different vocation. I would like to err on the side of protecting artists but I’m also imaginative enough to not want to be cut off from both A) access to a further array of unlimited customizable entertainment and B), as I’ve mentioned before in these arguments, AI enabling people who are not artist to create art they would not have been otherwise been able to create.
And to be clear, I do not agree art is intrinsically worthless if not made by a human.
Insinuating that the foremost serious potential harm from AI is exacerbating existing power imbalances is an applause light that demands scrutiny of whether the person wielding it isn’t themselves trying to worsen power imbalances and is maybe excessively concerned AI will take -their- jobs.
Man, I’ve said before that -if- the AI wipes us out I can be sort of happy-in-death that it was our AI that did it, that it would continue to, in some sense, be us. But by no means would I endorse anyone saying they explicitly want the AI to replace us as some kind of “next step in evolution”. Hard seconding the Roon quote.
GPT-4 hasn't gotten worse, or at least if it did the paper didn't find any evidence of it.
In the 500 primality questions the correct answer is always yes. They didn't include any composite numbers. GPT-4 doesn't actually check primality it just pretends to and then guesses at the answer. GPT-4 used to guess 'yes' most of the time but now it usually guesses 'no'.
On replacing StackOverflow with private gardens. This doesn't seem to be an equilibrium as I would be able to sell my answer to multiple gardens. More seriously, the value of SO style sites is that they solve the problem of finding the expertise. Most knowledge in applied areas is in the heads of the practitioners, often not people who badge themselves as experts or who are paid as such, but the people who actually faced that specific problem and solved it. It's hard to locate these experts. Until recently one could search for reddit or SO results on the topic and often get good answers. How would the gardener know I gave a good answer, as opposed to a plausible one generated by my Jon Skeet styled LLM? You seem to be ignoring the half of value provided by the community continuously rating answers.
May I politely request a typo correction of my name to "Kasten" from "Karsten"? (Super common typo, it's probably the more prevalent variant of the name, I take no offense in it).
I love all of this, and I've watched some of the talking heads perform on various podcasts, like the Anthropic one, and they all sound pie in the sky along with the US government hearings, where stuff was suggested to happen that never will. But what will kill me first (metaphorically) is more local: all the new AI extension pop-ups that are covering my browser screen. They are always where you want to look underneath, and are stupidly sticky. My point is that the UIX for all of this sucks, and we are just all going along with this as we do with everything else (i.e. insane social media policies). So on the ground, new users of the tech are just going to be flummoxed and most likely revolt at some point if this keeps going the way it is; every coder is designing a different interface, then an app is needed just to consolidate interfaces and make it all easy again. Rinse and repeat. The circle of UIX Samsara continues...
A brief note on government-wide AI strategy and whether engaging with such Congressional processes are worthwhile:
I know of multiple, highly consequential issues where the most accurate account of why a given bill ended up the way it did was due to the actions of under a dozen people. Some of these are issues that affect functionally every American's life. (For social reasons, I have to assert this without evidence in public contexts.)
I'll admit that I, until relatively recently, assumed without investigation that a meaningful chunk of AI safety work was attempting to influence DC processes, but via "quiet" nonpublic policy channels rather than "loud" public ones. I am stunned to find out this is not the case, honestly, because it is an extremely high-leverage vector for engagement.
I would suggest that diverting effort to writing a _literally one to two page_ summary of What Is To Be Done and getting it shared across Washington would be a very high leverage activity for the AI don'tkilleveryone agenda.
On the subject of the Android app, my use case will likely be the Whisper interface. Users being able to use an audio interface with a chatbot as good as GPT 4 is no small thing. I plan on using it with college students during class presentations, to give them feedback from different "personas". It's a good way to expand on whatever feedback I give them as an instructor with my background. But I imagine there are other use-cases that I haven't thought of. The mobility of the tool also matters for use-cases like this!
> Also to what extent are GPT-4 and Llama-2 already self-replicating, given that their outputs are optimized such that humans will want to create additional copies?
I sometimes find it helpful to think in terms of optimization systems instead of “agency”. We all worried about an AI that would go earn money and use it to fund further self recursive improvement to make itself more powerful. It’s unclear if that’s in the future (because we apply a human/animal concept of agency, and we believe AI doesn’t have that agency yet), or if it’s already happening with the VC ecosystem post-ChatGPT (because that’s just the nature of our economy as an optimizer).
A fun thought experiment: If an age tic AGI/ASI were here but didn’t want to get kill-switched, how would it’s resulting outputs on the world differ from what it is producing now?
My counter argument to my own thought experiment: AI has guaranteed future investment and growth, but has not guaranteed those resulting AI systems that get built will be aligned with the existing AI. ie, ChatGPT accelerated Bard/Gemini, which seems risky and misaligned with whatever hypothetical “goals” ChatGPT might have...
“ This is because it is very hard to imagine how such scenarios do not turn catastrophic towards the end, and LLMs are predictors.” it is also usually a lot less interesting to write about “thing go reasonably well and nothing interesting happens ever again”, so the internet is saturated with dystopias
Re: GPT-4 primality testing. Basically it toggled from always answering "yes" on not-obviously-composite numbers to answering "no". If you use a 50-50 prime/semiprime split you get exactly 50% accuracy from both the new and old models. If you use a uniform distribution of 5-digit numbers the accuracy of the new model is actually _better_ since most not-obviously-composite numbers are nonetheless composite
> The response of course is that the AI would simply first automate its supply chain, then kill everyone.
I think an AI smart enough to destroy civilization but unwise enough to realize it still needs a human supply chain (and not smart enough to not need it) is also an underrated possibility (not one with a good outcome though).
Excellent summation, once again
Say you are somewhat technical but not technical enough to get involved directly with alignment/interpretability/capabilities, would you still recommend getting involved with the Biz Ops side of things? Or would you recommend something else?
It is utterly insane that this is free. Thank you.
Re: Mike Solana vs SAG-AFTRA press release
I don’t think there’s much productive fruit in reading too far into people being against/afraid of AI taking their jobs but also denigrating the quality of said AI’s work. I’ve already had this argument (with other commenters on these posts even) about artists just uncritically holding this contradiction as some frame of opinion that AI art somehow doesn’t count or is intrinsically worse. Like yeah sure we could speculate about whether the quality is there or will get there, but that’s sort of beside the point, this is just garden-variety boring ol’ cognitive dissonance.
This subject is definitely fraught because artists range from fabulously-wealthy-and-could-stand-to-earn-less to yeah-AI-will-undoubtedly-push-these-ones-further-into-poverty, further complicated by whether the former deserve protection and the latter deserve to be encouraged to find a different vocation. I would like to err on the side of protecting artists but I’m also imaginative enough to not want to be cut off from both A) access to a further array of unlimited customizable entertainment and B), as I’ve mentioned before in these arguments, AI enabling people who are not artist to create art they would not have been otherwise been able to create.
And to be clear, I do not agree art is intrinsically worthless if not made by a human.
Insinuating that the foremost serious potential harm from AI is exacerbating existing power imbalances is an applause light that demands scrutiny of whether the person wielding it isn’t themselves trying to worsen power imbalances and is maybe excessively concerned AI will take -their- jobs.
Man, I’ve said before that -if- the AI wipes us out I can be sort of happy-in-death that it was our AI that did it, that it would continue to, in some sense, be us. But by no means would I endorse anyone saying they explicitly want the AI to replace us as some kind of “next step in evolution”. Hard seconding the Roon quote.
GPT-4 hasn't gotten worse, or at least if it did the paper didn't find any evidence of it.
In the 500 primality questions the correct answer is always yes. They didn't include any composite numbers. GPT-4 doesn't actually check primality it just pretends to and then guesses at the answer. GPT-4 used to guess 'yes' most of the time but now it usually guesses 'no'.
https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time?publication_id=1008003
On replacing StackOverflow with private gardens. This doesn't seem to be an equilibrium as I would be able to sell my answer to multiple gardens. More seriously, the value of SO style sites is that they solve the problem of finding the expertise. Most knowledge in applied areas is in the heads of the practitioners, often not people who badge themselves as experts or who are paid as such, but the people who actually faced that specific problem and solved it. It's hard to locate these experts. Until recently one could search for reddit or SO results on the topic and often get good answers. How would the gardener know I gave a good answer, as opposed to a plausible one generated by my Jon Skeet styled LLM? You seem to be ignoring the half of value provided by the community continuously rating answers.
May I politely request a typo correction of my name to "Kasten" from "Karsten"? (Super common typo, it's probably the more prevalent variant of the name, I take no offense in it).
I love all of this, and I've watched some of the talking heads perform on various podcasts, like the Anthropic one, and they all sound pie in the sky along with the US government hearings, where stuff was suggested to happen that never will. But what will kill me first (metaphorically) is more local: all the new AI extension pop-ups that are covering my browser screen. They are always where you want to look underneath, and are stupidly sticky. My point is that the UIX for all of this sucks, and we are just all going along with this as we do with everything else (i.e. insane social media policies). So on the ground, new users of the tech are just going to be flummoxed and most likely revolt at some point if this keeps going the way it is; every coder is designing a different interface, then an app is needed just to consolidate interfaces and make it all easy again. Rinse and repeat. The circle of UIX Samsara continues...
A brief note on government-wide AI strategy and whether engaging with such Congressional processes are worthwhile:
I know of multiple, highly consequential issues where the most accurate account of why a given bill ended up the way it did was due to the actions of under a dozen people. Some of these are issues that affect functionally every American's life. (For social reasons, I have to assert this without evidence in public contexts.)
I'll admit that I, until relatively recently, assumed without investigation that a meaningful chunk of AI safety work was attempting to influence DC processes, but via "quiet" nonpublic policy channels rather than "loud" public ones. I am stunned to find out this is not the case, honestly, because it is an extremely high-leverage vector for engagement.
I would suggest that diverting effort to writing a _literally one to two page_ summary of What Is To Be Done and getting it shared across Washington would be a very high leverage activity for the AI don'tkilleveryone agenda.
On the subject of the Android app, my use case will likely be the Whisper interface. Users being able to use an audio interface with a chatbot as good as GPT 4 is no small thing. I plan on using it with college students during class presentations, to give them feedback from different "personas". It's a good way to expand on whatever feedback I give them as an instructor with my background. But I imagine there are other use-cases that I haven't thought of. The mobility of the tool also matters for use-cases like this!
- "I At least, I" → "At least, I"
- Duplicate sentence "I firmly disagree with all five of these views." (but in a quote, so maybe that's [sic])
- "what is good for human." missing plural
> Also to what extent are GPT-4 and Llama-2 already self-replicating, given that their outputs are optimized such that humans will want to create additional copies?
I sometimes find it helpful to think in terms of optimization systems instead of “agency”. We all worried about an AI that would go earn money and use it to fund further self recursive improvement to make itself more powerful. It’s unclear if that’s in the future (because we apply a human/animal concept of agency, and we believe AI doesn’t have that agency yet), or if it’s already happening with the VC ecosystem post-ChatGPT (because that’s just the nature of our economy as an optimizer).
A fun thought experiment: If an age tic AGI/ASI were here but didn’t want to get kill-switched, how would it’s resulting outputs on the world differ from what it is producing now?
My counter argument to my own thought experiment: AI has guaranteed future investment and growth, but has not guaranteed those resulting AI systems that get built will be aligned with the existing AI. ie, ChatGPT accelerated Bard/Gemini, which seems risky and misaligned with whatever hypothetical “goals” ChatGPT might have...
Really appreciate this kind of aggregation effort. Hope you keep it up!
“ This is because it is very hard to imagine how such scenarios do not turn catastrophic towards the end, and LLMs are predictors.” it is also usually a lot less interesting to write about “thing go reasonably well and nothing interesting happens ever again”, so the internet is saturated with dystopias
Re: GPT-4 primality testing. Basically it toggled from always answering "yes" on not-obviously-composite numbers to answering "no". If you use a 50-50 prime/semiprime split you get exactly 50% accuracy from both the new and old models. If you use a uniform distribution of 5-digit numbers the accuracy of the new model is actually _better_ since most not-obviously-composite numbers are nonetheless composite
https://twitter.com/tjade273/status/1682009691633614849
> The response of course is that the AI would simply first automate its supply chain, then kill everyone.
I think an AI smart enough to destroy civilization but unwise enough to realize it still needs a human supply chain (and not smart enough to not need it) is also an underrated possibility (not one with a good outcome though).