The big news this week was that OpenAI is not training GPT-5, and that China’s draft rules look to be crippling restrictions on their ability to develop LLMs.
I did not intend to make a claim as strong as 'We can now go ahead and burn all our libraries without worrying.' I'm not saying China de facto doesn't exist, more that we need to stop treating it as a black box defect-bot that is only two steps behind us, running as fast as they can and almost as fast as we can, and can't be reasoned with, as a fully universal boogeyman. Or anything in the same state as that.
I do agree that if we banned AI forever, after some amount of time someone else would, if not stopped, surpass us, although I doubt it would even then be China at this rate.
I think the good take-away from the "compare AI safety to viral gain-of-function research" analogy (or to any number of other regulations) that everyone can agree is: no matter what people/governments CLAIM is "safety-improving", normal people need to aggressively audit what is actually being done and ensure that it actually does, and continues to do so in the future.
I don't think it's crazy to think that normal people, if exposed to the information, even pre 2020, would look at "pandemic safety via gain of function research!" and go "no thanks, that seems kinda unsafe?" while also looking at "pandemic safety via research into rapidly redeployable mRNA vaccines!" and say "yeah that does sound good!"
Similarly, my reaction to almost all of the general "We will achieve AI Safety by this comprehensive list of American Laws" is that I'm not entirely impressed with how well they've performed with hundreds of other objectives, especially when the object we are attempting to impose Safety on is fast, smart and motivated by money. (drugs?)
"it's about the whole low-level culture of seeing some new technology and by default assuming it will kill us all and trying to strangle it in its crib, a culture which has plausibly caused a lot more problems than any of the technologies it's afraid of."
While I am 100% on board with that for literally everything other than nukes/viral-gene-editing/AI, I think you may be going too far with "strangle" and "by default assuming it will kill us all" - I think AI risk people agree that AI will have a vast range of positive benefits, they just think that it is possible that a sufficiently small % of AI might have incredibly negative harms, and that we might not be able to differentiate.
You are free to totally disagree, but I think instead of trying to prove who is right about "is AI going to cause really had things" we need to figure out how likely it actually is. By definition, AI doom scenarios are ones outside normal prediction from historic events, and therefore going to be hard to persuade people who make predictions based on historic events. The only solution is to generate simulated, contained historic events to enable us to make more accurate predictions from.
> "I certainly never see them talking about it, other than occasional throat-clearing before they start screaming WE'RE ALL GONNA DIE again."
Isn't part of the AI doom theory that the reason that we will create ever more powerful AIs (that eventually kill us) is that we'll be getting so many wonderful things from them that we'll wonder "what better things could a more powerful one get us?"
> "That would be convenient, but how are you going to do that without full AI?"
I think I mentioned this before, but simulated/virtual MMO game style worlds - let the AI doomers take a breath from screaming and design a game world with whatever factors/variables/physics they think would be a realistic test (remember we've all got millions of powerful GPUs sitting around!) and let whatever the latest-AI-model-is loose in it, running at faster-than-life speed. If it wrecks it, and keeps wrecking it no matter how many times we reset the simulation, or change the factors, then we'll have learned something.
Definitely! We should do way more experiments, including (especially) new experiments informed by the failures/successes of past simulation. But I think the part I like best is the scaled down/iterative nature of it: if AIs existing in the real world with 70 quadrillion parameters is going to discover a way to cause harm, I assume that the same AI, existing in a simulated world with only 7 million parameters (and sped up) is going to discover a way to cause simulated harm first. Then we pull the plug, rip out of the hard drive and figure out why it did that. Then design some alignment rule - or perhaps a non-generalist-AI - that prevents it, and try again. Even if you think "AI kills everyone" is unlikely, it would be a great test bed for mundane stuff too - create a Gran Turismo game-style world with lots of pedestrians to test your AI car-driving model.
I am actively exploring doing things like this, yet I am not fooling myself that too many people would be convinced - goalposts would simply be moved, again, no matter how clear people were before the results came in. Still seems worth doing.
I would like to see statements of the form 'if a virtual world meeting requirements XYZ gets taken over then I will change my mind.'
(It does not bear on the actual foom scenarios as much as one might hope, I still know what I expect to see.)
How will we set up MMOs which don't have obvious dominant strategies? Game (and systems) design is hard and avoiding this kind of foom is one of the tricky parts.
that's why my recommendation would be to hire the most prominent/creative AI doomers to design those rules - and encourage them (though I think they'd be motivated to do so already) to make the "test for foom" worlds as general/realistic as possible. After all, that's exactly their assertion: that the real world is (to a superintelligent AI) a game that has obvious (eventually to it) dominant strategies.
Yes, definitely, the first few will be dominated by relatively weak AIs exploiting simple strategies, glitches or bugs (like any game!) that should've been easily foreseen, but that is good! The ways in which an AI does that (and our after-action assessment of the decision path they took to get there) will inform not only future simulations, but our design of rules for AI.
Eventually the goal is to get to a simulation that has enough variables (since world variables are just numbers to an AI, real-world or simulated) that it roughly approximates the number of variables that the real world has - I know this sounds naive, but if recursive application of GPUs can create >human intelligence, then recursive application of GPUs can create a simulated world with enough variables to start giving us ideas about how well it simulates AI behavior in the real world.
Non-general, limited-AIs play a big role here, not only in designing the rules for the worlds, but monitoring the AIs being tested, and providing essentially "NPCs" for the AI to paperclip (or not).
I feel like this could be useful. The AI doesn't even have to "keep" wrecking the worlds, as long as it happens semi-regularly and in different enough ways. We can look at the ways it happened, abstract them into more general stories, brainstorm how AIs might do the same things in the real world, and brainstorm how to prevent those types of things from happening. With even a few good examples, a smart but unconvinced person should be able to extrapolate enough to think "huh, those were only the first cases we found by trial and error, and look how much work we'd have to do to prevent them. What about all the cases that we haven't found yet?"
yes, that's my hope as well - right now the terrain of the argument of "will AI cause bad things?" is too abstract for most people to make decisions about. We need to solidify it, both to persuade people to have more informed opinions, and to find out what exactly those opinions should be informed by.
I agree that the essay wasn't trying and failing to engage with the question on the specific merits, and that essays like this don't do that, it's not part of the model being presented. I don't... see why saying that is unfair? If you think that move is fine then there's no 'and that's terrible' at the end.
This isn't really complicated. We can even use the printing press analogy people like so much.
When we used the printing press, we reduced our ability to produced hand written manuscripts. We lost some beauty in the drawn artwork.
When we began to use typewriters and emails in earnest, we mostly have exported our ability to handwriting en masses.
Now, we embark on a journey to export our thinking to technology, and it seems like accelerationists do not conceive the very normal consequences of atrophy.
When you make something that emulates humans enough, but better(even if it is just cheaper), you won't need humans anymore.
Being that I am human and have human children, I do not support this
One particularly silly opinion which seems to be constantly advocated is "because this happened, therefore it is good." Its essentially the optimism fallacy which Candide had mocked years ago, but it seems incalculated.
Was it good, for example, that WW2 was waged? Perhaps it led to modern technology. Therefore it is good that the Holocaust occurred? There's a fundamental fallacy of accelerationists which is basically, "It happened and therefore it is good."
But this doesn't really tell you anything about the road /not/ taken. Maybe the world would have advanced without WW2, for example. Maybe we lost on a number of options by taking the path we have taken. Perhaps it would have been better.
The desire to chase everything because "it is a solution" now without considering the consequences(or alterrnatives), which really seems to be a kind of norm is exactly what got us into a lot of present day problems.
To some morbid extent, insofar as this "hyperbolic discounting" appears to be the norm, it feels like we deserve whatever we get, good and hard.
This is a very silly take and can be dismissed as easily as asking why it might be wrong to run in the traffic in order to get to work faster. The ability to conceive of alternatives and calculate risk, is in fact, one of the strongest aspects of being human.
I am a strong believer in NIMBY, and sincerely hope that the same logical principles come to regulate AI(which it does seem to be coming).
Your alternative appears to be the Optimism fallacy, which I have already noted as bunk(would advocate the Holocaust as perfectly reasonable since we got jet engines from it, and cannot conceive if any other way of getting jet engines).
Hey I just started reading this, but when I click on something in the table of contents in the app it takes me to a different page instead of staying within the same page, which is very annoying. Substack is usually good on technical stuff, I’d reach out to them on this.
"This is Zvi Mowshowitz, my blog itself is funded but if you fund me I will hire engineers at generous salaries to try out things and teach me things and build demonstration projects and investigate questions and other neat stuff like that, maybe commission a new virtual world for LLM agents to take over in various ways"
I support this, but I don't have "hiring engineers" kinds of money! What else can I do?
I mean, every little bit helps! If you want to contribute, Balsa Policy Institute can accept donations at a 501c3, and I am happy to create an AI-directed fund that will work on AI-related issues only.
If you're an engineer or otherwise have expertise in related topics you could do work without the generous salary, with whatever time you have, and that would be great.
If your primary resource isn't money or skilled time... tell me more about your situation, I guess? You can email me or Twitter DM me.
I had not heard of Duolingo Max. I will likely sign up for it the instant it becomes available for English Speakers learning German (I already use Duolingo). However, it doesn't have the one feature I think would be super useful: longer form reading that sticks primarily to the vocabulary Duolingo knows that you know. The Role Play could serve a similar function, but I think having content to consume that is just reading, is level/vocab appropriate, and is longer than their current "stories" feature would be great. Especially for _very_ early language learners, finding medium length content to consume is _extremely_ difficult. It seems like GPT 4 should be able to generate such content pretty trivially.
Oh wow, that's... insanely great? As in, the AI has a database of words that it knows you know, it uses only those words (Dr. Seuss style!) and over time you can introduce new words (ideally obvious from context, potentially with translation, etc). Because you have an LLM it can carry on discussions like that, cover news stories, anything you want. Someone should totally do that.
This would be a more flexible version of the "in easy X" stories (for X a language) which educational publishers sell for learners, about $10 per slim volume. Those follow a particular curriculum where everyone will know these 200 words in level 1, these 1000 in level 2, and so on. Would there be a significant gain in dispensing with the curriculum? I'm not so sure the benefit would be that big. More useful perhaps would be summaries of new text content every day: CliffsNotes or Reader's Digest for the NYT/CNN/Fox/BBC/Le Monde, targeted to various audiences or based on one's own demonstrated active vocabulary.
"It is still early. The key claim of Fan’s is that the problems of AutoGPT are inherent to GPT-4 and cannot be fixed with further wrapping. If we are getting close to the maximum amount we can get out of creating a framework and using reflection and memory and other tricks, then that seems rather fast. We are only doing some quite basic first things here. Perhaps the core engine simply is not up to the task, yet there are definitely things I would try well before I gave up on it."
Agreed that there is a lot more that can be done by wrapping things around GPT-4, and that it would be surprising if this did not yield some results. That said, for many values of "the task", I do think GPT-4 is probably not up to it. When building a complex workflow out of multiple LLM invocations, if any one of those invocations goes wrong (i.e. hallucinates or otherwise generates bad output), it's hard to avoid spoiling the entire workflow. Meanwhile at the planning level, it's easy to get caught in loops or other bad attractor states. I am somewhat skeptical that a "lightweight" layer on top of an LLM, such as AutoGPT on GPT-4, can robustly manage complex tasks in general. Of course there will always be some simpler cases where this is less of a problem, and some of those will be useful to automate.
FWIW, I recently gave some serious thought to the question of what it would take to stretch GPT-4 to the point where it could undertake more complex tasks, such as a nontrivial software engineering project. I wrote up the results here, if you're interested (and I'd love to hear any thoughts you might have): https://amistrongeryet.substack.com/p/can-ai-do-my-job. A few relevant snippets:
> When a problem is too difficult to solve in a single intuitive leap, you need to undertake a process of exploration. ... Exploration entails a mix of activities: generating new ideas, modifying and refining existing ideas, breaking a problem into subproblems, exploring the context (e.g. reading through existing code), gathering new data, asking for help. There are always an infinite number of potential next steps, so judgement is constantly needed: is it time to give up on an idea, or should I keep tinkering with it? What information would help me make a decision? Have I reached a dead end, do I need to push back on the project requirements? Is this result good enough, or is it worth the effort to optimize it further?
>
> ...There are many ways for exploration to go wrong, so another important skill is self-monitoring, noticing when you’ve fallen into a bad pattern and need to change course.
>
> ...Arguably, the ability to effectively, efficiently, and astutely explore and refine a complex idea is the fundamental difference between shallow and deep thinking, and one of the critical elements missing from current LLMs.
Cool, I'll check it out for potential inclusion next week.
Some of the problems seem like 'I don't know if you can 100% solve this, that sounds hard, yet I am pretty sure I know how to 50% solve it and keep going from there.' For example, loops - how hard is it to notice when you're in a loop?
Yes, this is one of the trillion dollar questions. I'm sure you can solve some of the problems some of the time. How far will that take us? We'll find out!
Presumably the mechanisms for this in human beings are highly tuned (though perhaps not for modern tasks such as programming) and deeply integrated with the rest of the brain, and yet we still struggle: rat-holing on a flawed approach, gradually drifting into spaghetti code because we don't stop to rethink / refactor, over-optimizing, etc. This makes me think it is a hard problem.
To clarify, there aren't enough GPUs in the world to meet current AI demand, that is definitely a high-order bit. But Azure is also going to have a lot more capacity come online as orders get delivered. The 12-16 months is paraphrasing a comment Scott Guthrie made on how like there's no fast twitch muscle that lets you wave a wand and have a new datacenter appear, that's the absolute minimum time it takes between going "oh we need a new building to put the chips in" and being able to have that ready.
I think the medium-long term utility of AI is very high, but I am really struggling to get good utility out of these models in their current form. GPT is very good at coding quick prototypes or interpreting APIs but loses usefulness (for me) after an hour or two into a months long process. It is also good at automating busywork like data whitespace adjustments or something, I don’t do that very much but maybe some people do. I’d say maybe I’ve gotten like a 5% productivity increase? I tried to get it to summarize too, but I think most of the stuff I read is either already pretty concise or you lose a lot of it if you turn it into bullet points.
Not to be a dick, but I notice that a lot of people saying they’re getting 3x or 10x out of the current tools have jobs where I could not describe what they actually do at their job aside from generating a lot of emails and meetings.
As a relatively terrible coder who struggles with basic questions, the 3x-10x range seems totally right for me if the alternative doesn't involve an open line to someone who can serve as a tutor or mentor - it's vastly superior to existing non-human sources of info, although much less good in many ways than I'd like.
For many people, that 1-2 hours is most of their lives, in an important sense!
I don't believe OpenAI or any other major player if they say they are pausing or even slowing down. I'm more likely to believe the opposite, that they are accelerating their efforts towards the next big thing in secret. There is an AI race now, and in a race it makes sense to use every trick to confuse and screw with your opponents, which is every other AI company and also those people that want slow progress or at least prefer it be done more thoughtfully. All the companies involve will lie if they think it can gain them an advantage, and they face no real penalties for doing so.
Except, it's clear OpenAI have decided to change focus. When you are already training on the whole decent quality part of the Internet, going bigger isn't gaining you much (and might even reduce performance). This doesn't mean they have stopped working: Altman explicitly says they are improving their supporting tech, and that they are exploring different methods (perhaps things like adding Wolfram Alpha to the system, or a Python or Lisp interpreter).
Re #7: They Took Our Jobs: I've looked at different studies and they call out various occupations that will be impacted and by how much. Of course, A) it's too early to have a lot of confidence in these, B) there will be positive as well as negative ones and C) as you say, some jobs will get better while others will be eliminated (like with globalization).
That said, I think it's pretty clear women will be impacted more than men simply because a higher percentage of working women are in white collar jobs (~70%) vs. blue collar ones (~30%) vs men, where the ratio is roughly 50/50. This was intuitive to me but i did run the following analysis on the Goldman Sachs study as a test.
Thanks as always, best content around. Driverless cars: is a component of the problem how to rapidly make *most* cars driverless, which will then presumably make being a driverless car easier? Or perhaps how to make all cars able to communicate useful data about speed, direction, etc. etc. to each other which can be used to overrule the humans? My own car, newish VW Tiguan, does a surprising amount of overruling me: lane change protection when I want to change lane, automatic braking if it thinks it’s going to hit something *even when it is not!*, etc.) Feels like something manufacturers are unlikely to do unless incentivised, perhaps by the promise of lots of driverless subscriptions / subsidy handouts. Maybe linked to electrification too.
Lane change: steering wheel resists you crossing a laneunless you signal / apply more pressure. Mildly annoying, but does reinforce ‘good behaviour’. Can be turned off deep in settings. Automatic braking: slams on the parking brake during low speed manoeuvres if car perceives a likely collision. Have experienced it three times in ~2 years, confident it was a false positive (dirt / ice on sensor). Also can be turned off. More likely to cause an accident. I believe there is also automatic emergency stop, but haven’t triggered it and am not keen to try!
Aaronson strikes me as someone who does what he feels like doing then finds post-hoc rationalisations for why what he feels like doing is ok. (I mean, more than average.) He also definitely wants to see general AI in his lifetime more than he wants to make sure it doesn't kill anyone.
When I read his blog I imagine him as a cross between Dennis Nedry and Bishop.
It should be noted that the FDA does not certify anything. They "clear" medical devices and they "approve" drugs, but they do not do any independent assessments of the products they review. Self-audit amounts to self-assurance, which is not safe or reliable. "Certifications" are statements of assurance validated and verified by an independent third-party audit. Presumably, in China self-audit doesn't mean quite the same thing it would mean in the West because the CCP is so deeply embedded in every company in the country, so in this context it may mean something more like "internal political review".
I did not intend to make a claim as strong as 'We can now go ahead and burn all our libraries without worrying.' I'm not saying China de facto doesn't exist, more that we need to stop treating it as a black box defect-bot that is only two steps behind us, running as fast as they can and almost as fast as we can, and can't be reasoned with, as a fully universal boogeyman. Or anything in the same state as that.
I do agree that if we banned AI forever, after some amount of time someone else would, if not stopped, surpass us, although I doubt it would even then be China at this rate.
I think the good take-away from the "compare AI safety to viral gain-of-function research" analogy (or to any number of other regulations) that everyone can agree is: no matter what people/governments CLAIM is "safety-improving", normal people need to aggressively audit what is actually being done and ensure that it actually does, and continues to do so in the future.
I don't think it's crazy to think that normal people, if exposed to the information, even pre 2020, would look at "pandemic safety via gain of function research!" and go "no thanks, that seems kinda unsafe?" while also looking at "pandemic safety via research into rapidly redeployable mRNA vaccines!" and say "yeah that does sound good!"
Similarly, my reaction to almost all of the general "We will achieve AI Safety by this comprehensive list of American Laws" is that I'm not entirely impressed with how well they've performed with hundreds of other objectives, especially when the object we are attempting to impose Safety on is fast, smart and motivated by money. (drugs?)
"it's about the whole low-level culture of seeing some new technology and by default assuming it will kill us all and trying to strangle it in its crib, a culture which has plausibly caused a lot more problems than any of the technologies it's afraid of."
While I am 100% on board with that for literally everything other than nukes/viral-gene-editing/AI, I think you may be going too far with "strangle" and "by default assuming it will kill us all" - I think AI risk people agree that AI will have a vast range of positive benefits, they just think that it is possible that a sufficiently small % of AI might have incredibly negative harms, and that we might not be able to differentiate.
You are free to totally disagree, but I think instead of trying to prove who is right about "is AI going to cause really had things" we need to figure out how likely it actually is. By definition, AI doom scenarios are ones outside normal prediction from historic events, and therefore going to be hard to persuade people who make predictions based on historic events. The only solution is to generate simulated, contained historic events to enable us to make more accurate predictions from.
> "I certainly never see them talking about it, other than occasional throat-clearing before they start screaming WE'RE ALL GONNA DIE again."
Isn't part of the AI doom theory that the reason that we will create ever more powerful AIs (that eventually kill us) is that we'll be getting so many wonderful things from them that we'll wonder "what better things could a more powerful one get us?"
> "That would be convenient, but how are you going to do that without full AI?"
I think I mentioned this before, but simulated/virtual MMO game style worlds - let the AI doomers take a breath from screaming and design a game world with whatever factors/variables/physics they think would be a realistic test (remember we've all got millions of powerful GPUs sitting around!) and let whatever the latest-AI-model-is loose in it, running at faster-than-life speed. If it wrecks it, and keeps wrecking it no matter how many times we reset the simulation, or change the factors, then we'll have learned something.
Definitely! We should do way more experiments, including (especially) new experiments informed by the failures/successes of past simulation. But I think the part I like best is the scaled down/iterative nature of it: if AIs existing in the real world with 70 quadrillion parameters is going to discover a way to cause harm, I assume that the same AI, existing in a simulated world with only 7 million parameters (and sped up) is going to discover a way to cause simulated harm first. Then we pull the plug, rip out of the hard drive and figure out why it did that. Then design some alignment rule - or perhaps a non-generalist-AI - that prevents it, and try again. Even if you think "AI kills everyone" is unlikely, it would be a great test bed for mundane stuff too - create a Gran Turismo game-style world with lots of pedestrians to test your AI car-driving model.
I am actively exploring doing things like this, yet I am not fooling myself that too many people would be convinced - goalposts would simply be moved, again, no matter how clear people were before the results came in. Still seems worth doing.
I would like to see statements of the form 'if a virtual world meeting requirements XYZ gets taken over then I will change my mind.'
(It does not bear on the actual foom scenarios as much as one might hope, I still know what I expect to see.)
How will we set up MMOs which don't have obvious dominant strategies? Game (and systems) design is hard and avoiding this kind of foom is one of the tricky parts.
that's why my recommendation would be to hire the most prominent/creative AI doomers to design those rules - and encourage them (though I think they'd be motivated to do so already) to make the "test for foom" worlds as general/realistic as possible. After all, that's exactly their assertion: that the real world is (to a superintelligent AI) a game that has obvious (eventually to it) dominant strategies.
Yes, definitely, the first few will be dominated by relatively weak AIs exploiting simple strategies, glitches or bugs (like any game!) that should've been easily foreseen, but that is good! The ways in which an AI does that (and our after-action assessment of the decision path they took to get there) will inform not only future simulations, but our design of rules for AI.
Eventually the goal is to get to a simulation that has enough variables (since world variables are just numbers to an AI, real-world or simulated) that it roughly approximates the number of variables that the real world has - I know this sounds naive, but if recursive application of GPUs can create >human intelligence, then recursive application of GPUs can create a simulated world with enough variables to start giving us ideas about how well it simulates AI behavior in the real world.
Non-general, limited-AIs play a big role here, not only in designing the rules for the worlds, but monitoring the AIs being tested, and providing essentially "NPCs" for the AI to paperclip (or not).
I feel like this could be useful. The AI doesn't even have to "keep" wrecking the worlds, as long as it happens semi-regularly and in different enough ways. We can look at the ways it happened, abstract them into more general stories, brainstorm how AIs might do the same things in the real world, and brainstorm how to prevent those types of things from happening. With even a few good examples, a smart but unconvinced person should be able to extrapolate enough to think "huh, those were only the first cases we found by trial and error, and look how much work we'd have to do to prevent them. What about all the cases that we haven't found yet?"
At least, that'd be my hope.
yes, that's my hope as well - right now the terrain of the argument of "will AI cause bad things?" is too abstract for most people to make decisions about. We need to solidify it, both to persuade people to have more informed opinions, and to find out what exactly those opinions should be informed by.
I agree that the essay wasn't trying and failing to engage with the question on the specific merits, and that essays like this don't do that, it's not part of the model being presented. I don't... see why saying that is unfair? If you think that move is fine then there's no 'and that's terrible' at the end.
I am okay with strangling anything that will disempower humans in its crib, and the fact that it will probably disempower humans is almost undeniable.
This isn't really complicated. We can even use the printing press analogy people like so much.
When we used the printing press, we reduced our ability to produced hand written manuscripts. We lost some beauty in the drawn artwork.
When we began to use typewriters and emails in earnest, we mostly have exported our ability to handwriting en masses.
Now, we embark on a journey to export our thinking to technology, and it seems like accelerationists do not conceive the very normal consequences of atrophy.
When you make something that emulates humans enough, but better(even if it is just cheaper), you won't need humans anymore.
Being that I am human and have human children, I do not support this
It seems that your eagerness to export your thinking has already begun without the help of GAI.
One particularly silly opinion which seems to be constantly advocated is "because this happened, therefore it is good." Its essentially the optimism fallacy which Candide had mocked years ago, but it seems incalculated.
Was it good, for example, that WW2 was waged? Perhaps it led to modern technology. Therefore it is good that the Holocaust occurred? There's a fundamental fallacy of accelerationists which is basically, "It happened and therefore it is good."
But this doesn't really tell you anything about the road /not/ taken. Maybe the world would have advanced without WW2, for example. Maybe we lost on a number of options by taking the path we have taken. Perhaps it would have been better.
The desire to chase everything because "it is a solution" now without considering the consequences(or alterrnatives), which really seems to be a kind of norm is exactly what got us into a lot of present day problems.
To some morbid extent, insofar as this "hyperbolic discounting" appears to be the norm, it feels like we deserve whatever we get, good and hard.
This is a very silly take and can be dismissed as easily as asking why it might be wrong to run in the traffic in order to get to work faster. The ability to conceive of alternatives and calculate risk, is in fact, one of the strongest aspects of being human.
I am a strong believer in NIMBY, and sincerely hope that the same logical principles come to regulate AI(which it does seem to be coming).
Your alternative appears to be the Optimism fallacy, which I have already noted as bunk(would advocate the Holocaust as perfectly reasonable since we got jet engines from it, and cannot conceive if any other way of getting jet engines).
Hey I just started reading this, but when I click on something in the table of contents in the app it takes me to a different page instead of staying within the same page, which is very annoying. Substack is usually good on technical stuff, I’d reach out to them on this.
I reported this to Substack, got back that this is a known issue with the app, they are working on it and there is nothing an author can do.
"This is Zvi Mowshowitz, my blog itself is funded but if you fund me I will hire engineers at generous salaries to try out things and teach me things and build demonstration projects and investigate questions and other neat stuff like that, maybe commission a new virtual world for LLM agents to take over in various ways"
I support this, but I don't have "hiring engineers" kinds of money! What else can I do?
I mean, every little bit helps! If you want to contribute, Balsa Policy Institute can accept donations at a 501c3, and I am happy to create an AI-directed fund that will work on AI-related issues only.
If you're an engineer or otherwise have expertise in related topics you could do work without the generous salary, with whatever time you have, and that would be great.
If your primary resource isn't money or skilled time... tell me more about your situation, I guess? You can email me or Twitter DM me.
On the Larry Page thing; the more I see the ways some of the central figures talk about AI, the more I think these people are completely insane.
The link to your podcast appearance appears to be broken:
http://the%20week%20in%20podcasts/
Damn! Good catch. New link: https://www.youtube.com/watch?v=c9vCTTGfeHk&pp=ygUWenZpIG1vd3Nob3dpdHogcG9kY2FzdA%3D%3D
Well, I wanted to listen to it. Thanks!
I had not heard of Duolingo Max. I will likely sign up for it the instant it becomes available for English Speakers learning German (I already use Duolingo). However, it doesn't have the one feature I think would be super useful: longer form reading that sticks primarily to the vocabulary Duolingo knows that you know. The Role Play could serve a similar function, but I think having content to consume that is just reading, is level/vocab appropriate, and is longer than their current "stories" feature would be great. Especially for _very_ early language learners, finding medium length content to consume is _extremely_ difficult. It seems like GPT 4 should be able to generate such content pretty trivially.
Oh wow, that's... insanely great? As in, the AI has a database of words that it knows you know, it uses only those words (Dr. Seuss style!) and over time you can introduce new words (ideally obvious from context, potentially with translation, etc). Because you have an LLM it can carry on discussions like that, cover news stories, anything you want. Someone should totally do that.
This would be a more flexible version of the "in easy X" stories (for X a language) which educational publishers sell for learners, about $10 per slim volume. Those follow a particular curriculum where everyone will know these 200 words in level 1, these 1000 in level 2, and so on. Would there be a significant gain in dispensing with the curriculum? I'm not so sure the benefit would be that big. More useful perhaps would be summaries of new text content every day: CliffsNotes or Reader's Digest for the NYT/CNN/Fox/BBC/Le Monde, targeted to various audiences or based on one's own demonstrated active vocabulary.
"It is still early. The key claim of Fan’s is that the problems of AutoGPT are inherent to GPT-4 and cannot be fixed with further wrapping. If we are getting close to the maximum amount we can get out of creating a framework and using reflection and memory and other tricks, then that seems rather fast. We are only doing some quite basic first things here. Perhaps the core engine simply is not up to the task, yet there are definitely things I would try well before I gave up on it."
Agreed that there is a lot more that can be done by wrapping things around GPT-4, and that it would be surprising if this did not yield some results. That said, for many values of "the task", I do think GPT-4 is probably not up to it. When building a complex workflow out of multiple LLM invocations, if any one of those invocations goes wrong (i.e. hallucinates or otherwise generates bad output), it's hard to avoid spoiling the entire workflow. Meanwhile at the planning level, it's easy to get caught in loops or other bad attractor states. I am somewhat skeptical that a "lightweight" layer on top of an LLM, such as AutoGPT on GPT-4, can robustly manage complex tasks in general. Of course there will always be some simpler cases where this is less of a problem, and some of those will be useful to automate.
FWIW, I recently gave some serious thought to the question of what it would take to stretch GPT-4 to the point where it could undertake more complex tasks, such as a nontrivial software engineering project. I wrote up the results here, if you're interested (and I'd love to hear any thoughts you might have): https://amistrongeryet.substack.com/p/can-ai-do-my-job. A few relevant snippets:
> When a problem is too difficult to solve in a single intuitive leap, you need to undertake a process of exploration. ... Exploration entails a mix of activities: generating new ideas, modifying and refining existing ideas, breaking a problem into subproblems, exploring the context (e.g. reading through existing code), gathering new data, asking for help. There are always an infinite number of potential next steps, so judgement is constantly needed: is it time to give up on an idea, or should I keep tinkering with it? What information would help me make a decision? Have I reached a dead end, do I need to push back on the project requirements? Is this result good enough, or is it worth the effort to optimize it further?
>
> ...There are many ways for exploration to go wrong, so another important skill is self-monitoring, noticing when you’ve fallen into a bad pattern and need to change course.
>
> ...Arguably, the ability to effectively, efficiently, and astutely explore and refine a complex idea is the fundamental difference between shallow and deep thinking, and one of the critical elements missing from current LLMs.
Cool, I'll check it out for potential inclusion next week.
Some of the problems seem like 'I don't know if you can 100% solve this, that sounds hard, yet I am pretty sure I know how to 50% solve it and keep going from there.' For example, loops - how hard is it to notice when you're in a loop?
Yes, this is one of the trillion dollar questions. I'm sure you can solve some of the problems some of the time. How far will that take us? We'll find out!
Presumably the mechanisms for this in human beings are highly tuned (though perhaps not for modern tasks such as programming) and deeply integrated with the rest of the brain, and yet we still struggle: rat-holing on a flawed approach, gradually drifting into spaghetti code because we don't stop to rethink / refactor, over-optimizing, etc. This makes me think it is a hard problem.
Loop detection means memory or some ability to mark the environment. This is an active area of research.
To clarify, there aren't enough GPUs in the world to meet current AI demand, that is definitely a high-order bit. But Azure is also going to have a lot more capacity come online as orders get delivered. The 12-16 months is paraphrasing a comment Scott Guthrie made on how like there's no fast twitch muscle that lets you wave a wand and have a new datacenter appear, that's the absolute minimum time it takes between going "oh we need a new building to put the chips in" and being able to have that ready.
Why do you believe anything China says?
I think the medium-long term utility of AI is very high, but I am really struggling to get good utility out of these models in their current form. GPT is very good at coding quick prototypes or interpreting APIs but loses usefulness (for me) after an hour or two into a months long process. It is also good at automating busywork like data whitespace adjustments or something, I don’t do that very much but maybe some people do. I’d say maybe I’ve gotten like a 5% productivity increase? I tried to get it to summarize too, but I think most of the stuff I read is either already pretty concise or you lose a lot of it if you turn it into bullet points.
Not to be a dick, but I notice that a lot of people saying they’re getting 3x or 10x out of the current tools have jobs where I could not describe what they actually do at their job aside from generating a lot of emails and meetings.
As a relatively terrible coder who struggles with basic questions, the 3x-10x range seems totally right for me if the alternative doesn't involve an open line to someone who can serve as a tutor or mentor - it's vastly superior to existing non-human sources of info, although much less good in many ways than I'd like.
For many people, that 1-2 hours is most of their lives, in an important sense!
I don't believe OpenAI or any other major player if they say they are pausing or even slowing down. I'm more likely to believe the opposite, that they are accelerating their efforts towards the next big thing in secret. There is an AI race now, and in a race it makes sense to use every trick to confuse and screw with your opponents, which is every other AI company and also those people that want slow progress or at least prefer it be done more thoughtfully. All the companies involve will lie if they think it can gain them an advantage, and they face no real penalties for doing so.
Except, it's clear OpenAI have decided to change focus. When you are already training on the whole decent quality part of the Internet, going bigger isn't gaining you much (and might even reduce performance). This doesn't mean they have stopped working: Altman explicitly says they are improving their supporting tech, and that they are exploring different methods (perhaps things like adding Wolfram Alpha to the system, or a Python or Lisp interpreter).
Re #7: They Took Our Jobs: I've looked at different studies and they call out various occupations that will be impacted and by how much. Of course, A) it's too early to have a lot of confidence in these, B) there will be positive as well as negative ones and C) as you say, some jobs will get better while others will be eliminated (like with globalization).
That said, I think it's pretty clear women will be impacted more than men simply because a higher percentage of working women are in white collar jobs (~70%) vs. blue collar ones (~30%) vs men, where the ratio is roughly 50/50. This was intuitive to me but i did run the following analysis on the Goldman Sachs study as a test.
https://kenaninstitute.unc.edu/kenan-insight/will-generative-ai-disproportionately-affect-the-jobs-of-women/?
Thanks as always, best content around. Driverless cars: is a component of the problem how to rapidly make *most* cars driverless, which will then presumably make being a driverless car easier? Or perhaps how to make all cars able to communicate useful data about speed, direction, etc. etc. to each other which can be used to overrule the humans? My own car, newish VW Tiguan, does a surprising amount of overruling me: lane change protection when I want to change lane, automatic braking if it thinks it’s going to hit something *even when it is not!*, etc.) Feels like something manufacturers are unlikely to do unless incentivised, perhaps by the promise of lots of driverless subscriptions / subsidy handouts. Maybe linked to electrification too.
Those features sound super annoying and potentially accident causing. What is your experience with them?
Lane change: steering wheel resists you crossing a laneunless you signal / apply more pressure. Mildly annoying, but does reinforce ‘good behaviour’. Can be turned off deep in settings. Automatic braking: slams on the parking brake during low speed manoeuvres if car perceives a likely collision. Have experienced it three times in ~2 years, confident it was a false positive (dirt / ice on sensor). Also can be turned off. More likely to cause an accident. I believe there is also automatic emergency stop, but haven’t triggered it and am not keen to try!
Aaronson strikes me as someone who does what he feels like doing then finds post-hoc rationalisations for why what he feels like doing is ok. (I mean, more than average.) He also definitely wants to see general AI in his lifetime more than he wants to make sure it doesn't kill anyone.
When I read his blog I imagine him as a cross between Dennis Nedry and Bishop.
It should be noted that the FDA does not certify anything. They "clear" medical devices and they "approve" drugs, but they do not do any independent assessments of the products they review. Self-audit amounts to self-assurance, which is not safe or reliable. "Certifications" are statements of assurance validated and verified by an independent third-party audit. Presumably, in China self-audit doesn't mean quite the same thing it would mean in the West because the CCP is so deeply embedded in every company in the country, so in this context it may mean something more like "internal political review".