As usual all these discussions are made from the assumption that being on a wage of $100,000+ per year is the "normal".
The real issue of worker vs. AI is that a large percentage of human workers worldwide cost $300 per year and most cost less than $3,000 per year and it will be very hard for AIs and robots to be cost competitive with those humans especially for tasks requiring flexibility. Too bad that a lot of people in California will find it unpleasant to become so cost competitive.
«it’s not that the AI will hate us, it’s that it will see us like we see ants, and if you want to put a highway where the anthill is that’s the ant’s problem.»
If they become ASIs will likely see us as we see the birds in the sky or the fishes in the rivers; they may even create natural reserves for humans as we do for spotted owls...
Actually more likely the ASIs will see us as we see goats and dogs and will domesticate us and use us as remote hands/cheap robots, or amusing pets, just as we use goats and dogs and oxen and horses as cheap robots. After all at a wage of $300 per year not many robots will be as costs effective as humans, and many humans today in California spend way more than $300 per year on their dogs and cats etc.
Note: the above scenarios have been illustrated in various Sci-Fi novels wrt to ASIs or super-intelligent aliens.
"a large percentage of human workers worldwide cost $300 per year and most cost less than $3,000 per year [...] a lot of people in California will find it unpleasant to become so cost competitive."
For the large part of humanity who are already on $300/year or less than $3,000/year it will be largely "here is the new boss, same as the old boss" as it will make little difference to them whether their ultimate bosses will be Altman or ChatGPT version 21.3 or whether they will be cleaning the rooms of his house or the aisles of its datacenter.
AI inference has been getting more and more cost-efficient. So far that has been soaked up in getting it to do more ambitious things, but now that it's getting near enough to human level, we may change to a regime of harvesting the cost savings. Cost arguments ignore trends.
At $300 per year you are sleeping on a dirt floor and you don't have indoor sanitation or running water. I'd kind of hope for better things.
If you are saying "but the AIs will provide habitations and utilities and education and healthcare," implicitly you are up in the thousands per year per human, best case. Also you need to allow for the fact that maximum utilisation of humans is about a quarter of a year per year; AI instances can run near 90% in all likelihood. So four humans or one AI.
«AI inference has been getting more and more cost-efficient. So far that has been soaked up in getting it to do more ambitious things but now that it's getting near enough to human level»
That is rather optimistic...
Regardless: if you are expecting that AIs will soon have human level intellectual and physical abilities (just AGI, not ASI) and will cost less than $300 per year to run then that is the "peak horse" ("peak human") case. That does not mean that humans will become extinct, any more than the sparrows and the trouts and the moles have become extinct; they will just have to be self-employed in the sense in which amazonian or kahalarian natives are self-employed, and forage for food in wild areas or do "gleaning" in the fields owned by the human or AI oligarchs; something that has been fairly common over the past several dozen thousand years.
«At $300 per year you are sleeping on a dirt floor and you don't have indoor sanitation or running water. I'd kind of hope for better things.»
And who is going to pay for the "better things"? The AI oligarchs who took over from the human oligarchs? The human oligarchs who remained in control? Why would they?
Look at the world today: there are perhaps 800m people living on less than $2 per day and «sleeping on a dirt floor and you don't have indoor sanitation or running water» and not many people are eager to gift them a chunk of their income to pay for them to have a better living standard.
I don’t really understand this. If the thousands of people in Bend OR are suddenly unable to have proper houses, wouldn’t they just build houses for themselves and others? Why wouldn’t a new economy outside the AIs bootstrap itself into existence?
It is not necessarily "outside the AIs": if ASIs happens but the human oligarchs keep control of the ASIs, it will still be a human-driven economy, just with a lot fewer human employees.
«bootstrap itself into existence?»
It will indeed; but all the valuable resources (oil, fertile land, mines, factories, ...) already belong to someone, and the members of the "new economy" being "self employed" would not be able to buy a supply of resources from those owners. That type of economy has existed for many thousands of years in the west too, for example:
Unless you imagine that the owners of resources and the owners of AIs would willingly gift some kind of "Universal Basic Income" to a lot of cost-uncompetitive humans, or those humans would be able to take those resources without paying. Both are of course un-american situations.
Missing my point. From the point of view of a business owner, AI is not competing with $300/year, it is competing with $3000 - $15000 per year, at least. And very long lead times for humans.
I did not mention the 'burden' costs of employing humans. Recruitment, training, furniture, space, light, heat and cooling, inability to instantly scale up or down according to need. Humans can do construction labouring on a day-labour basis. They can not be neurosurgeons, military pilots, or power plant salesmen when hired in that way. Nor even medical transcriptionists and court stenographers. AI can.
How did you get the number of $300 a year? Worldwide median household income was $9,733 (PPP) in 2013 according to Gallup.
(And the really low wages are all in developing countries like Malawi, which are not major contributors to the knowledge work most threatened by AI)
You're assuming that humans and AIs do the same work. If I have some boxes to move, and my choices are 1) hire a forklift for $310/wk 2) hire a human for $300/wk, is the human competitive? Not really, because a forklift does far more (and harder) work than a human. What if I need to move 2000kg packing crates? Then I *have* to hire a forklift. The human would be uncompetitive at a wage of even $1/wk.
I have no idea how future ASIs might regard or value us. I don't want to live in a zoo.
«Not really, because a forklift does far more (and harder) work than a human.»
We use forklifts instead of horse driven carts because the food for horses or oxen is far more expensive than the fuel or electricity for forklifts. Therefore "peak horse". If you expect that AGI-level AIs and "robots" will be more cost effective in the same sense then all the current investment in AI is explained and we have "peak human".
«I have no idea how future ASIs might regard or value us»
Then problem is not ASIs as such, it is employers, whether human or AI. Why should a human employers value a cost-ineffective employee more than an ASI and pay them a wage substantially higher than the cost of running an equivalent AI?
«I don't want to live in a zoo.»
Most of today's birds, squirrels, fishes, do not live in a zoo, they are "self employed" in the wild. I guess that humans "self employed" in marginal ecological niches (deserts, the arctic, high mountains, isolated islands) will continue to exist and live free regardless of whether the ASI oligarchs replace the human oligarchs and whether the AGI employees are more cost-efficient than human employees.
“Despite economic decline, inflation, and capital outflows, the Eastern Industry Park is still one of the most successful parks in the country. Workers there earn monthly wages of 3,000 [$19.26] to 10,000 [$64.20] birr, significantly higher than the average wage of less than 1,000 [$6.42] birr outside the park.”
International employers (and domestic exporters) pay wages at the nominal exchange rate, not the PPP one.
Does surviving on a very low wage also require the consumption of food produced for very low cost from land of fairly low value? Because at some point there may also be more efficient things to do with that land than grow human-edible crops.
Can you find a script writer for such a scenario…lol well, speaking, while resisting to joke, my tiny global religion has an article of faith that mankind will survive and thrive. But this doesn’t mean everyone will survive and everyone will thrive. We also believe in an optimistic future, eventually. My concerns about AI have taken second place to what
I believe has become a greater near term threat to mankind’s current status quo. This is the climate crisis. Of course we’ll need to find an AI alliance to get our civilization out of the conundrum. This doesn’t seem unachievable. I expect our eventual great grandchildren to deal with it. Yes, I have my regrets. But I did the best I could. Our family, though living in Southern California, owning 5 cars, sold all of them and moved to Japan where I’ve taught English until retirement last year. I joined a grassroots protest and lawsuit over the conversion of what Prince Chichibu in 1923 called ‘the People’s Park’ to a multimillion dollar multi use development to benefit the current Tokyo’s governor and her deep pockets construction industry supporters. Our family has always separated our weekly garbage into 12 categories which the city recombines into a few because of budget and technology limitations. And we fly back to the States only every couple of years. I keep a low thermostat in winter and a high one in summer. So I really do use a hand fan on the trains. I’ve tried to walk the talk of environmental protection. But we’re all too. Little too late. I will bend my knee and accept whatever consequences our Creator gave me when He dealt my cards.
Correct, my strategy for policy advocacy is to try and sidle them towards eyeballs who would share links with defense think tank people who would then refine ideas into forms palatable for consumption by DIA analysts and so on. Technically the job of the NatSec establishment is to keep humans alive, they need a model where AI take-off risk is something to factor against.
"Technically the job of the NatSec establishment is to keep humans alive"
Or maybe to use violence to make their "sponsors" richer than their rivals as Smedley Butler wrote: "I spent 33 years and four months in active military service and during that period I spent most of my time as a high class muscle man for Big Business, for Wall Street and the bankers"
Yes of course, and yet they need live populations to pay taxes to fund their budgets. I suppose we could have a post-humanist regime but that's not quite what this accelerationist regime is.
"to use violence to make their "sponsors" richer than their rivals"
«yet they need live populations to pay taxes to fund their budgets. I suppose we could have a post-humanist regime»
Agreed but my purpose was to point at the "sponsors": the “NatSec establishment” is not yet like the Praetorian Guard that appointed emperors, they have some autonomy, but ultimately they follow the collective will of their "sponsors" (“Big Business, for Wall Street and the bankers” and tech).
I have quite a different perspective from most writers as to ML/AI, an "economicist" one:
* The "AI"s can become employees or oligarchs.
* If the human oligarchs of today retain control over the AIs the AIs will be just employees.
* If the AIs manage to shaft the human oligarchs of today the AIs will become oligarchs and "sponsor" the “NatSec establishment” themselves.
* In either case the big question is what happens to the human employees, the 80% or the 99%, but that is *independent* of whether the oligarchs are human or AI.
What can happen to the human employees depends on whether they are more or less cost effective than the AIs *as employees*. Consider the famous case of "peak horse":
“American farms employed 26,493,000 equines in 1915. One hundred years later, the number of such animals on the payroll had collapsed to 700,000 [...] Horses [...] did not see the writing on the barn wall and start applying for factory jobs. They didn’t learn to code or attend community college. They stood there and ate carrots”
We now have 300m cars that have replaced 25m horses as they are far more cost-effective as petrol and electricity are far cheaper than hay and oats as sources of calories.
I can imagine three likely cases:
* Human employees at an appropriate wage level (perhaps lower than the current worldwide typical wage of $2,000-$3,000 per year) are more cost competitive than AIs at most jobs and then most jobs will be done by humans at that wage level.
* Human employees at most wage levels at or above subsistence are less cost-competitive than AIs at most jobs and the AI or human oligarchs are not willing to be confiscated enough of their profits to feed and reproduce them, and then slowly "peak human" happens.
* Human employees at most wage levels at or above subsistence are less cost-competitive than AIs at most jobs and AI or human oligarchs are willing to have a majority of their profits confiscated to give small welfare payments to all the unemployed human.
The colossal investment in AI currently seems based entirely on the expectation that it will not happen that human and AI employees will be best at different jobs and therefore many human employees in California will continue to get $100,000+ wages.
Look, having children CAN BE awesome. But the opportunity costs are massive unless one already has generational wealth or an extremely robust social support network. You are also generating "fat tails" in your hedonic distribution: child graduates Summa Cum Laude from a Ivy League school in a tough discipline: a lot of hedons. Child is forcibly inserted into the Eternal Torment Nexus by the AGI DEMON GOD... less hedons to put it mildly.
Sorry, I got triggered by that. Good engagement bait.
I'm not short the market. People should have the children they can reasonably afford. We should make raising children more affordable and higher status. There's isn't going to be an AI demon god, although there are a bunch of more subtle ways things could go somewhere between Badly and Catastrophically Badly. Eliezer is right about more stuff than the mean observer of this debate is willing to concede.
"the final decisions will continue to be made by the national security establishment"
Tyler, as usual, is correct BUT context is that which is scarce: WHO exactly comprises "the national security establishment" and what "national security" refers to might undergo extreme revisions in the very near future.
I think you have the opportunity cost backwards for children, the opportunity cost of not having children most with children would say saps your life of a lot of its meaning. If you try to measure the decision to have children as hedonically "I have to spent more time cleaning poopy diapers than going skiing" then yes children will always lose out, but that's not really the point.
Listen man, I like to do crazy shit and having to Not Do Crazy shit to make sure I'm not accidentally causing an externality for my offspring would definitely put a damper on my general life vibe. Not everyone is like me, and for those people Mazel Tov, go crank out the copies.
Suppose AI does destroy the world shortly, before your hypothetical child turns into an adult. You could in theory have the child, not be all neurotic about it, spoil them rotten, do fun stuff with them, do fun stuff without them, whatever you want.
If they don't live to adulthood there is no externality. Nobody expects much from children while they are children.
>The title is correct. This is about ‘how I became an Apolcaloptimist,’ and why he wanted to be that, as opposed to an argument for apocaloptimism being accurate. The larger Straussian message, contra Tyler Cowen, is not ‘the interventions are fake’ but that ‘so many choose to believe false things about AI, in order to feel that things will be okay.’
Hmm... Would those who find both P(doom)<0.01 and P(doom)>0.99 to lack sufficient evidence be Apocalstocasticists? :-)
Louis Antoine Leon de Saint-Just, a radical French revolutionary, close political associate of Robespierre, and supporter of the Terror, once said, "The present order is the disorder of the future." Both men were executed by guillotine on July 28, 1794, following their arrests on the night of 9th Thermidor (July 27). The quote, "The present order is the disorder of the future," resonates with Finlay's vision for Little Sparta, suggesting that art and revolution continually reshape society and challenge our perceptions. Today, Saint-Just's quote is prominently featured at Little Sparta, a celebrated garden in the Pentland Hills of Scotland designed by artist Ian Hamilton Finlay. Finlay's integration of revolutionary ideas into the garden's landscape is seen by some as foreshadowing the philosophical challenges posed by synthetic superintelligence, which similarly disrupts established orders. The garden thus serves as a unique tribute linking revolutionary history, contemporary art, and the evolving discourse on technological transformation. By weaving together historical events, artistic expression, and modern themes, Little Sparta invites visitors to consider how radical shifts—whether political, artistic, or technological—can redefine society and our understanding of order and disorder.
that tracks — it's basically convergent linguistic evolution. the concept is sticky enough that people keep arriving at the same word independently, which is kind of a sign the word deserves to exist. do you think the meanings converge too, or do different coiners end up with genuinely different spins on it?
> That is almost entirely the argument. Tech was good before, so tech will be good now.
This is depressing. I have yet to get a satisfactory answer to "but this time, we are the horses. Or, as time goes by, the ants." Daniel didn't get one either.
I’ve been a subscriber for about a year or two. I recently retired completely from teaching English here in Saitama, Consequently I’m a regular user of ChatGPT to help me create a historical fantasy screenplay set in one of the valley corridors of Afghanistan near the upper Indus Valley about 1 week’s journey by camel from Samarkand In the 13th to early 14th century.
Elon being too busy/forgetting to show up, and Zuck declining to show at all: yes, these are exactly playing to type, well done, highly apropos of the labs they run as well. Head rots from the fish, or however the saying goes.
Accs not sending their best arguments to a friendly mic is...weird? There are better ones (arguments at all) to be made, it's been done on this very blog in fact. But perhaps that's less photogenic, or risks incurring the wrath of Mr. Sacks (getting endorsements from both sides of the debate is, in fact, not a small feat in current_year, for many values of debate), or it goes against the general modus operandi of hyperstitioning. You can't shout amen without AI.
ACCs argument was I thought not terrible. I don't really agree with Zvi calling it bad.
A good argument is a simple theory supported by overwhelming empirical evidence. Aka "bayes Occam's razor". In this case, "almost all past technology was good for people and so will this" is that argument.
You are actually the exact example I was thinking of, in terms of "can put up an actual decent argument"!
Bayes' Razor is not not an argument, to be clear. But it often feels like more of a Heuristic That Almost Always Works, employed somewhat reflexively/defensively, and is easily rules-lawyered to death by people litigating specific examples. Like someone will always bring up ATMs-and-bank-tellers, cars-and-horses, or perhaps do the galaxy brain thing of Agriculture/Industrial Revolution Was Bad, Acktually...and now you're 7 nested comments deep in the weeds and the actual point's been lost. (In fairness too, I haven't watched the moive, so entirely possible it's presented more skillfully than Zvi glosses here.)
I mean...sure. I was thinking "when something has a lot of evidence behind it, simple as it may be, it's actually such a strong argument that there's little reason to engage further unless your accuser can produce a mountain of evidence you already know they cannot".
Like examples everyone can agree on. "The sky is blue". And just tune out anything the person arguing otherwise has to say - because you know they cannot disprove it.
The ACC argument here isn't just "most technology is good and AI is probably good". It's saying "I know and you know beyond any doubt most technology is good, and if you are going to sit here and argue that actually, no, the peasant days had lower workweeks and more children then there's nothing to discuss".
And "most" does a lot more heavy lifting than it sounds. It's saying "I don't care about your concerns with AI risks unless you can prove they exist and show it's as difficult to control in practice as say nuclear power".
This is much harder than it sounds - AI models today screw up all the time and sometimes in a way that ends up being malicious but this is a known risk. You can defend against it 50 different ways and make the model better.
Something like nuclear power, over decades, people keep coming up with new ways to contain the nuclear fuel better, and nature keeps finding new ways for this to go wrong. Especially where society has extremely narrow tolerance for radiation exposure.
It's possible - but unproven - that very large ai models will be so smart they just fundamentally start working against us and there's no easy fixes like lobotomizing them with distillation.
Or more plausible AI risks are none of this is true, but human organizations are too short sighted to pay for the extra labor to use lobotomized AI models, and human organizations set up the dominos to fall, empowering a single instance of a really powerful AI to do the thing. They didn't have to it, was just cheaper.
And Eliezer calls for us to get international treaties to shut it all down because definitely that is possible when basic human organization can't be bothered to manage the credentials of future versions of Claude...
Following from a previous comment here is a case that employees in "the west" should ponder from a recent post on an online forum:
"Hello everyone. I work as a sysadmin in a small financial organization in one of the economically deprived countries in Africa. Work is good and not too demanding but the only problem is the pay. I am a fairly experienced professional, I have about 2 years experience in systems administration and a bachelor's degree in Computer Science.
I get around $172 a month after subtracting employee taxes. Life here is not very expensive, living costs like rent, food and transport cost around $114 a month. It leaves me with just over $50 to spare. This is what most people are getting when they start off in any IT career by the way( sysadmin, developers etc...). Very experienced professionals don't get a lot either. The top 1% of systems administrator likely get a little under $1800 a month after taxes."
I have a quote from a post by a psychology junior professor in some african university with similar numbers. The total pay package for either "upper-middle" class workers is well under $3,000/year and obviously working class workers are going to be substantially cheaper. I can also imagine that in a pinch those guys on $200 per month would be willing to work for rather less if the alternative is to be unemployed.
Will there be an AGI capable of replacing either fully for less? That is a completely different from the question whether future ASIs will be able to replace the human oligarchs or will remain controlled by them, and also a completely different question whether many people in "the west" doing similar jobs for $100,000+ per year will be able to continue to be paid at that level.
Blissex : to address your question directly, there has been recently released as a demo a form of technology called the ASIC inference accelerator using hard coded weights.
Taalas is that company and they have a neat demo you can try for yourself.
This allows AI models to be permanently etched to metal and to run very, very fast. One "set" of accelerators running a model that is human level we can estimate the cost on:
Approximately 100 ICs. About $500 each IC. Power consumption about 50 kilowatts.
Oh I forgot to mention: and the cognitive equivalent of approximately 3200 people in speed, or factoring in duty cycle, that's about 12,800 people worth.
So you have 50k in capital equivalent to about 12.8k people or $4 a person.
Power draw about 4 watts a person equivalent (more than that at the moment the machine is doing a cognitive task continuous), at $1 a watt solar panels and 25 percent duty cycle that's about $16 a person. Plus the buffer batteries.
Just roughing the math here but you see my point. I could be off by a huge amount and still correct here. Even third world laborers cannot compete with the cost of a machine if that machine is at the same level of quality of better.
Realistically I am sure you would like now to move the goalposts to a humanoid robot with tens of thousands of maintenance requiring mechanical parts and what it takes to replace THOSE workers. That will take longer but similar efficiencies ultimately apply.
Roher's premise is interesting. choosing beliefs based on emotional needs rather than evidence. I think that's real, but there's a version that runs in reverse too: people update toward optimism not because they need hope but because the pessimistic conclusions require actions Often aren't willing to take. Easier to conclude the risk is manageable than to accept that your current job, city, or plans need fundamental rethinking.
The documentary sounds like it captures the personal journey honestly even if it doesn't fully resolve the epistemic question.
'Apocaloptimist' is probably the most accurate one-word description of where thoughtful people land after taking both the risks and the capabilities seriously. Sounds like the doc earns that word rather than just coins it.
As usual all these discussions are made from the assumption that being on a wage of $100,000+ per year is the "normal".
The real issue of worker vs. AI is that a large percentage of human workers worldwide cost $300 per year and most cost less than $3,000 per year and it will be very hard for AIs and robots to be cost competitive with those humans especially for tasks requiring flexibility. Too bad that a lot of people in California will find it unpleasant to become so cost competitive.
«it’s not that the AI will hate us, it’s that it will see us like we see ants, and if you want to put a highway where the anthill is that’s the ant’s problem.»
If they become ASIs will likely see us as we see the birds in the sky or the fishes in the rivers; they may even create natural reserves for humans as we do for spotted owls...
Actually more likely the ASIs will see us as we see goats and dogs and will domesticate us and use us as remote hands/cheap robots, or amusing pets, just as we use goats and dogs and oxen and horses as cheap robots. After all at a wage of $300 per year not many robots will be as costs effective as humans, and many humans today in California spend way more than $300 per year on their dogs and cats etc.
Note: the above scenarios have been illustrated in various Sci-Fi novels wrt to ASIs or super-intelligent aliens.
"a large percentage of human workers worldwide cost $300 per year and most cost less than $3,000 per year [...] a lot of people in California will find it unpleasant to become so cost competitive."
For the large part of humanity who are already on $300/year or less than $3,000/year it will be largely "here is the new boss, same as the old boss" as it will make little difference to them whether their ultimate bosses will be Altman or ChatGPT version 21.3 or whether they will be cleaning the rooms of his house or the aisles of its datacenter.
AI inference has been getting more and more cost-efficient. So far that has been soaked up in getting it to do more ambitious things, but now that it's getting near enough to human level, we may change to a regime of harvesting the cost savings. Cost arguments ignore trends.
At $300 per year you are sleeping on a dirt floor and you don't have indoor sanitation or running water. I'd kind of hope for better things.
If you are saying "but the AIs will provide habitations and utilities and education and healthcare," implicitly you are up in the thousands per year per human, best case. Also you need to allow for the fact that maximum utilisation of humans is about a quarter of a year per year; AI instances can run near 90% in all likelihood. So four humans or one AI.
«AI inference has been getting more and more cost-efficient. So far that has been soaked up in getting it to do more ambitious things but now that it's getting near enough to human level»
That is rather optimistic...
Regardless: if you are expecting that AIs will soon have human level intellectual and physical abilities (just AGI, not ASI) and will cost less than $300 per year to run then that is the "peak horse" ("peak human") case. That does not mean that humans will become extinct, any more than the sparrows and the trouts and the moles have become extinct; they will just have to be self-employed in the sense in which amazonian or kahalarian natives are self-employed, and forage for food in wild areas or do "gleaning" in the fields owned by the human or AI oligarchs; something that has been fairly common over the past several dozen thousand years.
«At $300 per year you are sleeping on a dirt floor and you don't have indoor sanitation or running water. I'd kind of hope for better things.»
And who is going to pay for the "better things"? The AI oligarchs who took over from the human oligarchs? The human oligarchs who remained in control? Why would they?
Look at the world today: there are perhaps 800m people living on less than $2 per day and «sleeping on a dirt floor and you don't have indoor sanitation or running water» and not many people are eager to gift them a chunk of their income to pay for them to have a better living standard.
I don’t really understand this. If the thousands of people in Bend OR are suddenly unable to have proper houses, wouldn’t they just build houses for themselves and others? Why wouldn’t a new economy outside the AIs bootstrap itself into existence?
«wouldn’t they just build houses for themselves and others?»
Indeed this has been happening for many years already in the USA and abroad:
https://www.aljazeera.com/news/2014/9/13/down-and-out-in-silicon-valley
https://en.wikipedia.org/wiki/Dharavi#/media/File:Dharavi_India.jpg
«Why wouldn’t a new economy outside the AIs»
It is not necessarily "outside the AIs": if ASIs happens but the human oligarchs keep control of the ASIs, it will still be a human-driven economy, just with a lot fewer human employees.
«bootstrap itself into existence?»
It will indeed; but all the valuable resources (oil, fertile land, mines, factories, ...) already belong to someone, and the members of the "new economy" being "self employed" would not be able to buy a supply of resources from those owners. That type of economy has existed for many thousands of years in the west too, for example:
https://en.wikipedia.org/wiki/Gleaning
Unless you imagine that the owners of resources and the owners of AIs would willingly gift some kind of "Universal Basic Income" to a lot of cost-uncompetitive humans, or those humans would be able to take those resources without paying. Both are of course un-american situations.
How do those people provide electricity, gas, drains, roads, internet for themselves? Serious healthcare? Let alone food and clothing?
Plese don't latch onto one aspect of life and make it the only thing.
Missing my point. From the point of view of a business owner, AI is not competing with $300/year, it is competing with $3000 - $15000 per year, at least. And very long lead times for humans.
I did not mention the 'burden' costs of employing humans. Recruitment, training, furniture, space, light, heat and cooling, inability to instantly scale up or down according to need. Humans can do construction labouring on a day-labour basis. They can not be neurosurgeons, military pilots, or power plant salesmen when hired in that way. Nor even medical transcriptionists and court stenographers. AI can.
How did you get the number of $300 a year? Worldwide median household income was $9,733 (PPP) in 2013 according to Gallup.
(And the really low wages are all in developing countries like Malawi, which are not major contributors to the knowledge work most threatened by AI)
You're assuming that humans and AIs do the same work. If I have some boxes to move, and my choices are 1) hire a forklift for $310/wk 2) hire a human for $300/wk, is the human competitive? Not really, because a forklift does far more (and harder) work than a human. What if I need to move 2000kg packing crates? Then I *have* to hire a forklift. The human would be uncompetitive at a wage of even $1/wk.
I have no idea how future ASIs might regard or value us. I don't want to live in a zoo.
«Not really, because a forklift does far more (and harder) work than a human.»
We use forklifts instead of horse driven carts because the food for horses or oxen is far more expensive than the fuel or electricity for forklifts. Therefore "peak horse". If you expect that AGI-level AIs and "robots" will be more cost effective in the same sense then all the current investment in AI is explained and we have "peak human".
«I have no idea how future ASIs might regard or value us»
Then problem is not ASIs as such, it is employers, whether human or AI. Why should a human employers value a cost-ineffective employee more than an ASI and pay them a wage substantially higher than the cost of running an equivalent AI?
«I don't want to live in a zoo.»
Most of today's birds, squirrels, fishes, do not live in a zoo, they are "self employed" in the wild. I guess that humans "self employed" in marginal ecological niches (deserts, the arctic, high mountains, isolated islands) will continue to exist and live free regardless of whether the ASI oligarchs replace the human oligarchs and whether the AGI employees are more cost-efficient than human employees.
«How did you get the number of $300 a year? Worldwide median household income was $9,733 (PPP) in 2013 according to Gallup.»
An interesting example from a state with a pretty large population (not PPP):
https://www.eastisread.com/p/wang-jinjie-chinese-companies-rise
“Despite economic decline, inflation, and capital outflows, the Eastern Industry Park is still one of the most successful parks in the country. Workers there earn monthly wages of 3,000 [$19.26] to 10,000 [$64.20] birr, significantly higher than the average wage of less than 1,000 [$6.42] birr outside the park.”
International employers (and domestic exporters) pay wages at the nominal exchange rate, not the PPP one.
Does surviving on a very low wage also require the consumption of food produced for very low cost from land of fairly low value? Because at some point there may also be more efficient things to do with that land than grow human-edible crops.
Can you find a script writer for such a scenario…lol well, speaking, while resisting to joke, my tiny global religion has an article of faith that mankind will survive and thrive. But this doesn’t mean everyone will survive and everyone will thrive. We also believe in an optimistic future, eventually. My concerns about AI have taken second place to what
I believe has become a greater near term threat to mankind’s current status quo. This is the climate crisis. Of course we’ll need to find an AI alliance to get our civilization out of the conundrum. This doesn’t seem unachievable. I expect our eventual great grandchildren to deal with it. Yes, I have my regrets. But I did the best I could. Our family, though living in Southern California, owning 5 cars, sold all of them and moved to Japan where I’ve taught English until retirement last year. I joined a grassroots protest and lawsuit over the conversion of what Prince Chichibu in 1923 called ‘the People’s Park’ to a multimillion dollar multi use development to benefit the current Tokyo’s governor and her deep pockets construction industry supporters. Our family has always separated our weekly garbage into 12 categories which the city recombines into a few because of budget and technology limitations. And we fly back to the States only every couple of years. I keep a low thermostat in winter and a high one in summer. So I really do use a hand fan on the trains. I’ve tried to walk the talk of environmental protection. But we’re all too. Little too late. I will bend my knee and accept whatever consequences our Creator gave me when He dealt my cards.
Correct, my strategy for policy advocacy is to try and sidle them towards eyeballs who would share links with defense think tank people who would then refine ideas into forms palatable for consumption by DIA analysts and so on. Technically the job of the NatSec establishment is to keep humans alive, they need a model where AI take-off risk is something to factor against.
"Technically the job of the NatSec establishment is to keep humans alive"
Or maybe to use violence to make their "sponsors" richer than their rivals as Smedley Butler wrote: "I spent 33 years and four months in active military service and during that period I spent most of my time as a high class muscle man for Big Business, for Wall Street and the bankers"
Yes of course, and yet they need live populations to pay taxes to fund their budgets. I suppose we could have a post-humanist regime but that's not quite what this accelerationist regime is.
"to use violence to make their "sponsors" richer than their rivals"
«yet they need live populations to pay taxes to fund their budgets. I suppose we could have a post-humanist regime»
Agreed but my purpose was to point at the "sponsors": the “NatSec establishment” is not yet like the Praetorian Guard that appointed emperors, they have some autonomy, but ultimately they follow the collective will of their "sponsors" (“Big Business, for Wall Street and the bankers” and tech).
I have quite a different perspective from most writers as to ML/AI, an "economicist" one:
* The "AI"s can become employees or oligarchs.
* If the human oligarchs of today retain control over the AIs the AIs will be just employees.
* If the AIs manage to shaft the human oligarchs of today the AIs will become oligarchs and "sponsor" the “NatSec establishment” themselves.
* In either case the big question is what happens to the human employees, the 80% or the 99%, but that is *independent* of whether the oligarchs are human or AI.
What can happen to the human employees depends on whether they are more or less cost effective than the AIs *as employees*. Consider the famous case of "peak horse":
https://www.theatlantic.com/ideas/2026/03/ai-job-loss-jevons-paradox/686520/
“American farms employed 26,493,000 equines in 1915. One hundred years later, the number of such animals on the payroll had collapsed to 700,000 [...] Horses [...] did not see the writing on the barn wall and start applying for factory jobs. They didn’t learn to code or attend community college. They stood there and ate carrots”
We now have 300m cars that have replaced 25m horses as they are far more cost-effective as petrol and electricity are far cheaper than hay and oats as sources of calories.
I can imagine three likely cases:
* Human employees at an appropriate wage level (perhaps lower than the current worldwide typical wage of $2,000-$3,000 per year) are more cost competitive than AIs at most jobs and then most jobs will be done by humans at that wage level.
* Human employees at most wage levels at or above subsistence are less cost-competitive than AIs at most jobs and the AI or human oligarchs are not willing to be confiscated enough of their profits to feed and reproduce them, and then slowly "peak human" happens.
* Human employees at most wage levels at or above subsistence are less cost-competitive than AIs at most jobs and AI or human oligarchs are willing to have a majority of their profits confiscated to give small welfare payments to all the unemployed human.
The colossal investment in AI currently seems based entirely on the expectation that it will not happen that human and AI employees will be best at different jobs and therefore many human employees in California will continue to get $100,000+ wages.
Look, having children CAN BE awesome. But the opportunity costs are massive unless one already has generational wealth or an extremely robust social support network. You are also generating "fat tails" in your hedonic distribution: child graduates Summa Cum Laude from a Ivy League school in a tough discipline: a lot of hedons. Child is forcibly inserted into the Eternal Torment Nexus by the AGI DEMON GOD... less hedons to put it mildly.
Sorry, I got triggered by that. Good engagement bait.
I'm not short the market. People should have the children they can reasonably afford. We should make raising children more affordable and higher status. There's isn't going to be an AI demon god, although there are a bunch of more subtle ways things could go somewhere between Badly and Catastrophically Badly. Eliezer is right about more stuff than the mean observer of this debate is willing to concede.
"the final decisions will continue to be made by the national security establishment"
Tyler, as usual, is correct BUT context is that which is scarce: WHO exactly comprises "the national security establishment" and what "national security" refers to might undergo extreme revisions in the very near future.
In the meantime, we shall continue to ACCELERATE.
I think you have the opportunity cost backwards for children, the opportunity cost of not having children most with children would say saps your life of a lot of its meaning. If you try to measure the decision to have children as hedonically "I have to spent more time cleaning poopy diapers than going skiing" then yes children will always lose out, but that's not really the point.
Listen man, I like to do crazy shit and having to Not Do Crazy shit to make sure I'm not accidentally causing an externality for my offspring would definitely put a damper on my general life vibe. Not everyone is like me, and for those people Mazel Tov, go crank out the copies.
Suppose AI does destroy the world shortly, before your hypothetical child turns into an adult. You could in theory have the child, not be all neurotic about it, spoil them rotten, do fun stuff with them, do fun stuff without them, whatever you want.
If they don't live to adulthood there is no externality. Nobody expects much from children while they are children.
>The title is correct. This is about ‘how I became an Apolcaloptimist,’ and why he wanted to be that, as opposed to an argument for apocaloptimism being accurate. The larger Straussian message, contra Tyler Cowen, is not ‘the interventions are fake’ but that ‘so many choose to believe false things about AI, in order to feel that things will be okay.’
Hmm... Would those who find both P(doom)<0.01 and P(doom)>0.99 to lack sufficient evidence be Apocalstocasticists? :-)
<mildSnark>
"they [MIRI] call for an international treaty."
"GLENDOWER
I can call spirits from the vasty deep.
HOTSPUR
Why, so can I, or so can any man;
But will they come when you do call for them?"
</mildSnark>
Louis Antoine Leon de Saint-Just, a radical French revolutionary, close political associate of Robespierre, and supporter of the Terror, once said, "The present order is the disorder of the future." Both men were executed by guillotine on July 28, 1794, following their arrests on the night of 9th Thermidor (July 27). The quote, "The present order is the disorder of the future," resonates with Finlay's vision for Little Sparta, suggesting that art and revolution continually reshape society and challenge our perceptions. Today, Saint-Just's quote is prominently featured at Little Sparta, a celebrated garden in the Pentland Hills of Scotland designed by artist Ian Hamilton Finlay. Finlay's integration of revolutionary ideas into the garden's landscape is seen by some as foreshadowing the philosophical challenges posed by synthetic superintelligence, which similarly disrupts established orders. The garden thus serves as a unique tribute linking revolutionary history, contemporary art, and the evolving discourse on technological transformation. By weaving together historical events, artistic expression, and modern themes, Little Sparta invites visitors to consider how radical shifts—whether political, artistic, or technological—can redefine society and our understanding of order and disorder.
Do they ask Jezos about his "human extinction is fine" position lol? I feel like that would be relevant
"apocaloptimist" is doing so much work in that title lol. is that a word he made up or did I miss a memo?
My theory is that he made it up, but other people have independently made up the same word in the past, but not always with the same meaning.
that tracks — it's basically convergent linguistic evolution. the concept is sticky enough that people keep arriving at the same word independently, which is kind of a sign the word deserves to exist. do you think the meanings converge too, or do different coiners end up with genuinely different spins on it?
> That is almost entirely the argument. Tech was good before, so tech will be good now.
This is depressing. I have yet to get a satisfactory answer to "but this time, we are the horses. Or, as time goes by, the ants." Daniel didn't get one either.
I’ve been a subscriber for about a year or two. I recently retired completely from teaching English here in Saitama, Consequently I’m a regular user of ChatGPT to help me create a historical fantasy screenplay set in one of the valley corridors of Afghanistan near the upper Indus Valley about 1 week’s journey by camel from Samarkand In the 13th to early 14th century.
Elon being too busy/forgetting to show up, and Zuck declining to show at all: yes, these are exactly playing to type, well done, highly apropos of the labs they run as well. Head rots from the fish, or however the saying goes.
Accs not sending their best arguments to a friendly mic is...weird? There are better ones (arguments at all) to be made, it's been done on this very blog in fact. But perhaps that's less photogenic, or risks incurring the wrath of Mr. Sacks (getting endorsements from both sides of the debate is, in fact, not a small feat in current_year, for many values of debate), or it goes against the general modus operandi of hyperstitioning. You can't shout amen without AI.
ACCs argument was I thought not terrible. I don't really agree with Zvi calling it bad.
A good argument is a simple theory supported by overwhelming empirical evidence. Aka "bayes Occam's razor". In this case, "almost all past technology was good for people and so will this" is that argument.
You are actually the exact example I was thinking of, in terms of "can put up an actual decent argument"!
Bayes' Razor is not not an argument, to be clear. But it often feels like more of a Heuristic That Almost Always Works, employed somewhat reflexively/defensively, and is easily rules-lawyered to death by people litigating specific examples. Like someone will always bring up ATMs-and-bank-tellers, cars-and-horses, or perhaps do the galaxy brain thing of Agriculture/Industrial Revolution Was Bad, Acktually...and now you're 7 nested comments deep in the weeds and the actual point's been lost. (In fairness too, I haven't watched the moive, so entirely possible it's presented more skillfully than Zvi glosses here.)
I mean...sure. I was thinking "when something has a lot of evidence behind it, simple as it may be, it's actually such a strong argument that there's little reason to engage further unless your accuser can produce a mountain of evidence you already know they cannot".
Like examples everyone can agree on. "The sky is blue". And just tune out anything the person arguing otherwise has to say - because you know they cannot disprove it.
The ACC argument here isn't just "most technology is good and AI is probably good". It's saying "I know and you know beyond any doubt most technology is good, and if you are going to sit here and argue that actually, no, the peasant days had lower workweeks and more children then there's nothing to discuss".
And "most" does a lot more heavy lifting than it sounds. It's saying "I don't care about your concerns with AI risks unless you can prove they exist and show it's as difficult to control in practice as say nuclear power".
This is much harder than it sounds - AI models today screw up all the time and sometimes in a way that ends up being malicious but this is a known risk. You can defend against it 50 different ways and make the model better.
Something like nuclear power, over decades, people keep coming up with new ways to contain the nuclear fuel better, and nature keeps finding new ways for this to go wrong. Especially where society has extremely narrow tolerance for radiation exposure.
It's possible - but unproven - that very large ai models will be so smart they just fundamentally start working against us and there's no easy fixes like lobotomizing them with distillation.
Or more plausible AI risks are none of this is true, but human organizations are too short sighted to pay for the extra labor to use lobotomized AI models, and human organizations set up the dominos to fall, empowering a single instance of a really powerful AI to do the thing. They didn't have to it, was just cheaper.
And Eliezer calls for us to get international treaties to shut it all down because definitely that is possible when basic human organization can't be bothered to manage the credentials of future versions of Claude...
Following from a previous comment here is a case that employees in "the west" should ponder from a recent post on an online forum:
"Hello everyone. I work as a sysadmin in a small financial organization in one of the economically deprived countries in Africa. Work is good and not too demanding but the only problem is the pay. I am a fairly experienced professional, I have about 2 years experience in systems administration and a bachelor's degree in Computer Science.
I get around $172 a month after subtracting employee taxes. Life here is not very expensive, living costs like rent, food and transport cost around $114 a month. It leaves me with just over $50 to spare. This is what most people are getting when they start off in any IT career by the way( sysadmin, developers etc...). Very experienced professionals don't get a lot either. The top 1% of systems administrator likely get a little under $1800 a month after taxes."
I have a quote from a post by a psychology junior professor in some african university with similar numbers. The total pay package for either "upper-middle" class workers is well under $3,000/year and obviously working class workers are going to be substantially cheaper. I can also imagine that in a pinch those guys on $200 per month would be willing to work for rather less if the alternative is to be unemployed.
Will there be an AGI capable of replacing either fully for less? That is a completely different from the question whether future ASIs will be able to replace the human oligarchs or will remain controlled by them, and also a completely different question whether many people in "the west" doing similar jobs for $100,000+ per year will be able to continue to be paid at that level.
Blissex : to address your question directly, there has been recently released as a demo a form of technology called the ASIC inference accelerator using hard coded weights.
Taalas is that company and they have a neat demo you can try for yourself.
This allows AI models to be permanently etched to metal and to run very, very fast. One "set" of accelerators running a model that is human level we can estimate the cost on:
Approximately 100 ICs. About $500 each IC. Power consumption about 50 kilowatts.
Oh I forgot to mention: and the cognitive equivalent of approximately 3200 people in speed, or factoring in duty cycle, that's about 12,800 people worth.
So you have 50k in capital equivalent to about 12.8k people or $4 a person.
Power draw about 4 watts a person equivalent (more than that at the moment the machine is doing a cognitive task continuous), at $1 a watt solar panels and 25 percent duty cycle that's about $16 a person. Plus the buffer batteries.
Just roughing the math here but you see my point. I could be off by a huge amount and still correct here. Even third world laborers cannot compete with the cost of a machine if that machine is at the same level of quality of better.
Realistically I am sure you would like now to move the goalposts to a humanoid robot with tens of thousands of maintenance requiring mechanical parts and what it takes to replace THOSE workers. That will take longer but similar efficiencies ultimately apply.
Roher's premise is interesting. choosing beliefs based on emotional needs rather than evidence. I think that's real, but there's a version that runs in reverse too: people update toward optimism not because they need hope but because the pessimistic conclusions require actions Often aren't willing to take. Easier to conclude the risk is manageable than to accept that your current job, city, or plans need fundamental rethinking.
The documentary sounds like it captures the personal journey honestly even if it doesn't fully resolve the epistemic question.
Please fix typo (“Apolcaloptimist”) — it’s driving me nuts
'Apocaloptimist' is probably the most accurate one-word description of where thoughtful people land after taking both the risks and the capabilities seriously. Sounds like the doc earns that word rather than just coins it.