It has been quite a past two weeks, with different spheres in deeply divided narratives. In addition to the liberation of Kherson City, the midterm elections that of course took forever to resolve and the ongoing hijinks of Elon Musk taking over Twitter, there was the complete implosion of Sam Bankman-Fried (hereafter ‘SBF’), his crypto exchange FTX and his crypto trading firm Alameda Research.
re: Caroline's review of Barbarians at the Gate, it seems clear that this list:
- they all have fiduciary duties
- ???
- shareholder value is maximised
is riffing off the 1. step one 2. step two 3. ??? 4. profit meme, where the reader is not expecting the profit to ever exist. So I think she agrees with your reading!
she isn't specific, but based on a parallel example from SBF, I suppose she's talking about positive EV coin flip. SBF example read 75% win 25% loss or subpar EV.
Yup, it's not crazy if you're "cashing out" fairly soon. It's unfortunate that so many people claiming SBF/Caroline are "obviously wrong" aren't careful in stating the conditions under which the all-in strategy actually works. I speculate that this is part of why SBF still seems to believe his decision framework is rational.
It becomes crazy once you're dealing with a longer time horizon, extending far beyond what you can model precisely. As Zvi noted 4 years ago (https://www.lesswrong.com/posts/BZ6XaCwN4QGgH9CxF/the-kelly-criterion?commentId=3JEaWCYG2B5ocBJg7 ), "even if your utility outside the game is linear, inside of the game it is not". Investment opportunities tend to exhibit diminishing returns, and this makes the correct decisions under near-linear utility become more similar to those under logarithmic utility as the time horizon gets longer; you are then mostly stuck inside the game.
The other usually-convincing argument against SBF's "greedy algorithm" is noting that (i) you don't always know your edge and (ii) you can end up colossally negative, not just colossally positive. In combination, the result is that a single bad edge-estimate can be enough to sink the greedy algorithm. It's really unfortunate that the double-or-nothing thought experiment misled SBF and Caroline by only involving nonnegative numbers.
re: "The decline in crypto prices of ~20% does indeed seem too low to me. "
That part confused me, is that decline just "crypto vibes = bad" -> 20% drop? Obviously I can see why FTT or Serum would drop, but why would some unrelated token drop as well? Why would, to pick a non-random example, BNB drop too?
That is not a random example! Rather it is an interesting one that did relatively well, because Binance's relative position improved, rather than people doubting the safety and wisdom of exchange tokens in general. The answer is that a bunch of people lose capital, confidence is shaken, people worry about who is bust, space looks generally gloomy, people sell, people don't buy or invest, prices drop near term and then things are bad for a while. Your estimate of all of crypto's future value goes down a lot after these events if you did not see them coming, or should.
Okay, got it - "Your estimate of all of crypto's future value goes down a lot after these events" because the events reveal something about people's expectations of whether "things labeled crypto" will go up/down in general. I just wonder (idle speculation) what this means for whether people (or me) should invest in other crypto projects or exchanges that can verify they do not have the flaws of FTX/SBF, which may be currently depressed, but therefore have the potential to go up in the future.
If you do that, do so with the clear knowledge that there is *no underlying value* in any crypto. They could drop to zero dollars based wholly on the whims of the market. If there were a big selloff of BTC tomorrow (for instance a big investor decides to get out), then people might further flood the market with BTC and drive the price further down and so on in a spiral. Normal investments have a floor. That floor is the real value of assets the company holds or the real value of the thing itself. Gold is a tangible product used in making things like jewelry and electronics. It has a floor value even if the speculative value goes away completely. Crypto does not.
That's why you're struggling to determine if you should buy into a "depressed" market or not. A normal market can be evaluated for an expected value, which should in some way correlate with it's floor value (typically a company is evaluated based on a clear formula that includes profitability and baseline assets like land and buildings). Because crypto has no floor or value beyond speculation, any particular value can be too high or too low. The only way to measure current value is to wait until the future and look back, or look back into the past and guess at it. This is worse than buying overpriced tulips, because even they had *some* underlying value.
Non-rhetorically: Why do you think no cryptocurrency has any underlying value? Especially when the use, design and .... (advertised? aspirational?) underlying value... of each one varies so much? And why do you think that's the important metric in determining the floor, rather than people's perception of whether it will be worth more [than the current price] in the future? I am not trying to be pro- or anti- crypto, just trying to understand. National currencies would seem to have a similar "no underlying value" state. I guess I'm just not clear on what the "underlying value" concept gets us.
National currencies do have similar problems, but they also have a floor. Those countries will accept the money for debts, including taxes, which means the value is at least attached to the stability of that country. This is also one of the reasons that hyperinflation can exist, when a country is unstable and/or willing to devalue their own money.
Think about this, why is BTC worth more than Ethereum is worth more than fly-by-night new coin? There's nothing fundamentally different between BTC and [picks something cheap, randomly] ANKR, while one is worth upwards of $15,000 and the other is worth two cents. Whatever use value a digital currency has (which has a troubled history but could theoretically work) is probably closer to valuing it at the two cents than the $15,000. That's the underlying value. It's essentially worthless - it costs more to mine a Bitcoin in electricity than it's worth from a use-case. The speculative value, on the other hand, can be quite large. The problem you are having is a deeper issue, but I think one pegged to a losing proposition. Yes, you can make money on crypto by reading the room (is the price going to go up or down) and buying at a lower price than it will be in the future. Unfortunately, nobody has any real idea about whether it will actually go up or down, and more importantly, by how much. Maybe BTC will drop to $12,000, or go up to $20,000, or a million dollars. There's absolutely nothing above some minimum price (surely less than $1 each) that helps you track what the true price should be.
Check out the long term volatility and tell me there's a natural price that makes sense, for which you could argue that buying below that is somehow a good buy, but above that is too much?
I see what you're saying, and I'm not saying I know what the "right" price should be either, I just don't know that underlying value is helping me price other things either.
Re: "Those countries will accept the money for debts, including taxes, which means the value is at least attached to the stability of that country. "
There are lots of coins that are attached to companies that will indeed give you services in exchange for those coins. And obviously a small company with a few dozen people is vastly less stable than a country with millions of people, and indeed those coins seem (to my ignorant eye) to act like company stocks trading on the value of those services. Does that count as underlying value? (no matter how small)
"To be sure, it seems as though Bankman-Fried didn’t take side-constraints particularly seriously. But this doesn’t necessarily mean that he would have done any better had he taken them seriously. Intending to obey deontological constraints is no more a guarantee that you will obey them than intending to maximise utility is a guarantee that you will maximise utility. What’s needed is a more systematic analysis of how to act in situations of power, potentially including proposals for reform at the institutional level—not simplistic rules drawn from intuition."
" ‘heroes put the entire group, many innocent people, ‘the city,’ planet Earth or even the whole damn universe or multiverse in grave danger to save any main character because That’s What Heroes Do’ is ubiquitous in our fantasy media...it is rather mind-bogglingly awful. That kind of thinking needs to be widely condemned."
I get what you're saying. But didn't you also say earlier in this article
"Do not violate deontology for utilitarian ends. Seriously. Stop. Don’t do it."
And while as You say, Game Theory means not bargaining with hostage takers, the above formulation seems like you would be ok with the whole risking the city/world/multiverse scenario If the context were a bit different?
"Batman, this civilian has been turned into a living sun, If You don't kill her she will emit 10,000,000 degree heat and kill countless people. What do you say to that, Batman?"
"I won't kill her. Do not violate deontology for utilitarian ends. Seriously. Stop. Don’t do it."
You might say this is a heightened bizarre scenario totally divorced from reality and so the logic beneath it can't teach us anything. But then, if you're allowed to logically analyze one set of bizarre comic book scenarios, you're allowed to logically analyze another, no?
But the reason I said "living sun" instead of "living time bomb" or somesuch thing was the implication that by some absurd comic book physics the civilian would not be killed by this power even as it was killing countless other people.
And in that context, I don't think that Batman 2 canonically exists. At least, if what Batman 2 is saying is supposed to be a prelude to justified homicide in the name of the greater good, then I think that's not actually something canonical batman would do. He doesn't kill the Joker even though he's wracked up a body count in the millions by this point. He's certainly not going to kill an innocent person, even if it seems almost certain that doing so will save hundreds of thousands of lives.
Though my subcomment here is merely about character interpretation, not my view of ethics.
Well, that brings up two problems with fiction, philosophical hypotheticals, and comic books:
1: In a very different world (where people might become living suns and be unable to keep themselves from melting the world) there would be very different ethics based on the entirety of that world. Is Batman's deontological commitment to not kill one that extends to allowing the entire city/planet to get killed by following it? Most Batman (to my limited knowledge) does not have those kinds of stakes, where killing someone quick solves the problem. Which leads to...
2: Does Batman's deontology actually include not killing someone who will imminently destroy the world? Maybe not doing it if he has any other option, but it seems likely that he would plan for that final requirement. He did canonically spend a lot of time planning to defeat Superman, after all. We always see Batman opt for not killing when, honestly, killing would probably be better over all (see: Joker). Then again, Joker isn't an extinction level event, and if he was, it isn't clear to me that Batman would say "Whelp, nothing to be done, can't kill." My expectation would be that Batman would have a backup plan to kill Living Sun at the last second, in case the whole non-killing plan didn't work.
Then again, we will never know, because Batman doesn't exist as such, and is just a character in a fantasy world written by a variety of people who take things like this more or less seriously. *shrug*
Small thing: Not 100% certain, but I don't think "$5bb" is accepted notation. "$5mm" exists because it's m x m, where m = 1000. So, 5bb would be a 5 trillion or 5 quintillion, by analogy.
Re “Notice this VC ready to invest in SBF again if he asked”: this guy doesn't seem like a VC at all; if he is, he's definitely not anything like a top Silicon Valley VC. His Twitter bio says: “Chairman of O’Shares and Beanstox, CNBC Contributor,4 X Emmy Shark Tank's Mr. Wonderful, wine maker, watch, crypto & guitar guy, photographer & Chef”
> I do find his ‘the biggest thing you actually buy is a car’ attitude odd given the existence of houses.
I'm guessing you don't own a house.
As I tweeted at the time, a house is more of a strategy than a possession. Once you have a mortgage, especially one close to your maximum, it dominates ALL your financial decision making. It may seem like a house doesn't fit Rao's narrative but that's mainly because the bureaucracy that manages your home ownership is "you and your spouse".
That's true if you are middle class. It is NOT true if you're rich.
E.g. regular person buys $500k house, puts $200k down, plans around mortgage.
Rich guy buys $3 million house (or even $30 million house). Takes out a mortgage because why not, tax advantages and interest rate optionality, but it does not dominate his thinking. In the 8-figure net worth range a lot of people start buying vacation homes or second homes.
I'm interested in your flow of "long version", "short version", "shortest version." I would've thought it was better to go the other way so you can build up an incrementally more detailed mental model. Like a progressive jpeg. Do you know something I don't?
Think of the short version as a summary after having read the long version, to cement understanding or give you a basic understanding if long version was unclear.
Both Silver and Jason make good points here. I also don't want to anchor the reader on the short version, my take of which steps are most important, first.
Zvi, what is your advice to a software engineer in the AI compiler/infrastructure space? Not directly working on improving ML capabilities, but building tools for it. Is it possible to do anything good from within or only option is to get out?
Thanks for all your work, particularly the covid coverage. Love the blog.
I think good things are possible. I think that it is VERY easy to fool oneself on this and end up doing bad things instead, and 'I'll do it instead of someone else and I will care about doing it safely' isn't good enough.
I find it weird that still after all this time, SBF basically being a partisan hack, with donating to the Democrats being literally his main 'altruistic' cause, is still apaprently fine with absolutely everybody remotely related to the EA movement. Like, donationg millions to a party that recieved financial and other support from a majority of the largest corporations, all of the most prestigious universities and a majority of newspapers etc when you're being used as almost the face of EA or earning to give (80,000 hours literally listed him first on their 'About Us' page of people whose stories and support they are 'proud' of and mentioned his donations to the Democrats as main (only?) example of his "altruism").
Maybe you can make some sort of case for keeping Repubicans out of power truly being of supreme importance, but I think that that is not only kind of dumb, but also very distasteful for EA orgs and certainly makes me want to have nothing to do with them/actively cheer for their failure. And if nothing else, it strikes me as extremely ineffective altruism. Even being a huge donor, what material impact on elections would SBF reasonably have expected to have (vs other "altruism" options)?
It would have been one thing if these orgs just pinched their noses and accepted it for the sake of the cash, but it's now clear that they at least had no problem with it, and probably thought it was a good thing. Rob Wilbun etc blocking anyone on twitter who disputes the left wing narratives espoused in his tweets (e.g. basically being a heritability denier) certainly supports this, which is all the more reason these orgs losing funding should be considered a truly wonderful thing.
SBF's political actions were more a lot more complicated than 'partisan hack.' Some of the funds went to Biden 2020 and would have gone to Dem '24 if Trump got the nomination, yet the bulk of it was doing something more strategic (whether or not you agree with the strategies or goals involved.) He had specific policy goals, both in crypto and in pandemic prevention, plus at least one thing I can't name but will certify as Good Cause. Take that however you like. Also worth noting that Dustin Muskovitz is a major Democratic donor in way that mostly (as far as I can tell) ISN'T strategic. Again, not to be boo or yay, simply as important context.
Hey @zvi, Have you written about what the possible use cases for crypto? (I have some sense from Vitalik, but frankly I only understand about 60% of what V writes) I know block chain has uses, but crypto itself (as in tokens on its own blockchain) still seems limited, so would be interested in what you might be seeing? Also, do you agree on the some of the current thinking that crypto has turned out to be "centralising" whereas AI (eg Stable Diffusion) has turned out to be decentralising?
If you think AI is decentralizing, I have what I am guessing you will consider bad news. AI has been massively centralizing, giving huge advantages to big companies with big data available and big resources to use it. Their algorithms have the edge now, and also they are the ones who can influence the AIs. I expect such things to accelerate (and if AI does end up decentralizing, I'd mostly update in favor of us all definitely going to die from it).
Crypto doesn't seem centralizing to me, it's more that crypto started out with this anarco-capitalist dream of full decentralization which is something no one wants for many different reasons.
As for its use cases, haven't written about it, basically stayed away from crypto entirely until this other than regarding my game. You can think of it as crypto being the ability to have a public record. At all. Of anything. Without crypto, any public record is whatever the Powers That Be decide is the record, so it isn't a public record. Crypto changes that, whatever the blockchain says happened, happened. That can be tracking who has the money. It can be who said or agreed to what or did what, when. It can be hashed messages, or proving who you are. It can be claiming ideas. That's the core thing that has value, as I understand it.
Which is why e.g. Emergents wants to be on a blockchain. You want a Public Record of who has what cards and what attributes those cards have, that doesn't rely on a central authority (us) or allow that authority to change it whenever we feel like it.
Ah. I don't have a strong view on if AI is decentralising. But I read some thinkers who now think it might be. eg Ben Thompson (see here: futher down - https://stratechery.com/2022/narratives/ but I think Ben T is talking about a different form of it - eg Stable Diffusion in the hands of others. But I hear your argument that big tech / orgs now have the adge on their aglorithms etc. on influence AIs. I definitely dont feel I know enough about AGi etc. execpt some seeming experts in the field have non-zero worries about the area.
Note your argument on the value of a Public Record.
Completely, aside, I wonder if there is a small love story type element to the FTX/SBF/Caroline saga - it seems maybe not - but I'm interested in the flawed human decisions which led here.
Utilitarianism is an ethical theory, so it's something close to a category error to say that "something is deeply wrong with utilitarianism in practice." Maybe you mean that something is deeply wrong with how some people attempt to implement utilitarianism in practice. This makes clearer that the problem is with the people, and how they were thinking about the theory. There are better ways of thinking about it available (which are in fact central to the utilitarian philosophical tradition). I offer an outline (with academic citations) here: https://rychappell.substack.com/p/naive-vs-prudent-utilitarianism
I think the whole enterprise was a failure to incorporate very well known reasons to doubt FTX and SBF specifically. The vast majority of the financial world did not invest in FTX or crypto. Instead of taking that signal and updating on how good FTX was (not very), it appears that pro-crypto-investment individuals artificially cut off the other world of finance and only considered pro-crypto-investment perspectives. Let's say 90% of the financial markets are in a clear "crypto is dangerous and has no underlying value, don't buy this" and the other 10% is somewhere between willing to try it out and willing to invest heavily in it. If that 10% is wise, they'll keep in mind that 90% of the world will not even touch this stuff at all, let alone go long on it. What seems to have actually happened was for the 10% to say something like "Those [90%] don't understand crypto, I'll look at my peers in the know and recalibrate with them as a baseline" and suddenly those buying small amounts of crypto look like conservative investors. If you are willing to be a bit more risky than the most conservative, you'll stake a decent amount of your money on crypto, and still feel like you're only moderately risky. This is an obvious failure mode from the outside perspective. It's not surprising that once people increase their Overton Window on investment risk (at least when crypto is involved), they'll be more willing to overlook pretty obvious red flags. I mean, BC has had super obvious red flags from the start, but it's held value pretty well and made some people a lot of money (unfortunately, on the backs of people pumping money in later). So people tend to ignore the red flags. But can anyone really say that during a bad economic downturn (2008 certainly, let alone 1929) that BC will not lose a ton of value and leave people over-leveraged in it to go broke?
Then you find out things like FTX/Alameda not having complete lists of all of their *bank accounts* or even their *employees* and *employee compensation* and the whole thing just looks insane.
Can I interest you in some virtue ethics?
https://www.lesswrong.com/s/xqgwpmwDYsn8osoje
re: Caroline's review of Barbarians at the Gate, it seems clear that this list:
- they all have fiduciary duties
- ???
- shareholder value is maximised
is riffing off the 1. step one 2. step two 3. ??? 4. profit meme, where the reader is not expecting the profit to ever exist. So I think she agrees with your reading!
coin flip forever from Caroline:
she isn't specific, but based on a parallel example from SBF, I suppose she's talking about positive EV coin flip. SBF example read 75% win 25% loss or subpar EV.
this isn't crazy
Yup, it's not crazy if you're "cashing out" fairly soon. It's unfortunate that so many people claiming SBF/Caroline are "obviously wrong" aren't careful in stating the conditions under which the all-in strategy actually works. I speculate that this is part of why SBF still seems to believe his decision framework is rational.
It becomes crazy once you're dealing with a longer time horizon, extending far beyond what you can model precisely. As Zvi noted 4 years ago (https://www.lesswrong.com/posts/BZ6XaCwN4QGgH9CxF/the-kelly-criterion?commentId=3JEaWCYG2B5ocBJg7 ), "even if your utility outside the game is linear, inside of the game it is not". Investment opportunities tend to exhibit diminishing returns, and this makes the correct decisions under near-linear utility become more similar to those under logarithmic utility as the time horizon gets longer; you are then mostly stuck inside the game.
The other usually-convincing argument against SBF's "greedy algorithm" is noting that (i) you don't always know your edge and (ii) you can end up colossally negative, not just colossally positive. In combination, the result is that a single bad edge-estimate can be enough to sink the greedy algorithm. It's really unfortunate that the double-or-nothing thought experiment misled SBF and Caroline by only involving nonnegative numbers.
Great Summary! Thank you for putting this together.
re: "The decline in crypto prices of ~20% does indeed seem too low to me. "
That part confused me, is that decline just "crypto vibes = bad" -> 20% drop? Obviously I can see why FTT or Serum would drop, but why would some unrelated token drop as well? Why would, to pick a non-random example, BNB drop too?
That is not a random example! Rather it is an interesting one that did relatively well, because Binance's relative position improved, rather than people doubting the safety and wisdom of exchange tokens in general. The answer is that a bunch of people lose capital, confidence is shaken, people worry about who is bust, space looks generally gloomy, people sell, people don't buy or invest, prices drop near term and then things are bad for a while. Your estimate of all of crypto's future value goes down a lot after these events if you did not see them coming, or should.
Okay, got it - "Your estimate of all of crypto's future value goes down a lot after these events" because the events reveal something about people's expectations of whether "things labeled crypto" will go up/down in general. I just wonder (idle speculation) what this means for whether people (or me) should invest in other crypto projects or exchanges that can verify they do not have the flaws of FTX/SBF, which may be currently depressed, but therefore have the potential to go up in the future.
If you do that, do so with the clear knowledge that there is *no underlying value* in any crypto. They could drop to zero dollars based wholly on the whims of the market. If there were a big selloff of BTC tomorrow (for instance a big investor decides to get out), then people might further flood the market with BTC and drive the price further down and so on in a spiral. Normal investments have a floor. That floor is the real value of assets the company holds or the real value of the thing itself. Gold is a tangible product used in making things like jewelry and electronics. It has a floor value even if the speculative value goes away completely. Crypto does not.
That's why you're struggling to determine if you should buy into a "depressed" market or not. A normal market can be evaluated for an expected value, which should in some way correlate with it's floor value (typically a company is evaluated based on a clear formula that includes profitability and baseline assets like land and buildings). Because crypto has no floor or value beyond speculation, any particular value can be too high or too low. The only way to measure current value is to wait until the future and look back, or look back into the past and guess at it. This is worse than buying overpriced tulips, because even they had *some* underlying value.
https://en.wikipedia.org/wiki/Tulip_mania
Non-rhetorically: Why do you think no cryptocurrency has any underlying value? Especially when the use, design and .... (advertised? aspirational?) underlying value... of each one varies so much? And why do you think that's the important metric in determining the floor, rather than people's perception of whether it will be worth more [than the current price] in the future? I am not trying to be pro- or anti- crypto, just trying to understand. National currencies would seem to have a similar "no underlying value" state. I guess I'm just not clear on what the "underlying value" concept gets us.
National currencies do have similar problems, but they also have a floor. Those countries will accept the money for debts, including taxes, which means the value is at least attached to the stability of that country. This is also one of the reasons that hyperinflation can exist, when a country is unstable and/or willing to devalue their own money.
Think about this, why is BTC worth more than Ethereum is worth more than fly-by-night new coin? There's nothing fundamentally different between BTC and [picks something cheap, randomly] ANKR, while one is worth upwards of $15,000 and the other is worth two cents. Whatever use value a digital currency has (which has a troubled history but could theoretically work) is probably closer to valuing it at the two cents than the $15,000. That's the underlying value. It's essentially worthless - it costs more to mine a Bitcoin in electricity than it's worth from a use-case. The speculative value, on the other hand, can be quite large. The problem you are having is a deeper issue, but I think one pegged to a losing proposition. Yes, you can make money on crypto by reading the room (is the price going to go up or down) and buying at a lower price than it will be in the future. Unfortunately, nobody has any real idea about whether it will actually go up or down, and more importantly, by how much. Maybe BTC will drop to $12,000, or go up to $20,000, or a million dollars. There's absolutely nothing above some minimum price (surely less than $1 each) that helps you track what the true price should be.
Check out the long term volatility and tell me there's a natural price that makes sense, for which you could argue that buying below that is somehow a good buy, but above that is too much?
https://www.forbes.com/advisor/investing/cryptocurrency/bitcoin-price-history/#:~:text=It's%20sometimes%20easy%20to%20forget,nearly%20%2469%2C000%20in%20November%202021.
I see what you're saying, and I'm not saying I know what the "right" price should be either, I just don't know that underlying value is helping me price other things either.
Re: "Those countries will accept the money for debts, including taxes, which means the value is at least attached to the stability of that country. "
There are lots of coins that are attached to companies that will indeed give you services in exchange for those coins. And obviously a small company with a few dozen people is vastly less stable than a country with millions of people, and indeed those coins seem (to my ignorant eye) to act like company stocks trading on the value of those services. Does that count as underlying value? (no matter how small)
Would add https://forum.effectivealtruism.org/posts/rJ4cG9xcKPsqJmLTv/deontology-is-not-the-solution to your list of good takes.
"To be sure, it seems as though Bankman-Fried didn’t take side-constraints particularly seriously. But this doesn’t necessarily mean that he would have done any better had he taken them seriously. Intending to obey deontological constraints is no more a guarantee that you will obey them than intending to maximise utility is a guarantee that you will maximise utility. What’s needed is a more systematic analysis of how to act in situations of power, potentially including proposals for reform at the institutional level—not simplistic rules drawn from intuition."
" ‘heroes put the entire group, many innocent people, ‘the city,’ planet Earth or even the whole damn universe or multiverse in grave danger to save any main character because That’s What Heroes Do’ is ubiquitous in our fantasy media...it is rather mind-bogglingly awful. That kind of thinking needs to be widely condemned."
I get what you're saying. But didn't you also say earlier in this article
"Do not violate deontology for utilitarian ends. Seriously. Stop. Don’t do it."
And while as You say, Game Theory means not bargaining with hostage takers, the above formulation seems like you would be ok with the whole risking the city/world/multiverse scenario If the context were a bit different?
"Batman, this civilian has been turned into a living sun, If You don't kill her she will emit 10,000,000 degree heat and kill countless people. What do you say to that, Batman?"
"I won't kill her. Do not violate deontology for utilitarian ends. Seriously. Stop. Don’t do it."
You might say this is a heightened bizarre scenario totally divorced from reality and so the logic beneath it can't teach us anything. But then, if you're allowed to logically analyze one set of bizarre comic book scenarios, you're allowed to logically analyze another, no?
Batman 1: I have a prototype fusion reactor at Wayne Enterprises. Quickly, contain her while there is still time and she can power the world!"
Batman 2: Then she is already dead, and we must save everyone else while we can.
Batman 3: With that kind of energy output she will consume herself within micro-seconds! Drop her down the deepest abandoned mine in Gotham.
Batman contains multitudes :) (And hell, I don't even think it is always wrong to kill people.)
Amusing.
But the reason I said "living sun" instead of "living time bomb" or somesuch thing was the implication that by some absurd comic book physics the civilian would not be killed by this power even as it was killing countless other people.
And in that context, I don't think that Batman 2 canonically exists. At least, if what Batman 2 is saying is supposed to be a prelude to justified homicide in the name of the greater good, then I think that's not actually something canonical batman would do. He doesn't kill the Joker even though he's wracked up a body count in the millions by this point. He's certainly not going to kill an innocent person, even if it seems almost certain that doing so will save hundreds of thousands of lives.
Though my subcomment here is merely about character interpretation, not my view of ethics.
Well, that brings up two problems with fiction, philosophical hypotheticals, and comic books:
1: In a very different world (where people might become living suns and be unable to keep themselves from melting the world) there would be very different ethics based on the entirety of that world. Is Batman's deontological commitment to not kill one that extends to allowing the entire city/planet to get killed by following it? Most Batman (to my limited knowledge) does not have those kinds of stakes, where killing someone quick solves the problem. Which leads to...
2: Does Batman's deontology actually include not killing someone who will imminently destroy the world? Maybe not doing it if he has any other option, but it seems likely that he would plan for that final requirement. He did canonically spend a lot of time planning to defeat Superman, after all. We always see Batman opt for not killing when, honestly, killing would probably be better over all (see: Joker). Then again, Joker isn't an extinction level event, and if he was, it isn't clear to me that Batman would say "Whelp, nothing to be done, can't kill." My expectation would be that Batman would have a backup plan to kill Living Sun at the last second, in case the whole non-killing plan didn't work.
Then again, we will never know, because Batman doesn't exist as such, and is just a character in a fantasy world written by a variety of people who take things like this more or less seriously. *shrug*
Small thing: Not 100% certain, but I don't think "$5bb" is accepted notation. "$5mm" exists because it's m x m, where m = 1000. So, 5bb would be a 5 trillion or 5 quintillion, by analogy.
Anyway, I think you'd just write "$5b" or "$5B".
Re “Notice this VC ready to invest in SBF again if he asked”: this guy doesn't seem like a VC at all; if he is, he's definitely not anything like a top Silicon Valley VC. His Twitter bio says: “Chairman of O’Shares and Beanstox, CNBC Contributor,4 X Emmy Shark Tank's Mr. Wonderful, wine maker, watch, crypto & guitar guy, photographer & Chef”
> I do find his ‘the biggest thing you actually buy is a car’ attitude odd given the existence of houses.
I'm guessing you don't own a house.
As I tweeted at the time, a house is more of a strategy than a possession. Once you have a mortgage, especially one close to your maximum, it dominates ALL your financial decision making. It may seem like a house doesn't fit Rao's narrative but that's mainly because the bureaucracy that manages your home ownership is "you and your spouse".
That's true if you are middle class. It is NOT true if you're rich.
E.g. regular person buys $500k house, puts $200k down, plans around mortgage.
Rich guy buys $3 million house (or even $30 million house). Takes out a mortgage because why not, tax advantages and interest rate optionality, but it does not dominate his thinking. In the 8-figure net worth range a lot of people start buying vacation homes or second homes.
I'm interested in your flow of "long version", "short version", "shortest version." I would've thought it was better to go the other way so you can build up an incrementally more detailed mental model. Like a progressive jpeg. Do you know something I don't?
Maybe the reasoning is that if you start with the shortest version, no one will read the long version?
This way you can start on the long version and if it is to long for you skip down a bit for the short or shortest version.
Think of the short version as a summary after having read the long version, to cement understanding or give you a basic understanding if long version was unclear.
Both Silver and Jason make good points here. I also don't want to anchor the reader on the short version, my take of which steps are most important, first.
Zvi, what is your advice to a software engineer in the AI compiler/infrastructure space? Not directly working on improving ML capabilities, but building tools for it. Is it possible to do anything good from within or only option is to get out?
Thanks for all your work, particularly the covid coverage. Love the blog.
I think good things are possible. I think that it is VERY easy to fool oneself on this and end up doing bad things instead, and 'I'll do it instead of someone else and I will care about doing it safely' isn't good enough.
Trying to find those good things as best I can. Please keep the AI coverage coming!
I find it weird that still after all this time, SBF basically being a partisan hack, with donating to the Democrats being literally his main 'altruistic' cause, is still apaprently fine with absolutely everybody remotely related to the EA movement. Like, donationg millions to a party that recieved financial and other support from a majority of the largest corporations, all of the most prestigious universities and a majority of newspapers etc when you're being used as almost the face of EA or earning to give (80,000 hours literally listed him first on their 'About Us' page of people whose stories and support they are 'proud' of and mentioned his donations to the Democrats as main (only?) example of his "altruism").
Maybe you can make some sort of case for keeping Repubicans out of power truly being of supreme importance, but I think that that is not only kind of dumb, but also very distasteful for EA orgs and certainly makes me want to have nothing to do with them/actively cheer for their failure. And if nothing else, it strikes me as extremely ineffective altruism. Even being a huge donor, what material impact on elections would SBF reasonably have expected to have (vs other "altruism" options)?
It would have been one thing if these orgs just pinched their noses and accepted it for the sake of the cash, but it's now clear that they at least had no problem with it, and probably thought it was a good thing. Rob Wilbun etc blocking anyone on twitter who disputes the left wing narratives espoused in his tweets (e.g. basically being a heritability denier) certainly supports this, which is all the more reason these orgs losing funding should be considered a truly wonderful thing.
SBF's political actions were more a lot more complicated than 'partisan hack.' Some of the funds went to Biden 2020 and would have gone to Dem '24 if Trump got the nomination, yet the bulk of it was doing something more strategic (whether or not you agree with the strategies or goals involved.) He had specific policy goals, both in crypto and in pandemic prevention, plus at least one thing I can't name but will certify as Good Cause. Take that however you like. Also worth noting that Dustin Muskovitz is a major Democratic donor in way that mostly (as far as I can tell) ISN'T strategic. Again, not to be boo or yay, simply as important context.
Hey @zvi, Have you written about what the possible use cases for crypto? (I have some sense from Vitalik, but frankly I only understand about 60% of what V writes) I know block chain has uses, but crypto itself (as in tokens on its own blockchain) still seems limited, so would be interested in what you might be seeing? Also, do you agree on the some of the current thinking that crypto has turned out to be "centralising" whereas AI (eg Stable Diffusion) has turned out to be decentralising?
If you think AI is decentralizing, I have what I am guessing you will consider bad news. AI has been massively centralizing, giving huge advantages to big companies with big data available and big resources to use it. Their algorithms have the edge now, and also they are the ones who can influence the AIs. I expect such things to accelerate (and if AI does end up decentralizing, I'd mostly update in favor of us all definitely going to die from it).
Crypto doesn't seem centralizing to me, it's more that crypto started out with this anarco-capitalist dream of full decentralization which is something no one wants for many different reasons.
As for its use cases, haven't written about it, basically stayed away from crypto entirely until this other than regarding my game. You can think of it as crypto being the ability to have a public record. At all. Of anything. Without crypto, any public record is whatever the Powers That Be decide is the record, so it isn't a public record. Crypto changes that, whatever the blockchain says happened, happened. That can be tracking who has the money. It can be who said or agreed to what or did what, when. It can be hashed messages, or proving who you are. It can be claiming ideas. That's the core thing that has value, as I understand it.
Which is why e.g. Emergents wants to be on a blockchain. You want a Public Record of who has what cards and what attributes those cards have, that doesn't rely on a central authority (us) or allow that authority to change it whenever we feel like it.
Ah. I don't have a strong view on if AI is decentralising. But I read some thinkers who now think it might be. eg Ben Thompson (see here: futher down - https://stratechery.com/2022/narratives/ but I think Ben T is talking about a different form of it - eg Stable Diffusion in the hands of others. But I hear your argument that big tech / orgs now have the adge on their aglorithms etc. on influence AIs. I definitely dont feel I know enough about AGi etc. execpt some seeming experts in the field have non-zero worries about the area.
Note your argument on the value of a Public Record.
Completely, aside, I wonder if there is a small love story type element to the FTX/SBF/Caroline saga - it seems maybe not - but I'm interested in the flawed human decisions which led here.
re: "hot takes on ethics", I wish everyone would take more care to distinguish ethical theory and practice. See: https://rychappell.substack.com/p/ethical-theory-and-practice
Utilitarianism is an ethical theory, so it's something close to a category error to say that "something is deeply wrong with utilitarianism in practice." Maybe you mean that something is deeply wrong with how some people attempt to implement utilitarianism in practice. This makes clearer that the problem is with the people, and how they were thinking about the theory. There are better ways of thinking about it available (which are in fact central to the utilitarian philosophical tradition). I offer an outline (with academic citations) here: https://rychappell.substack.com/p/naive-vs-prudent-utilitarianism
I think the whole enterprise was a failure to incorporate very well known reasons to doubt FTX and SBF specifically. The vast majority of the financial world did not invest in FTX or crypto. Instead of taking that signal and updating on how good FTX was (not very), it appears that pro-crypto-investment individuals artificially cut off the other world of finance and only considered pro-crypto-investment perspectives. Let's say 90% of the financial markets are in a clear "crypto is dangerous and has no underlying value, don't buy this" and the other 10% is somewhere between willing to try it out and willing to invest heavily in it. If that 10% is wise, they'll keep in mind that 90% of the world will not even touch this stuff at all, let alone go long on it. What seems to have actually happened was for the 10% to say something like "Those [90%] don't understand crypto, I'll look at my peers in the know and recalibrate with them as a baseline" and suddenly those buying small amounts of crypto look like conservative investors. If you are willing to be a bit more risky than the most conservative, you'll stake a decent amount of your money on crypto, and still feel like you're only moderately risky. This is an obvious failure mode from the outside perspective. It's not surprising that once people increase their Overton Window on investment risk (at least when crypto is involved), they'll be more willing to overlook pretty obvious red flags. I mean, BC has had super obvious red flags from the start, but it's held value pretty well and made some people a lot of money (unfortunately, on the backs of people pumping money in later). So people tend to ignore the red flags. But can anyone really say that during a bad economic downturn (2008 certainly, let alone 1929) that BC will not lose a ton of value and leave people over-leveraged in it to go broke?
Then you find out things like FTX/Alameda not having complete lists of all of their *bank accounts* or even their *employees* and *employee compensation* and the whole thing just looks insane.