2023 Prediction Evaluations
It is that time of the year. One must ask not only whether predictions were right or wrong, whether one won or lost, but what one was and should have been thinking, whether or not good decisions were made, whether the market made sense.
The main subject will be the 2023 ACX Predictions, where I performed buy/sell/hold along with sharing my logic. The numbers quoted are from mid-February 2023, first Manifold, then Metaculus.
Section 1: World Politics
Last year I thought markets were too confident Putin would keep power. This year I think this is not confident enough and Metaculus is more accurate at 90%. Metaculus is also doing a better job adjusting as time passes. Things seem to be stabilizing, and every day without big bad news is good news for Putin here on multiple levels. I bought M500 of YES shares, which moved this to 86%.
I increased my position later, and won M179. The market had occasional spikes downward when Putin looked to potentially be in danger, and for a while it failed to decay properly.
Looking back, there was clearly risk that events in Ukraine could have led to Putin’s ouster, and he also could have head health issues. It was clear that I could have gotten much better per diem odds later in the year. So even though I won this bet, I don’t think it was especially good, and Metaculus was overconfident.
Getting Sevastopol is a heavy lift. Russia is not about to abandon it, Ukraine has other priorities first, and Ukraine’s ability to go on offensives is far from unlimited even in good scenarios. Metaculus is at 8% and once again that sounds more right to me. I bought M250 of NO here and M250 of NO in another similar market that was trading modestly higher, driving the price here to 13%.
I think this was a good bet. Certainly Russia could have completely collapsed, but even then holding onto Crimea was likely. I won M52 here and M50 in the other market. There wasn’t much decay until the second half of the year, but also things looked good for Ukraine for a while, so I think the market acted reasonably relative to itself.
This spent a bunch of time near 50% in early January, then went down. Once again Metaculus has been consistently lower and it is down at 13%. That feels very low, I’d probably be closer to 20% although I am doing much less work keeping up with the war these days, but 28% is a lot given how things are progressing right now. I bought M250 of NO shares, driving the price to 25%.
I won M92. I think the assessment of 20% here sounds reasonable. It would not have been that shocking if Ukraine had made it to Luhansk, but it was never likely.
Metaculus is at 69%. Here I’m more inclined to lean to the Manifold number, and would want to do research before I committed much. It is not great to be selling 81% shots in prediction markets, in general. I bought 10 NO shares to keep tracking.
This resolved YES. I bought out on September 6 at 94%, losing M4 on net. I am not following closely enough to know what the right price would have been.
Here Manifold roughly agrees and is at 24%, down several percent in the last few days which gives me pause about selling. As everyone digs in rhetorically and literally this does seem like it is getting less likely, and the criteria seems easy to not fulfil, but a year is a long time. I bought M10 of NO for tracking purposes.
On reflection I think this was too high, should have been more like 15%, the two sides have irreconcilable differences and defense is highly favored. Should have bet NO for real. I won M3.
This seems super high to me and an example of something specific that is brought to someone’s attention. It was no surprise when I found Metaculus and it came in at 21%, and that also seems high to me relative to the other markets - if any vehicle can cross on one particular date, this will count, so the condition seems hard to trigger. I bought M250 of no, moving this to 27%, since it was relatively illiquid.
I think this was reliably trading too high, the sales in October at 17% seem crazy good. I bought more NO at several points in the year, and won M171.
Yeah, no, that’s way too high, even if it’s possible to get a rather ticky-tacky evaluation to count, and Metaculus agrees down at 20%. I bought M500 of NO bringing this down to 32%. I’m highly confident the first half of that was right, the second half might not be depending on exact details of what ultimately would have counted.
There were a few scares, but I think I should have trusted my gut more on this and doubled down. My move stuck. I ended up winning M285, mostly on the initial move.
Metaculus says 29% and that seems nuts low to me? In the 20 year period starting in 2001 we had nine detonations, and now we have an actual war as justification. I’m going to buy M10 of YES for tracking, but don’t want to be on the wrong side of an arbitrage for size even if it’s not ‘real’ in any sense.
I ended up losing M160 on this, because I bought more later in the year. Metaculus vindicated the initial YES instinct by going higher for much of the year, and stayed above Manifold. I think we got lucky here, given the various states that could have done a test for various reasons, even if the chance Russia used one in the war was not so high.
I am below 4%. I wish I could confidently say below 2% as well, but I can’t. I bought M100 of NO, that’s about how much I would want given the year-long lockup for such a small payoff.
Yep, M4, that was purely a case of things that expire in a year trading at interest rate. On reflection I still do not think 2% was clearly too low, which is pretty scary.
I am very confident 7% is too high. I have been saying roughly 3% a year for a while, it is now February, and there is no party congress this year. Tensions are a little higher in other ways but I am not too worried here. I am buying M500 of NO, again stopping due to market structure considerations.
Link does not work here so presumably this ended up N/A, but presumably we can all agree this should have resolved NO. I continue to think 3% a year.
Turkey is objecting to Sweden. They are all right with Finland but Sweden and Finland will likely insist on a package deal. The question then is whether Turkey is actually willing to stop this from happening while they are under pressure, plus some percent of random other barrier. My prior is that Turkey will extract some amount of consideration for this, but that such objections rarely actually stop such things from happening - there is a long history of ‘everyone has to agree’ actually happening across Europe. So I mostly agree with the 85%, maybe a tad higher. There wasn’t much liquidity so I bought M100 of YES.
I won M24. My logic here is bad because I failed to ask the right question, which was whether Finland would be willing to join without Sweden. On reflection, yes, under the circumstances it made sense to bring them in, and given that backup scenario this should have traded closer to 90%. If we had been operating under the assumption that Finland wouldn’t join without Sweden, then this was overconfident. So either way, whoops.
In general people like to predict and bet on such changes more than they actually happen in any given year. I am sad I missed out on the really good sales early but this still seems like a screaming sale. I bought M500 of NO bringing this down to 25%.
I won M171. I continue to be confident that a policy of betting no in such situations will win over time. People think such things are more likely to happen than they are. By middle of the year this was down to 7%, half of which is ‘the interest rate.’
It is pointed out on Metaculus that the Tigray conflict had more casualties than Russia-Ukraine in 2022. That war is over for now, but we cannot be sure that will hold. The Metaculus prediction was down at 12%, then people thought about it more and bumped it to 24% without any news, Manifold seems to have shrugged through it. Without looking into details of various situations I don’t feel strongly here and I don’t see enough value in doing the work. I’ll buy M10 of NO for tracking.
I ended up winning M50 because I made December investments when the market didn’t crash fast enough, M2 on the initial wager. This was largely a bet on how likely other wars were to get deadly, especially African wars that everyone ignores but often have depressingly high death counts. I agree that I didn’t know enough to bet here.
This is a fun one. We all know there have been a lot more than 25 million COVID cases in China. The question is whether they will ‘confirm’ them. China has stopped reporting numbers entirely and the number that counts for this question is currently coming out to zero. I am choosing to buy M200 of NO here, because I think the default of ‘nothing changes and the number on the chart says zero for a long time’ seems rather likely.
I ended up investing a total of M400 and won M941. This was a very, very good trade, I think this was a heavy favorite to resolve NO. One can imagine what the other side was thinking, but they were asking the wrong question.
Section 2: Politics
The markets on this are not responding reasonably. Biden gave a campaign speech at the State of the Union that went well, no move. There has been a month and a half with no one declaring against him and no one making a serious move to do so. No move.
He is 62% to win the nomination according to PredictIt. Given the way the process works, most of the time that he is the nominee he is the most likely candidate at the end of 2023 - I’d say this is 95%+. If he is not the nominee, we can divide this into a few categories. If he has health or legal or other major issues that happen in 2024, he would be most likely in 2023, about 2/3rds of such issues happen in 2023 given the calendar. I’m guessing that’s roughly half the time that he isn’t the nominee. If it’s not that, then he simply gets beat in the primary. In those cases, it seems rather likely that he starts out 2024 as the favorite anyway.
So the math coming from PredictIt’s 62% is something like (60% of the 62% that he is the nominee) + (7% of the 19% that he withdraws/implodes) + (13% of the 19% that he gets challenged and loses). That’s 80%, and if anything I think the PredictIt number is low, it seems clear the Democrats are in on Biden here unless something major goes wrong. I’ve certainly given up on trying to move the needle.
I bought M$250 of yes, bringing this to 80%, with my fair something like 83%. It isn’t that correlated to the other markets and I see clear bias so I can be a little more aggressive here.
This was at 92% by April, and I bought a bunch in the 90s late in the year after it was very clearly health issue or bust, winning M132.
The calculation here looks good. The key question is whether we should have been far more than 62% confident he would get the nomination. I think clearly yes, and the prediction markets have been and remain to this day insane on this.
Biden is trading at 76% right now at PredictIt. Has there ever been a more screaming buy? Nothing I ever say is investment advice, but that is completely bonkers low.
I am short Gavin Newsom on PredictIt at 18%, it is now 14%. I would sell more now if I could. I don’t get prediction markets being so high on him, but here is one place there is a big difference between him winning the nomination, and the prediction market predicting he is most likely. The chance Newsom will ‘run away with it’ by January is almost zero. If Biden collapses, plenty of people will run and there will still be time. He could still be considered the favorite in a lot of those scenarios, but that would be a lot of polls and campaigning from now.
Is a conditional probability of 40% here for him sensible against the field? To me, the answer is clearly no, I do not think they can stay crazy that long. I bought M200 to move price to 7%, I can’t get excited enough about numbers this low to move it more. My fair is more like 4%.
All right, buying the NO on Gavin Newsome right now is actually even better than buying the YES on Joe Biden. I have no idea what the markets are thinking on this. I got M15 for my trouble.
As of writing this it is a dead heat with DeSantis at 38% each, Haley at 11% having announced. It seems clearly more likely that Trump is the most likely single nominee on 1 January than that he wins the nomination. Here the two scenarios can be ‘Trump or DeSantis wins the nomination’ or ‘someone else wins the nomination.’ In the first scenario, I think Trump is close to 50% to be the favorite going in. In the second scenario, I’d still have to give him at least 25% to be on top that long.
So I have to put the fair price here according to real money markets at something like 45%, and if you forced me to bet one way or the other on Trump vs. DeSantis I’d likely still pick Trump although it does seem close. Alas, the market is not very liquid, so I only bought M250 of YES here which already moves it to 41%. If it rebounded and I noticed I’d take more, potentially quite a lot more.
I only won M465 out of this market and I feel bad for not winning more. This was an easy game, a chance to combine two good bets. I gave the market way too much respect on these, forgetting how dumb political prediction markets can be.
So the theory here is that DeSantis is much more likely than Trump to be the favorite going into the primaries, but he isn’t as good at winning afterwards? I don’t see it. Trump is not good at playing from behind while looking like a loser. This seems some combination of out of date and pure defiance of the real money markets.
I bought M250 of NO to bring this down to 52%.
I got M553 on this one after doubling down a bunch, as the market took way too long to realize it was over. As my notes indicate, I think this was a crazy good play, it’s insanity to think this should have been a favorite under the uncertainty at the time. I don’t think that is hindsight.
The Supreme Court could go a variety of different ways here. Most of them involve what I would call ‘ruling against affirmative action’ and the court is giving every indication they are going to do that. The ‘tighten the scrutiny standard’ option is possible, but I do not expect Roberts to care that much about potential reputational damage here nor do I expect much of it. Given where Metaculus is and the resolution uncertainty this seems very close to me. I bought M10 of YES for tracking.
I dipped in a tiny bit more at 77% later, ended up winning M10. Given what happened with Dobbs it seems like we could have been more confident here.
The actuarial tables combined add up to about 13% for a full year, which should be adjusted downward for what we know of their current health. Would anyone retire otherwise? The conservative justices won’t do it if they have any choice. There’s still some corner cases, but historically they don’t come up much. So the question is whether a liberal will retire or Biden will pack the court. Biden won’t pack the court.
If Sotomayor and Kagan did not retire in 2022 when it made strategic sense to do so, how likely is it that they’d do it in 2023 when they could wait until 2024, which seems strictly better? I don’t see it. I bought M250 of NO to bring this down to 17%, my fair is still a few percent lower than that.
That was good for M59, once more I’m a bit of a coward. The incidents with Thomas made it even clearer than usual that you’re going to have to carry these nine out of there.
The argument in favor is that Truth Social is more a fraud than a real social network, it might not even last the year, and if Trump wants to run for President he is going to need Twitter. It will be impossible to resist. I am skeptical. Trump is all about doubling down. Trump is all about not being a loser. Without the prediction markets I’d have put this at 50% at most, likely more like 40%. It’s still tough to sell 62s when Metaculus is at 72%, so I’m only doing the nominal M10 of NO here despite my fair being something like 55%. If it happens, it likely happens late in the year.
We got exactly one and I lost M300 total, because even though I had this weighted towards later in the year there was pitifully little decay, and I continue to bet against Trump participating on Twitter.
No. Not that it is strictly impossible, but it would take something transformational to politics, where the last event that qualifies would probably have been 9/11. A war barely did anything. I can see the Republicans becoming less popular from a debt ceiling crisis or government shutdown, but even if Biden does everything right I don’t see how he gets to even. I bought 500 of NO to go down to 16%.
Should have bought more. On reflection this was essentially a Can’t Happen, would have required radical change, at most should have been 10%. I ended up winning M222.
This is a very typical kind of thing for people to get overexcited about. There are any number of reasons Trump richly deserves to be indicted, and why a regular person doing similar acts would be indicated. That does not mean those making the final decision wish to poke this particular bear. If they did want to poke it, this would have happened by now, and I don’t sense that the classified documents is a hill prosecutors are itching to die on. The Manifold market has a lot of trades and not that much liquidity, so I sold M200 of it down to 65% rather than push too hard. It will likely come back, and I’ll sell more if I remember and feel like bothering.
I lost M195. Given he got indicted a full four times, I have to assume I was incorrect, although one could view the decisions as highly correlated. Certainly various deals could have been struck, or orders given down to not move forward. I do think that as a reference class this type of thing is an easy NO, especially with Metaculus on my side. Oddly, it now feels hard to imagine how we could have ended up not indicting him, unless he stepped down from running.19
Assassination attempts that actually wound the target are not very common, and the number targeted at anyone other than the President looks very small when I do preliminary research here. ‘No matter how slight’ counts as wounded here, it still seems hard for the base rate to be as high as 17% for 10.5 months, and if anything tensions are deflating. Killing Biden to put Harris in charge seems like not a great opportunity in anyone’s eyes. I bought M100 of NO. Also I am disappointed in Metaculus for being too scared to allow this market.
I won M120, doubling down as decay was too slow. Given historical data I am very confident this was too high. I’d put the 2024 line higher than the 2023 line ‘should’ have been, given potential tensions around the election.
If the UK holds a general election, it would take a minor miracle for the Tories to hold on. So the way this isn’t true is most of the time there is an election, plus the times he loses the leadership. There doesn’t seem to be much love for Sunak but there is also no alternative, it’s a thankless job with no good candidates, and there is no obvious precipitating event on the horizon. The relationship between this and the next market seems reasonable to me. However, Sunak’s chances improve every day that things are quiet even more than the reduced time left, since his survival should be somewhat Lindy. Market isn’t adjusting. I bought M200 of YES to move it to 85%.
I didn’t come back to this and only won M37. I think this was modestly low, but the next question on elections at all is most of the risk and I don’t understand the details there all that well.
I never understood such dynamics. The party in power is currently in exceedingly deep trouble if there was an election. Why would they allow one to happen when they don’t have to and they have nothing much to win anyway? The chances here should be quite low. The problem is that argument proves rather too much. I like betting on Sunak’s survival more, especially given where Metaculus is. I’m buying M10 of YES for tracking purposes since I’d be OK hedging by betting on Sunak.
Yep, M10 lost. I think this was a good bet if and only if hedged, but again I don’t really know.
Section 3: Tech and Economics
Even if Musk wanted to sell Twitter, no one is going to offer him a price that lets him save face. He is stuck. There will be lawsuits but those should all take longer than a year. Yes, Musk moves fast and the situation is continuing to evolve and 88% is a lot, it still feels a little low to me so I bought M100 of YES moving it to 89%.
Yeah, 90% seems fine on reflection, on the chance that things could have come apart completely in various ways, but he certainly had no intention of leaving. M11.
Advertising revenue per minute spent on Twitter will probably be down due to worse relationships with advertisers plus economic conditions, although my guess is the effect is modest. Time spent on the site could go either way, as question 29 indicates. Sure, a bunch of people are complaining, but that’s what people do on the internet. I don’t see any substantial change, and my guess is this will be roughly static. So the question is whether Twitter can come up with other revenue streams, as Musk seems very keen on doing. It seems tough to unlock all that much here, so I am not excited, but I also don’t especially want to bet against him and Metaculus is on the high side. I bought M200 of YES, which moved this to 42%.
Whoops. That’s M200 I won’t see again, or would have been if I hadn’t realized my mistake later and hedged remarkably cheaply, so I only lost M10. Whatever one thinks of other metrics, revenue was clearly not a place you wanted to be bullish. Subscriptions were never going to make up for advertising. What I did not properly appreciate was the extent to which advertisers would be willing to ignore their own economic interest under the faintest public pressure. And yet, here we are, with no one stepping up. The whole ‘what if Disney had an add next to a fascist Tweet’ thing was always rather silly, but here we are.
Musk claimed that Number Go Up after he took over. I am not convinced, and my prior before knowing this was a question was definitely in the ~50% range. Given it is Musk who will decide what counts as a mDAU, and he is inclined to want to brag and has several ways to shift the probabilities here, I’m going to give the edge here to growth. I’m buying M100 of YES, brining it to 54%.
This was at 68% as late as October, then data came out from various sources and people strongly felt that the number went down. I lost M330, because I kept seeing claims that this number was going up, only to have people say it was going down. Was Twitter lying? Can we trust the new numbers? Who knows. The real answer looks like it is remarkably close, I’m not that confident this was resolved to physical reality, and having it as a coin flip was probably fine. As opposed to the revenue question, where I was clearly wrong.
We have our first very large disagreement. 5-year TIPS are at 2.44% but somehow never got above about 3.5. Two-year treasuries pay 4.6%. Median forecast of the Philadelphia Fed Survey is 3.1%, and 3.0% core. The Fed has been rather wrong about this for a while now, and my average expectation here is going to be more like 3.5%, but we’ve had about seven months with relatively contained inflation numbers. Where does that put this percentage? Presumably somewhere in the middle between the two numbers. I’m going to buy M200 of YES here (and put M10 on the Yglesias market).
I lost M430 in total after doubling down. I think the 27% number was pretty bonkers, as was getting 13% in June. We got a fortunate result, but it was far from clear that was going to happen. I’m sure I could have hedged this in various ways if I’d cared to.
We have about a 7% lead and in general stocks go up. This seems like a pure math question more than anything, and I’ll be slightly more bullish than whatever the raw numbers predict. AI impact isn’t directionally obvious for established firms. I don’t want to get updates on this so I’m sitting it out, put me down for basic math plus 1%.
On reflection I was wrong. AI impact was pretty damn obvious for established firms, so I should have been basic math plus a few percent, not only 1%.
Once again put me down for math plus 1%, whatever that is.
Same as the previous question.
I’ll go math plus 2% here. Covid dynamics should make this more likely to be a good year for stocks over there.
I do think it was reasonable to think that wasn’t priced in, but there’s a reason I keep my opinions mostly anchored to the market in such places.
It has to beat about $16.6k and is currently $24.5k, but volatility is super high, distributions are asymmetric, and crypto prices do not seem to be responding rationally to recent news - there’s an intensifying crackdown and all the hype is shifting to AI. We are about $2k higher than nine days ago when this market hit this level, in general it does not seem to be efficiently to crypto prices either, and I definitely can’t be bothered to try and do the math here for real. I don’t want updates so I’ll avoid trading, my fair here is probably in the mid-high 60s.
We closed above $40k. That indicates it was right to be short. If variance is that high, then you are much better off buying Bitcoin than you are betting that Bitcoin will go up. These markets consistently reflect that, I’d be tempted to do the (hedged) trade for size if I could. If you think Bitcoin is more likely than not to go up, it is a screaming buy, it can’t go below $0.
Does $16.6k-$30k make sense as an ~50% confidence interval for year-end BTC? I’d guess there is modestly more variance here than that, and this market is clearly not moving in rational fashion as news comes in, so I’ll do a min-size M10 buy on YES despite thinking that the 75% number was likely too high.
On reflection I should have bought this to 30%. This 23% and the 75% above do not live in the same universe. I bought a bit more later when it was even stupider, ended up making M101.
If Tether spends a full day below 98 cents there is a very good chance that the problem is much worse than a small breaking of the buck. The whole thing seems likely to collapse eventually, but it has been going for a while, they’ve survived a lot of shenanigans in the past, BTC is back up a bit, and if Tether was going to be a falling domino from FTX we would likely be hearing more rumblings by now. It is always tough to sell 18% when there’s a 25% on the board and it’s hard to bet against Tether given what I think of Tether, but I don’t think that 25% is realistic number. I bought M10 of NO as a nominal play.
I ended up making M239 getting much better odds later due to time decay, and as the situation became visibly less risky. On reflection I don’t think I was aggressive enough, at least later on. By mid-February, it was already wildly profitable to be Tether. Even if they had previously bled a bunch of capital, they would recover, and everyone knows this, so the chances of a run on the bank sufficient to break the buck here seem very low. I am definitely in the ‘Tether likely did all the crimes’ camp and also the ‘Tether was insolvent and printing money it couldn’t redeem’ camp, but I think even by February 2023 it was clear they’d gotten away with it. And that they could probably print a metric ton without backing now, if they wanted, collect the interest forever, and no one would ever call them on it.
So far, instead of going up, it’s gone down, while inflation has stayed largely under control. It does seem more likely than not it will get above 4% anyway as the Fed keeps raising rates, but it keeps not happening. I’m down at more like 60% that this happens. I bought M100 of NO, which knocked it to 70%.
I won M266. It was close. A lot of economists remain confused how this happened while inflation remained under control. I continue to suspect AI. And also that there is a ‘bunch of ruin’ in the unemployment rate here before it would rise substantially, although we’ve likely eaten through a good portion of that. If this is ‘trading at friction’ then it’s going to be sticky.
The moment for this has passed. I am factoring in some risk that the Dogecoin options at Tesla somehow count, but otherwise this seems highly unlikely to happen. No one would want the headlines that would follow, certainly not now. Somehow this was super liquid so I bought M1000 of NO and it only moved the price 1%. I don’t want to do too outsized a play here but this seems way too high to me.
There were some arguments but this did safely resolve NO and I won M539. This was an argument over grading, not over facts, everyone agreed no one was going to ‘really’ do this all that often, so ultimately not so interesting.
Why would they go back on their word here? A little trouble with A Good Bing? Let us be clear, no matter what OpenAI and its employees say they do not care about such considerations, and they do not care about AI Not-Kill-Everyoneism. Nor does Microsoft. They are releasing this. I bought M500 of YES to get this to 91%.
This was indeed a screaming buy, should have been >95%. Net of M58.
I know nothing here on a technical level, except that there was high confidence this would happen last year and it didn’t, and also one has to be a little less on the ‘Elon’s physical projects are totally going to work’ than one was last year, I’d think? So I’m buying M20 of NO on general principles, but not getting further involved.
It did indeed go off as NO, I did a few other small additions to net M196. I don’t claim to be an expert but this sure does seem like people were overconfident on many levels, including that SpaceX would be allowed a second chance.
Always know your judge. Scott Alexander claimed he had already won the bet. Edwin Chen then wrote a post explaining that no, Scott hadn’t won. I side with Edwin here on the narrow ‘did Scott win yet’ question. A raven on your head is not a raven on your shoulder, that does not count. However, it is worth noticing that Edwin’s response went a lot farther than that, making it clear his effective standard is more like humans reverse engineering the prompt rather than the picture matches the description. Those are two very different standards. It seems clear to me Edwin is not going to agree go down easy. I bought M270 of NO, taking out a resting buy order at 71%, causing me to note that you can sometimes get a better price buying in multiple steps where that shouldn’t be true. Weird.
This is currently sitting at 62% as I write this, waiting for the judge.
I generated two images for each with DALLE-3, got one that to me is a clear yes and one that could go either way, one that might get fixed with more sampling and two that seem like they won’t get fixed. My guess is that MidJourney 6.0 will get there, but again, know thy judge. Given that we got remarkably good AI detail prompting right on time and this was still trading ambiguously, this was definitely starting out too high.
If I was resolving this you could slam the yes, because China, where the death peak was probably in January or at least late in December, and they’re going to be losing people for a long time. Without China, however, the stats suggest that unless there is a new variant we will come in with over a 50% decline, at least in terms of official numbers. This question is thus mostly about how Scott estimates what happens in China, rather than about what is likely to happen in the future. My guess is that 71% is a bit high, but the catch is that this resolves ambiguous pretty often when it depends on China and not when there are enough deaths elsewhere, which gives value to being on the YES side, so I’m going to say 70% seems reasonable here.
This traded as if China’s death count was never that high, in which case the NO was clearly a strong favorite the whole way. As always, know how resolution works.
I mean, didn’t this already happen? The Omicron booster is not for the variants currently in circulation, and I’d say there is ‘substantial’ although partial escape already. So any new version of Covid that comes from the existing dominant strains would indeed quality. Scott didn’t mean the question that way, or he wouldn’t have asked it, which doesn’t mean he won’t resolve it that way. We also don’t know if this means versus infection or versus hospitalization and death. The intent is presumably ‘substantially more than the ones out there at the start of the year’ and even then it seems pretty vague. If I interpret this as ‘the virus once again does what Omicron did in terms of vaccine escape,’ a thing that Alpha and Delta notably did not do, then we have one example of this in three years, and 58% would be way too high, I’d be more like 30% give or take the other ambiguities. I’m putting a nominal M10 into NO and otherwise staying away due to ambiguity.
I ended up making M241 once it was clear this was going to be interpreted in a way that excluded all past events. Analysis seems like it aged well.
Augmented reality has been a next big thing for a while now. It is not an easy thing to do right, and it also is not about to be the new hotness unless it is integrated with AI, which I am guessing will take longer than a year to be ready to launch, and could cause a postponement. I bought M50 of NO here, but there wasn’t much liquidity and I haven’t done the research, so my fair here is still in the 50s somewhere. It does seem like a thing people mean to be doing.
I bought out of the initial NO in the 20s for a nice profit, but then messed up because I didn’t realize about one that ultimately counted, so I ended up losing M182. A remarkably large percentage percentage of my substantial losses at Manifold are ‘I misread the question’ or ‘I missed something dumb’ rather than anything else. On reflection this was clearly in the works, but given what I knew I think I acted well early on before blowing it later.
This does not seem like the style of thing our civilization is good at doing, and in particular is a thing that has time and again proven more difficult to actually implement than it looks. We’ve had self-driving cars moving around for a while in friendly conditions but I don’t see them letting these get injected into heavy traffic at rush hour. I am buying M250 of NO, knocking this down to 21%.
People held out hope for a while, but nope, up M73. Things actually took a step backwards.
I would bet heavily on such a product not being any good, at least by my standards, but that’s not the question, and the $29 product can be something like one piece of sushi so price is unlikely to actually stop this from happening, the same way quality won’t stop it either (see ‘beyond sushi’ which somehow is not ‘cooked fish.’) All it takes is one restaurant serving some tiny sampler as a curiosity. Not knowing any details, though, I’m only going to buy M250 of YES here, taking it to 37%.
I ended up losing M187 after taking some buyout late. I am still mildly surprised this ended up not happening. Now we know the whole area has mostly been a bust.
I keep hearing how this is going to happen. It is an interesting question why it is not happening. The voice copying in particular is scary good. Yet the most I have heard about is some fake porn of Twitch stars, which would not count here because no one involved believed it was real, it was a site selling explicitly faked porn. And That’s Terrible, but not the same level of terrible as people thinking it is real. What is stopping this from being much worse? Is it technical inability, or something else? At some point this surely will happen, but I don’t see the marginal progress in 2023 as more likely to trigger it than 2022 should have seemed if you knew the pace of advancement, or of it landing in 2024, and again something is slowing this down. Then again, the papers are suddenly more likely to highlight a relatively minor harm, and there’s no minimum requirement there, so maybe put this at a little over one-third, something like 35%, matching Metaculus. I sold 200 of NO, bringing this down to 39%.
I ended up losing M153. This stayed around 90% right until the end due to uncertainty over resolution. I’m still not entirely sure which article is considered to have counted, or what damage was inflicted. In spirit, I’m not convinced this actually happened. Who was harmed?
I am not going to challenge the resolution on Manifold, but I am curious what people think was the case that mattered.
The base rate here is rather high, and worth noting the jump to 48% was in the previous day or so. YES would have come in for 2009 (Swine Flu), 2014 (Ebola), 2016 (Zika), 2019 (Kivu Ebola), 2020 (Covid) and 2022 (monkeypox). Note however that starting in 2014 is cherry picking, and Covid is an ongoing emergency so it is unlikely to count as a new one. The chances implied by two other longer term markets are 30% and 40%. Swine flu is now more likely than before, which does bump this up a bit, and you can get an ‘emergency’ there without triggering a bunch of the ‘will this kill X people’ markets. This is trading at 8% (I bought a little NO here), and before recent developments this would have been more like 2%, at most ~4%. So we can tack on another 5% or so to the base rate of 6 incidents in 15 years (counting 2008 for fairness) which would get us to 45%, discounted to 40% for it being mid-February. I bought 200 of NO bringing it to 42%.
I got M20 on the side market and M219 on the main one. On reflection you normally get advance warning of a declaration, so I should have factored in more ‘time expired’ than I did, and also there will often be one potential source that is higher than usual but others will be lower. I think I was too conservative only going down to 42% and this should have been more like 30%.
This has not been rising at Metaculus, although Manifold has been catching up to it. This should damn well be rising, as both progress has been impressive recently with no signs of actual stopping due to risk, and there is now a ton more hype and funding and reason to put on this kind of show, and more incentive for ‘a major news outlet to report it,’ note that this does not require a front page or anything like that. I’m going to put M100 on YES, bringing this to 42%. I notice my gut-level fair is at least 50% here.
No sign that this happened, despite AI being remarkably good at coding, so I ended up down M74 and also was clearly wrong. If AI didn’t do a coding task in 2023 and you predicted it would, that’s on you.
Videos have to be at least 10 seconds long. It doesn’t have to be good at generating them. At pace of progress, it seems highly implausible that this won’t happen as defined. Even if the product isn’t ready for prime time, does that seem like it is stopping anyone these days? The only real risk is Scott saying ‘not good enough’ and I don’t put Scott on saying that without a lot of justification. I bought M1000 of YES here to bring it to 85%, I definitely have a fair in the low 90s similar to Metaculus.
Another clear case of ‘know your judge and know the exact criteria.’ I find all the video generations we have highly disappointing and rather useless, but they definitely count, and we were definitely going to get something that counted here. Ended up winning M200.
Overall Results
Directionally, I did well. In terms of logic, I think I did fine, but with some definite mistakes. Calibration is not something I calculate, once you anchor to the market it’s mostly about the market’s calibration rather than your own.
Of 42 evaluated markets where I traded, I went 30-12, with an average win of +M185 and a coincidentally identical average loss of -M185, due to scaling up and down of various predictions.
I did another pass to see what happens on fixed size. I found about a 58% expected win rate based on initial odds. There were a bunch of things over 80%, but I also did date some long shots. I ended up winning 71% of the time.
The key differentiator was that when I said something the market said was unlikely (25% or less) was even more unlikely, and I laid the odds, the event did not happen, whichever way the market was worded. This extended to other markets I traded in throughout the year, especially ones with odds in the high 80s or in the 90s. It was almost or entirely a clean sweep.
I’m not sure how much correlation was involved here, in the sense that the year lacked crazy events that we couldn’t have seen coming, which tend to cause other crazy events. Probably some.
I assume I ran lucky.
Can I do it again? Next up are the ACX predictions for 2024. Write up of that is coming up next. Whether we do an additional prediction post beyond that is to be determined. Will there be a third post with more predictions after that? That depends on interest, time available and how good I think the questions are.
> Will a successful deepfake attempt causing real damage make the front page of a major news source in 2023 (44%/33%)?
> I ended up losing M153. This stayed around 90% right until the end due to uncertainty over resolution. I’m still not entirely sure which article is considered to have counted, or what damage was inflicted. In spirit, I’m not convinced this actually happened. Who was harmed?
This one: https://www.cbc.ca/news/canada/newfoundland-labrador/deepfake-phone-scame-1.6793296
Your results align with a half-formed intuition I developed from a year of trading on Manifold that I am trying to find a way to express well -- here's one attempt: "outcomes that depend on people doing non-default specific things mostly don't end up happening". Even when they seem perfectly plausible, people seem to just plain do fewer things (especially even-slightly-novel things) than one would expect given the population and the incentives. Maybe this is just due to how big the space of possible actions is (so ~all possible actions stay unattempted), or maybe it is because of how non-strategic humans are most of the time (drifting along following our local incentive gradients)?