77 Comments

No edit. Delete fat thumb prior attempt and try again.

I just want to say that beyond your uberhigh intelligence I’m impressed by your energy and productivity.

Expand full comment

> Why would you want this negotiation bot?

Another reason is that if people put some work into negotiating a good price, they might be more willing to buy at the negotiated price, because they played a character who tried to get an acceptable price and agreed to it. IRL, if you didn’t need a particular version of a thing too much, but bargained and agreed to a price, you’ll rarely walk out. Dark arts-y.

> Pliny: they didn’t honor our negotiations im gonna sue.

That didn’t happen, the screenshot is photoshopped (via Inspect element), which the author admitted, I added a community note. The actual discount was 6%. They almost certainly have a max_discount variable.

Expand full comment

I only read 40% of this, but I will return to read the other 60% soon! I like the addition of a linked table of contents in the beginning.

Expand full comment

California has an initiative system where bills can pass with public vote. It is mostly terrible in all ways, but for the specific case of AI could it be used for good? If Americans (and specifically Californians) actually want more regulation and protection against AI than lawmakers, would it make sense to create a clearly worded initiative holding companies liable for the behavior of rogue AI? Or do you the effort would end up backfiring?

Expand full comment

On "Denying the Future": the entirety of the French AI Commission report that sets the strategy of France is based on this assumption. Facepalm.

Pause AI France is busy writing an incendiary response to this report, highlighting the obvious conflicts of interest (Yann LeCun, Arthur Mensch, Cedric O are part of the commission) and will throw a press conference to promote it in September in Paris. Stay tuned.

Expand full comment

Podcast episode of this post:

https://askwhocastsai.substack.com/p/ai-73-openly-evil-ai-by-zvi-mowshowitz

I really hope people are enjoying this "mutli-voice" podcast version of Zvi's posts. If you're finding it useful, please do consider becoming a paid subscriber. I love making these posts more accessible, but the human-sounding AI voice does come with a bit of a cost - about $0.24 per 1000 characters. No pressure at all, though if you do chip in, feel free to suggest any interesting articles you'd like me to consider for future episodes. Or if you prefer, there's also a ko-fi link https://ko-fi.com/askwhocastsai for one-time tips. Either way, I'm happy if I'm helping make these valuable posts more accessible to more people.

Expand full comment

Zvi, I would also like to bring to your attention this article by James Grugett, co-founder of Manifold. https://jamesgrugett.com/p/software-automation-will-make-us. I found the take very interesting, I would be curious to hear your opinion of it. I suspect it went unnoticed because of the weird title.

Expand full comment

> The city knows who they are and used to police them more actively, but now that’s become politically infeasible.

Nothing is politically infeasible. The “defund the police” extremism of the past few years has died down. Time to bring back and expand stop-and-frisk.

Last year in NYC, a non-hispanic Black was more than 25 times more likely to murder someone than a non-hispanic White.

Expand full comment

How about we start with you?

Expand full comment

Ziv writes that according to a former DC City Administrator, most of the violent crimes are committed by a small subset of the population. It is logical that people who commit more crimes should either be in prison, or be more closely monitored by the police.

As an example, meet Steven Hutcherson. This gentleman tried to kill two teenage girls visiting from Paraguary at Grand Central last Christmas, after yelling “I want all the white people dead.” Mr Hutcherson has had 17 prior arrests (!). This is the kind of man that should be stopped and frisked.

https://www.newsweek.com/steven-hutcherson-christmas-grand-central-stabbing-1855675

Expand full comment

I think you are the type of person should be stopped, frisked, and closely monitored by police.

Expand full comment

Why?

Expand full comment
Jul 19·edited Jul 19

Do to others what you want them to do to you. This is the meaning of the law of Moses and the teaching of the prophets.

— Matthew 7:12

Expand full comment

No, he should just be incarcerated. 10 years for 10 strikes is an underrated idea.

Expand full comment

He definitely should not have been roaming free.

I’d suggest the following: as long as someone is clearly not able to live in society, that person should be “inside,” either in a prison, psychiatric institution, or rehabilitation facility.

Having been arrested 17 times clearly fits the bill. (How many times was he caught by police but not arrested simply because such arrests don’t lead to anything?)

Some might see that as unfair, but the current situation is clearly unfair for all those involved:

- the 2 girls who got stabbed

- all the passerby’s who have to worry that this will happen to them

- if someone fights Steven Hutcherson, that person could be charged by the DA

- Hutcherson would presumably be better off inside

Expand full comment

Yes agree, strongly. We should err on the side of harming the person who has been arrested 17 times, and it really shouldn't be a close call.

Expand full comment

>Yes, as Samo Burja says here, common beliefs about things involving a potential technological singularity are a good example of how people’s beliefs, when articulated, turn out to be legitimately crazy. But also the common ‘elite’ or economic view of AI’s mundane utility in the medium term is far more insane than that.

Interested in hearing what you took Samo to mean by the median view on the technological singularity here? I'm not sure he's clearly stated his p(FOOM/apocalypse), although I took his tweet as thinking that it was insane. But he is very clear that he thinks that the economic/"take all our jobs" stuff is also overblown (without necessarily denying utility for those jobs with actual productive value). As far as I understand it his view is that you can't automate jobs that exist for political patronage reasons rather than economic utility reasons, as that misunderstands the point of most white collar/knowledge jobs. His view is that we stumbled onto UBI some time in the post-war period.

Expand full comment
author

I mean, the standard view is 'lalalala can't hear you not listening that can't happen,' so I took him to be (correctly) saying that view is insane.

I understand his view about jobs in the meantime, until that happens. I disagree with it, but it is possible, especially if things move in slower motion than I expect.

Expand full comment

Possibly, but I wouldn't be surprised if he was thinking of the median "SF elites" view and thought that the "idea that there will be a singularity" soon was insane. He very rarely discusses it, and in general his theme is we're much closer to civilizational collapse than take off. Would be pleasantly surprised if I was wrong but may be worth clarifying.

Expand full comment

I can't get over how dumb the Burja take on jobs is. One, "some people have dumb jobs" is not UBI. Just ask anyone who's been laid off. Two, most dumb jobs are there because no one could figure out how to get rid of them, not for political reasons. Companies and managers are primarily incented to make money, and pointless jobs do not make money. There are exceptions for empire building and whatnot, but the relevant point is that they are exceptions. In general, people being idle at work to some degree is a failure of management, often a temporary one in the big picture, rather than an attempt to provide a sinecure.

Expand full comment

I'd say that you're looking at too narrow a definition of "dumb" jobs here. There are plenty of people who do jobs which are marginally of very net social or economic value but also work extremely hard. Look at compliance documentation, bureaucracy, legal documentation, insurance brokers, HR, and so on. Most academics, many coders even. These aren't just sinecures for people to sit and twiddle their thumbs- they really do keep themselves busy. But the total value provided is extremely low because their time is mostly spent satisfying increasingly arbitrary processes.

Expand full comment

I guess this was written up before Trump's comments on Taiwan.

Geopolitics experts in DC are pretty sure that China is serious about wanting to invade Taiwan. They're just waiting for the right moment. How big is the risk of them invading and TSMC being destroyed? I guess we won't have to worry about AI safety in such a world, but we also won't have access to any high performance modern chips. No new cutting edge smartphones, computers or GPUs.

It feels unfathomable. Either Trump will reverse course after the US tech industry and military tell him to, or China will decide it's not worth it at the risk of upending the world economy. A third option is Taiwan surrendering, not wanting to risk it. If that happens, China might restrict the export of chips after they control TSMC and there will be a shortage, but not a catastrophic one.

Expand full comment

We would definitely still have no worry about AI safety in that world. TSMC has a lead, but it isn't a decade, or anything close to it. And a decade delay still wouldn't be anywhere near enough for clearly getting us out of the danger zone.

Expand full comment

Gemini says the new TSM plant in Phoenix should start producing chips next year in 2025. I don't know if it will produce enough--fast enough.

Expand full comment

Hanging / incomplete sentence:

"The good news is that this was mainly done on GPT-3.5 (IN MICE) and when they tested on GPT-4 they got smaller distortions. The optimistic"

Expand full comment
Jul 18·edited Jul 18

Not sure if you highlighted this earlier, but WebSim (https://websim.ai/) is a pretty interesting application of AI. It uses Claude 3.5 to generate fictional websites based on prompts or urls, which you can navigate like regular sites, clicking on links to generate new pages. For example, here's how Claude imagines the Reddit homepage in a world where AGI has just been released: https://websim.ai/c/gBqIeERhAvQLoC4qz

The site isn't very useful yet, although it feels like it's right on the cusp of being something useful- eventually, something like this may allow LLMs to create new interfaces and layouts for different kinds of interaction on the fly, rather than everything needing to fit in the chat UI.

Users have already gotten it to produce some pretty impressive things, like simple 3d games (https://websim.ai/c/GmEOkLEONJ4GlS9Pw) and a synthesizer (https://websim.ai/c/ePG8coAvlnx3jGdDH). As the models get better, being able to just ask for anything you might find on the web and getting a plausible result could become pretty powerful.

Expand full comment

Can someone explain to me why the capital gains tax proposal is so dangerous? I'm obviously not listening to those guys if Zvi says they still manage to go too far while personally believing it will cripple the economy (how far _is_ too far then?).

Expand full comment

A tax on unrealised capital gains would force the sale of assets to pay for the tax, materially depressing asset prices and jeopardising the PE/VC business model which depends on holding assets to accrue large unrealised capital gains that are liquidated on the holder’s schedule.

Expand full comment

...Thanks for the attempt but that didn't help. Why is having such a weird business model good? And where do the predictions for the rest of the economy following suit (rather than just another contained "tech bust") come from?

Expand full comment

Personally, I don't get it either. I think Zvi is being hyperbolic and evincing the very lack of imagination he accuses AI critics of in this post. There's a very strong belief in Silicon Valley that if VCs and founders can't reap billions of dollars from overinflated capital gains then the whole tech industry will grind to a halt. I wonder why they can't go back to the way capital used to work before the 80s and reap their rewards from cashflows, but they're set in their ways.

Expand full comment

The issue is that by forcing early sell-offs, founders won't be able to own a stake in the later cashflows. It pretty decisively kills start-ups as a concept.

It also really heavily penalises volatility - if you're sitting on some stock that spikes up at tax time, you now owe a huge bill, and when the stock goes back down, the government isn't sending you a refund.

Expand full comment

The US press may be different, but high-end UK newspapers have a reputation for highly partisan opinions. but rarely (ish) lying to the reader about basic facts. (The Daily Mail has ... um ... a bad reputation).

So it's pretty easy to imagine an LLM app that takes a Guardian news story, strips out the opinion from the facts, and inserts the kind of opinion you'ld find in the Telegraph. Or vice versa.

(Selective reporting of facts, where they omit to mention something important about the story, happens, but is less common)

Expand full comment

And it was The Sunday Sport that brought us "World War 2 Bomber Found on Moon"

Expand full comment

And "Freddie Starr Ate My Hamster" was The Sun.

Expand full comment

On Claude’s marketing blitz, I do see tons of billboards in the Bay for Claude, although they’re terrible. “A jet pack for your thoughts” doesn’t tell me what I get out of it

Expand full comment
author

Yeah I actually just saw one at a bus stop but they rotate faster than I could take out my phone camera.

Expand full comment

Can you explain why you think unrealized capital gains should never be taxed, please?

Expand full comment
author

I'll explain why this particular proposal kills VC and startups.

They do a good (if oversold) case on the podcast. If you tax 25% of unrealized gains each year, then founders end up selling most of and losing control of their companies, and the effective tax rate on VC investments (since they get hit for 20% multiple times) is prohibitive so they need much lower valuations, and the whole thing falls apart even if everyone is liquid.

Except they aren't liquid. What happens when you owe 25% of your 'unrealized gains' but there is no market for your stock and your company isn't profitable? How do you pay money you don't have?

In terms of the general case and why you indeed shouldn't ever tax unrealized capital gains (also known as: We think you are more likely to earn more money in the future, so you owe us a share of that now), it should mostly follow, but here's a thought experiment.

Suppose you go to college. Should we now tax you 25% of your anticipated NPV gains in income, and charge you that money now? Why shouldn't we tax that social capital gain? Why shouldn't we tax you on what you expect to earn as a partner in a law firm? Etc, etc.

Expand full comment
deletedJul 19
Comment deleted
Expand full comment
Jul 19·edited Jul 19

This is simple. Once you accept tax on unrealized capital gains for people with net worth above 100M, it is only a matter of time before the threshold is lowered (or inflation does its thing) so that the tax is assessed against everyone. There's a credible Schelling fence at `don't tax unrealized capital gains.' There's none at `net worth of 100m.'

Also, being a founder/venture capitalist is essentially being in the business of buying lottery tickets. If you say `ok you can buy lottery tickets, but if you win then we will take the money' greatly reduces the value of entering the lottery at all. Nobody's giving tax credits against unrealized capital losses, after all.

Expand full comment

A slippery slope argument seems reasonable in this case, but I'd hope that people claiming this could acknowledge that the law, as is, probably won't cause the whole startup economy to fall apart.

Taxes would disincentivize investment, but they also disincentivize income, and yet I still work. Rich people are of course fine with taxing me more. Just never tax them, as the economy will explode.

Expand full comment

That misses the point that the ones we care about are the ones that do make it to this stage. Remember Apple without Steve Jobs? Imagine Microsoft becoming what it did without Bill Gates, Google, etc etc.

And it would make financing these an order of magnitude or more riskier. Right now, one can (and the Marc Andreesens of this world have, many times) invest (say) $10m and see it revalued to say $150m before falling to essentially zero. Dem's de breaks, but if you are paying circa $30m tax along the way, it is a very different picture.

One can say "oh, we'd give a credit for losses", but 1. they won't, 2. can you imagine the tax revenue in a downturn year, and 3. the financers tend to do most of their loosing _after_ the big wins (it'sthe big wins that make them able to do the financing), so they may never realise those credits.

Expand full comment

The ones you care about, perhaps, but I don't necessarily agree. Maybe in a different world, we would see more numerous and less concentrated companies that do not rely on massive market capture, and the average person would be better off.

At the least, I really do believe the world would be better off without enabling people like Andreesen. I see everything he's done as a net negative on the world, and would be happy if the laws had prevented his success.

Expand full comment

Thank you, I appreciate the explanation.

Expand full comment

Would the better plan, assuming one accepts the goal to prevent the gains from never being taxed via the buy-hold-die and borrow-against strategies, be to remove the step-up on death and to....I suppose treat using your stock as collateral for a loan as being tax-equivalent to a sale at whatever the collateral is valued at?

Expand full comment

The Biden administration basically proposed this early on — indeed Biden ran in 2020 on a platform of closing all the capital gains loopholes. The proposal was shot down by blue state farmers, who worried about losing access to their generationally inherited farmland.

Expand full comment
author

For clarity: As a matter of public policy, taxing loans that exceed cost basis as if they are capital gains, and ending step-up on death, would be highly reasonable things to do, and a policy of 'lower overall cap gains % but change these rules to keep revenue overall constant' would probably be a good policy.

(Also I would not much care if there was a carve-out for small-scale farmland, that is not that big a % of the issue and we subsidize them anyway, just give them less other subsidy).

Expand full comment
Aug 8·edited Aug 8

Just removing the exceptions on death would in the accounting sense entirely solve the issue; delaying taxation isn't that big a deal - the government can borrow money very cheaply - it's the places where the tax gets avoided entirely that are problematic.

Maybe people would lobby for there to be no taxation event on assets inherited but not sold, but the "last sold at" number should never update without tax being paid accordingly.

Expand full comment

> Except they aren't liquid. What happens when you owe 25% of your 'unrealized gains' but there is no market for your stock and your company isn't profitable? How do you pay money you don't have?

Obviously, they will come gun you down like a dog. Or, more likely, you just say that your company's stocks aren't worth that much because they quite literally aren't?

> Suppose you go to college. Should we now tax you 25% of your anticipated NPV gains in income, and charge you that money now? Why shouldn't we tax that social capital gain? Why shouldn't we tax you on what you expect to earn as a partner in a law firm? Etc, etc.

We already increase taxes when the value of your house/land increases, before you actually realize the increase in value, in what are known as property taxes. This seems more closely analogous to unrealized capital gains. In fact, it seems worse, since property isn't all fungible, whereas stocks are - if an Amazon stock is trading at $183, then I can almost certainly sell it for $183, or at least $182. If my neighbor sells their house for $100,000, there's no guarantee I'll be able to sell mine at the same price.

Expand full comment

No that's totally different, we should tax land values more because land is so totally different to stocks. Notably, you can, and people do, create new stocks all the time.

But very few people create new land and even less in high land value areas.

Higher land taxes reduce the value of land and encourage turnover to higher and better uses, this is unbelievably good actually. Higher wealth taxes don't have any such effect, in fact they reduce the risk premium without reducing the risks and drive investment to less productive assets (bonds, "safe" shares).

Expand full comment

Property taxes are not just on land.

Expand full comment

No, but my comments were on taxing the land component, I would not tax the building component at all. If it helps think of my comments as assuming a split rate system with no change to the building rate but an increase in the land rate.

Expand full comment