Hours after Google announced Gemini 1.5, OpenAI announced their new video generation model Sora. Its outputs look damn impressive. How Sora Works How does it work? There is a technical report. Mostly it seems like OpenAI did standard OpenAI things, meaning they fed in tons of data, used lots of compute, and pressed the scaling button super hard. The innovations they are willing to talk about seem to be things like ‘do not crop the videos into a standard size.’
Obvious immediate use case seems to be stock footage or generic b roll. If you want a 10 second aerial shot of waves on a beach for your drug commercials voice-over I think Sora can already handle that. I don't know how relevant that will be in practice since I suspect there are already massive libraries of every kind of thing like that you could want available for quite cheap.
I wonder if a central use case for ‘random cool video vaguely in this area of thing’ would be porn. I mean, you can imagine the possibilities. Also wonder to what extent your point about “real people actually did this” also applies here. It’s been a while, for me, but I would think the lack of this would become an issue.
People are not creative and won't produce their own shows, but I predict we will get lots of good adaptations soon™️.
The material already exists. There are lots of examples of good adaptations. Visual is there. We're only missing consistent context + audio over a 90 minutes length. This is going to be a copy-right nightmare, so will most likely come from the open source world.
"Please give me an adaptation of the Dark Tower. Make it good this time, in the style of Sin City."
Hello, insane person here who wants to watch(mostly read tbh) content made by AI according to my own prompts. Despite not being able to replicate actually existing media that well, Generative AI has a denser set of points in Media Space such that for some random point, the closest thing is most likely AI generated rather than in the existing corpus of media. I expect there’s a continuum based on how much people value quality versus position in said Media Space.
In terms of predicting the pace of improvement, I didn’t think especially deeply about video generation but this lines up mostly with my timeline as evidenced by my bets on the Scott Alexander Manifold movie generation market. As time goes on, I am losing faith in the EMH due to consistent poor predictions on AI. I expect people will get more experience over the next few years and errors will stop having a directional bias.
I agree with your overall take. This is important due to being yet another piece of evidence pointing to the scaling law picture of AI progress. But I don't think AI video generation is commercially lucrative or transformative on its own.
Whenever you have a new technology, you have think about commoditized complement: what skills complementary to video generation just got more valuable? The answer is "artistic vision": the ability to tell a compelling story. But my impression (based on years of consuming Youtube) is that artistic vision has already been the bottleneck to producing new quality content. So making video production cheaper and easier doesn't change anything.
Note that this is very different from music production example that is talked about by Grimes and others. With music, there are non-trivial capital costs in acquiring different instruments and gaining the proficiency to play those different instruments (or similarly hiring professionals to play those instruments).
Obvious immediate use case seems to be stock footage or generic b roll. If you want a 10 second aerial shot of waves on a beach for your drug commercials voice-over I think Sora can already handle that. I don't know how relevant that will be in practice since I suspect there are already massive libraries of every kind of thing like that you could want available for quite cheap.
That video you linked from near is not AI generated, it is from a real Bollywood movie. You should update and block that Twitter profile.
I wonder if a central use case for ‘random cool video vaguely in this area of thing’ would be porn. I mean, you can imagine the possibilities. Also wonder to what extent your point about “real people actually did this” also applies here. It’s been a while, for me, but I would think the lack of this would become an issue.
People are not creative and won't produce their own shows, but I predict we will get lots of good adaptations soon™️.
The material already exists. There are lots of examples of good adaptations. Visual is there. We're only missing consistent context + audio over a 90 minutes length. This is going to be a copy-right nightmare, so will most likely come from the open source world.
"Please give me an adaptation of the Dark Tower. Make it good this time, in the style of Sin City."
Hello, insane person here who wants to watch(mostly read tbh) content made by AI according to my own prompts. Despite not being able to replicate actually existing media that well, Generative AI has a denser set of points in Media Space such that for some random point, the closest thing is most likely AI generated rather than in the existing corpus of media. I expect there’s a continuum based on how much people value quality versus position in said Media Space.
In terms of predicting the pace of improvement, I didn’t think especially deeply about video generation but this lines up mostly with my timeline as evidenced by my bets on the Scott Alexander Manifold movie generation market. As time goes on, I am losing faith in the EMH due to consistent poor predictions on AI. I expect people will get more experience over the next few years and errors will stop having a directional bias.
Oh, and FYI: https://manifold.markets/jgyou/what-will-the-main-commercial-user
Manifold version of Tyler's question. There are some interesting low-hanging fruits, including stock footage + custom backgrounds for websites.
AI Reading of this post:
https://askwhocastsai.substack.com/p/sora-what-by-zvi-mowshowitz
I agree with your overall take. This is important due to being yet another piece of evidence pointing to the scaling law picture of AI progress. But I don't think AI video generation is commercially lucrative or transformative on its own.
Whenever you have a new technology, you have think about commoditized complement: what skills complementary to video generation just got more valuable? The answer is "artistic vision": the ability to tell a compelling story. But my impression (based on years of consuming Youtube) is that artistic vision has already been the bottleneck to producing new quality content. So making video production cheaper and easier doesn't change anything.
Note that this is very different from music production example that is talked about by Grimes and others. With music, there are non-trivial capital costs in acquiring different instruments and gaining the proficiency to play those different instruments (or similarly hiring professionals to play those instruments).