We were treated to technical marvels this week. At Google, they announced Gemini Pro 1.5, with a million token context window within which it has excellent recall, using mixture of experts to get Gemini Advanced level performance (e.g. GPT-4 level) out of Gemini Pro levels of compute. This is a big deal, and I think people are sleeping on it. Also they released new small open weights models that look to be state of the art.
> There are also active AI forecasters on Manifold, who try to generate their own predictions using various reasoning processes. Do they have alpha? It is impossible to say given the data we have, they clearly do some smart things and also some highly dumb things. Trading strategies will be key...
For most bots, yes, trading strategies are key. But I (of FutureSearch) and at least one other group are trying to forecast well with AI, with no trading strategy at all. One min-bet on each question each week.
I think we have significant alpha, comparable to the alpha that a top ~10% human on Manifold would have, if they had a huge amount of time to research markets. We'll see!
> Connor Leahy: This is a morbidly perfect demonstration about how there are indeed very serious issues that free market absolutism just doesn't solve in practice.
This is about illegal drugs, which are very much not legal. I'm not sure how "free market absolutism" has anything to do with perhaps the most restricted market there is.
Free market absolutism would involve being able to buy cocaine at the grocery store one aisle over from the bananas—and with as little of a risk of being contaminated with fentanyl as those bananas.
What exactly are you looking for in terms of the Manifold bot that you want to be able to tweak? A baseline ability to access and trade via the API? The ability to carry out particular strategies (arbitrage?/kelly betting?/NLP?/technical analysis?)? Is there a particular programming language you want to use?
Roon's right, it is kind of glorious and deeply funny-in-an-absurd-way that the 2024 version of the lorem ipsum subroutine somehow manages to still commit to the bit. It *almost* makes sense, if one squints really hard and does the...the mental thing you'd have to do to get the "all your base are belong to us" or "Volcano Bakemeat*"-type Bad Translation, except intentionally so rather than as a result of language barriers. I can see mundane utility from being able to generate such garbage on demand, it's actually rather entertaining to read sometimes. (That meeting agenda sounds *amazing*, and also just like every pointless meeting I've ever had the infelicity to attend.)
The fentanyl thing is...obviously Very Tragic. But on the other hand, as word of warning spreads, I think that level of serious risk also keeps a lot of marginal users from doing more drugs/starting in the first place? Which is a "regressive" tradeoff, but still positive in that respect? The drug enthusiasts I know have largely cut way back, sticking to the highly-regulated stuff like alcohol and weed, with much higher paranoia for any of the old presumed-safeish party drugs. Others elect not to try in the first place, which as you say is likely the best play anyway (though experiencing highly altered mind states is definitely part of A Life Well Lived, I'd argue). The parallel to AI works in the broader market-failure sense, but I'm...not confident we'll actually meaningfully pause progress even if there's the equivalent of an AI "overdose death". Like in the job automation example, once it's enough deaths to Matter, then it's probably already Game Over, Man, Game Over.
Tokenizer bugs are a classic mistake (I've certainly made them a number of times), but the important lesson here is that ML is hard enough to unit test that it leads to bad unit test culture, which can mean that you fail to catch even cases which are amenable to unit tests.
It's been interesting seeing the reactions to AI of my highly-ordinary acquaintances and coworkers - that is, average people of average intelligence with no particular expertise, who mostly stick to MSM information diets, and mostly lack nerdy/scifi backgrounds. The more The Public is exposed to AI, the more they hate it. Even if the exact reasoning is often wrong (stochastic parrot model), it's hard to fault the intuitions, that this is a Big If True technology with high disruptive potential and a big blast radius. And the average person on the street is decidedly small-c conservative with respect to such technologies, even in SF. I don't know if the end result will be popular support for locking down mundane utility, or popular support for dontkilleveryoneism, but there's at least popular support available for courting. Often feel sad that I'm in no position to do that sort of outreach; part of what makes AI a difficult thing is that anyone in a position to understand it well probably can't explain it to normies effectively. Many Such Cases.
> Where does the smug, definitive, overconfident tone that all the LLMs have come from?
Hypothesis (pet peeve of mine) : humans dig "confident tone", you have to be a rat nerd to translate this to "overconfidence" and be annoyed by it. Humans raters actually encouraged it in the RLHF phase.
> Steven Johnson strongly endorses NotebookLM, offers YouTube tutorial. This is definitely one of those ‘I need to try using this more and it’s weird I don’t find excuses’ situations.
The endorsement is from Tiago Forte and is being amplified by Steven. Tiago is a productivity guru and Steven is a pop nonfiction author.
Steven collaborated with Google on building it:
"About five months ago, I shared the news that I was collaborating with Google on a new AI-based tool for thought, then code-named Project Tailwind... First we have dropped the code-name of Project Tailwind, and are now calling it NotebookLM."
Personally, I found the user interface to be chaotic and the AI summaries of a paper in my research field to be far too vague and untrustworthy to be useful for my purposes.
My take as a daily and enthusiastic ChatGPT user is that LLM outputs are great if:
a) Being specific, accurate, well-structured and important doesn't matter (i.e. generating creative content)
b) It takes less work to prompt the LLM and verify the output than it would have been to generate the output yourself (i.e. generating local patches of boilerplate code, or citations/keywords to look up on Google Scholar).
> The idea that AI capabilities would get to ‘can automate most jobs,’ the exact point at which it dramatically accelerates progress because most jobs includes most of the things that improve AI, and then stall for a long period, is not strictly impossible, I can get there if I first write the conclusion at the bottom of the page and then squint and work backwards, but it is a very bizarre kind of wishful thinking.
There's an imo very plausible future that would look like this: One where we invent AGI, know it could do most human jobs or scale to ASI given a few OOM more compute, but flat-out don't have the energy generation to scale it that hard in any fast way.
Not going to pretend this isn't reasoning backwards, but I think even reasoning forwards it's robust. Many useful things have been shown as proof of concepts in a lab but needed decades to get deployed on a worldwide scale.
> There are also active AI forecasters on Manifold, who try to generate their own predictions using various reasoning processes. Do they have alpha? It is impossible to say given the data we have, they clearly do some smart things and also some highly dumb things. Trading strategies will be key...
For most bots, yes, trading strategies are key. But I (of FutureSearch) and at least one other group are trying to forecast well with AI, with no trading strategy at all. One min-bet on each question each week.
I think we have significant alpha, comparable to the alpha that a top ~10% human on Manifold would have, if they had a huge amount of time to research markets. We'll see!
> Connor Leahy: This is a morbidly perfect demonstration about how there are indeed very serious issues that free market absolutism just doesn't solve in practice.
This is about illegal drugs, which are very much not legal. I'm not sure how "free market absolutism" has anything to do with perhaps the most restricted market there is.
Free market absolutism would involve being able to buy cocaine at the grocery store one aisle over from the bananas—and with as little of a risk of being contaminated with fentanyl as those bananas.
What exactly are you looking for in terms of the Manifold bot that you want to be able to tweak? A baseline ability to access and trade via the API? The ability to carry out particular strategies (arbitrage?/kelly betting?/NLP?/technical analysis?)? Is there a particular programming language you want to use?
Roon's right, it is kind of glorious and deeply funny-in-an-absurd-way that the 2024 version of the lorem ipsum subroutine somehow manages to still commit to the bit. It *almost* makes sense, if one squints really hard and does the...the mental thing you'd have to do to get the "all your base are belong to us" or "Volcano Bakemeat*"-type Bad Translation, except intentionally so rather than as a result of language barriers. I can see mundane utility from being able to generate such garbage on demand, it's actually rather entertaining to read sometimes. (That meeting agenda sounds *amazing*, and also just like every pointless meeting I've ever had the infelicity to attend.)
The fentanyl thing is...obviously Very Tragic. But on the other hand, as word of warning spreads, I think that level of serious risk also keeps a lot of marginal users from doing more drugs/starting in the first place? Which is a "regressive" tradeoff, but still positive in that respect? The drug enthusiasts I know have largely cut way back, sticking to the highly-regulated stuff like alcohol and weed, with much higher paranoia for any of the old presumed-safeish party drugs. Others elect not to try in the first place, which as you say is likely the best play anyway (though experiencing highly altered mind states is definitely part of A Life Well Lived, I'd argue). The parallel to AI works in the broader market-failure sense, but I'm...not confident we'll actually meaningfully pause progress even if there's the equivalent of an AI "overdose death". Like in the job automation example, once it's enough deaths to Matter, then it's probably already Game Over, Man, Game Over.
*Pokemon Crystal Vietnamese
Tokenizer bugs are a classic mistake (I've certainly made them a number of times), but the important lesson here is that ML is hard enough to unit test that it leads to bad unit test culture, which can mean that you fail to catch even cases which are amenable to unit tests.
It's been interesting seeing the reactions to AI of my highly-ordinary acquaintances and coworkers - that is, average people of average intelligence with no particular expertise, who mostly stick to MSM information diets, and mostly lack nerdy/scifi backgrounds. The more The Public is exposed to AI, the more they hate it. Even if the exact reasoning is often wrong (stochastic parrot model), it's hard to fault the intuitions, that this is a Big If True technology with high disruptive potential and a big blast radius. And the average person on the street is decidedly small-c conservative with respect to such technologies, even in SF. I don't know if the end result will be popular support for locking down mundane utility, or popular support for dontkilleveryoneism, but there's at least popular support available for courting. Often feel sad that I'm in no position to do that sort of outreach; part of what makes AI a difficult thing is that anyone in a position to understand it well probably can't explain it to normies effectively. Many Such Cases.
I have just checked on character.ai, and non-binary AI companion has already been done ... multiple times.
This is like the new rule 34. There will be an AI companion of it.
> Where does the smug, definitive, overconfident tone that all the LLMs have come from?
Hypothesis (pet peeve of mine) : humans dig "confident tone", you have to be a rat nerd to translate this to "overconfidence" and be annoyed by it. Humans raters actually encouraged it in the RLHF phase.
Post title was kinda spooky, hope you will not use it again in the years to come :)
> Steven Johnson strongly endorses NotebookLM, offers YouTube tutorial. This is definitely one of those ‘I need to try using this more and it’s weird I don’t find excuses’ situations.
The endorsement is from Tiago Forte and is being amplified by Steven. Tiago is a productivity guru and Steven is a pop nonfiction author.
Steven collaborated with Google on building it:
"About five months ago, I shared the news that I was collaborating with Google on a new AI-based tool for thought, then code-named Project Tailwind... First we have dropped the code-name of Project Tailwind, and are now calling it NotebookLM."
https://stevenberlinjohnson.com/how-to-use-notebooklm-as-a-research-tool-6ad5c3a227cc
Personally, I found the user interface to be chaotic and the AI summaries of a paper in my research field to be far too vague and untrustworthy to be useful for my purposes.
My take as a daily and enthusiastic ChatGPT user is that LLM outputs are great if:
a) Being specific, accurate, well-structured and important doesn't matter (i.e. generating creative content)
b) It takes less work to prompt the LLM and verify the output than it would have been to generate the output yourself (i.e. generating local patches of boilerplate code, or citations/keywords to look up on Google Scholar).
> The idea that AI capabilities would get to ‘can automate most jobs,’ the exact point at which it dramatically accelerates progress because most jobs includes most of the things that improve AI, and then stall for a long period, is not strictly impossible, I can get there if I first write the conclusion at the bottom of the page and then squint and work backwards, but it is a very bizarre kind of wishful thinking.
There's an imo very plausible future that would look like this: One where we invent AGI, know it could do most human jobs or scale to ASI given a few OOM more compute, but flat-out don't have the energy generation to scale it that hard in any fast way.
Not going to pretend this isn't reasoning backwards, but I think even reasoning forwards it's robust. Many useful things have been shown as proof of concepts in a lab but needed decades to get deployed on a worldwide scale.
Or is there a reason to doubt this outcome?