Re Anthropic hiring less engineers: there're benefits to developing a project with fewer higher-performing engineers. Throwing as many engineers as you can at the project is not always the best thing that you can do.
Another key consideration: if you were running a very-high-security national security project, you'd be THRILLED at the notion that you had to hire fewer new, less-vetted folks to join that project...
"If Dean is correct here, then the carbon cost of training is trivial." Note that Table 1 in that paper concerned models using 1e19 flops for training. If I understand correctly that's 1e6 times less than GPT-4, so that GPT-4 would have costed 1e5 tons of CO2 equivalents on the most inefficient setup listed in their Table 1 and 2e3 tons if efficient. No idea about inference costs. No idea about lifetime CO2e cost of the GPUs/TPUs.
I wonder if AI generated CSAM isn't a good thing on net that we should, while perhaps not encourage, maybe treat as less bad than real-world CSAM. There is obviously a market for CSAM that won't go away, and if this can be supplied more cheaply and with less risk than the actual child abuse stuff, and without harming children in the process, that seems better than the alternative.
I suppose the most important question is whether this would, on balance, more serve as a "gateway drug" to real CSAM and make CSAM more socially acceptable, or more satisfy pedophilic desires without any actual abuse going on. Also, of course, there is the question where you get the training data, but presumably enough of this exists that this shouldn't be an issue, and perhaps a good enough image generator could extrapolate this well enough without even needing any real CSAM.
One might also surmise that real CSAM could be claimed to be deepfaked to avoid or reduce punishment, which would definitely be an issue with this approach.
I understand that "pseudophotographs" (e.g. AI generated images) are already illegal under UK law, because the legislators anticipated the obvious problem that technology might get good enough that you cant tell whether an image is real of fake.
one way of resolving the hiring issue is the creation of a thing that I still don't get why it hasn't always existed: the applicant side equivalent of a head hunter agency.
There already exist agencies whose job it is to find people for a needed position. Why there aren't equivalent agencies whose job it is to find a position for an applicant I don't understand, but the new AI application world creates a _new_ reason:
These agencies are consistent, repeat players in the game-theory sense. They have an incentive to find _actually_ good candidates for fit. If they are repeatedly sending in people who have lied etc, then they get a bad reputation and companies won't trust them, and they can't place applicants and they go out of business.
And, since these people are hired _by_ the job applicant, they have an ability to actually find out real information about the candidate.
I expect a service like this to be expensive, but it also seems _obviously_ worthwhile.
And game theory optimum with repeated games fixes a lot of the current problems with AI application, AI hiring managers, and incentives to lie.
Job agencies and consulting companies provide this to some extent and some professionals hire or retain agents to do exactly this. Not having publicly accessible work that can be evaluated might mean 'reverse headhunters' can't do much more than edit resumés. Some job listing services do seem to match both sides too.
I guess in my mind, it's the people who don't have publicly accessible work who would gain the most value. If I can pay an agency to basically go through an interview process and find out the information that goes beyond a resume, but isn't easily legible, and if that company has a reputation for placing good candidates, then they can figure out where I'd be a good fit in ways that hiring companies can't afford to do (because finding that out is costly), then hard-to-assess candidates can get better positions.
Theoretically, being able to pay to guarantee an interview would accomplish something similar, except it would have massive perverse incentives.
Some recruiters do this already, tho I think it's only for particularly in-demand and well paid positions. There is also a long line of companies that tried and failed to do this even just for the relatively more legible { computer programming / software engineering }. Even those extremely narrow versions are, apparently, fatally difficult (for for-profit companies).
Maybe it's just an almost-perfectly insurmountable gap in the value between 'this person might be a good hire' and 'this person WAS a good hire – evaluated as-of a year later – by a company capable of putting them to some productive use'.
Think about it this way: Would you hire someone represented by an agent, knowing that as soon as you hire this person, the agent will get to work seeing who else will poach for more?
I don't understand why this would create a new risk.
Why an I going to continue to pay the agent to keep looking? If they think they can quickly and easily find me a better offer, why did they tell me to take this job?
Also, how is this different than the current situation? Why didn't you expect current employees to immediately start looking for a better job?
> Why an I going to continue to pay the agent to keep looking?
The agent works on commission, it's not like you pay them to keep looking.
> If they think they can quickly and easily find me a better offer, why did they tell me to take this job?
You now have more experience, perhaps a better title, higher pay — all these things set you up to be considered for a job that you might not have been able to get before. Or perhaps there is an opportunity that wasn't available six months ago.
> Also, how is this different than the current situation? Why didn't you expect current employees to immediately start looking for a better job?
Well, from the perspective of incentives it's not that different; but since an agent views this as their main job, on average they'll be much more persistent about it. Your agent doesn't care about your work relationships, or the career capital you get from having long tenure, or even really how much you like the job.
To put it another way, most people don't put nearly as much effort into job searching as they should (relative to effort spent actually doing their job) and having an agent changes that.
If you stay in the job long enough that it is _actually_ more experience, then I don't think that the hiring company has much room to complain
But the biggest reason this scheme works at all is that the _agencies_ get a reputation for placing good candidates. That's why companies will trust their assessment at all. But that cuts both ways. If they place a candidate who leaves in 6 months, that hurts their reputation, and the hiring company is less likely to take candidates from them in the future.
The equilibrium is companies placing candidates with an eye towards decently sustained success at the position. Yes, occasionally, it might happen, but any agency that does it too often is going to get a reputation for that and hiring companies won't hire from them.
"I am happy to hear that they are at least noticing that they might not want to release this in its current form."
Screw that. LET'S GO. I'm sick and tired of only the Illuminati Privilege Bros having access to this tech. Lets cut it loose. If we die, we die. It's a completely insane world where Zuck is the good guy, but that seems to be the current state of affairs.
Bryan Johnson has a point. We're all in a race against death and you have so little time. We're all scheduled to die anyways might as well try how far this avenue goes.
I think Hinton was arguably awarded for work that was not-physics, but Hopfield's work was squarely within the scope of physics as generally understood. Hopfield himself seems to think so
Also the condensed matter physics arxiv contains `disordered systems and neural networks' as a subarxiv. I've read it (and contributed to it) for over a decade, and `neural networks are part of physics' strikes me as an utterly obvious and uncontroversial take.
> I find myself excited to write actual code that does the things, but the thought of having to set everything up to get to that point fills with dread - I just know that the AI is going to get something stupid wrong, and everything’s going to be screwed up, and it’s going to be hours trying to figure it out and so on, and maybe I’ll just work on something else.
Yeah, the exact same concern put me off for a long time (and I have not yet fully pushed through it).
FWIW, I imagine the experience _might_ be better if you use JavaScript or Python. I went with Java because I have deep muscle memory for that language, but it's probably an unusual choice nowadays for starter projects. Regardless of whether LLMs would be better at giving troubleshooting advice, I think it's likely that in ReplIt and possibly Cursor you'd be following a more well-trodden path.
'Serious' JavaScript and Python are relatively painful – the 'script' side of both is very nice, but only for very small projects or prototypes.
As a professional, we just pay these costs infrequently, and I and my less excitable or adventurous fellow practitioners tend to pay smaller costs to upgrade a 'boring stack' over many years or even decades because the costs of spinning up new projects using an already setup stack can be driven down very low (which is, as one might imagine, very nice and very useful).
You and Zvi should pay a pro to setup the 'stack infrastructure' for you – your stack, your local dev environments, source/version control, and whatever initial deployments you want to start.
Any suggestions for how to find that pro? I'd be happy to pay.
(For context, I'm a professional myself, have probably written 1M lines of code over my career, but never liked or got good at the setup stage. I left my last company 20 months ago and am just getting back to dabbling with coding.)
If you know another, ask them for their 'cheerful price'! They might be happy to do it just as an excuse to hangout and talk shop.
It seems like you could absolutely do it yourself. Pick a boring standard stack for whatever you want to work with. JavaScript is the Wild West but Python has some 'all in one' packages that might Just Work for you. I don't like Docker, but maybe you'd like it fine.
Otherwise, check for someone using your stack on a (programming) projects site and ask them for what they recommend and a quote to help you get it all setup.
I have just had an idea for an AI application ... something that can answer questions about large codebases, such as the Linux kernel or the LLVM compiler. e.g. "Where in this code is the RISCV option string from the FDT parsed?" (I actually got the answer to that one using the traditional method "grep", but consider this an example of a type of question that you're asking all the time, and can't always be solved with a simple regular expression search.)
For the curious: those fine people in charge of the RISCV specification have ratified some more options, so the question one is of course asking is "Ok, where in the Linux kernel do I need to patch to support this new requirement?"
I once spend three days finding exactly where in the LLVM MIPS backend a bug was, given an example of code that triggered it. (Hint: there were 64 bit MIPS CPUs that were not MIPS64. Problem: find where in the compiler it is checking for target being MIPS64 rather that target being a 64 bit MIPS. Several days later, I have located the offending line of code).
I feel that this kind of thing could benefit from AI sutomation.
Since that code is open source, I would have assumed it's already part of the training sets for large LLMs. No idea whether that's actually true, though.
> I also think it’s fair to say that since there is no Nobel Prize in Computer Science, you have to put the achievement somewhere.
I think the Turing award is about equally prestigious to people in the field and has similar prize money. Hinton won a Turing award in 2018. That said, it's probably not considered as prestigious by normies.
That style of display-only store is depressing to me. I totally understand the business incentives for it, and "the market wants this" (or will be made to want it soon enough)...but it's another brick in the wall for Good Things in life, a step back towards those insular covid years when everyone stayed home, stayed apart, had everything delivered while the public commons withered. I still can't get over how many restaurant patrons still only come to pick up food pre-ordered on an app. Why not make everything a "ghost kitchen", at that point? Obviously there's industry context - it makes sense to run, say, a furniture store this way much more than a grocery or clothing store - just...I don't know. There's something myopic and sterile about a future increasingly lived at home, no matter how much more appealing that prospect gets each year. Eventually it doesn't seem much different to living an entirely digital life sans meatbag. This too is an AI future I wouldn't much value, even if we're still alive and technically flourishing on paper.
Don't worry about the lack of AC in SF, we have Certified Cooling Centers(tm) that the government makes freely available on hot days! Although it has been funny watching people adapt to several years of record heat and wildfires here...stores never used to run out of air conditioners and fans, if they carried them at all, then had sudden runs. Now there's less panic buying, fewer shortages, us retailers are better prepared for such seasonal rushes, and the public has some base knowledge of how to handle heatwaves. System Working As Intended. It'd still be better to have actual widespread AC (another victim of NIMBYism and historical preservation), but I am encouraged that people can indeed adapt to climate change with sufficient lag time and preparation. Even in a deeply sclerotic place that cares more deeply about The Symbolic Representation Of The Thing than The Thing.
"... But should we be afraid of artificial intelligence? Preventing the development of artificial intelligence, including superintelligence that begins to feel, distinguish smells, and has cognitive capabilities, which develops itself, is simply impossible. Preventing its development is impossible. And that means we need to lead it. At the very least, we must do everything to be among the leaders in this area. But how it will end, no one knows. These are the realities, at least for today.
Yes, we can talk now about possible restrictions, self-restrictions, and it is necessary to talk about reaching agreements between leaders to avoid creating conditions that could lead to dangers for humanity. At one time, when nuclear energy turned into a nuclear bomb and it became clear that the threat to those who possessed this weapon was growing, they began to negotiate. The threat and damage became unacceptable – they started negotiating. The same will probably happen with artificial intelligence. When the leaders in this field realize that certain threats are emerging, they will probably start negotiating. But before that, it is unlikely that any real agreements will be reached, though we certainly need to think about it today."
Re Anthropic hiring less engineers: there're benefits to developing a project with fewer higher-performing engineers. Throwing as many engineers as you can at the project is not always the best thing that you can do.
Another key consideration: if you were running a very-high-security national security project, you'd be THRILLED at the notion that you had to hire fewer new, less-vetted folks to join that project...
I’m confused about this. I know several engineers who have had recruiters reach out to them for interviews. Maybe this is very recent
I read it as a critique about breath not depth. there are things that need to get build that are simpler and currently no one is working on them.
Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/ai-85-ai-wins-the-nobel-prize
"If Dean is correct here, then the carbon cost of training is trivial." Note that Table 1 in that paper concerned models using 1e19 flops for training. If I understand correctly that's 1e6 times less than GPT-4, so that GPT-4 would have costed 1e5 tons of CO2 equivalents on the most inefficient setup listed in their Table 1 and 2e3 tons if efficient. No idea about inference costs. No idea about lifetime CO2e cost of the GPUs/TPUs.
No lighter side this week... I hope this does not mean things are getting bleaker
I wonder if AI generated CSAM isn't a good thing on net that we should, while perhaps not encourage, maybe treat as less bad than real-world CSAM. There is obviously a market for CSAM that won't go away, and if this can be supplied more cheaply and with less risk than the actual child abuse stuff, and without harming children in the process, that seems better than the alternative.
I suppose the most important question is whether this would, on balance, more serve as a "gateway drug" to real CSAM and make CSAM more socially acceptable, or more satisfy pedophilic desires without any actual abuse going on. Also, of course, there is the question where you get the training data, but presumably enough of this exists that this shouldn't be an issue, and perhaps a good enough image generator could extrapolate this well enough without even needing any real CSAM.
One might also surmise that real CSAM could be claimed to be deepfaked to avoid or reduce punishment, which would definitely be an issue with this approach.
I understand that "pseudophotographs" (e.g. AI generated images) are already illegal under UK law, because the legislators anticipated the obvious problem that technology might get good enough that you cant tell whether an image is real of fake.
4/5 dentists recommend living past the singularity for good mouth health!
one way of resolving the hiring issue is the creation of a thing that I still don't get why it hasn't always existed: the applicant side equivalent of a head hunter agency.
There already exist agencies whose job it is to find people for a needed position. Why there aren't equivalent agencies whose job it is to find a position for an applicant I don't understand, but the new AI application world creates a _new_ reason:
These agencies are consistent, repeat players in the game-theory sense. They have an incentive to find _actually_ good candidates for fit. If they are repeatedly sending in people who have lied etc, then they get a bad reputation and companies won't trust them, and they can't place applicants and they go out of business.
And, since these people are hired _by_ the job applicant, they have an ability to actually find out real information about the candidate.
I expect a service like this to be expensive, but it also seems _obviously_ worthwhile.
And game theory optimum with repeated games fixes a lot of the current problems with AI application, AI hiring managers, and incentives to lie.
Job agencies and consulting companies provide this to some extent and some professionals hire or retain agents to do exactly this. Not having publicly accessible work that can be evaluated might mean 'reverse headhunters' can't do much more than edit resumés. Some job listing services do seem to match both sides too.
I guess in my mind, it's the people who don't have publicly accessible work who would gain the most value. If I can pay an agency to basically go through an interview process and find out the information that goes beyond a resume, but isn't easily legible, and if that company has a reputation for placing good candidates, then they can figure out where I'd be a good fit in ways that hiring companies can't afford to do (because finding that out is costly), then hard-to-assess candidates can get better positions.
Theoretically, being able to pay to guarantee an interview would accomplish something similar, except it would have massive perverse incentives.
Some recruiters do this already, tho I think it's only for particularly in-demand and well paid positions. There is also a long line of companies that tried and failed to do this even just for the relatively more legible { computer programming / software engineering }. Even those extremely narrow versions are, apparently, fatally difficult (for for-profit companies).
Maybe it's just an almost-perfectly insurmountable gap in the value between 'this person might be a good hire' and 'this person WAS a good hire – evaluated as-of a year later – by a company capable of putting them to some productive use'.
Think about it this way: Would you hire someone represented by an agent, knowing that as soon as you hire this person, the agent will get to work seeing who else will poach for more?
I don't understand why this would create a new risk.
Why an I going to continue to pay the agent to keep looking? If they think they can quickly and easily find me a better offer, why did they tell me to take this job?
Also, how is this different than the current situation? Why didn't you expect current employees to immediately start looking for a better job?
> Why an I going to continue to pay the agent to keep looking?
The agent works on commission, it's not like you pay them to keep looking.
> If they think they can quickly and easily find me a better offer, why did they tell me to take this job?
You now have more experience, perhaps a better title, higher pay — all these things set you up to be considered for a job that you might not have been able to get before. Or perhaps there is an opportunity that wasn't available six months ago.
> Also, how is this different than the current situation? Why didn't you expect current employees to immediately start looking for a better job?
Well, from the perspective of incentives it's not that different; but since an agent views this as their main job, on average they'll be much more persistent about it. Your agent doesn't care about your work relationships, or the career capital you get from having long tenure, or even really how much you like the job.
To put it another way, most people don't put nearly as much effort into job searching as they should (relative to effort spent actually doing their job) and having an agent changes that.
If you stay in the job long enough that it is _actually_ more experience, then I don't think that the hiring company has much room to complain
But the biggest reason this scheme works at all is that the _agencies_ get a reputation for placing good candidates. That's why companies will trust their assessment at all. But that cuts both ways. If they place a candidate who leaves in 6 months, that hurts their reputation, and the hiring company is less likely to take candidates from them in the future.
The equilibrium is companies placing candidates with an eye towards decently sustained success at the position. Yes, occasionally, it might happen, but any agency that does it too often is going to get a reputation for that and hiring companies won't hire from them.
"I am happy to hear that they are at least noticing that they might not want to release this in its current form."
Screw that. LET'S GO. I'm sick and tired of only the Illuminati Privilege Bros having access to this tech. Lets cut it loose. If we die, we die. It's a completely insane world where Zuck is the good guy, but that seems to be the current state of affairs.
No.
found the privilege bro
Bryan Johnson has a point. We're all in a race against death and you have so little time. We're all scheduled to die anyways might as well try how far this avenue goes.
On the physics Nobel prize:
I think Hinton was arguably awarded for work that was not-physics, but Hopfield's work was squarely within the scope of physics as generally understood. Hopfield himself seems to think so
https://www.annualreviews.org/content/journals/10.1146/annurev-conmatphys-031113-133924
(Note that this essay was written a decade ago).
Also the condensed matter physics arxiv contains `disordered systems and neural networks' as a subarxiv. I've read it (and contributed to it) for over a decade, and `neural networks are part of physics' strikes me as an utterly obvious and uncontroversial take.
When I moved to San Francisco:
Me: Thanks for showing me these apartments today.
Rental Agent: No problem, love helping people move to The City.
Me: Does this next apartment have AC?
Her: AC? Uh, AC?
Me: ... "Air Conditioning".
Her: Oh! Ha, ha! Of course not!
Portable AC solves this, $200 from Amazon and your bedroom will be nice and cool.
Actually, in the three years I lived in Presidio Heights I only missed AC two days.
> I find myself excited to write actual code that does the things, but the thought of having to set everything up to get to that point fills with dread - I just know that the AI is going to get something stupid wrong, and everything’s going to be screwed up, and it’s going to be hours trying to figure it out and so on, and maybe I’ll just work on something else.
Yeah, the exact same concern put me off for a long time (and I have not yet fully pushed through it).
FWIW, I imagine the experience _might_ be better if you use JavaScript or Python. I went with Java because I have deep muscle memory for that language, but it's probably an unusual choice nowadays for starter projects. Regardless of whether LLMs would be better at giving troubleshooting advice, I think it's likely that in ReplIt and possibly Cursor you'd be following a more well-trodden path.
'Serious' JavaScript and Python are relatively painful – the 'script' side of both is very nice, but only for very small projects or prototypes.
As a professional, we just pay these costs infrequently, and I and my less excitable or adventurous fellow practitioners tend to pay smaller costs to upgrade a 'boring stack' over many years or even decades because the costs of spinning up new projects using an already setup stack can be driven down very low (which is, as one might imagine, very nice and very useful).
You and Zvi should pay a pro to setup the 'stack infrastructure' for you – your stack, your local dev environments, source/version control, and whatever initial deployments you want to start.
Any suggestions for how to find that pro? I'd be happy to pay.
(For context, I'm a professional myself, have probably written 1M lines of code over my career, but never liked or got good at the setup stage. I left my last company 20 months ago and am just getting back to dabbling with coding.)
If you know another, ask them for their 'cheerful price'! They might be happy to do it just as an excuse to hangout and talk shop.
It seems like you could absolutely do it yourself. Pick a boring standard stack for whatever you want to work with. JavaScript is the Wild West but Python has some 'all in one' packages that might Just Work for you. I don't like Docker, but maybe you'd like it fine.
Otherwise, check for someone using your stack on a (programming) projects site and ask them for what they recommend and a quote to help you get it all setup.
I have just had an idea for an AI application ... something that can answer questions about large codebases, such as the Linux kernel or the LLVM compiler. e.g. "Where in this code is the RISCV option string from the FDT parsed?" (I actually got the answer to that one using the traditional method "grep", but consider this an example of a type of question that you're asking all the time, and can't always be solved with a simple regular expression search.)
For the curious: those fine people in charge of the RISCV specification have ratified some more options, so the question one is of course asking is "Ok, where in the Linux kernel do I need to patch to support this new requirement?"
I once spend three days finding exactly where in the LLVM MIPS backend a bug was, given an example of code that triggered it. (Hint: there were 64 bit MIPS CPUs that were not MIPS64. Problem: find where in the compiler it is checking for target being MIPS64 rather that target being a 64 bit MIPS. Several days later, I have located the offending line of code).
I feel that this kind of thing could benefit from AI sutomation.
Since that code is open source, I would have assumed it's already part of the training sets for large LLMs. No idea whether that's actually true, though.
> I also think it’s fair to say that since there is no Nobel Prize in Computer Science, you have to put the achievement somewhere.
I think the Turing award is about equally prestigious to people in the field and has similar prize money. Hinton won a Turing award in 2018. That said, it's probably not considered as prestigious by normies.
That style of display-only store is depressing to me. I totally understand the business incentives for it, and "the market wants this" (or will be made to want it soon enough)...but it's another brick in the wall for Good Things in life, a step back towards those insular covid years when everyone stayed home, stayed apart, had everything delivered while the public commons withered. I still can't get over how many restaurant patrons still only come to pick up food pre-ordered on an app. Why not make everything a "ghost kitchen", at that point? Obviously there's industry context - it makes sense to run, say, a furniture store this way much more than a grocery or clothing store - just...I don't know. There's something myopic and sterile about a future increasingly lived at home, no matter how much more appealing that prospect gets each year. Eventually it doesn't seem much different to living an entirely digital life sans meatbag. This too is an AI future I wouldn't much value, even if we're still alive and technically flourishing on paper.
Don't worry about the lack of AC in SF, we have Certified Cooling Centers(tm) that the government makes freely available on hot days! Although it has been funny watching people adapt to several years of record heat and wildfires here...stores never used to run out of air conditioners and fans, if they carried them at all, then had sudden runs. Now there's less panic buying, fewer shortages, us retailers are better prepared for such seasonal rushes, and the public has some base knowledge of how to handle heatwaves. System Working As Intended. It'd still be better to have actual widespread AC (another victim of NIMBYism and historical preservation), but I am encouraged that people can indeed adapt to climate change with sufficient lag time and preparation. Even in a deeply sclerotic place that cares more deeply about The Symbolic Representation Of The Thing than The Thing.
I believe Bryan's intended meaning to be: "I want as many currently living humans as possible to survive until ASI arrives and makes us immortal."
This is from last year but recently I saw an interesting take from Putin on AI, in response to a question on deepfakes (https://www.youtube.com/watch?v=KbaKTz9FW2E, full transcript at https://www.kp.ru/daily/27594/4865984/). Translation of his answer is as follows:
"... But should we be afraid of artificial intelligence? Preventing the development of artificial intelligence, including superintelligence that begins to feel, distinguish smells, and has cognitive capabilities, which develops itself, is simply impossible. Preventing its development is impossible. And that means we need to lead it. At the very least, we must do everything to be among the leaders in this area. But how it will end, no one knows. These are the realities, at least for today.
Yes, we can talk now about possible restrictions, self-restrictions, and it is necessary to talk about reaching agreements between leaders to avoid creating conditions that could lead to dangers for humanity. At one time, when nuclear energy turned into a nuclear bomb and it became clear that the threat to those who possessed this weapon was growing, they began to negotiate. The threat and damage became unacceptable – they started negotiating. The same will probably happen with artificial intelligence. When the leaders in this field realize that certain threats are emerging, they will probably start negotiating. But before that, it is unlikely that any real agreements will be reached, though we certainly need to think about it today."