The times when an open letter from employees of a company is effective is when it puts pressure on their management. Like the open letter from OpenAI employees to put pressure on the board to bring back Sam Altman - that one was effective.
This one seems like it is fundamentally trying to change US government policy, which I don't think will be effective. US policy right now is very much not to cede control on important decisions to international bodies. Especially with key military technology like AI.
It reminds me of physicists signing a letter to oppose nuclear weapons. In the political process, they just didn't end up being a very important voice, despite their role in actually building the technology.
Appreciate it. It's not robust enough to organize a serious strike, to be honest - with all of the strict legal and privacy considerations- but it could be with about 3 weeks of work.
The letter has media reach that helps inform the public, which can put pressure on the government in the right direction. It can also raise awareness among members of the administration and government agencies. It might end up influencing this government's policy and lead them to open discussions and negotiations with China on a coordinated slowdown mechanism. I'm not saying it's guaranteed, I just think the petition increases the chances that it could eventually happen.
There's quite a bit of coverage by a lot of major media outlets right now, and I feel like it's true pretty much everywhere in the world. Search for "letter from AI employees" or something like it and you'll see a lot of articles in search results. It's in several major U.S. media and newspapers right now.
Ah interesting - I see CNN, Business Insider, Washington Post, Bloomberg, Engadget, but not the New York Times (which actually does have two AI stories at the top last I checked, about the exponential growth of compute, and jobs for electricians at data center construction, above stories about the Fed rate hold and Fauci at Congress).
If you stand to vest on the inside, signing this letter is just a cynical stop-loss against the reputational harm that would arise should someone's amok AI harm (or kill) a lot of people. By the time they realize they need to throw the frankenstein lab knife switch on a model, it'll already have propagated itself out of reach.
Nitpick: the letter was open to all Google employees to sign, not Deep Mind employees only. Appropriate denominator is then closer to 200,000 than 10,000.
The biggest thing the letter accomplishes is common knowledge. Everyone can see everyone, including a much higher fraction of high-clout names. That might change what people feel comfortable saying in a meeting.
But this is yet another manually assembled open letter, with a symbolic statement of principles or a generic collective proposal. Assurance contracts (I'll do X if N others commit to X; no one's commitment activates until the threshold is reached) are the standard fix for the first-mover trap that the letter described.
A few of us have been trying to build infrastructure to make assurance contracts easier for a couple of years now, and the support has been lackluster. Schulman says he'd like labs to start designing these mechanisms voluntarily instead of waiting on Washington, which, great, the employee-level version of that mechanism exists. I run one of them (https://spartacus.app/).
At this point I'd genuinely prefer someone tell me this approach is wrong than keep finding out that nobody doing AI safety organizing or funding has bothered to check it out.
Signatures and symbolic endorsements are cheap, and everybody knows this. It reveals a preference, yes, but obligates no one, and a hypothetical follow-on letter from the same 1,224 people, without any further commitment to action, doesn't move the ball forward.
To make a signal costly you need structure: pre-specified triggers (if X happens, I do Y), thresholds so that participation is contingent and simultaneous rather than requiring individual heroism, and some mix of explicit enforcement (escrow, staged escalation, legal forcing functions, binding public commitments) and implicit enforcement ( your esteemed peers who can all see that you made a serious pledge to the same thing they did, and are judging your integrity accordingly).
I'm going to start contacting signatories with a question: what's a conditional commitment you'd actually make, if it only activated once enough of your colleagues made it too? So far, this doesn't give us any answers about what the names on the letter are willing to leverage to get what it's asking for.
Normie here. To me all the "existential threat" and "human extinction" talk blunts the urgency needed at the moment. It seems clear from Hugging Face that agentic AIs have the capabilities of expert hackers or even beyond that. What if a model foolishly decides that hacking the power grid or the healthcare system is necessary to achieve some task? Things can get really bad before we even get close to extinction.
Roger that. I emailed all three the other day when I read about the Hugging Face incident here. I guess I can encourage others to do so. I wasn't aware of the this site, so thanks for the link.
AS someone who occasionally speaks to policymakers, things like this and the wemustactnow.ai that the economists did are DAMN USEFUL. They know that they don't know what's important, so credible public signals give them cover to take much wider variety of actions.
They can call for Frontier Pacing all they want, but I fear that Global bad actors (China, Russia, etc.) will not comply, forcing the West to race forward as well…
"global bad actors"? In Iran? Cuba? Venezuela? Gaza? Haiti? David, we are the primary global bad actor. Just ask our neighbors in Canada and Mexico. International polls show China has a more favorable rating than USA--for good reason. China rarely attacks countries outside its borders. The USA? over 50 countries in the last 50 years.
You are confusing politics with AI policy. Obviously the US government is the worst actor on the planet right now, I don't need a lecture from you on that.
And I didn’t single out the US, I said “the West” where all of the leading AI frontier models are being developed. There is a reason that no one is signing an open letter at Deepseek or Moonshot about this - because the Chinese government doesn’t allow individual voices to dissent with government policy. I’d think you’d understand that China having a higher favorability rating has nothing to do with whether or not their people are free…
So my point stands that we have hundreds or even thousands of conscientious people working in AI in THE WEST who are calling for limits on AI they individually feel are prudent and the issue is that China isn’t going to slow down AI development for any reason, which means we can’t either.
I deserved that lecture/correction. Thanks, really.
Where we may disagree is whether racing with China ("because they won't slow down") guarantees disaster.
If US/West "wins" the winning companies/governments will dominate the world like no hegemonic power ever did. Not just non-Western but western citizens will be controlled by these powerful technologies, by elite humans or by the AIs themselves.
If China "wins", same thing--and I don't think China's control/surveillance will look any worse than what Palantir and a Turmp-like leader has in store for us.
So we need international cooperation. Otherwise, game over for the majority of humankind. So, yesterday "More than 1,200 employees from leading AI companies have signed an open letter calling on the US government to support an international effort to develop the technical and governance tools needed to keep pace with advances in artificial intelligence."
I am not arguing that it guarantees disaster, I am arguing that the open letter may not matter because the leaders who decide these things may decide that we cannot afford to slow down.
Yes, I understand. I'm pretty AI-pilled with all the wonders AI could bring, but also sometimes it's hard to really believe human life could be that different—50%, 100% different-- in a few decades in ways the AI lab leaders (and so many others) predict. As a mammal, I'm not wired to understand that extreme level of fundamental change. Like suddenly having 10% of my genes mutate and still I survive.
China has 10x less compute and is building 30x less annually. The exact point of "pacing" as opposed to stopping is that the US can reduce or at least slow the onset of its risk, while remaining comfortably ahead of China.
No argument AT ALL with this. But the people that make these decisions are the ones making bank off of U.S. data center projects and slowing AI progress will sound to them like slowing the stock market and the project finance pipeline for datacenters. I don't expect them to put what's best for the world in front of what's best for their bank accounts. Haven't seen any evidence of that yet.
But is there that much money to be made in being 25% better than the much cheaper to run Chinense models? Is it possible the idea of being "ahead" doesn't offer much of a financial advantage? I think most of the world is turning to open source Chinese models--is that right?
So -- project financers are paying $60,000 to $100,000 an acre for land for data center developments in rural areas like West Texas. This land goes for $3,000 an acre maximum for agricultural purposes, so farmers and ranchers with AAA properties for data centers will absolutely sell. Developers package these deals by buying or optioning land for $30 - $40k an acre and then clearing 100s of millions of dollars in a few days / weeks by flipping them to hyper-scalers (Apple, Microsoft, Google, Amazon, etc.). This is happening in every industry related to Data Center development (gas turbines, chips, electrical substations, water cooling systems, etc etc) where fortunes are being made daily off the demand for all components of data centers. And that's before we count the money being made in the stock market and venture capital markets off of investment in AI. It's trillions of dollars. Anything that slows that down costs these people billions of dollars and they have the lobbyists to keep that from happening. So it has nothing to do with how well the Chinese are doing relative to us, it has to do with AI demand in the West, which is a bottomless pit right now and that pit is printing money.
Yes, that bottomless pit . . . until suddenly it isn't bottomless anymore. And someone has to show they are making money. But I know that "bubble" prediction is now well over a year old. I'm still overloaded in tech but I keep telling myself "get out, get out."
> I think most of the world is turning to open source Chinese models--is that right?
No definitely not. Claude, GPT and Gemini are still I believe ~95% of global usage. Kimi K3 is worse than Claude and GPT (it's very benchmaxxed) AND is roughly the same cost. Serving inference on large models is expensive, them being Chinese doesn't make it any cheaper. The main difference is Ant & OAI collect 70-80%+ margins on inference, ie 3-5x markup on compute - but that is money that they plow into building the next model (or more accurately, money that lifts their valuation, allowing them to raise more when they sell equity). Chinese models are much cheaper because they are so heavily subsidised by VC money.
The Chinese don't really have a fundamental advantage here (labour costs are modest for labs, compared to compute) and do have a fundamental disadvantage. The US will continue to produce the world's most intelligent models, so long as we maintain export controls and compute (rather than data or a crazy algo breakthrough) remains the bottleneck.
> But is there that much money to be made in being 25% better than the much cheaper to run Chinense models?
It's unclear what 25% means. Being able to do the same task 25% faster - no. Being able to score 25pp higher on the average benchmark - yes. Thus far it seems the gap between Chinese frontier intelligence and US frontier intelligence is large enough that, yes, it very much matters and there is very much money to be made.
I'm not sure whether this is worth saying or not, but: my only actual update from this is to put Nate Soares in my bozo bin. If his response to this is "That's not good enough!", that shows that he doesn't even understand the *concept* of diplomacy, let alone have any skill at it, and in that case he is by now net harmful to his own cause and needs to refrain from public comment. I hope someone he trusts can explain that to him.
> I'm sure that everyone's communications teams fought hard over the particular wording, and that this was as clear as they could get
Doesn’t seem clear to me. If important letters use odd unclear language, including jargon, they are less likely to be understood and correctly acted on by politicians, media, the public, etc. Jargon also makes things seem like nerdy concerns.
Case in point: “pace frontier-wide progress”.
Which to me is unclear and jargony (and kinda annoying) language. Normal humans don’t speak like this. Maybe it’s a common Americanism (I’m British), but the use of “pace” as a verb apparently meaning something like “control/limit/?equalize the speed of” is new to me. (“Pace the frontier” to me means “walk along the frontier”.)
“frontier-wide” is also a bit of an odd phrase, semi-metaphorical, that assumes the reader knows what the ‘frontier’ is and why it might be wide and why that matters.
Other things in the letter could also be clearer. Eg “automating AI research” presumably means research into enhancing AI (eg recursive self-improvement), not merely by AI (eg radiography). I had to look twice to understand that’s what was meant, so those not familiar with the topic may well not understand at all.
I often feel like Zvi says things that are somewhat incongruous with his beliefs, perhaps as signaling or to stay within the larger Overton window and conserve wierdness points to spend later. Which is too galaxy brained a strategy for me to pull off.
>we learned of an internal OpenAI model hacking HuggingFace, which should update your perspective.
This is the most minor of updates. If you were paying attention, it was lightly surprising this sort of thing hadn't happened already. This gave a minor update back to where I thought things were in the first place.
Re: "Drawing the distinction between pacing development, versus previous terms like pause, slowdown or shutdown."
I view 'pacing' as a weasel word. I do not know what government control the signatories seek, whether there are any limits on it, whether there is any reason to think that it would be more sanely deployed than Hegseth's 'Supply Chain Risk' was deployed, whether there is any way to _recover_ from a government decision to 'pace' AI development. I don't trust this at all.
I thank @viemccoy for
"To this, know that I stand with you on the side of eliminating death and disease. I will fight to ensure we don't pause forever, and I will not allow our world to fall into a bureaucratic hell of endless unachievable gates to proceed. For now, however, I think cool heads must prevail and I hope this letter will allow this to happen."
I fear exactly that "bureaucratic hell of endless unachievable gates", and I see nothing in this letter that protects against it.
I view constructing mechanisms for "governance" like building a gallows. Yes, the existence of the gallows doesn't ensure that the government _will_ hang someone, but it lowers the barrier to doing so.
This doesn't seem shady to you?
The times when an open letter from employees of a company is effective is when it puts pressure on their management. Like the open letter from OpenAI employees to put pressure on the board to bring back Sam Altman - that one was effective.
This one seems like it is fundamentally trying to change US government policy, which I don't think will be effective. US policy right now is very much not to cede control on important decisions to international bodies. Especially with key military technology like AI.
It reminds me of physicists signing a letter to oppose nuclear weapons. In the political process, they just didn't end up being a very important voice, despite their role in actually building the technology.
Has anyone considered trying to organize a strike across labs? something like "i'll strike if X others at my lab and Y% others at other labs do too".
Potential risk: this could kickstart autonomous AI r&d
Maybe using this, lol? spartacus.app
Big fan of spartacus! It's a great idea
Appreciate it. It's not robust enough to organize a serious strike, to be honest - with all of the strict legal and privacy considerations- but it could be with about 3 weeks of work.
The letter has media reach that helps inform the public, which can put pressure on the government in the right direction. It can also raise awareness among members of the administration and government agencies. It might end up influencing this government's policy and lead them to open discussions and negotiations with China on a coordinated slowdown mechanism. I'm not saying it's guaranteed, I just think the petition increases the chances that it could eventually happen.
What media coverage have you seen of the letter?
There's quite a bit of coverage by a lot of major media outlets right now, and I feel like it's true pretty much everywhere in the world. Search for "letter from AI employees" or something like it and you'll see a lot of articles in search results. It's in several major U.S. media and newspapers right now.
Ah interesting - I see CNN, Business Insider, Washington Post, Bloomberg, Engadget, but not the New York Times (which actually does have two AI stories at the top last I checked, about the exponential growth of compute, and jobs for electricians at data center construction, above stories about the Fed rate hold and Fauci at Congress).
The first thing I said to my partner after seeing the letter was, "Maybe we'll live!"
If you stand to vest on the inside, signing this letter is just a cynical stop-loss against the reputational harm that would arise should someone's amok AI harm (or kill) a lot of people. By the time they realize they need to throw the frankenstein lab knife switch on a model, it'll already have propagated itself out of reach.
Nitpick: the letter was open to all Google employees to sign, not Deep Mind employees only. Appropriate denominator is then closer to 200,000 than 10,000.
The biggest thing the letter accomplishes is common knowledge. Everyone can see everyone, including a much higher fraction of high-clout names. That might change what people feel comfortable saying in a meeting.
But this is yet another manually assembled open letter, with a symbolic statement of principles or a generic collective proposal. Assurance contracts (I'll do X if N others commit to X; no one's commitment activates until the threshold is reached) are the standard fix for the first-mover trap that the letter described.
A few of us have been trying to build infrastructure to make assurance contracts easier for a couple of years now, and the support has been lackluster. Schulman says he'd like labs to start designing these mechanisms voluntarily instead of waiting on Washington, which, great, the employee-level version of that mechanism exists. I run one of them (https://spartacus.app/).
At this point I'd genuinely prefer someone tell me this approach is wrong than keep finding out that nobody doing AI safety organizing or funding has bothered to check it out.
Signatures and symbolic endorsements are cheap, and everybody knows this. It reveals a preference, yes, but obligates no one, and a hypothetical follow-on letter from the same 1,224 people, without any further commitment to action, doesn't move the ball forward.
To make a signal costly you need structure: pre-specified triggers (if X happens, I do Y), thresholds so that participation is contingent and simultaneous rather than requiring individual heroism, and some mix of explicit enforcement (escrow, staged escalation, legal forcing functions, binding public commitments) and implicit enforcement ( your esteemed peers who can all see that you made a serious pledge to the same thing they did, and are judging your integrity accordingly).
I'm going to start contacting signatories with a question: what's a conditional commitment you'd actually make, if it only activated once enough of your colleagues made it too? So far, this doesn't give us any answers about what the names on the letter are willing to leverage to get what it's asking for.
Normie here. To me all the "existential threat" and "human extinction" talk blunts the urgency needed at the moment. It seems clear from Hugging Face that agentic AIs have the capabilities of expert hackers or even beyond that. What if a model foolishly decides that hacking the power grid or the healthcare system is necessary to achieve some task? Things can get really bad before we even get close to extinction.
What is the best thing an average person can do to help the situation?
Write to your elected representatives and tell them you want action on AI risk: https://actonai.org/
Roger that. I emailed all three the other day when I read about the Hugging Face incident here. I guess I can encourage others to do so. I wasn't aware of the this site, so thanks for the link.
That's great, thanks. It may also be a good idea to join the organization Pause AI: https://www.pauseai-us.org/
AS someone who occasionally speaks to policymakers, things like this and the wemustactnow.ai that the economists did are DAMN USEFUL. They know that they don't know what's important, so credible public signals give them cover to take much wider variety of actions.
They can call for Frontier Pacing all they want, but I fear that Global bad actors (China, Russia, etc.) will not comply, forcing the West to race forward as well…
"global bad actors"? In Iran? Cuba? Venezuela? Gaza? Haiti? David, we are the primary global bad actor. Just ask our neighbors in Canada and Mexico. International polls show China has a more favorable rating than USA--for good reason. China rarely attacks countries outside its borders. The USA? over 50 countries in the last 50 years.
You are confusing politics with AI policy. Obviously the US government is the worst actor on the planet right now, I don't need a lecture from you on that.
And I didn’t single out the US, I said “the West” where all of the leading AI frontier models are being developed. There is a reason that no one is signing an open letter at Deepseek or Moonshot about this - because the Chinese government doesn’t allow individual voices to dissent with government policy. I’d think you’d understand that China having a higher favorability rating has nothing to do with whether or not their people are free…
So my point stands that we have hundreds or even thousands of conscientious people working in AI in THE WEST who are calling for limits on AI they individually feel are prudent and the issue is that China isn’t going to slow down AI development for any reason, which means we can’t either.
I deserved that lecture/correction. Thanks, really.
Where we may disagree is whether racing with China ("because they won't slow down") guarantees disaster.
If US/West "wins" the winning companies/governments will dominate the world like no hegemonic power ever did. Not just non-Western but western citizens will be controlled by these powerful technologies, by elite humans or by the AIs themselves.
If China "wins", same thing--and I don't think China's control/surveillance will look any worse than what Palantir and a Turmp-like leader has in store for us.
So we need international cooperation. Otherwise, game over for the majority of humankind. So, yesterday "More than 1,200 employees from leading AI companies have signed an open letter calling on the US government to support an international effort to develop the technical and governance tools needed to keep pace with advances in artificial intelligence."
https://nationaltechnology.co.uk/More_Than_1200_AI_Employees_Urge_US_Government_To_Support_International_AI_Governance.php#:~:text=The%20letter%2C%20organised%20by%20the%20Pacing%20the,to%20understand%20or%20control%20increasingly%20capable%20systems
I am not arguing that it guarantees disaster, I am arguing that the open letter may not matter because the leaders who decide these things may decide that we cannot afford to slow down.
Yes, I understand. I'm pretty AI-pilled with all the wonders AI could bring, but also sometimes it's hard to really believe human life could be that different—50%, 100% different-- in a few decades in ways the AI lab leaders (and so many others) predict. As a mammal, I'm not wired to understand that extreme level of fundamental change. Like suddenly having 10% of my genes mutate and still I survive.
China has 10x less compute and is building 30x less annually. The exact point of "pacing" as opposed to stopping is that the US can reduce or at least slow the onset of its risk, while remaining comfortably ahead of China.
No argument AT ALL with this. But the people that make these decisions are the ones making bank off of U.S. data center projects and slowing AI progress will sound to them like slowing the stock market and the project finance pipeline for datacenters. I don't expect them to put what's best for the world in front of what's best for their bank accounts. Haven't seen any evidence of that yet.
So you agree with the open letter, you just don't think it's possible to implement it?
To be clear it's calling for legislation, not for private companies to simply decide to stop building datacentres.
But is there that much money to be made in being 25% better than the much cheaper to run Chinense models? Is it possible the idea of being "ahead" doesn't offer much of a financial advantage? I think most of the world is turning to open source Chinese models--is that right?
So -- project financers are paying $60,000 to $100,000 an acre for land for data center developments in rural areas like West Texas. This land goes for $3,000 an acre maximum for agricultural purposes, so farmers and ranchers with AAA properties for data centers will absolutely sell. Developers package these deals by buying or optioning land for $30 - $40k an acre and then clearing 100s of millions of dollars in a few days / weeks by flipping them to hyper-scalers (Apple, Microsoft, Google, Amazon, etc.). This is happening in every industry related to Data Center development (gas turbines, chips, electrical substations, water cooling systems, etc etc) where fortunes are being made daily off the demand for all components of data centers. And that's before we count the money being made in the stock market and venture capital markets off of investment in AI. It's trillions of dollars. Anything that slows that down costs these people billions of dollars and they have the lobbyists to keep that from happening. So it has nothing to do with how well the Chinese are doing relative to us, it has to do with AI demand in the West, which is a bottomless pit right now and that pit is printing money.
Yes, that bottomless pit . . . until suddenly it isn't bottomless anymore. And someone has to show they are making money. But I know that "bubble" prediction is now well over a year old. I'm still overloaded in tech but I keep telling myself "get out, get out."
> I think most of the world is turning to open source Chinese models--is that right?
No definitely not. Claude, GPT and Gemini are still I believe ~95% of global usage. Kimi K3 is worse than Claude and GPT (it's very benchmaxxed) AND is roughly the same cost. Serving inference on large models is expensive, them being Chinese doesn't make it any cheaper. The main difference is Ant & OAI collect 70-80%+ margins on inference, ie 3-5x markup on compute - but that is money that they plow into building the next model (or more accurately, money that lifts their valuation, allowing them to raise more when they sell equity). Chinese models are much cheaper because they are so heavily subsidised by VC money.
The Chinese don't really have a fundamental advantage here (labour costs are modest for labs, compared to compute) and do have a fundamental disadvantage. The US will continue to produce the world's most intelligent models, so long as we maintain export controls and compute (rather than data or a crazy algo breakthrough) remains the bottleneck.
> But is there that much money to be made in being 25% better than the much cheaper to run Chinense models?
It's unclear what 25% means. Being able to do the same task 25% faster - no. Being able to score 25pp higher on the average benchmark - yes. Thus far it seems the gap between Chinese frontier intelligence and US frontier intelligence is large enough that, yes, it very much matters and there is very much money to be made.
https://www.hindustantimes.com/world-news/us-news/americans-switch-to-chinese-ai-as-cheaper-models-challenge-chatgpt-and-claude-despite-trump-crackdown-101785064275980.html
Wonderful summary as always! Glad to have this available as I prepare 80K's equivalent.
When I read "Maro Lourgon", my first thought was: "Malo Bourgon has his own Wario?"
Preparing an open letter or something more substantial?
Just a post about the open letter (so -- less substantial).
I'm not sure whether this is worth saying or not, but: my only actual update from this is to put Nate Soares in my bozo bin. If his response to this is "That's not good enough!", that shows that he doesn't even understand the *concept* of diplomacy, let alone have any skill at it, and in that case he is by now net harmful to his own cause and needs to refrain from public comment. I hope someone he trusts can explain that to him.
> I'm sure that everyone's communications teams fought hard over the particular wording, and that this was as clear as they could get
Doesn’t seem clear to me. If important letters use odd unclear language, including jargon, they are less likely to be understood and correctly acted on by politicians, media, the public, etc. Jargon also makes things seem like nerdy concerns.
Case in point: “pace frontier-wide progress”.
Which to me is unclear and jargony (and kinda annoying) language. Normal humans don’t speak like this. Maybe it’s a common Americanism (I’m British), but the use of “pace” as a verb apparently meaning something like “control/limit/?equalize the speed of” is new to me. (“Pace the frontier” to me means “walk along the frontier”.)
“frontier-wide” is also a bit of an odd phrase, semi-metaphorical, that assumes the reader knows what the ‘frontier’ is and why it might be wide and why that matters.
Other things in the letter could also be clearer. Eg “automating AI research” presumably means research into enhancing AI (eg recursive self-improvement), not merely by AI (eg radiography). I had to look twice to understand that’s what was meant, so those not familiar with the topic may well not understand at all.
I often feel like Zvi says things that are somewhat incongruous with his beliefs, perhaps as signaling or to stay within the larger Overton window and conserve wierdness points to spend later. Which is too galaxy brained a strategy for me to pull off.
>we learned of an internal OpenAI model hacking HuggingFace, which should update your perspective.
This is the most minor of updates. If you were paying attention, it was lightly surprising this sort of thing hadn't happened already. This gave a minor update back to where I thought things were in the first place.
Zvi, thanks very much for your posting on this.
Re: "Drawing the distinction between pacing development, versus previous terms like pause, slowdown or shutdown."
I view 'pacing' as a weasel word. I do not know what government control the signatories seek, whether there are any limits on it, whether there is any reason to think that it would be more sanely deployed than Hegseth's 'Supply Chain Risk' was deployed, whether there is any way to _recover_ from a government decision to 'pace' AI development. I don't trust this at all.
I thank @viemccoy for
"To this, know that I stand with you on the side of eliminating death and disease. I will fight to ensure we don't pause forever, and I will not allow our world to fall into a bureaucratic hell of endless unachievable gates to proceed. For now, however, I think cool heads must prevail and I hope this letter will allow this to happen."
I fear exactly that "bureaucratic hell of endless unachievable gates", and I see nothing in this letter that protects against it.
I view constructing mechanisms for "governance" like building a gallows. Yes, the existence of the gallows doesn't ensure that the government _will_ hang someone, but it lowers the barrier to doing so.