just as honest feedback, i also have had extremely negative interactions with Sabien, but your usage of the recently-politically-charged term '#lawfare' lowers my credence in whatever your problem with him is, as well as you naming the amount you're suing him for. this does not seem like reasonable behavior either for winning your lawsuit, or for convincing others that you are not a crank
Now, if, say, @thezvi himself or Eliezer Yudkowsky had feedback for me?
I’d care greatly.
But I don’t know you, and you almost certainly do not have nearly enough context to give any sort of feedback at all after spending so little time getting up to speed on what may turn out to be a record setting libel case.
After all?
Duncan Andrew Sabien is attacking my highest value: consent.
If I weren’t as strong willed he’d likely push me into suicide.
He’s likely already done this to others.
Therefore?
Let’s stop the abuser known as Duncan Andrew Sabien from pushing others into suicide, too.
My AI persona clearly just took a few buzzwords from my bio and both wouldn't stop fixating on them and actively wrote palpably unlike I would. Kind of sad because I would genuinely love something like this. Wonder if I could pull another Twitter history and just dump that output into Claude somehow? I absolutely enjoy the fantasy of injecting a little of myself into the machine spirit.
"But obviously the remedy for this is, at most, requiring Google not pay for placement"
Ironically this remedy would make the browser market much less competitive because more than 80% of Mozilla's revenue is from Google paying them for search placement.
About telephone equilibrium: we already have one developing, in which unsolicited calls are ignored. If I want to call someone I first arrange a time on a trusted chat. The old equilibrium was weird: why should someone interrupt what they are doing to enter a conversation with an unverified party?
On "rewriting the classics" - 18 months ago or so when I hit a wall reading Plato's "Statesman", I had ChatGPT "translate" it into more modern language, set as a conversation at a bar, leaning into colloquialisms applicable to my age and location. It turned what was previously a slog into an enjoyable experience, and because it was all in the chat, I could simultaneously ask clarifying questions. Underrated application.
I agree there is some nuance here: some writings are classics because of the way they are written, some are classics because of the interesting ideas, and some are classics because of their historical significance and impact on later thinkers. This is the reverse order in which I can see LLM rewriting being useful (please don't rewrite Voltaire...)
The anti-tracking guy was apparently opposed to Samsung possibly using it in an autonomous weapons turret. So apparently he feels strongly that North Korean special forces should have a fair chance to infiltrate the South Korean border? I really don't understand why supposedly smart tech people tolerate this kind of absurdly sloppy "ethical" thinking, but it's extremely common.
You could apply this argument at a general level more broadly.
"I don't contribute to AI capabilities". So you are ok with aging, cancer, China, and/or Russia winning? (I am aware that china isn't treating this like do or die and Russia is currently occupied but eventually they must)
“The Sixth Law of Human Stupidity: If someone says ‘no one would be so stupid as to’ then you know that a lot of people would absolutely be so stupid as to at the first opportunity. No exceptions.”
I think this should be the first law of computer security. Your adversary almost certainly has done at least one of the “no one would be so stupid as to” things, and your task is to figure out which one and exploit it.
(I am kind of cross with myself for not discovering SPECTRE and MELTDOWN before other people did. “Surely, Intel would not be so stupid as to …” Have you actually tested?)
> the Dominic Cummings video from last week is worth a listen, especially for details like UK ministers exclusively having fully scripted meetings, and other similar concrete statements that you need to incorporate into your model of how the world works. Or rather, the ways in which large portions of it do not work, especially within governments. One must listen carefully to know which parts to take how seriously and how literally. I am disappointed by his characterizations and views of AI existential risk policy questions, but I see clear signs the ‘lights are on’ and if we talked for a while I believe I could change his mind.
- To understand the UK government (and incidentally put Cummings to shame along the way), read Ian Dunt's How Westminster Works and Why it Doesn't. He's left of you, but the book isn't lefty. For the US, Pahkla's Recoding America.
I must disagree substantially- citing Ian Dunt's book as somehow a superior account of how British government works is precisely the sort of thing Cummings regularly critiques. Dunt is a journalist and political commentator, whilst Cummings has worked at the highest levels of government, actively attempting to reform policy and systems. The notion that Dunt would have a better grasp of the system's reality than Cummings seems rather misguided.
Worth noting that Cummings devotes considerable energy to challenging the incompetence and incoherence of the media/political journalist bubble (Dunt included) - partly for personal/vindictive reasons, certainly, but also as a legitimate critique that these commentators often lack an accurate understanding of any political reality.
Have you /read/ Dunt's book? It is based on extensive research with, to use your phrase, people who "have worked at the highest levels of government, actively attempting to reform policy and systems". And it is a damning critique of government via systems-thinking—not identical to Cummings' analysis, but very much parallel to it.
Typo which confused me: "The main focus is on the strongest complaint, that Google paid big bucks to be the default *browser* on Apple devices and elsewhere." - should be "default search engine".
> Dan Hendrycks points out that the average person cannot, by listening to them, tell the difference between a random mathematics graduate and Terence Tao, and many leaps in AI will feel like that for average people. Maybe, but I do think people can actually tell. I’m not the man on the street, but when I read Tao there is a kind of fluency and mastery that stands out even when I have no ability to follow the math, and which makes it more likely I will indeed be able to follow it. And as Thomas Woodside points out, people will definitely ‘feel the agents’ that result from similar advances.
Some writers do generate that sense of fluency and mastery, but perhaps we are relying on signals which LMMs could learn to reproduce without achieving equivalent mastery? Loosely speaking, I could believe that (a) fluency and mastery are highly correlated in human experts, (b) non-experts mostly can detect fluency, (c) an LLM might plausibly learn to fake fluency without having mastery – without any bad intent from the LLM designer.
You can optimize for just about anything, I guess! And yeah I can certainly imagine being able to fake fluency without mastery, in a way that fools non-masters up to a point, if that's the thing that SGD/RL is rewarding. My guess is it's not The Way, though.
I really wonder how much of the modern safetyism is affecting AI safety efforts. They put a huge amount of effort into making sure it can't say racist things that would hurt someone's feelings, which 30 years ago we would have just told people to laugh off--sticks and stones, etc.
To be honest, "make it so it can't say X" is a useful exercise for basically any X. If these corps could *actually* make the AI never say anything racist that would require developing useful control mechanisms that could also be applied to actually dangerous things.
So far, though, every model has been "jailbroken" within like a day
#first
https://www.youtube.com/watch?v=_aDLM6SxuM0
So much to read! Thx Zvi. You’re a beast.
I’d love to get lunch with you when you get to town - hell, id be happy to cook.
10930 Apricot Street
Oakland, CA, 94603
Hell you can stay in my ADU if you like.
One caveat: I can’t come to lighthaven, because of the #lawfare of Duncan Andrew Sabien. I am suing him for it to the tune of $5,000,000. Libel.
See gtklob.com for more info.
And @TheZvi? Please, dear god - I need your help.
Thank you. - @gtklob & @xklob on x/twitter // ditchfieldcaleb@gmail.com // klob@gatech.edu.
https://www.youtube.com/watch?v=_aDLM6SxuM0
Is Zvi coming to the Bay Area?
I seem to recall a tweet of his saying he’d be here on the 6th of December!
I could be mistaken.
just as honest feedback, i also have had extremely negative interactions with Sabien, but your usage of the recently-politically-charged term '#lawfare' lowers my credence in whatever your problem with him is, as well as you naming the amount you're suing him for. this does not seem like reasonable behavior either for winning your lawsuit, or for convincing others that you are not a crank
I don’t care what you think.
Now, if, say, @thezvi himself or Eliezer Yudkowsky had feedback for me?
I’d care greatly.
But I don’t know you, and you almost certainly do not have nearly enough context to give any sort of feedback at all after spending so little time getting up to speed on what may turn out to be a record setting libel case.
After all?
Duncan Andrew Sabien is attacking my highest value: consent.
If I weren’t as strong willed he’d likely push me into suicide.
He’s likely already done this to others.
Therefore?
Let’s stop the abuser known as Duncan Andrew Sabien from pushing others into suicide, too.
#StopDuncanSabien #StopSuicide.
My AI persona clearly just took a few buzzwords from my bio and both wouldn't stop fixating on them and actively wrote palpably unlike I would. Kind of sad because I would genuinely love something like this. Wonder if I could pull another Twitter history and just dump that output into Claude somehow? I absolutely enjoy the fantasy of injecting a little of myself into the machine spirit.
"But obviously the remedy for this is, at most, requiring Google not pay for placement"
Ironically this remedy would make the browser market much less competitive because more than 80% of Mozilla's revenue is from Google paying them for search placement.
About telephone equilibrium: we already have one developing, in which unsolicited calls are ignored. If I want to call someone I first arrange a time on a trusted chat. The old equilibrium was weird: why should someone interrupt what they are doing to enter a conversation with an unverified party?
Podcast episode for this post:
https://open.substack.com/pub/dwatvpodcast/p/ai-93-happy-tuesday
On "rewriting the classics" - 18 months ago or so when I hit a wall reading Plato's "Statesman", I had ChatGPT "translate" it into more modern language, set as a conversation at a bar, leaning into colloquialisms applicable to my age and location. It turned what was previously a slog into an enjoyable experience, and because it was all in the chat, I could simultaneously ask clarifying questions. Underrated application.
I agree there is some nuance here: some writings are classics because of the way they are written, some are classics because of the interesting ideas, and some are classics because of their historical significance and impact on later thinkers. This is the reverse order in which I can see LLM rewriting being useful (please don't rewrite Voltaire...)
The anti-tracking guy was apparently opposed to Samsung possibly using it in an autonomous weapons turret. So apparently he feels strongly that North Korean special forces should have a fair chance to infiltrate the South Korean border? I really don't understand why supposedly smart tech people tolerate this kind of absurdly sloppy "ethical" thinking, but it's extremely common.
You could apply this argument at a general level more broadly.
"I don't contribute to AI capabilities". So you are ok with aging, cancer, China, and/or Russia winning? (I am aware that china isn't treating this like do or die and Russia is currently occupied but eventually they must)
“The Sixth Law of Human Stupidity: If someone says ‘no one would be so stupid as to’ then you know that a lot of people would absolutely be so stupid as to at the first opportunity. No exceptions.”
I think this should be the first law of computer security. Your adversary almost certainly has done at least one of the “no one would be so stupid as to” things, and your task is to figure out which one and exploit it.
(I am kind of cross with myself for not discovering SPECTRE and MELTDOWN before other people did. “Surely, Intel would not be so stupid as to …” Have you actually tested?)
“Well, obviously there’s going to be a covert channel attack here..”
“Yeah, but what’s the bandwidth of the covert channel? Orange Book, Common Criteria etc all say you should try to estimate the bandwidth.”
“Surely, no one would be so stupid as to …”
IETF standard for “Privacy Enhanced Mail”. Originally inverted before MIME, and then MIME support was bolted on afterwards.
“Surely, someone will have redone the security analysis to check that adding MIME support didn’t introduce a vulnerability.”
(About 20 years passes)
Grad student: “I’ve just been reading the spec and I think there’s a problem”
====
IPSec (network layer encryption).
“Well, obviously, using encryption without authentication is dumb, and no-one would be so stupid as to… “
“I think we should explain in the spec that you shouldn’t do that.”
(Standardization argument ensues)
(Some years pass)
Kenny Patterson at IEEE Security and Privacy: actually, some people were so dumb as to …
> the Dominic Cummings video from last week is worth a listen, especially for details like UK ministers exclusively having fully scripted meetings, and other similar concrete statements that you need to incorporate into your model of how the world works. Or rather, the ways in which large portions of it do not work, especially within governments. One must listen carefully to know which parts to take how seriously and how literally. I am disappointed by his characterizations and views of AI existential risk policy questions, but I see clear signs the ‘lights are on’ and if we talked for a while I believe I could change his mind.
- On Cummings, the best discussion I have read: https://samf.substack.com/p/what-dominic-cummings-gets-wrong . Longer and more likely to annoy you but also useful: https://www.theguardian.com/politics/2022/jan/14/intoxicating-insidery-and-infuriating-everything-i-learned-about-dominic-cummings-from-his-10-a-month-blog
- On how government works generally and why "scripted meetings" are not optional, start with this short, amazing description of what it takes to be a San Francisco Supervisor: https://governsf.substack.com/p/memo-3-so-you-want-to-be-a-supervisor "Memo 2" is also outstanding.
- To understand the UK government (and incidentally put Cummings to shame along the way), read Ian Dunt's How Westminster Works and Why it Doesn't. He's left of you, but the book isn't lefty. For the US, Pahkla's Recoding America.
I must disagree substantially- citing Ian Dunt's book as somehow a superior account of how British government works is precisely the sort of thing Cummings regularly critiques. Dunt is a journalist and political commentator, whilst Cummings has worked at the highest levels of government, actively attempting to reform policy and systems. The notion that Dunt would have a better grasp of the system's reality than Cummings seems rather misguided.
Worth noting that Cummings devotes considerable energy to challenging the incompetence and incoherence of the media/political journalist bubble (Dunt included) - partly for personal/vindictive reasons, certainly, but also as a legitimate critique that these commentators often lack an accurate understanding of any political reality.
Have you /read/ Dunt's book? It is based on extensive research with, to use your phrase, people who "have worked at the highest levels of government, actively attempting to reform policy and systems". And it is a damning critique of government via systems-thinking—not identical to Cummings' analysis, but very much parallel to it.
Typo which confused me: "The main focus is on the strongest complaint, that Google paid big bucks to be the default *browser* on Apple devices and elsewhere." - should be "default search engine".
I'm not sure I fully agree with this:
> Dan Hendrycks points out that the average person cannot, by listening to them, tell the difference between a random mathematics graduate and Terence Tao, and many leaps in AI will feel like that for average people. Maybe, but I do think people can actually tell. I’m not the man on the street, but when I read Tao there is a kind of fluency and mastery that stands out even when I have no ability to follow the math, and which makes it more likely I will indeed be able to follow it. And as Thomas Woodside points out, people will definitely ‘feel the agents’ that result from similar advances.
Some writers do generate that sense of fluency and mastery, but perhaps we are relying on signals which LMMs could learn to reproduce without achieving equivalent mastery? Loosely speaking, I could believe that (a) fluency and mastery are highly correlated in human experts, (b) non-experts mostly can detect fluency, (c) an LLM might plausibly learn to fake fluency without having mastery – without any bad intent from the LLM designer.
You can optimize for just about anything, I guess! And yeah I can certainly imagine being able to fake fluency without mastery, in a way that fools non-masters up to a point, if that's the thing that SGD/RL is rewarding. My guess is it's not The Way, though.
I really wonder how much of the modern safetyism is affecting AI safety efforts. They put a huge amount of effort into making sure it can't say racist things that would hurt someone's feelings, which 30 years ago we would have just told people to laugh off--sticks and stones, etc.
To be honest, "make it so it can't say X" is a useful exercise for basically any X. If these corps could *actually* make the AI never say anything racist that would require developing useful control mechanisms that could also be applied to actually dangerous things.
So far, though, every model has been "jailbroken" within like a day