Meta Pivots on Content Moderation

There’s going to be some changes made.

Table of Contents

1. Out With the Fact Checkers.

2. What Happened.

3. Timing is Everything.

4. Balancing Different Errors.

5. Truth and Reconciliation.

6. Fact Check Fact Check.

7. Mistakes Will Be Made.

8. Where We Go From Here.

Out With the Fact Checkers

Mark Zuckerberg has decided that with Donald Trump soon to be in office, he is allowed to care about free speech again. And he has decided it is time to admit that what was called ‘fact checking’ meant he had for years been running a giant hugely biased, trigger-happy and error prone left-wing censorship and moderation machine that had standards massively out of touch with ordinary people and engaged in automated taking down of often innocent accounts.

He also admits that the majority of censorship in the past has flat out been a mistake.

Zuckerberg later talked more about this, and many related and unrelated things, on the Joe Rogan podcast. He says many fun things, like that most companies need ‘more masculine energy’ to balance their feminine energy, and he gives his usual terrible takes on AI which I cover elsewhere.

Zuckerberg is going to overhaul Meta’s entire moderation and censorship structure, replacing it over several months with something akin to community notes. He’s going to move standards back in line with actual community standards. And he’s going to move his content moderation teams from California to Texas, and push back against censorship worldwide, highlighting Europe and Latin America.

What Happened

The current review process reportedly started when one of Zuckerberg’s own posts got throttled because of concerns over medical content, and then snowballed from there.

How did it all go so wrong? Zuckerberg tells the story on Rogan, that he took complaints about misinformation and the need for fact checking as sincere, then after he hired people for this the slippery slope took over and before long they were censoring things that are in the mainstream discourse.

Here is a parallel discussion, about similar pressures at Zuckerberg’s philanthropy efforts, where he got asked to resign from Facebook during a staff meeting for insufficiently moderating the actual sitting President of the United States.

Matthew Yglesias: Lots of moderate people end up embracing conservatism because of backlash against left-wing excess, but it’s funny to do it when the left-wing excess was literally your own hiring and business decisions.

Kelsey Piper: I think reasonably often middle-aged people hired recent college graduates through pipelines that had historically produced nice moderate liberals like themselves and discovered in shock that now they were producing illiberal leftists who thought the org should be run their way.

Of course the correct response here is to fire this person and hire the people you meant to hire, but hiring is hard and the people you wanted were suddenly hard to find and the existing processes were not producing them.

I know of a number of nonprofits that had an unpleasant shock waking up to this. Some said "wait, no, this isn't what we're doing" and had internal drama as they parted with the illiberal employees and survived. Some did not.

But I think it took a really unusual level of institutional leadership and courage to go in 2020 "what? no. that's not what we're doing here. if you want to do that, leave." And the orgs where it did happen tended to keep it quiet so they wouldn't be a target of outrage.

Also he claims the Biden administration would yell and threaten various people in phone calls, demanded they take down even true information about Covid if it would discourage vaccinations, said not doing so was ‘killing people,’ and when Zuckerberg drew the line at censoring true information (I’d say, ‘somewhat?’) the Biden administration suddenly made good on its jawboning threats and all the investigations came down on Meta’s head.

That’s his story. We have at least some documentary evidence that Facebook responded to White House demands by censoring and removing posts that ‘did not contain actionable misinformation.’ Some more evidence is this Patrick McKenzie report of his volunteers at VaccinateCA getting blocked on Facebook back in 2021 for their ‘unusual interest patterns.’ You can decide how much to believe Zuckerberg’s account of all this.

Timing is Everything

Now that the Biden administration is on its way out, and the vibes have shifted, it’s time for a change. Zuckerberg explicitly says he waited until after the election (partly because during one is an awkward time for major changes) and that he was deciding largely based on the vibe shifts.

Benjamin Hoffman: Zuckerberg is not literally a liberal; he’s a pragmatist used to passing as a good-enough simulacrum of a liberal to make other people doing the same feel comfortable with him.

If he were literally a liberal Facebook’s behavior in the past few years would have been liberal.

If he were even trying to be mistaken for literally a liberal, his explanation of Facebook’s censorship policy change would have been in terms of a subversion he’s noticing & correcting rather than in terms of vibe shift.

Having listened to the Rogan podcast, I do think Zuckerberg has some amount of preference for more free speech and other classical liberal preferences, but yes it all primarily sounds very pragmatic, and he’s definitely not left-wing in today’s parlance.

I also agree with DHH here that it would have been highly supererogatory for Zuckerberg to make these changes before Trump’s win rather than after, and that Zuckerberg has earned no one’s trust yet, but that it is not a reasonable ask to expect him to have done all this earlier, even if you don’t fully buy his stories about the Federal government half going to war against him.

Balancing Different Errors

So what to do now? He’s going to focus his filters, rather than scanning for any violation at all, on illegal and high severity violations, and only act on low severity actions if and when someone reports them. I worry that if this is the policy than there will be various people who decide it is their job to use their AIs (or just their eyes) to go searching for violations to report, but it would still have humans in the loop in every step.

Whereas ‘the filters make mistakes’ so he’s going to dial them back and require a lot more confidence than before (yes this is a trade-off, he discusses it more on Rogan, but there’s a lot of ‘what the hell were we doing before?’ here), essentially also admitting that there was no way to appeal to humans when those mistakes happened, or that those answering those appeals were insane. He’s going to ‘reduce the number of innocent people’s posts and accounts that we accidentally take down.’

Truth and Reconciliation

And accounts? Yeah, this kind of used to happen a lot, with no way to fix the mistakes. The first reaction I saw to this change was ‘I’ll believe this change matters when [X]’s account, which was banned without explanation, gets reversed.

I don’t see any indication of a plan to undo the mistakes of the past here?

He says they’re going to ‘bring back more civic posts,’ because people want to see such content again. Why not let people choose which content they want to see?

These are all highly welcome changes, especially the move to Community Notes and generally vastly raising the bar before things are censored, even if he basically admits that he was previously ‘going with the flow’ and bowing to pressure, and now he’s bowing to a different kind of pressure.

I hate to kick even this man while he’s making great changes. I want to be clear, my primary response is that these are centrally great changes.

It also seems like we need more before we can properly move on. There’s a lot of ‘you were doing WHAT?’ moments here, and a lot of ‘when did who know what about that exactly?’ all of which should rise to the level of requiring Truth and Reconciliation.

In particular, we need a blanket reinstatement of accounts whose violations would not get you banned under the new rules. Meta’s AI can presumably do reviews of the past content based on new standards and reinstate many accounts. The new Texas crew can then manually review any cases where confidence is not super high.

I also agree with Aella that if your platform bans porn, it isn’t really allowing free speech, which is especially relevant to Instagram for obvious reasons.

Fact Check Fact Check

Nate Silver writes of The Rise and Fall of Fact Checking. He points out that a lot of the bias in fact checking is in selecting what to ‘fact check,’ which usually targets unresolved or unresolvable claims, because if it was resolvable you didn’t need a fact check. And that ‘fact checking’ ended up often being a way to use an argument from ‘no evidence’ to call things the fact checkers disliked ‘misinformation,’ and the whole enterprise often aims primarily to scaffold and support a narrative.

Whereas the best use of professional and distinct fact checkers is to use them on yourself, as Nate Silver did for his book On the Edge. I would love to have the ability to do this prior to publication, but speed premium does not allow it.

If you want more detail on how insane ‘fact checkers’ and their claims of ‘misinformation’ had gotten, you can see this good post by John Barro. It was clear for a long time, but the treatment by such people of claims about Biden’s declining health made how they work fully common knowledge.

That post includes this quote for the ages from one such person:

Josh Barro: “Of course,” van der Linden replied. “We can't just be saying random stuff without expert assessment, especially on medical issues.”

Any questions?

Advocates for such policies are, as you would expect, ‘having a normal one.’

Casey Newton (Headline): Meta Surrenders to the right on speech.

“I really think this is a precursor for genocide,” a former employee tells Platformer.

One could say Zuckerberg ‘surrendered on speech’ by deciding that he is for it.

Or more elegantly, chef’s kiss:

To be fair to Newton, it sounds like the end of fact checking was relatively non-alarming to Meta employees, who were instead worried about the loosening of the content guidelines and thresholds. He cites, as many others did, the one especially bonkers policy, which is the main topic of the next section.

But the logic being argued for by such advocates is clear. They think that free speech causes harm, for very broad values of harm, so they are against free speech.

The Babylon Bee offers a thread of past ‘fact checks’ of its satirical posts. Many of these are pretty funny. The fact checks only make them funnier.

Mistakes Will Be Made

This is a good time to check out Asterisk’s piece on The Making of Community Notes.

There are also some… interesting choices in the new content policy?

Reading the entire rationale is looking into a world and philosophy very different from my own. I do want to emphasize that getting this kind of thing right is very difficult, mistakes will be made especially at first, they have to worry about laws and customs around the world very different from our own, and all that. It’s hard.

Alexa Corse, Meghan Bobrowsky and Jeff Horwitz: Meta on Tuesday also revised community standards to significantly loosen restrictions on content previously considered hate speech. For example, the updated rules permit “allegations of mental illness or abnormality when based on gender or sexual orientation” and strike down a prohibition on comparing women to “household objects or property.”

Specifically, Meta’s policy includes: “We do allow allegations of mental illness or abnormality when based on gender or sexual orientation, given political and religious discourse about transgenderism and homosexuality and common non-serious usage of words like “weird.””

Victoria: According to Meta's new community standards and how it defines hateful conduct, it is a violation of policy to call someone mentally ill or abnormal - unless that person is trans or gay. What the actual f***.

Both Claude and I think Victoria’s statement is accurate here? Which seems rather insane. I’m basically fine with ‘no one gets to call anyone mentally ill’ if you want to go that way. I’m also basically fine with ‘everyone gets to call everyone mentally ill for pretty much any reason, there’s a block button for a reason.’ But this? What?

It’s even weirder when the whole concept is general prohibition of attacks on individuals on the basis of ‘protected characteristics’ (unfortunate acronym PC) that includes sexual orientation, sex, gender identity and serious disease. What are we even doing?

At some point, [X] is going to say ‘[Y] is mentally ill’ and [Y] is going to say ‘how dare you accuse me of being mentally ill simply because I am mentally ill!’ and [X] is going to replay ‘it’s okay under Facebook rules, because that’s not why I said it. I said it because of your gender identity.’

404 Media is reporting this is causing ‘total chaos internally at Meta,’ based on having talked to five people. I am guessing it was not total chaos at Meta, but that there were people who were very understandably upset.

There’s also this rather amazing special exception for those breaking up:

Meta: Tier 2 not allowed: Targeted cursing, except certain gender-based cursing in a romantic break-up context, defined as:

• Targeted use of “f***” or variations of “f***” with intent to insult, such as “F*** the [Protected Characteristic]!”

• Terms or phrases calling for engagement in sexual activity [or contact with various things.]

This isn’t an accident:

Finally, sometimes people curse at a gender in the context of a romantic break-up. Our policies are designed to allow room for these types of speech.

Movie ideas:

1. Two people agree to fake a breakup in order to justify speech on Facebook. The speech escalates, they fight and targe each other for real, they of course fall in love.

2. Two people are breaking up, but neither wants to alter their relationship status on Facebook, because the other would then be able to post horrible things about them. So they have to keep pretending they are together. You know the rest.

   1. Bonus version: A version set in a slightly different policy, where whoever changes their status first, the other one gets to do this as the dumped person. So the two of them do increasingly convoluted things to get the other to change their status and acknowledge the breakup.

Finally, there’s this, and I don’t know if they thought this through or not?

Do not post:

• Content explicitly providing or offering to provide products or services that aim to change people’s sexual orientation or gender identity.

Where We Go From Here

I expect the change in approach to content moderation to be broadly permanent at Meta and elsewhere, unless it is brought down by foreign legal action.

I expect the new equilibrium to have a meaningful much higher level of allowed speech across the board.

That doesn’t mean every individual change will stick. While humans are in charge, we will continue the dance of fighting over exactly what the rules are and should be. There are some rather obviously ludicrous rules that were discussed in the last section, hopefully they will quickly be addressed.

The biggest threat to the new equilibrium will be foreign governments demanding various forms of censorship, especially the European Union. Facebook and Instagram are likely for this purpose a much bigger deal than Twitter, so they should put up a real fight, as their position on free speech is that they are strongly against it. I don’t know what will happen there. So far things seem unexpectedly quiet.

Indeed, the action we have is that Google is flat out telling the EU no on its legal demand that search results and YouTube videos use fact checkers and then use the results in ranking or removing content. Google says this ‘simply isn’t appropriate or effective for our services’ which is very true. Google says this won’t come as news to the relevant officials, who have been informed Google will ‘pull out of all fact-checking commitments in the Code before it becomes a DSA code of conduct.’

Something will have to give. The European Union is saying ‘thou shalt censor the ways we want’ and our largest tech companies are saying, ‘no actually we shalt not.’

I don’t know what happens. The EU demands here are not reasonable, but the EU very much sees it the other way around. If no one backs down, eventually the internet is going to fracture. I am confident Google and Meta do not intend to back down.

On a personal level, this makes it in theory possible to consider posting on Instagram or Facebook at all. When I was posting on Covid, I had requests to post my content to Facebook to help reach people who would not otherwise see it. I was already Against Facebook (with follow-ups) back in 2017, but when I considered reconsidering, I realized that if I posted my content there I would likely get my account banned. I want to keep access to my accounts with Meta, especially WhatsApp and using Facebook as a rolodex, so that wasn’t an option.

(In terms of contacting me, I will see friend requests and Facebook Messenger messages about once every few months, with a roughly Poisson distribution, and will approximately never see a Facebook post of any kind.)

Once these changes are made, it becomes an option again. I see no particular reason to take it, but doing a bit of exploration and controlled experimentation is non-crazy, and I will consider what might make sense.