37 Comments

"Schools should track data about the efficacy and effects of drills."

I wonder how exactly they expect to measure the "efficacy" of drills. As you rightly point out the probability of being involved in a shooting situation is vanishingly close to zero. So... what are they going to measure and compare? What kids do in subsequent drills?

The out and out lies the activists and politicians spread about school shootings have made people crazy. In the "entirely detached from reality" sense, that is; in a world where any school had a 5% chance of experiencing a mass shooting in a given year this might make sense, although at that point one should ask "why do we have these schools?"

Expand full comment
Jun 16, 2022·edited Jun 16, 2022

In theory "tracking efficacy" is not bad. We should measure the efficacy of fire drills, too.

Although if the mean-time-to-fires for each school was 6000 years maybe we shouldn't really bother that much.

Expand full comment

I'd push back and say even in theory tracking efficacy when doing so is impossible is a bad thing :) Chances are really good anything you produce that is greater than nothing is false and misleading, and it is going to cost you something to do. Even just implying that you are able to do such a thing as tracking efficacy is a misleading lie that I really wish people would stop telling.

Expand full comment
author

I mean yes, if it really was 5% a lot of what they were saying would make sense but also letting your child actually go to a school with that high a shooting risk is obviously insane.

Expand full comment

Indeed. I get the feeling many people believe the risk is in that ballpark instead of closer to 0.0005% or whatever it is. That might be not be enough leading zeros there. I expect it is a lot more like the "how many unarmed black men are killed by police each year?" numbers people report, which are usually off by an order of magnitude at least. Constant repetition mess with people's perceptions of the actual numbers involved.

Expand full comment

I've been trying to (patiently) push back – as gently as I can too! – against literal, unqualified, and apparently sincere, statements like "The police are useless.". I feel very Sad for us all given how warped our 'estimated base rates' can get from following most news sources. Oof! Shit's bad.

Expand full comment

Yea, getting people to understand relative risk is very difficult, an people get very touchy when you point out that something they are worried about is actually less likely to happen than many things they don't care about. Which is striking to me; you'd think people would be relieved and not annoyed!

I think part of it has to do with Smith's observation that while we are almost indifferent as to whether other people share our loves, we are very keen that they share our hates. So if I don't worry about the same things I am effectively saying I won't be a reliable ally against your enemies, at the emotional level.

Expand full comment

Very insightful!

Expand full comment

Funny enough tho, there is a kind of way to (eventually) get people to understand relative risk – the various therapies (e.g. CBT) for OCD!

Betting, or even just making advance predictions, is also pretty effective. (Tho some people will probably just refuse to bet/predict to avoid 'being wrong'; very sad.)

Expand full comment

That's an interesting point about CBT. That makes a lot of sense now that you mention it, as the whole goal is to walk people back from their tendency to overly worry. Haidt and Lukianoff make largely that point in The Coddling of the American Mind. I never quite linked that up myself.

I suppose the big issue is getting people to admit they have a mistaken thought process and want to change it. As you say, getting people to bet or predict is tricky, and even if they do they will often refuse to acknowledge that they were wrong if there is any wiggle room. If we could normalize betting and predicting though, that would be a big improvement :)

Expand full comment

To be at all feasible to measure, _every_ school across the country would have to basically perform _exactly_ the same drills, and even then I'd expect any conclusions to be so confounded by noise that no reasonable result would remain.

Expand full comment

Indeed. Anything produced would be far more likely to mislead, even if done in good faith. More likely, anything produced will say exactly what those who produced it want it to say for maximum leading.

Expand full comment

It definitely doesn't even seem remotely worth giving them a chance to bullshit us further!

Expand full comment

Presumably send someone disposable in with a paintball gun and see how many students he "kills" before they tackle and disarm him. Since students are practicing cowering rather than ambushing, I assume the goal is to maximize casualties.

I say someone disposable because the best ambush is one-hit-down, and even in a slower ambush, you can't count on kids stopping as soon as the shooter shouts "It's just a drill" or "I surrender". Stopping in a situation like that is actually hard.

Expand full comment

It's pretty horrible (or horrifying) to realize that, were the kids to be 'adequately trained', that should then result in them _killing_ whomever's playing the part of the attacker in a drill!

Expand full comment

It’s been interesting to see “trauma” become a more and more important word in our culture, with its definition evolving. Traditionally, a “trauma” was an event hat led to post-traumatic symptoms, which means that you can't necessarily call a bad event a "trauma" prospectively, without knowing whether it will lead to PTS symptoms, since a commonplace among students of trauma is that different people can have very different responses to the same event.

Now, many people, including Zvi here, seem to be using "trauma" to mean “a bad thing,” or “a thing that makes people feel bad.” So "traumatized" means someone being upset by an event. (Because the evidence of e.g. increasing social media posts about pain or being nervous certainly does not support a conclusion that active shooter drills are causing actual post-traumatic stress disorder, and if they were, it would almost certainly be in only a small fraction of kids.)



So when you say “we shouldn’t do X because it traumatizes kids,” it means the same thing as “we shouldn’t do X because it upsets kids.” And then you’re engaging in one-sided thinking, by only considering the harms of X, and not the benefits. Yes, it seems clear that active shooter drills upset kids, but their proponents no doubt think that temporary bad feelings are a reasonable price to pay in exchange for the benefit.

This post fails to steelman.

Expand full comment
author

I do think I consider the benefit, which I quite reasonably cap at reducing the damage from actual shooters? And I say explicitly that costs exceed benefits even if prevented all school shootings outright, which they obviously don't come close to doing? So I notice I'm confused here - you can disagree with the calculation but it wasn't skipped.

I agree that I failed to steelman, but... what's the steelman, beyond claiming it prevents shootings effectively (which is already a steelman, I very much doubt this)?

I do agree that often the word trauma has been used in places it doesn't belong, but I don't agree that I'm using trauma here to mean anything that upsets someone - I think this actually is trauma, making someone feel under attack with acute stress in ways that leave potential long lasting impacts.

Expand full comment

>And I say explicitly that costs exceed benefits even if prevented all school shootings outright...

I saw you said that, but there wasn't a lot of argumentation behind that. Proponents of these drills might think that lots of upset kids are worth it if it prevents one death (for example). Or even lessens the likelihood of one death. And if you think that is not the case, you need to make that argument. Or better yet, before making that argument, seek out some proponents of active shooter drills to understand their point of view. Because, if you don't see the steelman, you're probably not the right person to write a convincing argument on this.

> I think this actually is trauma, making someone feel under attack with acute stress in ways that leave potential long lasting impacts.

Thanks for that definition. But there aren't a lot of bad things that would not have *potential* long-lasting impacts, right? so I'm not seeing a lot of daylight between trauma as any bad thing and trauma as any bad thing that *might* have bad consequences.

Expand full comment

I felt like he DID make the argument that even knowing, to any sufficient certainty, that these drills saved _one life_ is practically impossible. Just 'knowing' in a 'p-value' kind of way – which is totally inadequate for this – would be extremely expensive and difficult. (We'd very probably end up with 'an interpretation of noise' instead accurate or useful knowledge.)

And if we're really arguing about the possibility of saving literally one life, at the cost of traumatizing thousands (or some number of OOM beyond) of kids, it just obviously seems like a terrible tradeoff. Where's the argument that _that_ is, actually, a totally reasonable tradeoff to make? Would that same argument form also apply to every other actually rare way that school-aged kids can die?

> so I'm not seeing a lot of daylight between trauma as any bad thing and trauma as any bad thing that *might* have bad consequences.

Are you denying that there is a spectrum of badness? That, e.g. a kid falling down and skinning their knee *might* have bad consequences and thus maybe is 'trauma' too?

It seems to me like there's a scale of trauma too. That 'trauma' is mostly a kind of very 'bad' thing that also persists longer than 'just bad' things do, prototypically anyways.

Bumping into something and bruising yourself is bad but probably not traumatic.

Falling and breaking a bone is pretty bad and maybe mildly traumatic – I'd expect a quite wide variation in potential badness or trauma, tho probably not particularly extreme trauma in almost all cases. (Someone or something deliberately breaking a bone probably _would_ be traumatic almost all of the time!)

I would expect that participating in an active shooter drill is very bad and pretty traumatic for most/a-lot-of kids.

I am very certain that being in a _real_ active shooter _situation_ (tragedy) is extremely bad and very traumatic.

What are the other "bad things" that you had in mind that you also think are definitely not trauma?

What is the scale of badness/trauma for various things that you think it's reasonable for schools to mitigate like they're doing with active shooter drills?

Is there anything that _always_ causes PTSD in anyone? I don't know of anything like that; nothing particularly common anyways. Is nothing 'really traumatic' then? Where do you think the line should be drawn? (And do you think human language works in a way that everyone could finally 'agree on what exactly words mean'?)

Expand full comment

I think Zvi was using 'trauma' in the 'traditional' sense you mention.

Expand full comment

Everything I ever hear about public (and most private) schools makes me more glad that I was homeschooled and more determined to homeschool my own children should I ever have any. This is complete madness.

Expand full comment
Jun 16, 2022·edited Jun 16, 2022

I don't believe the quant results of "bad" that the study claims. Every study that analyzes student reaction to anything (masks, drills, homework, changing vending machines from coke to Pepsi) always shows very strong results.

In reality, students are pretty resilient. I think we should adjust our priors against the studies.

At the same time, obviously, active shooter drills are extremely counterproductive to the point of irony and shouldn't be held. I don't think we need a lousy study to prove the point, I actually think it could weaken the point. (A shooting once every 5000 years is compelling enough)

Expand full comment

I think you're right that kids ("students"), and people generally too, are pretty resilient. That doesn't warrant us going out of our way to do bad things to them tho. If anything, it should mostly just temper are fear and anxiety about the most extreme worst-case predictions of the consequences.

I would be more willing to accept that student's reactions "always [show] very strong results", and what you think that implies about things like this, if the students/kids/people/survey-takers were given options to _trade_ among 'reactions' to different things. Otherwise, what's the 'cost' of answering 'Fuck that' to every stupid thing they ask about? (I know I would frequently answer that way.)

I'm not sure it's obvious to me that active shooter drills are "counterproductive" – I don't know what 'productive' would be. Would 'productive' mean that students fared better _if_ there was a real active shooter? It just doesn't seem sane to prepare for terrible but extremely rare events. _General_ 'emergency preparedness' seems sensible, but precisely because it applies to basically any terrible event.

I expect that a lot of people will be trained to interpret 'active shooter alarms' as _drills_. And I'd expect that people won't _be able to_ maintain any real enthusiasm for these drills if they do persist for any significant duration.

Expand full comment

$2.7 billion. Shit. We'd be better off if the money was literally lit on fire.

> $5 billion in additional funding for childhood trauma.

Omg this isn't just bad writing style, it really is funding FOR CAUSING trauma.

I think some of the people advocating for drills are trying to make the situation worse for the children because "if we make the trauma for them BAD ENOUGH then we can FINALLY ban guns." I don't know if that's true or not but it's evil to make kids pawns in this game.

Expand full comment
author
Jun 16, 2022·edited Jun 16, 2022Author

Yes, I probably should have said for causing it, edited. I went through a few versions of that and didn't realize the final one had an alternative interpretation.

Expand full comment

Considering how much the media and politicians over inflate the risk of school shootings, I would wager pretty heavily that the plan is "leverage fear about school shootings to garner support for banning guns." The status of children is of no concern in the end, only as a means. The more horrible the process of "preventing school shooting deaths" is, the easier it is to achieve their actual ends of banning guns.

And shoot, someone is getting paid to design and execute those drills. For that kind of money there are going to be a lot of people advocating pulling the trigger, whether the benefits are positive or negative.

Expand full comment

I remember the bomb drills that were supposed to protect us from nuclear attack back in the 50s. Even as a 5th grader, I can remember thinking how stupid they were. But I rather firmly believe that the resistance of Baby Boomers to nuclear power grows out of the fear engendered by the drills.

Most of us are aware of the attempt to tilt the 2020 election in the direction of the Democrats by infusions of money for the purpose of relaxing the rules surrounding such things as voting by mail. (No, I’m not saying the election was “stolen” - it was, however, tilted.) https://time.com/5936036/secret-2020-election-campaign/

With that in mind, the notion that a similar attempt is being made concerning active shooting drills is not unwarranted. Even if it is not what is happening, I’m sure that many people would regard "leverage fear about school shootings to garner support for banning guns" as a welcome side effect.

But I would say that the real side effect to worry about is the fear itself - that we are teaching children to live in constant fear of anything is crazy.

Expand full comment

Yikes! I don't think I'd considered how the nuclear drills might have, and fairly directly, caused the extremely low support for nuclear power. That's bleak!

Expand full comment

That's an interesting point! I remember thinking the nuke drills were crazy as a kid too, but I never considered that the ones who didn't probably were the same ones who opposed nuclear power later. I can now think of a few kids who I would bet good money fitting that to a T, however.

It is amazing to me to much "constant fear" has really become a standard feature in our lives. Not only in the THS, 1984 sense of "That's how they rule you!", but just in the sense of people putting up with it. As though a very large percentage of people believe they will never die unless something kills them, so they had better wrap themselves in bubble wrap. I don't know, but the trade off seems really skewed to me.

Expand full comment

I think, at least among the people that are being targeted for this manipulation, that they do, sincerely want to 'ban guns'.

But I would be surprised if _more_ people don't (loudly) 'revolt' about these drills. They seem much obviously worse than even the other terrible things schools do to their students.

Expand full comment

It’s a very good business model. Protecting people from something that is incredibly unlikely to happen means that your efficacy will probably never be measurable. Wait until a shooter says they personally enjoyed a good drill

Expand full comment

I've been wondering for awhile if lockdown is even an optimal strategy for mitigating spree shooters.

As far as I can tell we do not practice lockdown anywhere except for schools. Workplaces, public places, there are no drills and no lockdown plans. If there is a shooting, most people will instinctually flee from the sound of gunfire. Excepting coordinated ambushes, or mistakenly fleeing into a dead end, that seems like a good, if not the best, strategy.

Guns have a limited effective range, and most spree shooters are not trained marksmen. Distance and shooting angles scale linearly. If you present a two foot wide target at 10 feet away the shooter can shoot within a 12 degree angle to hit the target. At 20 feet it's about 6 degrees. At 40 feet it's about 3 degrees. At 400 feet about 0.3 degrees. So, getting distance greatly decreases your chance of getting hit.

Locking yourself into a room is effectively fleeing into a dead end with a door. If the shooter breaches the door, you're very close and have nowhere to run. If the walls are penetrable a shooter could just fire through them and have a good chance of hitting targets. If the walls aren't penetrable, the shooter could still fire into windows from outside.

Where's the evidence that lockdowns are a good strategy? Is the thinking that it's too hard to get kids to run away? Have you met kids?

Expand full comment

I was thinking the same thing – "Run!" is probably a _very_ good strategy in calls of dire emergencies!

I kinda like NYC's (school's) 'emergency' policy: https://www.schools.nyc.gov/school-life/safe-schools/emergency-readiness

It's general and generic. Things like "Form a single file line" seem fine, compared to, literally, just "Run!" – panicked 'stampeding' people can be dangerous. I'd be surprised if that's particularly well observed in actual emergencies.

But I suspect 'lockdowns' and even "Form a single file line" are due to fears about, e.g. a kid running away from the school, by themselves, or in a group, and then getting hurt. From a cost-benefit perspective, the "then getting hurt" is like an obvious cost to pay for a real emergency. From the seemingly more ubiquitous CYA perspective, it's a potentially arbitrarily large cost, even, or maybe especially, in a real emergency. Le sigh

Expand full comment

NYC seems surprisingly good about this: https://www.schools.nyc.gov/school-life/safe-schools/emergency-readiness

It seems good to have a general 'emergency protocol' and I didn't see any mention of active shooter drills, which was great.

I also didn't see anything about NYC schools performing this drills from a cursory review of an 'obvious' Google search; seems good too.

Do you know differently?

Expand full comment

Or, to put it another way: Active shooter drills are as exploitative and abusive of children as is dressing them up for "get ready to get shot on the first day of school" for a political ad. https://socraticgadfly.blogspot.com/2022/08/never-abbotters-go-dumpster-diving-for.html

Expand full comment