Is carrying a gun risky?

I honestly do wish the press — even the science press — would stop reporting on preliminary studies and incomplete results. Or maybe the fault is with the scientists who talk to the press about such results, and the publicity departments of their research organizations, which put out press releases.

New Scientist, which offers a mixed bag of good science reporting and stuff that the editors should have thrown in the rubbish bin, has just given us one in the latter category: “Carrying a gun increases risk of getting shot and killed”.

Packing heat may backfire. People who carry guns are far likelier to get shot — and killed — than those who are unarmed, a study of shooting victims in Philadelphia, Pennsylvania, has found.

It would be impractical — not to say unethical — to randomly assign volunteers to carry a gun or not and see what happens. So Charles Branas’s team at the University of Pennsylvania analysed 677 shootings over two-and-a-half years to discover whether victims were carrying at the time, and compared them to other Philly residents of similar age, sex and ethnicity. The team also accounted for other potentially confounding differences, such as the socioeconomic status of their neighbourhood.

The result? Well:

Overall, Branas’s study found that people who carried guns were 4.5 times as likely to be shot and 4.2 times as likely to get killed compared with unarmed citizens. When the team looked at shootings in which victims had a chance to defend themselves, their odds of getting shot were even higher.

Anyone staring at these pages for a while will know that I dislike guns. I’d love nothing more than to be able to take that result at face value, and to quote it far and wide. I’d love to have a definitive study showing such statistics.

This is not that study, and these results are useful only to prompt further study. As they stand, we can’t conclude anything from them.

The article itself does point out some of the problems, but many readers will miss them. First, this is not a randomized trial, nor even a review of other scientific work. They started with people who were shot. The article points out that practicality and ethics make it difficult to assign people to groups, but perhaps a study that selected people at random and then looked at what happened to them would have a better chance. As it is, the methodology here makes confirmation bias likely.

Second, while they attempted to control for factors such as age, sex, ethnic background, and socioeconomic status, they have not controlled for some major factors, not least of which involve the attitudes and behaviours of the subjects. We can’t say this enough: correlation does not imply causation. Even if we accept that they have shown a high correlation between carrying a gun and being shot, there is no sense in which they’ve shown any cause. Again, the article does note that, but not in so many words, and only in passing.

It’s entirely possible, for example, that the causation is exactly the other way around. It’s possible — I have no data to support this; it’s just hypothetical — that the people who were carrying guns were doing so because they often go places where they’re likely to be shot.

Third, they studied a city in the northeastern U.S., which has a certain view of guns. The results could be very different in, say, Dallas, where the gun culture is very different. Geographically diverse studies would be needed to account for regional differences in how we think about guns, and in the laws that regulate them.

None of this is to say that the work isn’t good, isn’t useful. It’s just that we can’t deduce anything directly from it. The value of studies like this is that they uncover apparent correlations and show us things that we can then go off and study more rigorously.

Unfortunately, it’s likely that many people will read reports of studies like this and won’t understand the limitations on interpreting the results. And in this case, trumpeting this study at the NRA’s gates would be a mistake, because its so easy to shoot it down (if you’ll excuse the metaphor).

On the other hand, I look forward to future studies that pursue the questions this one raises.