Cross-checks on ethics

We’ve another instance in a series of discredited scientific studies, this latest one in the medical field. The short version here is that Dr Timothy Kuklo, who worked for the U.S. Army, wrote a paper about a bone-growth product called “Infuse”, claiming that it was very effective for treating the traumatic bone injuries that many soldiers have been coming home with. He had some co-authors on the paper, including Dr Romney Andersen. Except the co-authors had nothing to do with the paper, and didn’t know it existed until Dr Andersen heard about it and started an investigation.

There’s little new here with respect to the paper itself, but what I want to talk about is some of the criticism being given to the peer-review and publication process that allowed this to happen. In particular, the New York Times article, saying that this “shows how medical journals may fail to conduct adequate due diligence on the studies they publish”, implies, thought doesn’t say outright, that the journal should have:

1. contacted all the paper’s co-authors,
2. checked the paper’s data, and
3. looked into the financial backing of the study.

In her comments about the incident, Janet Stemwedel, of the blog Adventures in Ethics and Science, supports the first point thus:

Dr. Andersen was never contacted by the British Journal of Bone and Joint Surgery that published the paper on which he was listed as a coauthor — he didn’t know about the paper (and his supposed role in it) until it had already been published. How hard would it have been for the journal editors to email all the named authors — to let them know the manuscript had been received, to transmit copies of the referee reports, to acknowledge receipt of the revised manuscript, etc.?

Indeed, that would be nice. As someone involved with conferences, journals, and magazines, I can say that some do routinely send correspondence to all authors, while others communicate only with a designated “corresponding author”, usually the one who put the submission into the tracking system. I can also say that some authors are pleased to get copies of the communication, and some decidedly are not, preferring to avoid extra routine email. But we could say that the benefit outweighs the small annoyance it gives some people.

Except, it doesn’t. First, it would require reprogramming some of the tracking systems even to permit it. Some of the systems only capture contact information for one author. Unless the email addresses of the other authors are in the text of the paper, those addresses aren’t available... and even if they are, it’s a manual process to get them.

But second, the information is all put into the system by the author who submitted it. If Dr Kuklo would put Dr Andersen’s name on the paper, would sign Dr Andersen’s name to the copyright release, why on Earth wouldn’t we imagine that he’d also supply bogus email addresses for Dr Andersen and the other alleged authors? He could simply sign up for some free ones that he checks himself, and no one at the Journal would be the wiser.

While it might be nice to check in with all the authors, doing so won’t really provide assurance of anything.

Now, there is another issue in this case, in that one of the journals (one that actually rejected the paper) refused to give co-author Andersen any details about the paper or its reviews, saying that they communicate that confidentially only with the corresponding author. That’s just bizarre, and that journal needs to change its policy.

On the question of checking the paper’s data, well, that’s what the peer-review process is for, and there’s no reason to think that any “due diligence” was lacking there. It’s simply not possible to independently repeat a study and replicate results before publishing a paper. When new data are developed, all we can do in the short term is to review the paper — review the methods, the controls, the data collection, and so on — and make a judgment about whether it meets reasonable standards. Perhaps reviewers might ask to see raw data, if the paper only has graphs or summaries. Maybe the reviewers would demand more information about the study’s methods before accepting the paper for publication.

But in the end, no one will know whether the data are reliable until other scientists do other experiments that either back up or refute the results. The journals simply can’t be held responsible for that, beyond the normal process of peer review.

To the point about the financial backing, it’s on the fringe of ethics, and what I consider an ethics violation, not to disclose relevant backing for your work. A simple footnote, such as, “Part of this work was done with funding from Medtronic,” would suffice, and one sees that all the time when the authors are following reasonable ethical guidelines.

But, again, the disclosure must be made by the authors, and it is they (in this case, Dr Kuklo alone) who are responsible for failure to disclose. It is not up to the journal (or the peer reviewers) to investigate this, and it cannot be so. They haven’t the time, nor the personnel, nor the other resources necessary for that.

The fact is that for the most part, the integrity of peer-reviewed scientific publication is dependent upon the integrity of the authors. And authors determined to publish at any ethical and scientific cost can and will do so. The best we can do is to expose them afterward, and to foster an environment where ethical behaviour is its own reward.