Above is an evidence-based practice pyramid (Creative Commons licensed) from the University of Washington Health Sciences Libraries page about evidence-based practice resources.

For those not familiar with the EBP pyramid, it is a tool used to teach about research articles in medicine with the most relevant to clinical care towards the top (Cochrane Library and then other systematic reviews of the literature) and least clinically relevant towards the bottom (clinical reference texts, case studies etc). I learned about this approach to research when I was studying health informatics in school and didn’t really give it a second thought. This Is The Way It Is and makes logical sense.

Lately I have really been rethinking that gold Cochrane capstone after the Pacific Northwest Chapter of the Medical Library Association (PNC/MLA) meeting in Portland last month.

Dr. Erick Turner’s presentation during the meeting of Selective Publication and Drug Efficacy: Don’t Believe Everything You (Don’t) Read (My Notes, His Slides) is a must-read for medical librarians and others concerned about pharmaceutical research and evidence-b(i!)ased medicine. The presentation gives much more background (charts in the slides) than what was published in the book Unhinged with the information about Turner and his work. The review documents for eight of the twelve antidepressants discussed are housed online in the Oregon Health & Science University Digital Resources Library.

At the end of his presentation one of the first questions asked was about the significance of this information for systematic reviews. He replied along the lines of (from my notes)

We use Cochrane as a search method, people genuflect at that name, but all the unpublished trials were missed. Starting point is published literature, FDA reviews are grey literature. Instead, he goes to the grey literature first to identify a cohort of pre-marketing trials then goes to published results to find them which misses post-marketing trials but questions spun factor.

Definitely read my notes or the presentation slides to get the full context of the spin factor since it’s pretty extreme.

Another Medical Librarian Perspective Along the Same Lines

Why Health Sciences Librarians Should Challenge the Flaws of the Peer Review System by Marcus Banks.

We seem to be on a similar wavelength questioning that which is currently seen as sacrosanct:

Yet, despite the recent provenance and pedestrian function of “peer review,” those two words have become holy totems among researchers and librarians.  In our classes we make sure to point students to the checkbox in various databases that limits to peer reviewed journals.  We extol peer reviewed research and excoriate Wikipedia.  And so on and so forth, all in the belief that “peer review” leads people to wonderfully validated, scientifically rigorous, and profoundly accurate research.

Oh, how it doesn’t. But we’re librarians so what can we do about it? Plenty as he suggests:

If we assume that the peer review system is structurally flawed, what should librarians do about it? Here are some options, arranged from least to most attractive:

• Worst: Continue to glorify the peer review system
• Better but still bad: Stop the glorification but go no further
• Better and now good: Vocally discuss the challenges of peer review system on campus
• Best: Formally challenge funding models/priorities at NIH and elsewhere that sustain peer review flaws.  If funding streams shift behavior change will follow

Many people would say that the best option isn’t for librarians–for policy types and researchers, maybe, but not for us.  We are here to serve, not to challenge.  I disagree, respectfully and yet strongly.  MLA’s Code of Ethics for Health Sciences Librarianship states that one of our roles is to facilitate “informed health care decisions.”  How can we ethically do this if we know that peer review has systemic flaws?

Indeed.