The following comments are from Bill Hunter, MD, Medical Director of Open Door Community Health Centers, based in Arcata with clinic sites in many Northern California communities:
I was confused by your recent blog post on topical NSAIDs (http://phcprimarycare.org/?p=619). So I asked the ECHO team in Tucson to comment on it during our weekly ECHO sessions. (Editor’s Note: Project ECHO, based out of the University of New Mexico School of Medicine, works to improve care of chronic, common diseases in rural areas.) They were rather appalled. Some of their thoughts:
(1) The study referenced in your blog post was a study trying to assess a particular delivery system. It was not designed to assess the effectiveness of the medicine itself.
(2) The study evaluated only ketoprofen. The main topical recommended by the ECHO team, and the one for which there is the most data, is diclofenac.
(3) There is a Cochrane Review on efficacy of topical NSAIDs, and they found evidence to support the efficacy of topical diclofenac. The abstract can be found at: http://summaries.cochrane.org/CD007400/topical-non-steroidal-anti-inflammatory-drugs-for-chronic-musculoskeletal-pain-in-adults.
(4) The author of the post was claiming to be scientific, when it seemed he already had conclusions in mind and was looking for data to support them. He seemed dismissive of the possibility topical NSAIDs could work because he did not think it was likely. The whole premise of the scientific endeavor is to be curious and not dismissive.
I’m not saying I think Partnership should cover all manner of expensive things. There are lots of treatments for various conditions that would be unsustainable to pay for. And I know there are problems with some compounding pharmacies. But I think our Partnership patients should be able to get Voltaren (diclofenac) gel. If not, well then that is the way of the world.
Robert Moore, MD, responded as follows:
Thanks for taking the time to review the post, think through the implications, and run them by the faculty of the ECHO program. The formulary status remains unchanged – topical NSAIDs remain an option for patients who have failed at least two oral NSAIDs. Blog postings do not reflect changes in our formulary. Our formulary changes are reviewed in a many-step process that takes at least 6 months, and ultimately goes through the Physicians Advisory Committee.
The issue of reviewing a new study in the context of the wider medical literature is important and your points are well taken. It would have been good to reference and discuss the Cochrane Review in this post. A more detailed look at the evidence for deep penetration of transdermal delivery of drugs would also have been good.
Sometimes early studies that lead to drug approval have significant biases, which only become evident with further study. Most recently, this came to light in the lack of benefit of injectable hyaluronic acid for osteoarthritis, compared to placebo. While the lack of efficacy of viscosupplementation is now accepted by the American Academy of Orthopedic Surgeons, it is only slowly being accepted by practicing orthopedists: http://www.aaos.org/news/aaosnow/jun13/cover1.asp. A close review of the evidence supporting injectable hyaluronic acid showed that much of the benefit accrued from the powerful placebo effect of the injection itself. This, combined with a small bias in early studies, created an early evidence base for benefit. When larger, better controlled studies were done later, it became clear that viscosupplementation had little benefit (but not until over a billion dollars had been spent in the U.S. on what turned out to be ineffective treatment). Even before the lack of efficacy was shown in larger studies, though, a review of the magnitude of the placebo effect should have given prescribers pause. But no publicity was given to this issue.
We have many medications on the PHC formulary that are very expensive, more expensive than topical NSAIDs. As a medical director staff, we look at the evolving science as a driver of change. We are stewards of taxpayer resources, so expensive drugs should not be used if there is no evidence of benefit. We opt for approving treatment if there is good scientific evidence of benefit, even if that benefit is small.
Looking deeper at the number of patients who benefit vs. placebo in the Cochrane Review is enlightening.
In studies lasting longer than 8 weeks (a common scenario for use of these medications), 50% of the patients in the placebo group showed improvement compared to 60% in the study group. Assuming the studies used for the Cochrane analysis are without any positive bias, this means 5 out of 6 people who had improvements on topical NSAIDs likely had the improvement due to the placebo effect. When studies are published, they often do not present their data this way; in many studies, determining the placebo effect requires the reader to do the math on the Relative Benefit and the Number Needed to Treat. If the studies used to show the Relative Benefit of topical NSAIDs of 1.2 have even a little bias in favor of a positive result, there may actually be no benefit for topical NSAIDs over placebo. And even though studies on a number of topical NSAIDs were reviewed, only diclofenac showed a benefit.
This slight benefit is evidence enough to allow non-formulary topical diclofenac to be approvable if two oral NSAIDs have been tried and failed. But it is worth bringing this to the attention of prescribers, who usually only hear one side of the story from pharmaceutical marketing. As prescribers, we need to know that if a patient says topical NSAIDs are working for longer than 8 weeks, there is at least a 5/6 (or 83%) chance that the benefit is due to the placebo effect. We clinicians tend to assume the benefit of a drug to the patient in front of us is due to the effectiveness of the drug.
As we medical directors look at the medical literature as new treatments are requested, we are struck by the frequent poor quality of studies. This may create an underlying bias that we need to be careful of. When we review a new study showing that prior studies were wrong, we need to look at the conclusion critically and take the whole literature in context. This is important, even when there is prevailing bias in the literature toward presenting positive results.
For a good review of the topic of bias in publication in science in general, see The Economist: http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble. It is disturbing and can rock one’s confidence in our system of establishing scientific truth.
It was not our intent to generate controversy, but a vigorous debate and detailed look at the evidence can only help PCPs prescribe more mindfully and help PHC formulate policy based on an informed review of the evolving scientific literature.