• Note that this meta-analysis of a decade’s worth ofNew England Journal of Medicine articles revealed that many common medical practices are overturned when larger, better-designed trials are performed.
• Be aware that the study was limited by the use of a single journal.
• There is also a significant risk of publication bias, as studies that confirm old results may not appear in a high-impact journal such as the New England Journal of Medicine.

Practice-changing research may not be as common as studies suggest, as a review of literature spanning a decade found that many new practices should require additional testing.

A review of 146 studies showed many standard-replacing practices were neither effective nor necessary, wrote Vinay Prasad, MD, of the National Cancer Institute in Bethesda, Md., and colleagues online in Mayo Clinic Proceedings.

The review reaffirmed 138 current practices and found inconclusive results for 139 others.

“Medical reversals occur when new studies — better powered, controlled, or designed than their predecessors — contradict current practice,” they noted, adding that weeding through less effective practices is necessary to cut down on waste and minimize adverse effects on patient outcomes.

The authors had assessed standard clinical practices previously in 2011, through a review of 2,044 medical research studies published in the New England Journal of Medicine from 2001 to 2010, and found a similar practice reversal rate. In the earlier study — published in the Archives of Internal Medicine, Prasad et al found that one in eight studies published inNEJM each year had findings that should change practice.

In the current review, the researchers identified studies of diagnostic tests, therapies (medical and procedural), and health system changes and rated the findings as superior to current standard of care, not superior to standard of care, affirmation of current standard, or a reversal of current standard.

Additionally, some studies were labeled “inconclusive” if no firm conclusions about the practice could be reached.

Practices were also divided into four categories based on the study results:

• Replacements: When a new practice was found to be superior to the current standard of care
• Back to the Drawing Board: When a new practice failed to surpass old standards of practice
• Reaffirmations: When an existing practice was found to be superior to a prior practice
• Reversals: When an existing practice was found to be inferior to a prior practice, or inferior to not using the existing practice at all

In addition, some studies were deemed “inconclusive” if no firm conclusions about the practice could be reached.

Of the total 2,044 reviewed studies, 1,344 addressed a medical practice, of which 981 looked at new medical practices and 363 looked at existing practices. This included 756 studies in which new practices surpassed current standards, 165 studies in which new practices failed to improve current practices, and 146 studies which were considered “reversals” because they found current practices to be inferior to older practices. Another 138 study results were categorized as reaffirmations, and 139 were inconclusive.

Among the 146 studies with reversals, 76% were randomized controlled trials, 8.9% were nonrandomized studies, and 13.7% were retrospective.

“Articles that tested new practices were more likely to find them beneficial than articles that tested existing ones (77.1% versus 38%, P<0.001),” while those testing existing standards were more likely to find them ineffective versus those looking at new practices (40.2% versus 17%, P<0.001), the authors noted.

“We were unable to identify any class of medical practice that did not have some reversal of standard care,” they concluded, adding that “when medical practices are instituted in error, most often on the basis of premature, inadequate, biased, and conflicted evidence, the costs to society and the medical system are immense.”

An accompanying editorial by John Ioannidis, MD, of Stanford University School of Medicine, noted that “these seemingly disappointing results may be extremely helpful in curtailing harms to patients and cost to the healthcare system,” adding that such reviews promote and disseminate information about practices that should be stopped.

He also raised a number of questions for future research on new practices, such as whether new practices should require higher levels of evidence before they change existing guidelines, whether dissemination of faults found in new practices should be done with the same level of energy as when the new guidelines are established, and whether there are incentives or other ways to promote retesting of new practices that may supplant established practices.

The authors noted that their research was limited by use of a single journal, their choice of classification scheme versus other schemes, and the lack of conclusive outcomes in studies that reversed practices.

• Reviewed by F. Perry Wilson, MD, MSCE Instructor of Medicine, Perelman School of Medicine at the University of Pennsylvania and Dorothy Caputo, MA, BSN, RN, Nurse Planner

last updated 07.23.2013