Clinical Practice Guidelines Require Scrutiny for Quality

September 26, 2013

Clinical Practice Guidelines Require Scrutiny for Quality

  1. Allan S. Brett, MD

Two studies revealed problems with endocrine and oncology guidelines.

  1. Allan S. Brett, MD

In two critical evaluations, investigators have assessed the reliability of clinical practice guidelines.

The first study concerned guidelines issued by the Endocrine Society, which uses the GRADE system (each recommendation is rated as strong or weak, and the quality of evidence supporting each recommendation is rated as high, moderate, low, or very low). Among 357 recommendations in 17 guidelines issued between 2005 and 2011, 121 (34%) combined a strong recommendation with low-quality evidence. Such guidelines require scrutiny because they strongly advocate a particular practice despite relatively weak supporting evidence. Using an explicit process, the authors found 33 instances in which no compelling justification for a strong-recommendation/low-evidence guideline existed.

In a second study, researchers reviewed 169 guidelines on prostate, lung, breast, and colorectal cancer published between 2005 and 2010. To determine whether guidelines were trustworthy, each was scored according to 8 standards published by the Institute of Medicine. On average, guidelines fulfilled only 2.75 of the 8 standards.


The proliferation of practice guidelines by professional societies, advocacy organizations, and government agencies presents considerable challenges: They often conflict with each other, and as these two studies show, many are flawed. The analysis of Endocrine Society guidelines reminds me of one of the society's 2012 guidelines that I found problematic: a strong recommendation — despite low-quality supporting evidence — for glucose targets <140 mg/dL (premeal) and <180 mg/dL (random) for hospitalized patients with noncritical illness (J Clin Endocrinol Metab 2012; 97:16).

Editor Disclosures at Time of Publication

  • Disclosures for Allan S. Brett, MD at time of publication Nothing to disclose


Reader Comments (4)

Rutenberg Physician, Pediatric Subspecialty, Pediatrics

Write a guideline, even if it's based on opinion, not fact, or real evidence, use it to deny care or select the cheaper treatment and save money, we all know, you cost the system more in your last 6 months than at any other time, so by using non evidence guidelines, your last 6 months is reduced to 3 months, my gosh, think of the savings, I follow the Buffet plan, treat me if you can. Warren Buffet has the resources to get the very best medical advice. Well, he's being treated for his prostate cancer, interesting, the Oracle of Omaha invests conservatively for the long term in his portfolio, but aggressively and for an immediate result for his health.
If Obama cared, he'd tell us the truth, there is opinion not fact guiding medical decisions, and if the government or the healthcare companies write the rules and regulations there are only 2 outcomes, they profit, we and our patients suffer, or if we want to contribute to solving the deficit, go die "and death shall have dominion."

SAMUEL PIERCE Physician, Internal Medicine, clinic

With regard to the Endocrine Society then, either the physicians making recommendations are incompetent, corrupt, or have a different standard of what " quality" is. I would hope and strongly suspect that the latter is the case here. Is there such a thing as a gold standard of quality ? Would it be possible to derive it from the explicit process mentioned above? For the sake of public safety, have the authors of the study, or anyone else attempted a dialog to revise the practice guidelines and prevent future errors?

Graeme French, M.D. Physician, Orthopedics, Colfax, WA

I find that there is rarely high-quality evidence applied for the development of any specific guideline. I think that secondary financial gain is almost always behind a strong recommendation has been made in the absence of strong evidence. This includes the financial increased to place specific subspecialty group and frequently represents the interest of an insurance company or other major payor. I think that it is the height of unethical professional behavior for a physician to participate in a process that results in guidelines of this quality. Frequently, the process developed to establish these guidelines are severely flawed. Claw's include having committees where limited numbers of the participants are actually qualified and had experience in treating the problem for which the guideline is being developed. The other major flaw with many of these guidelines is that the breadth of scientific evidence available is not screened or the selection process for inclusion is still biased as to excludes all data that that disagrees with the conclusion that the organization appears to be interested in reaching.7

RONALD HIRSCH Physician, Internal Medicine

The problem is that the experts in the field are also the ones being paid big money by pharma to do the clinical trials and speak around the country on their medications. The docs may deny it, but there will always be subconscious bias when the entity paying you a lot of money has a stake in a guideline review they are performing.

Your Comment

(will not be published)

Filtered HTML

  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Do you have any conflict of interest to disclose?
This question is for testing whether you are a human visitor and to prevent automated spam submissions.

Vertical Tabs

* Required

Reader comments are intended to encourage lively discussion of clinical topics with your peers in the medical community. We ask that you keep your remarks to a reasonable length, and we reserve the right to withhold publication of remarks that do not meet this standard.

PRIVACY: We will not use your email address, submitted for a comment, for any other purpose nor sell, rent, or share your e-mail address with any third parties. Please see our Privacy Policy.