Volume 8, Issue 3 , Pages 116-118, September 2008
Grading the Evidence: The Core of EBD
Article Outline
Three different types of systematic experiments are powerful tools to evaluate whether the low-level evidence on cause and effect indeed translates into tangible outcomes in humans. The evidence pyramid forms the core of evidence-based dentistry that allows grading the evidence.
Key Words: Evidence-Based dentistry, evidence pyramid, grade
“I think two and two is four, not three or five. I'm an evidence-based guy. We are trapped in a reality-based world.” Bill Clinton, Financial Times, Monday, September 24, 2007
We all live in an evidence-based world where 2 + 2 should equal 4, not 5 or 3. Unfortunately, judging the soundness of evidence in medicine or dentistry is not as straightforward as a simple addition. On many clinical questions, available evidence is arguably incomplete, conflicting, or nebulous, making it difficult to arrive at the yes/no decision required in daily clinical practice. In choosing a restorative material, how should one judge the common claim that gold is the “standard” restoration because gold has high corrosion resistance or malleability, versus the clinical trial evidence that the failure rate of gold cast restoration is 10%?1 In 1992, a landmark article proposed an evidence-based approach to medicine that addressed such challenges.2 One feature of an evidence-based approach is that formal rules have been developed in evaluating the reliability of clinical evidence.
These formal rules for grading evidence are often the result of prior disasters and the laws, regulations, and guidelines that followed the disasters. A benign sulfa drug dissolved in toxic diethylene glycol led to more than a hundred deaths—mostly children—and precipitated the Federal Food, Drug, and Cosmetic Act of 1938. Drug manufacturers were for the first time mandated to provide evidence of drug safety before marketing. The thalidomide crisis led to the 1962 Kefauver-Harris Amendments, which have been credited with the increased utilization of randomized controlled trials to provide evidence on effectiveness. Deaths associated with a device—the Dalkon Shield3—precipitated the Medical Device Amendment of 1976, which requires evidence that high-risk devices (eg, angioplasty catheters) are safe and effective. Possibly the controversies surrounding approved drugs such as encainade, rofecoxib, or rosiglitazone may lead to further changes in the methods and principles of evaluating clinical evidence.4 Such cycles of disasters and responses led to the recognition of a hierarchy of evidence upon which to judge drugs and devices. Evidence-based medicine came along and formalized rules in assessing the clinical literature not only on drugs and devices, but also clinical procedures. The advent of evidence pyramids in particular has helped in grading and translating evidence into a clinical recommendation.
In the evidence pyramid, the lowest levels of evidence are expert opinion, biological plausibility, bench research, animal studies, and case-series (Figure 1). Such findings are relegated to the lowest tier of evidence not because such research or thinking is methodologically unsound or because the evidence derived is necessarily unreliable. Quite the opposite is true. Animal research has been reported to form the basis for two thirds of the Nobel prizes in Physiology or Medicine.5 Bench research has led to discoveries such as the structure of DNA, the discovery of antibiotics, or the synthesis of insulin. Case-series, with implicit historical controls, led to the identification of bisphosphonates as a cause for osteonecrosis of the jaw or to a Nobel Prize in Physiology or Medicine for discovering bone marrow transplantation as a treatment for leukemia. Biological plausibility is at the basis of treatments such as root canals for the treatment of acute pulpitis, which are obviously effective. The adjective “low-level” does not refer to the intrinsic (or inherent) quality or value of the evidence, but rather as to how such evidence is valued when it is used as a basis for making clinical decisions for humans.
Figure 1.
The evidence pyramid ranks the reliability of evidence from low (bottom tier of pyramid) to high (RCT = randomized controlled trial). The 3 highest level of evidence consist of systematic experiments and became routine in the second half of the 20th century. With an inverted evidence pyramid, the reliability of evidence is ranked in a reverse order and biological plausibility is considered most reliable and RCTs are “used” to prove biological plausibility.
Expert opinion and low-level evidence can lead to disappointments and disasters. A New York gynecologist surmised that menopause was a disease and that estrogen replacement represented the miracle cure. Randomized controlled trials suggested that this expert opinion may have contributed to tens of thousands of unnecessary deaths. Biological plausibility appeared to strongly support the use of antioxidants to prevent cancer, yet all emerging evidence from randomized controlled trials suggest an increased mortality risk, not a decreased mortality risk.6 In dental practice, many issues related to diagnosis, etiology, and treatment remain based on low-level evidence. Only higher-level evidence studies will allow determining how many of these treatments, which are based on low-level evidence justification, will stand up to experimentation. For instance, one clinician suggested that teeth with periodontal involvement, when surrounded by teeth without periodontal involvement, should be extracted when an arch of teeth is being restored.7 If a randomized controlled trial on this topic were to be conducted, would it be refute or support such a recommendation? As long as no trials on such topics are conducted, we do not have a reliable answer.
The term “low-level” when characterizing evidence is used because leaps of faith are required to assume that knowledge on biological mechanisms, or results obtained from animal or bench experiments, or from clinical experience translate into clinical decisions that lead to a tangible patient benefit. Medical history has shown that such leaps of faith often have an unacceptably high chance of leading to harmful clinical decisions (think “disasters”). For instance, a vaginal cream effective at preventing HIV transmission in macaques unexpectedly increased HIV infection rates in humans.8, 9 Benefits or harms in animals may not translate to humans, or vice-versa. One physician-epidemiologist—Feinstein—has suggested that one of the historical hallmarks of the 20th century is the demise of low-level evidence and the advent of controlled human observations.
High-level evidence consists of controlled systematic experiments in humans: primarily case-control studies, cohort studies, and randomized controlled trials. Instead of using deductive reasoning to connect a cause to its effect in humans, observations are made on a sample of subjects and are generalized using inductive inference. In a case-control study, individuals with and without a disease or a condition are compared with respect to the prevalence of a suspected etiological factor. For instance, individuals with and without brain cancers can be compared with respect to the prevalence of past medical and dental diagnostic x-ray exposures. In a cohort study, exposed and nonexposed individuals are followed longitudinally and the incidence of the outcome of interest is monitored. For instance, individuals exposed and not exposed to water fluoridation can be followed longitudinally and monitored for the incidence of adverse events. Both case-control studies and cohort studies are commonly used to elucidate causes of disease.
Health recommendations that are based on such epidemiological study designs (ie, case-control and cohort studies) can also be disappointing. Both hormone replacement therapy and vitamin A provide 2 examples of treatments that were suggested to save lives, based on epidemiological studies, yet randomized controlled trials proved them to be harmful. A challenge with epidemiological studies is that both selecting individuals into studies in an unbiased fashion and controlling for the lifestyles and environment of these individuals is to a certain extent impossible and will always lead to small biases. While epidemiology can reliably identify large effects such as the impact of smoking on lung cancer, or sugar on caries, small effects are difficult to separate from small biases.
Unequivocal control for such small biases can best be achieved by a study design referred to as a randomized controlled trial, the top of the evidence-pyramid. In a randomized controlled trial, individuals are randomly assigned to exposures (most commonly treatments) and the incidence of the outcome of interest is monitored. For instance, individuals can be randomly assigned to either xylitol gum or sorbitol gum and the incidence of caries can be monitored. Many of the inherent and often uncontrollable biases present in case-control studies and cohort studies can be eliminated by the powerful, but delicate, mechanism of randomly assigning individuals to treatments. Certain evidence-based organizations, such as Cochrane, focus exclusively on randomized trial evidence when reporting on treatment effectiveness.
These 3 different types of systematic experiments are powerful tools to evaluate whether the low-level evidence on cause and effect indeed translates into tangible outcomes in humans. The evidence pyramid forms the core of evidence-based dentistry that allows grading the evidence.
References
- . Ceramic inlays for restoring posterior teeth. Cochrane Database Syst Rev. 2003;(1):CD003450
- Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420–2425
- . Another look at the Dalkon Shield: meta-analysis underscores its problems. Contraception. 1993;48(1):1–12
- Surrogate end points, health outcomes, and the drug-approval process for the treatment of risk factors for cardiovascular disease. JAMA. 1999;282(8):786–790
- AMP News Service. Nobel Award in Medicine Holds Strong Message for Animal Activists. Available at: http://www.amprogress.org/site/apps/nl/content2.asp?c=jrLUK0PDLoF&b=1311499&ct=4512083. Accessed March 10, 2008.
- . Mortality in randomized trials of antioxidant supplements for primary and secondary prevention: systematic review and meta-analysis. JAMA. 2007;297(8):842–857
- . When it is best to remove a tooth. J Am Dent Assoc. 1997;128(5):635–636
- Effectiveness of COL-1492, a nonoxynol-9 vaginal gel, on HIV-1 transmission in female sex workers: a randomised controlled trial. Lancet. 2002;360(9338):971–977
- Note: Sections of this text originate from a chapter in the textbook, Statistical and Methodological Aspects of Oral Health Research.
PII: S1532-3382(08)00114-0
doi:10.1016/j.jebdp.2008.05.010
© 2008 Elsevier Inc. All rights reserved.
Volume 8, Issue 3 , Pages 116-118, September 2008
