1. Does statistically significant always mean clinically significant?
No, statistical significance absolutely does not always equate to clinical significance.
Statistical Significance merely indicates that the observed difference or effect in the study sample is unlikely to be due to chance, assuming the null hypothesis is true (typically indicated by a p-value < 0.05) (Norman & Streiner, 2014). It is mathematically driven and heavily dependent on the sample size.
Clinical Significance refers to the practical importance of a finding. It indicates whether an intervention makes a genuine, palpable difference in patient care, treatment efficiency, or functional/esthetic outcomes—often referred to as the Minimal Clinically Important Difference (MCID) (Pandis et al., 2010).
Orthodontic Example: A study might find that a new aligner material corrects crowding 0.15 millimeters faster than a traditional material. If the study evaluates 5,000 patients, this 0.15 mm difference will likely be highly statistically significant (p < 0.001). However, 0.15 mm is imperceptible to both the orthodontist and the patient, rendering it completely clinically insignificant (Proffit et al., 2018).
2. What are the sensitivity and specificity of statistical significance when being used for clinical significance?
If we evaluate "Statistical Significance" (SS) as if it were a diagnostic test for detecting true "Clinical Significance" (CS), the diagnostic performance metrics are skewed:
Sensitivity (True Positive Rate) is High: If an orthodontic treatment truly has a massive, clinically significant effect (e.g., functional appliances reducing overjet by 6 mm), the statistical test will easily detect it. Thus, SS is highly sensitive to true clinical relevance, provided the study is adequately powered (Altman, 1991).
Specificity (True Negative Rate) is Low: Specificity asks: If an effect is NOT clinically significant, will the statistical test correctly flag it as non-significant? In modern research with large sample sizes, the test fails at this. Large studies will frequently detect tiny, meaningless differences and label them as statistically significant. Therefore, relying purely on p-values produces many "false positives" for clinical relevance (Button et al., 2013).
3. Is it true that when something has been found to be statistically significant, it must be clinically significant too?
No, this is one of the most common epidemiological fallacies in dental research. As explained above, statistical significance only proves that a difference exists, not that the difference matters. To establish clinical significance, an orthodontist must look past the p-value and examine the effect size (magnitude of the change) and the confidence intervals to determine if the treatment alters clinical protocols in the real world (Johnston, 2002).
References:
Altman DG (1991). Practical Statistics for Medical Research. Chapman and Hall/CRC.
Button KS, et al. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience.
Johnston LE (2002). Clinical studies in orthodontics: art, science, or nonsense? American Journal of Orthodontics and Dentofacial Orthopedics.
Norman GR, Streiner DL (2014). Biostatistics: The Bare Essentials. PMPH-USA.
Pandis N, Polychronopoulou A, Eliades T (2010). Failure to establish a clinically significant difference... American Journal of Orthodontics and Dentofacial Orthopedics.
Proffit WR, Fields HW, Larson BE, Sarver DM (2018). Contemporary Orthodontics, 6th Edition. Elsevier.
