Abstract
PURPOSE - Low statistical power and publication bias in sports and exercise science (SES) indicate potential replication issues, similar to psychology, but has yet to be examined. Therefore, our aim is to investigate the replicability of our field. We hypothesize that replication effect size estimates (ES) will be smaller than original ES. METHODS - Replication studies are selected as per a formalised protocol. Four studies are complete to date. Study 1 determined the effect of caffeine gum compared to a placebo on countermovement jump height (CMJ). Study 2 determined the effect of maximal fatigue on dynamic balance. Study 3 determined the effect of mental fatigue on CMJ. Study 4 determined the effect of aging on dynamic balance. Original and replication ES were compared using a z-test (α = 0.05). RESULTS - Replication 1 = A paired t-test showed a significant difference in CMJ height following the consumption of caffeine gum versus placebo chewing gum in the original study (p < 0.001; Cohen’s dz = 1.51) but not in the replication (p = 0.064, Cohen’s dz = 0.31). Replication and original ES were significantly different (z = 3.20, p = 0.002). Replication 2 = A 1 way repeated measures ANOVA showed a significant main effect of fatigue on dynamic balance for both the original (p = 0.025, partial eta squared (ηp2) = 0.376) and replication study (p = 0.005, ηp2 = 0.148). Replication and original ES were not significantly different (z = 1.73, p = 0.08). Replication 3 = A 2x2 ANOVA observed a significant main effect of mental fatigue on CMJ height versus control in the original study (p = 0.028, ηp2 = 0.344) but not in the replication study (p = 0.294, ηp2 = 0.038). Replication and original ES were significantly different (z = 2.49, p = 0.01). Replication 4 = An independent t-test showed a significant difference in dynamic balance in younger individuals compared to older individuals in the original study (p = 0.011; Cohen’s ds = 1.05) but not in the replication study (p = 0.67, Cohen’s ds = 0.18). Replication and original ES were significantly different (z = 2.02, p = 0.044). CONCLUSION - Of the 4 replication studies, 3 were not significant. Replication ES were significantly smaller than the original ES in 3 studies. Thus, some published SES research may not be replicable, potentially due to ES inflation and false positives.
Jennifer Murphy
PhD Researcher
My research interests include replication, open science and statistics.
Aaron Caldwell
PhD Supervisor
