Week 11 — Cautious & responsible claims

Scope & method. Rapid narrative review, not a systematic review. Structured search of the sport-science and health-communication literature (PubMed, Sports Medicine, BMJ, American Journal of Epidemiology, Significance). Findings reported as the cited studies state them; health-communication and epidemiology sources are flagged as adjacent.

The principle under review Week 11 · calibrating the claim to the evidence

Week 11 argues that a claim should be sized to its evidence on four dimensions: practical versus statistical significance; external validity (bounding the claim to the sample and conditions); provisionality (a single study's effect is likely inflated); and causal language matched to design (associational verbs for observational data). The empirical questions are whether the field over-states significance as importance, generalises beyond its samples, treats single findings as definitive, and attaches causal language to non-causal evidence.

1 · How well is it currently applied? the evidence

Causal language routinely outruns design

The most common calibration failure is attaching causal claims to non-causal evidence. In a systematic evaluation of observational health research, Haber and colleagues (2022) found action recommendations implying causality in about 44.5% of articles — frequently stronger than the results warranted. The stakes are shown by Young and Karr (2011), who assembled randomised trials testing 52 claims that had been suggested by observational studies: essentially none were confirmed in the direction of the original observational claim. Observational associations, however rigorously obtained, are an unreliable basis for the causal, intervention-licensing claims that are common in applied sport and exercise science.

Overstatement is amplified in communication

Claims tend to inflate as they move outward. Sumner and colleagues (2014) found that 33% of academic press releases contained exaggerated causal claims and 40% exaggerated advice, and that these exaggerations predicted corresponding exaggerations in the resulting news — locating the origin of hype upstream, in the research communication itself. Within the field specifically, Tiller, Sullivan and Ekkekakis (2023) issued a call to action on baseless and pseudoscientific claims in the sports, exercise and nutrition-science space, noting that extraordinary health and performance claims routinely outpace the ordinary evidence behind them.

Significance is over-read, and single findings over-trusted

Two further calibration failures are documented. Borg and colleagues (2023) show that an exclusive focus on statistical significance exaggerates the apparent magnitude of effects — the practical-versus-statistical confusion. And the provisionality problem is quantified directly by the field's replication data: Murphy and colleagues (2025) found only 28% of findings robustly replicated and effect sizes shrank by a median of 75%, so any single fresh finding — including one's own — should be treated as provisional and probably over-stated rather than definitive (Weeks 1, 7).

Source	Finding	Implication
Haber et al. (2022) Am. J. Epidemiology (adjacent)	~44.5% of observational studies made causal action recommendations	Causal language outruns design
Young & Karr (2011) Significance (adjacent)	~0 of 52 observational claims confirmed in RCTs	Observational associations are a weak basis for causal claims
Sumner et al. (2014) BMJ (adjacent)	33% of press releases had exaggerated causal claims; hype originates upstream	Overstatement is amplified in communication
Tiller, Sullivan & Ekkekakis (2023) Sports Medicine	Baseless/exaggerated claims pervade sports, exercise and nutrition science	The field recognises and is challenging over-claiming
Borg et al. (2023) J. Sci. Med. Sport	Exclusive focus on significance exaggerates effect magnitude	Statistical significance over-read as importance
Murphy et al. (2025) Sports Medicine	28% robust replication; effects shrank a median of 75%	Single findings should be treated as provisional

2 · Is there cause for concern? objective assessment

All four calibration failures are documented: causal language is attached to observational data in nearly half of studies; overstatement is amplified in press releases and media; statistical significance is read as practical importance; and single findings are treated as more definitive than the field's own 28% replication rate can justify. Because these claims are what practitioners, coaches and the public act on, mis-calibration here converts upstream methodological problems into real-world decisions based on effects that are smaller, more conditional, or less causal than stated.

The concern is partly offset by a growing culture of correction. The field is now explicitly naming and challenging exaggerated and pseudoscientific claims (Tiller et al., 2023), tools exist for auditing causal language (Haber et al., 2022), and the replication evidence provides a concrete basis for provisional language. Calibrated claiming requires no new resources — only discipline in matching the claim to the design, the effect size, and the weight of evidence.

3 · Immediate steps to improve actionable

1. Match causal language to design: use “associated with” or “predicted” for observational data, and reserve “improves” or “causes” for randomised designs.
2. Claim practical importance only when the effect — and the near end of its confidence interval — clears a smallest effect of interest, not merely because a result is statistically significant.
3. Bound the claim to the sample and conditions studied; treat any extension to other populations or settings as a hypothesis, not a result.
4. Use provisional language for single studies (“preliminary evidence that X may…; needs replication”), given that effects typically shrink on replication.
5. Calibrate the abstract, conclusion and any press release to the primary result, since overstatement originates in the research communication itself.

References