Week 7 — Sample size, power & precision

Scope & method. Rapid narrative review, not a systematic review. Structured search of the sport-science meta-research literature (PubMed, Journal of Sports Sciences, Sports Medicine, BMJ Open, SportRxiv), including several Sports Science Replication Centre (SSRC) papers. Findings reported as the cited studies state them; foundational statistical sources (neuroscience, psychology) are flagged as adjacent.

The principle under review Week 7 · adequate power, justified in advance

Week 7 argues that a small study gives a distorted answer, not merely a less precise one; that under-powered significant findings are systematically inflated (the winner's curse); and that sample size should be justified in advance against a smallest effect of interest. The empirical questions are how large studies in the field actually are, whether power is adequate, and whether sample sizes are justified.

1 · How well is it currently applied? the evidence

Studies are small and frequently under-powered

Direct field-specific evidence shows chronic under-powering. Mesquida, Murphy, Lakens and Warne (2023), applying a z-curve analysis to 89 studies in the Journal of Sports Sciences, estimated average observed power at 53% across all findings (61% for significant findings only) and found some indication of publication bias — well below the conventional 80% target, and lower still for the modest effects typical of the field. This sits on top of a median sample size of about 19 in the same journal (Abt et al., 2020). Editorial experience concurs: Impellizzeri, Murphy, Mesquida, Warne and colleagues (2025) note that most intervention studies submitted to their journal are under-powered, motivating a dedicated “Preliminary Report” submission category for small-sample studies.

The winner's curse is measurable in the field

Under-powering does not merely cause missed effects; it inflates the effects that do reach significance. In the field's replication project, Murphy and colleagues (2025) found that 88% of original effect sizes shrank on replication, with a median reduction of 75% — the pattern expected when significant results from small studies are inflated by chance (Type-M errors; Button et al., 2013; Gelman & Carlin, 2014). A literature built substantially on small, significant studies therefore over-states effect sizes systematically.

Sample sizes are rarely justified in advance

Pre-specification of sample size is uncommon. Mesquida and colleagues (2023) found the usage and reporting of a-priori power analyses to be suboptimal, with sample sizes often set by heuristics (“20 per condition”) rather than justified against a target effect. Schulz and colleagues (2022) similarly found that while 60% of sports-medicine trials reported a sample-size calculation, only 32% justified the expected effect size on which such a calculation depends. A further, subtler problem is that studies are frequently powered for a single outcome yet analysed across several: Gorman and Warne (2025) show that powering and testing across multiple dependent variables inflates the Type I error rate unless the multiplicity is accounted for.

Source	Finding	Implication
Mesquida, Murphy, Lakens & Warne (2023) J. Sports Sciences	Average power 53% (61% for significant findings); publication bias; suboptimal a-priori power use	Designs are under-powered and rarely justified
Abt et al. (2020) J. Sports Sciences	Median sample size of 19	Typical studies are small
Impellizzeri, Murphy, Mesquida, Warne et al. (2025) Sci. Med. Football	Most submitted intervention studies are under-powered	Under-powering is the norm, not the exception
Murphy et al. (2025) Sports Medicine	88% of effects shrank on replication (median −75%)	Winner's curse is measurable in the field
Schulz et al. (2022) BMJ Open	60% reported a sample-size calculation; only 32% justified the effect size	Justification is often absent even when a calculation is reported
Gorman & Warne (2025) J. Sports Sciences	Powering/testing across multiple dependent variables inflates Type I error	Single-outcome power under-states the true error rate
Button et al. (2013) Nat. Rev. Neurosci. (adjacent)	Low power inflates significant estimates and reduces reliability	Mechanism behind field-specific findings

2 · Is there cause for concern? objective assessment

The evidence is direct and field-specific: typical studies are small (median n ≈ 19), average power is around half rather than the conventional 80%, sample sizes are seldom justified against a target effect, and the predicted consequence — inflated significant effects — is measurable in the field's own replication data (88% of effects shrank on replication). Under these conditions, a significant finding from a typical study is likely to over-state the true effect, and a non-significant one is uninformative.

Two mitigating points apply. First, small samples are often genuinely unavoidable in elite sport; the failure is under-powering that is not acknowledged, not the small sample itself. Second, the field is developing concrete responses — sample-size justification frameworks, multi-site collaboration, and new journal formats such as the Preliminary Report for small-sample studies — which address the problem without pretending the constraint does not exist.

3 · Immediate steps to improve actionable

1. Run an a-priori power analysis against a smallest effect of interest, not against a heuristic sample size, and report it.
2. Justify the sample size honestly using one of the recognised approaches (a-priori power, resource-constrained plus sensitivity analysis, planning for precision, or an explicit statement of no justification).
3. Account for multiplicity when powering and analysing more than one dependent variable, rather than powering for a single outcome and testing many.
4. When the sample must be small, say so, report the power and a sensitivity analysis, and flag the winner's-curse risk on any significant finding rather than presenting it as definitive.
5. Pool across sites or reframe as a preliminary report when a single lab cannot reach adequate power, rather than publishing an under-powered study as conclusive.

References