Suppose you are planning to conduct a repeated-measures study, where outcomes are measured from the same subject at more than one point in time and the average within-subject effect is calculated using a paired t-test or linear regression. How might you calculate how many subjects need to be tested in order to find an effect?

Similar to calculating sample size for a 2 independent sample t-test, we can use Python’s statsmodels module to calculate sample size for a paired t-test. To find a given paired standardised mean difference of 1, we can calculate the following (output shown using >>> prompt, and documentation available here):

from statsmodels.stats.power import tt_solve_power

mean_diff, sd_diff = 0.5, 0.5
std_effect_size = mean_diff / sd_diff

n = tt_solve_power(effect_size=std_effect_size, alpha=0.05, power=0.8, alternative='two-sided')
print('Number in the group: {:.5f}'.format(n))

>>> Number in the group: 9.93785

Note, we are using the same standardised effect size (0.5/0.5 = 1), standard deviation of the paired difference (0.5), alpha (0.05) and power (0.8) that we used previously for a t-test on 2 independent samples, but the sample size is much smaller for a paired t-test (N=9.9) compared to a t-test on 2 independent samples (N=16.7). This is because for a paired t-test, each subject’s outcome is correlated with their previous outcome (ie. each subject acts as their own control) so the test becomes more powerful to detect an effect.

But remember, a paired or independent standardised effect size of 1 is a big effect, even for most fields of research. Standardised effect sizes vary substantially depending on the field of research, so it is important to know how big an effect needs to be detected for a given study, since much published research in neuroscience and clinical trials is already underpowered.

For a paired t-test, how does sample size change as effect size decreases?

for mean_diff in [0.5, 0.4, 0.3]:
    n = tt_solve_power(effect_size=mean_diff/sd_diff, alpha=0.05, power=0.8, alternative='two-sided')
    print('Number in the group when paired standardised mean difference is {:<4.2f}: {:.2f}'.format(mean_diff, n))

>>> Number in the group when paired standardised mean difference is 0.50: 9.94
>>> Number in the group when paired standardised mean difference is 0.40: 14.30
>>> Number in the group when paired standardised mean difference is 0.30: 23.79

For a given standard deviation of the paired difference, sample size increases as the standardised effect size decreases.

Summary

We used Python’s statsmodels module to calculate sample size for paired t-test. Sample size is sensitive to the size and variability of the paired difference, and tolerance to Type I and II errors.