Guide to Power Calculations

Learn how to perform power calculations for randomized controlled trials using Stata’s power command. This guide covers three essential scenarios: calculating required sample size, determining minimum detectable effect, and estimating statistical power for given parameters.

Learning Objectives

Power calculations are essential for designing effective randomized controlled trials (RCTs) and observational studies. They help researchers determine the sample size needed to detect a statistically significant effect, ensuring that studies are adequately powered to avoid type II errors (false negatives). This guide will help you:

  • Understand the concept of statistical power and its importance in research design
  • Learn how to perform power calculations using Stata’s power command
  • Apply power calculations to real-world scenarios, such as estimating required sample size, minimum detectable effect, and statistical power for given parameters

Practical Tools and Applications in Stata

Using the power Command in Stata

This guide uses Stata’s power command to perform sample size and power calculations, and to create sensitivity tables and graphs.1

📌 Type this in Stata for help:

help power

The India education intervention example illustrates three types of power analysis:

Goal: Determine how many participants you need to detect an expected effect.

Assumptions:

  • Effect size or minimum detectable effect: 0.2 S.D.
  • Statistical power: 80%
  • Variance of the outcome variable: 1 S.D.
  • A single treatment group and a control group
  • A single post-treatment data collection at endline
  • Program randomization at the individual child level
  • Perfect compliance
  • No attrition
  • No additional controls

Stata code:

%%stata
power twomeans 0 0.2 , power(0.8) sd(1)

Performing iteration ...

Estimated sample sizes for a two-sample means test
t test assuming sd1 = sd2 = sd
H0: m2 = m1  versus  Ha: m2 != m1

Study parameters:

        alpha =    0.0500
        power =    0.8000
        delta =    0.2000
           m1 =    0.0000
           m2 =    0.2000
           sd =    1.0000

Estimated sample sizes:

            N =       788
  N per group =       394

This tells Stata to calculate the required sample size for each group—treatment and control—to detect a 0.2 S.D. effect. The required study sample size to achieve a power of 80% is 788 students: 394 in the treatment group and 394 in the control group.

Stata code:

Now, suppose researchers expect a larger treatment effect of 0.4 standard deviations instead of 0.2 S.D., keeping all other assumptions the same. To calculate the required sample size for this new scenario:

%%stata
power twomeans 0 0.4 , power(0.8) sd(1)

Performing iteration ...

Estimated sample sizes for a two-sample means test
t test assuming sd1 = sd2 = sd
H0: m2 = m1  versus  Ha: m2 != m1

Study parameters:

        alpha =    0.0500
        power =    0.8000
        delta =    0.4000
           m1 =    0.0000
           m2 =    0.4000
           sd =    1.0000

Estimated sample sizes:

            N =       200
  N per group =       100

Stata code:

Now, changing the assumption: What if researchers want to increase the statistical power to 90%?

To calculate the required sample size for an effect size of 0.4 S.D. and 90% power:

%%stata
power twomeans 0 0.2 , power(0.9) sd(1)

Performing iteration ...

Estimated sample sizes for a two-sample means test
t test assuming sd1 = sd2 = sd
H0: m2 = m1  versus  Ha: m2 != m1

Study parameters:

        alpha =    0.0500
        power =    0.9000
        delta =    0.2000
           m1 =    0.0000
           m2 =    0.2000
           sd =    1.0000

Estimated sample sizes:

            N =     1,054
  N per group =       527

The required sample size would increase to 1,054.

Additional exercises are available in the following do file: Download power.do

Goal: Find the smallest effect size you can detect given your sample size.

Assumptions:

  • Sample size: 1,000 students with 500 per group
  • Power: 80%
  • Variance of the outcome variable: 1 S.D

Stata code:

%%stata
power twomeans 0 , n(1000) power(0.8) sd(1)

Performing iteration ...

Estimated experimental-group mean for a two-sample means test
t test assuming sd1 = sd2 = sd
H0: m2 = m1  versus  Ha: m2 != m1; m2 > m1

Study parameters:

        alpha =    0.0500
        power =    0.8000
            N =     1,000
  N per group =       500
           m1 =    0.0000
           sd =    1.0000

Estimated effect size and experimental-group mean:

        delta =    0.1774
           m2 =    0.1774

This calculates the minimum effect size you can detect with the given sample size.

Stata code:

As shown in the graph, with 500, 750, 1,000, 1,250, and 1,500 students, researchers can detect smaller treatment effects as the sample size increases.

%%stata
power twomeans 0 , n(500(250)1500) power(0.8) sd(1) graph

Additional exercises are available in the following do file: Download power.do

Goal: Determine the statistical power based on your sample size and expected effect.

Assumptions:

  • Study sample size: 1,000
  • Effect size: 0.2 S.D.
  • Variance of the outcome variable: 1 S.D.

Stata code:

%%stata
power twomeans 0 0.2 , n(1000) sd(1)

Estimated power for a two-sample means test
t test assuming sd1 = sd2 = sd
H0: m2 = m1  versus  Ha: m2 != m1

Study parameters:

        alpha =    0.0500
            N =     1,000
  N per group =       500
        delta =    0.2000
           m1 =    0.0000
           m2 =    0.2000
           sd =    1.0000

Estimated power:

        power =    0.8848

The estimated power under these assumptions is 0.8,848.

Stata code:

Smaller samples lead to less power. As sample size increases, studies reach a larger probability of avoiding a type II error or false negative. In the graph, as the sample size increases, the estimated power also increases. This means studies are more likely to detect a true effect with a larger sample.

%%stata
power twomeans 0 0.2 , n(100(100)1000) sd(1) graph

Additional exercises are available in the following do file: Download power.do

Additional Resources

Back to top

Footnotes

  1. For more details on the power command, see the Stata documentation.↩︎

Reuse