App H Calculation for District Sample

Appendix H

Calculation of Minimum Detectable Effects for District-Level Proportions Using the IDEA National Implementation Study (IDEA NAIS) District Sample

Let represent the population proportion of some characteristic of interest associated with school districts. We want to estimate . Assume that we select a simple random sample of school districts. Let the sample proportion based on districts be . The standard error (standard deviation) of is given by

where

is the population size of school districts.

Assuming that the sample proportion has a normal distribution with as the mean and s.e.( ) as the standard deviation, a 95% confidence interval for is given by

1.96 s.e.( ).

Since we do not have a simple random sample, we assume a design effect of 1.6. This is the ratio of the variance of under the sampling design used for the survey to the variance under simple random sampling. The effective sample size is

.

If we have a sample of 1,200 school districts and we have a response rate of 80%, then we have 960 schools in the sample. The effective sample size is

.

Let =0.5

The variance of is =

= (0.95718) 0.25/600 = 0.000399.

Therefore s.e.( )= 0.019975, assuming that the population of school districts is 13,988.

95% confidence interval for is 0.50 1.96 x 0.019975 which is

0.50 0.0391.

In percentages, a 95% confidence interval for the population percentage in this case is

50 3.9 percentage points.