PEELS wave 3 Nonresponse Bias Examination 10-22-08

Nonresponse Bias Examination

Wave 3 nonresponse bias examination

Non-response bias is usually small when the response rate is high or moderate, and weight adjustments can eliminate most of the bias. In Wave 3, PEELS was fortunate to have moderately high response rates, reducing the overall level of concern about response bias. Table 1 presents the Wave 1, Wave 2, and Wave 3 response rates for PEELS. The response rate for the teacher questionnaires is below 80 percent in Wave 1, which is a concern. However, it has been improved in later waves.

To explore the possibility of response bias in the Wave 3 collection, Westat generated tables showing the number and percentage of responses by child’s race/ethnicity and household income. Those data are presented in tables 2 and 3.

Response rates suggest some variation by child’s race/ethnicity. On the parent interview, response rates ranged from 82 percent for parents of Black or African American children to 93 percent for parents of non-Hispanic White children. On the child assessment, response rates ranged from 89 percent for American Indians to 97 percent for Asians, Native Hawaiians or other Pacific Islanders. Finally, on the teacher questionnaires, rates ranged from 76 percent of Black children’s teachers to 85 percent of non-Hispanic White children’s teachers (see table 2).

The data on response rates suggest an association between household income and response probability on the parent interview, with lower income families less likely to complete interviews. The response rates for all income groups were between 83 and 95 percent. Response rates for the child assessment and teacher questionnaires showed little variation by household income (see table 3).

This level of variation does not present a serious threat of nonresponse bias.

Table 1. PEELS Response rates, by wave and instrument

	Wave 1	Wave 2	Wave 3a
SEA questionnaire	100%	--	--
LEA questionnaire	87%	--	--
Parent interview	96%	93%	88%
Child assessment	96%	94%	93%
Teacher questionnaire	76%	86%	81%
Principal/program director questionnaire	76%	--	--

^a Wave 3 response rates are subject to change until the data have been finalized.

SOURCE: PEELS Study Management System. Previously unpublished tabulations, August 2006.

Table 2. Number and percentage of PEELS Wave 3 respondents, by race/ethnicity

	Total Sample	Parent		Assessment		Teacher
	n	n	%	n	%	n	%
Total	3,098	2,719	87.8	2,889	93.3	2,509	81.0
American Indian or Alaskan Native	19	17	89.5	17	89.5	15	78.9
Asian, Native Hawaiian or Other Pacific Islander	59	52	88.1	57	96.6	49	83.1
Black or African American	277	228	82.3	258	93.1	211	76.2
Hispanic	557	468	84.0	526	94.4	439	78.8
White, Non-Hispanic	1,730	1,604	92.7	1,653	95.5	1,467	84.8
More than one race	161	138	85.7	145	90.1	129	80.1
Missing	295	212	71.9	233	79.0	199	67.5

NOTE: Study withdrawals manually added to missing data line, column 1.

SOURCE: PEELS Study Management System. Previously unpublished tabulations, August 2006.

Table 3. Number and percentage of PEELS Wave 3 respondents, by household income

	Total Sample	Parent		Assessment		Teacher
	n	n	%	n	%	n	%
Total	3,098	2,719	87.8	2,889	93.3	2,509	81.0
Less than $20,000	723	599	82.8	677	93.6	574	79.4
$20,001 to $30,000	424	365	86.1	399	94.1	354	83.5
$30,001 to $40,000	359	330	91.9	341	95.0	296	82.5
$40,001 to $50,000	306	283	92.5	293	95.8	249	81.4
More than $50,000	960	913	95.1	918	95.6	815	84.9
Missing	326	229	70.2	261	80.1	221	67.8

NOTE: Study withdrawals manually added to missing data line, column 1.

SOURCE: PEELS Study Management System. Previously unpublished tabulations, August 2006.

Wave 4 nonresponse bias examination

Non-response bias is usually small when the response rate is high or moderate, and weight adjustments can eliminate most of the bias. In Wave 4, PEELS continued to have moderately high response rates, reducing the overall level of concern about response bias. Table 1 presents the Wave 1, Wave 2, Wave 3, and Wave 4 response rates for PEELS. Lower response rates in Wave 4 may be partially attributed to efforts to reduce the Wave 4 data collection budget.

To explore the possibility of response bias in the Wave 4 collection, Westat generated tables showing the number and percentage of responses by child’s race/ethnicity and household income. Those data are presented in tables 2 and 3.

Response rates suggest some variation by child’s race/ethnicity. On the parent interview, response rates ranged from 70 percent for parents of Black or African American children to 90 percent for parents of American Indians or Alaskan Natives. On the child assessment, response rates ranged from 79 percent for American Indians and Alaskan Natives to 90 percent for non-Hispanic Whites. Finally, on the teacher questionnaires, rates ranged from 75 percent of Black children’s teachers to 87 percent of non-Hispanic White children’s teachers (see table 2).

The data on response rates suggest an association between household income and response probability on the parent interview, with lower income families less likely to complete interviews. The response rates for all income groups were between 74 and 93 percent. Response rates for the child assessment and teacher questionnaires showed little variation by household income (see table 3).

This level of variation does not present a serious threat of nonresponse bias.

Table 1. PEELS Response rates, by wave and instrument

	Wave 1	Wave 2	Wave 3	Wave 4a
SEA questionnaire	100%	--	--	--
LEA questionnaire	87%	--	--	--
Parent interview	96%	93%	88%	80%
Child assessment	96%	94%	93%	85%
Teacher questionnaire	76%	86%	81%	81%
Principal/program director questionnaire	76%	--	--	--

^a Wave 4 response rates are subject to change until the data have been finalized.

SOURCE: PEELS Study Management System. Previously unpublished tabulations, July 2007.

Table 2. Number and percentage of PEELS Wave 4 respondents, by race/ethnicity

	Total Sample			Parent			Assessment			Teacher
		n		n	%		n	%		n	%
Total		3,104		2,488	80.2		2,632	84.8		2,502	80.6
American Indian or Alaskan Native		19		17	89.5		15	78.9		15	78.9
Asian, Native Hawaiian or Other Pacific Islander		58		50	86.2		48	82.8		50	86.2
Black or African American		276		193	69.9		224	81.2		207	75.0
Hispanic		545		418	76.7		480	88.1		435	79.8
White, Non-Hispanic		1,702		1,490	87.5		1,528	89.8		1,472	86.5
More than one race		160		127	79.4		131	81.9		127	79.4
Missing		344		193	57.1		206	59.9		196	57.0

NOTE: Study withdrawals manually added to missing data line, column 1.

SOURCE: PEELS Study Management System. Previously unpublished tabulations, July 2007.

Table 3. Number and percentage of PEELS Wave 4 respondents, by household income

	Total Sample		Parent			Assessment			Teacher
	n		n	%		n	%		n	%
Total	3,104		2,488	80.2		2,632	84.8		2,502	80.6
Less than $20,000	720		531	73.8		621	86.3		580	80.6
$20,001 to $30,000	412		327	79.4		358	86.9		331	80.3
$30,001 to $40,000	357		287	80.4		308	86.3		287	80.4
$40,001 to $50,000	300		256	85.3		265	88.3		260	86.7
More than $50,000	948		879	92.7		852	89.9		828	87.3
Missing	367		208	56.7		228	62.1		216	58.9

NOTE: Study withdrawals manually added to missing data line, column 1.

SOURCE: PEELS Study Management System. Previously unpublished tabulations, July 2007.