Ed Response to First Round of OMB Questions

Responses to Questions about the Evaluation of Secondary Math Teachers from Two Highly Selective Routes to Alternative Certification – Addendum Package (200904-1850-004)

We thank OMB for the thoughtful questions received on July 16, 2009. Below, we present our responses to the questions.

1. The use of “Confidentiality Statements.” While Ed already added the correct statements in response to OMB’s comments, you have still not deleted competing statements within the same document. As an aside, an information collection cannot use “confidential” in any pledge that also uses “statistical use only.” Since the forms all have a statement at the bottom that use “statistical”, the word “confidential” should not be used in the text on the main page. Please use accepted IES language and nothing more.

Response: We have revised all instruments to include only the accepted IES language (see attachments).

2. Incentives. We have been unable to locate a study addressing incentives in relation to returning of consent forms. The control number reference that Ed provided us with, 1850-0801, is actually a study with no incentives called "Evaluation of the Impact of Literacy Interventions in Freshman Academies--Follow-Up Forms for Students and Teachers (KI)." Currently, OMB is uncomfortable with the idea of incentives not directly related to assessment participation (i.e. classroom and student incentives for returning consent forms).

Response: We have proposed offering an incentive for the return of consent forms, regardless of the consent status, based on the literature which suggests that active consent procedures can reduce sample size and cause bias. Studies have found that when active parental consent is required participation rates are between 40 and 60 percent, in contrast to participation rates of 80 to 100 percent in studies with only passive consent.¹ Furthermore, there is evidence that disadvantaged and at-risk populations, such as minorities, low achievers, children with less educated parents or in less stable family structures, may be underrepresented in studies requiring active consent.² Hence, we are concerned that without incentives, a low rate of return of consent forms will significantly lower the sample size and our ability to generalize the findings to all students in the class.

We further based our decision on two OMB approved studies which offered incentives for the return of parental consent forms. In the Impact Evaluation of Mandatory-Random Student Drug Testing (OMB Approval #1850-0818), students would receive a movie ticket ($7 value) for the return of a completed consent form, regardless of the consent status. In the evaluation of the Youth Transition Demonstration Projects (OMB Approval # 0960-0687), sponsored by the Social Security Administration, we offered $10 Target gift cards or Metrocards to youth who returned completed consent forms, irrespective of the consent status.

2. Gift Cards. Ed’s response indicated that the gift cards given to students would be used at food outlets. Given the varying level on food quality associated with the establishments indicated in the response, OMB would prefer that the gift cards not be used solely for food outlets.

Response: We understand OMB’s concerns about the gift cards for food outlet stores. Instead, we will provide students with gift cards to a local general store such as Walmart or Target. Students participating in the pretests for the student assessment suggested general store gift cards if they could not have gift cards for movie theaters, which are too expensive, or gift cards for food outlets.

REFERENCES

Donovan, John, Richard Essor, and Frances M. Costa. “Syndrome of problem behavior in adolescence: A replication.” Journal of Consulting and Clinical Psychology, vol. 56(5), 1988, pp. 762-65.

Ellickson, Phyllis and Jennifer Hawes. “An assessment of active versus passive methods of obtaining parental consent.” Evaluation Review, vol. 13, 1989, pp. 45-55.

Esbensen, Finn-Aage, Elizabeth Deschenes, Ronald Vogel, Jennifer West, Karen Arboit, and Lesley Harris. “Active parental consent in school-based research: An examination of ethical and methodological issues.” Evaluation Review, vol. 20, 1996, pp. 737-53.

Kearney, Kathleen, Ronald Hopkins, Armand Mauss, and Ralph Weisheit. “Sample bias resulting from a requirement for written parental consent.” Public Opinion Quarterly, vol. 47, 1983, pp. 96-102.

Landis, S. and C. Janes. “The Claxton Elementary School health program: Merging perceptions and behaviors to identify problems.” Journal of School Health, vol. 65 (7), 1995, pp. 250-54.

Lueptow, L., S. Mueller, R. Hammes, and L. Master. “The impact of informed consent regulations on response rate and response bias.” Social Methods and Research, vol. 6, 1977, pp. 183-204.

McBride, C., S. Curry, A. Cheadle, c. Anderman, E. Wagner, P. Diehr, and B. Psaty. “School-level application of a social bonding model to adolescent risk-taking behaviors.” Journal of School Health, vol. 65(2), 1995, pp. 63-8.

Murray, David and P. Hannan. “Planning for the appropriate analysis in school-based drug-use prevention studies.” Journal of Consulting and Clinical Psychology, vol. 58(4), 1990, pp. 458-68.

Severson, Herbert and A. Biglan. “Rationale for the use of passive consent in smoking prevention research: Politics, policy, and pragmatics.” Preventive Medicine, vol. 18, 1989, pp. 267-279.

Severson, Herbert and D. Ary. “Sampling bias due to consent procedures with adolescents.” Addictive Behaviors, vol. 8, 1983, pp. 433-37.

Weeks, K., S. Levy, C. Zhu, C. Perhats, A. Handler, and B. Lay. “Impact of a social-based AIDS prevention program on young adolescents’ self-efficacy skills.” Health Education Research, vol. 10(3), 1995, pp. 329-44.

1 See, for example, Donovan et al. 1988; Lueptow et al. 1977; Severson and Bigland 1989, Severson and Ary 1983; Ellickson and Hawes 1989; Esbensen et al. 1996; Kearney et al. 1983; Landis and Janes 1995; McBride et al. 1995; Murray and Hannan 1990; and Weeks et al. 1995.

2 Ellickson and Hawes 1989; Esbensen et al. 1999.

3