Enviro Scan - Supporting Statement A revised

Natural Experiments and Model Career-Focused Schools:

An Environmental Scan

Supporting Statement – Part A

OMB #1830-NEW

Prepared by RTI International for:

Office of Career, Technical, and Adult Education

U.S. Department of Education

Table of Contents

A. Justification 1

1. Circumstances Making the Collection of Information Necessary 1

2. Purpose and Use of the Information Collection 1

3. Use of Technological Collection Techniques 2

4. Efforts to Identify and Avoid Duplication 2

5. Impact on Small Businesses or Other Small Entities 3

6. Consequences of Not Collecting the Information or Collecting It Less Frequently 3

7. Special Circumstances 3

8. Federal Register Notice and Consultation 3

9. Explanation of Any Payment or Gift to Respondents 3

10. Assurance of Confidentiality 3

11. Justification for Sensitive Questions 5

12. Burden Estimate 5

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 5

14. Estimate of Annualized Cost to the Federal Government 5

15. Explanation of Program Changes or Adjustments 6

16. Plans for Tabulation and Publication and Project Time Schedule 6

17. Display of OMB Approval Expiration Date 6

List of Exhibits

Exhibit 1. Key Research Questions…………………………………………………………………………………………………………2

Exhibit 2. Estimated Burden Hours……………………………………………………………………………………………………….5

Exhibit 3. Estimated Cost to Respondents……………………………………………………………………………………………5

1. Justification

This section provides detailed justification for the request for approval of interviews with high school principals for the study entitled “Natural Experiments and Model Career-Focused Schools: An Environmental Scan.”

1. Circumstances Making the Collection of Information Necessary

The growth of public school choice policies, particularly in urban areas, has led to growth in the number of secondary school options. College-and career-focused schools represent one potentially promising approach to secondary education. They are designed to offer a structured sequence of courses that spans secondary and postsecondary education, lead to an industry certification or license and a postsecondary certificate or degree, and enable graduates to gain employment in a high-growth industry upon completion.

The U.S. Department of Education is interested in identifying and disseminating information about promising school initiatives and in determining if there are natural circumstances that approximate random assignment in some of these schools. The Department believes that there are college- and career-focused school options that are oversubscribed—meaning that more students seek to enroll than the schools can accommodate—and that these schools admit students using a lottery or other process that approximates random assignment. If so, future research could use these naturally occurring experimental conditions to investigate differences in achievement for students who attend these types of schools.

1. Purpose and Use of the Information Collection

The purpose of the data collection is to identify and provide information about college- and career-focused schools that may be candidates for inclusion in a future randomized control trial (RCT). RTI International is conducting a one-time study of schools in three college- and career-focused networks or initiatives to collect information about the network/initiative program models and schools that are oversubscribed and use lotteries to select students.

The three networks include:

• College and Career Academy Support Network (CCASN)

• New Tech Network (NTN)

• National Academy Foundation (NAF)

RTI researchers will conduct semi-structured telephone interviews with up to 100 leaders of college-and career-focused schools in the networks to obtain information about each school’s program model and status of implementation, student admission practices, and any oversubscription experienced during the four most recent admission cycles (2011-12 to 2015-16 academic years). The project team will collect information during the 2015-2016 academic year. Copies of data collection instruments are attached in Appendix B.

Exhibit 1. Key research questions

1. Which schools/institutions exhibit the network-defined characteristics (i.e., college- and career-focused, program implementation, and status of implementation)?

2. Which schools are oversubscribed and use waiting lists?

3. Which oversubscribed schools select students by lottery or other practices that approximate random assignment?

RTI will provide tabular summaries of the collected information to the Department of Education. The results will not be shared publicly beyond Department of Education staff.

The Department will use the information to assess the feasibility of conducting experimental research on college- and career-focused schools. If the study findings indicate that a substantial number of college- and career-focused schools admit students through systems that approximate random assignment, the Department may support or solicit experimental research on some of the schools identified through the environmental scan.

2. Use of Technological Collection Techniques

RTI will collect only the minimum information necessary for the study. RTI has designed data collection procedures to minimize respondent burden. The researchers will use semi-structured telephone interviews with school principals or their designees to obtain a complete and accurate view of college- and career-focused activities and policies pertaining to admission lotteries, as well as to ease reporting and maximize the comparability of responses. Advantages of semi-structured telephone interviews over electronic methods for this particular type of information include: (1) the ability to ask for clarification of unclear responses, (2) interviewers can elicit longer, more complete answers than a respondent would give on their own, and (3) respondents are less likely to complete a longer questionnaire online than they would if talking with an interviewer. In the event that the research team is unable to contact a potential interviewee after a reasonable number of attempts (approximately three), the team will invite the interviewee to complete the interview via email in an effort to increase participation of non-responders. Allowing participants a choice of email or telephone may increase the likelihood of participation because it is sensitive to possible communication/mode preferences.

3. Efforts to Identify and Avoid Duplication

The information to be collected focuses on the characteristics that may make a college- and career-focused school a valid candidate for inclusion in a future experimental study. In designing the proposed data collection activities, RTI has endeavored to ensure that this effort does not duplicate ongoing data collection initiatives and that existing datasets do not already address the research questions.

Whenever possible, researchers have obtained publicly available information about the college- and career-focused schools through school websites and the Common Core of Data (CCD). The project team also consulted the three college- and career-focused network websites and conducted brief interviews with leaders of those networks to obtain information about their program models.

The remaining data items—such as the schools’ current admission policies and use of practices that may approximate random assignment—will be obtained through the proposed data collection because it is not available through the sources listed above or through existing public data collections. RTI reviewed the following data collections conducted and maintained by the National Center for Education Statistics and determined that none contain current data relevant to the schools and practices under investigation:

• Career/Technical Education Statistics (CTES)

• Education Longitudinal Study of 2002 (ELS)

• Fast Response Survey System (FRSS)

• High School and Beyond (HS&B)

• High School Longitudinal Study of 2009 (HSLS:09)

• High School Transcript Studies (HST)

• National Education Longitudinal Study of 1988 (NELS:88)

• National Teacher and Principal Survey (PSS)

• School Districts Demographics System (SDDS)

• Schools and Staffing Survey (SASS)

While two data collections include information related to special school admission requirements (CTES) and school choice (SASS), neither provides current and specific information about application and admission practices or the use of wait lists and lotteries when schools are oversubscribed.

4. Impact on Small Businesses or Other Small Entities

The respondents for the study are school principals. No small businesses or small entities will be involved in this data collection.

5. Consequences of Not Collecting the Information or Collecting It Less Frequently

This is a one-time collection. The Department of Education is collecting information about the admission practices of college- and career-focused schools to ascertain if future experimental studies are feasible. Without this data collection, the Department of Education will not be able to assess the feasibility of future RCTs, which would limit the Department’s ability to obtain evidence-based information through experimental research.

6. Special Circumstances

There are no special circumstances for the data collection.

7. Federal Register Notice and Consultation

The 60-day notice to solicit public comments was/will be published on September 28, 2015. The 30-day notice will be published to solicit additional public comments.

8. Explanation of Any Payment or Gift to Respondents

There are no plans to provide a payment or gift to respondents for participation.

9. Assurance of Confidentiality

No assurance of confidentiality will be made to respondents. Survey responses will be provided to the Office of Career, Technical, and Adult Education.

10. Justification for Sensitive Questions

The interview questions do not collect sensitive information. Data collection focuses on facts and opinions about schools, not on individual respondents themselves.

Respondents will be advised during the informed consent process of the voluntary nature of participation and their right to refuse to answer any question.

Burden Estimate

The one-time data collection will take place in late fall 2015 or early 2016. A 10–15 minute telephone interview will be conducted with up to 100 school principals or their designees. Potential interviewees who cannot be reached or who are unable to schedule a phone interview will be invited to respond to the interview via email. No record-keeping activity is required. Assuming an hourly rate of $45, the estimated cost per respondent to complete the interview by phone or email is $11.25.

Exhibit 2. Estimated burden hours

Type of Respondent	Name of Collection	No. of Respondents	No. of Responses/ Respondent	Burden / Response (Hours)	Total Burden (Hours)
Secondary school principal	Telephone or email interview	100	1	0.25	25

Exhibit 3. Estimated cost to respondents

Type of Respondent	Name of Collection	Total Burden Hours	Hourly Wage Rate	Cost / Respondent	Total Respondent Costs
Secondary school principal	Telephone or email interview	25	$45.00	$11.25	$1,125

11. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There are no direct costs to respondents beyond their time to participate in the study as documented in question 12 above.

12. Estimate of Annualized Cost to the Federal Government

RTI was awarded a 2-year contract from the Department of Education to complete this project. The estimated cost to the Federal government is $175,499, which includes the estimated cost of coordination with the U.S. Department of Education; project plan and schedule development; RTI IRB applications; overseeing of data collection; analysis; reporting; and progress reporting. The total cost includes evaluation activities that are not part of this OMB application, such as reviews of network and school websites.

13. Explanation of Program Changes or Adjustments

This is a new collection of information.

14. Plans for Tabulation and Publication and Project Time Schedule

The results of the study will be shared only with staff in the U.S. Department of Education and will not be published. Researchers will prepare tables for each school that include the findings of this data collection as well as information obtained from interviews with network leaders and from publicly available sources such as network and school websites and the Common Core of Data. RTI anticipates submitting the tables in spring of 2016, approximately six months after receiving OMB approval.

15. Display of OMB Approval Expiration Date

The OMB expiration date will be displayed on all data collection instruments.

2. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.