Volume 1 FRSS 107 Young Adults ELL Programs Feasibility Calls

Volume I:

Fast Response Survey System (FRSS) 107: Older Adolescent and Young Adult English Language Learner (ELL) Programs

• Feasibility Calls

OMB# 1850-0803 v. 109

July 16, 2014

National Center for Education Statistics

U.S. Department of Education

Justification

The National Center for Education Statistics (NCES), U.S. Department of Education (ED), requests OMB approval under the NCES system clearance for Cognitive, Pilot and Field Test studies (OMB #1850-0803) to conduct feasibility calls for Fast Response Survey System (FRSS) survey #107 on older adolescent and young adult English language learner (ELL) programs in public school districts. The Office of English Language Acquisition (OELA) requested that NCES conduct this FRSS survey.

According to the Center for Applied Linguistics, English Learners (ELs) are the fastest growing segment of the K-12 student population and they often struggle to be successful in school. A 2005 report by the Center for Adult English Language Acquisition notes that as high school exit criteria increase in rigor, many ELs are often unable to graduate on time and are turning to adult education programs to earn high school diplomas, acquire job skills, and improve their English language proficiency. However, little is known about the characteristics of older adolescent and young adult ELs, and there are no national data available on the characteristics of secondary education programs and services designed to serve them. The purpose of this FRSS survey is to collect national data from school districts on programs and services designed to serve ELs ages 14 to 21. Topics will include the availability of programs for these students, including the availability of programs designed specifically for a subset of the EL population: students 18 to 21 who were not able to earn a high school diploma within four years; the characteristics of the programs in terms of their purpose, content, staffing, and target population; the types of English language development programs offered by these programs (e.g., programs that use English and another language, programs that use English only); and the use of technology in the implementation of programs and provision of services.

The purpose of feasibility calls is to explore topics for potential survey items, identify and correct any potential issues with the content and format of the survey before conducting pretests, and to ensure that the survey captures the intended meaning of the questions and minimizes the burden imposed on respondents. A request to conduct pretest activities will follow completion of the feasibility calls. Early rounds of the feasibility calls will ask school district respondents to participate in a telephone discussion about the ELL population in their school district and the programs and services they provide to ELL students. This information will be used to develop the questionnaire. In later rounds of the feasibility calls, respondents will be asked to review, but not complete draft questionnaire items and ultimately, the draft questionnaire, and provide feedback by telephone. The pretests, approval for which will be requested at a later date, will involve asking respondents to complete the draft survey and participate in a telephone debriefing. Feasibility calls will be done before pretests to minimize the burden on respondents. Pretests will be done as a final test prior to OMB clearance submission to conduct the full-scale survey. The request to conduct the full-scale survey will be submitted at a later date under OMB generic clearance for quick response surveys (OMB#1850-0733), which are authorized under the Education Science Reform Act of 2002 (20 U.S.C. §9543). NCES has contracted Westat for all stages of this survey.

Design

Overview of Survey Development

NCES has contracted Westat to conduct the survey for FRSS 107, including development of the survey instruments. FRSS has established procedures for developing short surveys on a wide variety of topics. The techniques that we plan to use to shape the survey design on FRSS 107 include literature reviews on ELL programs, input from the NCES Quality Review Board (QRB), several rounds of feasibility calls, and up to two pretests. The specific ways we plan to use feasibility calls are discussed below.

We anticipate conducting up to three rounds of feasibility calls, each with fifteen or fewer respondents. With new survey topics such as that of older adolescent and young adult ELL programs, the initial feasibility calls use an open-ended interview guide rather than a questionnaire. As rounds of feasibility calls progress, respondents will be asked to review, but not complete draft questionnaire items and ultimately a draft questionnaire. Conducting multiple rounds of feasibility calls will systematically inform us about the availability and nature of ELL programs and services at the secondary level in order to draft a questionnaire, and in later rounds will provide in-depth information on respondents’ perceptions of the draft survey and response burden. The process will result in several iterations of the questionnaire items. The first round of calls will focus on learning more about the ELL populations ages 14 to 21 that school districts serve, the terminology districts use regarding these populations, and the characteristics of the ELL programs and services they provide at the secondary level. For the next round of feasibility calls, respondents will be asked to review draft survey questions, instructions, and definitions based on the initial round of feasibility calls. As a result of the feedback we receive, we will make any necessary changes to the survey items and draft a questionnaire. In the third and final round of feasibility calls, we will ask for respondents’ feedback on the draft questionnaire. The resulting draft of the survey will be reviewed by the NCES QRB and revised as necessary to prepare it for pretesting.

NCES Review and Consultations Outside of Agency

The NCES QRB members reviewed a draft list of questionnaire topics prior to this submission for the feasibility calls. Revisions were made to the list of topics based on input from the reviewers, and the list was used to develop an interview guide for the feasibility calls. In addition to staff from each of the three Divisions at NCES, the QRB also included staff from OELA; the Office of Career, Technical, and Adult Education (OCTAE); and the Office of Postsecondary Education (OPE). The QRB members for this survey are listed below:

Melissa Escalante, OELA

Carlos Martinez, OELA

Debra Suarez, OCTAE

Lenore Garcia, OPE

Chris Chapman, NCES

Eugene Owen, NCES (PIAAC)

Arnold Goldstein, NCES (NAEP)

Richard Reeves, NCES (IPADS)

Sharon Boivin, NCES (ATES)

Kashka Kubzdela, NCES

Sample, Burden, and Cost

In this submission, we are requesting approval for feasibility calls with members of the target population. We will conduct up to three rounds of feasibility calls for the survey, with 15 or fewer respondents per round. School districts will be recruited to participate in feasibility calls based on various district characteristics including level (secondary or unified), size, urbanicity (locale), and geographic region. Respondents will be recruited by telephone and will be identified as the person in the district who is most familiar with English language learner programs for older adolescents and young adults in their district.

Telephone interviewers will recruit participants for the feasibility calls using the recruitment script in Attachment 1. Interviewers will schedule an appointment to complete the feasibility calls with cooperating district-level personnel. Following telephone recruitment, interviewers will either email or fax a cover letter and draft questionnaire to the participating districts (as discussed below in the Data Collection Instrument section). In order to recruit 15 respondents per round, we anticipate contacting 45 public school districts per round (Table 1). On average, recruitment calls with respondents who agree to participate in the feasibility calls are expected to take about 10 minutes to explain the purpose of the call and set up an appointment to discuss the survey; all other recruitment calls are expected to take about 3 minutes. Prior to the feasibility calls, respondents will be asked to review (but not complete) either interview topics, draft questionnaire items, or a draft questionnaire, depending on the round of calls, which should take approximately 10 minutes. The feasibility call should take approximately 30 minutes to complete. The estimated burden for one round of feasibility calls is approximately 15 hours, and total estimated burden time for all three rounds of feasibility calls is approximately 45 hours. We anticipate that the estimated cost to the federal government will be approximately $6,000 for each round of feasibility calls.

Table 1. Estimated maximum burden time for up to three rounds of feasibility calls for FRSS 107

Respondents	Number of Respondents	Number of Responses1	Burden Hours per Respondent	Total Burden Hours
Recruitment: Districts not participating in the feasibility call	30	30	0.05	2
Recruitment: Districts participating in the feasibility call	15	15	0.17	3
Survey review and feasibility call	15	15	0.67	10
Total per round	45	60	--	15
Total for three rounds	135	180	--	45

¹ Counts each contact (e.g., recruitment and feasibility call are counted separately even when they are with the same respondents).

Data Collection Instrument

For each round of feasibility calls, a cover letter and topic list, or in later rounds, draft questionnaire items or questionnaire, will be emailed or faxed to each participating school district. The cover letter and topic list for the first round of feasibility calls are included in this document as Attachment 2. The cover letter thanks the respondent for agreeing to participate in the feasibility call, introduces the purpose and content of the survey, indicates that participation is voluntary, indicates that respondents should review the topic list on which we will base the telephone discussion, and provides contact information should any questions arise before the scheduled discussion with the survey manager. On the cover letter, respondents are assured that their participation is voluntary and their answers may only be used for statistical purposes and may not be disclosed or used in identifiable form for any other purpose unless compelled by law. The law is cited on the cover letter. The materials for subsequent rounds of calls will be similar, except the questions for each round will include modifications or new items that resulted from the previous rounds.

Interview Guide: overview of topics and interview guide

The purpose of this FRSS survey is to collect national data from school districts on programs and services designed to serve ELLs ages 14 to 21, with a particular interest in older adolescent and young adult ELLs who may be at risk for dropping out or not graduating on time. Little is known about the characteristics of older adolescent and young adult ELLs, and there are no national data available on the characteristics of secondary education programs and services designed to serve them. The interview guide (see attachment 3) will be used to learn more about ELL populations and programs and services at the secondary level in order to inform development of a questionnaire that is clear and relevant to respondents. The interview guide will cover several topics that include terminology school districts use to describe ELLs and ELL programs and services; the process districts use for identifying students eligible for district programs and services at the secondary level; the characteristics of programs and services provided to ELLs at the secondary level (e.g., the specific types of programs and services provided and the ages served, the extent to which students’ native language are incorporated into instruction and materials, and the extent to which technology is used to implement and administer programs and services); the system districts use to monitor ELL’s current status and progress; and what, if any, relationships districts have with other organizations regarding programs and services for ELLs ages 14 to 21.

Timeline

Feasibility call activities are expected to begin in July 2014, as soon as approval is received from OMB. Feasibility call activities are anticipated to take about four months to complete, including potentially three rounds of feasibility calls and the development of and revisions to the survey between each round.

Attachment 1: FRSS 107 Feasibility Call Recruitment Script

FRSS 107

Older Adolescent and Young Adult English Language Learner Programs Feasibility Call Recruitment Script

Hello, my name is __________________.

I am calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education, regarding a survey on older adolescent and young adult English language learner (ELL) programs. I would like to ask the district-level person who is most familiar with ELL programs in your district to provide us with information about programs and services in your district that serve students ages 14 to 21.

Who is the person in your district who is most knowledgeable about English language learner programs for older adolescents and young adults?

(This is often an ELL or multilingual program director or coordinator.)

May I please speak to that person?

CONNECTED TO DISTRICT–LEVEL PERSON MOST FAMILIAR WITH ENGLISH LANGUAGE LEARNER PROGRAMS IN THE DISTRICT

Hello, my name is __________________.

I’m calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education, regarding a survey on older adolescent and young adult English language learner (ELL) programs. We would like your help in developing a questionnaire about ELL programs and services at the secondary level. Specifically, we would like you to participate in a telephone discussion in which we ask you about the ELL population ages 14 to 21 in your district and the programs and services provided to those students. The call will take about 30 minutes.

1. How would you like me to send you the interview materials (email, fax)?

2. We ask that you review the interview materials before you talk to the survey manager. When would be a good time for the survey manager, Kimberley Raue, to call you to discuss the interview topics and obtain your comments? How about [SUGGEST A TIME]. [Just to be sure, you are in the [Eastern, Central, Mountain, Pacific] time zone?]

3. What is the best telephone number at which the survey manager can reach you?

Thank you. Your insights will be very helpful.

Attachment 2: FRSS 107 Cover Letter

U.S. Department of Education  Institute of Education Sciences  National Center for Education Statistics

[Date] 2014

Dear Participant,

Thank you for agreeing to give us feedback on the development of a survey on older adolescent and young adult English language learner (ELL) programs. Westat, a research company located in Rockville, Maryland, is conducting this survey for the National Center for Education Statistics, within the U. S. Department of Education. As part of our survey development, we are looking for feedback from district-level personnel about topics and questions that might be included in the survey. Your input will be essential in ultimately developing a questionnaire that is clear and relevant, and not overly burdensome to respondents. All information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose unless otherwise compelled by law (Education Sciences Reform Act of 2002, 20 U.S.C. § 9573).

We ask that you review the following list of interview topics prior to our telephone conversation.

1. Terminology used by your district to refer to students who are learning English and the programs and services provided to them;

2. The ways in which your district identifies English learners (ELs) ages 14 to 21 who are eligible for district programs and services;

3. The types of programs and services offered by your district for ELs ages 14 to 21;

4. The system your district uses to monitor ELs ages 14 to 21 and their status/progress;

5. Information about any formal or informal relationships your district has with other organizations regarding programs or services for ELs ages 14 to 21.

My colleague and I will call you at the scheduled time to get your feedback on the materials and to discuss any comments or suggestions you may have. In the meantime, feel free to call me at Westat’s toll-free number, 800-937-8281, ext. 3865, if you have any questions. You may also reach me by email at kimberleyraue@westat.com.

Thank you for your much needed assistance!

Sincerely,

Kimberley Raue

Westat Survey Manager