Pretesting Report

Attachment 1 – Pretest Report

Fast Response Survey System (FRSS) 108: Career and Technical Education (CTE) Programs in Public School Districts

OMB# 1850-0733 v.32

October 2016

National Center for Education Statistics

U.S. Department of Education

Date:	October 3, 2016
To:	Braden Goetz Sharon Miller
From:	Cindy Gray Laurie Lewis
Subject:	FRSS 108 Career and Technical Education Programs: Pretest Results

This memo describes findings from the pretest calls for the FRSS 108 survey on career and technical education (CTE) programs. Calls were conducted with district-level personnel most familiar with CTE programs in their district. Pretest calls were conducted in 12 districts located in 12 states: Alabama, California, Idaho, Indiana, Kentucky, Maine, Massachusetts, Nebraska, New Jersey, North Dakota, Rhode Island, and Tennessee. The districts included urban, suburban, and town/rural community types, with small, medium and large enrollment sizes.

The district respondents were asked to complete the draft questionnaire and fax it to Westat, and then discuss it by telephone with the Westat survey manager. Respondents were asked to consider whether the instructions and definitions were clear and helpful, whether the questionnaire items were clear and easy to interpret, whether they had the data available to answer the questions, and how long it took to complete the questionnaire. Respondents reported a range of 5 to 45 minutes to complete the questionnaire, with an average of about 20 minutes.

The pretest identified response issues for a small number of questionnaire items, which are discussed below. No issues were found for any of the other questionnaire items.

Question 2: Which entities provide the CTE programs that the district offers

The pretest included one CTE district that serves students in 4 municipalities. Students from these 4 areas can enroll in the CTE district instead of enrolling in their home districts. The respondent for the CTE district answered “yes” to Q2a (an area/regional CTE center or a group/consortium of school districts) but said that response does not precisely reflect his district’s situation. This is because he thinks of a “CTE center” as a part time arrangement where students only attend for CTE and not for their whole high school education. Since his district provides all of the students’ high school education, he thinks of it as a CTE district and not a CTE center.

Recommendation: We considered changing the wording of Q2a to incorporate CTE districts as well as CTE centers. However, the current wording has been tested and worked well for feasibility call respondents and the other pretest respondents. Even the respondent for the CTE district was able to answer Q2, even though he thought the answer was not precise. Since many states do not have CTE districts that students attend full time, we are concerned that changing the wording might confuse the respondents from regular districts who are not aware of CTE districts. Also, this issue only affects the CTE districts, which will be a small proportion of the sample and will be identified by their response to Q16. We do not think it is a good idea to risk confusing the regular district respondents, who will be the vast majority of the sample, by changing the tested wording. Therefore, we recommend keeping the current wording.

Question 4: Number of CTE programs the district offers

Of the 12 respondents in the pretest, 5 said that they were not sure what level of detail should be reported in Q4. These districts classified their CTE programs in multiple levels of specificity, such as program areas, programs of study, and pathways.

Recommendation: Districts often used some of the same terms (e.g., program areas, programs of study, pathways) but used them in different ways or to mean different things, depending on how the terms were used in their states. This means that if we add an instruction on the level of specificity to report, the instruction may be interpreted differently in different districts and may not make sense to districts that do not use that terminology. This variation by state and district makes it extremely difficult to collect counts consistently across all districts. We recommend dropping this question from the survey because it was not requested by OCTAE, may be confusing to respondents, and will not collect consistent data across districts.

Note that this issue only applies to Q4 and not to other questions on the survey. Respondents did not need to identify the level of specificity to answer the other survey questions and reported that the other questions were generally clear and easy to answer.

Question 13: Barriers to student participation in CTE programs

One of the pretest respondents gave a response in the “other (specify)” item of “Academic teachers’ negative perceptions of CTE.” This is similar to comments received during the feasibility calls about teachers’ and guidance counselors’ negative perceptions of CTE.

Recommendation: We recommend adding an item to Q13 for “Teachers’ or guidance counselors’ negative perceptions of CTE.”

U.S. DEPARTMENT OF EDUCATION NATIONAL CENTER FOR EDUCATION STATISTICS (NCES) WASHINGTON, D.C. 20202 Career and Technical Education Programs in Public School Districts FAST RESPONSE SURVEY SYSTEM

O.M.B. No.: 1850–0733 EXPIRATION DATE: 02/2018

NCES is authorized to conduct this survey by the Education Sciences Reform Act of 2002 (20 U.S.C. § 9543). While participation in this survey is voluntary, your cooperation is critical to make the results of this survey comprehensive, accurate, and timely. Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. § 9573).

This survey is designed to be completed by the person in the district most knowledgeable about career and technical education (CTE) programs for high school students.

Name of person completing this form:

Title/position:

Telephone number: E-mail:

Best days and times to reach you (in case of questions):

THANK YOU. PLEASE KEEP A COPY OF THIS SURVEY FOR YOUR RECORDS.

PLEASE RETURN COMPLETED FORM TO: IF YOU HAVE ANY QUESTIONS OR COMMENTS, CONTACT:

Mail: Cindy Gray (6197.03.01.02) Cindy Gray at Westat

Westat 800-937-8281, ext. 4336 or 301-251-4336

1600 Research Boulevard E-mail: cindygray@westat.com

Rockville, Maryland 20850-3129

Fax: 800-254-0984

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 1850–0733. The time required to complete this information collection is estimated to average 30 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate, suggestions for improving this form, or any comments or concerns regarding the status of your individual submission of this form, please write directly to: Quick Response Information System (QRIS), National Center for Education Statistics (NCES), PCP, 550 12^th Street, SW, 4^th floor, Washington, DC 20202.

FRSS Form No. 108, 01/2017

Instructions and Definitions

1. Please use the following definition for completing this survey:

Career and technical education (CTE) program: a sequence of courses at the high school level that provides students with the academic and technical knowledge and skills needed to prepare for further education and careers in current or emerging professions.

2. For this survey, include all CTE programs that your district offers to high school students, including programs provided by your district or by other entities (such as an area/regional CTE center, a consortium of districts, or a community or technical college).

3. Please report only for CTE programs offered to high school students.

4. Please report for the 2016–17 school year and the summer of 2016.

For this survey, include all CTE programs that your district offers to high school students, including programs provided by your district or by other entities (such as an area/regional CTE center, a consortium of districts, or a community or technical college).

1. Does your district offer CTE programs to students at the high school level? (See instruction box above.)

   Yes... (Continue with question 2.)

   No... (Skip to question 16.)

2. Which of the following entities provide the CTE programs that your district offers to your high school students? (Indicate yes or no for each item.)

Yes No

1. An area/regional CTE center or a group/consortium of school districts

2. Your district individually (not as part of a consortium)

3. 2-year community or technical college(s)

4. 4-year college(s) or universities

5. Other (specify)

3. At which of the following locations does your district offer CTE programs to high school students? (Indicate yes or no for each item.)

Yes No

1. At some or all of your district’s regular (comprehensive) high schools

2. At another district’s regular (comprehensive) high school

3. At a CTE-focused high school that students attend full time

4. At a CTE center that students attend part time (for example, students spend half the day at the CTE center and half at the regular high school)

5. At a 2-year community or technical college, or a 4-year college or university

6. Other (specify)

4. How many CTE programs does your district offer to high school students? (Include all CTE programs that your district offers to high school students, including programs provided by your district or by other entities.) ____________

5. About how many of the CTE programs offered by your district to high school students are structured as career pathways that align with related postsecondary programs? (Check one box.)

   None……..

   Few…….

   Some…….

   Most…….

   All…….

6. Does your district offer any CTE courses in which students may earn high school credits in math, science, English/language arts, or social studies?

   Yes

   No

7. Does your district offer any CTE courses for which students can earn both high school and postsecondary credits for the same course?

   Yes

   No

8. Does your district offer any CTE courses online (include courses in a blended/hybrid format)?

   Yes

   No

9. Which of the following are included in any of the CTE programs offered by your district to high school students? (Indicate yes or no for each item.)

Yes No

1. Student-run enterprises or services (for example, school store or restaurant, cosmetology services, automotive or construction services, child development facility)

2. Mentoring by local employers

3. On-the-job training, internships, practicums, clinical experiences, or cooperative education (co-op)

4. Apprenticeships or pre-apprenticeship programs (such as youth apprenticeships)

5. Other work-based learning (specify)

10. About how many of the CTE programs offered by your district to high school students require work-based learning activities (such as those listed in Question 9) for completion of the program? (Check one box.)

    None……..

    Few…….

    Some…….

    Most…….

    All…….

11. To what extent are employers involved in the following ways with the CTE programs offered by your district to high school students? (Check one box on each line.)

    Type of involvement	Not at all	Small extent	Moderate extent	Large extent	Very large extent
    a. Provide work-based learning opportunities
    b. Serve on your district’s CTE advisory council
    c. Advise about which occupations are in demand
    d. Provide advice on CTE programs to add or eliminate
    e. Review CTE program curriculum
    f. Provide guidance on industry standards
    g. Provide guidance about equipment or facilities
    h. Donate equipment
    i. Host student field trips
    j. Serve as guest speakers to CTE students
    k. Provide guidance for student CTE projects
    l. Judge student CTE competitions
    m. Provide training opportunities for CTE teachers
    n. Other (specify)

12. How much of a barrier to your district is each of the following in offering CTE programs to high school students? (Check one box on each line.)

    Barrier to district in offering CTE	Not a barrier	Small barrier	Moderate barrier	Large barrier	Very large barrier
    a. Lack of funding or high cost of programs (for example, cost of infrastructure or equipment)
    b. Facilities or space limitations
    c. Finding or keeping teachers for in-demand industries and occupations
    d. Limited availability of professional development in technical fields
    e. Difficulty keeping CTE teachers’ technical skills up to date
    f. CTE teachers who move into teaching from other occupations have difficulty obtaining a regular or standard state teaching certificate
    g. Difficulty developing partnerships with employers for work-based learning
    h. Other (specify)

13. How much of a barrier is each of the following to student participation in the CTE programs offered by your district to high school students? (Check one box on each line.)

Barrier to student participation in CTE	Not a barrier	Small barrier	Moderate barrier	Large barrier	Very large barrier
a. Lack of time in students’ schedules for CTE courses
b. Students’ or parents’ negative perceptions of CTE
c. Transportation to CTE programs outside of the high school campus
d. Transportation for work-based learning
e. Students’ costs for supplies, uniforms, or materials
f. Students’ difficulty finding work-based learning opportunities
g. Lack of student support services for special populations
h. Other (specify)

Questions 14 and 15 ask about adding or phasing out CTE programs. Please answer these questions about CTE programs for which your district has a role in making these decisions.

Check here and skip to question 16 if your district does not have a decision-making role in adding or phasing out CTE programs.

14. To what extent does each of the following factors influence your district’s decision on whether to add a new CTE program for high school students? (Check one box on each line.)

    Factor	Not at all	Small extent	Moderate extent	Large extent	Very large extent
    a. Student interest
    b. Facilities/space considerations (for example, whether appropriate space is available)
    c. Costs for new program
    d. Availability of qualified teachers
    e. Information on which industries and occupations are in demand
    f. Employer (business/industry) recommendations
    g. Postsecondary institution recommendations
    h. Recommendations from your state department of education
    i. Career pathways from the high school to the postsecondary level (for example, to structure new pathways or better align existing pathways)
    j. Other (specify)

15. To what extent does each of the following factors influence your district’s decision on whether to phase out a CTE program for high school students? (Check one box on each line.)

    Factor	Not at all	Small extent	Moderate extent	Large extent	Very large extent
    a. Enrollment or student interest
    b. Facilities/space considerations (for example, facilities are outdated, space is needed for other purposes)
    c. Cost of program
    d. Availability of qualified teachers (for example, a teacher leaves and is difficult to replace)
    e. Information on which industries and occupations are in demand
    f. Employer (business/industry) recommendations
    g. Postsecondary institution recommendations
    h. Recommendations from your state department of education
    i. Career pathways from the high school to the postsecondary level (for example, if a program does not align with a career pathway)
    j. Other (specify)

16. Some states have CTE school districts that provide only CTE programs. Students have the option of enrolling in the CTE district instead of enrolling in their home district. Do high school students within your district’s enrollment area have the option of enrolling in a separate CTE district instead of enrolling in your district?

Yes……..

No…….

Not applicable, this district is a CTE district…….

Thank you. Please keep a copy for your records.