Appendix D - Results

Fast Response Survey System (FRSS) 109: Teachers’ Use of Technology for School and Homework Assignments

Appendix D

Survey Development Results

– Feasibility Calls and Pretesting

(OMB# 1850-0803 v.202 and v.226)

OMB # 1850-0857 v.5

National Center for Education Statistics (NCES)

U.S. Department of Education

Institute of Education Sciences

Washington, DC

April 2018

revised May 2018

Date:	October 4, 2017
To:	John Ralph Chris Chapman
From:	Laurie Lewis Cindy Gray
Subject:	FRSS 109 Teachers’ Use of Technology for Teaching: Feasibility Calls, Round 1

This memo describes findings from the first round of feasibility calls for the FRSS 109 survey on teachers’ use of technology for teaching. These calls were conducted in August and September 2017 with 10 public school teachers located in 10 states: Alabama, Arkansas, California, Connecticut, Idaho, Kansas, Kentucky, Maine, South Carolina, and West Virginia. The schools where these teachers taught were located in urban, suburban, and town/rural community types, with small, medium, and large enrollments sizes. Five of the teachers taught in elementary schools, two taught in middle/junior high schools, one taught in a combined junior/senior high school, and two taught in high schools. The secondary-level (middle and high school) teachers taught math, science, English, critical reading and writing, and social studies. Below is a summary of the information collected from the respondents, presented by topic. Following the summary are our recommendations for the next round of feasibility calls.

District- or School-Provided Technology Available to Students

Students had substantial access to computers and the Internet while they were at school, and in some cases, students could take the district-provided computers home with them. District- or school-provided technology in these 10 schools broke out as follows.

• The two high schools are in districts that provide students with laptops that students keep with them all day at school and take home with them after school; in one of the districts, the laptops are issued to all students in grades K–12, and in the other district, they are issued to all students in grades 3–12.

• One middle school provides laptops to all students, and the laptops travel with the students throughout the day (except lunchtime), but do not go home with the students (they stay with the students’ homeroom teacher). In addition, each classroom has three extra laptops available for student use, and there are computers in the library media center.

• Three of the schools (two elementary and the combined junior/senior high) are 1:1 or almost 1:1 with laptops, but the laptops are assigned to classrooms rather than to students. At one of the elementary schools, the 1:1 laptops in classrooms is only for grades 5 and 6. At the elementary school that is almost 1:1 with laptops, every classroom has 15 laptops that stay in the classroom, and the teacher can request computer carts if all students need to be online at the same time. None of these schools allow students to borrow computers to take home.

• In four schools (three elementary and a junior high), there are computers in the school in classrooms, computer labs, and/or on carts, but the school does not have a computer for every student. None of these schools allow students to borrow computers to take home.

None of the schools reported having other devices (e.g., tablets, smartphones or other hand-held devices) or mobile hotspots available for students to use or take home.

Teacher Knowledge about Student Access Outside of School to Technology and the Internet for Doing School Assignments

Most teachers believed that they were fairly or somewhat knowledgeable, with a few teachers saying they were very knowledgeable, about their students’ access to technology and the Internet outside of school for doing school assignments. Teachers who considered themselves very knowledgeable sought this information from students and/or parents at the beginning of the school year through a technology survey or by directly asking students and/or parents about access at home. Other teachers developed a sense of what their students had as they worked with them (and one teacher said it was such a small community that everyone knew everything about everyone else). Some teachers knew that student access to the Internet was limited due to living in remote rural or mountainous areas where even cell phone access was problematic; teachers experienced this themselves, and sometimes schools had to install special equipment such as antennas for Internet access.

Student Access and Availability Outside of School and Outside of Class Time to Technology and the Internet for Doing School Assignments

The two high schools were in districts that provided laptops that students took home with them, so access and availability of appropriate devices was not an issue at these schools. Most of the students at these schools had access to the Internet at home, although a few students lived in remote rural areas where Internet was not as available. These high school students were also able to access the Internet at places like the Barnes and Noble bookstore and Starbucks, and at the homes of friends and relatives. Teachers also emphasized saving documents and other work onto the computers at school so that students could work offline if needed. Thus, access and availability was not an issue for students at these particular high schools.

At the elementary and middle/junior high school level, some students had access to computers at home, and a few had tablets, but for many students, the only device they had was a cell phone. For the middle/junior high school students, this was often a cell phone of their own, but for the elementary school students, it was usually a parent’s cell phone. Teachers noted that for most students whose only access to the Internet was through a cell phone, the access depended on a data plan; the expense of such plans restricted access for some students. One teacher noted that availability to use devices that a family had was highly dependent on a student’s age, with students in the fourth grade and up tending to have better availability, while younger students more often had to compete with older siblings and parents to use the device.

For students at the schools in this set of calls, the public library was not a useful resource for access to computers or the Internet. For most of the students, no matter where they live, getting to the library is problematic. This is compounded in small and rural communities, where the library hours are too limited to be available when students are out of school, and the library is not close to where students live. Most of the students ride the school bus, which leaves as soon as school is dismissed. The only other locations mentioned by teachers where their students could access computers and the Internet was at the homes of relatives, neighbors, or friends.

Schools provided access to computers and the Internet for students outside of class time by making computers in classrooms, computer labs, and library media centers available to students before school (school buses typically arrive early), at recess, and at lunch. In some schools, these resources are also available after school, but most students cannot use them after school because they ride the school bus home. Two of the schools also had an extra academic period (independent study or academic support) built into the school day when these resources were available for students to use for assignments from other classes.

Barriers and Challenges that Students Face in Using Technology and the Internet for Class Assignments Outside of School

Except for the high schools in these calls, which provided laptops for students to take home, consistent access and availability to an appropriate device and broadband Internet were the barriers and challenges mentioned most frequently. For many of the students, especially the younger students, cell phones were the only devices available, and for many students, the phone belonged to a parent. Teachers commented that phones were good for communication between teachers and families, for accessing information from the school such as online gradebooks and class web pages, and for looking up and reading information (for the older students), but were not good for most types of school work since they could not be used to create documents or do more complex assignments such as online problem sets for math and science. Cell phones were also often running off of data plans, which frequently limited how much students could use them.

Access to computers and to the Internet was affected by family financial resources in a number of schools. Teachers also mentioned that for some families, technology usage was not a part of their lives, and they did not bring technology (other than cell phones) into their homes. This was influenced by finances, of course, but also by parental education and occupation (including whether parents used computers at work), and by overall parental experience with technology. One teacher mentioned that the school served many migrant students, whose families were transient and without technology and the Internet at home. Another teacher mentioned that the school served a large refugee population, many of whom did not have experience with any electronic devices.

Families in rural areas, and especially in mountainous rural areas, often did not have good Internet access; some had slow dial-up access, and some did not have any access at all. One teacher mentioned that in their rural community, the phone company was turning people down for Internet access due to lack of bandwidth. In many of these rural communities, cell phone reception is also poor, which further limits student usage for school work. A couple of teachers from schools that were not in rural areas mentioned that families seemed to have Internet access for things such as Netflix, but did not have appropriate devices at home for students to use.

Language-related barriers were more often an issue for using technology to communicate with parents than for student use of the technology. Some teachers commented that many of their EL students also spoke some English, and tended to learn to use the technologies quickly. Some teachers mentioned that technology was a useful tool to help EL students since there were various applications and tools available to help with translation and skill-building. One teacher who taught chemistry and physics mentioned that technical terms did not translate well in the apps, but this was a specialized use of the technology.

When asked about other challenges, teachers had a variety of observations, as discussed below.

• One teacher did not agree with the public perception that students who grew up with technology can use technology for academic work. She thought this was not true because social media usage was very different from academic usage. Sometimes students brought habits regarding spelling and sentence structure into the classroom, and had to be taught explicitly not to use this “short-cut” approach in assignments or for communicating with teachers.

• Another teacher commented that about half of her students’ parents were not knowledgeable about the academic uses of technology, and another commented that a challenge they faced was making sure the parents knew how to use the technologies that the students were using and those for communicating with the school.

• One teacher commented that younger students have problems with accessing and navigating the website for homework, and that the student access page at home may look different from the teacher access page at school.

• One teacher was concerned that students do not have the opportunity at home to practice keyboarding skills since they do not have computers at home, only phones. This was a concern because their standardized assessments are all done online.

• A middle school teacher stated that the biggest problem she faces is that parents punish their children by taking away their phones, but then students cannot do their work at home (she sets up vocabulary and practice tests for students to do on their phones).

• The high school teachers, whose student keep the laptops with them all day and take them home, mentioned things like time management and discipline (it was so easy to watch a movie when they should be doing a homework assignment), and students using their devices in ways they are not supposed to, like live streaming videos and using social media when they are supposed to be working on papers.

How Student Access to Technology and the Internet Outside of School Influences Assignments Given, and the Kinds of Technology-based Assignments that are Given

A few teachers said student access to technology did not influence the assignments given, but there were different reasons for this. Some did not give much homework of any type because they taught younger elementary school students, and some teachers said the subject taught was not conducive to a technology-based assignment. One teacher commented that the discipline a teacher teaches can heavily influence the extent to which technology-based assignments can be used.

Most teachers said that student access to technology did influence the assignments given. Some teachers said that they do not give much work that requires a computer or Internet access at home because they know that not all of their students have appropriate devices and Internet access. One teacher said he does not use the eResources that are part of the curriculum, and which he would like to use, because he knows that not all of his students will be able to access them. One teacher said that it affected how she structured assignments. Because her students are primarily accessing the Internet on cell phones, she sets up vocabulary and practice tests with links that students can click on using their cell phones, rather than using the regular web interface, which is not cell phone-friendly.

The high school teachers, whose students all have district-provided laptops that they take home, reported that it affected assignments in a different way. The English teacher said that having the laptops has opened up their resources and allowed them to move faster. Students are assigned to do independent research and also class projects with presentations that are done on the computer. The high school physics teacher said the laptops have allowed him to assign online problem sets for students.

Teachers below the high school level said that when assignments that require Internet access are given, students are given time to do them at school. Teachers help younger students retrieve and print information that they need so they can complete the assignment at home without using the Internet. At the high school level, teachers stress having students save documents onto their laptops while they are at school so that the documents are available offline for students to use. The high school physics teacher sets up the online problem sets into Goggle Forms rather than to a website to make access easier. Teachers also work hard to ensure that students who need it have access to computers and the Internet at school outside of class time so that some students are not at a disadvantage. Teachers also stressed that any homework assignments they give are set up so that all students can complete them regardless of computer and Internet access at home. They do this by providing print resources and allowing students to complete assignments on paper, in addition to making school computer resources available to students outside of class time.

Policies about Teachers’ Use of Technology and the Internet for Homework Assignments

Teachers were not aware of written policies that addressed this issue. One teacher said that the school principal explained to them that alternative assignments must be available for students without Internet access so that no students are put at a disadvantage. Another teacher said that teachers are always given professional development presentations and discussions at the beginning of the year about how they should use technology.

Technology-based Instructional Management and Communication Tools and Practices

All teachers used some technology-based instructional management tools. Some states mandated the use of a particular management tool for functions such as attendance reporting and online gradebooks. Many teachers used multiple tools that provided different kinds of functions. Commonly mentioned uses of instructional management tools included attendance, posting assignments, online drop boxes for turning in assignments, and online gradebooks. These management tools had parent portals that allowed them to view functions such as attendance, assignments, and grades. Tools used by middle and high schools also included a student portal. Some teachers also mentioned being able to use the instructional management tool to create a class website, or to communicate with parents and students. Many of these systems allowed teachers to send and receive emails and text messages from individual teachers and students, and to send broadcast messages to all parents or students with general classroom announcements. All teachers mentioned communicating with parents by email, and often by text, without necessarily tying those communications to the instructional management tools. A few teachers used Facebook to communicate with some parents, primarily because the parents seemed most comfortable with this type of communication.

When asked how much parents used these technologies to obtain class or school information or to communicate with them, teachers responded that parents did not use them as much as the teachers would like. When asked if some parents had problems using the technologies to obtain information or communicate with them, teachers responded that some might have problems, but there was no way for teachers to know whether parents were not using the technology tools because of problems using the technology or for other reasons, such as lack of time or interest. Teachers also commented that students used the technology tools more often than parents did.

Other Teacher Comments

At the end of the calls, teachers were asked if they had any other comments that they would like to make. One teacher commented that it was his observation that many teachers do not understand or use technology much. Another commented that the research on using technology for instruction assumes that students have broadband Internet access, but students at the school did not have that type of access. An elementary school teacher commented that many students know how to use devices for playing and for getting some information, but not for learning. This teacher also commented that students need lessons on digital citizenship. Another teacher thought there were two important issues regarding technology access for students outside of school, a financial divide and a cultural divide (homes that do not regularly access and use technology). Another teacher commented that the very rural area in which the school was located drives many of their issues – access, poverty, and a growing drug problem.

Important Factors for Technology Access and Use Outside of School

During these calls, we identified a number of important factors at these schools that influenced technology access and use outside of school. Some of these are factors that are available on the CCD school sampling frame, including the instructional level of the school, community type (urbanicity), and poverty (using free and reduced-price lunch as a proxy). Other factors included the grade level of the self-contained class for the elementary school level, and the subject taught for the departmentalized instruction at the middle and high school level. Other characteristics of the population served by the school also affected access and use outside of school, including having sizeable populations of migrant students, refugees, and English learners.

In these calls, the factor that most influenced technology access and use outside of school was whether the district provided laptops that students could take home with them. When teachers knew that all students had appropriate devices and Internet access (mostly at home, but readily accessible to students at other locations if needed), they were able to take advantage of resources and use instructional practices that relied on computers and Internet access.

Recommendations

The first round of feasibility calls provided information on many topics related to technology access and use outside of school, summarized above. The next round of feasibility calls needs to narrow the topics discussed to move us closer to a questionnaire that both collects the information needed by NCES and meets the requirements of FRSS (three pages of questions with a respondent burden of about 30 minutes). Below we make recommendations about topics for the next round of feasibility calls.

Topics Recommended for Round 2 Feasibility Calls

• Technology provided by districts/schools

  • Whether the district/school provides laptops to all of the teachers’ students, and whether the students can take the laptops home

  • Whether the school allows students to borrow laptops to take home on a short-term basis

  • Whether the school provides students with access to computers outside of class time, such as before/after school or special periods during the school day

• Teacher knowledge of student devices and Internet access

  • How knowledgeable teachers think they are about their students’ access to computers and the Internet for doing school assignments outside of school

    • How teachers find out this information

  • The proportion of students estimated to have access to a computer at home

    • How available the computer is to the student to use

    • Whether the computer has reliable Internet access from home

  • The proportion of students that have access to a device other than a laptop at home

    • What that device is (e.g., tablet or phone)

    • How available it is to the student to use (whether the phone belongs to the student or a parent)

    • The type of Internet access the device has from home (whether it is running off a data plan)

  • What other locations outside of school students can use for computer and Internet access, and how useful these locations are for completing school assignments

• Influence of access to technology on homework assignments

  • The extent and ways that students’ access to technology (or the teacher’s perception of access) influences homework assignments

  • The types of assistance or accommodations made for students with limited access

• Barriers and challenges that students face in using technology and the Internet for class assignments outside of school

Topics Not Recommended for Round 2 Feasibility Calls

• Policies about teachers’ use of technology and the Internet for homework assignments

• Technology-based instructional management practices

• Technology-based communications with students and parents

Date:	February 16, 2018
To:	John Ralph Chris Chapman
From:	Laurie Lewis Cindy Gray
Subject:	FRSS 109 Teachers’ Use of Technology for School and Homework Assignments: Results of Feasibility Calls

This memo describes the feasibility calls for the FRSS 109 survey on teachers’ use of technology for school and homework assignments. These calls were conducted in fall 2017 and winter 2018 with teachers at the elementary, middle school, and high school level. Below is a brief summary of the calls and description of the resulting draft questionnaire (attached), presented by topic area of the questionnaire.

District- or School-Provided Computers Available to Students (Questions 1 through 5)

Question 1 asks whether the teacher’s students have a district- or school-provided computer that the student takes home. Because the Congressional request was for information about student access to technology at home, the key part of this question is whether the student takes the computer home. Some districts and schools have 1:1 programs where each student is provided a computer, but the program does not allow students to take the computers home. Such programs do not provide students with district- or school-provided technology access outside of the school.

If a teacher answers yes to question 1, question 2 asks whether there are some students who are not able to take their district- or school-provided computer home. Teachers reported that some students were not able to take their computers home because parents did not give permission for the student to bring the computer home, the student had previously damaged or lost a computer, or the computer would not be safe at home or during the trip home.

If a teacher answers no to question 1, question 3 asks whether the school allows students to borrow computers to take home on a short-term basis. Loaner programs provide at least some student access to technology at home using district- or school-provided computers. If a teacher answers no to question 1, question 4 asks whether students can access school computers outside of class time. Teachers reported that schools tried to provide as much access as possible to school computers so that students who did not have computers at home would be able to use school resources to complete technology-based assignments.

Question 5 asks whether the school has an additional academic period for all students during the school day when students can use computers and the Internet to work on school assignments from other classes. Teachers reported that this was another way in which schools tried to provide as much access as possible to school computers and the Internet so that students who did not have it at home would be able to use school resources to complete technology-based assignments.

Teacher Knowledge about Student Access Outside of School to Computers and the Internet for Doing School Assignments (Questions 6 through 8)

Question 6 asks how knowledgeable teachers think they are about their students’ access to computers and the Internet for doing school assignments at home. Knowledge about computers and Internet access differed, with teachers often reporting they were more knowledgeable about computers than Internet access. Question 7 asks teachers how they find out information about their students’ access to computers and the Internet at home. The response items are based on teacher reports across three sets of feasibility calls.

Question 8 asks teachers about the extent to which their students use various non-home locations for computer and/or Internet access to work on school assignments. The response items are based on teacher reports across three sets of feasibility calls. In addition, high school teachers sometimes reported providing students with information about places in the community where students could access these resources if they needed them.

Access and Availability of Computers at Home (Questions 9 through 11)

Question 9 asks teachers to estimate the percentage of their students with access to a computer at home. Responses from teachers during the feasibility calls indicated substantial variability in the reported access to computers at home. Even teachers whose students had district- or school-provided computers that they could take home (Q1=yes) often reported that less than 100 percent of their students had access to a computer at home because some students were not able to take their computers home. Question 10 asks teachers to estimate how available those home computers are for students to use for school assignments. Discussions with teachers indicated that home computers were often shared computers, and that there was variability in how available those home computers were for students to use for school assignments. Teachers extensively used the text after the availability labels, which was developed over the course of the feasibility calls, to guide their responses. Question 11 asks teachers to estimate how likely it is that the home computer has reliable Internet access from home. Discussions with teachers indicated variability in reliable Internet access for computers, particularly for students who lived in remote rural or mountainous areas, or whose families could not afford Internet service for a computer (although they may have it for a phone).

Access, Usefulness, and Availability of a Smartphone at Home (Questions 12 through 15)

Question 12 asks teachers to estimate the percentage of their students with access to a smartphone at home. Early discussions with teachers indicated that the two devices to which students would be likely to have access are computers and smartphones. Responses from teachers during the feasibility calls indicated variability in the reported access to a smartphone at home, particularly for younger students. Question 13 asks teachers how useful those smartphones are for completing the assignments that they give their students. Discussions with teachers indicated that some teachers designed some of their assignments in ways that could be easily done on a smartphone (e.g., vocabulary review or practice quizzes), while other teachers indicated that smartphones would not be useful for the types of assignments they give to students. Question 14 asks teachers to estimate how available those smartphones are for students to use for school assignments. Discussions with teachers indicated that smartphone availability varied, and that particularly for younger students, the smartphone may belong to a parent. Question 15 asks teachers to estimate how likely it is that the smartphone has reliable Internet access from home.

Technology and Homework (Questions 16 through 21)

Question 16 asks teachers how much influence their students’ access to technology and the Internet outside of school has on the homework assignments they give them. Technology is defined to include devices such as computers and smartphones, software such as computer programs and digital apps, and the Internet. Some teachers mentioned that students’ home computers or devices did not have the same software or digital apps available that were used on the school devices, such as tablets. This influenced the homework assignments given by these teachers.

Question 17 asks how often teachers assign technology-based homework to their students, and question 18 asks how often teachers assign homework of any type to their students. The question about assigning homework of any type (Q18) is asked to provide context for assigning technology-based homework. Both of these questions showed considerable variability during the feasibility calls, ranging from never to often for both categories of homework. Early feasibility calls identified age of the students and subject matter taught as influences on these questions. During early feasibility calls, questions 17 and 18 were in the opposite order, with the question about all types of homework first. We found that some (but not all) teachers were then responding to technology-based homework as a subset of all homework. For example, a teacher might say that they rarely assigned homework, and then say that they sometimes assigned technology-based homework. Discussion indicated that these teachers meant that, on the rare occasions when they did assign homework, it was sometimes technology-based homework. However, this is not the base that we would like teachers to use for technology-based homework, and we made two changes to address this situation. The main change was to ask about technology-based homework first. The second change was to the wording of question 18, to ask about “any type” of homework, and to add the parenthetical instruction to include both technology-based and non-technology-based homework.

Question 19 asks teachers about the extent to which their students have difficulty completing technology-based homework because they are not familiar with how to use technology. Question 20 asks teachers about the extent to which the parents of their students have difficulty helping the students with technology-based homework because they are not familiar with how to use technology. Feasibility calls showed variability on these items, particularly the item about parental assistance. Teachers from schools with lower parental education levels, a large number of migrant students, and sizeable refugee populations especially indicated that familiarity with technology had a large effect.

Question 21 asks teachers about the extent to which they provide various types of assistance for doing technology-based homework to their students who have limited access to technology and the Internet outside of school. The types of assistance are those reported by teachers during several rounds of feasibility calls.

Student Preparedness for Online or Computerized Assessments

Question 22 asks teachers how prepared their students are to use the technology required for online or computerized assessments given by their district or school. This question was added during the feasibility calls because some teachers mentioned that their students had trouble with these types of assessments, particularly those that included extended written responses, because they did not have much experience using a keyboard and mouse, although they might have experience using touch screens.

Teacher Instructional Setting

Questions 23 through 25 ask about the teacher’s instructional setting – self-contained classroom or departmentalized instruction, subjects taught (for departmentalized instruction), and grades currently taught at the school. These variables will be used for analysis.

Date:	May 10, 2018
To:	John Ralph Chris Chapman
From:	Laurie Lewis Cindy Gray
Subject:	FRSS 109 Teachers’ Use of Technology for School and Homework Assignments: Results of Pretest

This memo describes the pretest for the FRSS 109 survey on teachers’ use of technology for school and homework assignments. The pretest was conducted in April and May 2018 with teachers at the elementary, middle school, and high school level. Below is a brief summary of the pretest and a description of the minor changes made to the questionnaire as a result of the pretest.

Cover Page: The text above the bullets was revised slightly to include the statement that the survey is being conducted in response to a request from Congress. This reference was more meaningful to teachers than the reference to the Every Student Succeeds Act. Also, the dates in the first bullet were changed to the 2018–19 school year (the 2017–18 school year was used for the pretest). In addition, the burden estimate in the text at the bottom of the page was changed from 20 minutes to 15 minutes, which was based on the average time reported by pretest participants.

Question 5: The last part of the question stem was modified to include “homework or” and now reads “…when students can use computers and the Internet to work on homework or assignments from other classes.” This change was made to make the question more understandable to teachers of self-contained classrooms for whom the phrase “assignments from other classes” was not clear.

Questions 10 and 13: The phrase “percentage of your students” was changed to “percentage of the students you teach” to make it clearer that a teacher should respond for the students he or she teaches rather than all the students at the school.

Question 21, item a: The phrase “work on the homework assignments offline” was changed to “work on the homework assignments on a computer without access to the Internet.” Some of the teachers were confused by the term “offline.”

Question 22: The stem of the question was modified to include state assessments, and now reads “…assessments given by your state, district, or school?”