International Computer and Information Literacy Study (ICILS 2018) Pretest - Updated

International Computer and Information Literacy Study (ICILS 2018) Pretest

OMB# 1850-0803 v.221

(revised v.207)

Volume 2

Submitted by:

National Center for Education Statistics (NCES)

Institute of Education Sciences (IES)

U.S. Department of Education

Washington, DC

August 2017

revised November 2017

PAPERWORK BURDEN STATEMENT

The Paperwork Reduction Act and the NCES confidentiality statement are indicated below. Appropriate sections of this information are included in the consent forms and letters. The statements will be included in the materials used in the study.

Paperwork Burden Statement, OMB Information

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0803. The time required to complete this information collection is estimated to be an average of 175 minutes, including the time to review instructions and participate in the information collection.

If you have any comments concerning the accuracy of the time estimate, suggestions for improving this collection, or any other concerns, please write to: International Computer and Information Literacy Study (ICILS), National Center for Education Statistics, Potomac Center Plaza, 550 12th St., SW, 4th floor, Washington, DC 20202.

This is a project of the National Center for Education Statistics (NCES), part of the Institute of Education Sciences, within the U.S. Department of Education.

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

OMB No. 1850-0803 Approval Expires 07/31/2019

INTRODUCTION TO THE ASSESSMENT

As documented in Volume 1, there will be two pretesting sessions. The administrators will provide additional instructions to the students:

For this part of the study, we would like you to take your time exploring the system and interacting with the questions and system tools in different ways. While we would like you to provide answers to questions, we would like you to focus mainly on using the system tools and responding to the questions. Any response is acceptable for the questions as we really want you to experiment with how you can respond to questions and move from question to question. Have a little fun with things and explore the system!

Please take the assessment in this way until your time is up.

STUDENT DEBRIEFING QUESTIONS

After all students have finished the assessment in each session, field staff will conduct a debriefing activity with the students. Field staff will ask the debriefing questions out loud to the group of students. The questions may be paraphrased, and additional follow-up questions, such as those indicated in the bulleted lists below, may be asked. Students will respond by raising their hands and by calling out responses, as appropriate. Field staff will use their discretion about whether some or all of the questions will be asked, depending on the amount of prompting the students require, and on the amount of time remaining. The debriefing session should not exceed 10 minutes.

Field staff will write down the students’ responses to the questions.

1. What did you like the most about the assessment?

[Follow-up questions may include:]

• Can you explain that more?

• What did you like the least about the assessment?

• What was missing from the assessment?

• If you had to take a test, would you rather take it on a tablet, or on paper? Why?

• What do you like most about taking a test on a tablet? What do you like the least about taking a test on a tablet?

2. Did you see anything you thought was strange while taking the assessment?

[Follow-up questions may include:]

• Can you explain that more?

• Did questions or images seem slow to load at all to you?

• Was the system slow to move from one page/question to the next at all?

• Did you notice any freezes of your tablet or any error messages?

• Were you able to make these errors occur when you wanted to? How?

3. Did you find the system easy to use and/or easy to learn to use?

[Follow-up questions may include:]

• How did you like the tutorial?

• Did you use the Help screens? Were the Help screens useful?

• What did you think of the tools available? Did you use them?

• Which tool or tools did you like the most? Why?

• Did you have any difficulty using any of the tools? What did you find difficult?

• How did you find the mouse and keypad work with the assessment?

• Did you have any difficulty responding to questions using the available controls? If so, what made it difficult?

4. Is there anything you think would make the assessment better? If so, what?

5. Is there anything else you want to tell us about taking assessments on tablets?

SURVEY QUESTIONNAIRES

Students will take a set of survey questionnaire items during the ICILS pretest assessment session. For both sessions we will use the ICILS 2018 items that are finalized for the main study. Included here is the student questionnaire that will be used. In case any changes are made to the U.S. adaptations while the questionnaire is in review by the IEA, an update to this request will be submitted to OMB prior to these items being pretested.

IEA International Computer and Information Literacy Study

Student Questionnaire for the 2018 U.S. Main Study (Draft)

Confidential to ICILS 2018

Do not cite or quote

The National Center for Education Statistics (NCES), within the U.S. Department of Education, conducts ICILS in the United States as authorized by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0929. The time required to complete this information collection is estimated to average 30 minutes per student, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments or concerns regarding the accuracy of the time estimate(s), suggestions for improving the form, or questions about the status of your individual submission of this form, write directly to: International Computer and Information Literacy Study (ICILS), National Center for Education Statistics, Potomac Center Plaza (PCP), 550 12th St., SW, 4th floor, Washington, DC 20202.

OMB No. 1850-0929, Approval Expires 01/31/2021.

INTRODUCTION TO THE QUESTIONNAIRE FOR STUDENTS

This questionnaire is about Information and Communication Technology (ICT). In this questionnaire, ICT can refer to:

• desktop computers

• notebook or laptop computers

• netbook computers

• tablet devices

• smartphones, except when being used for talk and text.

In this questionnaire, you will find questions about:

• You, your home, and your family

• Where and how often you use ICT

• What you use ICT for

• Your views about the use of ICT.

Please read each question carefully and answer as accurately as you can. In this questionnaire, you will mostly answer by clicking on a button. You can change your responses at any time until you have clicked on ‘I’ve finished’ at the end of the questionnaire.

There are also a few questions where you will need to write a short response.

In this questionnaire, there are no right or wrong answers. Your answers should be the ones that apply to you.

You may ask for help if you do not understand something or if you are not sure how to answer a question.

The National Center for Education Statistics (NCES) is authorized to collect information from the questionnaire by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). You do not have to provide the information requested. However, the information you provide will help the U.S. Department of Education’s ongoing efforts to understand better how the educational system in the United States compares to that in other countries. There are no penalties should you choose not to participate in this study. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Your responses will be combined with those from other participants to produce summary statistics and reports.

Thank you.

About You

Q1	When were you born? (Scroll down to answer both parts to this question)
	January - December (Month) 
	1997 – 2008 (Year) 

Q2	Are you a girl or a boy?
	Girl	Boy
		

Q3	Are you Hispanic or Latino?
	(Please mark only one choice)
	Yes, I am Hispanic or Latino	
	No, I am not Hispanic or Latino	

Q4	Which of the following best describes you?
	(Please mark all choices that apply)
	White	
	Black or African American	
	Asian	
	American Indian or Alaska Native	
	Native Hawaiian or other Pacific Islander	

Q5	What is the highest level of education you expect to complete?
	(Please mark only one choice)
	Bachelor's degree (4-year college program) OR Master's degree or professional degree (MD, DDS, lawyer, minister) OR Doctorate (Ph.D. or EdD)	
	Associate's degree (2-year college program)	
	High school graduate	
	Some high school	
	Less than high school	

Your Home and Your Family

In this section you will be asked some questions about your family and your home.

Some of these questions will be about home and your parents or guardians who look after you — for example, step-parents or foster-parents. Select one parent or guardian as parent or guardian 1 and the other as parent or guardian 2.

If you share your time with more than one set of parents or guardians, please answer the following questions for those parents/guardians with whom you spend the most time. If you share your time only with one parent, please answer only the following questions for this parent.

Q6	In what country were you and your parents born?
	(Please mark only one choice in each column)
		You	Parent or guardian 1	Parent or guardian 2
	United States			
	Another country			

Q7	What language do you speak at home most of the time?
	(Please mark only one choice)
	English	
	Spanish	
	Another language	

Q8	Does your parent or guardian 1 work in a paid job?
	Yes		(Note: Student will be directed to Q10a and Q11a)
	No		(Note: Student will be directed to Q10b and Q11b)

Q9a	What is your parent or guardian 1’s main job? (for example, school teacher, cook, sales manager)
	(Please write in the job title)
Q10a	What does your parent or guardian 1 do in his/her main job? (for example, teaches high school students, helps prepare meals in a restaurant, manages a sales team)
	(Please use a sentence to describe the kind of work he/she does in that job)

Q9b	What was your parent or guardian 1’s last main job? (for example, school teacher, cook, sales manager)
	Please tell us his/her last main job. If he/she has never had a paid job, please write what he/she is currently doing.
	(Please write in the job title)
Q10b	What did your parent or guardian 1 do in his/her last main job? (for example, taught high school students, helped prepare meals in a restaurant, managed a sales team)
	(Please use a sentence to describe the kind of work he/she did in that job or what he/she is currently doing if he/she has never had a paid job)

Q11	What is the highest level of education completed by your parent or guardian 1?
	If you are not sure which box to choose, please ask the test administrator for help. (Please mark only one choice)
	Bachelor's degree (4-year college program) OR Master's degree or professional degree (MD, DDS, lawyer, minister) OR Doctorate (Ph.D., or EdD)	
	Associate's degree (2-year college program)	
	High school graduate	
	Some high school	
	Less than high school	

Q12	Does your parent or guardian 2 work in a paid job?
	Yes		(Note: Student will be directed to Q14a and Q15a)
	No		(Note: Student will be directed to Q14b and Q15b)
	N/A		(Note: Not yet approved by the IEA)

Q13a	What is your parent or guardian 2’s main job? (for example, school teacher, cook, sales manager)
	(Please write in the job title)
Q14a	What does your parent or guardian 2 do in his/her main job? (for example, teaches high school students, helps prepare meals in a restaurant, manages a sales team)
	(Please use a sentence to describe the kind of work he/she does in that job)

Q13b	What was your parent or guardian 2’s last main job? (for example, school teacher, cook, sales manager)
	Please tell us his/her last main job. If he/she has never had a paid job, please write what he/she is currently doing.
	(Please write in the job title)
Q14b	What did your parent or guardian 2 do in his/her last main job? (for example, taught high school students, helped prepare meals in a restaurant, managed a sales team)
	(Please use a sentence to describe the kind of work he/she did in that job or what he/she is currently doing if he/she has never had a paid job)

Q15	What is the highest level of education completed by your parent or guardian 2?
	If you are not sure which box to choose, please ask the test administrator for help. (Please mark only one choice)
	Bachelor's degree (4-year college program) OR Master's degree or professional degree (MD, DDS, lawyer, minister) OR Doctorate (Ph.D. or EdD)	
	Associate's degree (2-year college program)	
	High school graduate	
	Some high school	
	Less than high school	

Q16	About how many books are there in your home?
	Do not count magazines, newspapers, comic books or your schoolbooks.
	(Please mark only one choice)
	None or very few (0–10 books)	
	Enough to fill one shelf (11–25 books)	
	Enough to fill one bookcase (26–100 books)	
	Enough to fill two bookcases (101–200 books)	
	Enough to fill three or more bookcases (more than 200 books)	

Q17		How many of the following ICT devices are currently used in your home?
		(Please mark one choice in each row)
			None	One	Two	Three or more
a)	Desktop or laptop computers					
b)	Tablet devices or e-readers (e.g., iPad, Surface Pro, Kindle)					

Your Use of ICT

Q18	How long have you been using each of the following types of ICT devices?
	(Please mark one choice in each row)
		Never or less than one year	At least one year but less than three years	At least three years but less than five years	At least five years but less than seven years	Seven years or more
a)	Desktop or portable computers					
b)	Tablet devices or e-readers (e.g., iPad, Surface Pro, Kindle)					
c)	Smartphones except for using text and calling					

Q19	Who mainly taught you how to do the following activities?
	(Please mark one choice in each row)
		My teachers	My family	My friends	I taught myself	I have never learned this
a)	Communicate over the Internet					
b)	Create or edit digital documents					
c)	Create or edit digital presentations					
d)	Change settings on an ICT device					
e)	Find information on the Internet					
f)	Use programs and files in a computer network					

Q20	How often do you use ICT in these places?
	Please do not count the use of smartphones when making phone calls or writing text messages. (Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every day	Every day
a)	At school for school-related purposes					
b)	At school for other purposes					
c)	Outside of school for school-related purposes					
d)	Outside of school for other purposes					

Q21	How often do you use ICT for each of the following activities?
	(Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every day	Every day
a)	Write or edit documents					
b)	Use a spreadsheet to do calculations, store data or plot graphs e.g., using Microsoft Excel)					
c)	Create a simple “slideshow” presentation (e.g., using Microsoft PowerPoint)					
d)	Record or edit videos					
e)	Write computer programs, scripts or apps (e.g., using Scratch, Logo, VBA, Java)					
f)	Use drawing, painting or graphics software or apps					
g)	Produce or edit music					
h)	Build or edit a webpage					

Q22	How often do you use ICT to do each of the following communication activities?
	(Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every day	Every day
a)	Share news about current events on social media					
b)	Communicate with friends, family, or other people using instant messaging, voice or video chat (e.g., Skype, FaceTime, WhatsApp, Viber)					
c)	Send texts or instant messages to friends, family, or other people					
d)	Write posts and updates about what happens in your life on social media					
e)	Ask questions on forums or Q&A, question and answer websites					
f)	Answer other peoples’ questions on forums or Q&A, question and answer websites					
g)	Write posts for your own blog (e.g., WordPress, Tumblr, Blogger)					
h)	Post images or video in social networks or online communities (e.g., Facebook, Instagram or YouTube)					
i)	Watch videos or images that other people have posted online					
j)	Send or forward information about events or activities to other people					

Q23	How often do you use ICT to do each of the following leisure activities?
	(Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every day	Every day
a)	Search the Internet to find information about places to go or activities to do					
b)	Read reviews on the Internet of things you might want to buy					
c)	Read news stories on the Internet					
d)	Search for online information about things you are interested in					
e)	Use websites, forums, or online videos to find out how to do something					
f)	Play games					
g)	Listen to downloaded or streamed music					
h)	Watch downloaded or streamed TV shows or movies					

Using ICT for School

Q24	How often do you use ICT for the following school-related purposes?
	(Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every school day	Every school day
a)	Prepare reports or essays					
b)	Prepare presentations					
c)	Work online with other students					
d)	Complete worksheets or exercises					
e)	Organize your time and work					
f)	Take tests					
g)	Use software or applications to learn skills or a subject (e.g., mathematics tutoring software, language learning software)					
h)	Use the Internet to do research					
i)	Use coding software to complete assignments (e.g., in Scratch, Logo, VBA, Java)					
j)	Make video or audio productions					

Q25	At school, how often do you use ICT during lessons in the following subjects or subject areas?
	(Please mark one choice in each row)
		I don’t study this subject / these subjects	Never	In some lessons	In most lessons	In every or almost every lesson
a)	English					
b)	Spanish or other foreign language					
c)	Mathematics					
d)	Sciences (general science and/or physics, chemistry, biology, geology, earth sciences)					
e)	Human sciences/Humanities /Social studies (e.g., history, geography, civics, law, economics)					
f)	Creative arts (e.g., visual arts, music, dance, drama)					
g)	Information technology, computer studies or similar subject					
h)	Practical or vocational (e.g., mechanics and repair, healthcare occupations, construction trades)					
i)	Other (e.g., moral/ethics, physical education, personal and social development)					

Q26	When studying throughout this school year, how often did you use the following tools during class?
	(Please mark one choice in each row)
		Never	In some lessons	In most lessons	In every or almost every lesson
a)	Tutorial software or practice programs				
b)	Word-processing software (e.g., Microsoft Word)				
c)	Presentation software (e.g., Microsoft PowerPoint)				
d)	Spreadsheets (e.g., Microsoft Excel)				
e)	Multimedia production tools (e.g., media capture and editing, web production)				
f)	Concept mapping software (e.g., Inspiration, Webspiration)				
g)	Tools that capture real-world data (e.g., speed, temperature) digitally for analysis				
h)	Simulations and modelling software				
i)	Computer-based information resources (e.g., websites, wikis, encyclopedia)				
j)	Interactive digital learning resources (e.g., learning games or applications)				
k)	Graphing or drawing software				

Q27	At school, to what extent have you learned how to do the following tasks?
	(Please mark one choice in each row)
		To a large extent	To a moderate extent	To a small extent	Not at all
a)	Provide references to Internet sources				
b)	Search for information using ICT				
c)	Present information for a given audience or purpose using ICT				
d)	Work out whether to trust information from the Internet				
e)	Decide what information obtained from the Internet is relevant to include in school work				
f)	Organize information obtained from Internet sources				
g)	Decide where to look for information on the Internet about an unfamiliar topic				
h)	Use ICT to collaborate with others				

Q28	At school, have you learned about the importance of the following topics?
	(Please mark one choice in each row)
		Yes	No
a)	To change passwords regularly (e.g., network account, email, social media)		
b)	To check the origin of emails before opening attachments		
c)	To log out of a shared computer at the end of a session		
d)	To share information on social media responsibly		

Your Thoughts about Using and Learning about ICT

Q29	How well can you do each of these tasks when using ICT?
	(Please mark one choice in each row)
		I know how to do this	I have never done this but I could work out how to do this	I do not think I could do this
a)	Edit digital photographs or other graphic images			
b)	Create a database (e.g., using Microsoft Access)			
c)	Write or edit text for a school assignment			
d)	Search for and find relevant information for a school project on the Internet			
e)	Build or edit a webpage			
f)	Change the settings on your device to improve the way it operates			
g)	Create a computer program, macro, or app (e.g., in Scratch, Logo, VBA, Java)			
h)	Set up a local area network of computers or other ICT			
i)	Create a multi-media presentation (with sound, pictures, or video)			
j)	Upload text, images, or video to an online profile			
k)	Insert an image into a document or message			
l)	Install a program or app			
m)	Judge whether you can trust information you find on the Internet			

Q30	How much do you agree or disagree with the following statements about ICT?
	(Please mark one choice in each row)
		Strongly agree	Agree	Disagree	Strongly disagree
a)	Advances in ICT usually improve people’s living conditions.				
b)	ICT helps us to understand the world better.				
c)	Using ICT makes people more isolated in society.				
d)	With more ICT there will be fewer jobs.				
e)	People spend far too much time using ICT.				
f)	ICT is valuable to society.				
g)	Advances in ICT bring many social benefits.				
h)	Using ICT may be dangerous for people's health.				
i)	I would like to study subjects related to ICT after high school				
j)	I hope to find a job that involves advanced ICT				
k)	Learning how to use ICT applications will help me to do the work I am interested in				

Q31	When studying during the current school year, to what extent have you been taught how to do the following tasks?
	(Please mark one choice in each row)
		To a large extent	To a moderate extent	To a small extent	Not at all
a)	To display information in different ways				
b)	To break a complex process into smaller parts				
c)	To understand diagrams that describe or show real-world problems				
d)	To plan tasks by setting out the steps needed to complete them				
e)	To use tools to make diagrams that help solve problems				
f)	To use simulations to help understand or solve real world problems				
g)	To make flow diagrams to show the different parts of a process				
h)	To record and evaluate data to understand and solve a problem				
i)	To use real-world data to review and revise solutions to problems				

Q32	Do you study computing, computer science, information technology, informatics or similar in the current school year?
	Yes	
	No	

THANK YOU FOR YOUR TIME AND EFFORT IN COMPLETING THE QUESTIONNAIRE