International Computer and Information Literacy Study (ICILS 2018) Pretest

International Computer and Information Literacy Study (ICILS 2018) Pretest

OMB# 1850-0803 v.207

Volume 2

Submitted by:

National Center for Education Statistics (NCES)

Institute of Education Sciences (IES)

U.S. Department of Education

Washington, DC

August 2017

PAPERWORK BURDEN STATEMENT

The Paperwork Reduction Act and the NCES confidentiality statement are indicated below. Appropriate sections of this information are included in the consent forms and letters. The statements will be included in the materials used in the study.

Paperwork Burden Statement, OMB Information

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0803. The time required to complete this information collection is estimated to be an average of 175 minutes, including the time to review instructions and participate in the information collection.

If you have any comments concerning the accuracy of the time estimate, suggestions for improving this collection, or any other concerns, please write to: International Computer and Information Literacy Study (ICILS), National Center for Education Statistics, Potomac Center Plaza, 550 12th St., SW, 4th floor, Washington, DC 20202.

This is a project of the National Center for Education Statistics (NCES), part of the Institute of Education Sciences, within the U.S. Department of Education.

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

OMB No. 1850-0803 Approval Expires 07/31/2019

INTRODUCTION TO THE ASSESSMENT

As documented in Volume 1, there will be two pretesting sessions. The administrators will provide additional instructions to the students:

For this part of the study, we would like you to take your time exploring the system and interacting with the questions and system tools in different ways. While we would like you to provide answers to questions, we would like you to focus mainly on using the system tools and responding to the questions. Any response is acceptable for the questions as we really want you to experiment with how you can respond to questions and move from question to question. Have a little fun with things and explore the system!

Please take the assessment in this way until your time is up.

STUDENT DEBRIEFING QUESTIONS

After all students have finished the assessment in each session, field staff will conduct a debriefing activity with the students. Field staff will ask the debriefing questions out loud to the group of students. The questions may be paraphrased, and additional follow-up questions, such as those indicated in the bulleted lists below, may be asked. Students will respond by raising their hands and by calling out responses, as appropriate. Field staff will use their discretion about whether some or all of the questions will be asked, depending on the amount of prompting the students require, and on the amount of time remaining. The debriefing session should not exceed 10 minutes.

Field staff will write down the students’ responses to the questions.

1. What did you like the most about the assessment?

[Follow-up questions may include:]

• Can you explain that more?

• What did you like the least about the assessment?

• What was missing from the assessment?

• If you had to take a test, would you rather take it on a tablet, or on paper? Why?

• What do you like most about taking a test on a tablet? What do you like the least about taking a test on a tablet?

2. Did you see anything you thought was strange while taking the assessment?

[Follow-up questions may include:]

• Can you explain that more?

• Did questions or images seem slow to load at all to you?

• Was the system slow to move from one page/question to the next at all?

• Did you notice any freezes of your tablet or any error messages?

• Were you able to make these errors occur when you wanted to? How?

3. Did you find the system easy to use and/or easy to learn to use?

[Follow-up questions may include:]

• How did you like the tutorial?

• Did you use the Help screens? Were the Help screens useful?

• What did you think of the tools available? Did you use them?

• Which tool or tools did you like the most? Why?

• Did you have any difficulty using any of the tools? What did you find difficult?

• How did you find the mouse and keypad work with the assessment?

• Did you have any difficulty responding to questions using the available controls? If so, what made it difficult?

4. Is there anything you think would make the assessment better? If so, what?

5. Is there anything else you want to tell us about taking assessments on tablets?

SURVEY QUESTIONNAIRES

Students will take a set of survey questionnaire items during the ICILS pretest assessment session. For the first pretesting session the items previously administered as part of the ICILS field test will be used, and are included below. For the second pretesting session we anticipate using the ICILS 2018 items that will be finalized for the main study. An update to this request will be submitted to OMB when the final questionnaires are ready and prior to the second pretesting session.

IEA International Computer and Information Literacy Study

Student Questionnaire for the 2017 U.S. Field Test

For use in ICILS 2018

Do not cite or quote

The National Center for Education Statistics (NCES), within the U.S. Department of Education, conducts ICILS in the United States as authorized by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may only be used for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0929. The time required to complete this information collection is estimated to average 30 minutes per student, including the time to review instructions and complete and review the information collection. If you have any comments or concerns regarding the accuracy of the time estimate(s), suggestions for improving the form, or questions about the status of your individual submission of this form, write directly to: International Computer and Information Literacy Study (ICILS), National Center for Education Statistics, Potomac Center Plaza, 550 12th St., SW, 4th floor, Washington, DC 20202.

OMB No. 1850-0903, Approval Expires 07/31/2019.

The Australian Council for Educational Research

Student Questionnaire

This questionnaire is about Information and Communication Technology (ICT). In this questionnaire, ICT can refer to:

• desktop computers

• notebook or laptop computers

• netbook computers

• tablet devices

• smartphones, except when being used for talk and text.

In this questionnaire, you will find questions about:

• You, your home, and your family

• Where and how often you use ICT

• What you use ICT for

• Your views about the use of ICT.

Please read each question carefully and answer as accurately as you can. In this questionnaire, you will mostly answer by clicking on a button. You can change your responses at any time until you have clicked on ‘I’ve finished’ at the end of the questionnaire.

There are also a few questions where you will need to write a short response.

In this questionnaire, there are no right or wrong answers. Your answers should be the ones that apply to you.

You may ask for help if you do not understand something or if you are not sure how to answer a question.

The National Center for Education Statistics (NCES) is authorized to collect information from the questionnaire by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). You do not have to provide the information requested. However, the information you provide will help the U.S. Department of Education’s ongoing efforts to understand better how the education system in the United States compares to those in other countries. There are no penalties should you choose not to participate in this study. Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Your responses will be combined with those from other participants to produce summary statistics and reports.

Thank you.

About You

		G01
Q1	When were you born? (Scroll down to answer both parts to this question)
	January February March April May June July August September October November December
	1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

				G02
Q2	Are you a girl or a boy?
	Girl	Boy
		

				G03
Q3	What is the highest level of education you expect to complete?
	(Please mark only one choice)
	Bachelor's degree (4-year college program) OR Master's degree or professional degree (MD, DDS, lawyer, minister) OR Doctorate (Ph.D. or EdD)	
	Associate's degree (2-year college program)	
	High school graduate	
	Some high school	
	Less than high school	

Your Home and Your Family

In this section you will be asked some questions about your family and your home.

Some of these questions will be about home and your parents or guardians who look after you — for example, step-parents or foster-parents. Select one parent or guardian as parent or guardian 1 and the other as parent or guardian 2.

If you share your time with more than one set of parents or guardians, please answer the following questions for those parents/guardians with whom you spend the most time. If you share your time only with one parent, please answer only the following questions for this parent.

					G04
Q4	In what country were you and your parents born?
	(Please mark only one choice in each column)
		You	Parent or guardian 1	Parent or guardian 2
	United States			
	Another country			

				G05
Q5	What language do you speak at home most of the time?
	(Please mark only one choice)
	English	
	Spanish	
	Another language	

				G06
Q6	Does your parent or guardian 1 work at a paid job?
	Yes		(Note: Student will be directed to Q7a and Q7b)
	No		(Note: Student will be directed to Q8a and Q8b)

		G07A
Q7a	What is your parent or guardian 1’s main job? (for example, school teacher, cook, sales manager)
	(Please write in the job title)
		G07B
Q7b	What does your parent or guardian 1 do in his/her main job? (for example, teaches high school students, helps prepare meals in a restaurant, manages a sales team)
	(Please use a sentence to describe the kind of work he/she does in that job)

		G08A
Q8a	What was your parent or guardian 1’s last main job? (for example, school teacher, cook, sales manager)
	Please tell us his/her last main job. If he/she has never had a paid job, please write what he/she is currently doing.
	(Please write in the job title)
		G08B
Q8b	What did your parent or guardian 1 do in his/her last main job? (for example, taught high school students, helped prepare meals in a restaurant, managed a sales team)
	(Please use a sentence to describe the kind of work he/she did in that job or what he/she is currently doing if he/she has never had a paid job)

				G09
Q9	What is the highest level of education completed by your parent or guardian 1?
	If you are not sure which box to choose, please ask the test administrator for help. (Please mark only one choice)
	Bachelor's degree (4-year college program) OR Master's degree or professional degree (MD, DDS, lawyer, minister) OR Doctorate (Ph.D., or EdD)	
	Associate's degree (2-year college program)	
	High school graduate	
	Some high school	
	Less than high school	

				G10
Q10	Does your parent or guardian 2 work at a paid job?
	Yes		(Note: Student will be directed to Q11a and Q11b)
	No		(Note: Student will be directed to Q12a and Q12b)

		G11A
Q11a	What is your parent or guardian 2’s main job? (for example, school teacher, cook, sales manager)
	(Please write in the job title)
		G11B
Q11b	What does your parent or guardian 2 do in his/her main job? (for example teaches high school students, helps prepare meals in a restaurant, manages a sales team)
	(Please use a sentence to describe the kind of work he/she does in that job)

		G12A
Q12a	What was your parent or guardian 2’s last main job? (for example, school teacher, cook, sales manager)
	Please tell us his/her last main job. If he/she has never had a paid job, please write what he/she is currently doing.
	(Please write in the job title)
		G12B
Q12b	What did your parent or guardian 2 do in his/her last main job? (for example, taught high school students, helped prepare meals in a restaurant, managed a sales team)
	(Please use a sentence to describe the kind of work he/she did in that job or what he/she is currently doing if he/she has never had a paid job)

				G13
Q13	What is the highest level of education completed by your parent or guardian 2?
	If you are not sure which box to choose, please ask the test administrator for help. (Please mark only one choice)
	Bachelor's degree (4-year college program) OR Master's degree or professional degree (MD, DDS, lawyer, minister) OR Doctorate (Ph.D. or EdD)	
	Associate's degree (2-year college program)	
	High school graduate	
	Some high school	
	Less than high school	

				G14
Q14	About how many books are there in your home?
	Do not count magazines, newspapers, comic books or your schoolbooks.
	(Please mark only one choice)
	None or very few (0–10 books)	
	Enough to fill one shelf (11–25 books)	
	Enough to fill one bookcase (26–100 books)	
	Enough to fill two bookcases (101–200 books)	
	Enough to fill three or more bookcases (more than 200 books)	

				G15
Q15	How often do you use ICT to read e-books in your home?
	(Please mark only one choice)
	Never or hardly ever	
	At least once a month but not every week	
	At least once a week but not every day	
	Every day	

							G16
Q16		How many of the following ICT are currently used in your home?
		(Please mark one choice in each row)
			None	One	Two	Three or more
a)	Desktop computers					
b)	Portable computers (e.g., laptop, notebook, netbook)					
c)	Tablet devices or e-readers (e.g., iPad, Surface Pro, Kindle)					
d)	Smartphones					

				G17
Q17	Do you have an Internet connection at home?
	(Please mark only one choice)
	Yes	
	No	

Your Use of ICT

							G18
Q18	How long have you been using each of the following ICT?
	(Please mark one choice in each row)
		Never or less than one year	At least one year but less than three years	At least three years but less than five years	At least five years but less than seven years	Seven years or more
a)	Desktop or portable computers					
b)	Tablet devices					
c)	Smartphones except for using text and calling					

							C01
Q19	Who mainly taught you how to do the following activities?
	(Please mark one choice in each row)
		My teachers	My family	My friends	I taught myself	I have never learned this
a)	Communicate over the Internet					
b)	Create or edit documents					
c)	Create or edit presentations					
d)	Change settings on an ICT device					
e)	Find information on the Internet					
f)	Use programs and files in a computer network					

							C02
Q20	How often do you use ICT in these places?
	Please do not count the use of smartphones when making phone calls or writing text messages. (Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every day	Every day
a)	At home					
b)	At school					
c)	At other places (e.g., local library, Internet café)					

							A01
Q21	How often do you use ICT for each of the following activities?
	(Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every day	Every day
a)	Write or edit documents					
b)	Use a spreadsheet to do calculations, store data or plot graphs (e.g., using Microsoft Excel)					
c)	Create a simple “slideshow” presentation (e.g., using Microsoft PowerPoint)					
d)	Record or edit videos					
e)	Write computer programs, scripts or apps (e.g., using Scratch, Logo, VBA, Java)					
f)	Use drawing, painting or graphics software or apps					
g)	Produce or edit music					
h)	Build or edit a webpage					

							A02
Q22	How often do you use ICT to do each of the following communication activities?
	(Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every day	Every day
a)	Share news from current events on social media					
b)	Talk to friends, family, or other people using voice or video chat (e.g., Skype, FaceTime, WhatsApp, Viber)					
c)	Send texts or instant messages to friends, family, or other people					
d)	Write posts and updates about what happens in your life on social media					
e)	Ask questions on forums or Q&A, question and answer websites					
f)	Answer other peoples’ questions on forums or Q&A, question and answer websites					
g)	Write posts for your own blog (e.g., WordPress, Tumblr, Blogger)					
h)	Post images or video in social networks or online communities (e.g., Facebook, Instagram or YouTube)					
i)	Watch videos or images that other people have posted online					
j)	Tag others online in posts or images on social media					
k)	Send or forward information about events or activities to other people					

							A03
Q23	How often do you use ICT to do each of the following leisure activities?
	(Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every day	Every day
a)	Search the Internet to find information about places to go or activities to do					
b)	Read reviews on the Internet of things you might want to buy					
c)	Read news stories on the Internet					
d)	Search for online information about things you are interested in					
e)	Read posts or websites about things you are interested in					
f)	Use websites, forums, or online videos to find out how to do something					
g)	Play single-player games					
h)	Play multi-player games using the same device as the other player(s)					
i)	Play multi-player games where each player uses his/her individual device					
j)	Listen to downloaded or streamed music					
k)	Watch downloaded or streamed TV shows or movies					
l)	Buy things online					

Using ICT for School

							A04
Q24	How often do you use ICT for the following school-related purposes?
	(Please mark one choice in each row)
		Never	Less than once a month	At least once a month but not every week	At least once a week but not every school day	Every school day
a)	Prepare reports or essays					
b)	Prepare presentations					
c)	Work online with other students from your own school					
d)	Work online with other students from other schools					
e)	Complete worksheets or exercises					
f)	Organize your time and work					
g)	Write about your learning					
h)	Complete tests					
i)	Use software or applications to learn skills or a subject (e.g., mathematics tutoring software, language learning software)					
j)	Use the Internet to search for information					
k)	Use coding software to complete assignments (e.g., in Scratch, Logo, VBA, Java)					

							B01
Q25	At school, how often do you use ICT during lessons in the following subjects or subject areas?
	(Please mark one choice in each row)
		I don’t study this subject / these subjects	Never	In some lessons	In most lessons	In every or almost every lesson
a)	English					
b)	Spanish or other foreign language					
c)	Mathematics					
d)	Sciences (general science and/or physics, chemistry, biology, geology, earth sciences)					
e)	Human sciences/Humanities /Social studies (history, geography, civics, law, economics, etc.)					
f)	Creative arts (visual arts, music, dance, drama, etc.)					
g)	Information technology, computer studies or similar subject					
h)	Practical or vocational subjects					
i)	Other (moral/ethics, physical education, home economics, personal and social development)					

						B02
Q26	When studying throughout this school year, how often did you use the following tools during class?
	(Please mark one choice in each row)
		Never	In some lessons	In most lessons	In every or almost every lesson
a)	Tutorial software or practice programs				
b)	Word-processing software (e.g., Microsoft Word)				
c)	Presentation software (e.g., Microsoft PowerPoint)				
d)	Spreadsheets (e.g., Microsoft Excel)				
e)	Multimedia production tools (e.g., media capture and editing, web production)				
f)	Concept mapping software (e.g., Inspiration, Webspiration)				
g)	Tools that capture real-world data (e.g., speed, temperature) digitally for analysis				
h)	Simulations and modelling software				
i)	Computer-based information resources (e.g., websites, wikis, encyclopedia)				
j)	Interactive digital learning resources (e.g., learning games or applications)				
k)	Graphing or drawing software				

						B03
Q27	How often have the following activities taken place in your lessons?
	(Please mark one choice for each row)
		Never	In some lessons	In most lessons	In every or almost every lesson
a)	We use ICT to present information				
b)	My teacher uses ICT to present information				
c)	We do tests on ICT				
d)	My teacher uses ICT to provide students with feedback on what they have learned				
e)	We work on projects using ICT to share resources and ideas				
f)	We use the Internet for research				
g)	We use the Internet to contact external experts				

						B04
Q28	At school, to what extent have you learned how to do the following tasks?
	(Please mark one choice in each row)
		To a large extent	To a moderate extent	To a small extent	Not at all
a)	Provide references to Internet sources				
b)	Search for information using ICT				
c)	Present information for a given audience or purpose using ICT				
d)	Work out whether to trust information from the Internet				
e)	Decide what information obtained from the Internet is relevant to include in school work				
f)	Organize information obtained from Internet sources				
g)	Decide where to look for information on the Internet about an unfamiliar topic				
h)	Look for different types of digital information on a topic				
i)	Build or edit a webpage				

				B05
Q29	At school, have you learned about the importance of the following topics?
	(Please mark one choice in each row)
		Yes	No
a)	To keep anti-virus software on computers up-to-date		
b)	To change passwords regularly (e.g., network account, email, social media)		
c)	To check the origin of emails before opening attachments		
d)	To log out of a shared computer at the end of a session		
e)	To use social media responsibly (e.g., what types of information you share about yourself or other people)		

Your Thoughts About Using ICT

					C03
Q30	How well can you do each of these tasks when using ICT?
	(Please mark one choice in each row)
		I know how to do this	I have never done this but I could work out how to do this	I do not think I could do this
a)	Search for and find a file			
b)	Edit digital photographs or other graphic images			
c)	Create a database (e.g., using Microsoft Access)			
d)	Write or edit text for a school assignment			
e)	Search for and find relevant information for a school project on the Internet			
f)	Build or edit a webpage			
g)	Change the settings on your device to improve the way it operates			
h)	Use a spreadsheet to do calculations, store data or plot a graph			
i)	Create a computer program, macro, or app (e.g., in Scratch, Logo, VBA, Java)			
j)	Set up a local area network of computers or other ICT			
k)	Create a multi-media presentation (with sound, pictures, or video)			
l)	Upload text, images, or video to an online profile			
m)	Insert an image into a document or message			
n)	Install a program or app			
o)	Judge whether you can trust information you find on the Internet			
p)	Select information from within a digital source to use in an assignment			
q)	Create the best layout for a digital product to help viewers understand information			

						C04
Q31	How much do you agree or disagree with the following statements about ICT?
	(Please mark one choice in each row)
		Strongly agree	Agree	Disagree	Strongly disagree
a)	I would like to study subjects related to ICT after high school				
b)	I hope to find a job that involves advanced ICT				
c)	Learning how to use ICT applications will help me to do the work I am interested in				
d)	Knowing as much as possible about ICT will help me get a job				
e)	It is no longer possible to find work without knowing how to use ICT				

						C05
Q32	How much do you agree or disagree with the following statements about the use of ICT in society?
	(Please mark one choice in each row)
		Strongly agree	Agree	Disagree	Strongly disagree
a)	Advances in ICT usually improve people’s living conditions.				
b)	ICT helps us to understand the world better.				
c)	Using ICT makes people more isolated in society.				
d)	Using ICT helps people to relate better to others.				
e)	With more ICT there will be fewer jobs.				
f)	People spend far too much time using ICT.				
g)	ICT is valuable to society.				
h)	Advances in ICT bring many social benefits.				
i)	Using ICT may be dangerous for people's health.				

Studying Information Technology

						D01
Q33	When studying during the current school year, to what extent have you learned how to do the following tasks?
	(Please mark one choice in each row)
		To a large extent	To a moderate extent	To a small extent	Not at all
a)	Display information in different ways				
b)	Break a complex process into smaller parts				
c)	Understand diagrams that describe or show real-world problems				
d)	Plan tasks by setting out the steps needed to complete them				
e)	Identify the most efficient solution to a problem				
f)	Use tools to make diagrams that help solve problems				
g)	Use simulations to help understand or solve real-world problems				
h)	Make flow diagrams to show the different parts of a process				
i)	Record and evaluate data to understand and solve a problem				
j)	Work with instructions to complete or describe tasks				
k)	Use real-world data to review and revise solutions to problems				

						D02
Q34	How well do you think you can do the following tasks?
	(Please mark one choice in each row)
		Very well	Fairly well	Not very well	Not at all
a)	Display information in different ways				
b)	Break a complex process into smaller parts				
c)	Understand diagrams that describe or show real-world problems				
d)	Plan tasks by setting out the steps needed to complete them				
e)	Identify the most efficient solution to a problem				
f)	Use tools to make diagrams that help solve problems				
g)	Use simulations to help understand or solve real-world problems				
h)	Make flow diagrams to show the different parts of a process				
i)	Record and evaluate data to understand and solve a problem				
j)	Work with instructions to complete or describe tasks				
k)	Use real-world data to review and revise solutions to problems				

				D03
Q35	Do you study computing, computer science, information technology, informatics or similar in the current school year?
	Yes	
	No	

THANK YOU FOR YOUR TIME AND EFFORT IN COMPLETING THE QUESTIONNAIRE