The Early Childhood Longitudinal Study, Kindergarten Class of 2022-23 (ECLS-K:2023) Preschool Field Test Instruments Usability Testing

NCES System Clearance for Cognitive, Pilot, and Field Test Studies 2019-2022

Attachments 2-4 ECLS-K2023 Preschool FT Usability Testing Screener-Protocol-Consent

The Early Childhood Longitudinal Study, Kindergarten Class of 2022-23 (ECLS-K:2023) Preschool Field Test Instruments Usability Testing

OMB: 1850-0803

Document [docx]
Download: docx | pdf







The Early Childhood Longitudinal Study, Kindergarten Class of 2022-23 (ECLS-K:2023) Preschool Field Test Instruments Usability Testing




Attachments 2-4


Attachment 2: Recruitment Screener 2

Attachment 3: Usability Testing Protocol 6

Attachment 4: Participant Consent Form 24




OMB #1850-0803 v.253





Submitted by

National Center for Education Statistics

U.S. Department of Education






August 2019








The Early Childhood Longitudinal Study, Kindergarten Class of 2022-23 (ECLS-K:2023) Preschool Field Test Instruments Usability Testing




Attachment 2


Recruitment Screener




OMB #1850-0803 v.253





Submitted by

National Center for Education Statistics

U.S. Department of Education






August 2019





The Early Childhood Longitudinal Study, Kindergarten Class of 2022-23 (ECLS-K:2023)
Preschool Usability Recruitment Materials

Advertisement

Parents of preschoolers aged 4 or 5 needed for study – Receive $50

Parents of preschoolers aged 4 or 5 are needed to complete web survey testing and provide feedback. Westat is administering this study for the U.S. Department of Education, National Center for Education Statistics (NCES). The in-person session will last 90 minutes and take place at Westat offices in Rockville, MD.


To sign up or get more information, please email [CONTACT INFORMATION] or call [PHONE]. Westat staff members are not eligible, but their family and friends are welcome to participate.



Screener


Thank you for your interest in the study. As you may know from the ad, we are recruiting parents for an in-person testing of an online survey. We are interested in talking with parents of preschool-aged children (children aged 4 or 5, not yet in kindergarten) in order to help the U.S. Department of Education prepare a new study about early childhood education. If you are eligible, you will be asked to complete an online survey and give us your feedback on the experience. Your responses to the survey will be used for internal study purposes only. Your name will not be included in the report, and all your data will be destroyed once the report is written.


  1. First, are you an employee at Westat?

No_____

[THANK AND END] Yes_____



  1. Are you the parent of at least one preschool-age child? By preschool age, we mean children who have not yet started kindergarten or a kindergarten equivalent. The age range for this is typically 4 to 5 years of age.

Yes ____

[THANK AND END] No ____


  1. Will you have a preschool age child who attends kindergarten or a kindergarten grade equivalent in the next school year (that is, in the 2020-21 school year)?

Yes ____

[THANK AND END] No ____


  1. What is the current age of your preschool-age child(ren)?

[THANK AND END] Under age 4 ____

Age 4 ____

Age 5 ____

[THANK AND END] Age 6 ____




  1. For personal use, how often do you use ANY device (computer, tablet, or phone) to go online or use apps?

Constantly, all day long____

Many or multiple times a day____

A few times a day____

Once or twice a day ____


  1. For personal use, what type of device do you use most often to go online or use apps?

Computer____

Tablet____

Smartphone____


  1. How comfortable are you using that device to go online or use apps?

Very comfortable____

Somewhat comfortable____

Not very comfortable____


  1. How well do you think you would be able to use that device to complete an online survey?

Very well, with no trouble at all____

Somewhat well, with perhaps some trouble____

Not at all well, with a lot of trouble____



  1. When using your device to go online, what web browser do you use most?

Google Chrome____

Apple Safari ____

Firefox____

Internet Explorer or Edge____

Don’t Know____


  1. Do you have a smartphone? If yes, what kind?

No Smartphone____

Yes, Apple/iPhone____

Yes, Android____

Yes, Other____


    1. [IF YES HAVE SMARTPHONE] Would you be willing to bring your smartphone with you to complete the web survey during the testing session? We will provide free WiFi/internet access.

Yes ____

No ____




  1. So that we can be sure that participants of all backgrounds are represented in our testing, please tell me your racial/ethnic background. Which of the following do you consider yourself to be? You may select more than one. [RECRUIT AT LEAST 3 PEOPLE WHO IDENTIFY THEMSELVES AS SOMETHING OTHER THAN “WHITE, NON-HISPANIC”] [MARK ALL THAT APPLY]

Hispanic or Latino ____

White, Non-Hispanic ____

Black or African American, Non-Hispanic ____

Asian, Non-Hispanic ____

Native Hawaiian-Indian or Other Pacific Islander, Non-Hispanic ____

American Indian or Alaska Native, Non-Hispanic ____


  1. Please indicate YOUR age range.

Under age 20 ____

20-29 ____

30-39 ____

40-49 ____

50- over ____



  1. Which of the following categories best describes your annual household income?

Less than $20,000 _____

$20,000-$39,999 _____

$40,000-$59,999 _____

$60,000-$79,999 _____

$80,000-$99,999 _____

$100,000 or more _____


  1. Are you male or female?

Male _____

Female _____


Your opinions are very important to us and to thank you for your time if you participate in the web survey testing, we will give you $50 as a token of our appreciation.


I'd like to schedule an appointment when you could come in person to Westat offices in Rockville, MD. The web survey testing and discussion will take about 90 minutes. What days and times work well for you?


I have [DAYS/TIMES] available. Do any of these work for you?



As I mentioned, we will give you $50 as a token of our appreciation for your participation in this usability testing. I have you scheduled for <<DATE>> at <<TIME>>. Please come to Westat 1600 Research Blvd, Rockville, MD.


RESPONDENT NAME:


TELEPHONE/EMAIL:








The Early Childhood Longitudinal Study, Kindergarten Class of 2022-23 (ECLS-K:2023) Preschool Field Test Instruments Usability Testing




Attachment 3


Usability Testing Protocol




OMB #1850-0803 v.253





Submitted by

National Center for Education Statistics

U.S. Department of Education






August 2019





ECLS Preschool Usability Testing Protocol



Introduction

Thank you for coming in today. Let me start by telling you a little about Westat and what we are going to do today.

Westat is a research firm that has been hired by the U.S. Department of Education’s National Center for Education Statistics to test survey questions that will be used in a national education survey.

We would like you to read an introductory letter and follow the directions to go online and answer the survey questions, just as we will ask our survey participants to do. As you do the web survey, I am going to watch and will take some notes. If anything is unclear for you, or if you are not sure how to answer a question or are unsure what to do next, just pause and tell me. We are interested in knowing what is unclear and why.

At this stage, we are mostly interested in whether anything is not clear or confusing in the process of getting online and completing the survey. This will help us develop recommendations and improvements to make it as easy as possible for people to complete the final online survey. We are not testing your answers or your computer skills. We want to understand whether some tasks or survey items are working well or not. The session today will take approximately 90 minutes.

I also want to remind you that everything we cover in our discussions and all your answers in the web survey will be kept private—only the few people actually working on the project will have access to the information you share with us.

Your participation is voluntary. You may quit at any time. When we are finished, I’ll give you $50 as a token of our appreciation for completing the session.

If it’s OK with you, I would like to video record our session, just so that I can review it later when we’re working on the improvements to the survey.



Informed Consent

[HAND THE CONSENT FORM TO PARTICIPANT, ANSWER ANY QUESTIONS, AND OBTAIN CONSENT BEFORE CONTINUING.]

This form contains more information about the research and your rights during this study. Please read it over and let me know if you have any questions. If you are willing to take part in the study, please sign both copies. One copy is for you, and I will hold onto the other.

[START VIDEO/AUDIO RECORDER AND GET ORAL PERMISSION TO RECORD.] It is [DATE AND TIME], do I have your permission to record this conversation? [WAIT FOR CONFIRMATION.] Thank you.

IF NO CONSENT CONFIRMATION GIVEN: Thank you. We will not record this session. [STOP VIDEO/AUDIO RECORDER.]

Procedures

We are about to begin, but let me say a few things first. You told us previously that you are comfortable using [Chrome/Safari/Firefox/IE/Edge] as your browser, so we'd like you to use that browser today. You also told us you were comfortable using [a laptop computer/a mobile device], and we’d like you to use that.

As I mentioned before, households in the study will receive an introductory letter and follow the directions to go online and answer the survey questions.

FOR GROUP 1 ONLY HAND THE LETTER TO THE PARTICIPANT AND SAY:

Please follow the information in the letter and complete the survey. As you do this, if anything is unclear or you feel stuck as you are completing the survey, please let me know. Also, if you have any questions or comments as you work through the survey, please let me know, but otherwise, you can work through the survey at your own pace. After you have completed the survey, I will ask you some follow-up questions about your experience and the design of the survey. This will help me understand what worked well for you and what may need adjustment.

Do you have any questions before we begin?

Here is the letter that participants will receive in the mail. Please open it and follow the instructions to begin the survey.

FOR GROUP 2 ONLY SAY:

I’d like you to think aloud as you read the letter, follow its instructions to go online, and complete the survey. For example, talk about where you are looking and how you are deciding what to do next. I will be following along and observing. Sometimes I will interrupt you to ask questions.

Please share any reactions about starting the web survey, the layout, the clarity of the instructions of the survey questions, and how to enter your responses. We’d like to hear what you are thinking about in terms of aspects such as:

  • Finding the website and starting the survey,

  • Navigating within the survey,

  • Ease of use,

  • The placement of the next and back buttons,

  • The look of the web pages, like font size and color, and

  • The position of the question and response options on the screen.

Finally, after you have completed the survey, I will ask you some follow-up questions about your experience and the design. This will help me understand what worked well for you and what may need adjustment.

Do you have any questions before we begin?

HAND THE LETTER TO THE PARTICIPANT AND SAY: Here is the letter that participants will receive in the mail. Please open it and follow the instructions to begin the survey.

Item Specific Questions

Letter

  1. Are the instructions for starting the survey online clear?

    1. Was it easy for you to get to the website or did you have any trouble?

      • [IF HAD DIFFICULTY] Can you tell me more about what trouble you had?

      • [IF HAD DIFFICULTY] If you had trouble, would you call a help desk number for assistance?

    2. Are you familiar with using PIN numbers to gain access to online sites?

      • [IF NOT FAMILIAR WITH PINS] Can you tell me what your reaction was to the PIN? Did you have any concerns or questions that we could have answered?

    3. Are there other questions you have about the study that are not addressed in the letter? [IF NEEDED] What questions do you have?

    4. What are your impressions of the letter?

Home Screen/Eligibility Question

  1. What are your impressions of the first screen (the home screen)? [SHOW SCREENSHOT IF NEEDED]?

    1. On the website, was it easy or difficult to figure out where to start? Or what to do?

      • [IF DIFFICULT/PROBLEMATIC]: Tell me more about why you say that?

    2. If you took a break from completing the survey and were now returning to the survey after you began it in a previous session, what would you do here?

      • [IF DID NOT KNOW] Did you see the part of the screen with the link to return to the survey? How could that be clearer?

    3. Did you have any trouble answering the question, "Are there any children age 10 or younger living in this household?" Was anything unclear? Can you tell me more about what was unclear or the trouble you had?

Eligible Response/Additional Security

  1. Was it clear to you why you were asked for your email address or mobile phone number?

    1. Did you have any concerns about providing this information? [If YES] What were your concerns?

    2. {ASK IF NEEDED} Please describe to me how do you think the process will work if you wanted to exit the survey once you started it, then wanted to login again to finish the survey? What would happen? What would you do?



Survey Questions

IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM.

IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.

SCR030 Child Name (Test of typing in text box) Alternate items: INQ010/FSQ120a

QUESTION TEXT:

Please list the first names of all the children age 10 or younger who live in this household. You may provide nicknames or initials if you prefer. Be sure to include all small children, foster children, and babies."

Start with the youngest child who is age 10 or younger. Enter one child on each line. Enter names until you have listed all children age 10 or younger in this household, then click Next.

Child {NUMBER}:

DISPLAY THE NUMBER OF TEXT FIELD LINES IN SCR030 TO CORRESPOND TO THE NUMBER PROVIDED IN SCR020. LABEL LINE “Child {NUMBER}”

  1. Some of the questions required you to type in a response in an entry box.

    1. Did you have any difficulty using these boxes? If so, what was the difficulty?

    2. What did you think about the size of the entry boxes? Did you have enough space to type your answer?

    3. How comfortable did you feel typing in a response?

    4. I noticed that you entered [first name/initials/left blank]. How did you feel about providing this information?

    5. [IF INDICATED CONCERN ABOUT USING NAME] Did you know you could type in initials? Would you feel comfortable entering initials instead?



IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM. IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.

SCR080 Grade in Fall (Test of Probe) Alternate items: INQ015, FSQ040

QUESTION TEXT:

What grade or grade equivalent will {SCR030} be in fall 2020?



Child care for infants or toddlers in a home or center includes early childhood education programs, child care, or day care in a center, or child care with a relative or nonrelative other than a parent/guardian.



Preschool or child care for preschool-aged children in a home or center includes early childhood education programs, child care, or day care in a center, nursery school, preschool, prekindergarten, or child care with a relative or nonrelative other than a parent/guardian.



Kindergarten includes Transitional Kindergarten (TK), Early Transitional Kindergarten (ETK), Readiness Kindergarten, Transitional or Prefirst Grade, or a program that is a kindergarten equivalent but is ungraded or has multiple grades.



PROBE TEXT (IF QUESTION LEFT BLANK):

It looks like you haven’t answered the question yet. Please take another look at the question and provide an answer. Select a choice below to continue.



  1. [IF RESPONDENT DID NOT SEE PROBE SHOW SCREENSHOT WITH PROBE 1 TEXT] Please look at this screenshot. If you left this item blank and then saw this screen after selecting NEXT on this item, what is your reaction?

    1. [IF R MENTIONS TONE OR REDUNDANCY OF MESSAGE ASK]: Can you tell me more about why you feel that way?



  1. Would you understand why you are seeing this message? Is it clear? [IF NEEDED] A message appeared in red text that asked you to provide an answer.



  1. What would you do next?

  1. Was it difficult or easy to figure out what to do? Why?

  2. Is there anything you would change?





IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM.

IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.



INQ001 (Test for placement of response for "I cannot answer…")

QUESTION TEXT:

We would like to collect some information about {CHILD}'s care and home experiences. Your answers will be extremely valuable in better understanding the development of young children and their involvement in early care and education.

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

Are you a person in the household who knows about {CHILD}’s care and education?

Yes, I can answer questions about {CHILD}’s care and education.

No, I am not able to answer questions about {CHILD}’s care and education.

  1. I noticed that you selected the option [No I am not able to answer/Yes, I can answer].

  1. Can you tell me what you thought that question meant?

  2. Do you think there is more than one adult in your household that would be able to answer yes?

  3. Who should select the option No?

    • [IF IS UNSURE] What were you unsure about?

    • Can you tell me more about what sort of living arrangement in a household might make you unsure?

    • How can we improve this wording?



IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM.

IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.



PSQ050 (Test for difficulty to answer)

QUESTION TEXT:



Please select whether {CHILD} does the following. Have you observed that {CHILD}...

1

Yes

2

No

a. Tries repeatedly to communicate information which has not been understood?

○

○

b. Asks questions about information which is unclear?

○

○

c. Responds to questions in a thoughtful way that makes sense?

○

○

d. Rephrases questions or asks follow-up questions if {he/she} does not get the information {he/she} wanted?

○

○

  1. Did you have difficulty answering these questions?

    1. [IF YES] Please share your thoughts.

    2. [IF NEEDED] Were the question stems or the response options difficult or confusing?



  1. [IF RESPONSE OPTIONS WERE MENTIONED ASK] Is there anything you would change about the response options?

IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM.

IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.



PSQ055

QUESTION TEXT:



Now we would like to ask you about different words {CHILD} uses. Children understand many more words than they say. We are particularly interested in the words {CHILD} says. Here is a list of words. Please select the words your child uses in English. If {CHILD} uses a different pronunciation of a word, that’s OK.



Does {CHILD} say…



Select All That Apply.



CODES

1 hungry 14 stamp

2 baby 15 parent

3 doctor 16 lucky

4 down 17 furniture

5 bird 18 drip

6 fruit 19 measure

7 triangle 20 calm

8 turtle 21 lonely

9 plant 22 dive

10 last 23 skeleton

11 caterpillar 24 uncomfortable

12 castle 25 courage

13 excited

  1. [FOR DESKTOP/LARGE BROWSERS] For some of the questions, the response options are presented in two columns on the screen [SHOW SCREENSHOT IF NEEDED]. What do you think about how these are presented on the screen?

  1. Did you notice the second column?

    • [IF NO] Why not?

  2. Tell me what you were thinking as you read down the list?

    • Did you consider each word before moving on to the next word?

    • Did you review or scan the list once or more than once before you continued?



  1. [FOR MOBILE BROWSERS] For some of the question, there are many response options. What do you think about how these are presented on the screen?

    1. What did you do as you read the list?

    2. Tell what you were thinking as you read down the list?

      • Did you consider each word before moving on to the next word?

      • Did you review or scan the list once or more than once before you continued?



IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM. IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.

HEQ010 Home learning activities (matrix question) Alternate items: FSQ130, FSQ180

QUESTION TEXT:



Now we would like to ask about {CHILD’S} home environment.



In a typical week, how often do you or any other family members do the following things with {CHILD}?




1

Not at all

2

Once or twice a week

3

3-6 times a week

4

Every day

a. Tell stories to {CHILD}

○

○

○

○

b. Sing songs with {CHILD}

○

○

○

○

c. Help {CHILD} to do arts and crafts

○

○

○

○

d. Involve {CHILD} in household chores, like cooking, cleaning, setting the table, or caring for pets

○

○

○

○

e. Play games or do puzzles with {CHILD}

○

○

○

○

f. Talk about nature or do science projects with {CHILD}

○

○

○

○

g. Build something or play with construction toys with {CHILD}

○

○

○

○

h. Play a sport or exercise together

○

○

○

○

i. Practice reading, writing, or working with numbers

○

○

○

○



  1. [FOR DESKTOP BROWSER] This question has the same beginning question and then asks about different activities, which are displayed in a grid or chart. How easy or difficult would you say it was to answer this question while it was displayed in a grid?

    1. [IF DIFFICULT] Why was it hard?

    2. [IF DIFFICULT] Would a different format or layout be easier?



  1. [FOR MOBILE BROWSERS] This question has the same beginning question and then asks about different activities, and each activity is displayed on a separate screen. How easy or difficult would you say it was to answer these questions with each on a separate screen?

    1. [IF DIFFICULT] Why was it hard?

    2. [IF DIFFICULT] Would a different format or layout be easier?

    3. How did you feel about the speed or pace it took you to answer these questions? Too slow, too fast, or about right?



  1. A blue question mark icon is used here – Did you see the help/question mark icons when you read this question?

  1. Can you tell me why you think it is there?



  1. Why did you/did you not click on the question mark?

    1. [IF GROUP 1, SHOW SCREENSHOT of HEQ010h WITH HELP TEXT] Looking at the help text that appears with item h, what are your thoughts? Would this have been helpful or not helpful? If not, what would make it more helpful?

    2. [IF GROUP 2, Say:] Please click on the question mark icon before we continue. Looking at the help text that appears with item h, what are your thoughts? Would this have been helpful or not helpful?

      • [IF NOT HELPFUL] What would make it more helpful?

    3. Did you know that this additional information was available by looking at the screen?

    • What might make it clearer that the question mark icon could be clicked for help with the question?

IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM.

IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.



HEQ273a and HEQ273b Media engagement and usage (Test reporting in different units for the same question) Alternate items: CHQ086, CHQ087

QUESTION TEXT:

On any given weekday, how much time does {CHILD} spend watching shows, movies, and videos?



If less than an hour, enter '0' in the "Hours" section. If {CHILD} does not watch shows, movies, or videos, enter '0' in both the “Hours” and "Minutes" section.



Hours:

Watermark "Enter hours"



Minutes:

Watermark "Enter minutes"



  1. Did you think this question was easy or hard to answer? Why?

  1. Did you fill in a number for both hour and minute fields or only one? Why?

  2. If you entered more than 59 minutes, a message appears [SCREENSHOT A18].

    • Do you think this is helpful? [IF NO, HOW WOULD YOU CHANGE IT]

    • How would you answer this question if your child does not spend any time watching shows, movies, or videos?

IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM.

IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.



HEQ288 (Test for long response options, and more than 6 response options) Alternate items: CCQ560

QUESTION TEXT:



Now we'd like to ask you about some of the activities your child might do. Has {CHILD} ever participated in any of the following activities?



Select All That Apply.



CODES:

1 Organized athletic activities, like basketball, soccer, baseball, swimming, or gymnastics

2 Dance groups, classes, or lessons

3 Music, for example, piano, instrumental music, or singing lessons

4 Drama groups, classes, or lessons

5 Art groups, classes, or lessons, for example, painting, drawing, sculpture

6 Craft groups, classes, or lessons

7 Language groups, classes, or lessons (to learn English or another language)

  1. Was this question easy or difficult to answer?

    1. [IF DIFFICULT] Why? What would make it easier to answer?

    2. [IF DONE ON MOBILE]: Did you see all the response options before you answered the question?

    • Did you scroll in order to see all the options?

    • Some questions have longer response options or many response options, like this one. Did you find if easy or difficult to read through the response options here?

IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM.

IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.



KSQ010 (School selection from List) Test the usability of the lookup file

Alternate items: SCR100 (extended screener question)

QUESTION TEXT:



What is the name of the school where {CHILD} will attend {kindergarten/next year}?



Select the school from the list below. If you don't find the school, select "School not on list" and then type in the full school name. If you are considering more than one school, enter the name of the school that is most likely.

  1. {DISPLAY LIST OF UP TO 25 SCHOOLS} Were these instructions clear?

    1. Did you understand the instructions? Are they clear or confusing?

      • [IF CONFUSING] Why? What was confusing?

    2. Were you able to find the school you were looking for right away?

    3. What would you do if you didn't see the school on the list?

      • [IF NEEDED] Did you understand how to enter the school name if it did not appear on the list? If not, what confused you?

IF GROUP 1, SHOW SCREENSHOT OF THIS ITEM.

IF GROUP 2, STOP RESPONDENT AFTER THIS QUESTION HAS BEEN ANSWERED.

[NOTE: ASK FOLLOW UP QUESTION FOR FSQ WHEN ENTIRE FSQ010 – FSQ160 SERIES IS COMPLETE.]



FSQ010 (HH roster names), FSQ030 (HH roster ages), FSQ040 (HH roster sex)



FSQ010

QUESTION TEXT:



Now we have some questions about your household. Please confirm the names or initials of the household members you already told us about before and add the first names of all the other people who normally live here, including both adults and children. Please do not include anyone staying here temporarily who usually lives somewhere else.



We ask for first names so that we can ask questions about each person in the survey. If there is a typo, or you gave a nickname or initials, you may correct it here before continuing.



Please list each person in your household on a separate line, until you have listed all the people in your household.

Don't forget…

-your spouse or partner, or

-someone who is temporarily away from home or living in a dorm at school, or

-any babies or small children.



First name

{DISPLAY FIRST NAME OF RESPONDENT}

{DISPLAY FIRST NAME OF SAMPLED CHILD}

{DISPLAY FIRST NAMES OF ALL OTHER CHILDREN LISTED IN SCREENER, IF APPLICABLE}



FSQ030

QUESTION TEXT:



How old {are you/is {NAME}}?

For babies less than 1 year old, enter 0.

Watermark "Enter age"



FSQ040

QUESTION TEXT:



{Are you/Is {NAME}} male or female?}.

  1. [SHOW SCREENSHOT IF NEEDED] On these screens asking about your household, did you understand what to do here? Why or why not?

  1. When you were entering people on the list, who did you think about to add?

  2. Was there anyone you forgot? Was there anyone you were unsure of adding or not?

    • Who and why?

  3. I noticed that you entered [first name/initials/left blank], how did you feel entering this information?

  • [IF UNCOMFORTABLE] How would you have felt if you knew you could enter initials or nicknames?

FSQ110 (HH roster spouse Y/N), FSQ120 & FSQ120a (HH roster spouse name), FSQ120b (HH roster spouse age), FSQ120c (HH roster spouse sex)



FSQ110

QUESTION TEXT:

Do you have a spouse or partner who lives in this household?



FSQ120

QUESTION TEXT:

Who in the household is your spouse or partner?



Select the number next to the name of the person who is your spouse/partner.

If name not listed, select "not on list."



CODES

1 {DISPLAY HH MEMBER NAME 1}

2 {DISPLAY HH MEMBER NAME 2}

3 {DISPLAY HH MEMBER NAME 3}

4 {DISPLAY HH MEMBER NAME 4}

5 {DISPLAY HH MEMBER NAME 5}

6 {DISPLAY HH MEMBER NAME 6}

7 {DISPLAY HH MEMBER NAME 7}

8 {DISPLAY HH MEMBER NAME 8}



Not on list



FSQ120a

QUESTION TEXT:

What is the first name of your spouse or partner?

First name:



FSQ120b

QUESTION TEXT:

How old is {NAME}?

Watermark "Enter age"



FSQ120c

QUESTION TEXT:

Is {NAME} male or female?

  1. [IF R ADDED SPOUSE LATER] I noticed that you did not list your spouse on the first screen but added your spouse to the list later. Was this unclear what to do here or was it confusing?

  1. Did you have any difficulty? Please describe your thoughts.



FSQ130 (HH roster relationship to child series), FSQ140/150/155 (HH roster respondent relationship to child), FSQ160 (HH roster sibling)





FSQ130

QUESTION TEXT:

What is {your/{NAME}'s} relationship to {CHILD}?



CODES

1 Mother/female guardian

2 Father/male guardian

3 Other parent/guardian

4 Sister

5 Brother

6 Girlfriend or female partner of {CHILD}'s parent/guardian

7 Boyfriend or male partner of {CHILD}'s parent/guardian

8 Other partner of {CHILD}'s parent/guardian

9 Grandmother

10 Grandfather

11 Aunt

12 Uncle

13 Cousin

14 Other relative

15 Other non-relative



FSQ140

QUESTION TEXT:



{Are you/Is {NAME}} {CHILD}'s…



FSQ150

QUESTION TEXT:



{Are you/Is {NAME}} {CHILD}'s…

CODES

1 Biological or birth father

2 Adoptive father

3 Step father

4 Foster father or legal male guardian, or

5 Other male parent or guardian



FSQ155

QUESTION TEXT:



{Are you/Is {NAME}} {CHILD}'s…

CODES

1 Biological or birth parent

2 Adoptive parent

3 Step parent

4 Foster parent or legal guardian

5 Other parent or guardian



FSQ160

QUESTION TEXT:

{Are you/Is {NAME}} {CHILD}'s…



CODES

1 Full sister

2 Half sister

3 Step sister

4 Adoptive sister

5 Foster sister

  1. In this series of questions you were asked to pick the relationship between the adults in the household and your child. Was this task easy or hard?

    1. [IF HARD] Why? What made it difficult? How could it be improved?

    2. Was it clear how to answer the question and who the question was about? If no, what was confusing?

    3. Did you have any trouble finding the correct relationship for anyone? What was the confusion?

    4. [IF R SELECTED HELP TEXT] I noticed you clicked the question mark icon and additional text appeared. What was your reaction to that text?

    • Did the additional text that appeared help you to decide on your response? How so?

    • Was the text easy or hard to understand?

    1. [IF R DID NOT SELECT HELP TEXT AND HAD TROUBE] I noticed you did not select the question mark icon. What would you expect to see if you did click it?



General Questions

General Impressions about the Instrument

  1. Please tell me about your general impression about the survey.

    1. What did you think about the layout of the survey questions on the screener?

    2. What was your experience navigating between items?

      • Did you have any difficulty?

      • Is there anything that would make the navigation better?



Navigation



[PROBE FIRST ON ANY OBSERVED ISSUES WITH NAVIGATION]



  1. DID RESPONDENT TRY TO USE BROWSER BACK/NEXT BUTTONS OR USE THE INSTRUMENT BUTTONS FOR NAVIGATION?

  1. [IF USED BROWSER BUTTONS] Why did you use these (browser) to move (back/to the next question)?

    • Did you see [POINT TO INSTRUMENT BACK/NEXT BUTTONS] these?

  2. [IF USED INSTRUMENT BUTTONS] What do you think about the placement of the “Next” and “Back” button on the screen?

PROBE: Did you ever have difficulty finding the buttons? [SHOW SCREENSHOT]



  1. Did you notice the ‘Save and continue later’ button in the bottom right hand corner of the instrument?

PROBE: Why/why not? [SHOW SCREENSHOT]



  1. In general, was it easy or difficult for you to figure out what to do next after each question?

PROBE: What was easy/difficult?



  1. [SCREENSHOT OF PARENT PROBE WITH “I DON’T KNOW” OPTION INQ015, PLQ060, FSQ040]. If you saw this message what would you do next?

    1. Was it easy or difficult to figure out what to do? Please describe what you would do.

    2. Did you see that a don't know option is now available?

    3. Would you change anything about the “I don’t know” option? [PROBE FOR SIZE, LOOK, COLOR]





Stopping/Starting the Survey

[STOP Rs IN BOTH GROUP 1 AND 2 BEFORE THEY SUBMIT THE LAST QUESTION OF THE SURVEY]

  1. If you needed to stop in the middle of the survey before you were finished, what would you do?

    1. Please show me what you would do if you needed to walk away from the survey now, say if you had to stop the survey to start making dinner.

    2. [IF THE SURVEY IS STILL OPEN HAVE R LOGOUT OR CLOSE BROSWER] Now that the survey is closed, if you wanted to come back and finish the survey, what would you do?

      • Please show me what you would do.

    3. Did you find that process easy or hard? Why?

    4. Did you choose to receive the temporary code at your email or a text to your mobile phone?

      • Why did you choose that option?

      • Was there any reason you did not choose the other option?

Closing and Incentive

Those are all the questions I have for you. Do you have any other comments about the survey or its design that you would like to share with me?



Thank you for your time.

[STOP RECORDING]

[GIVE INCENTIVE AND HAVE RESPONDENT SIGN RECEIPT]










The Early Childhood Longitudinal Study, Kindergarten Class of 2022-23 (ECLS-K:2023) Preschool Field Test Instruments Usability Testing




Attachment 4


Participant Consent Form




OMB #1850-0803 v.253





Submitted by

National Center for Education Statistics

U.S. Department of Education






August 2019

The Early Childhood Longitudinal Study, Kindergarten Class of 2022-23 (ECLS-K:2023)
Preschool Usability Testing Consent Form


CONSENT FORM


PARTICIPATION IN A QUALITATIVE RESEARCH PROJECT


USABILITY TESTING PURPOSE

You have been recruited to participate in the testing of new web survey instruments for parents of preschool aged children. Your participation is voluntary. The information obtained from this activity will guide design of the web surveys for the upcoming Early Childhood Longitudinal Study (ECLS) in an effort to improve participation rates and collect valid data on education across the United States.


AGENCY CONDUCTING THE STUDY

This project is being conducted by the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education. Representatives from Westat, a research organization, will administer the usability testing sessions.


CONFIDENTIALITY

The National Center for Education Statistics (NCES) is authorized to conduct the Early Childhood Longitudinal Study (ECLS) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The data are being collected for NCES by Westat, a U.S.-based research organization. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).


Audio and video recordings will be used to write a summary report of what we learn through this qualitative research project. Your name will not be included in the report, and the recordings will be destroyed once the report is written.


QUESTIONS

If you have any questions about the research, please call Liz Bissett, the Westat ECLS project director, at 301-294-4414.


CONSENT

I, __________________________________, agree to participate in the usability testing session for the ECLS web survey.





Shape1

SIGNATURE




File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified0000-00-00
File Created2021-01-15
© 2024 OMB.report | Privacy Policy