Redesign of the Head Start Family and Child Experiences Survey (FACES 2012)

attachment A.5

Cognitive Interview Protocol for Family and Child Experiences SURVEY (FACES) Pilot Study: Survey of Well-Being of
Young Children

I. Interviewer Introduction Script

The following scripts should not be read verbatim. You, as the interviewer, need to be familiar enough with the scripts below to introduce the think-aloud process in a conversational manner. Text written in italics is suggested content for you to be thoroughly familiar with in advance. You should project a warm and reassuring manner toward the participant to develop a friendly rapport and should use conversational language throughout.

Hello, my name is [NAME].

Thank you for taking the time to help us today.

I am working with the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (DHHS) on a research study called the Family and Child Experiences Survey Pilot Study. Within the past few weeks you completed the Survey of Well-Being of Young Children (SWYC) [over the phone/on the web]. I have some questions about how you answered the questions on the survey, and about your experience answering them. The things you tell me today will help us make our surveys better in the future. This should take about 30 minutes.

Your participation today is voluntary. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is 0970-0355 and it expires 1/31/2015.

I will be taking notes so that we can use what you tell me today to make these items better. Your information will be kept private. Only people working on the research study will know about your answers and ideas, but your name will not be connected to those answers and ideas. We will not talk with anyone outside the study team, including anyone at your child’s Head Start center. Your contributions today may be used only for research purposes and may not be used for any other purpose except as required by law. I would like to record what we are doing today so that if I miss something when I am writing up my notes I can go back to the recording to make sure I got it right. No one else will hear the recording. Is it okay if I record our conversation?

Do you have any questions before we get started?

If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview. After answering questions and giving further explanation, begin the interview with the first item.

II. Survey Items

1. Developmental Milestones

The survey started with questions about your child’s development, and asked you to tell us how much your child does certain things, like “tells you a story from a book or tv” and “draws simple shapes, like a circle or square.”Your answer choices were not yet, somewhat, and very much.

Have the participant’s answers ready if he or she would like to be reminded of them. Note where the participant chose “somewhat” and “very much”.

These questions are about your child's development. Please tell us how much your child is doing each of these things. If your child doesn't do something any more, choose the answer that describes how much he or she used to do it.

Select one per row

	Not Yet	Somewhat	Very Much
a. Talks so other people can understand him or her most of the time	0 	1 	2 
b. Washes and dries hands without help (even if you turn on the water)	0 	1 	2 
c. Asks questions beginning with "why" or "how" - like "Why no cookie?"	0 	1 	2 
d. Explains the reasons for things, like needing a sweater when its cold	0 	1 	2 
e. Compares things - using words like "bigger" or "shorter"	0 	1 	2 
f. Answers questions like "What do you do when you are cold?" or "when you are sleepy?"	0 	1 	2 
g. Tells you a story from a book or tv	0 	1 	2 
h. Draws simple shapes - like a circle or a square	0 	1 	2 
i. Says words like "feet" for more than one foot and "men" for more than one man	0 	1 	2 
j. Uses words like "yesterday" and "tomorrow" correctly	0 	1 	2 
i. Says words like "feet" for more than one foot and "men" for more than one man	0 	1 	2 
j. Uses words like "yesterday" and "tomorrow" correctly	0 	1 	2 
k. Stays dry all night	0 	1 	2 
l. Follows simple rules when playing a board game or card game	0 	1 	2 
m. Prints his or her name	0 	1 	2 
n. Draws pictures you recognize	0 	1 	2 
o. Stays in the lines when coloring	0 	1 	2 
p. Names the days of the week in the correct order	0 	1 	2 

Participant Probes:

1. I see that you responded “somewhat” to a few items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. In answering this question, what did “somewhat” mean to you? Why did you choose “somewhat” instead of “very much?”

2. I see that you responded “very much” to a few items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. In answering this question, what did “very much” mean to you? Why did you choose “very much” instead of “somewhat?”

3. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

4. Were there any questions you did not understand?

5. Is there anything else you’d like to tell me about these questions?

2. Preschool Pediatric Symptom Checklist

The survey next asked questions about your child’s behavior. Read questions to participant. Your answer choices were not at all, somewhat, and very much.

Have the participant’s answers ready if he or she would like to be reminded of them. Note where the participant chose “somewhat” and “very much.”

Does your child…	Not At All	Somewhat	Very Much
a. Seem nervous or afraid?	0 	1 	2 
b. Seem sad or unhappy?	0 	1 	2 
c. Get upset if things are not done in a certain way?	0 	1 	2 
d. Have a hard time with change?	0 	1 	2 
e. Have trouble playing with other children?	0 	1 	2 
f. Break things on purpose?	0 	1 	2 
g. Fight with other children?	0 	1 	2 
h. Have trouble paying attention?	0 	1 	2 
i. Have a hard time calming down?	0 	1 	2 
j. Have trouble staying with one activity?	0 	1 	2 

Is your child…	Not At All	Somewhat	Very Much
a. Aggressive?	0 	1 	2 
b. Fidgety or unable to sit still?	0 	1 	2 
c. Angry?	0 	1 	2 

Is it hard to…	Not At All	Somewhat	Very Much
a. Take your child out in public?	0 	1 	2 
b. Comfort your child?	0 	1 	2 
c. Know what your child needs?	0 	1 	2 
d. Keep your child on a schedule or routine?	0 	1 	2 
e. Get your child to obey you?	0 	1 	2 

Participant Probes:

1. I see that you responded “somewhat” to a few items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. In answering this question, what did “somewhat” mean to you? Why did you choose “somewhat” instead of “not at all” or “very much?”

2. I see that you responded “very much” to a few items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. What does “very much” mean to you? Why did you choose “very much” instead of “somewhat?”

3. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

4. Were there any questions you did not understand?

5. Is there anything else you’d like to tell me about these questions?

3. Parent’s Concerns

Next, we asked if you had any concerns about your child. Read questions to participant. Your answer choices were not at all, somewhat, and very much.

Have the participant’s answers ready if he or she would like to be reminded of them. Note where the participant chose “somewhat” and “very much.”

	Not At All	Somewhat	Very Much
a. Do you have any concerns about your child's learning or development?	0 	1 	2 
b. Do you have any concerns about your child's behavior?	0 	1 	2 

Participant Probes:

1. I see that you responded “somewhat” to [one or two], and I’d like you to tell me more about how you arrived at your answer. Read items to participant. In answering this question, what did “somewhat” mean to you? Why did you choose “somewhat” instead of “not at all” or “very much?”

2. I see that you responded “very much” to [one or two] items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. What does “very much” mean to you? Why did you choose “very much” instead of “somewhat?”

3. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

4. Were there any questions you did not understand?

5. Is there anything else you’d like to tell me about these questions?

4. Family Questions

Next, we asked you some questions about your family. Read questions to participant. Your answer choices were yes and no.

Have the participant’s answers ready if he or she would like to be reminded of them. Note where the participant chose “somewhat” and “very much.”

Alcohol, Tobacco, and Substance Use

	Yes	No
a. Does anyone smoke tobacco at home?	1 	0 
b. In the last year, have you ever drunk alcohol or used drugs more than you meant to?	1 	0 
c. Have you felt you wanted or needed to cut down on your drinking or drug use in the last year?	1 	0 
d. Has a family member's drinking or drug use ever had a bad effect on your child?	1 	0 
e. In the past month was there any day when you or anyone in your family went hungry because you did not have enough money for food?	1 	0 

Participant Probes:

1. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

2. Were there any questions you did not understand?

3. Is there anything else you’d like to tell me about these questions?

4. If web: would you have answered these questions differently if an interviewer had asked them instead of you answering by yourself?

5. If phone: Would you have answered differently had you been taking the survey on your own, such as on paper or on the web?

Depression

Over the past two weeks, how often have you been bothered by any of the following problems?	Yes	No
a. Having little interest or pleasure in doing things?	1 	0 
b. Feeling down, depressed, or hopeless?	1 	0 

Participant Probes:

1. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

2. Were there any questions you did not understand?

3. Is there anything else you’d like to tell me about these questions?

4. If web: would you have answered these questions differently if an interviewer had asked them instead of you answering by yourself?

5. If phone: Would you have answered differently had you been taking the survey on your own, such as on paper or on the web?

Relationship with Spouse/Partner

Lastly, we asked you two questions about how you get along with your spouse or partner. Read first question to participant. Your answer choices for this first question were No tension, Some tension, A lot of tension, and not applicable.

In general, how would you describe your relationship with your spouse/partner?

Select one only

 No tension 0

 Some tension 1

 A lot of tension 2

 Not applicable 3

The second question was…Read second question to participant. Your answer choices were No difficulty, Some difficulty, Great difficulty, and Not applicable.

Do you and your partner work out arguments with…

Select one only

 No difficulty 0

 Some tension 1

 A lot of tension 2

 Not applicable 3

Participant Probes:

1. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

2. Were there any questions you did not understand?

3. Is there anything else you’d like to tell me about these questions?

4. If web: would you have answered these questions differently if an interviewer had asked them instead of you answering by yourself?

5. If phone: Would you have answered differently had you been taking the survey on your own, such as on paper or on the web?

III. General Questions

1. Survey Mode

Probes for web participants:

1. Did you have any trouble accessing the survey from the e-mail link?

2. How did you take the survey? On a desktop computer, laptop computer, tablet, or smartphone? Does that device belong to you or to someone else, such as a family member, friend, neighbor, public library, or your child’s Head Start Center?

3. Were the question screens easy or difficult to read? What made them easy or difficult to read?

Probes for phone participants:

1. Did you take the survey on a land line or cell phone?

2. How easy or difficult was it to understand the interviewer?

Probes for all participants:

1. How often do you read your e-mail?

2. Do you own or have access to a:

a. Desktop computer?

b. Laptop computer?

c. Tablet?

d. Smartphone?

3. Are you able to access the Internet on any of these devices? Which ones?

4. Where were you when you took the survey? At home, at work, someplace else?

5. Would you prefer to answer these questions on the web or over the telephone?

6. Is there anything that would have made the survey easier for you to complete?

2. Language (asked of all respondents)

Participant Probe:

1. You completed the survey in [English/Spanish]. How well would you say you speak [English/Spanish]: Very well, well, or not very well? If not very well: were any of the questions difficult to answer?

IV. Debriefing and Thank You for Participation

Thank participant for his/her time.

Before we finish, I’d like to hear any other thoughts you have about the survey.

Is there anything else you would like to tell me about answering the survey questions?

Is there anything you would like to ask me about what we did today? [Answer participant questions]

Thank you for helping us to improve our questionnaire.