Administration for Children and Families Youth Programs –Youth Participant Entry and Exit Survey Pretest

OMB No.: 0970-0355

Expiration Date: 10/31/2027

Instrument 9.

Youth Focus Group Discussion Protocol

THE PAPERWORK REDUCTION ACT OF 1995

This collection of information is voluntary and will be used to provide the Administration for Children and Families with information to help refine the performance measures. Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-0355, Exp: 10/31/2027. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Melissa Thomas at mthomas@mathematica-mpr.com

Summer 2025

Entry and Exit Surveys: Focus Group

Discussion Protocol

Check in participants

1. RECORD ATTENDANCE IN A STAND-ALONE FILE THAT CAN BE DELETED/SHREDDED

2. CONFIRM CONSENT/ASSENT FORMS RECEIVED

3. ONCE EVERYONE HAS ARRIVED, START THE INTRODUCTION

Introduction

Thanks for helping us today! My name is [NAME], and this is my colleague, [NAME]. We work for a research company called Mathematica. Mathematica was hired by the Administration for Children and Families, which is an agency within the U.S. Department of Health and Human Services, part of the federal government that works to support the health and well-being of people, including youth.

We’re here today because we want to get your feedback and thoughts about some questions that could be used on surveys with people your age who are about to participate in or have already participated in a program like [PROGRAM NAME]. We want to learn how well these survey questions capture the experiences and characteristics of youth in grades 6-12. Your participation today will help us make the questions better for when other youth complete the surveys. You can help us make sure the questions are clear and help us learn if there may be better ways to ask the questions. Although we don’t expect you to speak for everyone, we consider each of you an expert in how well these questions work for people your age. Thank you again for your help.

Our session today will take no more than an hour. I’m going to ask you to complete two brief surveys. After completing each survey, we will have a group discussion so I can hear your thoughts and opinions about the surveys.

Informed assent information: collected prior to arrival

You all should have completed a form before arriving, noting that you agreed to participate. Before we get started, I want to make sure all of you know your rights for participating in this discussion today.

• First, your participation is completely voluntary. That means you can decide if you want to participate or not participate and can stop participating at any time without any reason or consequences

• Your responses in the discussion will not be shared with anyone outside of the study team.

• During the discussion, you can decide to not answer any questions that I ask the group.

• We ask that everyone in this group respect each other’s privacy by NOT talking about the details of our discussion with anyone outside the group. However, we want everyone to understand that because this is a group discussion, we cannot guarantee your privacy of what you say in front of each other.

• When we share what we learned during today’s discussion we will NOT use your name or any other information that could identify you. We will only share information about you if required by law—that means that if we learn that you have been hurt or are in danger or if you tell us that you plan to seriously hurt yourself or someone else, then, by law, we must report it to the appropriate authorities to make sure you are safe.

• We would like to audio/video-record and take notes during our discussion. This is to help us remember all of your thoughts and ideas to make improvements to the survey questions. We will not share this recording with anyone outside of our study team and will destroy it once the study is finished. If anyone does not want to be recorded, we will not record the discussion. Does everyone agree to be recorded? CONFIRM EVERYONE’S ANSWER. IF ANYONE DISAGREES, SESSION WILL NOT BE RECORDED.

Does anyone have any questions? PAUSE FOR QUESTIONS.

Administer Entry Survey

Let’s get started. First, I would like you to complete this Entry Survey on paper. Do not put your name on the paper. You do not have to give your personal answers to these questions. We are more interested in hearing your feedback on how clear the questions are, how youth may think about them when responding, and how they could be improved. You can give fake answers, but we ask that you read the questions and responses carefully and think about how someone would answer them. As you read through the survey, we ask you to please circle or underline any words or phrases that are confusing or where you would use different words.

Please note we will collect your surveys, but we will not look at your specific responses. We will only review the surveys to look at any notes you made.

Are there any questions before we begin? PAUSE FOR QUESTIONS.

Go ahead and begin answering the Entry Survey questions. When you are done, please put your pen/pencil down so I know you’ve completed. I will be recording how long it takes to complete the survey.

[COLLECT COMPLETED SURVEYS WHEN COMPLETE]

Discussion group guidelines

We understand that participating in a group like this might be new to some of you. Before we begin our discussion, we want to establish a few group rules. Please feel free to add to this if you have other things we should consider as a group:

• There are no right or wrong answers to our questions. We just want to know your true opinions—all feedback is welcome!

• Remember to be respectful of everyone’s opinions, even if they are different from your own. We are really interested to hear if you have different thoughts and opinions about these questions, and welcome that discussion.

• Please do not share what we discuss today outside of this room. It is up to each of you to respect one another’s privacy and honor this request. 

1. Let’s go around and each say your first name and your favorite (movie, food, TV show). We will not record this part of the discussion.

2. Some of the topics we talk about today may be things you are not used to talking about. If we ask anything that you don’t want to answer or that makes you feel uncomfortable – please just let me know and we can skip that question.

3. As we go through specific questions, you don’t need to tell us what your answer is or would be. Our goal for today is to get your feedback on how these questions are worded and your thought process for these questions, so you can help us improve the questions.

4. Do you have any questions before we begin or anything to add to our group rules?

IN-PERSON FOCUS GROUP DISCUSSIONS

BEGIN DISCUSSION. REMIND PARTICIPANTS SESSIONS ARE BEING RECORDED. CHECK THAT RECORDING HAS BEGUN.

Entry Survey Discussion

We have some questions about the general process of answering these survey questions:

1. Overall, do you think people your age will feel comfortable providing honest answers to these questions? Why or why not?

2. Do you think it will be easy or difficult for people your age to answer these questions? If difficult: What do you think makes it difficult? Which questions were difficult to answer?

3. Do you think these questions make sense, to you and others your age? Which ones don’t make sense for others your age?

4. Were there any words or phrases that were confusing or seemed overly complicated? If so, which ones? What was confusing about them? What words or phrases would you use instead?

5. Are there any words or phrases that seemed out of date or not the way people your age might phrase things? If so, which ones? What words or phrases would you use instead?

6. Was it clear when you were supposed to select only one response or when you were supposed to select multiple responses?

7. Were there any questions where the answer choices didn’t seem to match what the question asked? If so, which ones? How would you change the question or response options to make more sense?

8. Were there any questions that didn’t seem to have an answer choice that could apply to all people your age? If so, which ones? What answer choices would you add?

9. What did you think about the length of the survey? Was it too long? Too short?

10. What did you think of the order in which the questions were asked? Would you change any order of the questions to make it easier to understand?

Now, let’s talk about some specific questions. Please share your thoughts on the wording of the following questions:

11. What age are you today?

SELECT ONLY ONE ANSWER

□            10 years

□            11 years

□            12 years

□            13 years

□            14 years

□            15 years

□            16 years

□            17 years

□            18 years

□            19 years

□            20 years

□            21 years

12. Are you currently…?

SELECT ALL THAT APPLY

□            In foster care

□            Unstably housed (moving from place to place), living outside (in a tent or in a car), in a hotel, or in

an emergency shelter

□            In a juvenile detention center, juvenile group home, and/or under the supervision of a probation

officer

□            None of the above

13. (PREP ONLY) In the past 30 days did you drink alcohol or use drugs that you didn’t get from a doctor?

SELECT ONLY ONE ANSWER

□            Yes

□            No

14. (SRAE ONLY) In the past 30 days did you...

    • A. drink alcohol (more than a few sips), including beer, wine, liquor, or lean?

    • B. smoke cigarettes or used other tobacco products?

    • C. vape or use electronic vapor products (such as JUUL, Vuse, MarkTen, and blu)?

    • D. use marijuana (also called pot or weed)?

    • E. use any other drugs that you didn’t get from a doctor?

15. Did you skip any questions? (HIGH SCHOOL AND OLDER VERSION ONLY: Did you skip any questions on sex: Q8-Q12?) If so, why did you skip this/these question(s)?

[REVIEW COMPLETED SURVEYS FOR ANY ISSUES, SUCH AS SELECTING RESPONSES WITHOUT MARKING “X,” SKIPPING QUESTIONS, OR CHANGE IN RESPONSE.]

ASK ABOUT EACH ISSUE: Why did you mark it this way? Why did you skip this question?

16. Is there anything that we didn’t discuss that would improve this question or the responses offered?

Administer Exit Survey

Now, I would like you to complete this Exit Survey on paper. The first few questions are similar to the first survey, but pretend it’s a month or so later, and you have completed a program like [PROGRAM NAME]. As with the Entry Survey, do not put your name on the paper. You do not have to give your personal answers to these questions – remember, we are more interested in hearing your feedback on how clear they are, how youth may think about them when responding, and how they could be improved. You can give fake answers, but we ask that you read the questions and responses carefully and think about how someone would answer them. As you read through the survey, we ask you to please circle or underline any words or phrases that are confusing or where you would use different words.

Please note we will collect your surveys, but we will not look at your specific responses. We will only review the surveys to look at any notes you made.

Are there any questions before we begin? PAUSE FOR QUESTIONS.

Go ahead and begin answering the Exit Survey questions. When you are done, please put your pen/pencil down so I know you’ve completed. I will be recording how long it takes to complete the survey.

[COLLECT COMPLETED SURVEYS WHEN COMPLETE]

Exit Survey Discussion

We have some questions about the general process of answering these questions:

1. Overall, do you think people your age will feel comfortable providing honest answers to these questions? Why or why not?

2. Do you think it will be easy or difficult for people your age to answer these questions? If difficult: What do you think makes it difficult?

3. Do you think these questions make sense, to you and others your age? Which ones don’t make sense for others your age?

4. Were there any words or phrases that were confusing or seemed overly complicated? If so, which ones? What was confusing about them? What words or phrases would you use instead?

5. Were there any questions where the answer choices didn’t seem to match what the question asked? If so, which ones? How would you change the question or response options to make more sense?

6. Are there any words or phrases that seemed out of date or not the way people your age might phrase things? If so, which ones? What words or phrases would you use instead?

7. Was it clear when you were supposed to select only one response or when you were supposed to select multiple responses?

8. Were there any questions that didn’t seem to have an answer choice that could apply to all people your age? If so, which ones? What answer choices would you add?

9. What did you think about the length of the survey? Was it too long? Too short?

10. What did you think of the order in which the questions were asked? Would you change any order of the questions to make it easier to understand?

Now, let’s talk about some specific questions. Please share your thoughts on the wording of the following questions:

11. As a result of being in the program, do you plan to abstain from sexual intercourse (not have sexual intercourse) in the next three months?

SELECT ONLY ONE ANSWER

□            Yes        

□            No     

□            Not sure            

12. How important are each of these reasons in your decision about having or not having sexual intercourse in the next three months?

SELECT ONLY ONE ANSWER PER ROW
	Very important	Somewhat important	Not important
a. how it might affect your plans for the future
b. the risk of getting a sexually transmitted infection (STI) or sexually transmitted disease (STD)
c. the risk of getting pregnant or getting someone pregnant

13. (PREP ONLY) What did you think about the note in the question 10? What timeframe did you think about when answering Q10.b?

Has being in the program made you more likely, about the same, or less likely to… (Note: If the program has not affected your likelihood to do the following, choose “About the same”.)

SELECT ONLY ONE ANSWER PER ROW

	More likely	About the same	Less likely
a. have sexual intercourse in the next three months?
b. have sexual intercourse while you are under the influence of alcohol or drugs?
c. use (or ask your partner to use) a condom if you were to have sexual intercourse in the next 3 months?
d. use (or ask your partner to use) birth control OTHER than condoms if you were to have sexual intercourse in the next 3 months? By birth control, we mean methods that can prevent pregnancy, like using birth control pills, the shot, the patch, the ring, IUD, or implant.

14. What did you think of the response options in Q11?

How often in this program…

SELECT ONLY ONE ANSWER PER ROW
	Most of the Time	Some of the Time	None of the Time
a. did you feel the material presented was clear?
b. did discussions or activities help you to learn program lessons?
c. did you feel respected by the facilitator?

15. Some questions had the response options: More likely, About the Same, or Less likely. Were these response options clear? Are there any response options that you would add here?

[REVIEW COMPLETED SURVEYS FOR ANY ISSUES, SUCH AS SELECTING RESPONSES WITHOUT MARKING “X,” SKIPPING QUESTIONS, OR CHANGE IN RESPONSE.]

ASK ABOUT EACH ISSUE: Why did you mark it this way? Why did you skip this question?

16. Is there anything that we didn’t discuss that would improve this question or the responses offered?

Wrap-up

Those were all the questions I had, but I want to make sure we don’t miss any more feedback from you to help us improve these questions.

Please take a moment to look back through the survey questions. Do you have any additional comments? Did we miss anything in our discussion that you would like to mention now? This can be about your general thoughts, feedback, or opinions on the surveys and the experience of answering the questions, rather than about specific questions. PAUSE FOR COMMENTS.

Thank you so much for joining the discussion today and sharing your thoughts with us.

Thanks, and have a wonderful [day/afternoon/evening]!