NCER- NPSAS Grant Study – Connecting Students with Financial Aid (CSFA) 2017 Cognitive Testing

National Center for Education Statistics

Attachments I - V

NCER- NPSAS Grant Study – Connecting Students with Financial Aid (CSFA) 2017 Cognitive Testing

OMB# 1850-0803 v.191

February 2017

Attachments:

Attachment I: Recruitment Procedures

Attachment II: Consent to Participate in Research

Attachment III: Sample Text for Interview Scheduling and Confirmation

Attachment IV: Interview Protocol

Attachment V: Interview Guide and Notes Form

Attachment I

Recruitment Procedures

Recruitment of Respondents

Respondents will be recruited primarily by Co-Principal Investigators, Dr. Bridget Long and Dr. Eric Bettinger using a variety of methods including networking referrals and invitations distributed via email, public postings, and social media announcements. Additional respondents may be recruited by Project Manager Lena Eisen.

Scheduling of Respondents:

Interested participants will be asked to contact the local researcher/interview administrator at their designated email or phone. Prospective participants will receive an email (Attachment III; Part A) describing the purpose of the survey interview process and details of participation, including the time commitment and consent requirement. After receiving the informational email, participants will be asked to select an interview time via an online sign-up tool. Participants will be scheduled on a first-come, first-served basis and sent a confirmation email for their scheduled time and, if applicable, location (Attachment III; Part B).

Attachment II

Consent to Participate in Research

Title of Research: Connecting Students with Financial Aid (CSFA) 2017– Cognitive Interview (OMB# 1850-0803)

Introduction and Purpose

You are being asked to participate in a cognitive interview carried out by Harvard University and Stanford University for a project funded by the National Center for Education Research (NCER) and sponsored by the National Center for Education Statistics (NCES), both part of the U.S. Department of Education. The purpose of the cognitive interview is to obtain your feedback on some of the questions that will be used in the survey portion of the Connecting Students with Financial Aid (CSFA) study. Feedback gathered from these interviews may be used to revise, improve, or confirm the clarity of the final study survey.

Procedures

You are one of up to 40 individuals who have been invited to participate in an interview during which you will be asked to answer a series of survey questions while discussing your understanding and reaction to the language used in those survey questions and their response options.

Notes will be taken during your interview to help us write a report summarizing the results of survey responses and our interview discussion. Your name will never be used in the report that we write.

Study Duration

Your participation in the interview will likely take about 30 minutes, and no more than 60 minutes.

Possible Risks or Discomforts

We do not anticipate that any of the discussion topics will make you uncomfortable or upset. However you may refuse to answer any question or take a break at any time.

Benefits

Your Benefits There is no monetary compensation for participating in the survey interview. However, light snacks and beverages may be provided if the interview is conducted in person.

Benefits for Other People We hope that this survey and interview process will help us improve students’ understanding of our survey questions.

Confidentiality

We are conducting these cognitive interviews for the National Center for Education Statistics (NCES) of the U.S. Department of Education. This study is authorized by law under the Education Sciences Reform Act (20 U.S.C., § 9543). Your participation is voluntary. Your responses are protected from disclosure by federal statute (20 U.S.C., § 9573). All responses that relate to or describe identifiable characteristics of individuals may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law.

Future Contacts

We will not contact you in the future as part of this interview.

Your Rights

Your decision to take part in this research study is completely voluntary. You can refuse any part of the study and you can stop participating at any time.

Your Questions

If you have any questions about the study, you may call Dr. Bridget Long at 617-495-3401 or Tracy Hunt White from the National Center for Education Studies (NCES) at 202-245-6507.

YOU WILL BE GIVEN A COPY OF THIS CONSENT FORM TO KEEP.

Your signature below indicates that you have read the information provided above, have received answers to your questions, and have freely decided to participate in this research. By agreeing to participate in this research, you are not giving up any of your legal rights.

________________ Date _____________________________________ _ Signature of Participant

____________________________________ __

Printed Name of Participant

I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above-named individual.

______________ Date _____________________________________ _ Signature of Person Obtaining Consent

____________________________________ __

Printed Name of Person Obtaining Consent

Attachment III

Cognitive Interview Sample Recruitment Email or Text

Part A: Response to initial inquiry from prospective participant

Greetings,

Thank you for your interest in participating in a survey and interview in support of the Connecting Students with Financial Aid (CSFA) study.

The goal of the interview is to get your feedback on a survey, which will be used in a study that aims to increase financial aid application completion and college persistence. The survey and interview will likely take about 30 minutes, but no more than 60 minutes.

If you are currently enrolled at least part time in a postsecondary education program and wish to participate in this research opportunity, please visit [INSERT ONLINE SCHEDULING TOOL LINK] to schedule your interview. You will receive email confirmation of your interview appointment.

No preparation is necessary for your participation in this interview and your participation is completely voluntary. Please contact me at bridget_long@gse.harvard.edu or 617-495-3401 if you have any questions.

Thank you again,

Bridget Long

Principal Investigator, CSFA Study

Part B: Confirmation of participation location and time

Greetings,

Thank you for your interest in participating in a survey and interview in support of the Connecting Students with Financial Aid (CSFA) study.

Our meeting is scheduled for [INSERT DATE] at [INSERT TIME]. We will meet at [INSERT LOCATION] [ALTERNATIVE LANGUAGE: I will call you at the phone number you provided.] The survey and interview will likely take about 30 minutes, but no more than 60 minutes.

The goal of this session is to get your feedback on a survey, which will be used in a study that aims to increase financial aid application completion and college persistence.

Your participation is completely voluntary. Please contact me at XXXX if you have any questions.

Attachment IV

Interview Protocol

Introduction and Welcome

Hello, my name is [INSERT INTERVIEWER NAME], and I’ll be conducting your interview today and taking notes. I am part of the Connecting Students with Financial Aid Study, a research program striving to increase completion of the Free Application for Federal Student Aid (FAFSA) and the likelihood of college persistence, particularly as a full-time student. Today, I hope to interview you as you complete a survey we plan to give to participants in our study.

Informed Consent

We are conducting interviews with you and other participants to evaluate the overall content and language within the survey we plan to use. The interview will likely take about 30 minutes, but no more than 60 minutes. We are particularly interested today in learning more about your understanding and comprehension of the questions in this survey and whether you have any suggestions about how we might improve it.

I will be taking notes so that we can review your ideas and comments after this interview is over. We will prepare a report that does not single out any one person but instead reports what we heard from all participants as a whole. No names will be mentioned in the report. Before we begin, I would like to describe what you can expect during your interview.

Interview Format

• I will be asking you to complete a short survey and may ask about some of your answers in more detail.

• I may ask you to read a question aloud during the survey or ask you to verbalize your answer as you make a selection.

• I may ask follow up questions to help us learn more about how you arrived at your answer and whether you have suggestions that would improve the question or response options.

I will begin the survey by asking you to answer some background questions about yourself and your educational background. I would like to remind you that your name will not be associated with what you say and that our report will not identify any names of persons participating in these interviews. This is not a test; there are no right or wrong answers. Your experiences and opinions are important to us, and we hope that you will feel free to share them openly. We thank you for your time and insight. Your participation will help to improve the project.

If you have no objections, we will start now.

Begin Survey

Note: The interviewer may ask follow up / general probe questions when the respondent expresses hesitation or suggests having difficulty with a survey item. Continue observing and listening, probing when appropriate, and noting respondent feedback as the interview progresses.

Examples of General Probes

• What do you think the question was asking?

• How would you rephrase the question in your own words?

• How easy or difficult was it to understand the question?

• [If answering a multiple-choice question:] Were you satisfied with the answer choices?

  • [If no:] What additional choices would you recommend be included?

• Did you find any of the language confusing?

• Do you see ways to make the questions or answer options more clear?

• How did you feel when answering the question?

• What was the reason for that comment?

• Probe for examples.

• Can you tell me more about that?

• You seemed to hesitate/take a moment before giving/entering your response. Can you tell me what you were thinking about?

• What do you mean by…?

• Is there any additional information that we should have provided to help understand the message?

• Does this term/question sound OK to you, or would you prefer something different?

Post-Survey

• Thank you for completing the survey. Do you have any additional questions or suggestions before we end our session?

• We really appreciate your involvement in this process. Your participation is very helpful as we prepare for the future use of the survey.

Attachment V

Interview Guide and Notes

2017 Connecting Students with Financial Aid (CSFA)

Cognitive Interview Notes

Interview ID		Date of Interview
Interviewer		Time of Interview
Consent Form Received and Signed
Welcome and Introduction Provided
Any questions about this process before starting? (Describe Below)

Background Questions:

PQ 1. Are you currently attending college at least part-time?
PQ 2. Have you ever completed a financial aid form?
PQ 3. Have you worked with other students trying to access higher education?

Start Survey: Online Survey will employ automatic skip logic to select appropriate question ordering

Q1. (Consent Page) Not relevant for testing.
Q2. Did you apply for financial aid from the government to help you with college expenses for the 2017-18 school year (i.e., this year)?
Q3 Did you apply for financial aid from anyone else (a non-government source) to help you with college expenses for the 2017-18 school year?
Q4 If you have ever applied for government financial aid, how did you learn about the process of applying? (Check all that apply) Options include: Friend, Relative, Communication or mailing from my school, College Faculty or Staff member, Community Organization, Online Website, Financial advisor, and I do not know how to apply for financial aid

Skip Logic: If Q2 = No (Respondent did NOT apply for government aid for 17-18)

Q5 Please choose the statements that apply to you: List of possible answers on the survey for why the student did not apply.
Q6: Do you know what the FAFSA is?

Skip Logic - If Q2 = Yes (Respondent DID apply for government aid for 17-18)

Q7: We’re interested in your experiences with applying for financial aid. Did you receive assistance from another person to complete the FAFSA the last time you did so? If yes, please indicate from whom.
Q8: Please tell us how you submitted your financial aid form.
Q9: We’re also interested to know your opinions about the financial aid process. Please select one response to each statement. (Strongly Disagree, Disagree, Neither, Agree, Strongly Agree)

All Respondents

Q10: How many credits are you taking this term (Fall 2017)
Q11: Are you taking more, fewer, or the same credits as the last time you enrolled in college courses?

Skip Logic - Ask if Q10 = fewer than 12 credits but not zero

Q12: Why are you taking fewer than 12 credits?
Q13: If your financial aid had been doubled, not including loans, would you have taken more credits or courses?
Q14: How many credits do you plan to take the next term?

Skip Logic – Ask if Q 10 = 12 credits or more

Q15: What made you decide to take 12 or more credits this term?
Q16: How many credits do you plan to take the next term?

All Respondents

Q17: We’re also interested to know how financial aid affects your decisions about college. Please select one response for each statement (SD to SA)
Q18: Have you taken out loans to pay for college? If so, what kinds of loans?

Skip Logic – If Q18 = Yes/Selected

Q19: How did you decide whether to take out a loan?
Q20:We’re interested to know what you think about your loans – Please select one response to each statement (SD – SA)

Skip Logic – If Q18 = No

Q21: How did you decide to avoid taking out a loan?
Q22: We’re interested to know what you think about college student loans. Please select one response to each statement (SD to SA)

Additional Notes

Summary Notes