2015-16 National Postsecondary Student Aid Study (NPSAS:2016) Graduate Student Cognitive Testing

National Center for Education Statistics

Attachments III, IV, and V

Screener Interview Materials

2015-16 National Postsecondary Student Aid Study (NPSAS:16) Graduate Student Cognitive Testing

OMB# 1850-0803 v.107

Attachment III – Screener

Attachment IV – Introduction Script, Interview Questions, and Debriefing

Attachment V – Interview Probes

July 2, 2014

Attachment III

Eligibility Screening Questions

Database Call Script:

Hello, this is . I’m calling from Shugoll Research, an independent market research company, on behalf of the National Center for Education Statistics which is part of the U.S. Department of Education. We are conducting a very brief survey among students and would greatly value your opinions. This is strictly market research; there is absolutely no sales effort involved. I’d like to ask you a few questions to determine if you are eligible to participate. The screening is expected to take approximately 3-5 minutes.

1. Did you attend graduate school at any time between July 1, 2013 and June 30, 2014?

Yes

No (Ineligible)

2. What is the name of the college, university, or trade school you attended at any time between July 1, 2013 and June 30, 2014 for graduate school?

3. Which of the following were you working on at [graduate school name] in the 2013-2014 academic year (July 1, 2013 - June 30, 2014)?

Master's degree

Doctoral degree--research/scholarship (for example, Phd, EdD, etc.)

Doctoral degree--professional practice (including: chiropractic, dentistry, law, medicine, optometry, pharmacy, podiatry, or veterinary medicine)

4. What is your specific degree program and major area of study? (recruit a mix)

5. Did you complete your graduate degree at any time during the 2013-2014 academic year?

Yes

No

6. What is your age? Are you…

Under 18 (Ineligible)

18-24

25-29

30-34

35-39

40 or older

7. What is your name?

8. Are you able to attend an interview in either of the following locations?

Bethesda, Maryland

Alexandria, Virginia

9. What days of the week and times of day would you be available to attend an interview?

10. Are you of either Hispanic or Latino origin?

Yes

No

11. What is your race?

Choose one or more.

White

Black or African American

Asian

American Indian or Alaska Native

Native Hawaiian or Other Pacific Islander

12. What is your gender?

13. On a scale of “1” to “10,” where a score of “10” means that the statement describes you extremely well and “1” means that the statement does not describe you at all, how well does the following statement describe you? (CIRCLE ONE NUMBER PER ROW)

	CIRCLE ONE
	EXTREMELY WELL									NOT AT ALL WELL
I feel comfortable expressing my opinion in a group or interview	10	9	8	7	6	5	4	3	2	1

NOTE TO INTERVIEWERS: IT IS EXTREMELY IMPORTANT THAT RESPONDENTS IN THIS STUDY BE ABLE TO PROVIDE AND CLEARLY EXPRESS INFORMATION ABOUT THEIR EXPERIENCE. IF RESPONDENT CANNOT OR WILL NOT EASILY GIVE A ONE TO TWO SENTENCE UNPROMPTED ANSWER IN WELL UNDERSTOOD ENGLISH, PLEASE TERMINATE.

IF ELIGIBLE: INVITATION SCREENER

We are conducting one-on-one interviews in our office with people like you to evaluate some questions that will be used for refining an education survey with special topics for graduate students. This is not a sales meeting of any kind. The interview is scheduled for __________ at (TIME). The interview will take about one and a half hours to complete and participants will receive $40 after the interview. Are you available to participate?

	CIRCLE ONE
Yes	1	(GIVE DIRECTIONS)
No	2	(THANK AND TERMINATE)

INTERVIEWER: RECORD APPOINTMENT DATE AND TIME.

RESPONDENT NAME:

DATE RECRUITED: RECRUITED BY:

DATE CONFIRMED: CONFIRMED BY:

NOTE TO RECRUITER:

• Valid proof of identification for all participants is required.

• Emphasize the amount of time (1.5 hours) the discussion will take, the beginning and end times, and the location the discussion will take place.

• Do not schedule anyone who is at all reluctant about the time, is hesitant about participating in a discussion, talks too low or speaks unclearly. That person will definitely not be considered a valid respondent and will not receive any incentive.

Attachment IV

NPSAS:16 Graduate Student Cognitive Interview Introduction Script, Interview Questions, and Debriefing Procedures

INTRODUCTION SCRIPT

WELCOME participant and thank them for their time.

INTRODUCE SELF: (Name/Shugoll Employee.)

READ OR PARAPHRASE: This study is being carried out by Shugoll Research on behalf of the National Center for Education Statistics, which is part of the U.S. Department of Education. We are currently planning to conduct a survey to learn about the experiences of individuals who were in graduate school in the 2013-2014 academic year, whether or not they are still enrolled. Today, I’m hoping to get your honest feedback on information about issues related to graduate study.

During the interview, I will read out the survey questions and ask you to respond aloud.

As you go through the survey, I will ask some questions about your answers, what you understood the question to mean, and whether the questions reflect your actual circumstances.

The interview will be recorded so that I can concentrate on what you are saying rather than having to take notes, and to make sure that we don’t miss anything that you say. The recording will be destroyed once the project is completed.

There are observers of this interview present in the observation room or listening on the phone. These people are not here to observe you in particular but to learn from what you say about your experience of answering the questions.

However, I can assure you that everything you say will only be used for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S. Code, Section 9573). You will not be identified by name in any report we produce.

Your participation in this interview is very important because it will help us improve an education study with topics related to graduate students. Once we start, the interview will last about an hour and a half. When we are done, I will give you $40 cash to thank you for your time.

Do you have any questions?

WRITE DOWN QUESTIONS:

PLEASE READ AND SIGN THE “CONSENT TO PARTICIPATE IN RESEARCH” FORM (see Attachment II).

IF CONSENT WAS GIVEN, START RECORDER.

There are no right or wrong answers. This is not a test. It is your experiences, views, and opinions that I am interested in. Some of the questions I ask I will not address further but other questions I may ask about in more detail. So I may ask you what you understood a particular term or phrase to mean, or if you could rephrase a question in your own words. However, please feel free to tell me anything that comes to mind or ask me anything you are unclear about.

CONDUCT INTERVIEW. ASK UNSCRIPTED PROBES AT QUESTIONS WHERE THE RESPONDENT HAS GIVEN SOME INDICATION OF DIFFICULTY WITH THE QUESTIONING BASED ON ANSWERS/BEHAVIORS, AS THE INTERVIEW PROGRESSES.

1. Graduate degree program in the 2013-2014 school year:

1. What degree program name/type and including field of study were you enrolled in?

   1. if in a dual program, please describe;

   2. if in a master’s degree program…is this on the way to a PhD in the same program? Is there a definite point at which the program becomes a PhD program?

   3. Is this an accelerated program? Please describe.

2. At what point are you in the progress of your program? (for example, year of program)

3. Were you enrolled full-time or part-time in the 2013-2014 school year? How about in the prior years for this program?

4. How many years are prescribed for completion of this program? How many years do you expect it will take to complete the program? [PROBE: What were you thinking of in answering this question?]

5. Were the requirements of your program made clear to you either in program materials or by program faculty or representatives? [PROBE: Can you tell me more about that?]

   1. Besides coursework, are there additional requirements needed to complete your graduate program? What are those requirements?

   2. Have you participated outside of the classroom in:

      1. Research (non-assistantship)

      2. International Experience

      3. Traineeship

      4. Internship/Practicum

If yes, to any of the above, please describe.

6. Do you plan to continue your graduate program in the 2014-2015 school year?

   1. If no, what were the major factors in your decision to leave your graduate program? At what point in your program did you leave?

   2. For all: Other than normal breaks, has your enrollment in your graduate program been continuous since you first started the program? If you had any significant periods of stopping out, what were the reasons?

2. Enrollment/Application:

1. Why did you choose to pursue graduate school?

2. Was there an entrance exam requirement for your program/which one?

3. How did you prepare for the exam? (if at all)

3. Financial Aid:

1. How much is tuition for your graduate program? Are there additional fees for the program? What are they?

2. Have you taken out any loans?

   1. What kinds? (for example, federal or private). For each type of loan taken out, why did you choose to take out that particular type of loan?

   2. Do you know what your interest rate is on those loans? What is it? (If unsure, provide your best guess).

   3. What are the repayment requirements for these loans?

   4. Do you have a strategy in place or in mind to pay back these loans?

   5. What can you tell me about income-based repayment plans? Do you see yourself making use of this option?

   6. Are you part of a loan forgiveness plan or expect to be part of one after your graduation? Please describe.

   7. Will this debt have an impact on your career decisions/trajectory? How so?

   8. Will this debt have an impact on other aspects of your life (e.g. family, housing, standard of living)? How so?

   9. What kind of information (from your school, from lenders, etc.) have you been given regarding your loans?

3. Other than loans, what, if any, other forms of financial assistance are you receiving? Please describe.

Specifically,

4. Do you have a…

   1. Fellowship?

   2. Assistantship of any kind? Other arrangement? (what kind is it?)

   3. Employer paid-tuition

   4. Veteran’s education benefits? Which program?

Is yes, please describe.

Specifically:

5. What is included in the compensation package? (if not mentioned, is there a stipend? Is tuition covered? In full or in part?)

6. To your knowledge, is any portion of your compensation taxable? Which portion?

7. Is it clear where this money is coming from or how it is disbursed? If you receive money directly, how do you receive it? Please describe.

8. What duties are required? Hours per week?

9. Does this assistance cover things like health insurance, fees, housing, books & supplies? Other?

5. Have you received any other grants/scholarships not previously mentioned?

   1. Source?

   2. How much?

   3. How do you receive the money? Directly? Through your school?

6. Do you use any money from a 529 college savings plan to pay for your tuition? (how much?) Have you ever borrowed against your 401k? Have you taken out a home equity line of credit to pay for graduate school?

7. Have you ever heard of/considered “crowdsourcing” arrangements? If so, what have you used it for?

8. About how much money (if any) do you pay each semester for your education, minus any assistance? [PROBE: How did you go about answering.]

4. Income & Expenses

1. What was your income in the 2013 calendar year?

2. What jobs have you held in graduate school (do not include assistantships)?

   1. Is this job related to your enrollment in graduate school?

   2. Have you used the skills you obtained in graduate school for the job?

   3. Do you work full-time or part-time? About how many hours per week?

   4. Are these jobs held during the school year and/or during breaks (for example, in the summer)?

3. While in graduate school, have you regularly given any friends or family who do not live with you more than $50/month to help them out? Please describe.

4. Do you have enough money saved to cover 3 months of expenses without additional pay or assistance? If yes, what about 6 months?

5. What is your current living arrangement? For how long have you been in this living arrangement?

6. If married or in domestic partnership…

   1. Is your spouse or partner currently employed?

   2. What percentage of your expenses is covered by your spouse or partner’s income?

7. In your estimation, how much money in wages are you foregoing each year to be in graduate school? Is there anything else you are foregoing in order to pursue your graduate program? [PROBE: What were you thinking of in answering these questions?]

8. In your estimation, what annual salary do you expect to make upon completion of your graduate program?

9. How much do you spend each semester on…

   1. Books & supplies? Where do you get your books from? Which supplies are you including?

   2. Research expenses?

   3. Conference attendance & professional development?

   4. Does your school/department/program help you pay for these expenses? How so?

10. Has your socioeconomic status (SES)/standard of living changed since enrolling in graduate school?

11. Do you expect your SES to change upon completion of graduate school? How so?

5. Career Preparation

1. In thinking about your graduate program as a whole, do you feel it is doing a good job of preparing you for your career? Please describe.

2. Is there anything you feel is lacking in your graduate training?

3. Are career planning resources available to you as a graduate student?

   1. Do you use these resources? How so?

   2. Are these resources relevant for your intended career path?

6. Background

1. Where were you born? Are you a U.S. citizen? Are you a citizen of any other country?

2. Do you speak any languages other than English?

   1. How well do you speak/read/write any other languages?

   2. What is your best language?

   3. What was your language of instruction before college?

   4. What language do you expect to use primarily in your future career?

DEBRIEFING PROCEDURES

• Ask for any last observations/comments/concerns.

• Stop the digital audio recorder.

• Thank the participant for coming in and for helping test the interview.

• Give them the $40 incentive and escort them from the interview room.

Attachment V

Interview Probes

As described in the memo, generic probes will be administered to participants on an as-needed basis to elicit additional information about the cognitive testing process. A script for the generic probes is presented below.

EXAMPLES OF GENERAL PROBES:

• What did you think the question was asking?

• Can you rephrase the question in your own words?

• How did you go about answering?

• What were you thinking of?

• How did you decide on your answer?

• What was the reason for that answer?

• How easy or difficult was it to answer the question?

• PROBE for examples.

• How did you feel about answering the questions?

• Could you explain what you understand by this term?

• Can you tell me more about that?

• You seemed to hesitate/take a moment before giving/entering your response. Can you tell me what you were thinking about?

• What do you mean by…?

• Is there any additional information that we should have provided to help you answer that question?

• Does this term/question sound ok to you, or would you prefer something different?