2016/20 Baccalaureate and Beyond (B&B:16/20) Full-Scale Study
Supporting Statement Part A
(OMB # 1850-0926 v. 11)
Submitted by
National Center for Education Statistics
U.S. Department of Education
December 2019
revised June 2020
revised August 2020
Contents
A. Justification 1
1. Circumstances Making Collection of Information Necessary 1
a. Purpose of this Submission 1
b. Legislative Authorization 2
c. Prior and Related Studies 2
2. Purposes and Uses of the Data 3
a. B&B:16/20 Purposes 3
b. B&B:16/20 Research and Policy Issues 4
c. Study Design for B&B:16/20 6
3. Use of Information Technology 6
4. Efforts to Identify Duplication 6
5. Method Used to Minimize Burden on Small Businesses 7
6. Frequency of Data Collection 7
7. Special Circumstances of Data Collection 7
8. Consultants Outside the Agency 7
9. Provision of Payments or Gifts to Respondents 8
10. Assurance of Confidentiality 9
11. Sensitive Questions 12
12. Estimates of Response Burden 13
13. Estimates of Cost 14
14. Costs to Federal Government 14
15. Reasons for Changes in Response Burden and Costs 14
16. Publication Plans and Time Schedule 14
17. Approval to Not Display Expiration Date for OMB Approval 15
18. Exceptions to Certification for Paperwork Reduction Act Submissions 15
B. Collection of Information Employing Statistical Methods & References
C. References
Appendices
Technical Review Panel (TRP) Members
Confidentiality for Administrative Record Matching
Sample Member Communication Materials for Panel Maintenance
Focus Groups/Cognitive Testing Report
Data Collection Contacting Materials
Survey Instruments
This request is for the National Center for Education Statistics (NCES), within the Institute of Education Sciences (IES), part of the U.S. Department of Education, to conduct the 2016/20 Baccalaureate and Beyond (B&B:16/20) full-scale study, including the survey and data collection design that was developed, in part, from the results of survey tryouts and focus groups (OMB# 1850-0803 v.242). A previous submission covered panel maintenance activities, plus matching data to administrative records (1850-0926 v.8). The B&B:16/20 full-scale data collection is scheduled to take place from Summer 2020 through Spring 2021.
B&B examines students’ education and work experiences after they complete a bachelor’s degree, with a special emphasis on the experiences of school teachers. The B&B-eligible cohort is initially identified in the National Postsecondary Study Aid Study (NPSAS). The first cohort (B&B:93) was identified in NPSAS:93 and consisted of students who received their bachelor’s degree in the 1992–93 academic year. NPSAS:93 provided the base-year data and students were surveyed in 1994 for the initial follow-up. The B&B:93 cohort was surveyed again in 1997 and 2003. The second cohort (B&B:2000) was selected from NPSAS:2000, which became the base year for a single B&B:00/01 follow-up. The third cohort (B&B:08) was selected from NPSAS:08, which became the base year for follow-up data collections in 2009, 2012, and the final follow-up in 2018. The fourth cohort (B&B:16) was selected from NPSAS:16, which is the base year for follow-up data collections in 2017, 2020, and 2026 (anticipated). The B&B:93 and B&B:08 cohorts included transcript collections. Please note that the B&B:08/18 field test and the B&B:16/17 full-scale study were in data collection at the same time. To accommodate this overlap in timing, B&B cohorts prior to B&B:16 are approved under OMB# 1850-0729 while the B&B:16 cohort is approved under OMB# 1850-0926.
Most of the data collection strategies to be employed for the B&B:16/20 full-scale study will be based on the methods used in prior studies. The strategy will include an aggressive data collection protocol for sample members who failed to respond to NPSAS:16 or B&B:16/17, the ever nonrespondents, and B&B:16/17 abbreviated respondents (N=7,622) and a default data collection protocol for respondents to both the NPSAS:16 base year and the B&B:16/17 first follow-up surveys, the double respondents (N=15,030). NPSAS:16 non-study members (N=1,248) and nonrespondents to both the NPSAS:16 and B&B:16/17 surveys (N=2,609) will be screened for study eligibility only. Data collection plans by group, including specific protocols and incentive amounts for each group, are detailed in part B of this request. The B&B:16/20 full-scale survey will be consistent with B&B:16/17 and B&B:08 cohort surveys, except for some item-level revisions based on results of the survey tryouts and focus groups, conducted from December 2018 through June 2019.
This submission includes the following appendices:
A membership list of the last Technical Review Panel conducted for the B&B:16 cohort (appendix A; v. 8 unchanged);
A description of the confidentiality procedures in place for the survey and administrative record matching (appendix B);
Contacting materials and address update form for panel maintenance of sample members selected for participation in the B&B:16/17 student survey (appendix C) (carried over from 1850-0926 v.8)
A summary of the results of survey tryouts and focus groups (appendix D);
Contacting materials and an address update form for panel maintenance of sample members selected for participation in the B&B:16/17 survey (carried over) and contacting materials for B&B:16/20 full-scale data collection (appendix E); and
The survey instruments (appendix F).
We anticipate submitting several change memoranda to update OMB on the following:
An item on parents’ marital status when the sample member was age 18 is being added to the survey. Two versions of the item were evaluated as part of cognitive testing for the 2020/22 Beginning Postsecondary Students Longitudinal Study (1850-0803 v. 260). The results of that evaluation informed the choice of wording to be used for the B&B:16/20 survey and a change memorandum with an updated Appendix F that includes the final wording was submitted in May 2020.
Part B describes the first calibration experiment which compares participation rates and sample representativeness resulting in two different ways of sending a $2 prepaid incentives – mailing cash, or including an index card announcing a $2 prepaid PayPal incentive. The results of that experiment were analyzed and reported in Part B.4 (Purpose of this Submission).
Parts A and B describe plans for a mini survey to be administered to B&B:16/20 nonrespondents toward the end of data collection. The mini survey will be offered as an online web survey and may be administered as a texted survey. Since Short Message Service (SMS) texting for surveying this population is a new mode, its effectiveness and ease of use will be evaluated following administration of civic engagement questions to sample members surveyed before November 3, 2020. A decision on whether or not a mini survey will be included as text mode will be submitted in a change memorandum by February 2021.
A second calibration experiment described in Part B will evaluate participation rates when nonrespondents are offered either a $2 prepaid incentive or a $10 flash incentive. The results of the experiment will be submitted in a change memorandum by February 2021.
B&B is a longitudinal series of surveys conducted by NCES in close consultation with other U.S. Department of Education offices, federal agencies, and organizations (see sections A.4 and A.8 of this document). B&B and NPSAS, the base-year study for B&B, are authorized under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543) and the Higher Education Opportunity Act of 2008 [HEOA 2008, 20 U.S.C. §1015(a)(k)]:
“Student aid recipient survey
(1) Survey required: The Secretary, acting through the Commissioner for Education Statistics, shall conduct, on a State-by-State basis, a survey of recipients of Federal student financial aid under subchapter IV of this chapter and part C of subchapter I of chapter 34 of title 42—
(A) to identify the population of students receiving such Federal student financial aid;
(B) to describe the income distribution and other socioeconomic characteristics of recipients of such Federal student financial aid;
(C) to describe the combinations of aid from Federal, State, and private sources received by such recipients from all income categories;
(D) to describe the—
(i) debt burden of such loan recipients, and their capacity to repay their education debts; and
(ii) the impact of such debt burden on the recipients’ course of study and post-graduation plans;
(E) to describe the impact of the cost of attendance of postsecondary education in the determination by students of what institution of higher education to attend; and
(F) to describe how the costs of textbooks and other instructional materials affect the costs of postsecondary education for students.
(2) Frequency: The survey shall be conducted on a regular cycle and not less often than once every four years.
(3) Survey design: The survey shall be representative of students from all types of institutions, including full-time and part-time students, undergraduate, graduate, and professional students, and current and former students.
(4) Dissemination: The Commissioner for Education Statistics shall disseminate to the public, in printed and electronic form, the information resulting from the survey.”
The B&B series provides longitudinal studies of the education, employment, financial, and personal experiences of individuals who have completed a bachelor’s degree at a given point in time. Four B&B cohorts, each sampled about eight years apart, have allowed researchers to evaluate how baccalaureate degree recipients have fared at differing times in recent history. Bachelor’s degree recipients are identified in NPSAS, a nationally representative trend study of postsecondary students designed to determine how students and their families pay for postsecondary education. B&B:16/20 is the second follow-up of a panel of baccalaureate degree recipients identified in NPSAS:16, and part of the fourth cohort (B&B:16) of the B&B series. The current cohort contains students who earned their bachelor’s degrees in 2015–16.
Table 1 presents the chronology of the previous administrations of the NPSAS study and its associated longitudinal components, including the Beginning Postsecondary Students Longitudinal Study (BPS). For all studies, except B&B:16/20, full-scale data collection was preceded by a field test data collection one year earlier in order to test methods and procedures planned for the full-scale study. In August 2018, NCES determined that a B&B:16/20 field test was not necessary given that several cycles of B&B have been conducted in recent years and most of the survey items anticipated for the B&B:16/20 full-scale data collection have been used in previous cycles of B&B.
The seven major issues addressed in these Sample Surveys Division studies are:
Undergraduate access/choice of institution;
Persistence;
Progress/curriculum;
Attainment/outcome assessment;
Financial aid and student debt;
Access to graduate programs; and
Benefits of postsecondary education to individuals and society.
Table 1. Chronology of NPSAS and its longitudinal components
Base year |
First follow-up |
Second follow-up |
Third follow-up |
NPSAS:90 |
BPS:90/92 |
BPS:90/94 |
— |
NPSAS:93 |
B&B:93/94 |
B&B:93/97 |
B&B:93/03 |
NPSAS:96 |
BPS:96/98 |
BPS:96/01 |
Administrative data matching only |
NPSAS:2000 |
B&B:2000/01 |
— |
— |
NPSAS:04 |
BPS:04/06 |
BPS:04/09 |
Administrative data matching only |
NPSAS:08 |
B&B:08/09 |
B&B:08/12 |
B&B:08/18 |
NPSAS:12 |
BPS:12/14 |
BPS:12/17 |
— |
NPSAS:16 |
B&B:16/17 |
B&B:16/20 |
B&B:16/26 (anticipated) |
NPSAS:20 |
BPS:20/22 |
BPS:20/25 |
— |
— Not applicable.
NOTE: BPS = Beginning Postsecondary Students; B&B = Baccalaureate and Beyond.
This section provides information on the purposes of B&B:16/20 and an overview of the primary research issues it addresses.
The primary purpose of the B&B longitudinal study series is to describe the post-graduation pathways of baccalaureate recipients, with a focus on their experiences in the labor market, postbaccalaureate education, and their education-related debt. B&B also focuses on the continuing education paths of science, technology, engineering, and mathematics (STEM) graduates, as well as the experiences of those who have begun careers in education of students through the 12th grade. Since graduating from college in 2015–16, members of this B&B cohort will begin moving into and out of the workforce, enrolling in additional postsecondary education, forming families, and repaying undergraduate education-related debt. Documenting these choices and pathways, along with individual, institutional, and employment characteristics related to those choices, provides critical information on the costs and benefits of a bachelor’s degree in today’s workforce.
The B&B series of studies is critical to understanding the education paths of all types of bachelor’s degree recipients. While these graduates are homogenous in many ways, there are important variations to consider. B&B represents both traditional-age and non-traditional-age college graduates of public, private nonprofit, and for-profit institutions, graduates who began their postsecondary education at a community college, and those who began at a four-year college or university. Findings based on B&B data are presented in multiple publications such as First Look and Statistics in Brief reports, and Web Table publications. The data can also be used to calculate statistics with PowerStats, QuickStats, and TrendStats, web-based software applications available to the public at https://nces.ed.gov/datalab, and to qualified researchers through the IES restricted-use data-licensing program.
The B&B:16/20 data will allow researchers to address a wide variety of policy-relevant topics, including the consequences of education debt, career paths into and out of STEM fields, the academic preparation and career paths of pre-kindergarten–12th grade teachers, and labor market outcomes of bachelor’s degree recipients in the four years following graduation.
Education debt and loan repayment. As the price of college has increased across the country, so has focus on the amount of education debt accrued and the burden of subsequent repayment. Borrowing for undergraduate education is predicated upon the assumption that future employment will allow for repayment of debt. B&B:16/20 will provide the latest information on recent college graduates’ debt burden, including their selection of income-driven repayment plans, and whether their income is sufficient to repay their loans. The data will allow an examination of the rates of repayment and default on federal student loans among students who graduated from various types of institutions. It will also allow for an examination of the relationship between loan debt and postbaccalaureate outcomes such as enrollment, employment, and in particular, whether contemporary college graduates make different graduate enrollment and employment decisions than they might otherwise, due to their education-related debt.
Science, technology, engineering, and mathematics. Given the emphasis that policymakers and business leaders place upon college graduates’ science, technology, engineering, and mathematics (STEM) expertise, analyzing the paths of graduates with STEM majors will provide information about why and when people move into and out of STEM occupations and graduate programs. The B&B data allow the study of whether college graduates with training in these fields are using that training in the workplace or pursuing graduate education in STEM fields.
The pre-kindergarten-12th (pre-K-12) grade teacher. B&B cohorts have historically focused on kindergarten-12th grade (K-12) teacher recruitment and retention as important issues for education policymakers. B&B’s focus on those who enter K-12 teaching after college graduation allows in-depth study of teacher experiences, satisfaction, and mobility into and out of the K–12 teaching profession. Because some states are raising certification and degree requirements to teach pre-K, and in response to a recommendation made by the B&B:16/17 Technical Review Panel (TRP), pre-kindergarten (pre-K) teachers are also be included in the definition of “teachers” studied in the B&B:16 cohort studies. B&B:16’s unique focus on pre-K–12 teachers will allow researchers to compare teachers with their similarly-educated, non-teacher peers in terms of career paths, workplace satisfaction, and salaries.
Labor market outcomes. Finally, B&B:16/20 will collect data on recent college graduates’ employment and job search experiences approximately four years after graduating. The B&B:16/20 data will allow researchers to examine employment and wages as well as unemployment and under-employment, job satisfaction, job search behavior, and the value of the bachelor’s degree in finding employment.
Some of the primary research and policy issues to be addressed with B&B:16/20 data will be:
Debt and finances;
How much do bachelor’s degree recipients owe on undergraduate student loans 4 years after college? What is the status of the loans?
If in repayment, what are the monthly payments? Are parents helping to repay the loans?
Has student loan debt influenced career or graduate school plans?
Do graduates consider the loans a worthwhile investment in their future? Would it have been possible to complete college in the same period without the loans?
What is their total household income including spouses’ or partners’ incomes?
Do they own or make payments on a home or a vehicle?
What are their monthly payments for housing, auto loans, and credit cards?
What is the total debt burden (monthly payments as percent of income) including student loans?
Are graduates living at home with parents, alone, or with others? Are they married or living with a partner? Do they have any children or other dependents?
Postbaccalaureate enrollment;
What percentage of recent college graduates apply to graduate or professional institutions, and what percentage enroll within 4 years of completing a bachelor’s degree? What factors are associated with postbaccalaureate enrollment?
What are their reasons for continuing their education?
What are the characteristics of those accepted for graduate/professional study who opt to defer enrollment?
How are they financing their postbaccalaureate education?
What institutions do they attend and why?
What types of programs do they pursue?
What percentage of bachelor’s degree recipients who enroll in graduate education do so in STEM fields, and what proportion of those had undergraduate majors in STEM fields? How does this enrollment relate to graduates’ career paths?
What paths do college graduates with bachelor’s degrees in STEM fields take after graduation? What percentage obtain employment or enroll in further education in STEM fields? What percentage of graduates who received a non-STEM bachelor’s degree are employed in STEM occupations?
How are postbaccalaureate enrollment patterns related to demographic characteristics, undergraduate enrollment histories, undergraduate academic performance, and financial factors?
Among those without postbaccalaureate enrollment, what are their reasons for delaying further education or not planning to attend at all?
What percentage are continuing education in undergraduate, vocational, or non-degree programs?
Pre-K–12 teaching;
How did the undergraduate experiences of graduates who taught differ from those of graduates who considered teaching, those who prepared to teach, and other graduates?
What percentage of college graduates were employed as a teacher within a year of receiving a bachelor’s degree? What percentage taught throughout the period?
What were the experiences of recent college graduates who are in the pre-kindergarten/ elementary/secondary schoolteacher “pipeline” (i.e., those who were trained, certified, or considered teaching as a career)?
What percentage of recent college graduates applied to and accepted teaching positions?
What were the reasons for not applying to or not accepting teaching positions?
Do those who entered teaching feel that they were well prepared?
What type of teaching position did they hold during the first 4 years, and how effective was the school in assisting them to teach?
With which aspects of teaching during the first 4 years were they satisfied or dissatisfied?
Labor market outcomes;
What type of careers are recent college graduates planning, and how successful are they in obtaining jobs leading to future careers within 4 years of graduation?
How successful are graduates with different majors in finding career-related employment? What is the relationship between graduates’ majors and the occupations and industries in which they become employed?
What were their employment statuses in the months and years following graduation?
What are the characteristics of the jobs held post-graduation?
What percentage are employed full-time and part-time, in temporary or permanent positions?
What is the annual income and what are the monthly wages across jobs held?
What percentage have participated in job-related training programs?
How do the B&B:16 cohort’s labor market outcomes 4 years after receiving a bachelor’s degree compare to those observed among the previous B&B cohorts of 1992–93, 1999-2000, and 2007-08?
Answers to these and other questions are vital to policymakers at the local, state, and national levels to respond adequately to the changing environment of postsecondary education and the labor market.
Data collection for B&B:16/20 will consist of a sample member survey and administrative data matching. The respondent universe for the B&B full-scale study consists of all persons who completed requirements for the bachelor’s degree during the 2015–16 academic year and received their degree by June 30, 2017. These sample members will be surveyed for B&B starting in summer 2020 through spring 2021. Data from sample members will be collected via an online survey, through web-based computer-assisted telephone interviews (CATI), and/or via text message.
In addition to the survey, administrative data matching for the B&B:16/20 sample will be conducted with several sources, including: the National Student Loan Data System (NSLDS), containing federal loan and grant files; the Central Processing System (CPS), which houses and processes data contained in the Free Application for Federal Student Aid (FAFSA) forms; the National Student Clearinghouse (NSC) which provides enrollment and degree verification; vendors of national undergraduate, graduate, and professional student admission tests; and possible other administrative data sources such as the Veterans Benefits Administration (VBA). These data will be obtained through file matching/downloading. We will conduct CPS and NSLDS matches prior to data collection, which begins in summer 2020. Tentative dates for additional matches include a CPS match in September 2020 and NSLDS, NSC, and VBA matches in January 2021. A description of the confidentiality procedures in place for administrative record matching is provided in appendix B.
To improve efficiency, the B&B:16/20 surveys – which will include a full, abbreviated, and mini survey – will use web-based questionnaires across two electronic modes of data collection: a self-administered survey, which is mobile-friendly to allow completion on a tablet or smartphone, and computer-assisted telephone interviewing (CATI). In the B&B:16/17 full-scale study, 61% of surveys were completed on a personal computer, 28% on a handheld device, and 11% through CATI. The survey will be available through a study website that resides on NCES servers. In addition, an SMS text message survey will be used to convey civic engagement questions about voting in the 2020 presidential election to individuals who responded to the B&B:16/20 survey on or before the November 3, 2020 United States Election Day. This will be the first time SMS text surveying is attempted with one of the NCES postsecondary samples. The platform for the text messaging, known as RTI International Short Message Service (ARTEMIS), was developed by RTI to support research applications since most commercial SMS platforms do not. If, based on participation rates and data quality, text surveying through ARTEMIS is successful, we will use the technology as an additional method to administer the mini survey at the end of data collection.
On a nightly basis, the data collection contractor will download survey data to their Enhanced Security Network (ESN) via a secure web service. Once in the ESN, data will be cleaned and undergo quality analysis. The websites used for the B&B:16/20 data collection will reside on NCES’ Secure Socket Layer-certified (SSL) servers with a secure data connection. SSL protocol is used to encrypt the data transmitted over the Internet, and all parts of the websites that collect respondent data are password protected. The forms that gather data on these websites will require session cookies to run in accordance with the U.S. Department of Education's privacy policy for the use of cookies. The most recent versions of Microsoft Internet Explorer, Google Chrome, and Mozilla Firefox allow for rejecting all cookies other than those required for session tracking.
Efforts to identify duplication have included NCES consultations with other federal offices, such as the U.S. Department of Education’s Office of Postsecondary Education; the Office of Planning, Evaluation and Policy Development; and other agencies, such as the Government Accountability Office(GAO); the Congressional Budget Office (CBO); and the Office of Management and Budget(OMB). In addition, NCES collaborates with the National Center for Science and Engineering Statistics (NCSES) at the National Science Foundation (NSF) to ensure that each unit is kept up-to-date on each other’s studies pertaining to postsecondary students and institutions. NCES and NSF meet on a regular basis to cover topical issues relevant to both offices and each has staff serving on study TRPs. NCES routinely consults with non-federal associations, such as the American Council on Education, the Career Education Colleges and Universities, the National Association of Student Financial Aid Administrators, the National Association of Independent Colleges and Universities, the Council of Graduate Schools, and the Institute for Higher Education Policy to confirm that data collected through B&B are not available from any other sources. NCES also consults with academic researchers, several of whom have attended past B&B TRP meetings. Beyond identification of duplication, these consultations provide methodological insights from the results of similar and related studies conducted by NCES, other federal agencies, and nonfederal sources. The consultations also assure that data collected through B&B will meet the needs of the federal government and relevant organizations.
No studies in the U.S duplicate the data produced by B&B. While both B&B and NCES’s National Teacher and Principal Survey (NTPS, formerly the Schools and Staffing Survey, or SASS) survey teachers, the studies’ aims are different. NTPS includes only K-12 teachers and focuses on teachers’ preparation, working conditions, and career development. With the exception of the five-year Beginning Teacher Longitudinal Study (BTLS) that concluded in 2011–12, NTPS does not follow teachers over time. B&B, in contrast, follows graduates for up to 10 years and supports comparisons between graduates who go into teaching with other graduates. The B&B study collects less detailed data than NTPS on current teachers but provides data on graduates who consider or prepare for teaching without taking a teaching job, as well as teachers who leave the teaching profession. Moreover, for the first time, B&B added pre-kindergarten to the definition of teachers included in the survey in 2017.
Target respondents for B&B:16/20 data collection are individuals. Data collection activities will not involve burden to small businesses.
A new B&B cohort has been created about every eight years since the first set of studies was initiated with NPSAS:93. B&B:93 and B&B:08 are the two cohorts for which college transcript data were collected. The current B&B study, B&B:16/20, will be conducted 3 years after the first follow-up B&B:16/17 data collection and 4 years after the base-year NPSAS:16 data collection.
NPSAS and its longitudinal spin-off studies, BPS and B&B, are conducted to reflect the large-scale and rapid changes in federal policy concerning postsecondary student aid and fluctuation in sizes of grant and loan amounts. A recurring study is essential to help predict future costs for financial aid because loan programs create ongoing obligations for the federal government for as long as loans are in repayment. Repeated surveys capture the changing nature of the postsecondary environment. With the longitudinal design of the NPSAS survey and B&B follow-ups, representative national samples of bachelor’s degree-receiving postsecondary students with similar base-year characteristics can be compared over time to determine the effects of changes in federal policy and programs.
No special circumstances of data collection are anticipated.
Recognizing the significance of the B&B data collection, several strategies have been incorporated into the project work plan that allow for the critical review and acquisition of comments relating to project activities, interim and final products, and projected and actual outcomes. These strategies include consultations with persons and organizations both internal and external to NCES, the U.S. Department of Education, and the federal government (see also section A.4 of this document).
Previous B&B implementations have benefited from TRPs composed of staff from several offices in the Department, representatives of OMB, CBO, GAO, NSF, and the Bureau of Labor Statistics; and non-federal members who are considered experts in postsecondary education issues, including employment and teaching outcomes of baccalaureate recipients. The last B&B TRP meetings were held in December 2016 and December 2017 to inform the B&B:16/17 data collection and the B&B:08/18 data collection, respectively, during which plans for the full-scale study designs for each study were presented and discussed. Given several cycles of B&B in recent years, each of which held TRP meetings to discuss full-scale study design plans, and that most survey items anticipated for the B&B:16/20 full-scale study data collection were administered in previous cycles of B&B, NCES determined that convening a B&B:16/20 TRP was not necessary. This is noted in appendix A, which lists the B&B:16/17 full-scale TRP members.
All eligible cases will be offered a monetary incentive for completing the survey according to the data collection protocol set for their group (see table 2). The use of incentives for completion of the survey can provide significant advantages to the government in terms of increased response rates and higher quality data with minimal nonresponse bias. In addition, the use of incentives may also result in decreased data collection costs due to improved efficiency. The following section summarizes plans for payments to survey respondents.
Table 2. Data collection incentive plan by respondent group
|
NPSAS:16 Nonstudy Members (n=1,248) and Double Nonrespondents (n=2,609) |
Ever Nonrespondents and B&B:16/17 Abbreviated Respondents (n=7,622) |
Double Respondents (n=15,030) |
Protocol |
Screening for Unknown Eligibility |
Aggressive |
Default |
Incentive plan |
$5 |
$2 prepaid + $35 promised with optional:
|
$0 prepaid + $30 promised with optional:
|
Protocol to Screen for Unknown Eligibility: Sample members who were nonstudy members after NPSAS:161 or were double nonrespondents having not participated in either the NPSAS:16 base year data collection nor the B&B:16/17 first follow-up have unknown eligibility for the study. They will be asked to complete an eligibility screener for which they will receive $5.
Aggressive Protocol: Sample members who participated in either NPSAS:16 or B&B:16/17 or who completed only an abbreviated survey for B&B:16/17 (Aggressive protocol) will receive a $2 prepaid incentive with a $35 promised incentive. Those who participate within the first 3 weeks of the start of data collection will receive an additional $5 as an early bird incentive ($42 total). During the nonresponse phase toward the end of data collection, those who have still not responded will be offered a $10 incentive boost ($47 total).
Default Protocol: Sample members who have participated in both prior round surveys will be offered a $30 promised incentive. If response rates at the end of production warrant, as a final attempt to obtain a full survey we will implement an experiment, before offering the abbreviated survey during the nonresponse conversion phase. One group of the remaining nonrespondents, assigned at random, will be offered a $2 prepaid incentive ($32 total) while the other group will be offered a $10 flash incentive if they complete the survey within two weeks of the offer ($40 total) to assess whether prepaid incentives continue to be more effective at later stages of data collection than are promised incentive boosts.
Prior to the start of data collection, all B&B:16/20 sample members will be matched to a federal database maintained by the U.S. Department of the Treasury’s Office of Foreign Assets Controls (OFAC). OFAC administers and enforces economic and trade sanctions based on U.S. foreign policy and national security goals. As part of its enforcement efforts, OFAC publishes a list of individuals and companies called the "Specially Designated Nationals List” or "SDN." Their assets are blocked and U.S. entities are prohibited from conducting trade or financial transactions with those on the list (https://www.treasury.gov/resource-center/sanctions/Pages/default.aspx). In order to determine if there are any B&B:16/20 sample members to whom NCES cannot offer an incentive, the sample members will be matched to the SDN using the Jaro-Winkler and Soundex algorithms recommended by OFAC. To avoid over-matching, B&B staff will review the cases based on full name, date of birth, and address. The small number of individuals that NCES anticipates to not be able to confirm as not matching the SDN list will receive a survey request without an incentive offer.
The primary contractor for this study is RTI International (Contract# ED-IES-13-C-0070). Data security and confidentiality protection procedures have been put in place for B&B:16/20 to ensure that RTI and its subcontractors comply with all privacy requirements, including:
The statement of work of this contract;
Family Educational Rights and Privacy Act (FERPA) of 1974 (20 U.S.C. §1232(g));
Privacy Act of 1974 (5 U.S.C. §552a);
Privacy Act Regulations (34 CFR Part 5b);
Computer Security Act of 1987;
U.S.A. Patriot Act of 2001 (P.L. 107-56);
Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9573);
Cybersecurity Enhancement Act of 2015 (6 U.S.C. §151);
Foundations of Evidence-Based Policymaking Act of 2018, Title III, Part B, Confidential Information Protection;
The U.S. Department of Education General Handbook for Information Technology Security General Support Systems and Major Applications Inventory Procedures (March 2005);
The U.S. Department of Education Incident Handling Procedures (February 2009);
The U.S. Department of Education, ACS Directive OM: 5-101, Contractor Employee Personnel Security Screenings;
NCES Statistical Standards; and
All new legislation that impacts the data collected through the contract for this study.
Furthermore, the contractor will comply with the Department’s IT security policy requirements as set forth in the Handbook for Information Assurance Security Policy and related procedures and guidance, as well as IT security requirements in the Federal Information Security Management Act (FISMA), Federal Information Processing Standards (FIPS) publications, Office of Management and Budget (OMB) Circulars, and the National Institute of Standards and Technology (NIST) standards and guidance. All data products and publications will also adhere to the revised NCES Statistical Standards, as described at the website: https://nces.ed.gov/statprog/2012/.
By law (20 U.S.C. §9573), a violation of the confidentiality restrictions is a felony, punishable by imprisonment of up to 5 years and/or a fine of up to $250,000. The B&B:16/20 procedures for maintaining confidentiality include notarized nondisclosure affidavits obtained from all personnel who will have access to individual identifiers; personnel training regarding the meaning of confidentiality; controlled and protected access to computer files; built-in safeguards concerning status monitoring and receipt control systems; and a secure, staffed, in-house computing facility. B&B:16/20 follows detailed guidelines for securing sensitive project data, including, but not limited to: physical/environment protections, building access controls, system access controls, system login restrictions, user identification and authorization procedures, encryption, and project file storage/archiving/destruction.
Additionally, the contractor will take security measures to protect the web data collection applications from unauthorized access. The web server will include an SSL certificate and will be configured to force encrypted data transmission over the Internet. Respondents to the survey will be required to confirm their identity with each log in session using security questions based on data previously collected on participants. These questions take the form commonly associated with credit check “pick lists.” A survey entrant will be presented with a series of similar answers to a background question (i.e. – name of the bachelor’s degree institution) and will be required to answer correctly before beginning (or resuming) the survey. The web survey will also be programmed to prevent backtracking to areas of the survey with personally identifiable information (PII). This measure is intended to prevent unauthorized access to PII within in-progress surveys. To further ensure confidentiality, the survey automatically logs out of a survey session after 20 minutes of inactivity.
Security measures have been put in place to protect data during file matching procedures described in A.3 of this document. NCES has a secure data transfer system, which uses SSL technology, allowing the transfer of encrypted data over the Internet. The IES File Transfer System will be used for all administrative data sources that do not have their own secure file transfer systems. All data transfers will be encrypted.
The Department has established a policy regarding the personnel security screening requirements for all contractor employees and their subcontractors. The contractor must comply with these personnel security screening requirements throughout the life of the contract, including several requirements that the contractor must meet for each employee working on the contract for 30 days or more. Among these requirements are that each person working on the contract must be assigned a position risk level. The risk levels are high, moderate, and low based upon the level of harm that a person in the position can cause to the Department’s interests. Each person working on the contract must complete the requirements for a “Contractor Security Screening.” Depending on the risk level assigned to each person’s position, a follow-up background investigation by the Department will occur.
B&B:16/20 and other NCES postsecondary studies include data linkages with many existing sources of valuable data, including the Department of Education’s CPS for FAFSA data, NSLDS, and NSC. These NCES studies also obtain data from postsecondary institution student records, information on military service records from VBA, and admissions test scores from ACT and The College Board. Many of these administrative data matches are not guaranteed and will require separate Memoranda of Understanding, Agreements, or Contracts with the entities owning or responsible for the individual data elements.
Regarding file matching with administrative sources, the Family Educational Rights and Privacy Act (FERPA, 34 CFR Part 99) allows the disclosure of personally identifiable information from students’ education records without prior consent for the purposes of B&B:16/20 according to the following excerpts: 34 CFR §99.31 asks, “Under what conditions is prior consent not required to disclose information?” and explains in 34 CFR §99.31(a) that “An educational agency or institution may disclose personally identifiable information from an education record of a student without the consent required by §99.30 if the disclosure meets one or more” of several conditions. These conditions include, at 34 CFR §99.31(a)(3):
“The disclosure is, subject to the requirements of §99.35, to authorized representatives of—
(i) The Comptroller General of the United States;
(ii) The Attorney General of the United States;
(iii) The Secretary; or
(iv) State and local educational authorities.”
B&B:16/20 is collecting data under the Secretary’s authority. Specifically, NCES, as an authorized representative of the Secretary of Education, is collecting this information for the purpose of evaluating a federally supported education program. Any personally identifiable information is collected with adherence to the security protocol detailed in 34 CFR §99.35:
“(a)(1) Authorized representatives of the officials or agencies headed by officials listed in §99.31(a)(3) may have access to education records in connection with an audit or evaluation of Federal or State supported education programs, or for the enforcement of or compliance with Federal legal requirements that relate to those programs.
(2) The State or local educational authority or agency headed by an official listed in §99.31(a)(3) is responsible for using reasonable methods to ensure to the greatest extent practicable that any entity or individual designated as its authorized representative—
(i) Uses personally identifiable information only to carry out an audit or evaluation of Federal- or State-supported education programs, or for the enforcement of or compliance with Federal legal requirements related to these programs;
(ii) Protects the personally identifiable information from further disclosures or other uses, except as authorized in paragraph (b)(1) of this section; and
(iii) Destroys the personally identifiable information in accordance with the requirements of paragraphs (b) and (c) of this section.
(b) Information that is collected under paragraph (a) of this section must—
(1) Be protected in a manner that does not permit personal identification of individuals by anyone other than the State or local educational authority or agency headed by an official listed in §99.31(a)(3) and their authorized representatives, except that the State or local educational authority or agency headed by an official listed in §99.31(a)(3) may make further disclosures of personally identifiable information from education records on behalf of the educational agency or institution in accordance with the requirements of §99.33(b); and
(2) Be destroyed when no longer needed for the purposes listed in paragraph (a) of this section.
(c) Paragraph (b) of this section does not apply if:
(1) The parent or eligible student has given written consent for the disclosure under §99.30; or
(2) The collection of personally identifiable information is specifically authorized by Federal law.”
Additionally, the study, including the administrative data linkage, qualifies for a 45 CFR Part 46 waiver of consent based on the following factors:
There is minimal risk to the participants. There is no physical risk and only minimal risk associated with linkage of data to sample members. The public-use and restricted-use data, prepared as part of the contract with the data collection contractor, will not include Social Security Numbers (SSNs) or other extremely sensitive Personally Identifiable Information (e.g., name, birthdate, etc.) even though these data elements are used for the linkage. Data will undergo disclosure avoidance analysis and disclosure treatment steps to further reduce the risk.
The waiver will not affect the rights and welfare of the sample members. Public-use and restricted-use data are only used for research purposes and lack direct individually-identifying information. The data are further protected through disclosure avoidance procedures approved by the NCES Disclosure Review Board.
The study cannot be conducted practicably without the waiver. To obtain written consent from sample members, multiple forms would have to be sent to them with multiple follow-up telephone and in-person visits. This process would add weeks to the data collection process and is not feasible from a time standpoint. Additionally, the value of these data would be jeopardized from a nonresponse bias perspective.
The following language will be included, as appropriate, in respondent contact materials and on data collection instruments for the survey:
NCES is authorized to conduct the 2016/20 Baccalaureate and Beyond Longitudinal Study (B&B:16/20) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543) and to collect students’ education records from educational agencies or institutions for the purpose of evaluating federally supported education programs under the Family Educational Rights and Privacy Act (FERPA, 34 CFR §§ 99.31(a)(3)(iii) and 99.35). The data are being collected for NCES by RTI International, a U.S.-based nonprofit research organization. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form, for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).
In addition to the language above, on data collection instruments, as appropriate:
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0926. The time required to complete this information collection is estimated to average approximately 35 minutes per survey response, including the time to review instructions, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate, suggestions for improving this survey, or any comments or concerns regarding the status of your individual submission of this survey, please write directly to: The 2016/20 Baccalaureate and Beyond Longitudinal Study (B&B:16/20), National Center for Education Statistics, Potomac Center Plaza, 550 12th St., SW, Room 4018, Washington, DC 20202. OMB Clearance No: 1850-0926 Expiration Date: 7/31/2022
The B&B:16/20 survey collects information about earnings, assets, marital and family status, gender identity, and sexual orientation. Regulations governing the administration of these questions require (a) clear documentation of the need for such information as it relates to the primary purpose of the study, (b) provisions to clearly inform sample members of the voluntary nature of participation in the study, as well as information about how their responses will be protected and may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).
The collection of data related to income, earnings, assets, indebtedness, and long-range employment outcomes is central to understanding key policy issues driving this study. Financial assets and obligations are important post-graduate outcomes of students and are important indicators of the rate of return of educational experiences to the respondent. The collection of information about marital and family status likewise facilitates the exploration of key policy issues. Financial and family-related obligations also influence decisions about employment, additional education, and loan repayment so it is important to collect information about marital status and dependents. Gender identity and sexual orientation can impact employment and enrollment experiences and outcomes, a topic of growing interest for social research and policy. Several procedures have been implemented (see section A.10) to provide assurances to sample members about the voluntary nature of participation in the study as well as the confidentiality provisions for survey responses.
Social Security Numbers (SSNs) will be needed to 1) conduct file matches to administrative records and 2) maintain the sample for the B&B longitudinal study. File matching to administrative records is a crucial element of the B&B study and would not be possible without the collection of SSNs. Data obtained from file matching will both minimize respondent burden and increase data quality.
Table 3 provides estimates for B&B:16/20 response burden, based on experiences from prior B&B collections. We expect the survey to require approximately 35 minutes, on average, for completion. Assuming average hourly cost of $29.262 for participating sample members, the 10,435 total burden hours (including the carried over burden hours associated with the panel maintenance activities) translate to an estimated respondent burden time cost of approximately $305,328.
Table 3. Estimated burden to fielded respondents in B&B:16/20 full-scale study
Data collection activity |
Sample |
Expected eligible |
Expected response rate (percent) |
Expected number of respondents |
Expected number of responses |
Average time burden per response (minutes) |
Total time burden (hours) |
Panel maintenance |
25,261 |
-- |
15 |
3,7901 |
3,790 |
3 |
190 |
Eligibility screener |
3,8572 |
-- |
33 |
1,273 |
1,273 |
5 |
|
Full survey administered on or before November 3, 2020 |
22,652 |
22,652 |
45 |
10,193 |
10,193 |
35 |
5,946 |
Full survey administered after November 3, 2020 |
22,652 |
22,652 |
22 |
4,984 |
4,984 |
35 |
2,907 |
SMS texted civic engagement questions |
10,193 |
10,193 |
95 |
9,6841 |
9,684 |
2 |
323 |
Abbreviated survey |
22,652 |
22,652 |
15 |
3,398 |
3,398 |
15 |
850 |
Mini survey |
22,652 |
22,652 |
6 |
1,360 |
1,360 |
5 |
113 |
New Main Study Activities (subtotal) |
|
|
|
21,208 |
30,982 |
|
10,245 |
Total |
-- |
-- |
|
21,208 |
34,682 |
-- |
10,435 |
Greyed-out rows represent burden requested in previous packages (OMB # 1850-0926 v. 8) and carried over here. All percentages are rounded.
1 Duplicative counts of individuals, not included in respondent totals.
2 The eligibility screener will be administered to sample members with unknown eligibility to determine final eligible sample. Approximately 25 percent are expected to be ineligible for the B&B:16 cohort because they either did not complete the bachelor’s degree or completed it outside the 2015-16 academic year.
There are no additional costs to the respondents.
The total cost to the federal government for this administration of the B&B:16/20 is $4,912,474, of which $4,542,195 is for the full-scale data collection. A summary of costs is shown in Table 4. Included in the estimates are staff time, reproduction, postage, and telephone costs associated with the data collection for which clearance is requested.
Table 4. Costs to NCES for the B&B:16/20 full-scale study
B&B:16/20 implementation |
Costs to NCES |
NCES Salaries and expenses |
$330,000 |
Panel Maintenance |
$40,279 |
Full-scale study |
|
Contract costs |
$4,542,195 |
Instrumentation and materials |
$808,950 |
Data collection |
$2,266,861 |
Systems and data processing |
|
Total |
$4,912,474 |
The apparent increase in response burden is due to the fact that the previous approval was for the B&B:16/20 panel maintenance activity, while this request includes the B&B:16/20 full-scale data collection.
The operational schedule for the B&B:16/20 full-scale study is shown in Table 5. The contract for B&B:16/20 requires multiple reports, publications, and other public information releases, as follows:
Descriptive summaries of significant findings for dissemination to a broad audience;
Detailed data file documentation describing all aspects of the full-scale study design and data collection procedures;
Complete data files and documentation for research data users in the form of both a restricted-file; QuickStats, a public-use data analysis system in which users create their own tables and charts using pre-defined categories from a subset of variables; and PowerStats, a public-use data system in which users create their own tables and charts using all of the variables, in addition to conducting regression analysis; and
Special tabulations of issues of interest to the higher education community, as determined by NCES.
Table 5. Operational schedule for B&B:16/20
B&B:16/20 activity |
Start date |
End date |
Full-scale study |
|
|
Panel maintenance for B&B:16/20 full-scale study |
Oct. 1, 2019 |
Feb. 1, 2020 |
Select sample |
Nov. 11, 2019 |
April 17, 2020 |
Initial contact greeting card |
June 2020 (upon OMB approval) |
June 2020 (upon OMB approval) |
Data collection announcement, mailed in waves |
July 2020 |
July 2020 |
Self-administered web-based data collection* |
July 6, 2020 |
Mar. 19, 2021 |
Conduct calibration experiment 1 on $2 prepaid incentive announcement* |
July 6, 2020 |
July 22, 2020 |
Conduct telephone interviews * |
July 20, 2020 |
Mar. 19, 2021 |
Texted survey for civic engagement questions among early responders* |
November 4, 2020 |
December 4, 2020 |
Conduct calibration experiment 2 on nonresponse conversion incentives* |
December 14, 2020 |
March 19, 2021 |
Process data, construct data files |
July 20, 2020 |
Sept. 17, 2021 |
Prepare/update reports |
Jan. 18, 2021 |
Nov. 11, 2022 |
The expiration date for OMB approval of the information collection will be displayed on data collection instruments and materials. No special exception is being requested.
There are no exceptions to the certification statement identified in the Certification for Paperwork Reduction Act Submissions of OMB Form 83-I.
1 NPSAS:16 staff identified key variables across the various data sources to define a minimum set of required student-level data elements necessary to support the analytic objectives of the study. Staff classified sample members for whom data for the key variables were available as study members, the NPSAS:16 unit of analysis. Those lacking the requisite data were considered nonstudy members. (Wine, Siegel, & Stollberg, 2018).
2 The estimated average hourly rate was obtained using the average salary of the B&B:12 cohort 4 years later (Cataldi, E.F., Siegel, P., Shepherd, B., and Cooney, J. (2014). Baccalaureate and Beyond: A First Look at the Employment Experiences and Lives of College Graduates, 4 Years On (B&B:08/12) (NCES 2014-141). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC. Retrieved [date] from http://nces.ed.gov/pubsearch), adjusted for inflation.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Chapter 2 |
| Author | spowell |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |