30-day Notice Public Comment Response Summary

Ronald E. McNair Postbaccalaureate Achievement (McNair) Program

Summary of Public Comments on

Proposed Changes to the 2012-13 McNair Annual Performance Report (APR)

Following 30-Day Review Period

On March 22, 2013, the Department of Education (Department) published a Notice of Proposed Information Collection Request (Notice) in the Federal Register inviting comments by May 21, 2013, on the proposed annual performance report (APR) for the Ronald E. McNair Postbaccalaureate Achievement (McNair) Program. The Department reviewed each of the comments and concerns and made a number of changes to the APR to reduce burden and clarify the reporting requirements. Subsequently, the Department published, on August 29, 2013, a 30-day notice inviting comments on the revised proposed APR by September 30, 2013. Sixteen (16) commenters submitted approximately 42 individual comments (i.e., multiple comments from individual commenters). By and large, most of the commenters were very pleased to see that the Department had taken the time to address their concerns as well as, where applicable, incorporate many of their suggestions.

A summary and discussion of the comments received as a result of the 30-day comment period on the proposed McNair APR as well as information on the actions taken follows.

Fields 4 & 5: Student’s Last Name and Student’s First Name

Comments: Commenters continue to be concerned regarding not being permitted to update a student’s name because students commonly change their names (particularly after marriage). The commenters stated that while they appreciated the opportunity to update students’ name during the Tier 1A Student Verification Process, they still had concerns because they would not be able to effectively track participants as required by the Department since the time frame for tracking participants was protracted (10-years post baccalaureate degree attainment). One commenter stated that students become very upset if the project refuses to change the participant’s name after they have notified them of the change, which often results in students simply not responding to requests for updates to their educational status.

Discussion: The Department recognizes the challenges projects encounter when tracking participants. Unfortunately, because the Department cannot solely rely on the SSN for matching purposes and does not assign a unique student identifier, the Department must rely on the name and date of birth to match records across reporting years. A change to any of these fields may result in a non-match and can potentially impact a project’s Prior Experience (PE) points and other TRIO data analysis. Nonetheless, the Department does appreciate the effort project’s make in tracking students over a long period of time and as such a field has been added to address projects’ concerns.

Actions Taken: Added field 48—Student’s Name Change—Optional (Full Name.)

Field 19: First School Enrollment Date (Postsecondary Education)

Comments: Two commenters stated that the information reported in this field for non-traditional students could negatively skew the results because oftentimes these students first enrolled in college in the 1980s or 1990s but dropped, then re-enrolled; therefore, their time to degree completion for the PhD could be skewed. The commenters’ recommendation was for the Department to clarify that the intent of this field includes non-traditional students as well.

Discussion: For the purposes of tracking time to degree completion, the date of first enrollment into postsecondary education includes any student who enrolled in a program of undergraduate study; therefore, this definition includes students who stopped-out for a period of time and later re-enrolled.

Action Taken: Revised the instructions as follows: “Enter participant’s date of first enrollment in postsecondary education. The first postsecondary enrollment date is the first date a participant enrolled in a program of postsecondary education. For participants who stopped-out for a period of time and subsequently re-enrolled, enter the date the participant first enrolled in postsecondary education. Please note that this date does not include the date a participant enrolled in a high school bridge-to-college program/dual enrollment program or the first enrollment date a participant took college courses while enrolled in high school.

Field 19 -- Attendance at community college

Comments: One commenter suggested that for clarification purposes the field name should be renamed “Attended Community College/2-Year Institution” since the student will have transferred and/or attended community college or a 2 year institution prior to being enrolled in the McNair program. They stated that as currently written, the word “attendance” implies the act of attending in the present tense. 

Discussion: The Department agrees with the commenter’s suggestion and will make the revision.

Action Take: Revise to “Attended Community College/2-Year Institution”.

Field 20: Project Entry Date

Comments: One commenter asked if the project entry date will be used to calculate the eligibility criteria, that is, two-thirds first generation and low income and one-third underrepresented. 

Discussion: The determination of whether a project met the eligibility criteria (i.e., two-thirds must be first-generation and low-income and one-third may be underrepresented) is based on the numbers served in the reporting period. Please note that the Department does not use the project entry date to establish cohorts for calculating prior experience points and GPRA. For example, the cohort for the student outcome “graduate enrollment” is based on the number of participants who earned a baccalaureate degree in the reporting period (i.e., field 29, Graduating Cohort Year of Bachelor’s Degree).

Action Taken: None

Field 25 -- Funding Source

Comments: One comment suggested that since the intent of Field 25 is to “determine if the cost of serving the student in the McNair project was supported with Federal funds only or partially supported with non-federal funds, option 1 should read “Federal funds only.”

Discussion: The Department agrees with the commenter and will revise option 1.

Action Taken: Revised option to: Federal funds only

Field 26 -- STEM Major
Comments: One commenter stated that by focusing on STEM and non-STEM they are losing the ability to track different fields such as linguistics, English, and history as well as to see the difference within STEM fields. They stated that there are differences with underrepresented populations in science (biology vs. chemistry) which they will not be able to track. 

One commenter stated that there appears to be a distinction between the social/behavioral sciences, the "hard sciences” (biology, chemistry, computer sciences, etc.), and non-STEM majors. The commenter requested clarification as to whether the social/behavioral sciences fall under the STEM category. They further stated that this could be an important factor in recruitment for those McNair projects that are engaging strategies in attracting more STEM students into their program. The commenter asserted that the distinction either “broadens their net or shrinks it”; and in either case, clarification on this would be very helpful as they move forward in supplying data on STEM students served.

Response: Under the FY 2012 grant competition, McNair applicants were given the opportunity to address one or more of the three competitive preference priorities listed in the Notice Inviting Applications for New Awards for FY 2012. The goal of Competitive Preference Priority 1: Promoting STEM education is to increase the number and proportion of McNair participants prepared for graduate study in the STEM fields. If an applicant chose to address this priority and received priority points, the Department expects the project to focus on serving students who have chosen to major in one of the STEM disciplines, as proposed in the project’s approved application. Furthermore, one of the goals of field 26 is to measure student outcomes for project participants who are in enrolled in a STEM field. To accomplish this goal, the Department has mapped the field with both the Survey of Earned Doctorate (SED) and the Baccalaureate & Beyond (B&B) Survey.

Finally, for the purposes of tracking participants in the STEM fields as it relates to Competitive Preference Priority 1, STEM refers to fields in the “hard sciences” and not in the behavioral/social sciences. Again, the reason for the separate options is to better align with B&B and SED.

Action Taken: None

Fields 32—Research,

Field 33—Other Scholarly Activities, and

Field 34—McNair Internships

Comments: Most of the comments received concerned fields 32, 33, and 34. While commenters were elated that the Department had taken into consideration and included “other scholarly activities” in the calculation of prior experience points, they still had the following issues and concerns and requested further clarification:

1. Several commenters expressed concerns about the explicit definition of “other scholarly activities”. More specifically, the commenters stated that in a previous comment period (i.e., McNair regulations), commenters stated the following:

“….the definition of research or scholarly activity in Sec. 647.7 should be expanded to include examples such as developing a research proposal, implementing reporting, presenting and publishing research, and attendance at professional conferences. They argued that adding these activities as examples in the definition would clarify that ``research'' encompasses a range of scholarly activities that are more rigorous than typically available to undergraduates in a classroom setting.”

According to the commenter, the Department’s response was:

“…research may include a wide variety of scholarly activities, and we intend for the defined term research or scholarly activity to include activities such as those mentioned by the commenter. These examples are appropriate parts of a doctoral program and accordingly, could satisfy the requirement for research or scholarly activity under the McNair program. However, because there are so many examples of activities that could be covered in this definition, we are not including any examples in the regulations, but may include them in non-regulatory guidance."

Based on the Department’s most recent response “attendance at professional conferences,” is not accepted unless the student is presenting research at the conference. The commenter states that they were previously led to believe (based on comments by the Department at conferences and the definition cited immediately above) that “other scholarly activities” were allowed to be broader in scope than the formal definition, which, according to the commenter, “is a rigid interpretation of all four of the elements listed in the definition and now in the new document.” The commenter stated that activities such as:

“IRB training and attendance at research conferences (which would not necessarily have the specific “guidance of an appropriate faculty member with experience in the relevant discipline,” but would certainly be “more rigorous than…typically available to undergraduates in a classroom setting and WOULD be under the guidance of credentialed McNair staff…..”

are the kinds of activities that doctoral students engage in. The commenter is concerned that the definition is more rigorous than in previous discussions which may impact the project’s PE points for the research or other scholarly activities objective.

2. One commenter stated that participants who are freshman and sophomores should be exempted from the research activity as these students do not have the experience to be able to compete for external research opportunities nor will they have work that could be presented at a professional society conference. If the students cannot be exempted then projects should be allowed to count these students as having participated in “other scholarly activities” such as Institutional Review Board training, conference attendance (i.e., to enable students to begin to learn how others in their field present research results), library skills training, and attending seminars that either enhance research skills or expose students to research in their fields.

3. Two commenters asked for clarification as to whether projects needed to fulfill all of the three activities in order to be eligible for prior experience points. Furthermore, they asked whether every participant in the program had to participate in all three activities.

4. One commenter asked if field 32 (Research) was a subset of field 34 (McNair Internship). This commenter also asked if the fields are being defined differently from those of previous APRs.

5. One commenter requested that field 33, option 3 be reworded to say "Did not participate in scholarly activities," instead of "did not participate in research?" They stated that this would be consistent with the wording of Fields #32 and #34.

Discussion: Regarding “a” and “b” above, the standard objectives for the FY 2012 grant competition established the definition for research and other scholarly activities. The standard objectives were part of the 2012 application package. More importantly, the current McNair regulations in Section 647.22(a) (2) (How does the Secretary evaluate prior experience points?) states that “the Secretary uses the approved project objectives….to determine the number of PE points to be allotted.” The approved project objectives are in the McNair Profile sheet of the FY 2012 application and it is these objectives the regulations are referencing. The definition for the research and scholarly activities objective as defined in the McNair Profile sheet of the FY2012 application is defined as follows:

“Research and Scholarly Activities: an educational activity that is more rigorous than is typically available to undergraduates in a classroom setting, that is definitive in its start and end dates, contains appropriate benchmarks for completion of various components, and is conducted under the guidance of an appropriate faculty member with experience in the relevant discipline.“

The very essence of McNair is the research component; therefore, activities such as merely attending a conference/workshop, conducting a literature review, etc., is not considered research as it does not meet the definition in terms of a more rigorous activity.

Regarding “c” above, projects do not need to fulfill all of the three activities in order to be eligible for prior experience and not every participant in the program has to participate in all three activities.

With regard to “d” above, field #32 and #34 are not subsets as a project can have participants in their project who do not receive a McNair stipend (internship). For the definition of research and other scholarly activities, please refer to the instructions under “Definition That Apply”.

With regard to “e”, the Department agrees with the commenter’s suggestion and will make the revision.

Action Taken: “3 = Did not participate in scholarly activity”

Field 35 – Graduate School Admission Test

Comments: One commenter stated that the department should allow the program to report only on students who have graduated with a BA degree and those who have taken the graduate school admission test because students may take the graduate exams many times prior to graduating with BA. For this reason, the Department should allow reporting only for students who have graduated with a BA degree, have taken the admission test (regardless of how many times prior). This eliminates confusion relating to the number of times an admission test was taken. 

One commenter stated that the field should not be limited to just those participants who graduated with a bachelor’s degree because oftentimes students will take, for example a GRE, prior to the attainment of the bachelor’s degree which would result in delays in reporting.

Discussion: To reduce misreporting including leaving questions blank, the online data collection system requires projects to provide a response for each field. However, to further clarify how to respond to this question, the Department has revised the instructions to address how to report for students who take graduate admissions tests multiple times.

Regarding the second comment, this field was revised based on comments received from the 60-day comment period. Specifically, to reduce burden several commenters felt that there should be a point in time in which to report whether the participant has taken a graduate school admissions test. They all stated that the most logical point in time was at the time the participant earned a bachelor’s degree as this is most applicable to students who take a graduate admissions test. Furthermore, the intent of this field is to address issues related to sample selection bias when conducting the comparative analysis for the congressionally mandated reports. The cohort sample is based on bachelor’s degree recipients; therefore, capturing this information at the time of degree attainment aligns with the criteria used to draw the sample.

Action Taken: The following revision has been made to the note below the field options: “NOTE: Select “Yes” only if the participant has earned a bachelor’s degree and has taken a graduate admissions test. If you do not know if the participant has taken a graduate admissions test, select “0” (Unknown). If a participant has taken multiple admissions, only report it at the time the participant earned their bachelor’s degree. Once reported, you should not have to update in subsequent reporting periods.”

Field #36 – Graduate School Enrollment Status (at the beginning of the 2013-14 academic year)

Comments: One commenter asked if students who obtain postbaccalaureate or research fellowships such as NIH or Fulbright while undergraduates could be considered as having enrolled in graduate school.

Discussion: The McNair Program Profile which is governed by the regulations under 647.22(a)(2) for the objective “graduate school enrollment” is very specific—to earn PE points a McNair scholar must enroll in graduate school in the fall term immediately after attaining a baccalaureate degree. Therefore, for the purpose of calculating a project’s PE points, baccalaureate degree recipients who did not enroll in graduate school in the fall immediately after completing their baccalaureate degree because they enrolled in a highly competitive program (e.g., Fulbright) cannot be counted as having enrolled in graduate school.

Action Taken: None

Field 37-- Graduate Institution First Attended

Field 38—Graduate Student’s Main Field of Study (at time of entrance)

Comments: One commenter asked if it was possible for there to be a pop-up on the questions that links to the NCES coding as well as a pop-up for field 38.

Discussion: The NCES codebook contains codes for over 6,000 institutions and since we do not know the graduate institution the participant first attended, it would not be feasible to list all of the institutions. However, if the project clicks on the link provided in either the form or the information box (i) on the web application, the project will be routed to the NCES locator web site. Regarding a pop-up display for field 38, again the list for the Main Field of Study, which is provided in the instructions, is rather large and so a pop-up would not be feasible.

Action Taken: None

Field 39— First-Year Graduate Student Persistence Status (at the beginning of the 2013–14 academic year), and

Field 40— Current Year of Graduate Study (during academic year reported)

Comments: One comment requested clarification on how the Department will use these two fields together to track the first to second year persistence of all students who enter graduate school in any given year, whether or not that student entered graduate school the fall directly following the attainment of his or her bachelor’s degree.

Discussion: Field #39, Year Graduate Student Persistence Status (at the beginning of the 2013–14 academic year) and field #40, Current Year of Graduate Study (during academic year reported) are two separate fields and will not be used in combination to track the first to second year persistence rate for the graduate persistence objective. In order to provide transparency in the calculation of PE points, the goal of the new APR is to reduce the number of fields needed to calculate PE. Therefore, for the exception of the doctorate degree attainment objective, the other objectives will only require a single field be used in the calculation of PE. This will require grantees to update certain fields every year (e.g., field 38 and 39). Field 40 may be used to calculate the GPRA outcomes and to conduct comparative analysis of student outcomes and is not a PE field. In addition, depending on the participant’s status in graduate school, Field 40 may or may not require an update.

Action Taken: None

Field 41 – Graduate Student’s Main Field of Study (at time of entrance)

Comments: One commenter stated that field 41 does not provide instructions for how to code students who have received a Bachelor’s degree, but have not yet entered graduate school. The commenter further stated that the Note makes reference to the participant’s “undergraduate main field of study.” Should this read “graduate main field of study?”

Discussion: The Department concurs with the commenter’s concerns and has revised the form and instructions.

Action Taken: The “Note” in the form as well as the instructions will be revised as follows: “NOTE: Using the Main Field of Study List provided in the instructions, enter the three-digit code for the main field of study at the time the participant entered graduate school. If the participant has not earned a bachelor’s degree enter “001”. If you do not know the participant’s main field of study at time of entry into graduate school, enter “000”. Also enter “000” if the participant earned a bachelor’s degree but did not enter graduate school. Once reported, you should not have to update in subsequent reporting periods.”

Field 42 – Graduate Assistantships
Comments: One commenter stated that the department should consider the burden in collecting this information and delete this field as the question is not relevant to any of the 4 McNair objectives. 

Discussion: The field is relevant because the Department is mandated by law to conduct comparative analysis on program outcomes. In order to comply with the law, we must collect additional information. As such, field 40, Graduate Assistantships is an important element in understanding graduate persistence—a statutory mandated outcome criterion.

Action Taken: None

Field 43 – Reason Left Graduate School

Comments: Two commenters suggested additional changes to field 43 to capture more accurate information for participants who (1) left graduate school after attaining a Master’s and (2) left graduate school because they needed to complete graduate program requirements as a condition for doctoral admission. The commenters felt that these two scenarios were different than for example, left graduate school because they needed to work.

Discussion: The Department agrees with the commenter’s concerns related to participants who either do not pursue a doctoral degree or do not immediately enroll in a doctoral degree program; therefore, the field has been revised

Action Taken: Revised options as follows:

10 = Complete graduate program requirements (e.g., required for doctoral program admission)

11 = Attained Master’s degree and did not enroll in doctoral degree program

Field 44—Highest Graduate Degree Earned.
Comments: One commenter asked for clarification on how to code a student who is pursuing an Engineering degree (as a terminal degree) and not continuing to a Ph. D. program?

Another commenter stated that the guidance provided by Survey of Earned Doctorate (SED) regarding what constitutes a research doctorate degree is still vague and open to varied interpretations. According to the commenter the SED states that: “a research doctorate is a doctoral degree that (1) requires the completion of an original intellectual contribution in the form of a dissertation or an equivalent project (e.g. musical composition), and (2) is not primarily intended as a degree for the practice of a profession. The most common research doctorate degree is the PhD. Recipients of professional doctoral degrees such as MD, DD, JD, DPharm and PsyD are not included in the SED. Thus, this definition identifies only the most common doctorate and categorically eliminates only a handful others.” The commenter suggested that it would be very helpful if the Department provided a comprehensive listing of all the doctorates that the Department would accept.

Discussion: The goal of McNair is the doctorate degree. If the participant earns a doctorate in engineering, then the project should select the appropriate degree code. If the participant did not earn a doctorate (e.g., Master’s) then the project should code the participant accordingly.

The Department agrees with the commenter’s concerns regarding the lack of guidance related to which doctoral degree programs are considered research-intensive. Because institutions vary on what constitutes a research doctorate degree, it is difficult for the Department to provide a comprehensive list. For example, a Doctor in Social Work (DSW) may be considered a research doctorate at University A but may not be considered one at University B. The institution granting the doctorate will define whether the doctorate is research-based; therefore, because of the variations the Department encourages projects to contact the university where the doctorate was earned in order to ascertain whether the doctorate is research-based. Nonetheless in an effort to provide and clarify the instructions, the instructions have been revised.

Action Taken: Revise the instructions to field 44 as follows:

“NOTE: Select the option that indicates the highest graduate degree obtained by the end of the academic year. Do not update this information in subsequent reporting period unless a new or more advanced degree has been earned. Select option 4 only if the doctorate degree was in a research-intensive program. For a list of research doctoral degrees recognized by the Survey of Earned Doctorates, please click: http://www.nsf.gov/statistics/sed/2011/pdf/taba1.pdf. If you are not certain about whether the doctorate is research-based, please contact the institution where the doctorate was granted (e.g., Dean of Graduate Studies). “

Field 47 -- Doctorate Recipient’s Employment Activity

Comments: One commenter stated that there is no clarification in the participant structure, in the instructions, nor in the “Summary,” of when this information should be reported – immediately after the doctorate is awarded? The commenter further stated that many recipients may not find employment immediately following graduation, especially as there is increasing competition for university level jobs, however, a Ph.D. recipient may find employment in research or teaching in a later reporting period.

Discussion: The instructions “Once reported, you should not have to update in subsequent reporting periods.” means that once you report an outcome (e.g., employed in a research activity) you do not have to update this information. If a project has not reported an outcome (e.g., not applicable, participant earned doctorate but is not currently employed) and the participant obtains employment in a subsequent reporting in which the project is still required to track the student, the project would update the field to the relevant employment activity.

Action Taken: None

Summary of Comments on the 2012-13 Ronald E. McNair Postbaccalaureate Achievement Program APR (30-day) Page 8