Parts B ELS 2002 3rd Follow-up PETS and FAFS

The respondent universe for the Education Longitudinal Study of 2002 third follow-up postsecondary education transcript study (ELS:2002 PETS) collection and financial aid student record feasibility study (ELS:2002 FAFS) is described in the first section below. Postsecondary transcript collection will be conducted in three phases with the first phase of transcript collection occurring concurrently with the financial aid student record feasibility study. The second section describes the sampling and statistical methodology proposed for the ELS:2002 PETS and FAFS. The other sections describe methods for maximizing response rates, the special tests of procedures and methods, and the statisticians and other persons responsible for designing and conducting the study.

B.1 Respondent Universe

B.1.a Institution Universe

To be eligible for ELS:2002 PETS and FAFS, an institution will be required to:

be attended, or have been attended, by one or more ELS:2002 third follow-up sample members;
offer an educational program designed for persons who had completed secondary education;
offer at least one academic, occupational, or vocational program of study lasting at least 3 months or 300 clock hours;
offer courses that are open to more than the employees or members of the company or group (e.g., union) that administered the institution;
be located in the 50 states or the District of Columbia; and
be other than a U.S. Service Academy¹.

Institutions providing only vocational, recreational, or remedial courses or only in-house courses for their own employees will be excluded. U.S. Service Academies are excluded from the financial aid feasibility study because of their unique funding/tuition base.

B.1.b Student Universe

The students eligible for inclusion in the sample are any ELS:2002 third follow-up sample members who attended, or attend, one or more of the eligible institutions.

B.2 Statistical Methodology

The ELS:2002 PETS postsecondary transcript collection will be divided into three phases. The FAFS data collection coincides with the PETS first phase.

Phase I. A sample of institutions will be asked to participate in the ELS:2002 PETS transcript collection. That same subset of institutions will be asked to participate in the ELS:2002 FAFS student record collection. .
Phase II. Additional institutions not part of the sample in phase I will be contacted to provide postsecondary transcript data.
Phase III. Institutions identified as part of the phase I and phase II postsecondary transcript collections will be contacted to provide postsecondary transcript data.

This section describes the institution and student sampling for the phase I and the procedures for identifying institutions and students across phase II and phase III.

B.2.a Institution Sample for Phase I

Postsecondary transcripts will be collected during phase 1 (and 2 and 3 as well) of PETS; the FAFS collection coincides specifically with phase 1 of PETS.

The institution sampling frame for ELS:2002 FAFS and phase 1 of ELS:2002 PETS will be constructed from the list of institutions attended as reported by ELS:2002 third follow-up sample members and the latest available IPEDS header, Institutional Characteristics (IC), 12-Month Enrollment, and Completions files. The list of institutions reported by ELS:2002 third follow-up sample members will be linked with information contained in the IPEDS files and the set of eligible institutions reported by ELS:2002 third follow-up sample members will define the institution sampling frame for this phase of the study.

A sample of the eligible institutions reported by ELS:2002 third follow-up sample members² will be selected using stratified simple random sampling. Institutions will be classified into 9 strata derived from the nine sectors traditionally used for postsecondary analyses with the private for-profit 2-year or more sector split into two strata: 2-year and 4-year and with the public 2-year and public less-than-2-year sectors combined. These strata are based on institutional level, control, and highest level of offering:³

1. public 2-year or less⁴

2. public 4-year non-doctorate-granting

3. public 4-year doctorate-granting

4. private nonprofit less-than-4-year

5. private nonprofit 4-year non-doctorate-granting

6. private nonprofit 4-year doctorate-granting

7. private for-profit less-than-2-year

8. private for-profit 2-year

9. private for-profit 4-year.

One rationale for splitting the private for-profit 2-year and 4-year institutions is to reflect the recent growth in enrollment in for-profit 4-year institutions.

While the overall target number of responding institutions is 300, a review of NPSAS:08 data collection activities indicates that some institutions in the created sampling frame may be ineligible, may not agree to participate in the study if selected, or may not be able to abstract student records in the brief feasibility study data collection window even if willing to participate in the transcript collection. Consequently, we propose oversampling institutions in order to adjust for the likelihood of such events. Institutional response targets by stratum are shown in Table 1. Note that since frame counts will not be available until completion of data collection for the ELS:2002 third follow-up, it is possible that the number of eligible institutions in a particular stratum is not sufficient to reach a response goal of 30 institutions within each stratum. If the number of eligible institutions is too small to support the proposed sampling strategy, sampling strata will be combined until the number of eligible institutions in each stratum is sufficient to support a response goal of 30 institutions per stratum. Based on second follow-up enrollment information, there appears to be only one stratum that may lack a sufficient number of eligible institutions: the “Private nonprofit less-than-4-year. Based on current estimates for participation rates and response goals, only one additional institution in this stratum needs to be reported during the third follow-up. However, if this does not occur, the 41 already-identified institutions in the stratum can be included.

Exhibit B-1. ELS:2002 institution sample sizes and yield

Institutional Stratum	Number of Institutions⁵	Response Goal⁶	Adjustment Factors⁷	Number Sampled
Total	2,518	300	1.20	359

Public
2-year or less	770	41	1.14	47
4-year non-doctorate-granting	269	34	1.13	38
4-year doctorate-granting	255	34	1.13	38

Private
Nonprofit less-than-4-year	41	30	1.40	42⁸
Nonprofit 4-year non-doctorate-granting	505	37	1.17	43
Nonprofit 4-year doctorate-granting	245	34	1.20	41
For-profit less-than-2-year	156	30	1.33	40
For-profit 2-year	157	30	1.17	35
For-profit 4-year	120	30	1.17	35

In order to provide information on how institution participation and data quality are impacted by the number of students for whom information is requested, within each institutional stratum, an additional stratification will be implemented that classifies institutions by a measure of compliance burden. Based on prior or current attendance of ELS:2002 third follow-up sample members, institutions will be classified, within stratum, as “high burden” (e.g., the number of ELS:2002 sample members in attendance is higher than the median) or “low burden” (e.g., the number of ELS:2002 sample members in attendance is lower than the median) members. Table 2 reports the number of sampled institutions by stratum, the median number of students per institution per stratum, the definition used to classify an institution as “high burden”, and the estimated student sample per stratum.

Exhibit B-2. ELS:2002 phase I burden estimates

Institutional Stratum	Number of Sampled Institutions	Median Number of Students Per Institution⁹	Number of Students Defining a “High Burden” Institution	Estimated Student Sample Size
Total	359	2	N/A	1,670

Public
2-year or less	47	4	5 or more	312
4-year non-doctorate-granting	38	4	5 or more	214
4-year doctorate-granting	38	12	13 or more	547

Private
Nonprofit less-than-4-year	42	1	2 or more	75
Nonprofit 4-year non-doctorate-granting	43	2	3 or more	123
Nonprofit 4-year doctorate-granting	41	3	4 or more	200
For-profit less-than-2-year	40	1	2 or more	67
For-profit 2-year	35	1	2 or more	64
For-profit 4-year	35	1	2 or more	68

If the overall number of sampled institutions in a given stratum is even, then half of the sampled institutions will be randomly selected from the “low burden” stratum and half from the “high burden” stratum. If the overall number of sampled institutions (N) in a given stratum is odd, then (N-1)/2 institutions will be randomly selected from the “low burden” stratum and (N-1)/2 institutions will be randomly selected from the “high burden” stratum. One additional institution will be drawn from either the “low burden” or “high burden” following a randomization step that assigns the additional draw to the “low burden” stratum with 50% probability and to the “high burden” stratum with 50% probability.

Because additional student-institution linkages will be identified during the ELS:2002 third follow-up main study data collection, the final classification of institutions as “high” or “low” burden cannot occur until after completion of the main study data collection. The third follow-up collection is scheduled to culminate in December 2012 and this phase 1 of the postsecondary transcript collection and financial aid feasibility study is scheduled to begin in January 2013. Therefore, it is possible that there will not be a sufficient number of “high” or “low” burden institutions within a given institutional stratum to support the proposed sampling design. In such an event, the second level of stratification will not be applied to institutions in the corresponding stratum.

B.2.b Student Sample for Phase I

All ELS:2002 third follow-up sample members who attend, or attended, one or more of the sampled institutions – as identified by the sample members themselves – comprise the student sample. While, based on second follow-up postsecondary enrollment information, the average number of ELS:2002 third follow-up sample members to have attended, or attend, an institution is roughly 5, since approximately half of the sampled institutions are “low burden,” we estimate a total of 1,455 students to be represented by the estimated 300 respondent institutions. The 1,455 is calculated by estimating the average number of students that would be expected if the target number of institutions participate in the study. The calculation of the 1,455 is carried out by estimating the average number of students for each stratum and summing the averages. For a given stratum, the average number of students is estimated by assuming that half¹⁰ of the target number of institutions will be “low burden” and half will be “high burden” and then calculating the weighted¹¹ average of the mean number of students attending “high burden” institutions and the mean number of students attending “low burden” institutions.

B.2.c Institution Sample Phase II and Phase III

The institution sample for the second phase of transcript collection will comprise all eligible institutions reported by ELS:2002 third follow-up sample members excluding those institutions sampled for the first phase of ELS:2002 PETS and FAFS. The first and second phase of the postsecondary transcript collection only involves those institutions directly reported by ELS:2002 third follow-up sample members during the third follow-up data collection or earlier.

The third phase of transcript collection involves reviewing transcripts collected during phases one and two, identifying new student-institution linkages not otherwise previously reported by third follow-up sample members, and following up with the institutions so identified. Note that some institutions identified following transcript review will have previously been contacted during phase one or phase two and will be re-contacted in order to retrieve information for the newly linked students. Similarly, some institutions may be identified that were not included during phase one or phase two. These institutions will be contacted in order to retrieve information for the associated students.

B.2.d Student Sample for Phase II and Phase III

All ELS:2002/12 third follow-up sample members who attend, or attended, one or more of the institutions in the third phase institution sample – as identified during processing of phase 1 and phase 2 transcripts – will be included in the third phase student sample.

B.3 Methods for Maximizing Response Rates

The success of the ELS:2002 postsecondary education transcript study (ELS:2002, PETS) and financial aid feasibility study (ELS:2002 FAFS) is fully dependent on the active participation of sampled institutions. The cooperation of an institution’s coordinator is essential as well, and helps to encourage the timely completion of the data collection. Telephone contact between the project team and institution coordinators provides an opportunity to emphasize the importance of the study and to address any concerns about participation.

Proven Procedures. ELS:2002 procedures for working with institutions will be developed from those used successfully in other studies with transcript collections such as the BPS 2009 Postsecondary Education Transcript Study (PETS:09) which combined transcript collections for the B&B:08/09 and BPS:04/09 samples, as well as studies involving student records data collection from institutions, specifically, NPSAS:08 and NPSAS:12 (which is in data collection now). ELS:2002 will use an institution control system (ICS) similar to the system used for PETS:09 to maintain relevant information about the institutions attended by each ELS:2002 cohort member. Institution contact information obtained from the Integrated Postsecondary Education Data System (IPEDS) will be loaded into the ICS and used for all mailings and confirmed during a call to each institution to collect the name of the registrar and financial aid director (or other appropriate contact), address information, telephone and facsimile numbers, and email addresses. This verification call will help to ensure that transcript and student records request materials are properly routed, reviewed, and processed.

Endorsements. In past studies, the specific endorsement of relevant associations has been extremely useful in persuading institutions to cooperate. Endorsements from 17 professional associations were secured for PETS:09 and 26 for NPSAS:12. We plan to contact appropriate associations to request endorsement for ELS:2002 PETS and the ELS:2002 FAFS collections as well; a list of potential associations is provided in appendix 2.

Minimizing burden. Different options for collecting data for sampled students are offered. Institution staff is invited to select the methodology of greatest convenience to the institution. The optional strategies for obtaining the data are discussed later in this section.

Another strategy that RTI has used successfully to increase the efficiency of institution data collections, encourage participation, and minimize burden is to solicit support at a system-wide level rather than contacting each institution within the system separately. A timely contact, together with enhanced verification procedures, is likely to reduce the number of remail requests, and minimize delay caused by misrouted requests.

B.3.a ELS:2002 Postsecondary Education Transcript Study (ELS:2002, PETS) collection

RTI recently completed PETS:09, a collection of approximately 45,000 postsecondary transcripts for the BPS:04/09 and B&B:08/09 samples. The same processes and systems found to be effective in PETS:09 will be adapted for use for the ELS:2002 PETS. They are described below.

Data request materials and prompting. Transcript data will be requested for sampled students from all institutions attended since high school. The descriptive materials sent to institutions to request data will be clear, concise, and informative about the purpose of the study and the nature of subsequent requests. The package of materials sent to the transcript coordinators, provided in appendix 1, will contain:

A letter from RTI providing an introduction to the ELS:2002 PETS,
An introductory letter from NCES on U.S. Department of Education letterhead,
A letter of endorsement from the American Association of Collegiate Registrars and Admission Officers (AACRAO),
A list of other endorsing agencies,
Information regarding how to log on to the study’s secure website and access the list of students for which transcripts are requested as well as a form in which they can request reimbursement of expenses incurred with the request (e.g., transcript processing fees), and
Descriptions of and instructions for the various methods of providing transcripts.

During the phase I collection of the ELS:2002 PETS, the institutions will also be asked to participate in the ELS:2002 FAFS by providing financial aid student records data (described in greater detail in a separate section below). After phase I of the ELS:2002 PETS, the remaining institutions that ELS:2002 cohort members have attended will be contacted for transcripts.

Follow-up calls to ensure receipt of the packet and answer any questions about the study will occur 2 days after the initial phase I mailing. We anticipate that telephone prompting also will be required to obtain the desired number of transcripts. Despite the relatively routine nature of the transcript request many institutions give relatively low priority to voluntary research requests. Telephone follow-up is necessary to ensure that the request is handled within the schedule constraints. In addition to telephone prompting, institutions will be contacted by email prompts, letters, and postcard prompts.

Experienced staff from RTI’s Call Center Services (CCS) will carry out these contacts and will be assigned a set of institutions that is their responsibility throughout the process. This allows RTI staff members to build a relationship and maintain rapport with the institution staff and provides a reliable point of contact at RTI. Project staff members will be thoroughly trained in transcript collection and in the purposes and requirements of the study, which helps them establish credibility with the institution staff. Because institution coordinators are a critical element in this process, communicating instructions about their transcript collection tasks clearly is essential.

Data submission options. Several methods will be used for obtaining the data including: (1) institution staff uploading electronic transcripts for sampled students to the secure study website; (2) institution staff sending electronic transcripts for sampled students by secure File Transfer Protocol; (3) institution staff sending electronic transcripts as encrypted attachments via email; (4) for institutions that already use this method, RTI requesting/collecting electronic transcripts via a dedicated server at the University of Texas at Austin; (5) institution staff sending electronic transcripts via eSCRIP-SAFE™, in which institutions send data to the eSCRIP-SAFE™ server by secure internet connection where they can be downloaded only by a designated user; (6) institution staff transmitting transcripts via a secure electronic fax after a test submission of nonsensitive data confirms that the institution has the correct fax number; and as a last resort, (7) sending transcripts via Federal Express. Each method is described below.

A complete transcript from the institution will be requested as well as the complete transcripts from transfer schools that the students attended, as applicable. To track receipt of institution materials and student transcripts, we will add a Transcript Control System (TCS) to the IMS developed for ELS:2002 PETS. The TCS will track the status of each catalog and transcript request, from initial mailout of the requests through follow up and final receipt.

Uploading electronic transcripts to the secure study website. Goals for ELS:2002 PETS include reducing the data collection burden on institutions (thereby reducing project costs), expediting data delivery, improving data quality, and ensuring data security.

Because the open internet is not conducive to transmitting confidential data, any internet-based data collection effort necessarily raises the question of security. However, we intend to incorporate the latest technology systems into our web application to ensure strict adherence to NCES confidentiality guidelines. Our web server will include a Secure-Sockets Layer (SSL) Certificate, and will be configured to force encrypted data transmission over the Internet. The SSL technology is most commonly deployed and recognizable in electronic commerce applications that alert users when they are entering a secure server environment, thereby protecting credit card numbers and other private information. Also, all of the data entry modules on this site are password protected, requiring the user to log in to the site before accessing confidential data. The system automatically logs the user out after 20 minutes of inactivity. This safeguard prevents an unauthorized user from browsing through the site.

Files uploaded to the secure website will be immediately moved to a secure project folder that is only accessible to specific staff members. Access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. It is necessary for the files to be stored on the project share so that they can be backed up by ITS in case any problems occur that cause us to lose data. ITS will use their standard procedures for backing up data, so the backup files will exist for 3 months.

Institution staff sending electronic transcripts by secure File Transfer Protocol. FTPS (also called FTP-SSL) uses the FTP protocol on top of SSL or TSL. When using FTPS, the control session is always encrypted. The data session can optionally be encrypted if the file has not been pre-encrypted.

Files transmitted via FTPS will be copied to a secure project folder that is only accessible to specific staff members. As with uploaded files, access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. After being copied, the files will be immediately deleted from the FTP server. It is necessary for the files to be stored on the project share so that they can be backed up by ITS in case any problems occur that cause us to lose data. ITS will use their standard procedures for backing up data, so the backup files will exist for 3 months.

Institution staff sending electronic transcripts as encrypted attachments via email. RTI will provide guidelines on encryption and creating strong passwords. Encrypted electronic files sent via email to a secure email folder will only be accessible to a few staff members on the project team. These files will then be copied to a project folder that is only accessible to these same staff members. Access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. After being copied, the files will be deleted from the email folder. The files will be stored on the network that is backed up regularly to avoid the need to recontact the institution to provide the data again should a loss occur. RTI’s information technology service (ITS) will use standard procedures for backing up data, so the backup files will exist for 3 months.

Institution staff sending electronic transcripts via eSCRIP-SAFE™. This method involves the institution sending data via a customized print driver which connects the student information system to the eSCRIP-SAFE™ server by secure internet connection. RTI, as the designated recipient, can then download the data after entering a password. The files are deleted from the server 24 hours after being accessed.

The transmission between sending institutions and the eSCRIP-SAFE™ server is protected by Secure Socket Layer (SSL) connections using 128-bit key ciphers. Remote access to the eSCRIP-SAFE™ server via the Web interface is likewise protected via 128-bit SSL.

Downloaded files will be moved to a secure project folder that is only accessible to specific staff members. Access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. It is necessary for the files to be stored on the project share so that they can be backed up by ITS in case any problems occur that cause us to lose data. ITS will use their standard procedures for backing up data, so the backup files will exist for 3 months.

Institution staff transmitting transcripts via a secure electronic fax (e-fax). We expect that few institutions will ask to provide hardcopy transcripts. In such cases, we will encourage one of the secure electronic methods of transmission. If that is not possible, we will accept faxed transcripts. Although fax equipment and software does facilitate rapid transmission of information, this same equipment and software opens up the possibility that information could be misdirected or intercepted by individuals to whom access is not intended or authorized. To safeguard against this, as much as is practical, RTI protocol will only allow for transcripts to be faxed to an electronic fax machine and only if institutions cannot use one of the other options. To ensure the fax transmission is sent to the appropriate destination, we will require a test run with nonsensitive data prior to submitting the transcripts to eliminate errors in transmission from misdialing. RTI will provide schools with a fax cover page that includes a confidentiality statement to use when transmitting individually identifiable information.

Transcript data received via e-fax are stored as electronic files on the e-fax server, which is housed in a secured data center at RTI. These files will be copied to a project folder that is only accessible to project staff members. Access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. After being copied, the files will be deleted from the e-fax server. The files will be stored on the network that is backed up regularly to avoid the need to recontact the institution to provide the data again should a loss occur. RTI’s information technology service (ITS) will use standard procedures for backing up data, so the backup files will exist for 3 months.

Institution staff sending transcripts via Federal Express. We expect that few institutions will ask to provide hardcopy transcripts. In such cases, we will encourage one of the secure electronic methods of transmission or fax. If that is not possible, we will accept transcripts sent Federal Express. Before sending, we will instruct the institution staff to redact any personally identifiable information from the transcript including student name, address, data of birth, and Social Security Number (if present).

Paper transcripts will be scanned and stored as electronic files. These files will be stored in a project folder that is only accessible to project staff members. Access to this project folder will be set so that only those who have authorized access will be able to see the included files. The folder will not even be visible to those without access. The files will be stored on the network that is backed up regularly to avoid the need to re-contact the institution to provide the data again should a loss occur. RTI’s information technology service (ITS) will use standard procedures for backing up data, so the backup files will exist for 3 months. The original paper transcripts will be shredded.

Collecting electronic transcripts via a dedicated server at the University of Texas at Austin. We will also request and collect transcripts electronically via a dedicated server at the University of Texas at Austin for institutions that currently use this method. Approximately two hundred institutions are currently registered to send and receive academic transcripts in standardized electronic formats via a dedicated server at the University of Texas at Austin. The server supports Electronic Data Interchange (EDI) and XML formats. Additional institutions are in the test phase with the server, which means that they are preparing and testing using the server but not currently using it to send data.

The dedicated server at the University of Texas at Austin supports the following methods of securely transmitting transcripts:

email as MIME attachment using PGP encryption
regular FTP using PGP encryption
Secure FTP (SFTP over ssh) and straight SFTP
FTPS (FTP over SSL/TLS)

Files collected via the dedicated server at the University of Texas at Austin will be copied to a secure project folder that is only accessible to specific staff members. The same access restrictions and storage protocol will be followed for these files as described above for files uploaded to the study website.

We do not anticipate that active student consent for the release of transcripts will be required for ELS:2002, PETS. For certain agency purposes, the Family Educational Rights and Privacy Act of 1974 (FERPA) permits institutions to release student data to the U.S. Department of Education and its authorized agents without consent. In compliance with FERPA, a notation will be made in the student record that the transcript has been collected for use in the ELS:2002 longitudinal study.

Despite the relatively routine nature of the transcript request, it is anticipated that telephone prompting will be required to obtain the desired number of transcripts. We will also use email prompts, letters, and postcard prompts, which have proven to be effective tools in gaining cooperation. Email, in particular, has proven to be a low-cost and effective means of reaching institution officials who cannot be reached by phone. Because institutions can request reimbursement for expenses incurred in handling the request, it is unlikely that refusals will become a significant problem. However, in the event that an institution expresses resistance to the transcript request, seasoned institutional contactors and other project staff are trained to sensitively listen to institutional concerns, address any roadblocks to participation, and negotiate with institution staff to resolve them.

Quality control and initial processing. As part of our quality control procedures, we will emphasize to registrars the importance of collecting complete transcript information for all sampled students. Transcripts will be reviewed for completeness. Institutional Contactors will contact the institutions to prompt for missing data and to resolve any problems or inconsistencies.

Transcripts received in hardcopy form will be subject to a quick review prior to recording their receipt. Receipt control clerks will check transcripts for completeness and review transmittal documents to ensure that transcripts have been returned for each of the specified sample members. The disposition code for transcripts received will be entered into the TCS. Course catalogs will also be reviewed and their disposition status updated in the system in cases where this information is necessary and not available through CollegeSource Online. Hardcopy course catalogs will be sorted and stored in a secure facility at RTI, organized by institution.

The procedures for electronic transcripts will be similar to those for hardcopy documents—receipt control personnel, assisted by programming staff, will verify that the transcript was received for the given requested sample member, record the information in the receipt control system, and check to make sure that a readable, complete electronic transcript has been received.

The initial transcript check-in procedure is designed to efficiently receipt returned materials into the TCS as they are received each day. The presence of an electronic catalog (obtained from CollegeSource Online) will be confirmed during the verification process for each institution and noted in the TCS. The remaining catalogs will be requested from the institutions directly and will be receipted in the TCS as they are received. Transcripts and supplementary materials received from institutions (including course catalogs) will be inventoried, assigned unique identifiers based on the IPEDS ID, reviewed for problems, and receipted into the TCS.

Data processing staff will be responsible for (1) sorting transcripts into alphabetical order to facilitate accurate review and receipt; (2) assigning the correct ID number to each document returned and affixing a transcript ID label to each; (3) reviewing the materials to identify missing, incomplete, or indecipherable transcripts; and (4) assigning appropriate TCS problem codes to each of the missing and problem transcripts plus providing detailed notes about each problem to facilitate follow-up by Institutional Contactors and project staff. Project staff will use daily monitoring reports to review the transcript problems and to identify approaches to solving the problems.

Web-based collection will allow timely quality control, as RTI central staff will be able to monitor data quality for participating schools closely and on a regular basis. When institutions call for technical or substantive support, we will be able to query the institution’s data and communicate much more effectively regarding any problems.

Transcript data will be destroyed or shredded after the transcripts are keyed, coded, and quality checked at a time to be negotiated with NCES.

Transcript Keying and Coding. Once student transcripts and course catalogs are received and missing information is collected, keying and coding of transcripts and courses taken will take place. As part of PETS:09, RTI updated and enhanced the course classification structure used on NELS:88. The result, the 2010 College Course Map¹², is a hybrid PETS coding taxonomy that made it easier for KCs to select an appropriate code for the courses they identify on transcripts.

During the pilot test, RTI will review the adequacy of this coding system for ELS:2002, PETS. One form of review will be analysis of courses coded as “other” to see if clusters of similar courses may merit additions to the taxonomy. Guidelines and recommendations for modifications to the taxonomy will be delivered to NCES for review and feedback prior to full scale keying and coding.

Keyer-Coders will have full access to all transcript-related documents including course catalogs or other course listings provided. All transcript-related documents will be thoroughly reviewed before data are abstracted from them.

Transcript Keying and Coding Quality Control. A comprehensive supervision and quality control plan will be implemented during transcript keying and coding. At least one supervisor will be onsite at all times to manage the effort and simultaneously perform QC checks and problem resolution. Verifications of transcript data keying and coding at the student level will be performed. Any errors will be recorded and corrected as needed.

Once the transcripts for each institution are keyed and coded, transcript course coding at the institution level will be reviewed by expert coders to ensure that (1) coding taxonomies have been applied consistently and data elements of interest have been coded properly within schools (2) program information has been coded consistently according to the program area and sequence level indicators in course titles (3) records of sample members who attended multiple institutions do not have duplicate entries for credits that transferred from one institution to another and (4) additional information has been noted and coded properly.

B.3.b ELS:2002 Financial Aid Feasibility Study (ELS:2002 FAFS) collection

RTI will offer several options to institutions for providing the financial aid student records for students, similar to those used for NPSAS:12 and invite the institution coordinator to select the methodology that is least burdensome and most convenient for the institution. The optional methods for providing financial aid student record data are described below.

Student Records obtained via a web-based data entry interface. The web-based data entry interface is flexible and allows the coordinator to enter data in one of two data entry modes. One data entry mode resembles a spreadsheet (referred to as “grid mode”) and as such, the coordinator can view and edit multiple student records at a time or, where data elements require it, all academic years for one student at a time. The other data entry mode displays one student at a time, one year at a time, and the coordinator may enter data in a top to bottom fashion before moving onto the next student.

Student Records obtained by completing an Excel workbook. An Excel workbook will be created for each institution and will be preloaded with the sampled students’ ID, name, date of birth, and SSN (if available). To facilitate simultaneous data entry by different offices within the institution, the workbook contains a separate worksheet for each of the following topic areas: Student Information, Financial Aid, Enrollment, and Budget. The user will download the Excel worksheet from the secure ELS:2002 institution website, enter the data, then upload the data to the website. Validation checks will occur both within Excel as data are entered and when the data are uploaded via the website. Data will be imported into the web application such that institution staff can quality control their data.

Student Records obtained by uploading CSV (comma separated values) files. Institutions with the means to export data from their internal database systems to a flat file may opt for this method of supplying financial aid records. Institutions that select this method will be provided with detailed import specifications, and all data uploading will occur through the project’s secure website. Like the Excel workbook option, data will be imported into the web application such that institution staff can check their data before finalizing.

Institution coordinators will receive a guide that provides instructions for accessing and using the website. In conjunction with the transcript collection pilot test, institution contacting staff at RTI will make initial telephone calls to notify institutions that the financial aid student records data collection has begun. Using daily status reports that summarize the progress of the institutions, staff will also call institutions periodically to prompt completion of the financial aid student records collection. Help desk project staff are available by telephone or by email to provide assistance if institution staff have questions or encounter problems.

B.4 Statisticians and Individuals Responsible for Designing and Conducting the Study

A number of individuals have consulted with NCES and RTI on the design and analysis plans for the ELS:2002 Postsecondary Education Transcripts Study (PETS) and Financial Aid Feasibility Study (FAFS). Members of the TRP have been described in Part A of this submission. In addition, the following personnel at NCES have reviewed and approved the statistical aspects of the study: Elise Christopher, Sarah Crissey, Tracy Hunt-White, Jeff Owings, Marilyn Seastrom, Sean Simone, Ted Socha, and Matt Soldner. Exhibit B-3 provides the names of RTI consultants on statistical aspects of the ELS:2002, PETS and ELS:2002, FAFS, while Exhibit B-4 lists other principal RTI professional staff assigned to the study.

Exhibit B-3. Contractor Consultants on Statistical Aspects of the ELS:2002 PETS and ELS:2002 FAFS

Name	Affiliation
James Chromy	RTI
Steven J. Ingels	RTI
Daniel J. Pratt	RTI
John Riccobono	RTI
Peter H. Siegel	RTI
David Wilson	RTI
Jennifer S. Wine	RTI

Exhibit B-4. Other Contractor Staff Responsible for Conduct of the ELS:2002 PETS and ELS:2002 FAFS

Name	Affiliation
Chris Alexander	RTI
Mike Bryan	RTI
Ben Dalton	RTI
Kristin Dudley	RTI
Donna Jewell	RTI
Erich Lauff	RTI
Tiffany Mattox	RTI
Annaliza Nunnery	RTI
Jim Rogers	RTI

1 While U.S. Service Academies will not be part of the first phase of transcript collection, transcripts will be requested from them during the second and third phases of transcript data collection.

2 Transcripts and financial-aid information will be pursued not just for third follow-up respondents but also for third follow-up nonrespondents who specified postsecondary enrollment as of the second follow-up.

3 The institutional strata can be aggregated by control or level of the institution for the purposes of reporting aggregate statistics.

4 The Public, 2-year and Public, less-than-2-year sectors are combined due to the small number of institutions in the Public, less-than-2-year sector.

5 The number of institutions is derived from postsecondary enrollment information provided by sample members as of the second follow-up.

6 The response goals for the nine institutional strata were determined by first setting a goal of 30 per stratum, which would yield a total response of 270, so 30 additional institutions were allocated proportionally to the five largest strata.

7 Adjustment factors are derived from NPSAS:08 institutional eligibility rates, institutional agreement-to-participate rates, and institutional student record abstraction rates. For purposes of sample size calculation, the average overall eligibility rate was assumed to be 95%. The average institution agreement-to-participate rate was assumed to be 88%. The average rate at which participating institutions provide record abstraction data was assumed to be 98%.

8 The total number of nonprofit less-than-4-year schools should increase with the collection of third follow-up data so that 42 institutions will be able to be sampled; however, if this does not occur, then the full set of 41 already-identified institutions in this stratum will be included.

9 Medians are estimated from the post-secondary enrollment information collected as of the second follow-up.

10 This is true if the number of target institutions is even. If the number of target institutions is odd, say N, then (N-1)/2 institutions are assumed to be “high burden” and (N+1)/2 institutions are assumed to be “low burden.”

11 The weights will be .5 if the target number of institutions (N) is even and will be (N-1)/2N and (N+1)/2N if odd.

12 Bryan, M. & Simone, S. (2012). 2010 College Course Map (NCES 2012-162REV). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC. Retrieved [date] from http://nces.ed.gov/pubsearch.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Chapter 2
Author	spowell
File Modified	0000-00-00
File Created	2021-01-30