Att_TAD OMB Part B

This package requests approval for a data collection for the National Evaluation of the Technical Assistance and Dissemination (TA&D) Program. Data collection will focus on gathering relevant information on the Office of Special Education Programs (OSEP)-funded TA&D Program from program grantees and from state agency staff. A separate package for a subsequent data collection – that will be shaped, in part, by findings from the first collection – will be submitted for review at a later date.

Introduction

Part D of the Individuals with Disabilities Education Act (IDEA) specifies that the TA&D Program will provide technical assistance, support model demonstration projects, disseminate useful information, and implement activities that are supported by scientifically based research (IDEA 2004, P.L. 108-446 Part D Section 663, 118 Stat. 2781). While the federal government has been funding projects that provide technical assistance related to the education of individuals with disabilities for four decades, the TA&D Program assumed its current structure with the 1997 reauthorization of IDEA. The current TA&D Program (funded at 49.5 million dollars in FY 2010) awards grants in nearly 20 subprogram areas, with grants ranging in size from approximately $65,000 per year to approximately $2.8 million per year. Program grantees are located throughout the U.S. and recipients include institutions of higher education, for-profit organizations and private nonprofit institutes and organizations.

The IDEA TA&D Program is based on the assumption that outcomes can be improved for children with disabilities when a knowledge base is disseminated to practitioners (and families) through technical assistance. In turn, the Program awards grants to fund a network of organizations staffed by skilled technical assistance providers. Exhibit A-1 (see Part A, Justification) depicts the theory of action that describes how the TA&D Program should work. It begins with the assumption that useful evidence-based knowledge exists. This knowledge is drawn on, and added to, by TA&D Program grantees who determine the best methods for translating the knowledge into forms that can be used by practitioners. For the sake of efficiency, and to ensure that the knowledge reaches the broadest target audience of practitioners, the grantees work through other entities that have direct relationships with practitioners. These entities include state education agencies (SEAs) and Part C lead agencies, institutes of higher education (IHE) faculty and researchers, other relevant stakeholders such as other child-serving agencies, and, in some cases, families and children.

As Exhibit A-1 illustrates, the focus of the technical assistance that grantees provide is informed by both the knowledge base and the need for technical assistance expressed by the entities served. OSEP has determined that the most powerful course for change is the provision of technical assistance to SEAs and Part C lead agencies, which builds state capacity and enables the state agencies to better support local education agencies and local Part C organizations. This technical assistance ultimately results in changes in local policy and local practice that have a positive impact on children. In sum, the theory of action underpinning the TA&D Program is that OSEP-supported grantees can translate and disseminate evidence-based knowledge into forms that can be disseminated, through technical assistance activities, to practitioners whose changed practice will lead to improved outcomes for children with disabilities.

While the model depicted in Exhibit A-1 (Part A of this submission) focuses on the TA&D Program, it should be noted that the TA&D Program is part of a larger system of technical assistance funded by OSEP called the TA&D Network. In recent years, OSEP has evolved its TA&D efforts toward this concept of a network of centers that provide complementary services. The TA&D Network is a group of 46 centers (plus the center responsible for coordination), funded from across OSEP Part D programs and organized into 13 categories. These centers are intended to coordinate their efforts to provide states and other recipients with appropriate assistance and without duplication of efforts to work toward the goal of improved outcomes for children with disabilities and their families. Of the current 86 TA&D Program grantees, 27 of them are also members of the TA&D Network. The remaining 60 TA&D Program grantees are not members of the TA&D Network. While an evaluation of network functioning is an important topic, it is not the focus of this evaluation.

All funded projects provide technical assistance and dissemination services with the broad goals of (1) ensuring that Parts B and C of IDEA are implemented effectively and (2) improving results for children with disabilities. However, projects vary in structural and substantive ways. Most broadly, the currently active grantees can be described as belonging to one of five groups: 1) National centers, which focus on a particular topic area; 2) Regional Resource Centers, which serve and support state needs; 3) the PEPNet program, which has as its mission to improve transition services and educational access for students who are deaf or hard of hearing 4) Model demonstration centers, which have as their goal to examine a specific practice in a limited number of sites; and 5) State deaf-blind centers, which serve students in this population within their state. Grantees also differ in the way in which they provide their services, including the topic areas of focus; type of TA provided; intended outcomes; methods of service delivery; level/intensity of services and activities provided; and population and number of customers served.

Overview of the Evaluation

The National Evaluation of the TA&D Program is being conducted by the National Center for Education Evaluation (NCEE) in the Institute of Education Sciences (IES) at the U.S. Department of Education (ED). While technical assistance is often hailed as a critically important area for the effective implementation of federal policy, no independent data are currently available on the role that the TA&D Program plays in supporting state agencies in their implementation of IDEA. This evaluation will provide important information on the needs that state agencies have for technical assistance in special education, the products and services provided by the TA&D Program, and the extent to which these are meeting the needs of the state agencies for the particular topic areas upon which they focus. This information will set the stage for a more focused examination of how technical assistance might relate to improved state-capacity, changes in local policies and practices, and ultimately, improved child outcomes.

This data collection will focus on gathering specific information on the TA&D Program from program grantees and from officials at SEAs and Part C lead agencies. A TA&D Program grantee questionnaire/interview will yield detailed descriptive information about TA&D Program grantees, including the topic areas on which they focus, the particular practices and outcomes on which grantees are focused, as well as the technical assistance products and services provided by the TA&D Program grantees and to whom they provide them. State surveys will provide information concerning the needs that SEAs and Part C lead agencies have for technical assistance to support the implementation of IDEA and support improvement of child outcomes, and the technical assistance services and products that have been received or accessed at the state level from OSEP TA&D Program centers and satisfaction with those services and products.

A subsequent data collection for the evaluation is planned to focus on implementation of practices at the local level following the state agency’s receipt of technical assistance from TA&D Program-supported centers. The design of the second data collection will be informed by the findings of the first, particularly those findings that provide information about improving state-capacity, changing local policies and practices, and ultimately, improving child outcomes. These data collection plans will be submitted for public comment and OMB review under a separate package at a later date.

B.1 Respondent Universe and Sampling Methods

Data collection will include: (1) a TA&D Program grantee questionnaire/interview and (2) state surveys. No sampling is proposed for either component of the data collection.

TA&D Program Grantee Questionnaire/Interview

All TA&D Program grantees that are active as of August 1, 2011 (including those under a no-cost extension) will be included, with the exception of two groups of centers: state deaf-blind project grantees and model demonstration grantees. Both of these two groups of grantees are under evaluation through other data collection efforts, and we do not want to duplicate effort by obtaining information from them at the same time.

Based on the currently active grantees, there will be 27 TA&D Program grantee project directors that will be included as respondents for the TA&D Program grantee questionnaire/interview:

National Consortium on Deaf-Blindness (NCDB)
Center for Appropriate Dispute Resolution in Special Education (CADRE)
Early Childhood Outcomes Center (ECO)
National Early Childhood Technical Assistance Center (NECTAC)
Center for Early Literacy Learning (CELL)
Technical Assistance Center on Social Emotional Intervention for Young Children (TACSEI)
Center on Positive Behavioral Interventions and Supports
National Center on Response to Intervention (NCRTI)
Center on State Implementation and Scaling-up of Evidence-based Practices (SISEP)
IDEA Partnership
National Dissemination Center for Children with Disabilities (NICHCY)
Project Forum
National Center on Educational Outcomes (NCEO)
National Post-School Outcomes Center (NPSO)
National Dropout Prevention Center for Students with Disabilities (NDPC-SD)
National Secondary Transition Technical Assistance Center (NSTTAC)
Technical Assistance Coordination Center (TACC)
Northeast Regional Resource Center (NERRC)
Mid-South Regional Resource Center (MSRRC)
North Central Regional Resource Center (NCRRC)
Southeast Regional Resource Center (SERRC)
Mountain Plains Regional Resource Center (MPRRC)
Western Regional Resource Center (WRRC)
PEPNet-Northeast
PEPNet-Midwest
PEPNet-South
PEPNet-West

State Surveys

The state surveys will be sent to all 50 states, the District of Columbia, Puerto Rico, the Virgin Islands, American Samoa, the Commonwealth of the Northern Mariana Islands, and Guam.

B.2 Information Collection Procedures

TA&D Program Grantee Questionnaire/Interview

To help ensure a high response rate among TA&D Program grantees, we will provide information to all the TA&D Program Project Officers, including a presentation at OSEP’s annual Project Director’s meeting, so they can let their grantees know about the upcoming data collection effort. We do not anticipate challenges in achieving participation among TA&D Program grantees.

A personalized cover letter and the TA&D Program Grantee Questionnaire will be mailed to the TA&D Program grantee Project Director. We will make follow-up contact after 1 week to ensure that the questionnaire was received and remind respondents to complete it. At this time, we will also explain the purpose of the two semi-structured interviews and schedule time to complete them. Once the questionnaire and interviews have both been completed, the data collection will be closed out.

State Surveys

The following six steps will be followed in the course of state survey administration and collection of information.

Step 1: Collect, and confirm or update, contact information (name, address, telephone, e-mail address) of primary survey respondents through publicly accessible databases and websites and by follow-up calls to state offices if necessary.

Step 2: Mail an advance letter from Westat to inform respondents of the forthcoming survey, to share with them the purpose of the study and its importance, and to ask for their cooperation.

Step 3: Mail a personalized cover letter that explains the survey and what participation entails, provides assurance of confidentiality, and provides the web address for the on-line survey with a separate set of instructions for completing the on-line survey. After 1 week, send this information by email if the respondent has not yet logged into the survey.

Step 4: Make follow-up contact by telephone or email to confirm receipt of the survey information packet by the correct person and to address technical difficulties with the web-based survey or answer questions about specific items. These calls will be made within 10 days of the initial letter if the respondent has not yet logged in.

Step 5: Continue email and telephone contact with respondents on an ongoing basis, prioritizing those respondents who have not begun or have completed only a limited portion of the survey.

Steps 4 and 5 will also be conducted at the module level, e.g., with the secondary respondents that are identified by the state Part B director or state Part C coordinator. All follow-up within a given state will be conducted by the same individual to allow a cohesive process of data retrieval, and to immediately identify any cases where a secondary respondent may not have received correct information.

Step 6: Close out data collection.

B.3 Methods to Maximize Response Rates

Section B.2 describes the procedures that we will use to implement the TA&D Program grantee questionnaire/interview and the state surveys. These procedures were developed to encourage cooperation and completion of the activities within the data collection period. We anticipate that we will easily be able to obtain a 100% response rate with the TA&D Program grantees. Exhibit B-1 highlights the specific strategies we will employ to maximize response rates for the state surveys and deal with issues of non-response.

Exhibit B-1: Strategies to Maximize Response Rates
Design a high quality and user-friendly instrument	The state survey has been pre-tested to ensure that the questions are clear and as user-friendly as possible (in particular, many of the items are answered by checking off boxes rather than writing in responses), and the survey can be completed quickly. It has also been kept short by excluding requests for information that can be obtained from other data sources. Use of skip patterns and looping as a way to present a seamless experience from the perspective of the respondent.
Advance notification of survey	Gain support and cooperation of state administrators by providing advance notice of the survey
Provide clear instructions and user-friendly materials	Send introductory letter from Westat along with a personalized cover letter that explains the survey and what participation entails, provides assurance of confidentiality, and provides the web address for the on-line survey along with instructions for completing the on-line survey.
Offer technical assistance for survey respondents	Provide toll-free technical assistance telephone number Provide study website with instructions for web-based survey completion
Monitor progress regularly	Produce weekly data collection report of completed surveys Maintain regular contact between study team members to monitor response rates, identify non-respondents, and resolve problems Use follow-up and reminder calls and e-mails to non-respondents

B.4 Test of Procedures

The TA&D Program grantee questionnaire/interview and state surveys were tested internally, and information derived from the tests was used to refine them. The TA&D Program grantee questionnaire/interview was also pre-tested with four current Project Directors of non-TA&D program centers. These Project Directors were selected because they operate technical assistance centers that provide similar services as the centers under evaluation, serve similar populations, and therefore are likely to have similar experiences to the TA&D program grantee project directors. As part of the pre-testing, the respondents completed the questionnaire and then participated in telephone calls or in-person interviews with project staff in order to pre-test the interview items. The information obtained from the pre-test respondents was used to refine questionnaire/interview items and to assess potential burden. The Part B and Part C state surveys were tested with five former state Part B Directors and three former state Part C Coordinators. Again, these individuals were selected because their experiences were likely to be similar to the Part B and Part C survey respondents. Pre-test respondents were sent the survey to review/complete and then asked to participate in a telephone call with project staff to go over their responses and comments. The information obtained from the pre-test respondents was used to refine survey items and to assess potential burden. It should be noted that pre-testing of the state surveys involved various iterations of the surveys to ensure confidence that the refined survey items were capturing key constructs.

B.5 Individuals Consulted on Statistical Aspects of Design

These data collection plans were developed by Westat. The research team is led by Tamara Daley, project director and Tom Fiore, Principal Investigator. Other members of the evaluation team who worked on the design include Julie Bollmer and Chris Lysy from Westat. The NCEE project officer, Meredith Bachman, also played a central role in data collection plans. The senior consultants, Debra Price-Ellingstad and Sharon Walsh, reviewed multiple drafts of the instruments and provided comments. In addition, one of the TWG members, Jim Hamilton, reviewed the grantee protocol from the perspective of having been a former director of a technical assistance center. Additional input was provided by Jonathan Jacobson, NCEE. Contact information for these individuals is provided below.

Meredith Bachman

Institute of Education Sciences, U.S. Department of Education

202-219-2014

Jonathan Jacobson

Institute of Education Sciences, U.S. Department of Education

202-208-3876

Tamara Daley

Westat

919-474-8038

Tom Fiore

Westat

919-454-0329

Julie Bollmer

Westat

301-517-8094

Chris Lysy

Westat

919-474-2485

Debra Price-Ellingstad

Minnesota Department of Education

651-582-8568

Sharon Walsh

Walsh-Taylor Associates

703-250-4935

Jim Hamilton

Retired

703-532-9435

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Tamara Daley
File Modified	0000-00-00
File Created	2021-01-31