THE SUPPORTING STATEMENT
Specific Instructions
Please do not remove or alter the headings below
A. Justification
Circumstances Making the Collection of Information Necessary
The Administration on Developmental Disabilities (ADD) oversees the implementation of the Developmental Disabilities Assistance and Bill of Rights Act of 2000 (DD Act) (P.L. 106-402) (42 USC 15062). The purpose of the DD Act is to assure that individuals with developmental disabilities and their families participate in the design of and have access to needed community services, individualized supports, and other forms of assistance that promote self-determination, independence, productivity, and integration and inclusion in all facets of community life.
As defined in the DD Act, the term “developmental disabilities” means a severe, chronic disability of an individual that is attributable to a mental or physical impairment or combination of mental and physical impairments that is manifested before the individual attains age 22 and is likely to continue indefinitely. Developmental disabilities result in substantial limitations in three or more of the following functional areas: self-care, receptive and expressive language, learning, mobility, self-direction, capacity for independent living, and capacity for economic self-sufficiency. It is estimated that four million people in America have developmental disabilities.
The DD Act authorizes appropriations for three programs in the States to achieve the purposes of the Act:
State Developmental Disabilities Councils;
State Protection and Advocacy Systems to Protect the Rights of Individuals with Developmental Disabilities; and
The National Network of University Centers for Excellence in Developmental Disabilities, Education, Research, and Service.
ADD is currently conducting the National Study of the State Developmental Disabilities Programs (National Study) to assess the effectiveness and achievements of the three programs under the DD Act – the State Developmental Disabilities Councils, State protection and Advocacy Systems, and National Network of University Centers for Excellence – and collaboration among them,. This study received OMB approval on December 31, 2009 (ICR Reference number 200907-0970-002; OMB Control Number 0970-0372).
A modification to the study calls for an assessment of ADD to examine ADD’s efficiency and effectiveness to support these DD Network programs. One component of the ADD assessment is a web-based questionnaire to all Executive Directors of DD Network programs. This survey is the subject of this supporting statement and the purpose of this submission. All other data collection for the ADD assessment consists of qualitative interviews with Federal employees or fewer than 10 people who represent ADD stakeholder groups.
There are several legal and administrative requirements that as a combined constellation necessitate the National Study. As described in the previous supporting statement to OMB on the National Study, these include initiatives over the past several Administrations to promote accountability of federally funded programs.
The DD Act requires a system of accountability for the DD Act programs. Specifically Section 105 requires that submit to the President, Congress, and the National Council on Disability a report that describes the goals and outcomes of programs.
The Government Performance and Results Modernization Act of 2010 provides another basis for conducting the information collection. Among the purposes of GPRA is to: improve the confidence of the American people in the capability of the Federal Government, by systematically holding Federal agencies accountable for achieving program results; improve Federal program effectiveness and public accountability by promoting a new focus on results, service quality, and customer satisfaction; help Federal managers improve service delivery, by requiring that they plan for meeting program objectives and by providing them with information about program results and service quality; and to improve internal management of the Federal Government.
Executive Order 13450 Improving Government Program Performance calls for improving the effectiveness and efficiency of the Federal Government and promoting greater accountability of that Government to the American people.
The current Administration has outlined in the document “Building a High Performing Government”. It calls for improving results and outcomes for Federal Government programs while reducing waste and inefficiency. It also calls for program evaluations.
Purpose and Use of the Information Collection
The purpose of the National Independent Study of the State Developmental Disabilities Programs (National Study) is to assess program effectiveness and achievements of the programs funded under the Developmental Disabilities Assistance and Bill of Rights Act of 2000 (DD Act), including collaborative efforts among the state developmental disabilities (DD) network programs and to develop valid and reliable measurement matrices for determining program effectiveness. A component of the study will be an examination of the ADD’s efficiency and effectiveness to support these DD Network programs. Research questions for the ADD Assessment are contained in Exhibit A-1. The web-based survey is one component of the ADD Assessment.
Exhibit A-1. ADD Assessment study questions
Structure-related questions:
Process-related questions:
Output-related questions:
Outcome-related questions:
|
The National Study is divided into two phases. The first phase carried out from October 2005 – September 2008 involved: (1) the development of data collection instruments and draft measurement matrices for determining program effectiveness and achievements, and (2) the implementation of a pilot study. The second phase includes two stages: (1) obtaining OMB approval for the evaluation tools (e.g., data collection instruments) developed during Phase I; and (2) full implementation of the evaluation using the data collection instruments developed during Phase I and finalization of measurement matrices that contain performance standards. Data collection for phase 2 was approved by OMB (OMB Control Number 0970-0372). ADD is now adding a component to assess the agency’s efficiency and effectiveness to support these DD Network programs.
It is not the purpose of the National Study to analyze ADD’s current measurement system that is used by grantees to report on their activities. The current measurement system requires a yearly report to ADD on the activities and compliance with the Act of each DD Network program in each state and territory. Instead, the purpose is to have an objective, outside contractor use a measurement system and related evaluation tools designed specifically to determine the effectiveness and achievements of the national DD Network programs.
The information collected through the National Study will be used to provide in-depth performance information to several stakeholders:
Members of Congress
OMB and the Administration
Grantees
Individuals with developmental disabilities
Family members
Advocates
Other federal agencies
The Administration on Developmental Disabilities will be able to use the information to make program improvements. It is anticipated that findings from this study will be a helpful tool for policymakers, demonstrating the excellent work done throughout the DD Network, as well as some of the challenges experienced by the grantees. In addition, the performance criteria that are being developed as part of this study will be used to describe the overall program effectiveness and achievements.
Use of Improved Information Technology and Burden Reduction
Two components of the proposed information collection will use technology to reduce burden. Executive Directors will be asked to complete a web-based survey electronically. In addition, panels will review draft performance criteria electronically to provide feedback. Thus, study participants will be able to provide their feedback in an efficient manner and will avoid the necessity of traveling to meet in person.
Efforts to Identify Duplication and Use of Similar Information
ADD has a number of mechanisms for monitoring DD Network programs and ensuring that they are complying with the Developmental Disabilities Assistance and Bill of Rights Act of 2000 (DD Act). Moreover, there is a number of strategic planning and accountability requirements in the DD Act itself that programs must comply with through reports to ADD. The result is that DD Network programs are already required to provide ADD with a considerable amount of data.
A pilot study was conducted that examined whether existing data can be used for the evaluation. All recent reports submitted to ADD by pilot study programs were reviewed to determine whether data from these reports would be able to answer the questions in the pilot study questionnaires. Through a crosswalk, the approximate location of data in several reports to ADD were located that might be able to answer the questions. Then, the existing data was incorporated from those reports into questionnaire binders. For each question for which existing data had been located, evaluators determined whether the existing data was able to answer the question.
It was found that the existing data was incomplete, out of date, not related to the dates of interest or not specifically related to the indicator and question. It was also noted that there was considerable inconsistency in definitions used by each program (e.g., in the NIRS data), and differences in formatting and contents of each report. Although much of the data was useful as background, none was able to specifically answer the questions in the questionnaires, which were based directly on the benchmarks and indicators that had been developed. Moreover, because of the inconsistency in definitions and data collection methodology, data would not be considered reliable enough to meet OMB requirements and it would not be possible to combine program data for roll-up to the national level.
There has never been an effort launched by ADD to examine efficiency and effectiveness to support the DD Network programs at the national level. Therefore, data does not exist for this purpose.
Small Entities
Some of the programs funded by ADD described under (1) are small in comparison to others. For example, under the DD Council program, some of the programs are considered ‘minimum allotment’. These Councils receive the smallest amount of funding. Because the funding formula for the DD Councils is based partly on population, typically, ‘minimum allotment’ DD Councils are those in States and territories with a small population.
The full-scale National Study, by necessity, will produce a certain amount of burden to all of programs included in the study sample. Programs will be asked to assemble individuals to be interviewed and to collect specific materials to send to the evaluator. The questionnaires were reduced considerably so interviews will be shorter and programs will be asked to collect fewer materials.
Consequences of Collecting the Information Less Frequently
This is a one time project.
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances.
Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
The current information collection clearance is to add an information collection tool to the National Study for the assessment of ADD. The 60 day Federal Register Notice was published in 75 FR 33309 on 06/11/2010. Most of the comments received from the First Federal Register notice addressed the National Study and related tools that is already underway. The tools for this study have been developed and underwent OMB clearance. Therefore, these comments were not relevant.
For this new data collection, one person commented on the additional information collection tool that is the subject of this package. The following comments were received:
It does not appear to address one of the key outcomes - leadership in federal disability community. There are no proposed evaluation questions that would determine if ADD is being successful with regard to this important outcome.
The proposed respondents on this survey are all internal to the ADD Network (Councils, P&As, UCEDDs). In order to address the outcomes stated above, it seems that it would be important to ask disability stakeholders outside of the Network as well as key administrators within ACF and policymakers in the legislature.
Finally, I have difficulty with the idea that the Commissioner of ADD will be a key informant, particularly to address historical evidence (proposed data collection is for 2005, 2007). The current Commissioner has not been in her position for enough time to have this level of knowledge.
In response to comment number 1, the tool has been revised to incorporate questions related to leadership in the federal disability community.
In response to comment number 2, the plan includes interviews with other stakeholders, however these did not require OMB clearance because of the number of people to be interviewed.
In response to comment number 3, it is not expected that the current Commissioner will address historical issues. The contractors will rely on federal staff for historical information.
ADD made significant efforts to consult outside the Agency for the full-scale national study. These efforts were described in the previous OMB supporting statement on this study and are summarized below.
Independent contractor: ADD solicited for an outside contractor, Westat, to conduct the National Study. Westat took three years to develop the information collection instruments for the National Study.
Advisory Panel: As part of it’s work, Westat established an Advisory Panel that included people with developmental disabilities, family members, other consumers, advocates, researchers, representatives from the DD network programs and policy specialists. It also established an advisory panel for developing the ADD assessment tool for Executive Directors.
Working groups: Westat organized and conducted P&A, DD Council, UCEDD and Collaboration Working Group meetings in person and by telephone and web cast throughout the spring, summer, and fall of 2006 to consult with experts in developing the information collection instruments.
Feedback from Programs: Westat provided ADD programs in each state with opportunities to provide feedback and comments on the evaluation and draft documents in 2007.
Pilot Study: Westat conducted a Pilot Study in 2008. The objectives of the pilot study were to: (1) inform the revision or elimination of some the benchmarks and indicators; (2) test qualitative data collection instrument guides for measuring the indicators; (3) inform the further development of performance standards; (4) determine the usefulness of existing data in reports to ADD; and (5) test the logistics for a full-scale National Study. Information was collected from less than 9 people at each site using interview guides and open-ended questions. Data collection instruments were guides for interviewers to ensure that all topic areas were covered. Interviews were different at all sites. The pilot study provided the contractor with insights into whether such a methodology was feasible for collecting the type of information that was required and could be analyzed.
Validation Panels: Westat convened Validation Panels in July 2008. Advisory Panel members made recommendations to Westat on the membership of Validation Panels within the following categories: was a person with a developmental disability or family member; was an advocate; and had a familiarity with research and policy. In addition, panel members needed to have an understanding of consumer needs; have an understanding of the purpose of the programs; have an appreciation for outcomes; be at least somewhat involved in the DD Network system; and have a proven track record of self-advocacy (e.g., DD Council members; self-advocates outside the programs). Westat also obtained a mix of urban and rural representation (with some thought to geographic representation) and a mix of senior and junior program staff. Each person reviewed the instruments and provided feedback.
For this data collection – administration of a web-based survey to Executive Directors of all DD Network programs in each state and territory – the contractors worked with one Executive Director from each type of program to develop the instrument.
Explanation of Any Payment or Gift to Respondents
In this additional data collection effort, respondents will not be compensated.
Assurance of Confidentiality Provided to Respondents
The proposed information collection instrument has received IRB approval.
In addition, participants in the evaluation are subject to the assurances and safeguards as provided by the Privacy Act of 1974 (5 USC 552a), which requires the safeguarding of individuals against invasion of privacy. The Privacy Act also provides for the confidential treatment of records maintained by a Federal agency according to either the individual’s name or some other identifier.
The contractor implementing this information collection activity, Westat, is firmly committed to the principle that the confidentiality of individual data obtained through Westat interviews must be protected. This principle holds whether or not any specific guarantee of confidentiality was given at time of interview (or self-response), or whether or not there are specific contractual obligations to the client. When guarantees have been given or contractual obligations regarding confidentiality have been entered into, they may impose additional requirements which are to be adhered to strictly.
Privacy and confidentiality will be protected throughout the data collection and processing operations and will not be shared with anyone outside this study, except as otherwise required by law. The contractor will separate names and other direct identifying information from electronic survey response data by storing them in separate files linked only by a nonmeaningful study ID. Files with identifying information, which will not be needed for analysis, will be stored in directories separate from the response data. All contractor staff, including all field staff, will be required to sign a confidentiality pledge.
Justification for Sensitive Questions
No questions of a sensitive nature will be asked.
Estimates of Annualized Burden Hours and Costs
Tables 1 and 2 below show the total annualized burden hours and costs estimated for this information collection activity.
Given the variability in the hourly rates for Executive Directors, an average hourly rate of $30 was used for the cost estimates.
Tables A-1 and A-2 contain the burden calculations for the full-scale study. The first row of Table 1 consists of all Executive Directors of all three programs. These are the participants of the additional data collection. The remaining rows cover the rest of the study. Table A-2 is a summary table of the full-scale study (including the additional web-based survey).
Table A-1: Estimate of Annualized Burden Hours and Costs for the Proposed Information Collection Instruments and the Full-scale Study
Instrument |
No. of Respon-dents
|
No. of Responses per Respondent
|
Average Burden Hours per Response |
Total Burden Hours
|
Total Cost |
ADD Assessment – Executive Director Interview – web-based survey* |
180 |
1 |
1 |
180 |
$5,400 |
DD Council: Executive Director Interview |
20 |
1 |
4 |
80 |
$2,400 |
DD Council: Interview with Council Chair/Council Members |
60 |
1 |
.75 |
45 |
$1,350 |
DD Council: Group Interview with Policymakers, Collaborators, and Grantees |
160 |
1 |
2 |
320 |
$9,600 |
DD Council: Group Interview with Recipients of Self-Advocacy and Leadership Education and Training |
100 |
1 |
.75 |
75 |
$2,250 |
DD Council: Group Interview with Recipients of Education and Training to Improve Community Capacity |
100 |
1 |
.75 |
75 |
$2,250 |
DD Council: Self-administered Form |
20 |
1 |
41.5 |
830 |
$24,900 |
*Addition to full-scale study
Table A-1: Estimate of Annualized Burden Hours and Costs for the Proposed Information Collection Instruments (continued)
Instrument |
No. of Respon-dents
|
No. of Responses per Respondent
|
Average Burden Hours per Response |
Total Burden Hours
|
Total Cost |
P&A: Executive Director Interview |
20 |
1 |
4 |
80 |
$2,400 |
P&A: Staff Interview |
60 |
1 |
.75 |
45 |
$1,350 |
P&A: Board of Directors (Commissioners)-Chair and Members |
60 |
1 |
.75 |
45 |
$1,350 |
P&A: Group Interview with Policymakers and Collaborators |
160 |
1 |
2 |
320 |
$9,600 |
P&A: Interview with Recipients of Community Education |
100 |
1 |
.75 |
75 |
$2,250 |
P&A: Interview with Clients |
100 |
1 |
.75 |
75 |
$2,250 |
P&A: Self-administered Form |
20 |
1 |
41.5 |
830 |
$24,900 |
UCEDD: Interview with Executive Director |
20 |
1 |
4 |
80 |
$2,400 |
UCEDD: Telephone Interview with Current and Graduated Students |
100 |
1 |
.75 |
75 |
$2,250 |
UCEDD: Interview with the Consumer Advisory Committee |
60 |
1 |
.75 |
45 |
$1,350 |
UCEDD: Interview with Peer Researchers and Colleagues |
100 |
1 |
.75 |
75 |
$2,250 |
UCEDD: Interview with Recipients of Community Services or Members Of Organizations/Agencies that are Trained to Provide Community Services |
100 |
1 |
.75 |
75 |
$2,250 |
UCEDD: Self-administered Form |
20 |
1 |
41.5 |
830 |
$24,900 |
TOTAL |
1,560 |
1 |
149.75 |
4,255 |
$127,650 |
Table A-2. Summary: Estimate of Total Burden Hours and Costs for Activities to Support Administration of Proposed Information Collection Instrument (Original Full-scale Study and ADD Assessment that includes the Web-Based Survey)
|
Total no. respondents |
No. responses per respondent |
Total average burden hours per response |
Total burden hours |
Annualized total burden hours |
Total cost |
Original full-scale study |
1,380 |
1 |
148.75 |
4,075 |
2,037.5 |
$61,125 |
Addition |
180 |
1 |
1 |
180 |
90.0 |
$2,700 |
|
|
|
|
|
|
|
Full study |
1,560 |
1 |
149.75 |
4,255 |
2,127.5 |
$ 63,825 |
*includes time to collect, organize, and submit advance materials and materials collected on site; identify key informants, obtain consent; prepare agenda; schedule interviews; make logistical arrangements; and participate in an exit interview
Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers
There will be no cost to respondents for this data collection. The cost will be incurred by the Federal government.
Annualized Cost to the Federal Government
The total annual cost to the Federal government was $750,000 ($1,500,000 over 2 years) (Table A-3). With the additional data collection for the ADD Assessment, the total annual cost is $1,540,883 or $770,442 over 2 years.
Table A-3. Costs to Federal government – Full-scale study and ADD Assessment
Federal cost |
Original Full-scale Study |
ADD Assessment |
Total |
Full cost |
$1,500,000 |
$40,883 |
$1,540,883 |
Per year |
$750,000 |
$20,442 |
$770,442 |
The additional cost for the ADD Assessment (including the web-based survey) to the Federal government is $40,883 or $13,628 per program. Over a 2-year time period, the cost is $20,442 per year and $6,814 per year per program.
Explanation for Program Changes or Adjustments
This is a new project.
Plans for Tabulation and Publication and Project Time Schedule
Except for this web-based survey, most of the data collection for the ADD assessment will be qualitative. The contractor will analyze data from each type of stakeholder separately. Because there are so few interviews (fewer than 10 per stakeholder group), recordings will not be transcribed. However, the contractor will use recordings as a backup to assist in summarizing interviews within a pre-defined outline consistent with the layout of the semi-structured interview guide. Analysis of this qualitative data will primarily be descriptive. Westat will also conduct a descriptive analysis of the data extracted from background materials. Some data will be transformed into tabular format (e.g., to demonstrate budget appropriations or cost allocations over time).
The contractor will use SAS to analyze data from the web-based questionnaire. Analysis will also be primarily descriptive, consisting of frequency distributions, cross-tabulations, and calculations of means, standard deviations, and medians for continuous data (as appropriate). Westat will analyze findings from each program separately. Westat will also conduct two separate analyses of the data – one for those 60 EDs who participated in the National Study, and another for all 180 EDs.
The ultimate goal of the ADD assessment analysis will be to determine the effectiveness of ADD processes in producing desirable outcomes, as well as the efficiency of ADD inputs in producing high quality and useful products. Thus, the analysis will seek relationships and patterns between inputs and outputs and processes and outcomes using an iterative approach to qualitative data analysis.
Contingent on OMB approval, we anticipate that the web-based survey will take 1 month to administer and 1 month to analyze. . Findings from the web-based survey (which is part of the ADD assessment) will be incorporated into the project’s final report.
Table A-4 provides a description of the deliverables for the full National Independent Study on Developmental Disabilities Programs
Table A-4. List of Deliverables
Item/Task |
Description |
No. of Copies |
Due Date after contract Award |
Task 2: Implement Independent Evaluation Study |
|||
Subtask 2.1 |
Develop a Plan for Implementing the Study |
1 electronic copy |
Within 2 months of start date |
Subtask 2.2 |
Identify Study Sample and Contact Participants |
1 electronic copy |
Within 3 months of start date |
Subtask 2.3 |
Use Evaluation Tools to Collect Data |
|
Within 6 months of start date |
Subtask 2.4 |
Train Research Staff on the Use of Evaluation Tools |
|
Within 4 months of start date |
Task 3: Finalize Performance Standards for the National DD Network Programs and Include in Final Package to ADD |
|||
Subtask 3.1 |
Further Develop Performance Standards by Building Upon Work Conducted in Phase I |
1 electronic copy |
24 months |
Subtask 3.2 |
Finalize the Performance Standards |
1 electronic copy |
24 months |
Task 4: Synthesize Findings and Develop Recommendations |
|||
Subtask 4.1 |
Synthesize Findings |
1 electronic copy |
24 months |
Subtask 4.2 |
Develop Recommendations |
1 electronic copy |
24 months |
Task 5: Progress Reports |
|||
Subtask 5.1 |
Periodic Progress Reports to ADD Grantees |
Electronic when requested |
As requested |
Subtask 5.2 |
Quarterly technical progress reports |
1 electronic copy |
Quarterly
|
Subtask 5.3 |
Develop and submit to PO final report with recommendations |
2 hard copies, 1 electronic copy |
24 months |
Task 6: Conduct an Assessment of ADD |
|||
Subtask 6.1
|
Develop and implement a methodology for conducting the ADD assessment. |
1 electronic copy |
8 months from date of modification |
Subtask 6.2
|
Develop Data Collection Tools |
1 electronic copy |
5 months from date of modification |
Subtask 6.3
|
OMB Clearance of Data Collection Tools |
Electronic when requested |
6 months from date of modification |
Reason(s) Display of OMB Expiration Date is Inappropriate
Not applicable
Exceptions to Certification for Paperwork Reduction Act Submissions
Not applicable
| File Type | application/msword |
| Author | ACF |
| Last Modified By | Department of Health and Human Services |
| File Modified | 2011-06-15 |
| File Created | 2011-06-15 |