Department of Health and Human Services
Administration for Children and Families,
Children’s Bureau
Data Collection Plan for a Follow-up Survey with
Child Welfare Information Gateway Customers
Office of Management and Budget
Clearance Package Supporting Statement
and Data Collection Instruments
October 2012
TABLE OF CONTENTS
Section Page
A. Justification 1
A1. Circumstances Making the Collection of Information Necessary 1
A2. Purpose and Use of the Information Collection 1
A3. Use of Improved Technology and Burden Reduction 2
A4. Efforts to Identify Duplication and Use of Similar Information 2
A5. Impact on Small Businesses and Other Entities 2
A6. Consequences of Less Frequent Data Collection 2
A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 3
A8. Comments in Response to the Federal Register Notice 3
A9. Explanation of any Payment or Gift to Respondents 3
A10. Assurance of Confidentiality 3
A11. Sensitive Questions 3
A12. Estimates of Annualized Burden Hours 4
A13. Annualized Cost for Respondents 4
A14. Annualized Cost to the Federal Government 5
A15. Explanation of Program Changes or Adjustments 6
A16. Plans for Tabulation and Publication and Project Time Schedule 6
A17. Waiver of Expiration Date Requirement 6
A18. 5 CFR 1320.9: Exceptions to Certification for Paperwork Reduction Act Submissions 6
B. Collection of Information Employing Statistical Methods 7
B1. Respondent Universe and Response Rates 7
B2. Procedures for the Collection of Information 11
B3. Maximizing Response Rates and the Issue of Non-response 13
B4. Pretesting 13
B5. Contact Information 13
LIST OF TABLES
Number Page
A-1, Annual Burden Estimates 4
A-2, Annualized Cost to the Federal Government……………………………………………….. 5
A-3, Project Schedule 6
B-1, Expected Response Rates by Strata 8
LIST OF APPENDICES
Supporting Statement for the
Paperwork Reduction Act of Submission
Child Welfare Information Gateway (CWIG) is a service of the Children’s Bureau, a component within the Administration for Children and Families, and is dedicated to the mission of connecting professionals and concerned citizens to information on programs, research, legislation, and statistics regarding the safety, permanency, and well-being of children and families. CWIG seeks to continuously identify the needs in the child welfare field, determine the extent to which those needs are being met, and assess the perceived impact of information products and services. The proposed information collection activity includes conducting a follow-up survey with a sample of professional CWIG users. The follow-up survey will gather data about how professionals use Child Welfare Information Gateway’s information services in their work and will assess the perceived impact of CWIG products and services. Follow-up survey findings will be applied to make continuous improvements to Child Welfare Information Gateway’s website and other information services.
The follow-up survey is designed to assess the perceived impact of CWIG on professional users by measuring short-term outcomes and intermediate outcomes that result from the information services provided by CWIG. Short-term outcomes include in changes in users’ knowledge, awareness, attitudes, and skills while intermediate outcomes include behavioral changes and the adoption of new policies and practices. The follow-up survey will be administered to a sample of professional users who complete the customer satisfaction survey (approved under OMB #: 0970-0401). The last question on the customer satisfaction survey will ask if the respondent is willing to participate in a follow-up survey. If so, the participant will provide their name as well as their email address and/or phone number, for contact purposes only, in order to receive and complete the follow-up survey by phone or online. An automated system will email the follow-up survey to everyone who provides an email address on a biweekly basis. In the event that customers who are emailed the follow-up survey do not respond within two weeks, a second attempt will be made to conduct the follow-up survey over the telephone.
The primary means of gathering information is through electronic format including e-mailed surveys. The evaluation team also imports online survey data from a password protected server into SPSS, a statistical software package used for storing, managing, and analyzing survey data. Per guidance outlined in 5 CFR 1320.8, the focused use of electronic data collection methods is intended to reduce the burden on respondents.
The follow-up survey is the only assessment of Child Welfare Information Gateway’s perceived impact; therefore, there is no similar information already available.
We do not plan to survey small businesses.
A6. Consequences of Collecting the Information Less Frequently
The data collection plan utilizes the minimum amount of data collection necessary to obtain valid, reliable, and useful information that can help inform Child Welfare Information Gateway products and services. Reducing data collection further would jeopardize the quality and integrity of the results including the generalizability to all Information Gateway professional customers.
Responding to this data collection effort is completely voluntary. Individuals recruited for this effort will be given the opportunity not to respond at all, and to respond whenever it is convenient for them during the data collection period. All customers have the option of refusing a survey request without penalty or to request the survey be offered at a more convenient time for them.
As required by CFR 1320.8, a Federal Register notice appeared on July 23, 2012 (Volume 77, Document Number: 2012-17812) Page 43091-43092. The first notice received one public comment, “america [sic] does not need this information. This is wasted taxpayer dollars. Let the profiessionals [sic] pay their own way. 1 out of 2 american [sic] taxpayers are living in poverty and this is for the benefit of those high flying high paid "professionals". Shut down this project. Cut the budget for this project to zero. This comment is for the pubilc [sic] record.”
The follow-up survey was also piloted with eight child welfare professionals to help ensure that the survey was an appropriate length.
Not applicable, the data collection plan does not call for payment to respondents for participation.
Contact information (including customer’s name, email, and phone number) collected from a customer satisfaction survey (approved under OMB #: 0970-0401) will only be used for the purpose of conducting the follow-up survey and will be stored according to the Federal guidelines set forth by the National Institute of Standards and Technology (NIST)1. We will inform respondents of the voluntary nature of their participation with the follow-up survey and survey responses will be kept private to the extent permitted by law.
Sensitive issues will not be raised at any time during the conduct of the survey or focus groups.
Estimates of response burden are outlined in the following table.
Table A-1: Annual Burden Estimates
Instrument |
Affected Public |
Number of Respondents |
Participation Time |
Burden |
Follow-up Survey |
Private Sector |
140 |
0.17 |
17 |
State, Local, or Tribal Governments |
60 |
0.17 |
17 |
|
TOTAL Estimated Annual Burden Hours |
34 hours |
|||
There will be no cost to survey respondents or focus group participants.
We estimate the annualized costs to the Federal government to be $32,640. This cost estimate is based on the work plan under the cooperative agreement for this effort. As outlined below, the estimated annual Federal costs associated with this effort include costs to manage and implement the data collection, analysis, and reporting activities outlined in this application.
Table A-2: Annualized Cost to the Federal Government
|
Total Project Cost |
# of Years |
Annualized Cost |
Gateway Staff Hours – (design, development, testing, phone surveys, management, data analysis, reporting) |
$97,200 |
3 |
$32,400 |
Indirect Costs – (Web costs, maintenance, conference calls) |
$720 |
3 |
$240 |
Totals |
$97,920 |
3 |
$32,640 |
This is a new data collection effort.
Table A-3 presents the data collection and reporting schedule:
Submit OMB revision package: July 2012
Ongoing data collection : January 1, 2013 – September 29, 2016
Reporting: Quarterly and annual reporting to inform Child Welfare Information Gateway’s products and services.
Analysis of data will primarily include basic descriptive statistics such as frequencies, means, and percentages. Additional analyses may include ANOVAs, t-tests, and chi-squares if appropriate.
This section does not apply. We are not seeking approval to not display the
expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are requested.
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
This data collection effort is designed to reach professional customers who previously used Child Welfare Information Gateway services to learn about how they are applying the information and/or resources they received. Data will be collected by delivering follow-up surveys to customers who had previously received Information Gateway services approximately four weeks earlier. A convenience sampling method was used to determine the estimates outlined in Table B-1.
Table B-1: Expected Response Rates by Strata |
|||||
Instrument |
Response Offers |
Expected Response Rate |
Expected Number of Responses by Strata |
Total Responses |
|
Private Sector |
State / Local / Tribal Governments |
||||
Follow-up Survey |
400 |
50% |
140 |
60 |
200 |
Total |
140 |
60 |
200 |
||
Our response rate calculations were based upon a response rate obtained for a similar data collection effort conducted by ICF International in 2011. Expected response rates were calculated based on a follow-up survey conducted by the National Training and Technical Assistance Center (NTTAC) for the Office of Juvenile Justice and Delinquency Prevention (OJJDP). We anticipate that approximately 400 professionals will agree to be contacted for a follow-up survey by submitting their contact information with a customer satisfaction survey. Of these customers, we expect that approximately 50% will actually complete the survey either via email or phone, resulting in a total of 200 survey responses.
Based on customer survey data obtained during FY13 Quarter 1, we expect that 70 percent of follow-up survey responses will be from the private sector and 30 percent of follow-up survey responses will be from state/local/tribal governments. A power analysis using the Gpower computer program (Faul & Erdfelder, 1998) indicated that if using a t test between means with alpha at .05, a total sample of 200 respondents (140 in Private Sector and 60 in State/Local/Tribal Governments) has 93% chance to detect medium effects (d=.5), and 77% chance to detect small effects (d=.3).
The follow-up survey is designed to assess the perceived impact of CWIG on professional users by measuring short-term outcomes and intermediate outcomes that result from the information services provided by CWIG. Short-term outcomes include in changes in users’ knowledge, awareness, attitudes, and skills while intermediate outcomes include behavioral changes and the adoption of new policies and practices. The follow-up survey will be administered to a sample of professional users who complete the customer satisfaction survey (approved under OMB #: 0970-0401). The last question on the customer satisfaction survey will ask if the respondent is willing to participate in a follow-up survey. If so, the participant will provide their name as well as their email address and/or phone number, for contact purposes only, in order to receive and complete the follow-up survey by phone or online. Third-party software, SurveyGizmo, will be used to send an email with the follow-up survey to everyone who provides an email address on a biweekly basis. In the event that customers who are emailed the follow-up survey do not respond within two weeks, a second attempt will be made to conduct the follow-up survey over the telephone.
B3. Maximizing Response Rates and the Issue of Non-Response
Child Welfare Information Gateway has taken steps to maximize response rates. First, we have ensured that the follow-up survey is concise and clear. In addition, all customers who consent to take the follow-up survey have already demonstrated a willingness to complete a satisfaction survey and may be more willing to agree to participate with the follow-up survey. Finally, two attempts will be made to contact customers via email and/or phone.
B4. Pretesting of Procedures or Methods
All instruments for this data collection effort were tested for usability by Child Welfare Information Gateway staff and by eight professionals working in the child welfare field who agreed to pilot test the survey.
B5. Contact Information
Should you have any questions about the contents of this OMB submission package, please contact one of the following individuals:
La Chundra Lindsey, Children’s Bureau, 1250 Maryland Avenue, SW, Room 8148, Washington, DC 20024, telephone: 202.205.8252, email: Lachundra.Lindsey@acf.hhs.gov
Helena Wallin-Miller, ICF International, 9300 Lee Highway Fairfax, VA 22031, telephone: 910.255.6124, email: hwallin-miller@icfi.com
Christine Leicht, ICF International, 9300 Lee Highway Fairfax, VA 22031, telephone: 703.934.3000, email: cleicht@icfi.com
1 A Privacy Impact Assessment was performed to ensure that the collection of Personally Identifiable Information on the customer satisfaction survey meets appropriate data security standards.
| File Type | application/msword |
| File Title | Supporting Statement for the |
| Author | Shamegan |
| Last Modified By | 15058 |
| File Modified | 2013-02-04 |
| File Created | 2013-01-28 |