Pilot Test Data Collection (OMB 0970-0355)
Supporting Statement Part A
Measurement Development: Quality Relationship of Family and Family Services Staff in Head Start/Early Head Start
The Office of Planning, Research and Evaluation (OPRE) and the Office of Head Start (OHS) of the Administration for Children and Families (ACF), within the U.S. Department of Health and Human Services (HHS), are proposing a data collection activity as part of the development of measures to assess relationships between families and family services staff in Head Start and Early Head Start programs. This data collection activity is part of the continuing Family and Provider/Teacher Relationship Quality (FPTRQ) project to develop measures of the quality of family and provider/teacher relationships that will be (1) applicable across multiple types of early care and education settings and diverse program structures, including Head Start/Early Head Start; (2) sensitive across cultures associated with racial, ethnic, and socioeconomic characteristics; and (3) reliable in both English and Spanish. As a step in developing these measures (see Table 1), OPRE and OHS request permission to conduct a pilot test of two measures developed for Head Start/Early Head Start family services staff and the families they serve. The pilot test data will be used exclusively to examine the psychometric properties of these measures in order to ensure that the measures are reliable and that items are working well in Head Start/Early Head Start programs.
Table 1. Previous OMB Packages for the FPTRQ Project
OMB Number |
Clearance Date |
Description |
0970-0356 |
April 2012 |
Focus groups with parents and providers/teachers were conducted to assess the extent to which the FPTRQ conceptual model matched the perceptions and experiences of target populations and to help guide item development. |
0970-0355 |
November 2013 |
Cognitive interviews were conducted to gather targeted information from family services staff and families specifically about the family services staff-family relationship. |
0970-0420 |
January 2013
|
A pilot and a field test were conducted using the parent, provider/teacher, and director measures to collect data for statistical analyses to examine the distribution of items, reliabilities of constructs and subscales, and to conduct psychometric analyses to finalize the measures. |
The information collected in this effort will not be the primary subject of any published ACF reports; however, information may be made public through methodological appendices or footnotes, reports on instrument development, instrument user guides, descriptions of respondent behavior, and other publications describing findings of methodological interest. When necessary, results will be labeled as exploratory in nature. Methodological results of this pre-testing research may be prepared for presentation at professional meetings or publication in professional journals.
A growing literature on early care and education indicates that the family and provider/teacher relationship is an important domain in early care and education settings. Specifically, research has highlighted the value of the interactive role that families and programs play in fostering positive developmental outcomes of children in these settings (Dunst, 2002; Johnson, 2000; Mendez, 2010). The benefits of early care and education are especially important for children living in poverty who encounter disproportionate risks to meeting academic and social demands needed to excel in school (Bornstein & Bradley, 2003). The nation’s largest and most comprehensive response to the needs of preschool children is the Head Start/Early Head Start program. Head Start/Early Head Start is a social welfare initiative to give young children growing up in poverty the skills and experiences necessary for subsequent school success (Improving Head Start for School Readiness Act of 2007). Head Start/Early Head Start offers young children an enriched preschool experience and offers families a range of health and social service interventions, including family case management. Indeed, Head Start/Early Head Start employs a two-generation approach, focusing on the welfare of both children and their parents, in order to achieve positive outcomes for the family overall (Henrich and Gadaire, 2008). An important component of two-generation approaches is case managers, called family services staff, who work directly with families (Duch, 2005). Family services staff also serve as liaisons between the Head Start/Early Head Start program and its families and are critical to the achievement of Head Start/Early Head Start’s performance standards. It is often the collaboration between the parent, teacher, and family services staff member that creates an optimal supportive network both for the child at school and the parenting at home (Webster-Stratton, Reid, & Hammond, 2001). For this reason, understanding the family services staff-family relationship is imperative for meeting the goals of Head Start/Early Head Start, and no measure of this relationship currently exists. Greater focus should be brought to the family services staff-family relationship if the quality of the family-school connection is to be fully understood.
While there are a number of federal surveys that collect data on the early care and educational experiences of families and children, such as the National Survey of Early Care and Education and the National Household Education Survey, none include measures that tap into multiple dimensions of family and family services staff relationship quality in general, are applicable across diverse populations in particular, and focus on the role of Head Start/Early Head Start family services staff. The FPTRQ project has developed measures to address these gaps. The new FPTRQ family services staff measures will be tools that policymakers, researchers, and Head Start/Early Head Start programs can use to gather valid and reliable information about the quality of family and family services staff relationships. As part of this project, two family services staff surveys were developed to assess the quality of the family services staff and family relationship. One survey will be completed by family services staff, and the other will be completed by parents regarding their relationship with the family services staff member who serves their family.
The proposed data collection activity is part of the fifth step in the process of developing the FPTRQ family services staff measures.
In the first step, we began with an extensive review of the literature and extant survey measures and developed a conceptual model of family and provider/teacher relationships to guide our work.
In the second step, focus groups (conducted under OMB Formative Generic Clearance 0970-0356) with parents and providers were used to assess the extent to which our conceptual model matched the perceptions and experiences of our target populations and to help guide item development. The focus groups found that, for the most part, the FPTRQ conceptual model and definitions of the elements within the model accurately reflected provider and parent perceptions of strong family-provider/teacher relationships. In particular, both parents and providers/teachers spontaneously agreed with the elements within the attitudes, knowledge, and practices constructs in the conceptualized model and generally agreed with the environment construct after they were prompted to provide their opinions. In sum, focus groups findings confirmed the constructs in the conceptual model and helped to streamline the construct definitions.
In the third step, we conducted an extensive review of existing items, honed our definitions (with the help of the focus group findings), and revised and developed new items when necessary to create new measures of family-provider relationships.
In the fourth step, we tested how well the measures we developed work using cognitive interviews (conducted under OMB Pretesting Data Clearance 0970-0355) to gather targeted information from family services staff and families specifically about the family services staff-family relationship.
In the fifth step and where we now seek OMB generic clearance approval, we will conduct a pilot test of the family services staff and family services staff parent measures. This step in the project will result in sound and reliable measures that will tap into multiple domains of family services staff and family relationships and can be used with Head Start/Early Head Start family services staff and parents.
The purpose of the pilot test data collection is to examine distribution of item responses across response categories and internal reliability of the subscales in the measures. The information obtained through this data collection will be used exclusively to evaluate the psychometric properties of the FPTRQ constructs and subscales and to identify problematic items that should be dropped from the measures. The FPTRQ measures for family services staff and parents about family services staff (English and Spanish versions) are presented in Appendices A, B, C, and D. In addition, telephone interviewer scripts for recruiting program directors and family services staff are presented in Appendices H and I.
The family services staff measure is for Head Start/Early Head Start family services staff, referred to as family service workers in the measure. It asks respondents questions about how they work with all parents of children in Head Start/Early Head Start programs, such as how often they have offered parents ideas or suggestions about parenting and how often they have helped families find services they need. The family services staff measure includes 112 items (including 17 demographic items) and takes about 15 minutes to complete on average. (See Appendices A and C.)
The family services staff parent measure is for parents to complete about the Head Start/Early Head Start family services staff member (referred to as family service worker in the measure) who serves their family. It asks respondents questions about how they work with their family service worker, such as how often their family service worker remembers personal details about their family and how comfortable they feel sharing information about certain topics with their family service worker. The family services staff parent measure includes 75 items (including 9 demographic items) and takes about 10 minutes to complete on average. (See Appendices B and D.)
Scripts for recruiting Head Start/Early Head Start program directors and family services staff will be used when calling them for participation in the pilot test. Using the scripts, our telephone interviewers will present a brief description of the pilot test, ask directors’ permission to contact family services staff from their programs, and collect contact information from directors and family services staff. It will take about 10 minutes to complete the recruitment process. (See Appendices H and I.)
Whenever possible, advanced technology will be used to collect and process data to reduce respondent burden and make data processing and reporting more timely and efficient. In the pilot test, self-administered questionnaires will be mailed to respondents to collect information. The questionnaires will take approximately 10-15 minutes to complete and have a simple, check-box design. Self-administered questionnaires allow respondents to complete them at times that are convenient to them.
Every effort has been made to determine whether similar measurement tools exist by searching various databases (e.g., national and scholarly), reviewing existing early care and education quality measures, and consulting with experts in the field. Our review of extant literature uncovered family and provider/teacher relationship measures; however, none measured multiple domains of family and provider/teacher relationships, assessed the family services staff-family relationship, or were appropriate for program evaluation. We have also consulted with experts in the early care and education field and they concur that the field lacks appropriate and psychometrically sound measures that assess the quality of family services staff and family relationships.
We will only collect data from family services staff from Head Start/Early Head Start programs and from families served by Head Start/Early Head Start. Although these family services staff and families are not considered small businesses/entities, we will attempt to reduce the impact on them by allowing them to complete the measures on days, at times (e.g., evenings and weekends), and in locations that are convenient to them.
This request is for clearance for the pilot test of the family services staff measures. No additional data collection activities are planned after this pilot test.
There are no special circumstances requiring deviation from these guidelines. As such, this request fully complies with regulation 5 CFR 1320.5.
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the generic clearance for information collection. This notice was published on June 10, 2011, Volume 76, page 34078, and provided a sixty-day period for public comment. The second notice was published on August 29, 2011, Volume 76, page 53682, and provided a thirty-day period for public comment. ACF did not receive any comments.
The FPTRQ project has benefited from focus groups and consultation with OHS, as well as from consultation with many outside experts, including attendees of the “Family-Sensitive Caregiving and Family Engagement Working Meeting: Identifying and Measuring Common Concepts,” a meeting that was sponsored by OPRE in June 2010, and the FPTRQ Technical Work Group.
Non-federal attendees of the Family-Sensitive Caregiving and Family Engagement Working Meeting were:
Gina Adams, Urban Institute
Don Bailey, RTI International
Juliet Bromer, Erikson Institute
Concha Delgado-Gaitan, Consultant
Carl Dunst, Smoky Mountain Research Institute
Jay Fagan, Temple University
Nikki Forry, Child Trends
Anne Henderson, Consultant, Annenberg Institute for School Reform
Lee Kreader, National Center for Children in Poverty
Michel Lahti, University of Southern Maine
Laurie Linscott, Michigan State University
Tammy Mann, United Negro College Fund
Lisa McCabe, Cornell University
Christy McWayne, Tufts University
Diane Paulsell, Mathematica Policy Research
Toni Porter, Bank Street College of Education
Eva Marie Shivers, Indigo Cultural Center
Amy Susman-Stillman, University of Minnesota
Bobbie Weber, Oregon State University
The FPTRQ Technical Work Group is comprised of the following experts in the fields of measurement development, family and provider/teacher relationships, and early care and education:
Catherine Ayoub, Harvard University
Carl Dunst, Smoky Mountain Research Institute
Julia Henly, University of Chicago
Judith Jerald, Save the Children
Elena Lopez, Harvard University
Doug Powell, Purdue University
Lori Roggman, Utah State University
Julia Mendez, University of North Carolina at Greensboro
Suzanne Randolph, University of Maryland
Participants will be informed of the voluntary nature of the participation in the study and of the privacy provision in the initial cover letter and on the surveys, stating that their responses will not be disclosed or used in identifiable form for any other purposes.
In the cover letter that will be given to participants along with a hard copy of the FPTRQ family services staff measures, participants will be made aware of the extent to which their privacy will be protected as part of the pilot test. Specifically, participants will be assured that their names will not be documented on any reports, that their responses will not be shared with others outside of the study team, and that their personally identifiable information will not be linked to their responses or analyses. In order to protect respondents’ privacy, a study-specific identification number will be assigned to each respondent and will be used for all study materials.
All information collected will be kept private to the fullest extent required by law. Westat assumes responsibility for the security of data it collects during the pilot test. Efforts are directed primarily at preventing any form of data security violations, whether they result from malfunction of the computer system, environmental hazards to the facility, or accidental or intentional misuse or misappropriation of data or systems. Monitoring of these security efforts is achieved through carefully planned management practices, control procedures, and facility and equipment standards.
Westat staff are trained in the importance of protecting data privacy, and all staff are required to read and sign Westat’s "Employee or Contractor’s Assurance of Confidentiality of Survey Data." Data collected in hard-copy form are kept in locked cabinets. Likewise, system-generated output containing private data is stored in locked areas until no longer needed, and names linking to data are destroyed in accordance with project requirements.
For hard copy survey materials, we have a survey receipt control system that is designed to track the location of paper documents and, thus, detect any missing materials. When the materials are no longer needed, they will be securely shredded.
No sensitive questions will be asked as part of this data collection.
The total annualized hours for this data collection activity is estimated to be 65 hours.
Table 2. ESTIMATED ANNUAL RESPONSE BURDEN AND ANNUAL COST
Respondent |
Respondent N |
Annual Number of respondents |
Number of responses per respondent |
Average burden hours per response |
Total burden hours |
Average hourly rate |
Total annual cost |
Recruitment script for directors |
1201 |
120 |
1 |
0.17 |
20 |
$21.13 |
$422.60 |
Recruitment script for family services staff |
60 |
60 |
1 |
0.17 |
10 |
$14.49 |
$144.90 |
Family services staff measure |
60 |
60 |
1 |
0.25 |
15 |
$14.49 |
$217.35 |
Family services staff parent measure |
120 |
120 |
1 |
0.17 |
20 |
$15.55 |
$311.00 |
TOTAL |
300 |
|
1 |
|
65 |
|
$1,095.85 |
Note: 1We anticipate a 50% cooperation rate from directors when contacting them for participation in the pilot test by phone.
Estimates of Annualized Costs. There is an estimated annualized burden over the 1-year data collection period to respondents of $1,095.85.
For parent respondents, an average hourly salary of approximately $15.55 is assumed based on the Bureau of Labor Statistics (BLS) estimates for median hourly wages for high school graduates. For FSS respondents, an average hourly salary of approximately $14.49 is assumed based on BLS estimates for median hourly wages for child care workers.
There will be no direct cost to the respondents other than their time to participate in the pilot test.
There will be no capital, operating, or maintenance costs to the respondents.
The analysis of the family services staff measures will be conducted using descriptive statistics, factor analysis, and reliability testing using Cronbach’s alphas. The information collected in this effort will not be the primary subject of any published ACF reports; however, information may be made public through methodological appendices or footnotes, reports on instrument development, instrument user guides, descriptions of respondent behavior, and other publications describing findings of methodological interest. When necessary, results will be labeled as exploratory in nature. The results of this pre-testing research may be prepared for presentation at professional meetings or publication in professional journals.
The plan is to conduct the pilot test between December 2014 and March 2015. Upon the OMB approval, the recruitment of participants will start and data collection will begin with a two month field period. Westat will recruit participants from different Head Start/Early Head Start programs in the Washington, DC metropolitan area and across other large metropolitan and rural areas in the United States, such as Chicago, IL, Detroit, MI, Seattle, WA, and Los Angeles, CA. Approximately three months will be allotted for both recruitment and data collection.
Table 3. Tentative Pilot Test Time Schedule
Activity |
Tentative Timeline |
|
PILOT TEST |
Commencement |
Duration |
Recruitment and Data Collection after OMB clearance received |
One week after OMB approval |
10 weeks (including follow-up) |
Data Analysis |
8 weeks after data collection starts |
3 weeks (including preliminary and final analyses) |
Report |
Upon completion of data analysis |
2 weeks |
The OMB number and expiration date will be displayed at the top of the first page of the consent form that will be given to each participant in the pilot test, as well as on the front of each of the measures. When recruiting Head Start/Early Head Start directors and family services staff by telephone, we will read the OMB number and expiration date at the start of any conversation.
No exceptions are necessary for this data collection.
References
Bornstein, M.H., & Bradley, R.H. (Eds.) (2003). Socioeconomic status, parenting, and child development. Mahwah, NJ: Erlbaum.
Duch, H. (2005). Redefining parent involvement in Head Start: a two-generation approach. Early Child Development and Care, 175(1), 23-35.
Dunst, C. J. (2002). Family-centered practices: Birth through high school. Journal of Special Education, 36, 139-147.
H.R. 1429--110th Congress: Improving Head Start for School Readiness Act of 2007. (2007). In www.GovTrack.us. Retrieved September 4, 2013, from http://www.govtrack.us/congress/bills/110/hr14
Henrich, C. C., & Gadaire, D. M. (2008). Head Start and parent involvement. Infants & Young Children, 21(1), 56-69.
Johnson, B. H. (2000). Family-centered care: Four decades of progress. Families, Systems, & Health, 18(2), 137-156.
Mendez, J. L. (2010). How can parents get involved in preschool? Barriers and engagement in education by ethnic minority parents of children attending Head Start. Cultural Diversity and Ethnic Minority Psychology, 16(1), 26-36.
Webster-Stratton, C., Reid, M. J., & Hammond, M. (2001). Preventing conduct problems, promoting social competence: A parent and teacher training partnership in Head Start. Journal of clinical child psychology, 30(3), 283-302.
Willis, G. B. (2005). Cognitive interviewing: a tool for improving questionnaire design. Thousand Oaks, California: Sage Publications, Inc.
Appendices
Appendix A: FPTRQ Family Services Staff Measure (English version)
Appendix B: FPTRQ Family Services Staff Parent Measure (English version)
Appendix C: FPTRQ Family Services Staff Measure (Spanish version)
Appendix D: FPTRQ Family Services Staff Parent Measure (Spanish version)
Appendix E: Prenotice letter to Head Start directors
Appendix F: Cover letter for Family Services Staff
Appendix G: Cover letter for Head Start parents
Appendix H: Recruitment scripts for Head Start directors
Appendix I: Recruitment scripts for Family Services Staff
Appendix J: Mailing address form for token of appreciation
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | jwest |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |