Revised May 11, 2007
.
SUPPORTING STATEMENT
FOR PAPERWORK REDUCTION ACT SUBMISSION
A. Justification
1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.
REL Midwest at Learning Point Associates, part of a federally funded network of 10 regional educational laboratories, exists to bring the latest and best research and proven practices into school improvement efforts. Serving the seven states of Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, and Wisconsin, REL Midwest provides policymakers and practitioners with resources based on the highest quality evidence as defined by scientifically valid research principles. You can find out more at www.learningpt.org
The current authorization for the Regional Educational Laboratories program is under the Education Sciences Reform Act of 2002, Part D, Section 174, (20 U.S.C. 9564), administered by the Institute of Education Sciences’ National Center for Education Evaluation and Regional Assistance. The legislation states that one of four primary functions of the Regional Educational Laboratories program is to develop “a plan for identifying and serving the needs of the region by conducting a continuing survey of the educational needs, strengths, and weaknesses within the region, including a process of open hearings to solicit the views of schools, teachers, administrators, parents, local educationalH. R. 3801—29agencies, librarians, and State educational agencies within the Region.”
As defined above, one major activity required of each Regional Educational Laboratory (REL) includes a rigorous assessment of regional education needs to be conducted through continuing surveys of the education needs, strengths, and weaknesses within the region. The findings are to be compiled into the regional education needs assessment analysis conducted under Subtask 1.1 (Regional Education Needs Analysis and Training and Technical Assistance Response Unit) in the REL contract. These findings will be used to accomplish three major objectives, including (1) identifying training and technical assistance priorities and needs in the region, (2) monitoring such needs and training and technical assistance activities, and (3) ensuring that the activities respond to the region’s needs.
The REL Midwest proposes to assess the needs of its region in collaboration with Public Agenda, one of the well-respected survey, research, and communications firms in the country. Continuous needs sensing and reporting will ensure that the applied research and technical assistance activities conducted are indeed on target and meet the specific needs of the Midwest region.
We propose a carefully integrated program of quantitative surveys and qualitative research that can powerfully inform REL Midwest’s regional work.
The quantitative and qualitative research with local and state educational leaders will take place in 2007, 2008 and 2009. 2008 and 2009 will repeat many questions developed for the basic instrument in 2007 in order to track responses over time. (REL Midwest and Public Agenda expect some questions to be adjusted over time due to changing policies or emerging trends and hypotheses but plan to hold the core of the quantitative work steady in order to effectively track changes in opinion.) Additionally, a separate sample of teachers in Year 2009 will provide opportunities to enrich the data sets with fresh perspectives.
The quantitative questionnaire/survey will be administered to a representative sample of school and district educators and leaders across the region. The survey will only be administered to those who have current experience in K-12 education, and we will conduct focus groups with this population as well. REL Midwest will assess the needs and priorities of state-level policymakers and practitioners in the qualitative component of our data collection, as well as a qualitative component with parents.
Quantitative Survey. The survey will be administered in fall 2007, spring 2008, and spring 2009. During the first and third year of data collection, REL Midwest will survey 700 superintendents and 700 school board members, with survey administration divided evenly across the seven states. The second year of data collection will consist of 1,400 teachers and 1,400 principals divided evenly across the seven states.
Surveying district level stakeholders in fall 2007 and spring 2009 allows REL Midwest to track potential changes in the needs and priorities of stakeholder groups before and after critical milestones, such as the reauthorization of NCLB and the presidential election. School board presidents and district administrators directly serve both policymakers at the state level, and also work with teachers and students at the school level. Thus, this district group should be knowledgeable about and concerned with the potential changes in state and federal policy, and how these policies ultimately affect teacher practice and student learning. Because they sit at the crossroads between state policy and local practice in this way, REL Midwest believes that resulting changes in needs and priorities will be most easily detected at this level.
During the second year, REL Midwest will survey the opinions of principals and teachers, as they are on the front lines and are acutely aware of the obstacles affecting teacher and student performance. While the needs and priorities of local school-level practitioners is critical to examine, REL Midwest believes that school practitioners’ needs are affected less directly, and more slowly, by federal and state policy changes. Thus, surveying this group once during the three year data collection period is adequate for understanding the broader issues that affect teachers over the life of the REL contract. A description of the school (principals and teachers) and district (superintendent and school board presidents) level participants, as well as the methods to be used in randomly sampling school and district participants are described below.
School-level sample. REL Midwest will collect 200 surveys per state from both K-12 principals and K-12 teachers (total surveys= 2,800) during the second year of data collection.
The principal sample will be purchased through MDR, a division of Dunn and Bradstreet. MDR conducts a census of educational institutions nationwide twice a year and updates their principal directories to include names, position titles, geographic data, physical work address, e-mail address, and phone numbers. The MDR data base covers every district and school in the country.
To ensure regional representation, the school-level survey sampling will begin with stratifying each state in the region based on type of locale, percentage of students who receive free or reduced-price lunch, and student minority status. Schools will be randomly and proportionally selected from these strata. Stratifying the sample in this way will ensure that we will be able to reliably analyze and report on opinions of defined positions both within a state and across the Midwest region.
Teacher sample
The teacher sample will be generated by contacting principals by e-mail in the beginning of year 2 and asking them to randomly select teachers from their schools. A random number (n) will be generated prior to contacting the principals and the principals wilI be asked to provide contact information for the n and 2n teacher from their roster to be used. If necessary to increase response, principals will be contacted by telephone as well. By getting the teachers from the principals themselves, we will have a more accurate representation than if we were to rely on purchased sample, as such teacher sample is often derived from purchased lists and other misrepresentative sources.
Principals contacted will not be the same principals contacted to complete the survey so that the survey questions will not bias their participation levels in furnishing teacher information. The principal information will also be purchased from MDR and will be stratified in the same manner as the principal sample used in our survey.
District-level sample. REL Midwest will collect 100 interviews per state from both superintendents and school board presidents in the first and third year of data collection. This sample will also be purchased from MDR and will be stratified the same way to reliably analyze and report on opinions of defined positions both within a state and across the Midwest region.
Weighting the sample
In following the sample selection procedures described above for both the school and district level sample, REL Midwest will select a sample that is an approximate microcosm of the school and district samples as defined above. When we report on the region as a whole, results will be weighted on the population figure rather than sample size. Weight variables will be constructed based upon statewide population estimates for each sample group—teachers, principals, superintendents and school board presidents.
Among district-level survey respondents, an independent random sample will be generated for each data collection period (2007, 2008, 2009).
REL Midwest will work with Princeton Data Source to administer the CATI and online surveys. Princeton Data Source (PDS) is an affiliate of Princeton Survey Research Associates International, located in Princeton, N.J. and Washington D.C. PDS specializes in providing high-quality research by telephone, mail and online from a state-of-the-art facility in Fredericksburg, Virginia. Founded in 1999, PDS provides data for clients who need independent, reliable information about the attitudes, knowledge and behavior of the general public, as well as of defined subgroups.
Response rate and call design
We believe that we can attain an 80% response rate with the sample as noted in the revised table 4 below and the indicated number of interviews. Our mixed mode design of phone and online interviews, coupled with our prenotification letters and letters of encouragement as well as successive reminder phone calls (up to 15) will increase our response rates to the proper level.
Ongoing regional needs assessment is required as part of the U.S. Department of Education contract with REL Midwest. All of our research and technical assistance must be driven by regional needs. The priority that the population assigns to their needs may indeed change each year based on a variety of factors stemming from local changes (new hiring and training guidelines for teachers in a particular district, e.g.), state changes (new funding mandates, e.g.) and on the national level (e.g., Presidential election, NCLB reauthorization, etc.). Conducting our research each year and comparing the results to the previous years will help REL Midwest track that trend.
While we do not expect the broad topic or content area needs (e.g., high school reform, teacher quality, etc.) to change from year to year, it is likely that states’ and school districts’ policy, programmatic, and research priorities within the larger categories of need will change from year to year. In other words, states and districts’ overarching priorities may remain consistent; however, the specific research and technical assistance needed to address these overarching priorities is much more fluid. Needs change at the state level when new policies are enacted, existing policies are reauthorized, or when changes occur in key state leadership positions. At the district level, NCLB may influence more rapid changes in district priorities, as more and more districts face corrective action and restructuring and must make changes to internal structures and processes.
With NCLB reauthorization expected soon and the upcoming presidential election, monitoring needs before, during, and after these events will be important to ensure that REL Midwest can respond quickly to state and district needs. In addition, our needs sensing data will be helpful for states and districts to track how these events may impact the stakeholders’ priorities at different levels of the education system.
Fast response projects (research projects that occur within a year or less) and technical assistance can be proposed and implemented (once approved) at any time during a contract year. This allows us to be responsive to states’ needs as they emerge.
Each of the strategies that we plan to employ to assess the region’s educational needs and priorities has different, and complementary, strengths. Well-constructed and properly fielded quantitative surveys can tell us with considerable confidence about the distribution of views across a given population. But quantitative surveys do not always lend themselves very well to exploring underlying dynamics, or testing out different kinds of approaches or “what if” scenarios. By contrast, qualitative methods, such as carefully designed and administered interviews and focus groups, are eminently suitable for such purposes. Taken together, these needs-assessment strategies form a varied, practical, flexible, and comprehensive approach to soliciting the views of stakeholders. Through these strategies, we will directly target and probe the thinking of those whom we serve—educators, policymakers, and education stakeholders within REL Midwest’s seven states—thus informing our planning processes overall.
Qualitative research methods will enable us to probe more deeply into the thinking of respondents and uncover the dynamics underlying the attitudes demonstrated through the surveys. The qualitative research methods can help explain why people hold certain beliefs. Moreover, the findings from both focus groups and state leadership interviews will help to illuminate broader research questions. Insights from focus groups and interviews also can contribute to the design of future surveys and track changes in public attitudes in a timely and cost-efficient manner.
Focus groups. Focus groups will follow standard recruitment procedures to attain target audiences and avoid bias. Typically, REL Midwest will recruit and host focus groups through established market research facilities. Public Agenda created the specifications for the composition of each group and a screener for the market research facility to use to recruit participants from its own database, through sample provided by MDR, and/or through random calling. Best-practice screening procedures will be observed, such as screening out respondents who are or have been directly involved in market research-related fields and those who have attended focus groups within the past several months (as a screen against “professional” focus group respondents).
Focus group and interview respondents will be recruited through the most methodologically sound and cost-effective procedures and mechanisms possible, which will naturally vary by type of respondent. In some cases, this recruitment will involve providing financial incentives and working through market-research facilities, subcontracting the recruitment of focus groups according to REL Midwest specifications and recruitment script. In other cases, it will involve volunteer informants recruited through REL Midwest’s database of stakeholder contacts and other “networking” strategies for recruiting respondents.
State level Interviews (Instead of surveys)
Interviews. In depth interviews will be conducted in each state with (1) two state legislators or key members of these legislators’ education staff and (2) two state education department leaders. REL Midwest and its umbrella organization, Learning Point Associates, has established longstanding relationships with both state legislators and state department of education leadership, as our organization has over 20 years’ experience conducting policy research, technical assistance, and evaluation work for the seven state region. In addition, several state-level policymakers and practitioners across the Midwest states sit on our Board of Directors and regularly engage with REL Midwest and Learning Point Associates to share information and request assistance. As a result, REL Midwest is confident that our strong relationships and history with both groups will enable us to purposively sample a group of state-level participants for these interviews. While REL Midwest informally engages in conversations with state policymakers and practitioners on a regular basis, we believe that formal interviews are also important for systematically gathering answers to questions of importance and probing for detailed responses to provide us with an in depth and rich description of state needs and priorities, This information will also be systematically collected to connect with and add to the quantitative survey responses administered to district and school level respondents.
State level interviews will be one on one interviews conducted by telephone. The responsibilities of state legislators and Department of Education leadership are such that establishing a focus group would be quite difficult and burdensome. In addition, interviews will be conducted in private with assurances that the information shared will be kept confidential. This helps to ensure that the perspectives of each state-level interview participant are more reflective of their true beliefs.
The focus group moderators and interviewers are either staff members of Public Agenda or consultants that Public Agenda have worked with extensively in the past. These moderators have extensive experience in working with stakeholders on education policy topics.
Data analysis.
Qualitative Data Analysis. Public Agenda’s inductive approach to qualitative analysis and reporting incorporates systematic methods of managing data through reduction, organization, and connection (Dey, 1993; LeCompte, 2000). Public Agenda begins by summarizing interview and focus group transcripts, highlighting the common and unique experiences that diverse stakeholder groups report. Individual interview summary reports are then compared and synthesized to identify common themes and divergent cases. Descriptive findings are included in the final report to add depth and context and to enhance understanding of various stakeholders’ beliefs reported in the quantitative survey.
Reports from qualitative focus group and interview data will be designed to be descriptive, revealing a range of attitudes and beliefs about the issues that emerge and the perceptions that drive them. Analyses will be written up as narratives that detail the findings and incorporate direct quotes when appropriate.
Individual interviews will be conducted with key state-level personnel, as it can be very difficult logistically to convene a group of state-level personnel for a focus group. In addition, state and local policymakers and practitioners are more likely to reveal their true perspectives in individual interviews. This is because individual focus group members with positions of power or authority, or the particular dynamics of a group, can often influence how other members of the group choose to respond. State level stakeholders have positions that vary in terms of their roles, responsibilities, decision-making power and authority, to a greater degree than teachers or parents. For these reasons, we plan to conduct interviews with state-level and district level stakeholders and focus groups with teachers, parents, and principals and superintendents..
2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.
Data collected will be used to inform REL Midwest’s planning processes and substantiate the research, training and technical support it provides to the region. In addition, needs sensing results will be disseminated to inform planning and improvement efforts in REL Midwest states. Broadly speaking, these efforts will allow REL-Midwest to support the goals of the recently reauthorized Elementary and Secondary Education Act (ESEA) by exploring and uncovering regional needs in such critical topics as:
High School Reform
Teacher Quality
Achievement Gap
Supplemental Educational Services
Special Needs, including English Language Learners
No Child Left Behind Act
Continuous needs sensing and reporting will ensure that the applied research and technical assistance activities conducted are indeed on target and meet the specific needs of the Midwest Region. The audiences for whom the data collection strategies will target include all educators, policymakers, and education stakeholders within REL Midwest’s seven states. Findings will be disseminated through three quarterly reports and an annual report.
The REL Midwest contract for 2006-2010 addresses the goals of the recently reauthorized Elementary and Secondary Act (ESEA), which includes a more focused effort on conducting scientifically-based research and providing training and technical assistance on the latest findings from scientifically valid research that pertain to ESEA goals and regional needs. Needs sensing activities and deliverables will provide a critical data source to ensure that research, training, and technical assistance activities are aligned with the needs of the region. The U. S. Department of Education (ED) will use monthly reports to monitor REL Midwest’s ongoing research efforts, regional needs analyses, training and technical assistance requests, and contractors’ responses to these requests. ED will use quarterly and annual reports to stay informed on regional priorities and areas of greatest need.
REL Midwest is not including parents in the yearly sample. The needs assessment questionnaire is designed for educators at the school, district and state level, and many of the questions we ask require knowledge of school, district and state operations which many parents have no familiarity.
REL Midwest did not include teachers in the 2007 and 2008 survey samples. Databases available through MDR and other education database companies provide information on only a small subset of teachers in the region and do not allow us to collect data from a representative sample of teachers. In addition, unlike administrators, teachers often do not have their own office telephones, and occasionally may not have email addresses. As a result, the response rates for teachers will be lower and will be demographically disproportionate. Since REL Midwest and Public Agenda plan to achieve the highest response rate possible and conduct the most rigorous survey possible, we chose not to include teachers in our main survey.
Since is possible to conduct much of the survey instrument with teacher populations, and REL Midwest added an over sample of teachers in 2009. This way, REL Midwest can assess teachers’ needs in a quantitative manner, but keep the response rate effects isolated to one year only.
REL Midwest will use focus groups to deepen our understanding of teacher and parent needs and further new areas of inquiry for future projects. If their needs run counter to the district and state level sample then REL Midwest will report that and use it to decide next steps.
3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision of adopting this means of collection. Also describe any consideration of using information technology to reduce burden.
Public Agenda, REL Midwest’s primary needs assessment subcontractor, will use the Total Design Method of survey data collection, the highest professional standard of quality for survey research, and will use established design principles to ensure that the questionnaire functions as intended. In addition to standard best practices, Public Agenda will develop questions with the goals of using the exact same construction (a unimode design) in two computer-assisted modalities: computer-assisted telephone interviewing and a web-based survey. This study will be carefully designed to be equivalent across modes, with question formats and sampling procedures carefully designed and tested to be comparable
The primary purposes for utilizing multiple modalities (e.g., web-based and telephone surveys) in survey administration are (1) to increase the efficiency and probability of capturing a representative cross section of responses across the region and (2) to achieve the highest possible survey response rates. Web-based surveys allow for efficient communication and distribution to educators across the region, helping to ensure that responses are collected in the region’s most rural areas. Using web-based surveys increases response rates by making it much easier and more efficient for respondents to complete. Features such as pop-up boxes to provide special instructions when needed; scroll bars that allow for easy access to all survey sections with a click of the mouse; and drop down boxes that hide options from review until needed are a few of the features that can influence higher response rates.
A variation of the Total Design Method (Dillman, 2000) will be used to increase the quantity and quality of the responses received. This method begins with the development of an easy to read and comprehend survey questionnaire, using a layout that is in accordance with visual principles of design for comprehension and easy response. The Total Design Method then follows a system of five or more compatible contacts. First, a brief pre-notice letter will be mailed to respondents a few days prior to the survey questionnaire. The pre-notice letter will (1) very briefly explain the purpose of the study, (2) will provide a phone number and a web address to provide respondents the option of conducting the survey either by telephone (using a CATI program) or online (using a web-survey program), and (3) will emphasize that the person’s response would be greatly appreciated. Follow up reminders will be conducted via a phone call and/or an e-mail, always offering either mode for the respondent and being receptive to their needs or concerns. In addition, a second letter may be sent to non-respondents after five to seven business days. This second letter will indicate the completed questionnaire has not yet been received and urge the recipient to respond.
Dillman, D. (2000). Mail and internet surveys: The tailored design method. New York: John Wiley & Sons, Inc.
REL Midwest does not mean to suggest that falling short of the target response rate in one area can be made up by a different collection mode. REL Midwest intends to achieve the expected 80% response rate in the survey and use the focus groups, one on one interviews and online engagement work to deepen the quantitative work with in depth open-ended responses that will underscore thee needs facing the region.
We believe that we can attain an 80% response rate with the sample as noted in the revised table 4 below and the indicated number of interviews. Our mixed mode design of phone and online interviews, coupled with our prenotification letters and letters of encouragement as well as successive reminder phone calls (up to 15) will increase our response rates to the proper level.
4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use of the purposes described in Item 2 above.
The data to be collected in these instruments are not available from any other source on a national basis. While needs sensing data was continuously collected, analyzed and disseminated under the previous contract with the North Central Regional Educational Laboratory (NCREL), REL Midwest has become the new mechanism through which the region’s changing educational needs are to be captured and disseminated. No other federally funded research entity targets its needs sensing data collection efforts in such a comprehensive manner within and across all Midwest states.
State organizations and regional committees are designed to assist in needs-sensing activities. In addition, reports on state goals, priorities, and needs are readily available on state websites and through REL Midwest’s ongoing communications with state and local educational agencies. However, the information collected from these entities are used as archival data sources that are triangulated with survey and interview data (1) to provide in-depth information on regional needs across the region and (2) to validate findings from smaller scale investigations of state and local needs.
To verify that no duplication will occur, several conversations occurred between REL Midwest and state agency during the first contract year. Through these conversations, REL Midwest confirmed that (1) a large scale effort to collect data on statewide needs and priorities had not been established in any Midwest state and (2) the needs assessment data REL Midwest plans to collect is unlike information already available in the Midwest states.
In addition, both the REL Midwest Needs Assessment methodology and quantitative and qualitative instruments were presented to REL Midwest Board Members in October, 2006. The purpose for our presentation was to (1) review with REL Midwest states the methods for collecting, analyzing and reporting Needs Assessment data, (2) to provide a forum through which Board Members could offer feedback on our methodology and data collection instruments, and (3) gather assurances that no similar forms of needs assessment data collection were being conducted, or had been conducted in the recent past, in any of the seven Midwest states. The Board Members- including state superintendents, researchers, and other state education officials representing each of the seven REL Midwest states- provided feedback and suggested changes, which REL Midwest and Public Agenda integrated into the quantitative survey instrument and focus group protocols. These procedures occurred before REL Midwest submitted the final instruments and supporting statement for OMB review, in order to prevent data collection duplication with any REL Midwest state.
Each of the data collection strategies that Public Agenda and REL Midwest plan to employ to assess regional needs and priorities has different, and complementary, strengths. Well-constructed and properly fielded quantitative surveys can tell REL Midwest with considerable confidence about the distribution of views across a given population. By contrast, qualitative methods, such as carefully designed and administered interviews and focus groups, are eminently suitable for such purposes. In addition, for time-series results, REL Midwest are likely to gain the greatest insight for the research dollar by doing quantitative work every other year, using its qualitative work to dig deeper into topics and specific needs that emerge from its surveys..
5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-1), describe any methods used to minimize burden.
Collection of needs-sensing data is voluntary and will not impact small businesses.
REL Midwest and Public Agenda will take care to ensure that individuals representing educational organizations in rural areas will not be unduly burdened. We will attempt to contact them through the same means we use to contact other respondents and will be highly cognizant of their situation as a small entity. If we contact rural respondents and they are busy, we will work with them to either find a time amenable to them or we will cease contacting them if that is their wish.
6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.
The systematic collection, analysis, and reporting of needs across the seven state region is required to fulfill REL Midwest’s contract obligations with ED. It is important to continually assess needs and priorities to understand how to best serve the Midwest states, as their educational needs and priorities will change over time. Withholding permission to collect needs assessment data as described would prevent REL Midwest from using results to determine with confidence what are the major needs and priorities of the seven Midwest states. As a result, any proposals for fast response research or state specific technical assistance would be based on anecdotal data, which violates requirements of the Regional Educational Laboratory Program, as defined by The Education Sciences Reform Act of 2002, Part D, Section 174, (20 U.S.C. 9564). Other short and long term consequences for not collecting needs assessment data as described include (1) the possibility of proposing and conducting research that is not relevant to the needs of the region and (2) proposing and conducting technical assistance that does not address current state or local level priorities. Over the long term, such consequences could result in REL Midwest’s inability to build a relevant and useful library of research to inform state level policies and program, which could ultimately prevent states from building capacity to improve student achievement.
Participation in all data collection activities is voluntary. Multiple data collection strategies are planned, which include surveys, focus groups, individual interviews. In addition, other activities such as webcasts, open-hearings, and policy and research forums integrate the dissemination of technical assistance with information to inform ongoing needs. REL Midwest and its subcontractors (Public Agenda and the Princeton Survey Research Center) have extensive experience in administering surveys, facilitating focus groups, and conducting interviews. REL Midwest’s expertise in these areas, combined with the establishment of strong and supportive relationships under NCREL, will help to ensure high response rates across a number of data collection activities. Should one data source fall short of expectations in terms of the percent response rate, validity and reliability of results, or depth of information provided, the multiple measures included in data collection will ensure that the data collection activities as a whole will accurately reflect the needs of the region and its subgroups.
7. Explain any special circumstances that would cause an information collection to be conducted in a manner:
requiring respondents to report information to the agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records of more than three years;
in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.
No special circumstances apply.
8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instruction and record keeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.
Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every three years – even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.
There are several strategies in place to ensure that participant burden will be minimized. First, random sampling will occur with individuals from various sectors of the educational field. Because REL Midwest and Public Agenda are generating an independent random sample each time they conduct surveys and focus groups, burden will be minimized by maintaining a low probability that they will speak with the same participants within a set of organizations multiple times over the five-year contract period.
Participation in all data collection activities is voluntary and participants’ rights of refusal and retraction will be communicated in all data collection activities. Additionally, pretests and pilots will be conducted on all data collection instruments. Revisions will be made to data collection protocols and processes based on feedback obtained from these efforts, to make the entire reporting process clearer and less burdensome. REL Midwest plans to garner ongoing feedback from its Board of Directors on the data collection, analysis and reporting process. REL Midwest plans to incorporate changes from this feedback into its work.
As indicated in question four above, REL Midwest consulted with the Board of Directors on October 4, 2006. Per the Board’s request, a few small wording changes were made in the quantitative instrument before it was sent to OMB for review and approval
The primary contractors involved in this data collection effort are provided in the table below.
Table 1. Contractor Staff
Organization |
Name |
Telephone Number |
Public Agenda |
Will Friedman |
212-686-6610 |
Princeton Data Source |
Evans Witt |
202-293-4710 |
Farkas, Duffett Research |
Steve Farkas |
201-332-5471 |
Marketechnique |
Hart Hooten |
212-414-2824 |
9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.
No payment or gift of any kind will be provided to respondents.
10. Describe any assurances of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.
Through Public Agenda, REL Midwest does not anticipate collecting any sensitive information, such as Social Security numbers, during the course of this project; any personally identifiable information will be protected with industry-standard encryption technology.
Public Agenda follows the confidentiality and data protection requirements of IES (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183). Public Agenda will protect the confidentiality of all information collected for the study and will use it for research purposes only. No information that identifies any study participant will be released. Information from participating institutions and respondents will be presented at aggregate levels in reports. Information on respondents will be linked to their institution but not to any individually identifiable information. No individually identifiable information will be maintained by the study team. All institution-level identifiable information will be kept in secured locations and identifiers will be destroyed as soon as they are no longer required. Public Agenda obtains signed NCEE Affidavits of Nondisclosure from all employees, subcontractors, and consultants that may have access to this data and submits them to our NCEE COR.
Neither REL Midwest nor Public Agenda will release any data sets to any persons or organizations not named within this proposal. They are committed to protecting the confidentiality of individually identifiable data as outlined in the Privacy Act of 1974 (amended), the E-Government Act of 2002, and the U.S. Patriot Act of 2001.
Public Agenda has a nondisclosure policy; this policy states that employees will not release any individually identifiable data to any person not sworn to the preservation of confidentiality. All staff with access to individually identifiable information will use that information only for purposes stated within the contract.
Respondents will be told—in writing or over the phone, depending upon the survey form—that their answers may be used only “for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law.” In addition, REL Midwest will adopt a reporting convention that omits or combines responses whenever the cell size falls below 3 (excepting cells of 0).
Public Agenda uses industry-standard procedures and technology to secure its Windows 2003 local area network (LAN). The live Web servers, which host Public Agenda Online, are provided by Digex Inc., an MCI company (www.digex.net). Both the local network and Web servers are under the supervision of Public Agenda’s Online Department.
Public Agenda procedures are designed to guard the five key areas of network security: physical, personnel, procedural, data backup, and virus/spyware protection.
The LAN servers are on-site at Public Agenda and are kept in a secure, temperature-controlled room with UPS power backup in case of a blackout. Firewall and IP filtering technology is in place to prevent outside intrusion. The live Web servers are firewalled, under 24/7 monitoring by Digex technicians and located at the secure Digex facility in Beltsville, Maryland. Digex uses SysTrust certified security and backup procedures. Passwords to access the servers are held on a strict need-to-know basis and changed regularly.
All Public Agenda employees are aware of the organization’s acceptable Internet use policies. Sensitive data is compartmentalized, so that only employees with a specific need-to-know have access to it. All network, e-mail, and database access is controlled by usernames and passwords, which are changed on a regular basis.
The Online Department maintains complete documentation for procedures and systems, including vendor and support information, backup plans, and system status.
LAN data security is provided by daily tape backups, with the tapes being stored off-site on a daily basis. Digex servers have full weekly and daily incremental backups, with off-site storage.
All servers and every desktop computer at Public Agenda are equipped with antivirus and antispyware protection, with file definitions updated regularly. Windows updates are installed automatically.
In addition to these measures, Public Agenda will abide by any other recommended industry and government best practices with respect to data that are collected.
11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. The justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.
None of the questions utilized in surveys, interviews, or focus groups concern topics commonly considered private or sensitive, such as religious beliefs or sexual practices.
12. Provide estimates of the hour burden of the collection of information. The statement should:
Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.
If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-1. Provide estimates of annualized cost to respondents of the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14.
Compliance burden on the public are minimal, as all needs sensing data collection activities in the REL Midwest contract are voluntary. Public Agenda plans to conduct surveys, and focus groups with policymakers and educators during the first year of data collection. Informal focus groups were conducted in 2006 with approximately tfhree groups of less than 9 participants, resulting in a total of 27 focus group participants. In addition three individual interviews were conducted over the phone with rural administrators. A total of 2,800 surveys will be administered to educators and policymakers in REL Midwest states. Each state-level sample will be designed to yield approximately 100 interviews across three major categories: government officials, education interest groups, and business/community leaders. This approach will result in approximately 700 state-level interviews across the entire region. District-level interviews surveys will be conducted with 300 100 superintendents and 100 school board members educators in each REL Midwest state (7 states total) with the following groups:in the first and third year of data collection. superintendents and principals, school board members, and local union members and leaders.In the second year of data collection 200 principals and 200 teachers will be surveyed as the school level sample. In addition, a pilot of 100 interviews surveys will be conducted, including 75 75 district school respondents and 25 state district respondents.
In table 2 and 3 below, burden estimates are provided for the survey and the interviews and focus groups. The tabled estimates specify the number of respondents, annual burden hours, and costs to respondents.
Respondent Category |
Number Of Responses |
Est. Hours per Response |
Est. Hours Total Burden |
StateDistrict -level survey respondents |
7251400 |
15/60 minutes |
181.25350 |
District-School level survey respondents |
28002,175 |
15/60 minutes |
543.75700 |
Interview participants |
28 |
60/60 minutes |
28 |
Focus group participants |
84 (14 groups x 6 part.) |
90/60 minutes |
126 |
Total Respondent Burden |
3,0124312 |
|
8791204 |
Respondent Category |
Hourly Rate |
Total Number of Hours |
Number of Responses |
Estimated Data Collection Cost to Respondents |
District -level survey respondentsState-level survey respondents |
$40 |
15/60 minutes |
1400725 |
$725014000 |
School level survey respondentsDistrict-level survey respondents |
$40 |
15/60 minutes |
28002,175 |
$21,75028000 |
Interview participants |
$40 |
60/60 minutes |
28 |
$1,120 |
Focus group participants |
$40 |
90/60 minutes |
84 |
$5,040 |
Totals |
|
|
3,0124312 |
$35,160$48160 |
Data collection plan during 2007:
40 participants (5 groups x 8 part./group) x 90/60 hours/focus group = 60 hours
20/60 hours x 2,800 respondents= 933.3 hours
Total hours requested during 2007= 993.3 hours
Describe any other costs to respondents.
No additional costs are associated.
Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expenses that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.
Below is an estimation of costs to the federal government. See Appendix C for the line item budget, which quantifies the percentages of labor hours associated with total costs.
Tasks for FY 2007:
Capital/startup costs |
|
Production and publication |
$10,000 |
Interactive Online distribution |
$10,000 |
Technology |
$7,000 |
Rent/Utilities/RE taxes |
$35,000 |
Miscellaneous |
$1,000 |
Overhead |
$74,550 |
Survey software |
$5,000 |
Total annualized capital/startup costs |
$142,550 |
|
|
O&M costs |
|
Staff |
$243,590 |
Consultants |
$48,000 |
Focus Groups |
$31,660 |
Surveys |
$10,000 |
Travel |
$13,500 |
Total O&M costs |
$346,750 |
|
|
Total annualized cost requested |
$489,300 |
15. Describe any changes in the burden from prior approvals
There are no changes in the burden from prior approvals.
16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of the report, publication dates, and other actions.
The results of this data collection may be used in several ways. Public Agenda will use the data to prepare an annual State of the Region Report that provides regional and state-specific results of educational needs and priorities in the REL Midwest region.
Additionally, REL Midwest may use the raw needs assessment data or annual report results to prepare presentations at regional or national conferences. Finally, reports, presentations, and other training and technical assistance activities specified in the contract will be conducted using the results from 2007 data collection activities. Below is a description of the analytic plan for the quantitative and qualitative components of the needs assessment work, followed by a timeline for conducting the work.
Quantitative Analysis
For the quantitative analyses, Public Agenda will work with Princeton Data Source to ensure that datasets are thoroughly cleaned through use of a computer validation program. Upon completion of data checks, Princeton Data Source will run initial simple summary statistics for Public Agenda to use for examining emerging relationships. Public Agenda will analyze the quantitative survey data by applying a mix of descriptive statistics, cross-tabulations, t-tests and correlations as relevant to the question structure and number of responses. The frequency of each response category for total respondents, selected target subpopulations, and other meaningful points of analysis will be identified. Additional t-tests and correlations will be run depending on the question structure. Analysis of the quantitative survey will explore significant outcomes by parceling results by state, stakeholder group, and other demographic characteristics. Relationships within and between groups will be explored using tests of significance when applicable.
Public Agenda will prepare a final highly accessible print document that will summarize all that is learned from this phase of the project. It will include background material, references of key findings, significant geographic and demographic group analyses, and a discussion of interactions of the various questions in the survey where significant.
Qualitative Analysis
Public Agenda’s inductive approach to qualitative analysis and reporting incorporates systematic methods of managing data through reduction, organization, and connection (Dey, 1993; LeCompte, 2000). Public Agenda begins by summarizing interview and focus group transcripts, highlighting the common and unique experiences that diverse stakeholder groups report. Individual interview summary reports are then compared and synthesized to identify common themes and divergent cases. Descriptive findings are included in the final report to add depth and context and to enhance understanding of various stakeholders’ beliefs reported in the quantitative survey.
Reports from qualitative focus group and interview data will be designed to be descriptive rather than predictive, revealing a range of attitudes and beliefs about the issues that emerge and the perceptions that drive them. Analyses will be written up as narratives that detail the findings and incorporate direct quotes when appropriate.
A revised timeline for data collection, report preparation, and production is below. This timeline assumes that OMB approval will be granted before May, 2007.
Public Agenda General Timeline and Program of Work
2007
Needs-assessment activities in 2007 will involve quantitative research, qualitative research, and ongoing online engagement activities.
First Quarter 2007. (February 1– April 30, 2007) Begin online engagement; Continue quarterly reporting.
Public Agenda will do as follows:
Hold monthly online advisory group discussions on various educational needs.
Hold Digital Public Hearing inviting wide input on regional educational needs.
Produce quarterly reports.
Consult and coordinate with REL Midwest throughout.
Second Quarter 2007. (May 1– July 31, 2007) Begin conducting leadership interviews and focus groups; continue online engagement and quarterly reporting.
Public Agenda will do as follows:
Pretest and pilot the survey questionnaire in iterative rounds of instrument development. (Pending OMB approval).
Analyze survey results.
Conduct one-on-one, in-depth interviews with (1) two state legislators or key members of these legislators’ education staff and (2) two state education department leaders.state and local education leaders (at least four interviews per state). Respondents will come from government (e.g., education staff from the governor’s office, state department of education professionals, the state board of education, and legislators and/or staff on education committees), education interest groups (state-level education union leaders, state PTA) and business/community leaders (business organizations such as the state chamber of commerce and/or business roundtable, and civic and community groups and organizations that have an education interest, including heads of community organizations heavily involved in education advocacy such as La Raza or the Urban League.) These interviews will discuss continuing and emerging educational needs as well as information from REL Midwest’s ongoing fast-response studies to gain the best possible feedback on fast-response efforts.
Conduct focus groups (covering each REL Midwest state) with a variety of stakeholders (such as parents, teachers, and principals) in urban, suburban, and rural settings.superintendents and school board members (Pending OMB approval).
Hold monthly online advisory group discussions on various educational needs.
Produce quarterly reports.
Hold Digital Public Hearing inviting wide input on regional educational needs.
Consult and coordinate with REL Midwest throughout.
Third and Fourth Quarters 2007. (August 1– January 31, 2008) Analyze survey results; create Needs Assessment database; Produce and disseminate Needs Assessment report; continue online engagement and quarterly reporting; update Needs Assessment database.
Public Agenda will do as follows:
Field the quantitative survey with superintendents and school board members.
Create and manage (in coordination with REL Midwest) database to create archive of all data and input sources. (For more, see Database subsection in Section II on Methodology).
Continue one-on-one, in-depth interviews with state and local education leaders (at least four interviews per state).
Continue focus groups (covering each REL Midwest state) superintendents and school board memberswith a variety of stakeholders (such as parents, teachers, and principals) in urban, suburban, and rural settings.
Provide a thorough analysis of research, culminating in a final highly accessible print document that will incorporate all it has learned from the quantitative and qualitative phases of work in 2007.
Create Web-based version of the report for download from REL Midwest and Public Agenda websites and other sites that will link to the research.
Do press work, leadership briefings, and other forms of dissemination to widen and deepen impact and discussion of research results.
Hold monthly online advisory group discussions on various educational needs.
Produce quarterly reports.
Update (in coordination with REL Midwest) database to archive of all data and input sources.
Consult and coordinate with REL Midwest throughout.
Plan for 2008 work.
2008
Needs-assessment activities in 2008 will involve quantitative research, qualitative research, and ongoing online engagement activities.
First Quarter 2008. (February 1– April 30, 2008) Continue online engagement and quarterly reporting
Public Agenda will do as follows:
Hold monthly online advisory group discussions on various educational needs.
Hold Digital Public Hearing inviting wide input on regional educational needs.
Produce quarterly reports.
Consult and coordinate with REL Midwest throughout.
Second Quarter 2008. (May 1– July 31, 2008) Continue online engagement and quarterly reporting; conduct one-on-one interviews; conduct focus groups.
Public Agenda will do as follows:
Conduct one-on-one, in-depth interviews with 1) two state legislators or key members of these legislators’ education staff and (2) two state education department leaders.state and local education leaders (at least four interviews per state). Respondents will come from government (e.g., education staff from the governor’s office, state department of education professionals, the state board of education, and legislators and/or staff on education committees), education interest groups (state-level education union leaders, state PTA) and business/community leaders (business organizations such as the state chamber of commerce and/or business roundtable, and civic and community groups and organizations that have an education interest, including heads of community organizations heavily involved in education advocacy such as La Raza or the Urban League.) These interviews will discuss continuing and emerging educational needs as well as information from REL Midwest’s ongoing fast-response studies to gain the best possible feedback on fast-response efforts.
Conduct focus groups (covering each REL Midwest state) with with a variety of stakeholders (such as parents, teachers, and principals) in urban, suburban, and rural settings.).parents and teachers
Hold monthly online advisory group discussions on various educational needs.
Produce quarterly reports.
Hold Digital Public Hearing inviting wide input on regional educational needs.
Consult and coordinate with REL Midwest throughout.
Third and Fourth Quarters 2008. (August 1– January 31, 2008) Field survey; Analyze survey results; create Needs Assessment database; Produce and disseminate Needs Assessment report; continue online engagement and quarterly reporting; update Needs Assessment database.
Public Agenda will do as follows:
Field the quantitative survey with parents and teachers.
Create and manage (in coordination with REL Midwest) database to create archive of all data and input sources. (For more, see Database subsection in Section II on Methodology).
Continue one-on-one, in-depth interviews with state and local education leaders (at least four interviews per state).
Continue focus groups (covering each REL Midwest state) with parents and teachers with a variety of stakeholders (such as parents, teachers, and principals) in urban, suburban, and rural settings.
Provide a thorough analysis of research, culminating in a final highly accessible print document that will incorporate all it has learned from the quantitative and qualitative phases of work in 2008.
Create Web-based version of the report for download from REL Midwest and Public Agenda websites and other sites that will link to the research.
Do press work, leadership briefings, and other forms of dissemination to widen and deepen impact and discussion of research results.
Hold monthly online advisory group discussions on various educational needs.
Produce quarterly reports.
Update (in coordination with REL Midwest) database to archive of all data and input sources.
Consult and coordinate with REL Midwest throughout.
Plan for 2009 work.
2009
Needs-assessment activities in 2009 will involve quantitative research, qualitative research, and ongoing online engagement activities.
First Quarter 2009. (February 1– April 30, 2009) Continue online engagement and quarterly reporting
Public Agenda will do as follows:
Hold monthly online advisory group discussions on various educational needs.
Hold Digital Public Hearing inviting wide input on regional educational needs.
Produce quarterly reports.
Consult and coordinate with REL Midwest throughout.
Second Quarter 2009. (May 1– July 31, 2009) Continue online engagement and quarterly reporting; conduct one-on-one interviews; conduct focus groups.
Public Agenda will do as follows:
Conduct one-on-one, in-depth interviews with 1) two state legislators or key members of these legislators’ education staff and (2) two state education department leaders.
state and local education leaders (at least four interviews per state). Respondents will come from government (e.g., education staff from the governor’s office, state department of education professionals, the state board of education, and legislators and/or staff on education committees), education interest groups (state-level education union leaders, state PTA) and business/community leaders (business organizations such as the state chamber of commerce and/or business roundtable, and civic and community groups and organizations that have an education interest, including heads of community organizations heavily involved in education advocacy such as La Raza or the Urban League.) These interviews will discuss continuing and emerging educational needs as well as information from REL Midwest’s ongoing fast-response studies to gain the best possible feedback on fast-response efforts.
Conduct focus groups (covering each REL Midwest state) with a variety of stakeholders (such as parents, teachers, and principals) in urban, suburban, and rural settings.).superintendents and school board officials
Hold monthly online advisory group discussions on various educational needs.
Produce quarterly reports.
Hold Digital Public Hearing inviting wide input on regional educational needs.
Consult and coordinate with REL Midwest throughout.
Third and Fourth Quarters 2009. (August 1– January 31, 2009) Field survey; Analyze survey results; create Needs Assessment database; Produce and disseminate Needs Assessment report; continue online engagement and quarterly reporting; update Needs Assessment database.
Public Agenda will do as follows:
Field the quantitative survey including a sample of teachers
Create and manage (in coordination with REL Midwest) database to create archive of all data and input sources. (For more, see Database subsection in Section II on Methodology).
Conduct one-on-one, in-depth interviews with 1) two state legislators or key members of these legislators’ education staff and (2) two state education department leaders.
Conduct focus groups (covering each REL Midwest state) with superintendents and school board officials
Continue one-on-one, in-depth interviews with state and local education leaders (at least four interviews per state).
Continue focus groups (covering each REL Midwest state) with a variety of stakeholders (such as parents, teachers, and principals) in urban, suburban, and rural settings.
Provide a thorough analysis of research, culminating in a final highly accessible print document that will incorporate all it has learned from the quantitative and qualitative phases of work in 2009.
Create Web-based version of the report for download from REL Midwest and Public Agenda websites and other sites that will link to the research.
Do press work, leadership briefings, and other forms of dissemination to widen and deepen impact and discussion of research results.
Hold monthly online advisory group discussions on various educational needs.
Produce quarterly reports.
Update (in coordination with REL Midwest) database to archive of all data and input sources.
Consult and coordinate with REL Midwest throughout.
17. Describe arrangements for displaying the number provided by OMB and its expiration date.
The approval number provided by OMB and its expiration date will appear in the heading on all surveys and on the “description of the work” page for focus group participants.
Exceptions to OMB Form 83-I
There are no exceptions to the certification statement identified in item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.
| File Type | application/msword |
| File Title | SUPPORTING STATEMENT |
| Author | NCERL |
| Last Modified By | Sheila.Carey |
| File Modified | 2007-06-05 |
| File Created | 2007-06-05 |