ED Response to OMB Qs

December 2006

Responses to OMB Comments on Priority Needs for REL Southwest

1850-NEW

1. Is this planned as a one-time survey or do you expect to administer it again in the future?

We propose to have this survey administered again in 2008 and possibly in 2010.

The constituent responses collected through this research survey in 2007 will enable ED and REL Southwest to enhance their understanding of the educational research needs of their constituents in the southwest and to develop a baseline measure of how educational research is viewed currently in the southwest. By administering this survey again in 2008 and possibly in 2010, ED and REL Southwest will be able to revaluate this measure and track any changes in the perceptions and needs of their constituents regarding educational research. By administering these surveys in the subsequent years, ED and REL Southwest will continue to read the “pulse” of their constituents and will be able to plan their future research development and dissemination agendas accordingly. It will also enable ED and REL Southwest to identify continuous needs as well as those needs that change with time. We will also be able to regularly evaluate how research that is being disseminated satisfies stated needs in the southwest region.

2. Please explain the ways in which you believe that the research needs of educators/ parents/ business leaders in the SW will differ from the research needs of those in other areas of the country.

The Southwestern region is among the most unique of the ten laboratory regions. Its five states cover an enormous landmass that is geographically, demographically, culturally and economically varied. Such a huge diversity may highlight that the research needs of the educators/parents/business leaders in the southwestern region are also diverse and may be different from those in other areas of the country. This baseline research can either prove or dispel the likeness or differences to similar constituents in other regions. This is one of the objectives in the study as this has not yet been measured. A few notable factors that contribute to the differing research needs of our constituents are:

• High Poverty

Looking east, we share the delta region with the Southeast. Areas of the Delta region are desperately poor with high levels of student failure. Displacement due to the hurricanes is high and thousands of our Gulf Coast neighbors have re-situated in other southwestern states.

Research by national nonprofit organizations such as the Rural School and Community Trust have also highlighted the poverty levels in the south west region, especially New Mexico and Louisiana. In a 2005 publication of the Rural School and Community Trust titled “Why Rural Matters 2005”, researchers Johnson and Strange published a “Poverty Gauge Rank” in which New Mexico was ranked 1^st and Louisiana was ranked 3^rd.

	AR	LA	NM	OK	TX
Why Rural Matters 2005 Poverty Gauge Rank	7th	3rd	1st	6th	16th
Why Rural Matters 2005 Overall Priority Ranking	7th	4th	2nd	6th	16th

Johnson and Strange, 2005, Why Rural Matters 2005

• High Native American Populations

Looking north and west (Oklahoma and New Mexico), we have a high proportion of Native Americans, and there are vast rural areas with an agricultural base. Although 23% of Native Americans live in poverty, 34% of Native American children do, and an equal number have mothers with less than a high school education.

• Large ELL Populations

Looking west, we share a long border with the western region. We have many ELL and immigrant students here. New Mexico and Texas share a vast border with Mexico. This border has been called “porous” as increasing flow of immigrants into these two states have pushed state legislators (Richardson- New Mexico, Bonilla- Texas) to call for a State of Emergency along the Mexican border. The rapid increase in numbers of immigrant children challenges the resources and capabilities of our schools.

• High Student Mobility

Throughout the region there exists high student mobility and in-migration from Mexico, South America and other countries. Texas continues to have an influx of population as the general U.S. population moves southward. Migrant work is also high in this region and the mobility has unique effects on the children of migrant workers.

• Large Rural Populations

In each of our five states, large numbers of public school students reside in rural areas where resources are limited and the need for teachers and leaders who are prepared to handle these challenges is acute. Half of the public schools in Oklahoma (51%) and Arkansas (49%) are in rural areas followed by New Mexico (35%), Louisiana (32%) and Texas (23%).

	AR	LA	NM	OK	TX
Number of students enrolled in rural schools	157,909	183,623	56,180	197,157	532,378
Percent of students enrolled in rural schools	35.1%	25.7%	18.3%	31.7%	12.8%
Percentage of public schools in rural areas	49.0%	32.0%	35.0%	51.0%	23.0%
Percentage of students attending small rural schools	21.8%	14%	11.3%	24.2%	5.9%
Percentage of rural students who are minorities	18.2%	35.3%	70.6%	35.7%	34.9%

Johnson and Strange, 2005, Why Rural Matters 2005

Secondary research conducted by REL Southwest has revealed that little primary research has been conducted to accurately identify research needs of educators/parents/business leaders in the southwest region. Nor has the available research been projectable to or representative of the constituents in our five state region. By completing the proposed research, REL Southwest will be able to accurately identify the unique research needs of our constituents in the southwest and in the future we may compare their unique needs nationally (as this is not in the scope of the current study as proposed).

3. Please explain what information you used to distinguish the low response rates groups that receive the incentive from the higher response rate groups that will not receive the incentive. Why were the specific groups targeted to receive the $10 incentive? Is there empirical data that support these groups? For example, it would seem just as likely that Hispanics may be hard to reach.

Within our target survey populations (please see sampling chart below from pages 2-4, Item 2, OMB Form Part B.2), we identified several groups to be low response rate groups that will receive a $10 incentive to complete our survey. Identification of some of these groups as low response rate groups (such as native American parents) was based on our discussions with our board members including Dr. Belinda Biscoe (Director of Mid-Continent Comprehensive Center (MC3) in Oklahoma) and Irowena Whitehair, who is a Teacher and belongs to the Native American community of New Mexico.

In addition, identification of these groups was also based on their relatively small population sizes. As per our sampling methodology (please see explanation of sampling methodology on page 13, Item 16, OMB Form Part A.2), to project research results to a specific group/population within the desired confidence levels, we need to collect responses from a predetermined number of respondents who belong to that group. However, given the small sizes of some of the target populations, we realize that it would be difficult to contact and collect data from respondents belonging to such finite populations.

For example, from the chart below, we know that the population for “Superintendents” in urban/suburban Arkansas is 34. Based on our sampling methodology, we need to collect research data from at least 25 respondents for this population. To meet this survey quota, we would need a response rate of over 70%, which is extremely high to achieve without offering respondents from this category an incentive to take the survey. Conversely, we are also surveying populations such as “Teachers” in rural Texas who we know to be quite large (73,760), and even though we need a relatively larger sample of 96 respondents from this population to complete our survey, we know that we have a large pool of respondents to approach. Additionally, from prior experiences with this population, we estimate that we will be able to fulfill our survey quotas for this population without offering any incentives.

Segment	Sub Segment	Title	AR	LA	NM	OK	TX	Sub Total	Total Sample Required
Pre K-12	Rural	Superintendents	67	31	44	80	85	_	307
	Rural	*MDR Population	p=217	p=45	p=80	p=481	p=742	_
	Urban/Suburban	Superintendents	25	22	11	45	74	_	177
	Urban/Suburban	*MDR Population	p=34	p=29	p=12	p=83	p=321	_
	Rural	Special Education Director	66	34	44	79	84	_	307
	Rural	*MDR Population	p=207	p=51	p=81	p=426	p=669	_
	Urban/Suburban	Special Education Director	26	25	10	44	73	_	178
	Urban/Suburban	*MDR Population	p=36	p=33	p=11	p=80	p=298	_
	Rural	Curriculum Director	68	48	40	74	83	_	313
	Rural	*MDR Population	p=231	p=95	p=67	p=326	p=620	_
	Urban/Suburban	Curriculum Director	35	37	14	47	79	_	212
	Urban/Suburban	*MDR Population	p=54	p=60	p=16	p=91	p=431	_
	Rural	Testing/Assessment Director	53	21	23	67	74	_	238
	Rural	*MDR Population	p=118	p=27	p=30	p=222	p=318	_
	Urban/Suburban	Testing/Assessment Director	17	14	5	22	58	_	116
	Urban/Suburban	*MDR Population	p=20	p=16	p=5	p=28	p=145	_
	Rural	ELL/Bilingual Director	36	12	23	48	73	_	192
	Rural	*MDR Population	p=56	p=13	p=30	p=94	p=306	_
	Urban/Suburban	ELL/Bilingual Director	15	9	6	19	56	_	105
	Urban/Suburban	*MDR Population	p=17	p=10	p=6	p=23	p=133	_
	Rural	Principal/Asst. Principal	88	88	83	89	94	_	442
	Rural	*MDR Population	p=992	p=1,042	p=592	p=1,175	p=3,558	_
	Urban/Suburban	Principal/Asst. Principal	86	91	80	89	95	_	441
	Urban/Suburban	*MDR Population	p=790	p=1,681	p=487	p=1,116	p=12,541	_
	Rural	Teachers (all grades, all subject areas, all pre-school, elementary, middle, high)	96	96	95	96	96	_	479
	Rural	*MDR Population	p=20,988	p=19,985	p=9,620	p=23,216	p=73,760	_
	Urban/Suburban	Teachers (all grades, all subject areas, all pre-school, elementary, middle, high)	95	96	95	96	96	_	478
	Urban/Suburban	*MDR Population	p=15,212	p=35,462	p=9,355	p=22,692	p=260,189	_
	Rural/Urban/Suburban	Librarian	87	89	82	88	95	_	441
	Rural/Urban/Suburban	*MDR Population	p=913	p=1,131	p=535	p=1,030	p=6,242	_
Higher Education	Rural/Urban/Suburban	Higher Education 2 Year Colleges (College Department Chair, Dean, College Admissions Director, College Student Affairs Director)	81	82	79	84	93	_	419
	Rural/Urban/Suburban	*MDR Population	p=503	p=568	p=426	p=645	p=3,291	_
	Rural/Urban/Suburban	Higher Education 4 Year and Graduate Colleges (College Department Chair, Undergraduate/ Graduate Dean, College Admissions Director, College Student Affairs Director)	89	93	88	92	95	_	457
	Rural/Urban/Suburban	*MDR Population	p=1,291	p=3,318	p=1,019	p=2,227	p=10,041	_
Pre K-12 and Higher Education	Rural	Parents						1,125
	Urban/Suburban	Parents						1,125
	Ethnicity - Caucasian	Parents	75	75	75	75	75	375
	Ethnicity- Black/African-American	Parents	75	75	75	75	75	375
	Ethnicity- Asian/Pacific Islander	Parents	75	75	75	75	75	375
	Ethnicity- Hispanic	Parents	75	75	75	75	75	375
	Ethnicity- Native American	Parents	75	75	75	75	75	375
	Ethnicity- Multiracial	Parents	75	75	75	75	75	375
	Education Level - Less than High School	Parents						1,125
	Education Level - High School or more	Parents						1,125
	Total Parents	Parents							2,250
	Business Leaders in Rural/Urban/Suburban	Business Leaders	100	100	100	100	100	_	500
Total Surveys Completed									8,052

Based on our more than 52 years of combined experience in the full spectrum of market research methodology and projectable data collection, we know that often it is quite difficult to get respondents to voluntarily spend some of their time in completing an online survey. From past data collection experiences, we know that it becomes even more difficult to complete online survey quotas if the target populations are small or difficult to reach. Hence to complete online data collection in a timely manner when surveying small populations, we recommend offering a small monetary incentive to respondents. By offering a small monetary incentive, we have been successful in being able to interest even those respondents who, without an incentive, might not have spared the time to attempt our survey.

Please note that in comparison to our preselected populations that are quite small in size, the Hispanic populations in our five states are quite large and not as difficult to reach. Listed below are some population estimates for Hispanic/Latino populations that are currently available with the U.S. Census Bureau:

State	2005 Total Population Estimate	Percentages of Hispanic/Latino Origin	Number of Hispanic/Latino Origin
Texas	22,859,968	34.6%	7,909,549
New Mexico	1,928,384	43.3%	834,990
Oklahoma	3,547,884	6.3%	223,517
Arkansas	2,779,154	4.4%	122,283
Louisiana	4,523,628	2.8%	126,662

(Ref: http://quickfacts.census.gov/qfd/states/22000.html)

4. If quotas are not met for an incentive group, will incentives still be offered during the phone mode?

If quotas are not met for a group that was preselected to receive modest incentives during our online data collection, incentives will subsequently be offered during the phone mode to that specific group only. Please note that incentives would not be offered to any group that has not been pre-identified on pages 3 and 4, Item 9, OMB Form Part A.2.

5. Please clarify if the incentive is provided at the professional email or the personal email.

Incentive will be provided at the email entered by respondent. It will be at the respondent’s discretion as to what type of email (professional or personal) is provided. Please note that REL Southwest will use the email provided only to send incentive.

6. Why does ED expect a 5% response rate for this survey (especially since you are providing a $10 incentive)?

With the growth of online research surveys, there has been a lot of discussion on estimated response rates for online surveys. In the article titled “Measuring response rates in online surveys” published in the leading online market research journal, Quirk's Marketing Research Review, (ref: http://www.quirks.com/articles/article.asp?arg_ArticleId=583), it is estimated that “for a Web survey, expect a response rate of 1 to 3 percent of total contacts made”. Wikipedia, the largest online encyclopedia, estimates online response rates to be between 2% - 30% (ref: http://en.wikipedia.org/wiki/Statistical_survey).

REL Southwest has estimated a 5% response rate based on its prior experiences with similar online data collection efforts. By assuming a 5% response rate, REL Southwest is able to calculate the exact number of survey invitations that need to be sent out to meet survey quotas in a timely manner. Please note that the $10 incentive is not being provided to all those who attempt the survey but only to specific, preselected populations.

As per our market research sampling methodology (please see explanation of sampling methodology on page 13, Item 16, OMB Form Part A.2), to project research results to a specific group/population within the desired confidence levels (95% confidence level and +/-10% confidence interval), we need to collect responses from a predetermined number of respondents who belong to that group.

7. Please describe the sampling frame from which you are drawing parent and business leader names. Will you stratify business leaders by industry or only by region? What is a business “leader” (as opposed to a business person)? Also, will MDR provide the sampling frame for all other respondent types? If so, how comprehensive is their database?

MDR will only provide the sampling frame for professional educators.

The parent sample will be obtained from InfoUSA, Inc., a leading industry sample provider for online surveys. We will obtain a targeted sample of parents that reside within our 5 state region and are not professional educators. The parent sample will also be stratified by the selected race categories for parents. (please see chart on page 7, Item 2, OMB Form Part B.2.)

The business leader sample will be obtained from several state and local Chambers of Commerce that have agreed to provide to REL Southwest, emails of their members who are local business leaders and are involved in education in their cities/states. REL Southwest will supplement the business leader sample obtained from the Chambers of Commerce with additional sample from leading industry list providers such as InfoUSA, Inc. We will obtain a targeted sample of business leaders who reside within our 5 state region and are not primarily professional educators or professionally involved in the education industry. Business leaders will not be stratified by any other specific industry though we will survey a mix of different industries (such as retail, manufacturing, etc.) that are active within our 5 state region. Business leaders will be stratified by state only to meet predetermined quotas (please see chart on page 7, Item 2, OMB Form Part B.2).

REL Southwest defines a business “leader” as somebody who is different from a business “person” because he/she is somebody who:

1. has a direct stake in running a business within the southwest region

2. is on the front lines of his/her business to see the impact that, elementary, secondary or higher education being imparted in his/her state has on the youth entering the workforce, and

3. wants to have a “voice” in the way education needs are identified and met within his/her state.

Business leaders will typically have the following professional titles: CEO, President, Vice President, Business Owner/Principal, Human Resource Director/Manager.

8. Please explain how ED will use the information on how satisfied respondents are with the existing research out there (Qs 16 – 19 on the educator survey).

The information from questions 16-19 will assist ED and REL Southwest in identifying first impressions and perceptions of its constituents in the southwest, regarding currently available educational research. In order to know where the current strengths and weaknesses are or how to improve educational research for use by its constituents under a continuous improvement model, one must know the opinions and experiences of its users. This research will provide this information. ED and REL Southwest will also use this information to identify how constituents in the southwest feel about the quality and relevancy/applicability of education research that is currently available. This information would allow ED and REL Southwest to keep a “pulse” of its constituents and learn about their perceptions of the effectiveness of the currently available educational research. REL Southwest will also utilize this information to plan its future research development and dissemination agendas.

Additionally, questions 16-19 will also allow ED and REL Southwest to develop a baseline measure of how educational research is viewed in the southwest in 2007. By revaluating this measure in the coming years, ED and REL Southwest will be able to track any changes (positive or negative) in the perception of its constituents regarding educational research available in the southwest.

ED could also use the information obtained here to make comparisons between the southwest region and nationally, to identify and evaluate how its constituents in different regions/states feel about the education research that is available in their state or region. Subsequently, ED could also leverage this information in formulating research policies and making changes to existing research guidelines.

9. Why doesn’t ED ask whether the respondent would use educational research if the right research were available to them? For example, a business leader in the financial services industry may have fully and accurately answered your questions, but that person may never use your research regardless; perhaps it makes sense to know this about the respondent?

ED does not ask whether the respondent would use educational research if the right research were available to them, as by asking the question in this manner, ED would induce a “bias” in the survey. The bias in this case would be the implied assumption that respondents do not use educational research because it is not available to them. By framing the question in this manner, ED would have left out any other possible reasons due to which respondents do not use educational research (e.g. “I doubt the quality of research that is available”, “Research available is difficult to understand”, “Available research is relevant, but I don’t know how to apply to my job”). Additionally ED would have also missed on the opportunity to identify the impact percentages of other possible resistance drivers.

In learning about the respondents’ perceptions about available education research, ED has drafted the questions in a manner that does not induce any type of bias. To maintain a “bias-free” mode of surveying, ED first inquires if the respondent currently uses educational research (Q12, Educator Online Survey). If the respondent indicates that he/she does not use educational research then in the next question (Q13, Educator Online Survey), ED enquires the respondents’ reasons for not using educational research. By asking the question in this manner, ED does not induce any bias but is able to identify all possible reasons for why a respondent does not currently use educational research. Then in the next question (Q21), ED inquires about the specific education issues that are important to the respondent. By this methodology, ED is first able to identify all the reasons that prevent a respondent from using educational research and then is also able to identify the top education issues that are most important to the respondent.

10. Please describe the security measures in place to secure the online responses.

Online data collection will be done using a state of the art web survey development and data collection system that has been commercially available for a decade, has more than 22,000 customers world wide , is proprietary, is built to ensure secure data collection and has not been known to have any data collection security breaches. Additionally, REL Southwest will leverage the services of a Tier 1 AT&T hosting facility to host surveys. All commercially available forms of physical and technical security are leveraged to prevent unauthorized access to survey environment and data. All accounts are password protected and under the control of REL Southwest. Examples of technical and physical security options that are included be REL Southwest to ensure secure online data collection are: SSL encryption, several layers of firewalls, biometric entry to hosting facility, service hardening on application servers, and advanced network intrusion detection systems.

REL Southwest will maintain control at all times on who within ED and REL Southwest can access online surveys and collected data. At all times, the actual publishing/launching of all surveys and collection of online responses will be controlled by a REL Southwest officer or a senior market research staff member.

11. Please clarify why the use of a phone survey is expected to improve response rates in the hurricane-impacted areas. Early in the package, a reference was made to the lack of “infrastructure,” which would make phones a better mode. Later in the package, reference was made to email being the preferred mode of communication in these areas. These two statements appear inconsistent.

We will be using online surveys to fulfill survey quotas and complete data collection because the majority of our constituents in the southwest are known to have access to the internet. Additionally, it has been our experience that online data collection is the most cost effective and efficient form of data collection. However, based on the visits made by REL Southwest directors to hurricane impacted areas of Louisiana and rural areas of Arkansas and New Mexico, and our discussions with education leaders and other constituents in these states, we know that access to internet in certain areas are lower than the rest of the southwest region. For such regions, we feel that phone surveys might be another viable alternative to collecting survey information and fulfilling our survey quotas in a timely manner.

12. Please clarify the collection of personal information (especially with regard to Question 5 above). The package states that you are not collecting personal information, yet first and last names are collected.

First and last names are being collected only to ensure that future contact with respondents (who wish to receive research developed by REL Southwest or wish to participate in future research (only for educators)) can be made in a professional manner by addressing them by their first and last names. Please note that at the beginning of the question, it is specified to all respondents that it entirely “optional” for them to provide any information about themselves.

13. Please explain what the quota targets are and how they will be defined. This needs to be determined in advance.

Please refer to quota methodology and sampling chart on page 2-4, Item 2, OMB Form Part B.2. For your convenience, we have copied the methodology explanation and chart form OMB Form Part B.2 below.

A combination of secondary and primary research efforts will be undertaken for this study. The secondary research will be conducted prior to launching the primary research. Through secondary research, most current estimates of the actual constituent populations will be identified for each of the 5 REL Southwest states. This secondary research will be used to guide the stratification and development of quotas to be used in the primary methodology. Data collection will occur every year. In the primary phase, a survey will be deployed to fulfill project research objectives. The survey will be deployed via the internet. For identified constituents who do not have access to the internet, a phone survey may be deployed to meet quota targets if necessary. For example, a phone survey may be deployed to meet survey quota targets in hard to reach places in hurricane-impacted Louisiana or rural schools/districts or with parents who do not have access to the internet and thus cannot complete an online survey.

• Stratified random sampling will be done of the entire survey population to ensure collection of statistically projectable data that is representative of all educators involved in elementary, secondary and higher education in the southwest.

• Stratified random sampling will also be done to ensure collection of statistically projectable data that is representative of all parents and business leaders involved in elementary, secondary and higher education in the southwest.

• Survey sampling and quotas will be set to fulfill 95% confidence level limits (at +/-10% confidence interval).

• Sampling of the populations will be done using the below listed two part formula. The first part calculates the sample size as though the population is infinite. This is defined as n. The second part corrects for the population size.
  1) n=(z/m)^2*p*(1-p) where:

• z is the standard normal statistic

• m is the desired error

• p is the probability of the outcome

• ^2 means "the square of the resulting number"

2) n'=n/(1-n/N) where:

• n is the result from the equation in part 1) above

• N is the population size

• Accuracy of sampling will be maintained by using educator contact information that is accurate and reflects accurate information for 2006, from renowned and prestigious list providers such as Quality Education Data (QED) and Market Data Retrieval (MDR).

• Below are stratification details for each targeted educator title. All sample counts have been based on Market Data Retrieval’s Educator 2005-2006 Population Count.

(Chart note: *MDR Population = Market Data Retrieval’s 2005-2006 Educator Database)

Segment	Sub Segment	Title	AR	LA	NM	OK	TX	Sub Total	Total Sample Required
Pre K-12	Rural	Superintendents	67	31	44	80	85	_	307
	Rural	*MDR Population	p=217	p=45	p=80	p=481	p=742	_
	Urban/Suburban	Superintendents	25	22	11	45	74	_	177
	Urban/Suburban	*MDR Population	p=34	p=29	p=12	p=83	p=321	_
	Rural	Special Education Director	66	34	44	79	84	_	307
	Rural	*MDR Population	p=207	p=51	p=81	p=426	p=669	_
	Urban/Suburban	Special Education Director	26	25	10	44	73	_	178
	Urban/Suburban	*MDR Population	p=36	p=33	p=11	p=80	p=298	_
	Rural	Curriculum Director	68	48	40	74	83	_	313
	Rural	*MDR Population	p=231	p=95	p=67	p=326	p=620	_
	Urban/Suburban	Curriculum Director	35	37	14	47	79	_	212
	Urban/Suburban	*MDR Population	p=54	p=60	p=16	p=91	p=431	_
	Rural	Testing/Assessment Director	53	21	23	67	74	_	238
	Rural	*MDR Population	p=118	p=27	p=30	p=222	p=318	_
	Urban/Suburban	Testing/Assessment Director	17	14	5	22	58	_	116
	Urban/Suburban	*MDR Population	p=20	p=16	p=5	p=28	p=145	_
	Rural	ELL/Bilingual Director	36	12	23	48	73	_	192
	Rural	*MDR Population	p=56	p=13	p=30	p=94	p=306	_
	Urban/Suburban	ELL/Bilingual Director	15	9	6	19	56	_	105
	Urban/Suburban	*MDR Population	p=17	p=10	p=6	p=23	p=133	_
	Rural	Principal/Asst. Principal	88	88	83	89	94	_	442
	Rural	*MDR Population	p=992	p=1,042	p=592	p=1,175	p=3,558	_
	Urban/Suburban	Principal/Asst. Principal	86	91	80	89	95	_	441
	Urban/Suburban	*MDR Population	p=790	p=1,681	p=487	p=1,116	p=12,541	_
	Rural	Teachers (all grades, all subject areas, all pre-school, elementary, middle, high)	96	96	95	96	96	_	479
	Rural	*MDR Population	p=20,988	p=19,985	p=9,620	p=23,216	p=73,760	_
	Urban/Suburban	Teachers (all grades, all subject areas, all pre-school, elementary, middle, high)	95	96	95	96	96	_	478
	Urban/Suburban	*MDR Population	p=15,212	p=35,462	p=9,355	p=22,692	p=260,189	_
	Rural/Urban/Suburban	Librarian	87	89	82	88	95	_	441
	Rural/Urban/Suburban	*MDR Population	p=913	p=1,131	p=535	p=1,030	p=6,242	_
Higher Education	Rural/Urban/Suburban	Higher Education 2 Year Colleges (College Department Chair, Dean, College Admissions Director, College Student Affairs Director)	81	82	79	84	93	_	419
	Rural/Urban/Suburban	*MDR Population	p=503	p=568	p=426	p=645	p=3,291	_
	Rural/Urban/Suburban	Higher Education 4 Year and Graduate Colleges (College Department Chair, Undergraduate/ Graduate Dean, College Admissions Director, College Student Affairs Director)	89	93	88	92	95	_	457
	Rural/Urban/Suburban	*MDR Population	p=1,291	p=3,318	p=1,019	p=2,227	p=10,041	_
Pre K-12 and Higher Education	Rural	Parents						1,125
	Urban/Suburban	Parents						1,125
	Ethnicity - Caucasian	Parents	75	75	75	75	75	375
	Ethnicity- Black/African-American	Parents	75	75	75	75	75	375
	Ethnicity- Asian/Pacific Islander	Parents	75	75	75	75	75	375
	Ethnicity- Hispanic	Parents	75	75	75	75	75	375
	Ethnicity- Native American	Parents	75	75	75	75	75	375
	Ethnicity- Multiracial	Parents	75	75	75	75	75	375
	Education Level - Less than High School	Parents						1,125
	Education Level - High School or more	Parents						1,125
	Total Parents	Parents							2,250
	Business Leaders in Rural/Urban/Suburban	Business Leaders	100	100	100	100	100	_	500
Total Surveys Completed									8,052

14. Regarding the pilot test, please explain how feedback information will be used to improve the full collection. Is a follow-up planned? If the only measure is “time to complete,” this could be misleading in terms of determining an understanding of the clarity of questions.

In the pilot test, we will direct our sample deployment partners (such as MDR) to send out survey invitations to less than 1% of our total sample base. The pilot test will enable our sample deployment partners to :

1. Test for any unforeseen technical issues.

2. Evaluate if the question format allows data to be easily collected in the desired manner (open versus close ended questions).

We will review the data collected from the pilot test to :

1. Review if respondents are able to complete all the survey questions within the set time frame

2. Identify are any questions where respondents are having any difficulty in understanding or interpreting the question language.

After the pilot test, if we find that there are any survey/technical issues that need to be revised, we will make the required revisions and run another pilot test to ensure that the actual data collection will go as planned. A full scale deployment will then be done and invitations will be sent to the entire sample base.

15. On the survey the race/ ethnicity question does not meet OMB standards. Please put this into the two question format and direct applicants to “select one or more.” You can refer to the Department’s proposed guidance for specific language.

The following 2 question format will be followed:

QA. Please select your ethnicity: (Select one)

1. Hispanic origin

2. Not of Hispanic origin

QB. Please select your race: (Select one or more)

1. American Indian or Alaskan Native

2. Asian or Pacific Islander

3. Black

4. White

15