Appendix B NAEP System Clearance 2014-2016 External Consultants

National Center for Education Statistics

National Assessment of Educational Progress

Appendix B

External Consultants

Request for System Clearance for

NAEP Assessments for 2014-2016

OMB# 1850-0790 v.36

Table of Contents

A. JUSTIFICATION 1

1a. Circumstances making the collection of information necessary 1

1b. Overview of NAEP assessments 2

1c. Overview of 2014–2016 NAEP assessments 5

1d. Rationale for OMB System Clearance 7

2. How, by whom, and for what purpose the data will be used 9

3. Use of techniques to reduce burden 10

4. Efforts to identify duplication 12

5. Burden on small businesses or other small entities 12

6. Consequences of collecting information less frequently 12

7. Consistency with 5 CFR 1320.5 13

8. Consultations outside the agency 13

9. Payments or gifts to respondents 14

10. Assurance of confidentiality 15

11. Sensitive questions 18

12. Estimation of respondent reporting burden (2014–2016) 18

13. Cost to respondents 24

14. Estimates of cost to the federal government 25

15. Reasons for changes in burden (from last System Clearance submittal) 25

16. Time schedule for data collection 26

17. Approval for not displaying OMB approval expiration date 26

18. Exceptions to Certification Statement 26

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 1

1. Potential respondent universe 1

2. Procedures for collection of information 1

3. Methods to maximize response rates and deal with issues of nonresponse 3

4. Tests of procedures or methods to be undertaken 4

5. Consultants on statistical aspects of the design 4

Appendix A Statute Authorizing NAEP A-1

Appendix B External Consultants

NAEP Design and Analysis Committee B-1

NAEP Validity Studies Panel B-2

NAEP Quality Assurance Technical Panel B-3

NAEP Socio-Economic Status Panel B-4

NAEP National Indian Education Study Technical Review Panel B-5

NAEP Civics Standing Committee B-6

NAEP Economics Standing Committee B-7

NAEP Geography Standing Committee B-8

NAEP Mathematics Standing Committee B-9

NAEP Reading Standing Committee B-10

NAEP Science Standing Committee B-11

NAEP Technology and Engineering Literacy Standing Committee B-12

NAEP U.S. History Standing Committee B-13

NAEP Writing Standing Committee B-14

Appendix C Example of Sample Design Document (2013 Assessment) C-1

NAEP Design and Analysis Committee

Name Affiliation

Betsy Becker Florida State University

Peter Behuniak University of Connecticut

Johnny Blair Abt SRBI Associates, Bethesda, MD

Lloyd Bond Carnegie Foundation, Washington, DC

Kadriye Ercikan University of British Columbia

Huynh Huynh University of South Carolina

Brian Junker Carnegie Mellon University, Pittsburgh, PA

Kenneth Koedinger Carnegie Mellon University, Pittsburgh, PA

David Lohman University of Iowa

Ingram Olkin Stanford University, Stanford, CA

Sophia Rabe-Hesketh University of California, Berkeley

Lynne Stokes Southern Methodist University, University Park, TX

NAEP Validity Studies Panel

Name Affiliation

Peter Behuniak University of Connecticut

George Bohrnstedt American Institutes for Research, Washington, DC

Jim Chromy Research Triangle Institute, Research Triangle Park, NC

Phil Daro University of California/SERP/Pearson

Lizanne DeStefano University of Illinois

Richard Duran University of California

David Grissmer University of Virginia

Larry Hedges Northwestern University, Evanston, IL

Gerunda Hughes Howard University, Washington, DC

Robert Linn University of Colorado at Boulder

Ina Mullis Boston College

Scott Norton Council of Chief State School Officers, Washington, DC

Gary Phillips American Institutes for Research, Washington, DC

Lorrie Shepard University of Colorado at Boulder

David Thissen The University of North Carolina at Chapel Hill

Karen Wixson The University of North Carolina at Greensboro

NAEP Quality Assurance Technical Panel

Name Affiliation

Jamal Abedi University of California, Davis

Chuck Cowan Analytic Focus LLC, San Antonio, TX

Kadriye Ercikan University of British Columbia

Gail Goldberg Gail Goldberg Consulting, Ellicott City, MD

Brian Gong National Center for the Improvement of Educational

Assessment, Dover, NH

Mark Reckase Michigan State University

Bill Schafer The University of Maryland

Richard Wolfe University of Toronto

Rich Hill National Center for the Improvement of Educational

Assessment, Dover, NH

NAEP Socio-Economic Status Panel

Name Affiliation

Chris Chapman National Center for Education Statistics, Washington, DC

Margaret Beale Spencer University of Chicago

Stephen L. Morgan Cornell University, Ithaca, NY

Bob Hauser University of Wisconsin–Madison

Robert Kominski U.S. Census Bureau, Washington, DC

Samuel R. Lucas University of California, Berkeley

Chuck Cowan Analytic Focus, Alexandria, VA

Hank Levin Columbia University, New York City, NY

NAEP National Indian Education Study Technical Review Panel

Name Affiliation

Henry Braun Boston College

Bryan Brayboy Arizona State University

Robert Cook Crazy Horse Memorial, Crazy Horse, SD

Steve Culpepper University of Colorado, Denver

Susan Faircloth The Pennsylvania State University

Valeria Littlecreek Oklahoma State Department of Education

Christopher Lohse Council of Chief State School Officers, Washington, DC

Larry Ludlow Boston College

Debora Norris Arizona Department of Education

Ted Wright Haskell Indian Nations University

Tarajean Yazzi-Mintz Indiana University

NAEP Civics Standing Committee

Name Affiliation

Patricia Avery University of Minnesota

Steve Camicia Utah State University

Michael Fischer Encino, CA

Daphne Greene Doe Elementary School, Mountain City, TN

Lewis Huffman South Carolina Department of Education

Peter Levine Tufts University, Medford/Somerville, MA

Kimberly Eggborn Howard County Public Schools, Ellicott City, MD

Ayo Magwood Cesar Chavez Public Charter School, Washington, DC

Rick Miller Princeton Regional Schools, Princeton, NJ

Richard Ochoa Alta High School, Sandy, UT

Clarissa Peterson DePauw University, Greencastle, IN

Joseph Stewart, Jr. Clemson University, Clemson, SC

Marcie Taylor-Thoma Maryland State Department of Education

Kim Tetzlaff Banyan Elementary School, Newbury Park, CA

Jackie Viana Hialeah Gardens Middle School, Miami, FL

NAEP Economics Standing Committee

Name Affiliation

Kris Bertelsen Little Rock Branch-Federal Reserve Bank of St. Louis,

Little Rock, AR

Stephen Buckles Vanderbilt University, Nashville, TN

Steven L. Cobb University of North Texas

Jaime Festa-Daigle Lake Havasu High School, Lake Havasu City, AZ

Julie Heath University of Memphis

Richard MacDonald St. Cloud State University, St. Cloud, MN

Andrea Morgan Oregon Department of Education

Kevin Smith Renaissance High School, Detroit, MI

William Walstad University of Nebraska–Lincoln

NAEP Geography Standing Committee

Name Affiliation

Sarah Bednarz Texas A&M University

Osa Brand National Council for Geographic Education, Washington, DC

Karen Thomas-Brown The University of Michigan–Dearborn

Charlie Fitzpatrick ESRI Schools, Arlington, VA

Donna Fontaine Westside Elementary School, Smithfield, VA

Paul Gray Russellville High School, Russellville, AR

Harry Hanna Frederick County Public School, Walkersville, MD

Briavel Holcomb Rutgers University, New Brunswick, NJ

Niem Huynh Association of American Geographers, Washington, DC

Lauren Mitterman Gibraltar Middle School, Fish Creek, WI

David Palmer Eaglecrest High School, Centennial, CO

Harris Payne Omaha Public Schools, Omaha, NE

Joe Stoltman Western Michigan University

Kelly Swanson Johnson Senior High, St. Paul, MN

Fred Walk Illinois State University

Sonja Yow Eastern Kentucky University

NAEP Mathematics Standing Committee

Name Affiliation

Sareeta Carter Benjamin Banneker Academic High School, Washington, DC

Herb Clemens Ohio State University

Marcia Cole District of Columbia Public Schools, Washington, DC

Carl Cowen Indiana University–Purdue University

Bradford R. Findell Ohio Department of Education

Diana Rivera Garcia Departamento de Educación Puerto Rico

Anne Gonzales South Gate Middle School, South Gate, CA

Julie Guthrie Texas Education Agency

Kathleen Heid The Pennsylvania State University

Russ Keglovits Nevada Department of Education

Judy Kinley Morris Elementary School, Des Moines, IA

Carole Phillip Alice Deal Middle School, Washington, DC

Penny Roberts Greenville Elementary School, Greenville, KY

Daren Starnes The Lawrenceville School, Lawrenceville, NJ

JT Sutcliffe St. Mark’s School of Texas, Dallas, TX

NAEP Reading Standing Committee

Name Affiliation

Peter Afflerbach University of Maryland

Patricia Alexander University of Maryland

Christine Carriere Carl Von Linne School, Chicago, IL

Mary Beth Curtis Lesley University, Cambridge, MA

Patsy Dunton Maine Department of Education

Arzie Galvez Office of Curriculum, Instruction and School Support,

Los Angeles, CA

Pamela Mason Harvard Graduate School of Education, Cambridge, MA

Joe McGonegal Catholic Memorial High School, West Roxbury, MA

Margaret McKeown University of Pittsburgh

Katie Oliver Morgan Park Middle School, Duluth, MN

Timothy Shanahan University of Illinois

Alfred Tatum University of Illinois at Chicago

Lisa White Plymouth Public Schools, Plymouth, MA

Junko Yokota National–Louis University, Skokie, IL

NAEP Science Standing Committee

Name Affiliation

Alicia Cristina Alonzo Michigan State University

Carlos Ayala Sonoma State University, Rohnert Park, CA

Mary Thandi Buthelezi Wheaton College, Wheaton, IL

Susan Craft Hanahan High School, Hanahan, SC

George Deboer American Association for the Advancement of Science,

Washington, DC

Alex Decaria Millersville University, Millersville, PA

Crystal Edwards Lawrence Township Public Schools, Lawrenceville, NJ

Ibari Igwe Shrewd Learning, Elkridge, MD

Michele Lombard Swanson Middle School, Arlington, VA

Ellen Mingione Delaware Department of Education

Brett Moulding Utah Partnership for Effective Science Teaching and Learning,

Ogden, UT

Amy Pearlmutter Littlebrook Elementary School, Princeton, NJ

Steve Semken Arizona State University

Gerald Wheeler National Science Teacher Association, Arlington, VA

David White Berlin, VT

NAEP Technology and Engineering Literacy Standing Committee

Name Affiliation

Keith Barton Indiana University, Bloomington

John Behrens Pearson eLEADS Center, Mishawaka, IN

Brooke Bourdelat-Parks Biological Sciences Curriculum Study (BSCS),

Colorado Springs, CO

Barbara Bratzel Shady Hill School, Cambridge, MA

Lewis Chappelear James Monroe High School, North Hills, CA

Britte Haugan Cheng SRI International, Menlo Park, CA

Meredith Davis North Carolina State University

Chris Dede Harvard Graduate School of Education, Cambridge, MA

Richard Duran University of California, Santa Barbara

Maurice Frazier Oscar Smith High School, Chesapeake, VA

Camilla Gagliolo Arlington Public Schools, Arlington, VA

Christopher Hoadley New York University

Eric Klopfer Massachusetts Institute of Technology

Beth McGrath Stevens Institute of Technology, Hoboken, NJ

Greg Pearson National Academy of Engineering, Washington, DC

John Poggio University of Kansas

Erin Reilly Annenberg School for Communication & Journalism,

Los Angeles, CA

Troy Sadler Missouri University Science Education Center,

Columbia, MO

Kimberly Scott Arizona State University

Teh-Yuan Wan New York State Education Department

NAEP U.S. History Standing Committee

Name Affiliation

Keith Barton Indiana University

Konstantin Dierks Indiana University

Cosby Hunt Bell Multicultural High School, Washington, DC

Linda Levstik University of Kentucky–Dickey Hall

Louisa Moffitt Marist School, Atlanta, GA

Alana Murray Loiederman Middle School, Silver Spring, MD

Caleb Perkins International Education Curriculum, Instruction, & Assessment,

Olympia, WA

Kim Rasmussen Auburn Washburn Unified School District, Topeka, KS

Tina Reynolds Woodfill Elementary School, Fort Thomas, KY

Larry Starr Nebraska Department of Education

Kathleen Anderson Steeves National Council for History Education, College Park, MD

Diana Turk New York University

Rebecca Valbuena Stanton School, Glendora, CA

Bruce Vansledright University of Maryland

NAEP Writing Standing Committee

Name Affiliation

Arthur Applebee University at Albany–SUNY

Diane August Center for Applied Linguistics, Washington, DC

Margretta Browne Montgomery County Public Schools, Silver Spring, MD

Robert Crongeyer Robla School, Sacramento, CA

Elyse Eidman-Aadahl University of California, Berkeley

Nikki Elliot-Schuman Office of Superintendent of Public Instruction, Olympia, WA

Rayna Goldfarb Abraham Lincoln High School, Philadelphia, PA

Carol Jago University of California, Los Angeles

Charles MacArthur University of Delaware

Michael McCloskey Johns Hopkins University

Norma Mota-Altman San Gabriel High School, Alhambra, CA

Sandra Murphy University of California, Davis

Drew Sterner Tamanend Middle School, Warrington, PA

Sherry Swain National Writing Project, Berkeley, CA

Victoria Young Texas Education Agency