Supporting Statement A for Request for Clearance:
COLLABORATING CENTER FOR QUESTIONNAIRE DESIGN AND EVALUATION RESEARCH
Research and Development Survey, Round 9
OMB No. 0920-0222
Expiration Date: 01/31/2026
Contact Information:
Amanda Titus
Behavioral Scientist, Collaborating Center for Questionnaire Design and Evaluation Research
Division of Research and Methodology
National Center for Health Statistics/CDC
3311 Toledo Road, Room 5451
Hyattsville, MD 20782
301-458-4579
atitus@cdc.gov
November 2, 2023
Table of Contents
A. Justification
A.1. Circumstance Making the Collection of Information Necessary 3
A.2. Purpose and Use of Information Collection 4
A.3. Use of Improved Information Technology and Burden Reduction 4
A.4. Efforts to Identify Duplication and Use of Similar Information 5
A.5. Impact on Small Businesses or Other Small Entities 5
A.6. Consequences of Collecting the Information Less Frequently 5
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 5
A.8. Comments in Response to the Federal Register Notice and
Efforts to Consult Outside the Agency 6
A.9. Explanation of Any Payment or Gift to Respondents 6
A.10. Protection of the Privacy and Confidentiality of Information Provided
by Respondents……………………………………………. 8
A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions 11
A.12. Estimates of Annualized Burden Hours and Costs 12
A.13. Estimates of Other Total Annual Cost Burden to Respondents
or Record Keepers 12
A.14. Annualized Cost to the Federal Government 12
A.15. Explanation for Program Changes or Adjustments 12
A.16. Plans for Tabulation and Publication and Project Time Schedule 12
A.17. Reason(s) Display of OMB Expiration Date in Inappropriate 12
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions 12
LIST OF ATTACHMENTS
Attachment 1: Questionnaire
Attachment 2: Nondisclosure Affidavit
Attachment 3: CIPSEA Information Protection Plan
Attachment 4: Comparison of Topics between RANDS 8 & 9
Supporting Statement A
Collaborating Center for Questionnaire Design and Evaluation Research
The staff of the National Center for Health Statistics’ (NCHS) Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, exp. 01/31/2026), in collaboration with the Division of Research and Methodology (DRM), plans to conduct a methodological survey as part of its Research and Development Survey (RANDS) program. This round of RANDS, RANDS 9, will primarily be used to conduct mixed-method question evaluations. As with most of the previous rounds of RANDS that have been approved in the past, the survey content for this round will be related to health and will generally mirror content found on the National Health Interview Survey (NHIS), the National Survey of Family Growth (NSFG), the Household Pulse Survey, and the Behavioral Risk Factor Surveillance System (BRFSS). The overall goals of RANDS 9 will be to continue to develop NCHS’ statistical and weighting approaches to using commercial survey panel data and to evaluate potential questions for NCHS surveys.
RANDS 9 will continue to use NORC’s AmeriSpeak Panel as the primary sample source, though similar to what was previously approved for RANDS 8 (OMB approval 5/31/2023), it will also include a supplemental opt-in sample that will allow NCHS to broaden its methodological research. Note that unlike the special RANDS during COVID-19 series (OMB No. 0920-1298, expiration: 11/30/2020 and OMB No. 0920-1323, expiration: 8/31/2021), NCHS is not requesting clearance to release prevalence estimates of any variables in these surveys, which will be conducted for methodological purposes related to the underlying generic clearance. As is typical for RANDS projects, these data will be used by NCHS for methodological purposes only, and public data files will be released along with extensive documentation about the limitations of the data1.
We propose to begin fielding RANDS 9 as soon as we receive OMB approval. This survey will collect complete responses from 6,600 AmeriSpeak (6,000 via web, with an additional 600 via phone interview) panelists and 10,000 opt-in (all collected via web) panelists.
A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
As the nation’s principal health statistics agency, the National Center for Health Statistics (NCHS) is responsible for not only producing high-quality statistics on the health and wellbeing of the American public and the state of the country’s health care system, but also for contributing to the development of survey methodologies that will allow the agency to continue producing these health statistics in the future. NCHS uses commercially available, pre-existing survey panels to supplement its ongoing examinations of measurement error and estimation techniques. This methodological survey system—NCHS’ RANDS—has allowed the Center to not only conduct meaningful subgroup analyses of survey response patterns, explore patterns of survey item interpretation and potential reporting errors at a population level, but also to explore how estimates from commercial survey panels might be used in the future to supplement NCHS’ official statistics information collections and survey systems. To date, there have been 8 rounds of RANDS and 3 rounds of RANDS during COVID (previously approved under CCQDER’s generic clearance with OMB No 0902-0222 on 2/19/18; 11/19/18; 2/5/20, 7/30/21; as well as the three rounds RANDS during COVID-19 series under their own OMB Nos. 0920-1298 and 0920-1323), totaling 11 rounds2.
As with previous rounds, DRM RANDS 9 functions a methodological study that will provide a testbed for NCHS’ estimation and question evaluation research. Specifically, it will focus on:
Evaluating potential calibration variables that will allow NCHS to re-calibrate RANDS’ weights to surveys other than just the NHIS.
The response processes behind those calibration variables.
Exploring how statistically sampled and opt-in samples can be meaningfully combined for analytic purposes.
The intersection of measurement and sample error, particularly in small population subgroups and intersectional groups.
Refining processes for integrating web probes with post hoc cognitive interviewing findings for question and questionnaire evaluation purposes.
Additionally, NCHS will continue to use these proposed rounds of RANDS to supplement its ongoing question evaluation work, including evaluations of items collecting information on gender identity, race and ethnicity, discrimination, stigma, intellectual and developmental disability, and intimate partner violence.
2. Purpose and Use of Information Collection
Questionnaire Content
The questions we plan on evaluating and administering in the proposed round of RANDS are included as Attachment 1. In addition to the inclusion of the standard RANDS variables that allow NCHS to calibrate RANDS to the NHIS, this questionnaire covers other topics that will help DRM and NCHS advance its methodological research, including:
Novel Calibration Variables: To this point, DRM has found success calibrating RANDS data to the NHIS using a combination of demographic and chronic condition variables. DRM and NCHS’ Division of Health Interview Statistics have developed an expanded list of potential calibration variables. Specific items include questions about language use at home, volunteerism, health care access, internet use, and functional disability. NCHS plans on using the proposed cognitive interviews and RANDS to further evaluate these variables’ response patterns and how the inclusion of these variables in its calibration approach affects estimation.
Discrimination: NCHS plans to use RANDS 9 to continue its evaluation of measures of discrimination. Specifically, RANDS 9 will include both the Everyday Discrimination Scale and the Heightened Vigilance Scale and a set of accompanying probe questions. These data, combined with data from previously conducted cognitive interviews and rounds of RANDS, will allow NCHS to better understand the measurement properties of these scales, which are already included on the NHIS.
Diabetes and Obesity Stigma: Staff with CDC’s National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) approached NCHS with a set of questions regarding diabetes stigma for inclusion on a round of NCHS’ Rapid Survey System (RSS). After reviewing the questions and further discussions with NCCDPHP staff, the decision was made to continue developing and evaluating questions on this topic before administering them on RSS. To this end, a set of diabetes stigma questions were adapted from an existing set of questions designed to measure obesity stigma. Both question sets, along with a follow-up probe, are being administered on RANDS 9 in order to begin understanding the response processes behind answering items related to stigma generally, and about diabetes specifically. Specifically, half of the sample will receive the obesity stigma set, while the other half will receive the diabetes set. CCQDER plans on comparing differences in the outcomes of these sets to understand whether or not respondents appear to respond differently to questions on specific types of stigma, which will allow NCHS to determine whether or not future qualitative and mixed-method question development research on this topic is justified. The purpose of adding these specific questions onto the RANDS 9 questionnaire is not to test this specific set of questions for inclusion on other federal surveys at this time, but rather to take advantage of an existing stigma question set to propel this research.
Intellectual and Developmental Disability (IDD): DRM plans on RANDS 9 to continue evaluating potential IDD questions, following its previous analysis of cognitive interviewing and RANDS data. On RANDS 9 specifically, a set of questions about learning difficulties will be included and administered alongside a probe designed to gather information about how respondents comprehend the item about learning to read and write.
Intimate Partner Violence: DRM plans to continue its collaborative work with the National Center for Injury Prevention and Control (NCIPC)’s National Intimate Partner and Sexual Violence Survey (NISVS) program. RANDS 9 will be used to experimentally compare two approaches to asking about physical violence—one approach uses a longer set of eight items, whereas the other approach combines those eight items into two items. The analysis of this experiment will focus on whether these approaches produce different estimates, and whether the lead to different levels of objective (i.e., response latency) or subjective burden.
Race and Ethnicity: NCHS is participating in the cross-agency work to test and refine the newly proposed race and ethnicity questions to be used on federal information collections under Statistical Policy Directive No. 15 (SPD 15)3. In collaboration with the Interagency Technical Working Group on Race and Ethnicity Standards (ITWG), CCQDER conducted cognitive interviews on the recently released, proposed question. The analysis of these interviews revealed potential false negative responses among Middle Eastern and North African (MENA) respondents. One hypothesis regarding these response errors that emerged from the cognitive interviews was that respondents did not know MENA was an answer category, and since many people who are from that region have been previously instructed to answer “White,” they simply chose that response (as it is typically the first answer category presented). After discussing these findings with ITWG leadership, the working group and DRM collectively decided to embed an answer category order experiment in RANDS 9. One half of the respondents will receive the race and ethnicity question with the usual order of answer categories (i.e., “White” will come first); the other half will receive the answer categories in alphabetical order. DRM plans to evaluate whether and how the prevalence of the various race and ethnicity groups vary across the experimental conditions.
Gender Identity: NCHS is continuing its work on developing novel approaches and evaluating existing gender identity items. In light of CCQDER’s recent work testing the new one-step gender item on the U.S. State Department’s passport form4, CCQDER plans on exploring how response to a one-step approach in a household survey context differs from that of a two-step approach where respondents are asked to provide their sex at birth/sex on birth certificate. This also builds on work previously approved and carried out on RANDS 5 and in previous cognitive interviewing projects5.
3. Use of Improved Information Technology and Burden Reduction
Respondents will complete the survey using NORC’s web interface or telephone. Respondents will submit their responses electronically; and these responses will be tallied electronically and put into a database by survey software. Electronic collection will minimize the burden on survey respondents and facilitate the most rapid processing of survey results.
4. Efforts to Identify Duplication and Use of Similar Information
The CCQDER at NCHS is the only government facility that currently conducts testing and development of NCHS or other CDC questionnaires. Similar facilities at the Bureau of the Census and the Bureau of Labor Statistics bear the responsibility for testing survey questionnaires associated with their own agencies.
In order to identify duplication across federal agencies, CCQDER hosts a publicly accessible online searchable database, Q-Bank, that contains all CCQDER evaluation reports, including reports derived from RANDS data. CCQDER encourages all agencies to submit their evaluation reports so that it is possible to track the work done across agencies as well as to build in existing knowledge.
5. Impact on Small Businesses and Other Small Entities
This data collection does not impact small business or other small entities.
6. Consequences of Collecting the Information Less Frequently
This is a one-time data collection activity.
7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5
This request fully complies with the regulation 5 CFR 1320.5.
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agencies
A Federal Register notice for the CCQDER Generic data collection was published on September 30, 2022 (Vol. 87, No. 189, p. 59425). No additional comment period is required for GENICs submitted under this generic.
Consultants:
The following individuals have been consulted within the past year on survey methodology and/or on a specific project:
Katherine Irimata
Associate Service Fellow
NCHS
301-458-4075
John Dombrowski
Client Services Manger
AmeriSpeak
312-230-9059
9. Explanation of Any Payment or Gift to Respondents
NCHS will not provide an incentive to panelists participating in RANDS, though NORC does provide non-cash incentives in the form of “AmeriPoints” that panelists can accumulate and exchange for items such as gift cards.
10. Protection of the Privacy and Confidentiality of Information Provided by Respondents
The CCQDER continues to collect, on a confidential basis, data needed in order to conduct CCQDER studies. The process of informing respondents of the procedures used to keep information confidential begins with the telephone screener and will carry through to the interviewer and all communications with potential respondents. Materials will include all elements of informed consent, including the purpose of the data collection, the voluntary nature of the study, audio or video recording of the interview, and the effect upon the respondent for terminating the interview at any time.
Confidentiality provided to respondents is assured by adherence to Section 308(d) of the Public Health Service Act (42 U.S.C. 242m) which states:
“No information, if an establishment or person supplying the information or described in it is identifiable, obtained in the course of activities undertaken or supported under section...306 (NCHS legislation),...may be used for any purpose other than the purpose for which it was supplied unless such establishment or person has consented (as determined under regulations of the Secretary) to its use for such other purpose and (1) in the case of information obtained in the course of health statistical or epidemiological activities under section...306, such information may not be published or released in other form if the particular establishment or person supplying the information or described in it is identifiable unless such establishment or person has consented (as determined under regulations of the Secretary) to its publication or release in other form,...”
In addition, legislation covering confidentiality is provided according to the Confidential Information Protection and Statistical Efficiency Act or CIPSEA (44 U.S.C. 3561-3583), which states:
“Whoever, being an officer, employee, or agent of an agency acquiring information for exclusively statistical purposes, having taken and subscribed the oath of office, or having sworn to observe the limitations imposed by this section, comes into possession of such information by reason of his or her being an officer, employee, or agent and, knowing that the disclosure of the specific information is prohibited under the provisions of this subchapter, willfully discloses the information in any manner to a person or agency not entitled to receive it, shall be guilty of a class E felony and imprisoned for not more than 5 years, or fined not more than $250,000, or both.”
The CIPSEA legislation authorizes the designation of agents (“designated agents” or “agents”) to perform statistical activities on behalf of an agency. These agents function under the supervision of the agency’s employees and are subject to the same provisions of law with regard to confidentiality as an agency’s employees. A Designated Agent Agreement between the agency and the designated agents (e.g. contractors) must be executed before the agents can acquire information for the agency for exclusively statistical purposes under a pledge of confidentiality. This requirement is outlined in an OMB Notice, published in the Federal Register on June 15, 2007, entitled “Implementation Guidance for Title V of the E-Government Act, Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA).”
A Designated Agent Agreement between NCHS and any CCQDER contractor will be executed if any contractors are hired to acquire information for the NCHS for exclusively statistical purposes under a pledge of confidentiality (i.e. complete any of the five types of activities described in this generic clearance request). Additionally, the agents (contractors) will be required to complete NCHS Confidentiality Training (https://www.cdc.gov/nchs/training/confidentiality/training/), submit a certificate of completion, and sign a pledge to maintain confidentiality (Nondisclosure Affidavit; see Attachment 2) prior to completing CCQDER work. If the CCQDER contractor hires subcontractors to complete CCQDER work, the subcontractors must adhere to the same confidentiality and security requirements as CCQDER staff and contractors.
Data in identifiable form are collected for linkage of various CCQDER forms (informed consent documentation and respondent demographics) and audio and video recordings. The CCQDER also uses some identifiable data (name, phone number, email address) to contact previous respondents for CCQDER studies. The ability to match respondents to other data (informed consent documents, respondent demographics, and audio/video recordings) greatly expands the usefulness of the data at a very low cost.
As outlined in the informed consent form, access to personal information is restricted to CCQDER staff who can only access the personal information for statistical, training and research purposes. Additionally, other NCHS staff, designated agents such as CCQDER contractors, or subcontractors may access the personal information for statistical purposes only after signing a Designated Agent Agreement with NCHS. CCQDER staff, designated agents, and staff from collaborating agencies must complete annual NCHS confidentiality training (https://www.cdc.gov/nchs/training/confidentiality/training/), submit a certificate of completion, and sign the NCHS affidavit of nondisclosure (see Attachment 2) prior to being granted access to any personal information.
The collection of information in identifiable form requires strong measures to ensure that private information is not disclosed in a breach of confidentiality. Storage of confidential data is protected through procedures such as an internal QDRL LAN, passwords and restricted access.
Confidentiality of responses and safeguarding of data at NCHS
Following the completion of this round’s field period, NORC will process the survey data and prepare data files. The data files will not include the respondents’ names, addresses, or any other primary personally identifiable information (PII), including any ISP (internet service provider) address data NORC has about the computer from which the respondent replied to the survey. All metadata tying the respondents to their inclusion in the RANDS sample will be eliminated from the NORC servers, including the backups, following final delivery. The data files will be transferred to NCHS via either a secure File Transfer Protocol (FTP) web portal or by loading them directly on an encrypted memory stick.
Following the delivery of the RANDS datasets and the final methodological report, NORC will remove all RANDS data from its servers, including backups. This will include not only the responses to the survey itself, but the metadata associated with the RANDS (including response, non-response, participation, and sampling flags identifying the RANDS sample). NORC has extensive cyber and physical security in place, including a CIPSEA Information Protection Plan approved by the NCHS Confidentiality Officer and the NCHS Information Systems Security Officer, in order to protect both the security of the front-end survey interface and the back-end storage of the survey’s data (Attachment 3). Additionally, all NORC employees working on the RANDS will complete NCHS confidentiality training, sign the NCHS affidavit of nondisclosure (see Attachment 2), and will be NCHS designated agents via the Designated Agent Agreement between NORC and NCHS. The sampling frame information for AmeriSpeak will be protected under Section 308(d) of the Public Health Service Act [42 U.S.C. 242m(d)] and the CIPSEA [44 U.S.C. 3561-3583].
11. Institutional Review Board (IRB) and Justification for Sensitive Questions
NCHS Ethics Review Board (ERB) and CDC Human Research Protection Office (HRPO) approved this data collection on September 22, 2023.
Informed Consent and Voluntary Nature
All Panel participants have been fully screened and a substantial amount of background data have already been collected (e.g., health and well-being, socio-economic and occupational status, media usage, political views, age, gender, race, ethnicity, etc.), which will be attached to the final files delivered by NORC to NCHS, allowing for extensive non-response bias analysis.
As with previous rounds, the RANDS survey itself will begin with an introduction screen (or introduction text for telephone respondents) like what is seen at the beginning of Attachment 1, explaining the general purpose of the survey, and providing the confidentiality and Paperwork Reduction Act language. As signed consent is not possible for internet surveys where the population of respondents is anonymous to NCHS, as in a commercial panel, we will receive a waiver of signed consent from the NCHS ERB (as has been the case in all previous rounds of RANDS). The introduction page will require the respondent to manually click through to the first page of questions (or agree to continue and not hang up for telephone respondents); this action therefore implies consent.
12. Estimates of Annualized Burden hours and costs (Table 1 was OMB approved on January 31, 2023 for CCQDER Generic data collection.)
Table 1: Estimated Annualized Burden Table: (Approved hours under OMB. No. 0920-0222)
Types of Respondents |
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Average hours per response (in hours) |
Total Burden Hours
|
Individuals or households |
RANDS (Methodological Survey) |
49,800 |
1 |
15/60 |
12,450 |
Table 2: Burden table for RANDS 9:
Types of Respondents |
Form Name |
Number of Participants |
Number of Responses/Participants |
Average hours per response |
Response Burden (in hours) |
Individuals or households |
RANDS 9 |
16,600 |
1 |
15/60 |
4,150 |
|
Total |
16,600 |
1 |
15/60 |
4,150 |
13. Estimates of Other Total Annual Cost Burden to Respondents and Record keepers
There is no annual capital or maintenance costs to the respondent resulting from this collection of information.
14. Annualized Costs to the Federal Government
Funding for RANDS is from the Data Modernization Initiative funds. It is expected that collecting and analyzing the data from RANDS 9 will cost to the government $300,000. It consists mainly of the salaries of the CCQDER and contracted staff that will (1) program and collect survey data, and (2) assist in the analysis of the results and recommend changes in questionnaire wording.
15. Explanation for Program Changes or Adjustments
This is a generic IC. This one-time data collection will use 4,150 hours.
16. Plans for Tabulation and Publication and Project Time Schedule
This clearance request is for questionnaire development activities to be conducted prior to survey production and for developmental work that will guide future questionnaire design. Final reports will be written that document how the question performed in the RANDS survey, including question problems as well as the phenomena captured by the survey question, web probes and experiments as applicable. All reports will be placed on Q-Bank for public access. Reports are used to provide necessary information to guide designs for redesigning a question prior to fielding as well as to assist end users when analyzing the survey data. For field tests/pilot interviewing activities, qualitative and quantitative analysis will be performed on samples of observational data from household interviews to determine where additional problems occur. Because NCHS is using state-of-the-art questionnaire development techniques, methodological papers will be written which may include descriptions of response problems, recall strategies used, and quantitative analysis of frequency counts of several classes of problems that are uncovered through the cognitive interview and observation techniques.
17. Reason(s) Display of OMB Expiration Date is Inappropriate
The expiration date will be displayed.
18. Exceptions to Certification for Paperwork Reduction Act Submissions
The certifications are included in this submission.
4 Willson, S., Miller, K. (2022). Cognitive Interview Evaluation of X Gender Marker Definitions for the U.S. Passport Application Form. National Center for Health Statistics - QDRL. Hyattsville, MD. https://wwwn.cdc.gov/qbank/report.aspx?1225
5 Miller, K., Willson, S., Ryan, V.. (2021). An Initial Cognitive Evaluation of a 2-Step Gender Identity Measure. National Center for Health Statistics - QDRL. Hyattsville, MD. https://wwwn.cdc.gov/qbank/report.aspx?1219; Ryan, V. “Examining Measurement Error in a Sexual Identity Question.” AAPOR 77th Annual Conference, Chicago, IL. May 13, 2022.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2025-05-19 |