B. Collection of Information
Please Note: Statistically significant sample size and statistical data analysis are not relevant to a qualitative formative study. Qualitative research uses small numbers. The relevant qualitative data collection procedures are described in this Section B.
Respondent Universe and Sampling Methods
This qualitative testing project will include a maximum of 52 respondents across six different testing locations relevant to census and the subject matter. Participant recruiting will be conducted to ensure ten test subjects in focus groups and seven test subjects for individual testing in each location who meet the eligibility requirements outlined in the recruitment package. To ensure diversity of respondents, each location will use a participant screener to recruit along U.S. census demographic statistics. The participant recruitment package is included as Attachment B.
We anticipate that approximately 50% of consumers contacted for this study will qualify and be willing to participate. Assuming this percentage, we will need to screen 104 individuals to recruit 52 participants. This estimate is based on the professional research sites’ recommendation per their recruiting process and the contractor’s previous testing experience.
Procedures for the Collection of Information
ONC will contract with an experienced research and communications firm, to conduct the six rounds of iterative, one-on-one, cognitive usability testing. The contractor will conduct 1-2 Focus Groups with 10 participants each (up to 20 total), 1 90-minute, one-on-one, cognitive usability pre-test interview with 4 participants, and 4 90-minute, one-on-one, cognitive usability interviews seven participants at each of four sites, not to exceed 52 interviews.The contractor will use professional testing and recruiting facilities for each round.
The professional testing and recruiting facilities include a testing room equipped with a one-way mirror, observation room, usability software, and audio/videotaping equipment. Trained and experienced contractor staff will serve as moderators, note-takers, and observers during the actual testing sessions.
Before each test, participants will read and sign a statement (Attachment C) confirming that participation is voluntary and stating how privacy is maintained. Each participant will fill out a brief demographic questionnaire to confirm and document the diversity of the 52 respondents (Attachment D).
The contractor will conduct focus groups and usability tests using a combination of paper and electronic formats for more detailed comprehension, usability, and navigation data. During the 90-minute interview, participants will look at either a paper-based or online screen of the improved NPP, provide their unguided impressions through a think-aloud protocol, and then answer focused questions regarding design, content, navigation, comprehension, and comparative decision-making.
The data collection is qualitative, and will be analyzed by using the following sources of information:
Participant discussion of questions posed in the Moderator’s Guide (Attachment A) during the sessions
Notes taken during each session using a note taker’s log of observations
Debriefing notes taken by the note taker, moderator, and observer after each session
Logs from usability software that tracks participants’ onscreen navigation and use of the model for the online sessions
DVDs/videotapes of the sessions (The sources listed above will be the primary sources of information. The videotaping will only be used to clarify responses as needed.)
As a qualitative study, analysts will identify critical incidents from the testing sessions that report
perceptions of the participants about the model’s layout, organization, and design,
whether participants could find key pieces of information,
whether participants could understand the information in the model, and
whether participants could form inferences about that information in order to make decisions about covered entities.
The analysis will be tied to questions that are answered either implicitly by the participants during the sessions or explicitly when directly asked by the test moderators.
Methods to Maximize Response Rates and Deal with Nonresponse
Participants will be contacted by recruiting facilities via phone to schedule each of the 52 respondents at each of the sites. To maximize response rate, test sites send a confirmation letter to respondents in advance of the test date and then also call respondents the day prior to the test order to confirm participation and schedule.
Test of Procedures or Methods to be Undertaken
The proposed qualitative cognitive usability testing methodology of an unstructured think-aloud protocol followed by a series of focused, direct questions about the improved NPP’s content and structure is a qualitative method used for many successful OMB-approved document development and information design cognitive usability testing projects with several government agencies. Quite different than conducting large quantitative, statistical studies or drug trials, this type of qualitative testing facilitates the development of clear, usable, and understandable information about serious and relevant health information based on consumer feedback and documented data. For examples of other qualitative studies to develop clear information, please see:
|
|
|
|
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Statistical analysis is not relevant to a qualitative formative research project of this size.
Joy Pritts, Chief Privacy Officer, Office of the National Coordinator for Health Information Technology (ONC), is directing this qualitative formative research project through a contract. The Project Officer is Scott Weinstein, Program Analyst, ONC.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Department of Health and Human Services |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |