Rationale for App-based Data Collection

ATTACHMENT 4C: OVERVIEW, RATIONALE, DATA SECURITY, AND RTI’S SIMILAR EXPERIENCE FOR THE POSITEV SMARTPHONE GEOLOCATION APP AND APP-BASED QUESTIONNAIRES

Overview of the POSITEv App

A geolocation app has been developed specifically for the Point of Sale Intervention for Tobacco Evaluation (POSITEv) to provide a real-time, low-burden measure of campaign exposure. The campaign media team will provide the addresses of all stores that carry the campaign materials in the treatment counties and the addresses for similar stores (convenience stores that sell tobacco products) in the control counties. The app vendor will use this information to create a virtual fence (geofence) around the stores. Mobile phones use Global Positioning Systems (GPS) and wireless networks to capture a phone’s longitude and latitude (Donaire-Gonzalez et al, 2016). Many smartphone apps (such as Google maps) rely on capturing the phone’s geolocation (for example, to provide directions). Geofencing uses the latitude and longitude of specific locations to create virtual fences around areas of interest. When the latitude and longitude of the phone overlaps the geofenced area of interest, the app detects that the person has entered or exited this area. Using this approach, the POSITEv app will register a date and time stamp and the RTI-generated convenience store id number whenever the user enters or exits one of the selected stores. Instead of sending the user a questionnaire or ad in response to their store visit, as marketing apps do, the app will simply log the participant’s visit to this location.

Analysts from the RTI project team will then compile data on the number of visits to each of the stores for each participant, as well as how much time they spent in each store on each occasion. Next, we will use this information to calculate total number of visits to stores that contain the ads (or comparable stores in the control condition). We will also calculate the sum of the number of minutes spent in all stores. Number of visits per store, total visits, and total time spent in stores will be calculated at specific time intervals to match the store visit data to self-reported data from the Waves 2-4 evaluation questionnaires and the three brief app-based questionnaires. We will examine the relationship between the store visit variables created from the app data and self-reported campaign exposure from the evaluation questionnaires and brief questionnaires. We will also examine the relationship between the store visit variables and our outcome variables of interest, which include intention to quit, quit attempts and tobacco use.

The app will also prompt the participant to complete a brief questionnaire approximately three times over 18 months (approximately every 6 months). The primary purpose of these questionnaires is to ensure that the app is still working and to keep participants aware that the app is still collecting information from their phone. In addition, these questionnaires will provide additional longitudinal data points for several of the variables measured in the Waves 1, 2, 3, and 4 outcome evaluation questionnaires. The app-based questionnaire will ask about tobacco use, intention to quit, recent tobacco purchases, cravings to smoke, and aspects of the point of sale environment that the participant noticed the last time he or she purchased tobacco products.

Study participants who do not have a smartphone or who choose not to download the app will still be able to participate in every other part of the evaluation. We anticipate that approximately 2,308 participants who complete the Wave 1 questionnaire will agree to download the optional smartphone app and use it (assuming constant 54% participation of Wave 1 completers over the life of the study). This estimate is based on the estimate that 77% of Americans had smartphones in 2017 (Smith, 2017) and the assumption that 70% of those with cell phones will download the app. The three app-based questionnaires are scheduled to occur over 18 months (approximately every 6 months). These questionnaires will fall between the four primary data collections (Waves 1-4). Each time they complete a questionnaire, RTI project staff will send the participants a $5 electronic gift card, for a total possible value of $15 throughout the course of the study. After the study has ended, the app will be deactivated, and all data obtained from participants will be scrubbed from the app vendor’s data storage systems.

Rationale for the POSITEv app

• Valid measure of campaign exposure

  • Accurately capturing campaign exposure is a central feature of campaign evaluations. A custom-developed, smartphone-based data collection app offers the opportunity to supplement the self-reported survey data with passively collected data on frequency and duration of exposure to the campaign. Campaign exposure data will be captured in real time, continuously through the duration of the campaign, an accomplishment that cannot be achieved through active survey data collection.

  • This app-based component of the evaluation will allow us to answer several key questions about campaign exposure (Exhibit 4C-1). Passive measures of exposure to the campaign collected by the app will determine whether there is any contamination of control participants (i.e., whether they enter any retail spaces that contain the campaign ads). The app-based portion of the study will also inform the utility of using smartphone technology to passively measure exposure to a print-based media campaign.

Exhibit ‎4C-1. Campaign Key Evaluation Questions, Geo-tracking and App-based Questionnaires

Campaign Exposure

• What proportion of 25- to 54-year-old current smokers in the treatment counties are exposed to the campaign ads?

• What is the frequency and duration of exposure to the ads?

• Were any of the participants in the control counties exposed to the advertisements (has contamination occurred?

• How does campaign exposure recorded by the app compare to participants’ self-reported exposure to the app?

• Methods used to objectively measure campaign exposure in previous evaluations are insufficient for this evaluation

  • For most media campaign evaluations, we are able to capture objective measures of media exposure through Gross Rating Points (GRPs). GRPs measure the number of people who “tune in” to radio and TV broadcasts. When we are able to measure GRPs in a certain geographic area, we can match this information to the participant’s place of residence and calculate an objective (or passive) measure of campaign exposure.

  • However, it is not possible to use GRPs for the Point of Sale Intervention for Tobacco Evaluation (POSITEv) because POSITEv is a convenience store-based campaign. Print (or “static”) ads are posted in convenience stores around the United States.

  • Objective measures of media presence do not exist for convenience stores. Similarly, sales data is not available by store, and therefore it is impossible to calculate “traffic” to specific stores based on sales data.

  • In addition, because the ads are posted in convenience stores, which are visually complex, and the ads are static, we anticipate that self-reported recall will be less accurate for POSITEv compared to previous campaigns. In previous evaluations, we have been able to show participants video clips that last several seconds to assess exposure. We are more likely to get an accurate measure of campaign exposure from this type of stimuli than from showing participants a print ad.

• The app provides a low-burden way to capture campaign exposure

  • The passive exposure data captured by the app will not depend on respondent recall, providing for superior data quality.

  • Little to no action is required from the respondent to capture and transmit the data, reducing responded burden (relative to surveys) and perhaps also reducing the likelihood that participants will self-consciously alter their behavior due as they might with active data collection.

• Most participants already have apps installed on their phone that track location

  • Mobile phones use Global Positioning Systems (GPS) and wireless networks to capture a phone’s longitude and latitude (Donaire-Gonzalez et al, 2016).

  • Many smartphone apps (such as Google maps) rely on capturing the phone’s geolocation (for example, to provide directions).

  • Geofencing uses the latitude and longitude of specific locations to create virtual fences around areas of interest. When the latitude and longitude of the phone overlaps the geofenced area of interest, the app detects that the person has entered or exited this area.

  • Marketing agencies and retail stores have then used such apps to send users ads or coupons relevant to the specific location.

• Geofencing has been successfully used for public health research in the past

  • Using a combination of qualitative and quantitative methods, Naughton et al (2016) found that using geolocation for smoking research was reliable, feasible, and sufficiently accurate to justify its use. Naughton et al (2016) tested an app among participants interested in quitting smoking. The app used geolocation data to trigger the delivery of supportive messages to individuals attempting to quit smoking. Although 20% of participants stopped participating before the end of the study, geolocation data was accurate, participants found the messages helpful, and participants unanimously reported no privacy concerns about using the app. They also noticed no appreciable impact of the app on battery life.

  • In 2013, Evenson and Furberg (2016), the latter of whom works at RTI International, developed a smartphone app called “Moves” to track physical activity. The app uses geolocation technology (GPS), Wifi, cellphone towers, and the phone’s accelerometer (a measure of movement and speed at which the phone is traveling) to create a daily diary of physical activity and places visited throughout the day. The app has been purchased by Facebook.

  • Donaire-Gonzalez et al (2016) compared the accuracy of an app called CalFit that tracked location and physical activity to data obtained from a wearable GPS tracker among 162 participants. They found that the app produced accurate estimates of travel and activity. Kirchner et al (2013) gathered continuous location data from 475 participants trying to quit smoking cigarettes to determine how frequently they had visited tobacco retailers over the course of a month. They used the participant’s phone to prompt participants to record real-time cravings to smoke three to four times per day; 79% of which participants responded to the prompts. The researchers found that exposure to tobacco retail outlets was positively associated with lapses in cessation when cravings to smoke were low. Kirchner et al (2014) also conducted a longitudinal study that tracked geolocation data of 550 participants over 8 months. The researchers mapped participant location and then overlaid this data onto a map of convenience stores and similar retailers. They found that this was a viable method of mapping retailer density for future public health research.

• The app-based brief questionnaires will provide additional data for the evaluation

  • The app-based questionnaire will ask about tobacco use, intention to quit, recent tobacco purchases, cravings to smoke, and aspects of the point of sale environment that the participant noticed the last time he or she purchased tobacco products (Exhibit 4C-2).

Exhibit 4C-2. Questionnaire Items by Type and Intended in Analysis, smartphone App-based Media Questionnaire

Type of Item	Questionnaire Items	Intended Use in Analysis of Media Tracking Data
Tobacco Use	Not including today, how many cigarettes did you smoke on the most recent day you smoked? A pack usually has 20 cigarettes in it.	Outcome variable
Campaign Awareness	In the past 3 months, how frequently have you seen or heard the following slogan or theme? Every Try Counts.	Outcome variable
Intention to Quit	Do you plan to quit smoking cigarettes for good… (in the next 7 days, 30 days, 6 months, next year, more than a year, not planning to quit, or already quit)	Outcome variable
Tobacco Purchasing Behaviors	On how many of the past 7 days did you go to a store that sells cigarettes? On how many of the past 7 days did you purchase cigarettes in a store?	Control variable
Quit Attempts	In the past 30 days, how many times have you intentionally quit smoking for at least 24 hours?	Outcome variable
Ad Exposure	In the past 7 days, when you went to a store that sold cigarettes, did you see any of the following advertisements? Select all that apply. (Anti-smoking advertisement, tobacco display advertisement, weight-loss advertisement, food advertisement, lottery advertisement).	Control variable

• Extensive privacy and security protocols have been put in place

  • The app vendor has experience working with research organizations that require confidentiality for participant responses.

• The app is completely optional

  • Study participants who do not have a smartphone or who choose not to download the app will still be able to participate in every other part of the evaluation.

• After the study has ended, the app will be deactivated, and all data obtained from participants will be scrubbed from the app vendor’s data storage systems

Privacy and Data Security for the Point of Sale Intervention for Tobacco Evaluation (POSITEv) App

• The app developer will not have access to participants’ identifying information.

• The app will not log or store any identifying information about the participant.

• The app will log time, date, and store location of visits to convenience stores and responses to a brief questionnaire administered every 6 months.

• Participants will log into the app with an RTI-assigned case identification number. All data logged by the app will contain this number only, and no other identifying information.

• The subcontractor, Question Pro, who has developed the app, underwent a data security and privacy review with RTI’s Privacy Officer and has been approved by RTI’s Cloud Data Storage committee.

• Data obtained from the app will be stored on a cloud server that meets federal requirements for data security and is managed by the app developer. Only the authorized app developer staff will have access to this data in the cloud.

• Data obtained from the app will be transferred from the app developer’s cloud server to RTI’s network server by a nightly process that executes at QuestionPro via the Secure File Transfer Protocol (SFTP), which encrypts the data in transit. Access to this folder is limited to QuestionPro and select RTI project members.

• Following receipt from the SFTP, app data will be stored only on RTI password protected, secured servers separately from PII.

• Participants who participate in the optional app-based portion of the study will sign a separate consent form that outlines the risks and benefits of participation.

• RTI will e-mail the incentives ($5 electronic gift card) for completion of the brief app-based questionnaires to participants so that RTI, not the app vendor, will have access to participants’ e-mail addresses. When a questionnaire is completed, the app vendor will notify RTI of the participant’s case identification number. Then, RTI will use that identification number to locate the participant’s e-mail address and e-mail the incentive to the participant.

• Only RTI will have access to the file that links the case ID to participants’ identifying information, and RTI project staff will only have access to this file as necessary.

• We will collect contact information for respondents, including their mobile phone number and/or e-mail address if they consent to receive reminders to complete the app-based questionnaire via text and/or e-mail. Phone numbers will be stored on RTI’s network server and only accessible to authorized staff. Once securely transmitted from the field to RTI, this information will be stored separately from all other data collected as part of the evaluation.

• Data obtained by the app will be encrypted while at rest and while in transit to the QuestionPro (app vendor) cloud and the RTI SFTP folder.

• The app vendor will delete the data after RTI has confirmed that they have received their copy of the data and before the end of their subcontract with RTI.

References

Donaire-Gonzalez D, Valentin A, de Nazxelle A, Ambros A, Carrasco-Turigas G, Seto E, Jerrett M, Nieuwenhuijsen MJ (2016). Benefits of mobile phone technology for personal environmental monitoring. JMIR Mhealth and Uhealth, 4(4): e126.

Evenson KR, Furberg RD (2016). Moves app: a digital diary to track physical activity and location. British Journal of Sports Medicine, Published online May 3, 2016.

Kirchner TR, Cantrell J, Anesetti-Rothermel A, Ganz O, Vallone DM, Abrams DB (2013). Geospatial exposure to point-of-sale tobacco: Real-time cravings and smoking-cessation outcomes. American Journal of Preventive Medicine, 45(4): 379-385.

Kirchner TR, Gao H, Anesetti-Rothermel A, Carlos H, House B (2014). Longitudinal human mobility and real-time access to a national density surface of retail outlets. The 3^rd International Workshop on Urban Computing, New York, NY.

Naughton F, Hopewell S, Lathia N, Schalbroeck R, Brown C, Mascolo C, McEwen A, Sutton S (2016). A context-sensing mobile phone App (Q Sense) for smoking cessation: a mixed-methods study. JMIR Mhealth and Uhealth, 4(3): e106

Smith, A (2017) Record shares of Americans now own smartphones, have home broadband. Pew Research. http://www.pewresearch.org/fact-tank/2017/01/12/evolution-of-technology/. January 12, 2017. Accessed August 10, 2017.