Download:
pdf |
pdfAppendix A – System Usability Scale (SUS)
Participant #________ Condition________
Please check the box that reflects your immediate response to each statement. Don’t think too long about each
statement. Make sure you respond to every statement. If you don’t know how to respond, simply mark “3.”
Overall, I would rate the user-friendliness of this application as
Worst
Imaginable
Awful
Poor
Fair
Good
Excellent
DISTRIBUTION A. Approved for public release: distribution unlimited.
7
Best
Imaginable
Appendix B – Subjective Usability Questionnaire (SUQ)
Please answer the following questions as completely as possible. Place a circle around the number that best represents
your rating of your experience.
SCREEN
1. Characters on the mobile screen
hard to read
1
2
3
2.
3.
4.
4
5
6
7
8
easy to read
9
N/A
Characters on the mobile screen
vague
1
2
3
4
5
6
7
8
obvious
9
N/A
Organization of information on screen
confusing
1
2
3
4
5
6
7
8
very clear
9
N/A
Sequence of screens
confusing
1
4
5
6
7
8
very clear
9
N/A
5
6
7
8
consistent
9
N/A
Terminology used is related to the task you are doing
never
1
2
3
4
5
6
7
8
always
9
N/A
Location on screen of messages and notifications provided to you by the application
inconsistent
1
2
3
4
5
6
7
8
consistent
9
N/A
Messages that require your input are?
confusing
1
2
3
2
3
TERMINOLOGY AND SYSTEM INFORMATION
5. Use of terms throughout application
inconsistent
1
2
3
4
6.
7.
8.
9.
4
5
Application keeps you informed about what is going on
never
1
2
3
4
5
10. Error messages
unhelpful
1
LEARNING
6
7
8
clear
9
N/A
6
7
8
always
9
N/A
4
5
6
7
8
helpful
9
N/A
4
5
6
7
8
easy
9
N/A
12. Exploring new features by trial and error
difficult
1
2
3
4
5
6
7
8
easy
9
N/A
13. Remembering names and use of commands
difficult
1
2
3
4
5
6
7
8
easy
9
N/A
2
3
11. Learning to operate the application
difficult
1
2
3
DISTRIBUTION A. Approved for public release: distribution unlimited.
8
14. Tasks can be performed in a straight-forward manner
never
1
2
3
4
5
6
7
8
always
9
N/A
15. Help messages on the screen
unhelpful
1
2
3
4
5
6
7
8
helpful
9
N/A
16. Supplemental reference materials
confusing
1
2
3
4
5
6
7
8
clear
9
N/A
APPLICATION CAPABILITIES
17. Correcting your mistakes
difficult
1
2
4
5
6
7
8
easy
9
N/A
3
DISTRIBUTION A. Approved for public release: distribution unlimited.
9
Appendix C – Satisfaction Questionnaire
Please answer the following questions as completely as possible. Place a circle around the number that best represents
your rating of your experience.
OVERALL REACTIONS TO THE APPLICATION
useless
0
1
2
3
4
5
6
7
8
useful
9
N/A
0
1
2
3
4
5
6
7
8
easy
9
N/A
frustrating
0
1
2
3
4
5
6
7
8
satisfying
9
N/A
1
2
3
4
5
6
7
8
flexible
9
N/A
1
2
3
4
5
6
7
8
trustworthy
9
N/A
difficult
rigid
0
untrustworthy
0
DISTRIBUTION A. Approved for public release: distribution unlimited.
10
Appendix D – Sample Knowledge Assessment Questions
1. Based on data from the Consumer Expenditures Survey, a
middle-come family will spend approximately $___ annually
per child.
2. After your child ages out of TRICARE, they may purchase
TRICARE Young Adult up until their ___ birthday.
3. Linda is a Service member who is three months pregnant and
is covered by TRICARE. She has yet to find out the gender of
her baby and therefore has scheduled an ultra sound to do so.
After her ultrasound she received a medical bill that TRICARE
would not cover. Why did she have to pay for the ultrasound?
4. Janet bought a car with an extremely low interest rate.
However, the interest rate slowly increased. What predatory
loan practice did she fall victim to?
5. Carrie can't pay her loan off early or else she gets hit a hefty
fee. Which predatory loan practice is this?
6. Arya is looking to purchase a house in a year, so she is
working on improving her credit score. To effectively monitor
factors affecting her credit score Arya should ____?
a) $5,460
b) $10,320
c) $12,980
d) $15,670
a) 21st
b) 22nd
c) 25th
d) 26th
a) She already used her limit
b) She was no longer covered under TRICARE
c) TRICARE doesn’t cover ultrasounds used only
to determine the gender of the baby
d) TRICARE made a mistake
a) Teaser Interest Rate
b) Balloon Payment
c) Negative Amortization
d) Balloon Teaser Interest Loan
a) Early Bird Penalty
b) Surprise fee
c) Penalty Payment
d) Breadcrumb fee
a) Request a free copy of her credit report from
all 3 agencies
b) Purchase her credit report from an
independent company
c) Focus on saving up money and trusting that
her credit report is accurate
d) Stagger requesting a free copy of her credit
report from each of the three agencies by 4
months.
7. Tom wants to ensure that he has great credit. What can he
do on a regular basis that will have the biggest impact on
increasing his credit score?
a) Pay all of his bills on time
b) Open new lines of credit
c) Review his score with multiple services
d) Open a card as young as possible
8. You’ve been working on raising your credit score for the past
few years, and you periodically check your credit score through
free sites. How will this affect your credit score?
a) Only credit scores you personally check will
lower your credit score
b) Personal inquiries will help bring your credit
score up because it shows lenders that you are
actively monitoring the state of your credit score
c) Both personal inquiries and inquiries from
lenders will help bring your credit score up
d) Personally checking your credit score will not
affect your credit score in any way
DISTRIBUTION A. Approved for public release: distribution unlimited.
11
Appendix F – Informed Consent
TITLE:
Sen$e Summative Evaluation
PROTOCOL NO.:
IRB Protocol not specified yet
SPONSOR:
The Office of Financial Readiness
INVESTIGATOR:
United States
STUDY-RELATED
PHONE NUMBER(S):
Purpose
The purpose of this assessment is to gather end-user feedback regarding a financial readiness mobile
application being developed. We are interested in your perspective and expectations while using the app. We
are also interested to see how well the information can help you learn about various financial topics. The
assessment should take no longer than 1hr to complete.
Procedure
You will be asked to fill out some questionnaires before interacting with the mobile application. You will then
work through some scenarios while using the mobile application. While interacting with the application, we
may ask you to ‘think-aloud’. After you have completed all the scenarios, you will fill out some remaining
questionnaires. Lastly, you will be asked some final questions regarding your interaction.
Risks
The risks associated with the survey, the interview procedures, or the interaction with the application are no
more than one would typically expect to encounter while when using an application on a mobile device. It is
estimated that there is minimal to no risk involved in volunteering in this data collection process.
Potential Benefits
The potential benefits to volunteers include gaining a better understanding of personal financial readiness,
knowing that volunteering and providing responses will better inform product design and implementation, and
contributing to team knowledge about military service members’ financial readiness.
Compensation & Voluntary Participation
Volunteers will not be paid for their time. Additionally, if volunteers wish to stop participation at any time,
they are free to do so, and choosing to not participate in the data collection process is a valid alternative.
Volunteers who choose to not participate or discontinue participation will not result in any loss of benefits to
which they are otherwise entitled.
DISTRIBUTION A. Approved for public release: distribution unlimited.
23
Confidentiality
QIC verifies that the anonymity and confidentiality of individual subject data will be upheld and preserved.
Personal identifiers to study data will not be recorded and there will be no linkages to individual identifying
information. The information gathered will only be used for Sen$e project-related tasking and will be stored
on either a password protected computer with password protected software and/or in locked file drawer inside
an office requiring keycard entry.
Contact
If you have questions, concerns, or complaints, or think the research has hurt you, talk to
CEO, Quantum Improvements Consulting,
,
You can call a subject representative at IntegReview IRB if you have concerns or complaints you would like
to talk about with someone who is not working on the study. If you have a study consent form, please have it
available during the call. IntegReview, https://integreview.com/, (512) 326-3001.
I agree to participate in the summative evaluation being conducted by Quantum Improvements Consulting
investigating the usability and effectiveness of Sen$e, a mobile application. Sen$e is a unique financial support
tool developed to aid Service members and their families in developing optimal financial decision-making
through customized life-long education.
I understand that participation in this summative evaluation is voluntary and I agree to immediately raise any
concerns or areas of discomfort during the session with the evaluation facilitator.
Please sign below to indicate that you have read and you understand the information on this form and that any
questions you might have about the session have been answered.
Date:_________
Please print your name: ____________________________________________________
Please sign your name: ____________________________________________________
Researcher’s print name: ____________________________________________________
Researcher’s signature: ____________________________________________________
Thank you!
We appreciate your participation.
DISTRIBUTION A. Approved for public release: distribution unlimited.
24
Appendix G – Demographics
Participant ID: _________
Demographics Questionnaire
Please provide the following information to the best of your ability. Do not write your name or other
personally identifying information on this page.
1. General Information
a. Age (yrs): _____ b. Gender: ___M ___F
2. Military Experience (if applicable)
a. How many years have you been in the military? __________
b. Which branch of the military are/were you in? ___________
c. Current/Past rank ____________
d. What is/was your MOS? ________________
3. Smartphone Experience
a. What type of phone do you use? Apple Android
Other_____________
b. How
1 long have you2been using a smartphone?
3
Never
Less than 1
1-3 years
year
c. How often do you use your smartphone?
1
2
3
Less than 1
1 hour a day
2 hours a day
hour a day
4
4-6 years
5
7-10 years
6
10 years or
more
4
3 hours a day
5
4 hours a day
6
More than 4
hours a day
d. For each of the following questions, circle the response that best describes how often you use your
smartphone to:
Check email
1
15 mins a
day
Play mobile games
1
N/A
15 mins a
day
N/A
2
30 mins a
day
3
45 mins a
day
4
1 hour a day
5
2 hours a
day
6
More than 2
hours a day
2
30 mins a
day
3
45 mins a
day
4
1 hour a day
5
2 hours a
day
6
More than 2
hours a day
DISTRIBUTION A. Approved for public release: distribution unlimited.
25
Browse the internet
1
N/A
15 mins a
day
Use financial apps
1
N/A
15 mins a
day
Learn new information
1
N/A
15 mins a
day
2
30 mins a
day
3
45 mins a
day
4
1 hour a day
5
2 hours a
day
6
More than 2
hours a day
2
30 mins a
day
3
45 mins a
day
4
1 hour a day
5
2 hours a
day
6
More than 2
hours a day
2
30 mins a
day
3
45 mins a
day
4
1 hour a day
5
2 hours a
day
6
More than 2
hours a day
DISTRIBUTION A. Approved for public release: distribution unlimited.
26
Appendix H – Debrief Questions
1. What is your overall impression of the application?
• Did you think the application was useful?
• Was the interaction the way you expected it to be?
• Did you learn anything? Do you think the way information was presented could support learning the content?
2. What do you like most about the look and feel of the application?
3. What do you like least about the look and feel of the application?
4. Were you able to navigate to the instructed locations? Why?
5. Was the content well organized? Was content in locations where you thought it would be?
6. Was the content terminology easy to understand? Why?
• Were you familiar with the terms already? Were there one’s you didn’t know?
• Any spelling, typo, or grammatical errors?
7. If you were the developer, what is the first thing you would change/fix?
8. What do you feel is missing from the system that would be very helpful/important?
9. Do you have any final comments or questions?
DISTRIBUTION A. Approved for public release: distribution unlimited.
27
| File Type | application/pdf |
| Author | Jennifer Murphy |
| File Modified | 2019-04-03 |
| File Created | 2019-01-09 |