| A | B | C | D | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | AC | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Principle | Scope | Measure | Evaluation | example value | normalized example value | weight (1-10) | example weighted value | potential max | |||||||||||||||||||
2 | for all of the below 1-10 (1 - strongly disagree, 10 - strongly agree) | |||||||||||||||||||||||||||
3 | Data control | patient | The user has control over their data (incuding revoking, sharing, transferring, exporting, avoiding data loss, etc) | The user has control over their data (including revoking, sharing, transferring, exporting, avoiding data loss, etc) | 2 | 2 | 10 | 20 | 100 | |||||||||||||||||||
4 | Data control | all users | The user can control their profile information | user can control profile information | 8 | 8 | 7 | 56 | 70 | |||||||||||||||||||
5 | Data control | patient | Consents and opt-ins/outs are implemented, confirmed and clear for any data sharing | Consents and opt-ins/outs are implemented, confirmed and clear for any data sharing | 7 | 7 | 10 | 70 | 100 | |||||||||||||||||||
6 | Data control | patient | Data privacy and sharing is communicated clearly to the user | data privacy and sharing is communicated to user clearly | 7 | 7 | 10 | 70 | 100 | |||||||||||||||||||
7 | Data control | patient | Data security approach and storage location is communicated clearly to the user | Data security approach and storage location is communicated clearly to the user | 8 | 8 | 5 | 40 | 50 | |||||||||||||||||||
8 | Design | all users | The user experience matches current paradigms of digital interaction | the experience has no obtrusive advertising | 10 | 10 | 3 | 30 | 30 | |||||||||||||||||||
9 | Design | all users | Prompts for additional information are contextual and timely | the experience matches current paradigms of digital interaction | 10 | 10 | 7 | 70 | 70 | |||||||||||||||||||
10 | Design | all users | The experience can be easily navigated and the user knows where they are at all times in the experience | The experience can be easily navigated and the user knows where they are at all times in the experience | 4 | 4 | 5 | 20 | 50 | |||||||||||||||||||
11 | Design | all users | On-boarding is completed in a timely manner and the user clearly understands the capabilities of the application after on-boarding | On-boarding is completed in a timely manner and the user clearly understands the capabilities of the application after on-boarding | 6 | 6 | 5 | 30 | 50 | |||||||||||||||||||
12 | Design | all users | The information is presented to the user in easily scannable format | information is presented to the user in easily scannable format | 6 | 6 | 10 | 60 | 100 | |||||||||||||||||||
13 | Design | all users | User can control notifications in a flexible way | it is simple to login | 10 | 10 | 5 | 50 | 50 | |||||||||||||||||||
14 | Design | all users | Login is simple | prompts for additional information are contextual and timely | 6 | 6 | 8 | 48 | 80 | |||||||||||||||||||
15 | Design | patient | In-experience advertising is not obtrusive | Conversational language is favored over medical jargon and uses the same terminology that clinicians use with patients | 2 | 2 | 7 | 14 | 70 | |||||||||||||||||||
16 | Design | all users | Conversational language is favored over medical jargon and uses the same terminology that clinicians use with patients | the experience can be easily navigated | 8 | 8 | 5 | 40 | 50 | |||||||||||||||||||
17 | Performance | patient | It is easy to connect peripherals and related services | it is easy to connect peripherals | 6 | 6 | 5 | 30 | 50 | |||||||||||||||||||
18 | Satisfaction & Delight | all users | Experience provides moments of levity and delight to the user | Experience provides moments of levity and delight to the user | 8 | 8 | 7 | 56 | 70 | |||||||||||||||||||
19 | Support & Feedback | all users | There is a means to provide feedback on the experience | there is a means to provide feedback on the experience and the feedback is reviewed on a regular basis | 8 | 8 | 6 | 48 | 60 | |||||||||||||||||||
20 | Support & Feedback | all users | There is a simple and fast way to get help with the application | there is a simple and fast way to get help with the application | 4 | 4 | 5 | 20 | 50 | |||||||||||||||||||
21 | Support & Feedback | all users | Upon imminent or actual system or user error there is clear communication and graceful recovery. | Upon imminent or actual system or user error there is clear communication and graceful recovery. | 10 | 10 | 10 | 100 | 100 | |||||||||||||||||||
22 | Support & Feedback | all users | The system provides informative feedback to the user about actions they are about to take or have taken. | The system provides informative feedback to the user about actions they are about to take or have taken. | 4 | 4 | 6 | 24 | 60 | |||||||||||||||||||
23 | Value | all users | Information needed for a particular task or decision making is grouped together on a single screen | Information needed for a particular task or decision making is grouped together on a single screen | 10 | 10 | 5 | 50 | 50 | |||||||||||||||||||
24 | Value | all users | the experience provides valuable information in a easily digestible format | the experience provides valuable information in a easily digestible format | 4 | 4 | 10 | 40 | 100 | |||||||||||||||||||
25 | Value | all users | The experience enables "jobs to be done" or tasks to be completed effectively | The experience enables "jobs to be done" or tasks to be completed effectively | 2 | 10 | 10 | 100 | 100 | |||||||||||||||||||
26 | Value | clinician | The experience is useful for clinical documentation and clinical decision support | The experience is useful for clinical documentation and clinical decision support | 8 | 8 | 5 | 40 | 50 | |||||||||||||||||||
27 | Value | clinician | The experience minimizes the creation of unjustified workflow for clinicians | The experience minimizes the creation of unjustified workflow for clinicians | 10 | 10 | 10 | 100 | 100 | |||||||||||||||||||
28 | provide values for all of the below | 0 | ||||||||||||||||||||||||||
29 | Performance | all users | Screens load and system actions execute in a timely manner | average load time for screens or wait times for system actions based on sampling of no less than 5 or 20% of screens (whichever is more) in seconds (0 = > 10 seconds, 1 = > 5 seconds, 2 = > 2 second, 3 = 1 second, 4 = 1-0.5 second, 5 = < .5 second) ) | 4 | 8 | 10 | 80 | 100 | |||||||||||||||||||
30 | Performance | all users | The error rate is low | number of errors as percentage per interaction | 3 | 9.7 | 9 | 87.3 | 100 | |||||||||||||||||||
31 | Research | process | Users were involved in validation research | how many users were involved in validation research? (0=0,1 = < 3, etc) | 3 | 3 | 8 | 24 | 80 | |||||||||||||||||||
32 | Research | process | Users were involved in your product development research | how many users were involved in your product development research? (0=0,1 = < 3, etc) | 4 | 4 | 10 | 40 | 100 | |||||||||||||||||||
33 | Satisfaction & Delight | all users | The experience has a good net promoter score (i.e. on scale of 1-10 how likely are you to recommend this app) | net promoter score (i.e. on scale of 1-10 how likely are you to recommend this app) of a sampling of no less than 5% of users in the last 3 months | 7 | 7 | 10 | 70 | 100 | |||||||||||||||||||
34 | Satisfaction & Delight | all users | The experience has an acceptable adoption rate | % of invited users who have used the app or service for at least two sessions in the last 6 months | 60 | 3 | 5 | 15 | 50 | |||||||||||||||||||
35 | Special needs & accessibility | patient | The experience is accessible and section 508/ADA compliant | For section 508/accessibility what is the W3 conformance level? (A=1, AA=2, AAA=3) | 3 | 5 | 8 | 40 | 80 | |||||||||||||||||||
36 | for all of the below answer YES/NO | 0 | ||||||||||||||||||||||||||
37 | Data control | patient | The app explains any sensors on personal devices it will use and why | does the app explain any sensors on personal phones it will use and why? | yes | 10 | 8 | 80 | 80 | |||||||||||||||||||
38 | Performance | all users | If a secondary system there is SSO/seamless embedding from the primary system | if a secondary system there is SSO/seamless embedding from the primary system | yes | 10 | 10 | 100 | 100 | |||||||||||||||||||
39 | Research | process | The relevant academic literature reviewed as evidence basis for your solution | was the relevant academic literature reviewed as evidence basis for your solution? | no | 0 | 8 | 0 | 80 | |||||||||||||||||||
40 | Research | process | Usability testing was conducted | was usability testing conducted? | yes | 10 | 8 | 80 | 80 | |||||||||||||||||||
41 | Research | process | Personas were developed as part of product creation | were personas developed as part of product creation? | yes | 10 | 10 | 100 | 100 | |||||||||||||||||||
42 | Special needs & accessibility | patient | The population evaluated for special needs (physical disabilities, hardware, connectivity, technical literacy) | was the population evaluated for special needs (physical disabilities, hardware, connectivity, technical literacy)? | yes | 10 | 10 | 100 | 100 | |||||||||||||||||||
43 | Special needs & accessibility | patient | If the population has special needs, measures were implemented to address them | if special needs were measures implemented to address them? | no | 0 | 5 | 0 | 50 | |||||||||||||||||||
44 | Special needs & accessibility | patient | Proxy access is available (spouses, caregivers, 3rd parties) | is proxy access available (spouses, caregivers, 3rd parties)? | n/a | 5 | 5 | 25 | 50 | |||||||||||||||||||
45 | Special needs & accessibility | patient | There is language support for languages of populations greater than 5% of total population | is there language support for languages of populations greater than 5% of total target population? | yes | 7 | 5 | 35 | 50 | |||||||||||||||||||
46 | Support & Feedback | all users | Human support is provided in addition to digital support | do you provide human support in addition to digital support? | yes | 8 | 5 | 40 | 50 | |||||||||||||||||||
47 | total | 2142.3 | 3110 | |||||||||||||||||||||||||
48 | score | 6.9 | ||||||||||||||||||||||||||
49 | ||||||||||||||||||||||||||||
50 | ||||||||||||||||||||||||||||
51 | ||||||||||||||||||||||||||||
52 | ||||||||||||||||||||||||||||
53 | ||||||||||||||||||||||||||||
54 | ||||||||||||||||||||||||||||
55 | ||||||||||||||||||||||||||||
56 | ||||||||||||||||||||||||||||
57 | ||||||||||||||||||||||||||||
58 | ||||||||||||||||||||||||||||
59 | ||||||||||||||||||||||||||||
60 | ||||||||||||||||||||||||||||
61 | ||||||||||||||||||||||||||||
62 | ||||||||||||||||||||||||||||
63 | ||||||||||||||||||||||||||||
64 | ||||||||||||||||||||||||||||
65 | ||||||||||||||||||||||||||||
66 | ||||||||||||||||||||||||||||
67 | ||||||||||||||||||||||||||||
68 | ||||||||||||||||||||||||||||
69 | ||||||||||||||||||||||||||||
70 | ||||||||||||||||||||||||||||
71 | ||||||||||||||||||||||||||||
72 | ||||||||||||||||||||||||||||
73 | ||||||||||||||||||||||||||||
74 | ||||||||||||||||||||||||||||
75 | ||||||||||||||||||||||||||||
76 | ||||||||||||||||||||||||||||
77 | ||||||||||||||||||||||||||||
78 | ||||||||||||||||||||||||||||
79 | ||||||||||||||||||||||||||||
80 | ||||||||||||||||||||||||||||
81 | ||||||||||||||||||||||||||||
82 | ||||||||||||||||||||||||||||
83 | ||||||||||||||||||||||||||||
84 | ||||||||||||||||||||||||||||
85 | ||||||||||||||||||||||||||||
86 | ||||||||||||||||||||||||||||
87 | ||||||||||||||||||||||||||||
88 | ||||||||||||||||||||||||||||
89 | ||||||||||||||||||||||||||||
90 | ||||||||||||||||||||||||||||
91 | ||||||||||||||||||||||||||||
92 | ||||||||||||||||||||||||||||
93 | ||||||||||||||||||||||||||||
94 | ||||||||||||||||||||||||||||
95 | ||||||||||||||||||||||||||||
96 | ||||||||||||||||||||||||||||
97 | ||||||||||||||||||||||||||||
98 | ||||||||||||||||||||||||||||
99 | ||||||||||||||||||||||||||||
100 |