Participatory Design
Participants interact with their Workstations in their homes, controlling videophone, lights and small appliances; identifying problem incidents as they occur during the course of their daily usage. Event-logs are collected nightly, reviewed each morning, checked against Voice-memos and followed-up as necessary with face-to-face debriefings. These analyses are then fed back into the iterative design loop to inform subsequent redesigns, as depicted in this Remote Usability Testing Schematic:
Participants engaged in this home-based Participatory Design Process, consisting of:
-
User needs assessments
-
Prototype deployment in Participant's homes
-
In-home usability testing, including automatic 24/7 event logging
-
Feedback from Participants, including System and Participant prompted Voice-memo reports
-
Iterative Design, based on the Usability Problems identified from the Contextual Incident reports
Remote Usability Testing
The goal of our design for the automated Participant-generated usability feedback component of CareWheels was to provide developers with rapid feedback for iterative user-centered design. The advantages of this approach are:
-
Feedback from realistic situations and actual system use in a home environment
-
Automated data collection and data transcription
-
Ability to collect usability feedback over long periods of time
-
Ability to collect information on unanticipated use of the system
The System is embodied as “Merlin” - an animated Anthropic Agent, providing an Interactive Voice Response (IVR) interface. Participants issue verbal commands to run programs and SmartHome functions by speaking the attention phrase: “Merlin old chap” followed by the desired commands.
Over the course of a typical day, we wish to provide opportunities for both Participant and System prompted feedback. These interactions are encoded as Events: script-driven system functions with build-in usability tests. For example, every afternoon at 3 PM, if the Participant is present, the DailyFeedback Event script is triggered. Merlin asks the Participant:
“Is this a good time for you to give your Daily Feedback about this system?”
If the Participant answers “No”, then Merlin replies:
“OK ... I'll check back with you later.” Merlin waits a half hour, then asks again.
Otherwise, if the Participant answers “Yes”, then Merlin continues:
“At the tone, please tell me in what ways you have used the system today.”
The Take-a-Memo function records the Participant’s comments. Then Merlin asks:
“On a scale of one to ten, with ten being the most satisfied, how satisfied are you with the system? At the tone, please say the number.”
Merlin makes one final request:
“At the tone, please record your comments or suggestions.”
Merlin completes the transaction by expressing appreciation to the Participant:
“Thank you for your daily feedback.”
Once the Participant has completed this DailyFeedback Event, Merlin will not run it again until the next day.
Contextual Incident Report
This Sample Log File Excerpt is marked-up to distinguish the components of the Participant-System dialog. Note that this dialog is between Participant "John" and System "Merlin", pictured below, the interactive Anthropic Agent that serves as the embodiment of the System. In this case, Merlin mis-recognized John's request. John's digital recording files in Data Fields 7 and 9 have been transcribed to demonstrate how the co-incident date and time stamped Incidents and Voice-memos are integrated into a Contextual Incident report.
The Contextual Incident reports were analyzed to determine which events warranted face-to-face debriefing between the Participant and Evaluator. The resulting Usability Problem Description provided the basis for the Redesigned Solution, which was fed back into the iterative design process. By this Remote Usability Research Method, CareWheels has developed the means to evaluate and evolve technologies that will provide a reliable foundation for future TeleCare Services.