Samuel Cheng
UX Designer & Engineer
Summary

This project focused on the development of a photo reminiscence mobile application to assist caregivers of individuals with dementia. We focused on five major components: user research, ideation, design process, prototyping, and evaluation.

The goal of our application is to provide a simple and intuitive way for a caregiver (and their family) to organize photos into a story slideshow showing a timeline of a person's life, important details (e.g. who, what, when, where), and other events that the person with dementia may be interested in. These stories would then be played for the dementia patient helping them reminisce on past events as well as their current family. We believe this can help alleviate tension that often develops between the caregiver and the person with dementia as well as give the caregiver a brief reprieve from having to answer the same questions over and over again.

Details:

  • August - December 2015
  • Team of 4
  • User research, literature review, Sketch, Framer.js, CoffeeScript, heuristic evaluation, usability testing, video editing
Introduction

Dementia is a generic term for the wide range of symptoms related to an overall decline in brain function, memory, and the ability to think and function in daily life. Alzheimer's disease (AD) is the most common form of dementia, affecting nearly 5.3 million Americans in 2015 [1]. As Alzheimer's progresses, the afflicted individual’s memory and confusion may worsen, which can lead to the individual not recognizing former familiar people and places. Additionally, the individual may lose the ability altogether to communicate, and complete daily functions such as getting dressed, bathing, and eating [2].

Therefore, caregivers are important for patients with dementia or AD for a variety of reasons. Caregivers perform tasks involved in assisting with instrumental activities of daily life, such as getting the recipient dressed, helping them bathe or shower, fixing their meals, and helping with medication. Whether the caregiver is a spouse or family member, AD is known as a family disease due to the high stress levels that family members experience [3]. The related literature has also shown that caregivers of individuals with AD are more likely to self-report their health as fair or poor [4]. Additionally, caregivers are less likely to engage in healthy behaviors and less likely to take care of themselves while they are caregiving. Furthermore, the rate of depression is overall higher in this caregiver population compared to the non-caregiver rate [4].

One of the major challenges presented for both the caregivers and patients is magnified with respect to communication. A common occurrence caregivers experience is that they need to frequently repeat phrases that they have just said and their responses can give the appearance of not listening to what the person with dementia is saying. This can be frustrating and disempowering for people with dementia and distressing for the family and care staff they are trying to communicate with.
Technology Solutions

There are a variety of systems and tools that have been developed to assist caregivers and individuals with dementia. One popular branch of those technologies are reminders for individuals with dementia, such as medication reminder devices [5] and calendar event reminder frameworks [6]. Relink Device is an example of a smart clock for individuals with dementia [7]. Companion, a touch screen application, delivers scheduling reminders for activities during key times throughout the day [8]. Despite that, Companion can also display family photographs and audios such as preferred pieces of music and personalized messages recorded by family members. Some researchers have found that pre-recorded audio and visual cues stimulate positive memories and foster healthy daily routines. Kyoto etc. [9] studied the effectiveness of personalized reminiscence photo videos for individual with dementia. They demonstrated that photos and videos received more positive responses than TV series. An additional system, CIRCA, is an interactive, multimedia touch screen system that contains a wide range of stimuli to prompt reminiscing. Their results suggest that interacting with the touch screen system is engaging for people with dementia.

Based on our research, however, all the personal reminiscence systems so far heavily rely on the raw input materials, such as pre-recorded voices or photos with manually created descriptions, which burden the caregivers more and require more of their time. Our goal is to design a system to support caregivers in initiating a personal reminiscence system for their loved one with dementia, focusing on a system that is easy to use, has a shallow learning curve, and is very simple to operate. Our solution is Memory, an iPad application. Memory is based on reminiscence with the help of photos of family members and stories which are generated from photos according to life events. Caregivers can manage their family photos, and create stories that help the patients recollect memories from their life. While the patients interact with the stories, our tool will allow caregivers to regain control over their own independence and further maintain a more balanced set of emotions.
Target Users and the Problem

Our target users are primary caregivers for a family member diagnosed with Alzheimer's disease or dementia. In 2014, friends and family of people with Alzheimer's disease and other dementias provided an estimated 17.9 billion hours of unpaid care, a contribution of the nation valued at $217.7 billion [1]. Among them, approximately 34% are age 65 or older. More than half of the primary caregiver take care of their own parents. In order to better understand our target users and their preferences and needs, we conducted an anonymous survey of over 30 users as well as 3 semi-structured interviews. These methods allowed us to develop empathy with these individuals and witness a resolve to help their family members. The problem space has been revealed during the interaction with our target population. The typical user persona is shown below.



Survey

The purpose of our survey was to gain responses to our questions that could assist us in understanding our target users, specifically their demographics and main problems they face daily. From our survey, we found that the caregivers are experiencing a wide range of emotions. Most of these emotions included stress and frustration, demonstrating that caregivers are having a difficult time emotionally. When it comes to the challenges, the two most common responses we found was the caregiver dealing with the patient’s negativity and being able to juggle everything from work to caregiving since caregivers are experiencing a loss of freedom themselves.

Emotions Experienced on a Daily Basis


Top 3 Caregiver Challenges


Interviews

We also conducted 3 semi-structured interviews with participants that matched the target user population. The objective was to gain a deeper understanding into the firsthand accounts of the daily activities of the dementia individuals and challenges that a caregiver faces. Multiple individuals we interviewed indicated that when the individual with dementia was browsing some old photos about him/her or other relatives, even when they can not remember who they are, they showed strong interest and positive emotions during the experience. These experiences allowed the caregiver to have time to manage their own household and other duties while their loved one kept busy.
Design Process

Based on our initial idea and data collected from the survey and interviews we conducted in the last phase, our design process consisted of two informed brainstorming sessions and a series of continuous discussions. In the first brainstorming session, we adopted a divergent approach to generating possible directions. Then, a more convergent method was used in the second brainstorming session. We focused on our specific problem space that had been narrowed down through our discussion.

Three possible design directions were identified at the end of the first session.





Based on the result from our last brainstorming session, we used a different approach to idea generation in the second session in order to generate more detailed and feasible design alternatives. Before diving into the brainstorming part, we wrote down our problem space and strong evidence extracted from our survey and interviews on the whiteboard. While doing the brainstorming, we tried to create scenarios instead of keywords. Then, we shared our scenarios on the whiteboard and discussed each idea. Though the number of possible ideas was limited compared to the first session, it allowed us to go deeper with each alternative. After this round of brainstorming, we translated two design directions from the first session to more concrete design ideas and ended up with three design ideas: a smart stress ball, memory reminiscence, and social caregiving.






Our group faced some difficulty deciding between the three design ideas presented. We ultimately decided to hear from our potential users and created a short YouTube video where we presented the design ideas and asked viewers to give us survey feedback on which idea they would use. The overwhelming majority favored memory reminiscence so based on this data, we decided to pursue the memory reminiscence application.

We then engaged in several whiteboarding sessions to determine the appropriate look and feel for the user interface, its interactions and information placement.

We decided not to use a hamburger menu. Since we only have four main modes (i.e. photo library, stories, family tree, and profile information), we didn't need to have an expandable menu according to iOS guidelines. The second reason is that with a hamburger menu, the user is required to take an extra step to their desired action.

We also made the bottom navigation bar taller than normal iPad applications to account for our older adult population. We wanted to make sure older adults could still easily read the bottom navigation bar without it obscuring the main features of the screen.

When it came to design, we chose green, as research has shown that green not only facilitates creativity but it is also associated with nature, life, and growth. Similar to blue, green has a calming effect [12,13]. We chose a Sans-Serif font, as we didn’t need to have users read a very large block of text. Most of our text consist of headings, and stylings, which are easier to read with a Sans-Serif font.

Design and System Prototype

Our application, Memory, is an application designed based on photo reminiscence. Photo reminiscence has been proved to be useful in dementia care services [10]. By using photographs and videos, Memory helps caregivers answer repeated questions related to family-shared memories from individuals with dementia. Three main features are designed with existing technologies to address our user group’s problems.

Photo Library

Our Photo Library is a gallery that stores all family photos. Memory allows Caregivers to manage Photo Library with ease. By using third party APIs, Memory enables various sources where caregivers can add photos from, such as Facebook and Instagram. This saves the caregiver numerous effort in terms of building a library. Caregivers are also able to edit or delete photos with a few clicks.

Memory also generates a set of attributes for each photo upon being added to Photo Library automatically. The application extracts information about when and where a photo was taken from the photo's metadata. We also plan on utilizing Face Recognition techniques to detect faces in photos. After a brief training session with caregivers, Memory should be able to know which family members are in the photo and their ages as well.

These features in our photo library will help the individual with dementia feel more independent, and may assist them in answering their questions, reducing frustration for the caregiver.

Story

Our Story feature is essentially a slideshow element. With background music and smooth transitions, a story creates an immersive experience that helps individuals with dementia reminisce on their family memories. Memory supports two types of stories.

A Personal Story is based on the timeline of a family member and is generated by Memory automatically. With automatic face-recognition, Memory can easily select all photos of a specific family member and then create a Personal Story by sorting her/his photos by age or other filters. A Personal Story is therefore an effective way to reminisce the life history of each family member. Caregivers also can create event-based stories which are called Created Stories. By selecting photos and adding event-specific information, caregivers are able to create a story quickly.

Our hope is that these stories can be enjoyed together with the caregiver and the individual with dementia, helping them maintain their relationship with positive experiences. It also has the potential to give caregivers more time for themselves while the individual with dementia watches the story on their own.

Family Tree

Our Family Tree feature is a diagram that consists of an overview of relationships and information of all family members. It groups all family members together with each person having a picture of themselves. Their personal information, such as their relationship with the person living with dementia and where they currently live, will be shown on a side panel if their picture is selected.


The Family Tree helps the individual with dementia answer their questions as well, and help them see who each family member is, when needed.

Prototype

The designs of each screen were built in the Sketch 3 graphics editor. Each of these designs were imported into Framer.js to build out a high-fidelity interactive prototype using Framer's CoffeeScript language.
Usability Criteria

Due to our users being in an older demographic, as well as the sensitive nature of the problem we attempted to assess, there are a few usability criteria we wanted to focus on.

Perceptible User Interface (UI)

For older users, age-related changes in vision, including visual acuity, color perception, and susceptibility to glare, are important considerations. To address those issues, Memory uses clean typography, larger font sizes, and a proper color scheme with enough contrast.

Recognition Rather Than Recall

Memory is also affected by the aging process. It is important to us that our users do not have to rely on additional memory resources when using our application. Even for younger caregivers with such a busy schedule, our application intuitive to use. We carefully use icons, phrases, and metaphors that map with real-world objects to avoid introducing new concepts to users. We also follow industrial conventions and guidelines so that caregivers who are familiar with the operating system of the device can adapt to Memory quickly.

Error Prevention and Recovery

With a lot of editing features, it is important that our users do not become frustrated or confused with unexpected actions they may accidentally perform. It is also crucial that users can revert certain actions that lead to errors. First, we use constraints on some screens to limit actions that users could take. One example is disabling buttons that are not available in a current mode. We also use confirmation dialogs to confirm some crucial actions that the user attempts to take, such as deleting a photo or terminating a process. Once an error occurs, we make sure that users can receive proper feedback and error information that informs them on how to remedy actions.

User Control and Freedom

Since our user group is quite varied, we expect to see various behavioral patterns. Memory needs to have enough user control and freedom to facilitate users' interactions. For instance, users are able to view the personal story of a family member in two ways, either from the Personal Story list on Stories screen or from that particular family member’s personal profile panel on Family Tree screen.
Evaluation

We conducted both heuristic evaluations and usability tests to evaluate our application, Memory.

Heuristic Evaluation

Our team conducted five heuristic evaluations of our memory reminiscence mobile application based on the "10 Usability Heuristics" by the Nielsen Norman Group. We chose to do a heuristic evaluation in order to get quick feedback from other expert evaluators and designers. We focused mainly on a subset of the usability heuristics that we believed were more important to consider: the visibility of system status and recognition rather than recall.

We prepared the following four task scenarios and each evaluator was given two to perform.

  1. Upload two photos and edit its related attributes.
  2. Create a story.
  3. Play a personal story.
  4. Find out where Betsy Johnson’s son lives.
Tasks 1, 2, and 3 were chosen to explore the main functionality of our application: the editing of photo information, the creation of photo stories, and the recognition between automatically-generated personal stories and manually created created stories. Task 4 was chosen in order to explore a secondary functionality of our application: the family tree diagram and the organization of people's information in relation to the person with dementia. Successful completion of each task requires some form of understanding of the conceptual model underlying our application. By this we can gauge whether or not the design succeeded in helping the user develop his or her mental models of the application, taking advantage of pre-existing metaphors to enhance understanding as well.

Usability Testing

After our heuristic evaluation, we conducted usability testing with caregivers of an individual with dementia. Participants were recruited by reaching out to the local Georgia chapter of the Alzheimer's Association. Several other attempts to recruit from connections within Georgia Tech, Emory and other adult day care centers around the Atlanta area proved unsuccessful. We ended up with four participants which met our inclusion criteria: English-speaking, adult, family caregivers of a loved one with dementia or Alzheimer's disease between the ages of 45-70. Participants were given the IRB-approved informed consent document and all agreed to have a screen capture of the iPad record their interactions and voice.

We had each participant complete the following six tasks that covered the main functions of our application.

  1. Upload two photos (from the tablet’s photo library) and edit its related attributes.
  2. Create a story.
  3. Play a personal story.
  4. Find out where Betsy Johnson’s husband lives.
  5. Find pictures between Betsy Johnson and her husband.
  6. You realize that one of your photos has an incorrect date. Find the photo and correct its date.
Tasks 1, 2, 3, and 6 were chosen to explore the main functionality of our application: the editing of photo information, the creation of photo stories, and the recognition between automatically-generated personal stories and manually created created stories. Tasks 4 and 5 were chosen in order to explore a secondary functionality of our application: the family tree diagram and the organization of people’s information in relation to the person with dementia. Successful completion of each task requires some form of understanding of new terminology and the conceptual model underlying our application.

While users completed these tasks, we measured whether they completed the task successfully, task completion time, the number of errors they made, and their efficiency. A user’s efficiency in completing a task was measured through a "lostness" formula [14] that indicated to us whether the user was "lost" when performing the given tasks. After users completed the given tasks, they also filled out the System Usability Scale (SUS), which measured how usable they found our application.

Several notable usability issues are described below:

  • In some tasks that required switching modes, our users had trouble recognizing the tab bar due to its' visibility. One user even said on one page that there "was no back button," when the correct method they needed to complete was simply changing the mode.

  • Most users did not immediately recognize the "Add Photos" button. Instead their main focus was on the photos shown. Even after looking at the photos, some users still seemed to neglect that specific button.

  • Some users had trouble understanding what the difference was between personal story and created story. Some other users did not see the contrast in the play button that overlays the story, so they did not realize such story was clickable.

  • Some users did not see the top bar as a progress bar for their creation of a story. One user tried clicking on the bar, since they did not see the “Next” button in the upper right corner. Primary tasks on the screen are not obvious enough. Contrasts between clickable elements and non-clickable elements are not obvious enough.
In an informal interview, we found that users still greatly enjoyed Memory and thought it could be very effective in their daily lives. The System Usability Scale score of 84.4 indicated that users loved our application and found it highly usable. Our application received a lostness score of 0.21. Anything less than 0.40 meant that users did not exhibit any observable characteristics of being lost. When it came to completing our tasks, while users seemed to have some initial trouble, once they understood the system, they were able to complete the tasks effortlessly.
Lessons Learned

Running a successful usability test is harder than it looks especially when it comes to organizing the script and making sure that everything is in place. However, as a skill, it gets better with practice. We discovered that not only did we have to run the test and keep the participant on task, we also had to encourage him or her when the tasks seemed too difficult as a way to raise their confidence.

The quality of feedback we received from our users was remarkable in that it opened our eyes to areas that we were previously blind. Furthermore, we always learned something that surprised us such as how someone's conceptual model of the application did not match what we were trying to achieve. We noted that even within a relatively specific target population, there was a wide disparity between how comfortable each participant was with using an iPad photo application and technology in general. Usability testing is crucial to the success of any design. Though implementing the prototype is important, teams must be always be analyzing and evaluating the effectiveness of the design. Design should be user-centered and if there is any important takeaway it is this: we are not our users.
Conclusion and Future Work

The purpose of Memory is to help caregivers and their families to tell stories of their shared memories to the individual with dementia. This application aims to help caregivers managing photos and generating stories in various ways according to the metadata that is extracted automatically. We are considering using machine learning algorithms to extract rich information from the raw materials to reduce the workload of caregivers. Personal stories will be generated according to the metadata of the photos. Caregivers are given the freedom to create stories for their special memories. Family trees are organized by family member information and their relationships. During the design process, we take usability criteria into consideration. We conducted heuristic evaluation and usability testing for our product, and received positive feedback from real potential users.

In the future, we are going to introduce a family account system into the product. The family may include multiple family members who might have precious shared memories with one another. An in-app tutorial will be shown to the users on the first time they open the application in order to eliminate the confusion about the specific terms in our product. Also, we will further investigate what is the most effective way of displaying the stories to people with dementia, using simple interactions like swiping to advance to the next photo or just presenting photos in one video without any interaction. Helping people with dementia to recollect memories in engaging and enjoyable way and reducing the burden of caregivers will positively benefit for both of them.
References

1. Alzheimer’s Association. 2015. Alzheimer's disease facts and figures. Alzheimer's & dementia: the journal of the Alzheimer's Association, 11(3), 332.

2. Alliance Family Caregiver. 2001. Fact sheet: Selected caregiver statistics. San Francisco, CA: Family Caregiver Alliance. October, 19, 2011. 


3. Caregiver.org. 2015. Alzheimer’s Disease and Caregiving. 16 September 2015, from https:// caregiver.org/alzheimers-disease-caregiving. 


4. Clyburn, Leah D., Michael J. Stones, Thomas Hadjistavropoulos, and Holly Tuokko. 2000. Predicting caregiver burden and depression in Alzheimer's disease. Journals Of Gerontology Series B, 55(1), S2-S13.

5. Kamimura, Tomoko, Rina Ishiwata, and Takenobu Inoue. 2012. Medication reminder device for the elderly patients with 
mild cognitive impairment. American journal of Alzheimer's disease and other dementias, 27(4), 
238-242.

6. Wolters, Maria Klara, Fiona Kelly, and Jonathan Kilgour. 2015. Designing a spoken dialogue interface to an intelligent 
cognitive assistant for people with dementia. Health informatics journal.

7. http://www.relinkdevice.com/.

8. http://www.simplec.com/companion/

9. Astell, Arlene J., Maggie P. Ellis, Lauren Bernardi, Norman Alm, Richard Dye, Gary Gowans, and Jim Campbell. 2010. Using a touch screen computer to support relationships between people with dementia and caregivers. Interacting with Computers 22, no. 4 (2010): 267-275.

10. Yasuda, Kiyoshi, Kazuhiro Kuwabara, Noriaki Kuwahara, Shinji Abe, and Nobuji Tetsutani. 2009. Effectiveness of personalised 
reminiscence photo videos for individuals with dementia. Neuropsychological Rehabilitation, 19(4), 
603-619.

11. Jastrzembski, T. S., & Charness, N. (2007). The Model Human Processor and the older adult: parameter estimation and validation within a mobile phone task. Journal of Experimental Psychology: Applied, 13(4), 224.

12. Lichtenfeld, S., Elliot, A. J., Maier, M. A., & Pekrun, R. (2012). Fertile green green facilitates creative performance. Personality and Social Psychology Bulletin, 38(6), 784-797.

13. Calkins, M. P. (2002). Using Color as a Therapeutic Tool. Ideas Institute. 14. Albert, W., & Tullis, T. (2013). Measuring the user experience: collecting, analyzing, and presenting usability metrics. Newnes.