A Brain-Inspired Self-Organizing Episodic Memory Model for a Memory Assistance Robot
Chiao-Yu Yang, Edwinn Gamborino , Li-Chen Fu, Yu-Ling Chang
This paper presents a novel approach to memory assistance robots targeting the elderly. The aim is to develop a robot companion with an effective memory assistance system. To achieve this, an end-to-end memory assistance system is proposed and implemented on the Pepper robot, integrating event observation, memory storage, and personalized assistance. The essential component is a brain-inspired episodic memory model, which incorporates a hierarchical long-term memory storage model capable of memory clustering, merging, and summarization, mimicking human memory processes. Using reinforcement learning and social interaction, the robot learns effective cues for stimulating memory recall, offering tailored assistance. Positive user feedback validates the practical implications. The proposed model demonstrates promising results in experiments. The memory assistance robot significantly improves participants' recall accuracy for activities, providing valuable retrospection and strengthening memory. The self-organizing episodic memory model exhibits fast computing and adaptability on limited hardware. This research has potential implications for memory support and human-robot interaction. The developed brain-inspired memory model enhances the capabilities of memory assistance robots, benefiting elderly users and potentially extending to other applications requiring memory assistance.
Interactive Question-Posing System for Robot-Assisted Reminiscence from Personal Photos
Yi-Luen Wu, Edwinn Gamborino, Li-Chen Fu
This paper presents the implementation of a robot companion that assists individuals in reminiscing about personal memories through the analysis of photos. The aim is to enable the robot to associate relevant concepts with the photo content and engage users through relatable questions. To achieve this, deep learning techniques are employed to recognize events, objects, and scenes in the photos, and a Markov Random Field-based algorithm incorporating common sense knowledge is utilized to infer associated concepts and autonomously generate semantically related and socially appropriate topics. The system poses appropriate questions based on these topics to guide users in reminiscing through conversation. An integrated, end-to-end robotic system capable of real-time user interactions is created. It can pose relevant questions and engage users effectively, providing the potential for organized reminiscing. Experimental validation demonstrates the high accuracy of the event recognition network on both a dataset and personal photos. The human-robot interaction experiment confirms the system's capability to ask coherent questions related to the image and previous user utterances, with the potential to guide memory recall. Although some results lack statistical significance, they exhibit a generally positive outcome, offering insights for improving question coherence in future studies.
Inferring Stressors from Conversation: Towards an Emotional Support Robot Companion
Yu-Cian Huang, Edwinn Gamborino, Yan-Jia Huang, Xiaobei Qian, Li-Chen Fu, Su-Ling Yeh
This paper presents a conversational robot system that provides emotional support to individuals facing stressful situations. The aim is to identify stressors from user speech and offer appropriate support strategies based on their detected stressors and personality traits. A novel stressor-specific knowledge graph derived from ConceptNet is created to enable the robot to infer stressors from verbal utterances and exhibit artificial empathy. The system then employs a pre-trained neural network to determine the most suitable emotional support strategy for the user with a specific personality. The proposed system achieves real-time interactions through an embodied robotic agent capable of engaging in naturalistic conversations, conducting small talks, and providing emotional support when stressors are detected. Evaluation results demonstrate the system's ability to accurately infer stressor categories, with a high average Likert scale score, and effectively predict suitable emotional support methods, achieving a high Dice coefficient. The system successfully identifies stressor categories from self-disclosure in human-robot interaction experiments and generates appropriate supportive responses. This paper addresses a research gap by developing an emotional support conversational agent in the context of social robotics, contributing to providing guidance and comfort in stressful situations.
Adaptive Recommendation Dialogue System through Pragmatic Argumentation
In an aging society, we expect that a robotic caregiver is able to persuade an elderly person to perform a healthier behavior. In this work, pragmatic argument is adopted to make a user realize that an action is really worthwhile. Therefore, an adaptive recommendation dialogue system through pragmatic argumentation is proposed. There are three objectives in this work: first, we build a knowledge base for pragmatic argument construction, and the knowledge base concerns not only the effect of an action but also the reason for the effect; second, the robot is endowed with the ability to recommend an action that adopts to different users, and the recommended action is determined based on the integration of both robot’s and user’s value systems. In other words, the robot knows how to compromise with the user; third, the robot will persuade the user through his/her value system, and it will try to select the perspective whereby the user can be more interested in the action recommended by the robot.
Interactive Question-Posing System for Reminiscing about Personal Photos
Reminiscence is a lifelong activity that happens throughout our lifespan. While memories can serve as the topics in people’s chit-chat, recalling the past can also help people to build self-esteem and increase the level if happiness. In our work, we apply deep learning techniques to recognize events, objects, and scenes in the image to understand the content in the photo. Then, we use Markov random field to consider the observations from the photo and the user utterance. Afterwards, we utilize the loopy belief propagation to infer possible associated concepts and topics. Finally, we give appropriate questions about the selected topics to RoBoHoN. The result show that the proposed system can pose proper and related question to interact with the user, and has a potential to help and guide the user to recall the past in an organized way.
Context-aware Topic Proposal in Phatic Communion for Interaction Initiation of a Robot Companion
Recently, robot companions have become more and more popular; they are expected to provide not only physical assistance, but also social supports for human. In order to provide social connections and improve user experience, the capability to actively initiate an interaction in a natural way is important. When people try to engage with others, they often start from topics that are related to the context. In this work, we aim to develop a context-aware robot that initiate a non-task-oriented interaction with similar strategy. We focus on how a robot could interpret its environment as semantic concepts and then generate a proper topic proposal to start an interaction. With the help of object detection components developed in computer vision field, it can detect semantic entities in the environment, and combine such detection results using the commonsense knowledge to establish a context model based on hinge-loss Markov random field. Afterwards, according to the scores of each concept in the context model and the social relation with the user, the robot generates a sentence to initiate the interaction with the user. In the end, an experiment has been conducted to examine the effectiveness of the context-aware topics.