RARA Newsletter vol. 9 Creating a Warm-Hearted World through Visualization of the Human Mind and New Relationships with Robots: Interview with Professor Shima Okada

Various collaborations and links within and beyond the University. Researching communication robots that inspire human emotions.

 

This newsletter features an interview with RARA Fellow Shima Okada, Professor at the College of Science and Engineering.

Professor Okada said, “By visualizing changes in people’s minds and relationships, and intervening appropriately, I want to create new relationships between people and machines and ultimately improve the quality of human relationships.” Her goal is to create a society in which more positive human relationships are fostered by utilizing communication robots that connect people to machines and people to people.

 

Prof. Okada is also a member of the Research Center for Sport and Health Science and the Robotics Research Center. In addition to active collaboration with other RARA Fellows, Associate Fellows, and researchers in and outside the university in various fields, she works with companies and local governments.

 

How can robots positively change people’s minds, bodies, behaviors, and human relationships? We asked Prof. Okada about her various activities, motivation, and vision.

 

(The following is a summary composed by the writer based on the discussion with Prof. Okada.)

 

Toward better human relationships

 

I always think of ways to optimize human relationships. Of course, people are humans—sometimes people fight, sometimes we do not get along with people we have just met, and sometimes we have poor relationships with family and friends. It happens to everyone at different levels.

 

I think it is important that these things do not become a burden on people’s lives, and that relationships become positive through their daily lives. This is an important element of wellbeing.

 

Therefore, I aim to achieve better relationships using a variety of technologies. To achieve this, I want to go beyond my specialty and combine various technologies.

 

Utilizing “empathy data” at matchmaking events

 

Numerous professors and companies from both within and beyond the University have approached us and we are working together on research and development.

 

For example, we used our technology at a matchmaking event in Shiga Prefecture the other day. With parts of the prefecture facing depopulation and aging, the local government is promoting matchmaking activities. We collaborated with Shiga Prefecture and a local dating agency to organize a matchmaking event to try to match people through “visualization of the mind” by measuring participants’ mental information and matching couples based on it.

I have heard that many people who meet each other for the first time at a matchmaking event choose their potential partner based on appearance and conditions. Conversely, we introduced a method of selection that focuses on empathy data. We achieved high satisfaction among participants, and I believe that it was a successful trial.

 

Specifically, we used wearable devices to measure data regarding participants’ autonomic nervous systems.

 

There is a phenomenon whereby people’s autonomic nervous systems synchronize when they have the same experience in the same space. At this matchmaking event, we measured the participants’ degree of empathy for each other and provided them with results that suggested who they are likely to empathize with, and used it to facilitate 8-on-8 pairings.

 

How much can we increase the matchmaking rate by providing results derived from empathy data, such as the alignment of intangible elements like values and personalities? Does it increase the satisfaction of the persons concerned? Although I was worried, the results of the survey showed that participants were highly satisfied with the event, with responses such as “It was good to know each other’s level of empathy when meeting each other for the first time” and “It was good to be able to choose a partner based on empathy as well as conditions.”

 

Does machine-to-human stimulation change a person’s behavior and spirit?

 

Considering the assumption that people do not change, I am conducting research on methods to encourage people to change their behavior. I previously worked on a project on getting people lacking exercise habits to exercise, but I found it difficult to change their behavior unless they were strongly motivated.

 

At RARA, my topic of research is whether stimulation from physical robots can change people’s behavior by intervening in people’s minds and communication.

 

The field of engineering robotics has long promoted the integration of man and machine.

 

Cybernetics is a well-known academic field centered on communication between humans and machines. Until recently, academia, including cybernetics, understood communication as limited to a one-way flow from the humans to machines, seeking to operate machines effectively by connecting people to machines.

 

What we are exploring now is a system in which people and machines are unified. In this system, machines stimulate human functions to influence human behaviors and minds, and the machine takes in the change and alters the stimuli. There is relatively little research on such process.

 

Improved sleep quality? Robot Pillow Project

 

The general idea is that machines do not give feedback to or interfere with humans. For example, when asked something by a human, generative AI sources the answer from an abundance of information; it does not change the physiological function or behavioral motivation of the person in any way.

 

However, it is surprisingly easy to change a person’s biological functions with external input. Our research has demonstrated that physical stimuli can positively affect a person’s biological functions.

 

For example, we are working on a “pillow” project in collaboration with a company to improve sleep quality. This is a joint research project using a transformable robotics pillow called “Nemurisu” with Ax Robotix, a company that develops sleep-related products and services.

It is known that micro-rocking motion initiates sleep and deep sleep, which is referred to as the “rocking phenomenon.” Rocking is conventionally done using hammocks and cradles, but in this study, we used “Nemurisu” to experiment with rocking only the head.

 

More specifically, we used the height adjustment feature of the “Nemurisu” to rock the user’s head slightly up and down during sleep. A plate connected to the silent motor under the pillow repeats a small up and down movement of about 2cm. It takes 4 seconds to rise and 4 seconds to fall.

 

Research indicates that this successfully improves the quality of sleep compared to a stationary pillow.

The graph on the right shows an increase in “delta waves” that occur during deep sleep.

 

I presented a paper on this research at the “IEEE 42nd International Conference on Consumer Electronics” held in Las Vegas in January 2024, for which I received a Best Session Award.

 

I intend to conduct further research to optimize the rocking motion and help users achieve ideal sleep. Professor Satoshi Fujita (RARA Associate Fellow) of the College of Sport and Health Science is investigating the detailed biochemical response.

 

In joint experiments with partner companies, we take the utmost care to ensure that participants do not suffer any side effects. We verify any side effects associated with the main effect and make careful experimental plans to prevent unexpected side effects. Moreover, at the time of social implementation, we explain the side effects in advance and propose countermeasures.

 

The COVID-19 pandemic dramatically changed the way people communicate, creating problems of loneliness and isolation

 

Inspired by my grandfather, who was an anatomist, I always wanted to be a researcher. Many of the people around me were medical professionals, but I wanted to develop technology that would help a wider range and larger number of people rather than providing direct medical care.

 

When I was a high school student, I saw an information pamphlet for Ritsumeikan University, where I learned about the field of biomedical engineering, which connects robots and humans. I thought, “This is where I want to study.” In my third year of bachelor’s degree, I joined a biomedical engineering laboratory and studied under my academic supervisor and mentor, Professor Masaaki Makikawa.

 

Upon completing my Master’s degree, I joined and worked for a company but returned to pursue a PhD after getting married. I experienced pregnancy and childbirth during the PhD program and after completing my degree, I finally became a researcher.

 

My research was originally focused on the development of sensors and biometric processing. This focus shifted to relationships and communication only recently.

 

We witnessed a dramatic change in human behavior and communication during the COVID-19 pandemic. A growing number of people began feeling lonely and isolated. Also, I became acutely aware of the communication problems of the younger generation through supervising students.

 

Realizing the difficulty of changing people’s behavior

 

The issue of bullying is part of this. In many cases, simply telling a bully, “Stop, this behavior is bad,” does not solve the problem. I believe it is important to support both the bully and the victim. I want to find a solution to how to change the behavior of bullying.

 

It is not very realistic to expect that everyone will get along in all relationships, but I want to create a society where no one suffers or becomes isolated. This desire is my driving force.

 

I participated in the Ministry of Education, Culture, Sports, Science, and Technology’s “Center of Innovation Science and Technology based Radical Innovation and Entrepreneurship Program” (COI STREAM) and worked with the Tokyo University of Arts and experts in psychology on the topic “Bright Future for All Ages with Health Innovation by Daily Exercise” to pursue research on how to encourage people to start and continue exercising. Our research sought to create and sustain an exercise habit among middle-aged and older adults to maintain their health.

 

However, we discovered that people without exercise habits rarely change their behavior, regardless of how strongly we emphasize the importance of health and explained future risks. I thus came to understand the difficulty of changing behavior.

 

Similarly, with issues like bullying, changing behavior can be very difficult. Simply saying, “Stop this, it is bad,” does not solve the problem unless you approach the root cause. Solving such a problem is a significant motivation for my research.

 

Can robotic intervention change people’s behavior?

 

Behavioral change is a difficult topic in any research field. External rewards and punishments can be used to change a person’s behavior, but without changing the fundamental motivation, you cannot expect a behavior change. I think this is generally understood.

 

The pillow is just one example; surprisingly, subtle functions of the human body can be influenced by external stimuli or robotic intervention. I similarly believe that robots can change human-to-human communication by intervening in groups.

 

Consider, for example, a robot intervening in the case of an isolated child in elementary school. If the robot befriends the child, the other children might find it interesting and gather around naturally. Communication can be generated through the robot, resulting in a small group forming around the child as people around gain a better understanding.

 

Robots could also help redirect social attention and reduce bullying. I think robotic intervention can be a possible factor in changing people’s behavior.

The importance of enjoying eating explored by the degree of rise in blood glucose level

 

One of my research themes is the effect of good relationships on the physical state of the body. This research compares postprandial blood glucose levels between “eating while having a good conversation with a friend,” “eating alone,” and “eating with a person you have just met while nervous.”

 

Results showed that blood glucose levels rose the least when the participants were having a pleasant time with friends, and rose significantly when nervous or eating alone. This indicates that the environment when eating has a significant impact on health and is an important factor in the prevention of lifestyle-related diseases.

We were very surprised by the evaluations of professors in various fields, which were more positive than we expected.

 

Spreading research through RARA, leading to social implementation

 

I find RARA a very interesting environment, one in which I can connect with other Fellows and Associate Fellows in various ways, enabling interdisciplinary engagement and collaboration to expand my research into their fields.

 

Fellows and Associate Fellows have no barriers, and some of them find my proposals interesting. They provide advice on the theory of research, verification of actions and side effects, and identify issues in the research. Being able to ask professional experts for help with problems that you cannot solve by yourself is a major boon. I can realize protocols for experiments I am unable to do on my own and even access research data I might not have been able to obtain before. I often feel like a student myself, expanding my knowledge and research with the help of other professors.

 

For example, I have been conducting international collaborative research with Professor Fujita. We are building a relationship whereby we advance each other’s research by providing research data.

 

Through the RARA framework, I have been able to do things that would be impossible alone, which is a great draw. I think such flexible collaboration is possible because the professors are selected independently of college or department and are active across the board.

 

While such technologies are being developed and implemented in the fields of psychology and engineering, my current research does not fall squarely into either field but rather combines parts of each.

 

With advice from RARA professors, I have been able to expand collaborations with companies and advance discussions for social implementation. I think the high degree of external interest can be attributed to the influence of RARA. While fundamental research needs to be carried out thoroughly, we also need to implement new technologies in society as soon as they are developed.

 

Ultimately, our goal is to create a ‘warm-hearted world.” The COVID-19 pandemic changed many things and systems. I will continue to research ways to connect people more successfully and positively, facilitating the realization of a “warm-hearted world”.

 

I also hope to inspire children and students to pursue research as a profession through my endeavors.