A robot and supervised learning algorithms for physical rehabilitation

What if robotics and machine learning could help ease your back pain? The KERAAL project, led by IMT Atlantique, is working to design a humanoid robot that could help patients with lower back pain do their rehabilitation exercises at home. Thanks to supervised learning algorithms, the robot can show the patient the right movements and correct their errors in real time.

Lower back pain, a pathology primarily caused by aging and a lack of physical activity, affects a large majority of the population. To treat this pain, patients need rehabilitation from a physical therapist, and they must perform the prescribed exercises on their own on a daily basis. For most patients, this second step is not carried out very diligently, leading to real consequences for their health. How can they receive personalized assistance and stay motivated to perform the rehabilitation exercises long-term?

The KERAAL project, funded by the European Union under the Echord++ project, led by IMT Atlantique in partnership with Génération Robot and CHRU de Brest, has developed a humanoid robot capable of showing physical therapy exercises to a patient and correcting the patient’s errors in real time. The researchers have used a co-design approach, working with physical therapists and psychologists to define the most relevant exercises to be implemented, and be as specific as possible in defining the robot’s gestures and verbal instructions, and study how the robot is received by patients and therapists.

“With a coach at home, patients have a physical presence and moral and emotional support that encourages them to correctly pursue their rehabilitation,” explains Mai Nguyen, project coordinator and researcher at the Computer Science Department at IMT Atlantique.  “The robot offers a way to monitor the patient’s performance of daily repetitive exercises, a task that is tiresome for therapists. At the same time, it prevents patients from having to make daily trips to the rehabilitation center.”

A supervised learning algorithm that corrects the patient’s movements in real time

In 2014, the researchers began tests with the Nao robot, developed by SoftBank Robotics. “We found that Nao did not have enough joints to allow him to reproduce the rehabilitation exercises,” Mai Nguyen explains. “This is why we finally chose to work with the Poppy robot, which has a backbone. Therefore, he can move his back, which is better adapted to the treatment of lower back pain.”

The small humanoid robot is equipped with a 3D camera and algorithms capable of extracting the “skeleton” of the person being filmed and detect their movements. The IMT Atlantique team worked on a supervised learning algorithm capable of analyzing the movement of the patient’s “skeleton” by comparing it to the demonstrations of the exercises previously shown to the robot by the health professional.  “The algorithm we are working on will determine the common features between the physical therapist’s different demonstrations, and will identify which variations he must reject,” Mai Nguyen explains. “There are some differences in execution that are acceptable. For example, if the exercise focuses on the arm muscles, the position of the feet is not important, whereas in the movement of the shoulders, every detail counts. The same level of precision is not required for every body part at all times.”

The robot has a list of common errors in the physical therapy movements that have been previously identified and are associated with specific instructions for the patient. If the robot detects an error in the execution of the exercise, it will be able to verbally communicate with the patient while performing the correct movement. “Our goal was to create an interactive system that could respond to the patient in real time, without requesting the physical therapist’s assistance,” Mai Nguyen explains. “The data from the robot could then be used by the caregivers for more thorough follow-up.”

A friendly and motivating presence

“For the time being, what we have observed is that the system is functional. The initial tests show that the robot is able to perform the ongoing monitoring of patients, but we are awaiting the end of the clinical tests to come to a conclusion on the robot’s effectiveness in terms of motivation and rehabilitation as well as on the patients’ and physical therapists’ experiences,” Mai Nguyen explains.

As part of the co-design approach, Poppy was subject to a pre-experiment phase during the initial development of the project with five senior citizens who seldom use technology. Following the robot-mediated physical exercise sessions, the psychologists interviewed the participants. The goal was to understand how they perceived the machine, and if they had correctly understood its movements. “Before the session, the subjects were very apprehensive about the idea of working with a robot, but Poppy was perceived very positively, and provided a friendly dimension that was very appreciated,” Mai Nguyen explains. “The subjects were very motivated to do their exercises right!” Tests have been carried out with six patients suffering from lower back pain at the CHRU in Brest and at the rehabilitation center in Perharidy.

For the time being, all experiments have been carried out in a hospital setting. But the researchers’ long-term goal is to propose a robot that the patient can take home, with a program of personalized exercises implemented by the physical therapist. “We are trying to develop a system that is as lightweight as possible, with only one camera as a sensor,” Mai Nguyen explains. The researchers have also launched a study of the business model with the perspective of the potential industrial production of this physical therapist robot.

“The project had very theoretical roots, but its completion is becoming more and more concrete!” Mai Nguyen explains. “We believe that, in a few years, this solution may be available on the market, bringing real advances in patient care.”

The work presented here was partially funded by the European project EU FP-7 ECHORD++ KERAAL, by the CPER VITAAL project funded by FEDER, and by the RoKINter project by UBO.