In this day and age, human-robot interactions are increasingly prevalent, both in our real life and in the cultural zeitgeist. Many groundbreaking projects are emerging worldwide, and Kazakhstan is not all that far behind with the APRIL project team at Nazarbayev University successfully employing a unique therapy that involves therapeutic robots—NAO and Mirai—to assist autistic children in adapting to their daily interactions.
Powering through unexpected challenges—be it faulty internet connections or even the complete lack of access—the team's research caught the eye of the international scientific community and was selected as the representative of this year’s Best Studies Paper at the 18th Annual ACM/IEEE International Conference on Human-Robot Interaction in Stockholm.
QazMonitor caught up with Dr. Anara Sandygulova, the project lead and Associate Professor at NU, to discuss the novel usage of robots in therapy sessions and the many moving parts of the whole ordeal, exploring how true technological passion can sometimes lead to something remarkable for the betterment of all.
Why robot therapy? How did the idea for APRIL come about?
We started APRIL as a research project. Our main motivation was to create a technically advanced system involving social robots working with autistic children, and since it has a lot of things that are technically challenging—engagement recognition to determine whether the child is engaged or not—we’ve integrated artificial intelligence algorithms to achieve that.
When the algorithms were in place, we needed to test their performance in real-world scenarios. So, we got our ethical approval for a big experiment at the National Children’s Rehabilitation Сenter in Astana. It was back in 2018, and to be honest, when we realized how effective it was...
So, children would meet with a robot ten times every day for fifteen minutes, and since these kids came from all over Kazakhstan, they would stay for three weeks, living with their parents. They would have music classes, ergonomics, and various physiotherapy sessions, such as swimming classes, and we integrated daily classes with our robot into their schedules.
After one child completed ten meetings with the robot, one parent told us with teary eyes, 'My daughter started to look into my eyes for the first time in her life. What did you do?'. It was shocking to hear this feedback. Before, the girl had never made eye contact with her mother, but because she spent all this time looking into the robot's eyes—which had facial recognition and eye contact functionality—it was as if the robot was following the child's gaze, and she really liked that. We believe that after this, she became curious about her mom's eyes.
According to some parents, it can take months or years to train eye contact. A newborn takes a few weeks before they begin to make eye contact with others; autistic children, however, might have absence of eye contact because they aren’t comfortable or because there is an unpleasant stimulation they receive from eye contact, so they tend to avoid it.
So imagine us just being driven by our technological research, yet seeing these impressive results and this impact on the children! The reason why we were able to do it was because our university is super close to the National Children’s Rehabilitation Сenter, and we were able to go there every day to work and collect our data; plus, every three weeks we had new children coming in.
Can you share more about what NAO and Mirai can do? And how does the therapy work with these robots?
We target different social, emotional, and cognitive skills that children need to learn, so there are specific activities in which children engage. We programmed twenty-two evidence-based practices in our APRIL curriculum.
One day, the child and the robot meet to listen to a story. Then, on another day, in addition to the story that they’ve listened to, they play a game that helps develop the child's imitation skills, and if the child is doing something correctly, the robot claps and says, ‘Well done!’
So, the robots are highly interactive. For example, NAO has twenty-five degrees of freedom – it can move its arms and legs and even demonstrate some yoga. It's not very big, about 57 centimeters tall, and it has a cute appearance, and all of this is very attractive to children. Mirai, on the other hand, has a completely different functionality because it has a graphical face. Essentially, it has a projector that projects a face, and its facial expressions can vary: it can be frowny, or it can demonstrate sadness and laughter, etc.
Autistic children often need to learn emotional expressions because they tend to have difficulties recognizing them, and that's where Mirai steps in. It has all of these facial expressions that appear realistic. We can project various faces – male, female, a child’s face, and it can even [project] a fictional character, such as one from anime, and we actually make use of this a lot. Say, we change the voice of a male [mask] to that of a firefighter, which explains how fire can be dangerous, and if it’s a female voice, the robot acts as a doctor for the child to practice a doctor’s appointment.
We've integrated a methodology called Social Stories into our therapy, allowing the robot to serve as a stand-in for a doctor. It would say, 'I'm happy to welcome you to my office. We are going to measure your temperature,' and then we would have a real thermostat and stethoscope for the child to use. This way, the child practices, and when they go to a real doctor, and see a thermostat or a stethoscope, they won't be afraid of it, they won't be stressed.
Your work involves communicating with parents and children and dealing with technical issues. Which aspect do you find to be the most challenging?
The part with the robots is challenging enough, but it is plain difficult to even talk about these things because not everyone can understand what robot-assisted autism therapy is. Not everyone understands what autism is, and it's not a well-understood subject either, so it's where [these] two different challenges converge.
Because we come from a technical background, the autism part was more challenging for us because we were learning about it by trial and error. Sometimes we were submitting a paper and, we would refer to children as ‘autistic children’, but back in 2018, it wasn't considered appropriate to use the word ‘autistic’. We would have reviewers telling us that we were not using inclusive language, and it's the opposite nowadays – it is encouraged now to say, ‘autistic children’.
It's 2023—six years have passed since we started—and the scientific community learned more things and the awareness of it all has changed. Now there are many autistic adults who are advocates for their cause, and the community responds, changing these terminologies.
Since APRIL was initially launched at the National Children’s Rehabilitation Center, can you tell us where else your therapy is currently being applied?
Last year, we were contacted by autism centers that asked us, 'Could you assist us? We want to purchase these robots. Could you help us understand which robots we should acquire?’ Because when the robots are delivered, they don't speak Kazakh or Russian. However, since we already had everything set up, we were able to support these centers. That's how the commercialization of our research began.
We partnered with four centers around Kazakhstan: there are centers in Pavlodar, Shymkent, Taldykorgan, and Oskemen. I would say that about 400 children use the robots on a daily basis, as each center has 100 children attending every day. Apart from these four, we’ve made agreements to establish four more centers by the end of this year.
What are your plans for the future?
Right now, we are working on adding software that will enable specialists to track the child's progress. We would maintain a log of how much attention each child is paying and how long they spend on various activities because what we've come to understand – parents are keen to know what's happening at the center during the day. [With our new software accessible to the parents], if a child is in the early stages of learning joint attention, parents will be aware of their progress.
Another motivation for doing this is that parents will begin to understand what their child has already achieved and what they still need to achieve. It’s especially important with social skills because progress in social skills is not as straightforward in terms of progress in comparison to cognitive skills.
All of this is not that obvious, and parents of autistic children will perhaps, for the first time, have the opportunity to be aware of what's going on at the center during the day and how to keep track of their child's progress.
