Philips researchers have already been using artificial intelligence techniques for decades, to automatically scan for diseases in medical images. More recently, they’ve started to apply AI to other types of video and also audio. In a project, together with trainees from the PDEng Software Technology program, they explored new possibilities in telehealth and customer support for hospital staff.
Artificial intelligence is a hot topic in healthcare. At Philips, AI already plays a role in half of the projects. “Doing smart things to otherwise unstructured data,” Marcel Quist calls it. “The data in itself isn’t actionable; the information you can derive from it is.” Within the company’s research department for professional health services and solutions, Quist is leading a team that is looking into the application of remote communication in the healthcare domain.
Riding the digitalization wave, medical professionals are increasingly utilizing the power of video conferencing to optimize the efficiency of their communication. Caretakers are setting up Skype-like calls to remotely talk to their patients, who can thus remain in the comfortable surroundings of their home. Hospital staff is connecting to customer support, anywhere in the world, to get assistance in operating medical devices or to resolve technical issues.
Artificial intelligence can also be very supportive in remote communication. One of the research lines pursued by Quist’s team is to add AI capabilities as a layer of intelligence. “We have an overflow of ideas from real-life, customer-driven, use cases,” says Quist. Being a former trainee himself, he turned to the PDEng Software Technology (ST) program to help contain the overflow. “At Philips, we always like to work with talented young people, and we can provide building blocks of the right size for them to innovate and build on quickly, without a learning curve of several months. And the trainees get the chance to work in a professional environment. We sensed a win-win.”
Not one but three solutions
The PDEng ST is a two-year, salaried, post-master technological designer program on a doctorate level for MSc graduates with a degree in computer science or a related field. It prepares the trainees for a career in industry by strengthening their theoretical basis and confronting them with challenging problems from industrial partners. Yanja Dajsuren, PDEng ST Program Director: “In our program, we aim to find a variety of clients that offer complex system/software architecture and design-related challenges. Our trainees learn to develop innovative solutions meeting industry standards, while mastering all the aspects of teamwork, different roles and professional skills.”
Recently, the PDEng ST has expanded its program, covering the field of AI via courses, workshops, hackathons and training in companies. “Last year, 25 percent of our graduation projects were about data-driven architecture and intelligent systems, in collaboration with, to name a few, Hendrix Genetics, EIT Digital and Thermo Fisher Scientific,” clarifies Dajsuren. “According to the McKinsey Global Institute, the rapidly expanding field of AI is estimated to create an additional 13 trillion dollars of value annually by the year 2030. We’re confident that our program will contribute to the development of innovative AI platforms, tools and methods, and strengthen the collaboration between industry and academia.”
With Philips, eighteen PDEng ST trainees from twelve different nationalities got a crash course in the booming domain of healthcare AI. “We briefed them on the use cases for telehealth and remote services, we supplied our platform for remote communication, we referred them to the AI cloud services of vendors like Amazon, Google and Microsoft, and wished them an inspiring journey,” Quist explains. His colleague Zoran Stankovic, who as an architecture lead provided technical support along the way, adds: “We laid down our expectations, but not what to do exactly. That they had to figure out by themselves.”
In two stretches, two weeks at the end of last year and six and a half weeks in February and March, the trainees completed the assignment. Using an Agile approach, they devised not one but three solutions. One group developed a condition monitoring system based on Microsoft AI services to analyze a video stream and generate a history of faces and a graph of emotions, which can be used by caretakers to track their patients’ well-being over time, or by tech support to measure customer satisfaction. The second group came up with a Google AI-based solution to remove private, on-screen patient information from video conferences between hospital staff and support personnel. The third group again employed artificial intelligence from Microsoft for real-time speech-to-text conversion, translation and annotation, providing remote communication partners on the fly with transcripts, subtitles and extra contextual information.
Experiences gained
PDEng ST trainee Robin Mennens was assigned project manager. His main task was to facilitate the teams and keep everyone productive from the get-go. “We started with five teams. Following a two-week brainstorm, they presented their ideas to Philips, which selected the three most promising ones. After a break in January, we reconvened and reorganized into three groups according to everyone’s preferences and technical skills. In the remaining six weeks they developed the use cases. This project shows what a group of talented individuals can achieve when they’re allowed to work autonomously and make their own decisions.”
For the three team leads, one of the main challenges was to set the project boundaries. “The assignment we were given was very broad, too big actually for the time we had,” explains Hossain Muctadir, who led the condition monitoring team. “So in collaboration with Philips, we scoped it down until it was feasible, prioritizing the functionality into must-have, nice-to-have and optional.” Meram Salih, leader of the de-identification team: “We had to solve the client’s problem while keeping it realistic for us to do. I learned that when you listen to the people in your team and reach to decisions together, everything falls into place.”
With twelve nationalities in one team, communicating with each other was a challenge as well. “It’s really interesting to see how people from different parts of the world interpret things differently,” observes Muctadir, a Bangladeshi by birth. Palestine-born speech AI team lead, Yousef Fadila, agrees: “Never assume people are on the same page unless you have put them on that page and have verified that they’re really there. In every daily stand-up, I checked if everybody was aligned with one another.”
The architect role was new to many team members, Nobahar Arian being one of them. “I used to work as a back-end software developer in Iran. In this project, I had to show leadership skills. AI is very interesting to developers and it comes naturally to them to focus on the features. Being software engineers prior to the program, this is also what we did at the beginning of the project. But as an architect, you have to make informed design decisions and you need to control your team to follow you. Thanks to all the experiences gained here, I now know exactly what to do.” Vahe Pezeshkian hadn’t been an architect before either. “Looking at a problem from a different perspective was eye-opening for me. It was also really fun to see our activities get more orchestrated and finally see the teams flow in the same direction.”
Agile was a first for some trainees as well. “I had to get familiar with the process, which is not that easy, as it’s not exactly set in stone,” explains Konstantinos Manos. “Every company has its own version, so there’s a lot of ways to do it, and a lot of documentation out there. I had to find out what fitted our team best.”
Further development
The trainees’ responsiveness and self-reliance amazed Marcel Quist and Zoran Stankovic of Philips. “After we had given them the broad assignment, they took several steps that were very surprising to us, in a positive way,” Stankovic recalls. “Despite the initial vagueness, everything went pretty smoothly.” Quist: “They already showed eighty percent of the functionality after only two weeks of development.”
The expectation of a win-win panned out. “The PDEng trainees had the opportunity to work with the technology building blocks we have to offer for remote communication and AI services, and they experienced our way of working: no longer technology driven, but use-case driven. That’s a valuable combination,” Quist thinks. “There’s also a win for the ecosystem as a whole: the project has shown that universities can play a role as a solution partner, contributing to the total solution and taking ownership.”