Making electron microscopes smarter, Tesla-style

“In Tesla’s electric cars, the hardware is deliberately over-dimensioned to leave room for adding functionality later through software. This is a strategy from which we can also greatly benefit in our electron microscopes,” argues Yolanda van Dinther of Thermo Fisher Scientific. “By putting more sensors in our systems, we can anticipate new possibilities. We’re not there yet, but it’s a nice aspiration.”

For the past three years at Thermo Fisher Scientific, Van Dinther has been leading a growing group of software specialists driving the digital transformation of the company’s electron microscopy division. Her team of 26 is contributing globally. In addition to her home base of Eindhoven, she has people working in the Czech Republic (Brno), France (Bordeaux) and the US (Hillsboro).

“Software, data and artificial intelligence are our areas of expertise,” Van Dinther summarizes the work of her team. On the one hand, this involves ease of use. “Just as it’s a piece of cake these days to shoot the best pictures with your cell phone, we’re also incorporating more and more intelligence to make the lives of our microscope users more comfortable through automation. Software, data and AI are how we’re realizing that increasing support.”

On the other hand, it’s about the transformation to more software-driven innovation, Van Dinther explains. “Microscopy has traditionally been a discipline of physicists. Innovations have always been primarily physical in nature and rooted in hardware development. We’re shifting that to a way of working that’s much more integral and software minded.”

Hooked on leading

Van Dinther has more than earned her spurs in high-tech software. After graduating from Eindhoven University of Technology, in the footsteps of the famous Edsger Dijkstra, she went to work for Philips in 1987. As a software engineer, architect and consultant, she was involved in a variety of developments there. “I modeled the behavior of electronics components at the predecessor of the CFT, I worked on the Carin navigation system at TASS, I set up a service for software architecture consultancy within Research and I continued that at Atos Origin.”

With her move to Philips Semiconductors in 2001, Van Dinther’s career took a decisive turn. “The idea was to continue on the same footing in software architecture, but then my new boss asked me to lead the entire software team. I never thought that could be so much fun. Since then, I’ve been hooked on leading technical teams. A few years ago, I even obtained a post-bachelor’s degree in counseling psychology to further solidify my coaching skills.”

As a manager, too, Van Dinther has learned the ropes. “At Semiconductors, I led the embedded-software team for the Ultimate One Chip and LED One Chip families of TV chipsets, while at Healthcare, I managed platform development, among other things. In a foray into Cimpress, the parent company of Vistaprint and Drukwerkdeal, I headed the integration of several production printing software lines. Back at Philips, I was in charge of software development at Handheld Diagnostics until that activity was unfortunately discontinued.”

Van Dinther is apparently attracted by big software challenges. “That has been the common thread in my journey through a lot of fields,” she admits. Thermo Fisher Scientific is her most recent stop. “I joined the company five years ago as a strategist overseeing the challenge of bringing two machine platforms together and connecting them to application software. For the last three years, I’ve been building an expert team around data and connectivity.”

Predictive maintenance

“We’re getting more and more data from our electron microscopes,” Van Dinther notes. “We do that by connecting to our customers via the cloud. In my team, we’re therefore very much concerned with security. And because the data remains the property of our customers, we also make sure that there are clear agreements in place about its use.”

At the moment, that use is mainly limited to reading out error messages and analyzing the status and health of the microscopes. “From the error codes, we can figure out what went wrong. Based on the status information, we can do predictive maintenance. We also make these insights available to our customers. In a dashboard, they can monitor the various components in their microscopes. If one of them is close to breaking down, we can detect that, warn the customer and send a service engineer.”

For now, the microscope image data from customers is mostly being left untouched. “Handling that data is extremely challenging,” Van Dinther explains. “Those images are huge, they’re actually more like movies, inconveniently big to transfer to the cloud. Plus, they often contain valuable information, which makes sharing them a lot more sensitive. In fact, some customers prefer not to be connected to the internet at all. When we retrieve data, there’s usually an IT department in between anyway, double-checking whether the data can really leave the premises.”

At several customers, Thermo Fisher Scientific does work with the image data on-premises. “There, for example, we deploy AI training techniques to make it easier to use the microscope.”

Up and running

In addition to reading out error messages, status information and (limited) image data, Van Dinther and her team also help retrieve machine configurations. “We too are making our software increasingly modular and the updates smaller and more frequent. To prevent a breakdown after a new release, it’s important to carefully check for compatibility with the existing code. From the configuration we read out, we can deduce which software, firmware and hardware versions are running in a system. Based on that, we can then determine which updates we can deploy. Especially with our diverse installed base, that’s crucial to know.”

“Another of my team’s responsibilities is to provide a standard mechanism to keep our systems up-to-date and up and running, whether it’s one of our service engineers installing the new software or the customers themselves,” Van Dinther goes on to explain. “Such a mechanism is essential as more and more features get implemented in software. In the past, each product was more or less unique. The focus was on the hardware and once delivered, the functionality didn’t change. The digital transformation turns that completely on its head. From one-time revenue from hardware, we’re going to recurring revenue with software.”

It’s a journey with an ambitious objective – incorporating hardware headroom to accommodate the future addition of functionality through software. “Our initial focus is on making additional data sources available, for example by including extra sensors,” Van Dinther indicates. “With more data, we can run more elaborate analyses and further improve our systems in areas such as predictive maintenance and user-friendliness.”

Thus, Van Dinther and her team are working hard to make Thermo Fisher Scientific’s electron microscopes even smarter with software, data and AI. “We’ve been collecting data from our systems for at least ten years, so that’s something we already have quite some experience with. We have more and more data at our disposal, from more and more systems. We’ve only just embarked on our journey toward software-driven innovation, but we’ve made a good start. That has been our primary focus for the past two years, and those efforts are starting to pay off. It’s great to see a traditional hardware company go through such a cultural transformation.”

At the Bits&Chips Event on 12 October 2023 in Eindhoven, Yolanda van Dinther will elaborate on the application of data science at Thermo Fisher Scientific.