Alexander Pil
23 February 2022

Are artificial intelligence and machine learning hypes? Sometimes it sure seems that way. Consultants and specialists promise you the world, but in the day-to-day life on the factory floor, those predictions rarely pan out. Omron’s Tim Foreman has experienced hands-on where things go wrong and now knows how to do it right. To Bits&Chips, he tells the real story.

For Omron, the journey into the land of artificial intelligence began about six years ago. After the founding of the Jheronimus Academy of Data Science (JADS) in ’s-Hertogenbosch in 2016, Tim Foreman immediately sought contact. As R&D manager of the local Omron site, he saw many opportunities for AI, machine learning and data science. “The beauty of our location in ’s-Hertogenbosch is that we have R&D, engineering and production under one roof,” says Foreman. “Beautiful machines, with real problems. It’s a playground for data scientists.”

Foreman and JADS program director Stef van Eijndhoven opted for a PDEng construction, the industry-oriented variant of a PhD track. “I can wholeheartedly recommend that to anyone who wants to get started in this field,” says Foreman. “Such a student is supported by the university and brings a lot of knowledge to your company.”

With a PDEng student from JADS, an interested internal engineer and himself, Foreman had his team complete. The only thing they needed now was a good problem they could focus on. “I spoke to some operators in the factory,” Foreman recalls. They pointed him to an issue in the final inspection of a printed circuit board. “To check whether the board is good, we test it before it goes into its casing. We do this on a testbed with contact pins. Over time, however, those pins wear out and you see more tests fail. Is it a failure in the PCB or a worn contact pin?” The task Omron gave itself was simple: can we predict the wear of the contact pins? In other words, what’s the remaining useful lifetime?

Keep it small

Although the testbed already logged quite a bit of data, the researchers could do very little with it. “I had been warned that 80 percent of data science time is spent cleaning, understanding and completing the data,” Foreman notes, giving an example. “If a service engineer replaces one of the pins and doesn’t report it properly, you suddenly see an inexplicable kink in the data. That maintenance may be listed somewhere – in a digital system if you’re lucky – but it’s not linked to the data you’re looking at. Good luck making sense of that.”

BCe24 save the date

Due to the dirty, incomplete data, among other things, Foreman and his team were ultimately unable to make a good prediction of the remaining lifetime of the test pins within the set period of the project. “It also had to do with the fact that it’s very difficult to keep focus. Once you open the hood of the machine, there’s so much data thrown at you, and you see so many strange things happening that you’re easily distracted. Especially if you’re as curious as I am,” Foreman observes, laughingly.

Another challenge was that it took Omron a while to get to the right problem. “Initially, we wanted to increase the throughput of the entire assembly line,” recollects Foreman. “At the beginning, you think: how hard can it be? Well, it turned out to be anything but simple. So in the end, we made it very, very small.” That’s one of Foreman’s most important lessons from that first project: “Make the task much smaller than you think is necessary.”

Omron Tim Foreman 01
Credit: Omron

The right picture

Despite the somewhat disappointing result of that first experiment, Foreman was quite happy. “We learned a lot from that first step,” he says. Since then, he knows how crucial it is to gain good insight into the data. “If you’re a beginner in data science, forget artificial intelligence and machine learning for a while. Start by gaining insight into the data. For example, is the flow of products a nice, flat distribution? Or are there all kinds of hiccups? Such simple questions often provide enough leads for improvement. Only after you’ve solved those issues and have a smooth process in place, it makes sense to start with AI.”

The first visualizations showed, among other things, that not all operators delivered the same number of products. “That’s to be expected, of course, but you’re treading on thin ice with that,” Foreman warns. “In Germany, it’s even forbidden to collect such data so that an employer isn’t tempted to judge his employees on it.” One of the operators scored well below average. “We solved that by open and honest conversations. It turned out that the operator had made a process step unnecessarily complex for himself. We’ve fixed that – everyone happy.”

According to Foreman, the importance of good visualization can hardly be overestimated. “With the right picture, you always generate a reaction, from the operator, the engineer or the factory manager. Hey, what’s going on? If that response doesn’t come, then it’s not because of the recipient, but because the data scientist has made the wrong visualization.” He still sees it regularly during company visits. “They’ve made a flashy dashboard with all the colors of the rainbow. A true Picasso, but you get tired just looking at it. And no one can tell what’s actually happening.”

Seeing the pattern

Such a wow moment certainly came with the visualization in Omron’s second project. “We examined the visual inspection of an SMT line,” tells Foreman. “Has enough solder paste been applied? Is the component in the right place? Is the solder good? We already did an extensive quality check with a lot of data logging. We asked the second PDEng student to take a look at that data.”

The result was shocking. “He’d made a heat map of the PCB to show where things often go wrong,” says Foreman. “Of course, you’d expect a more or less even distribution, but it turned out that all the mistakes were happening in one place. A big red dot!” Closer examination revealed that the PCB wasn’t properly supported in that spot, causing it to sag.

Didn’t any of the operators or engineers notice that the production of those boards always failed at the same point? “No, that’s the interesting thing about data science – you shouldn’t apply it to problems that you stumble upon. First, you have to fix all the errors everyone can see. Only then thorough data analysis is useful, for issues that a human doesn’t see or of which he doesn’t recognize the pattern,” Foreman answers. In a subsequent project, for example, the data showed that a machine slowed down by one second every week. “We only found that out when we compared the data from the past few months. No one had noticed it, but with the right visualization, you could see it straight away.”

Today, many machines in the Omron factory have a monitor that shows how the machine is performing, compared to historical data. Foreman: “It’s a cheap way to start with data science in your production. Only then you can start thinking about machine learning. Because it’s an obvious next step to monitor all that data with algorithms and to set alarms if irregular behavior is detected.”

Omron Tim Foreman 02
Credit: Omron

Lost sensor

After a number of instructive projects at his home base in ’s-Hertogenbosch, Foreman dared to apply the gained experience at an Omron factory in Italy. “By that time, we’d captured our knowledge in an AI controller, a data logger that could write all data neatly and with a timestamp in a database. The system measures industrial signals, makes visualizations and recognizes patterns,” explains Foreman.

This AI controller was attached to the Italian production line. “Every now and then, their machines malfunctioned and they had to restart,” says Foreman. “Because products roll off the line every second, such interruptions were disrupting the output.”

The data that the AI controller showed on the screen immediately gave many insights. “We saw some strange things, which we verified with the local engineers,” Foreman says. For example, it turned out that a position sensor had shifted and there was a bug in the software that sometimes caused asynchronous process steps that would upset the machine. “We also saw a flat line on a sensor output. Weird, of course, because you’d expect something to happen every now and then. It turned out that there was no sensor at all, just a loose cable.”

Within a few weeks, Foreman and his colleagues were able to uncover all those errors simply from analyzing the available data. “The trick is to get the right data. That’s only possible if you work together with local engineers, operators and mechanics. You need their knowledge to be able to properly interpret the measurements and data,” Foreman points out. Ultimately, all improvements resulted in 4.5 hours of extra production time per month.

Complicating variations

In the same factory near Rome, Omron also produces components for the automotive industry. To meet the high demands of that market, many of the products are checked extensively. But because they’re mass-produced, 100 percent control is out of the question, so the occasional faulty component would slip through.

“A clear problem with a clear financial pain, because recalls are expensive,” Foreman states. Together with the local engineers, his team set up an experiment in which the AI controller was again used to collect data. “We also added a sensor. Normally, I prefer to avoid that because it entails risks. In this case, it was inevitable because it’s very difficult to extract good data from pneumatics.”

A force sensor in the head of the pneumatic cylinder provided a wealth of information. In the period of 200 ms that the cylinder puts pressure on the component, two hundred measurements can be made. That paints a clear picture, which means that Omron is able to classify the action as right or wrong. This feature has now been integrated into the production line to such an extent that a trained algorithm detects an error much earlier than an operator ever could.

“There’s a lot of knowledge involved in that,” concludes Foreman. “We’ve discovered, for example, that it matters whether the injection molded part has just been produced or whether it’s already a day old. That gives a different force profile. In practice, industrial processes contain more variations than you thought of beforehand. However, you have to take them into account and that makes the application of data science in industry complex.”

Main picture credit: Omron