Co-designing hardware and software for a sustainable future of AI

In its annual Digimeter, research institute Imec surveys a representative portion of the Flemish population regarding their views on technological developments. The 2023 edition included some questions about artificial intelligence, and the responses were in line with expectations, shares Ben Stoffelen, the department manager of AI at Imec. “AI has made significant inroads into the general population: surveyed individuals replied that they understand it, use it and believe that it makes them more efficient. This is primarily due to ChatGPT, which reached a high adoption rate in an incredibly short timeframe.”

Another domain where AI is gaining adoption is autonomous driving, where vehicles use AI on the edge to process data from an array of sensors. “In both the Generative Pre-trained Transformer instance and the case of edge AI, the ongoing advances have been enabled by the introduction of high-powered GPU chips,” Stoffelen elaborates. “Yet, both training and inference of these AI models consume copious amounts of energy, resulting in substantial environmental impact. As an illustration, researchers have estimated that training OpenAI’s GPT-3 large language model emits as much CO₂ as driving a car to the Moon and back. If GPT models become mainstream and if autonomous driving achieves a breakthrough, we’ll have a huge sustainability problem.”

Co-designing hardware and software

It would be unfortunate if our society were unable to reap the benefits of these powerful AI models. Hence, researchers at Imec’s AI department are looking for methods to keep developing better AI models in an environmentally friendly manner. Stoffelen continues: “One method is using computational devices that are less generic. Given that CPUs and GPUs are general compute solutions, they aren’t particularly energy-efficient: they’re designed to execute a diverse set of generic processor instructions. But what if we designed a chip specifically tailored for an application, like a heart rate sensor or a sensor in an autonomous vehicle?”

According to Stoffelen, achieving the significant improvements in energy efficiency that we need to keep improving AI applications necessitates a co-design approach for hardware and software. “AI application developers shouldn’t wait until Nvidia releases their next accelerator chip. And chip manufacturers shouldn’t solely aim to create more powerful AI chips. If we want to reach our sustainability goals and create better AI applications, we need to address these issues from the root. This means developing purpose-built hardware for the application. This will permit us to incorporate AI within all types of smart sensors.” Imec regards accelerating this co-design approach as part of its mission.

Breaking away from Von Neumann

Furthermore, Stoffelen posits that to develop these application-specific, energy-efficient designs, we must break away from the conventional Von Neumann architecture that we’ve been accustomed to since the 1940s. “There exist numerous alternative computer architectures, providing us more flexibility to engineer efficient chips. We’re examining neuromorphic chips, which closely mimic how the human brain works. Another promising approach involves analog processors: using the physical characteristics of analog devices for computation. One of our ongoing research programs aims to build AI accelerators based on an analog computer architecture. By embracing new architectures, we’ll potentially reduce an AI model’s energy consumption down to nanojoules or picojoules per inference.”

If we manage to address this sustainability issue in AI, Stoffelen predicts the emergence of numerous new applications and breakthroughs in stagnant domains. “Progress in autonomous driving is currently slowing down. Although we have robotaxis, they only fare well because they’re restricted to a geofenced and relatively controlled environment. For ordinary cars, the technology plateaued at adaptive cruise control, lane following and similar features. However, cars eventually encounter unpredictable scenarios, such as harsh weather conditions or pedestrians dressed in dark clothing, leading to detection problems. Manufacturers are attempting to overcome this by adding more and more sensors, but this strategy isn’t feasible. Thus, we require energy-efficient AI to better use existing and novel sensor data.”

This article was written in close collaboration with the Advanced Engineering expo.