With Achmea and Prorail as long-time customers and Thermo Fisher Scientific as the first Brainport client, Axini is pushing hard to make a name for itself in high tech as well. Coming up to the year of their company’s fifteenth anniversary, Machiel van der Bijl and Menno Jonkers share their vision of achieving fault-free software through model-based testing.
Axini – isn’t that that company from Amsterdam doing some complicated model-based software engineering stuff? This is how founders Machiel van der Bijl and Menno Jonkers were typically encountered after taking the A2 motorway, or the train, down to the Dommel valley. It’s also exactly the sentiment they’re determined to turn around. “We’ve set our sights on that market,” says Van der Bijl. “Our target for 2022 is to add at least three high-tech companies to our customer base.”
Three high-tech companies next to Thermo Fisher Scientific, that is. The Eindhoven site of the American life sciences specialist recently became Van der Bijl and Jonkers’ first Brainport client. It’s going to use Axini’s model-based testing approach to further increase its software quality and streamline the development process. “They have a grand vision and an extensive program for applying model-driven techniques,” explains Jonkers. “They saw the added value of our approach and made us part of their plans.”
An important part of Axini’s added value for high-tech companies, Van der Bijl points out, is the ability to test the software before the hardware is available. “Engineers don’t want to wait until system integration for any errors to manifest themselves. Detecting problems earlier can speed up the process tremendously. We can help achieve that.”
The asset managers at Robeco, the insurance salesmen at Achmea and the railway monitors at Prorail, to name a few, are already well aware of what Axini can do. “At Prorail, for example, we’re model-based testing parts of ERTMS, the European Railway Traffic Management System,” illustrates Van der Bijl. “That project is months ahead of schedule, we’ve saved thousands of testing hours and had virtually no integration problems.” Supported by partners ICT Group and Sioux Technologies, Axini is now pushing hard to make a name for itself in high tech as well. With the model-based way of thinking being well-rooted there, Brainport seems ripe for the picking.
“Our approach creates a model that describes the environment of the software that’s being developed, with all the connecting interfaces,” explains Van der Bijl. “This environment can consist of other software modules – in the case of Prorail, for example – or hardware components – in the case of Thermo Fisher. From the model, test cases are automatically generated, which are then automatically executed against the software under test, giving the development team immediate feedback on the functional behavior, both the happy flow and the bad and sad weather.”
Van der Bijl distinguishes three flavors of model-based testing (MBT): before, during and after development. “MBT after the fact is a kind of insurance. You’re moving software to production and want to be sure that it works. This is where we started with Prorail. Over the years, however, we’ve learned that, although it’s okay for insurance purposes, it yields a very bad return on investment. It almost always uncovers problems that really need solving before going to production, incurring extra work and thus additional costs that could have been avoided – which is also why project managers don’t like it.”
“MBT during development provides a better return on investment,” Van der Bijl continues. “Making a model while you’re building the software allows you to find and fix problems much earlier, instead of at the last minute. You won’t catch errors in your specification, though. To uncover these, you need to model right from the start.”
It won’t come as a surprise that MBT before development is Van der Bijl’s method of choice. “Expressing the specification in a model, rather than just on paper, gives you the best ROI by far. It results in a huge improvement in software quality, if only because you’re scrutinizing every interface. At the same time, it brings the biggest change. Managers generally aren’t used to having those kinds of discussions at the beginning of a project. And it cuts across disciplines, as test engineers also start making models, which may cause confusion. We’ve come to realize that for companies adopting model-based testing, it’s much more a process change than a technology change, and we’ve adjusted our approach accordingly, focusing more on the new way of working.”
“I’ve studied computer science at the University of Twente and ever since my graduation project, building a neural network simulator, I’ve been fascinated by the question: how can I be sure that my software is correct?” recalls Van der Bijl, asked about the origins of his passion. “In 1997, I started at Utopics, the precursor to IT giant Topicus, where I had the opportunity to dive deeper into test automation. But after a couple of years, I realized that it just doesn’t scale and I set out to look for better ways to achieve fault-free software.”
This quest brought him back to his alma mater in 2002. “At the University of Twente, they said they had long solved it, using formal methods, model checking and model-based testing. They invited me to come to Twente and do a PhD on the subject, which is what I did,” Van der Bijl goes on to tell. “But as the project neared its end and I proposed to apply my work in practice, realizing that it was promising but still theoretical, they dismissed the idea, saying that was something for industry. So I returned to industry, founding Axini in 2007 – only to finish my PhD a couple of years later.”
Van der Bijl reached out to his friend and former Utopics colleague, Jonkers. He had also studied computer science with a specialization in AI, but at the Vrije Universiteit Amsterdam. “After graduating in automatic credit card fraud detection and working at Interpay for a year, I joined Utopics’ financial automation branch in 1996. Machiel and I became good friends and we kept in touch after I moved on in 2000. I went to Tryllian, a high-potential dot-com startup in distributed agent technology, where I stayed for five years, during which the company crashed and restarted. I was doing interim management work when Machiel came calling in 2007.”
Jonkers gladly accepted the invitation to build up a company together. “Machiel told me that he had received the open-sourced TorX tooling as a kind of dowry from the university and that he wanted to industrialize it. He had also just won a €25k valorization grant from the Dutch technology foundation STW. I was done with being a gun for hire and with his enthusiasm, he sold me on joining his mission for fault-free software. We incorporated on the last day of 2007.”
TorX, from “T or X,” true or false, proved to be a very powerful tool, but not for every problem. “It takes the data of the system to be modeled and tries to capture it in 0s and 1s, as the name suggests. When that’s possible, the tool yields amazing results. That was a real eye-opener for us. There was this problem of faulty software and we had something of a solution,” Van der Bijl notes. “In practice, however, there’s also a lot of data that can’t be captured that way, like dates and text.”
So, in practice, Axini had to extend its toolbox to contain more than just the TorX screwdriver. Step by step, new instruments were added, both formal and non-formal. “We built our own solutions on the go. Along the way, we created a nice mix of formal tooling and our own brute-force methods and other smart tricks – at the university, they called it cheating, but hey, it worked,” Jonkers says, smilingly.
A mix under the hood, one product to the outside world. “Depending on the needs of the customer, projects would lean more towards the formal side or more towards the other side. But we always made sure to bring everything together in a single offering,” Jonkers points out. “From day 1, we set out to be a product company.”
Aspiring to be a product company and actually being one are two different things, the Axini entrepreneurs learned. Jonkers: “Like many other techies, we’ve been overly optimistic about the maturity of our product. A couple of years in, we already thought we were there, but it turned out we still had some way to go. Our first customers didn’t hire us for our product but for us to come and employ it for them. We often got asked the question: does it work because the tool is so good or because you’re so smart? Slowly but surely, we’ve matured the product, readying it for use by people without a PhD and including stuff like training and support – the necessary prerequisites for companies to do everything themselves.”
The Axini approach has come to rest on three pillars. “It starts with modeling, or rather computer-aided specification,” clarifies Van der Bijl. “The next step is no-code test automation, as people like to call it these days – the system is tested fully automatically, without having to create test data or scripts by hand. Finally, application code is generated from the model – this we do mainly with our customers in finance, not yet with tech clients like Thermo Fisher.”
With Axini, according to Van der Bijl, the domain expert is in the driver’s seat. “Our approach is based on a Lean philosophy of process optimization. How do you get the best specification? Many organizations suffer from a Chinese whispers situation there, in that they have several architects coming in from different angles, each drawing up their own piece of the spec. We optimize this by putting one domain expert at the steering wheel right from the start and having them build the models, making explicit the implicit assumptions and clearing up vaguenesses and inconsistencies.”
The software can then be built incrementally, Agile style. “During the sprint, you’re implementing and modeling in parallel,” Van der Bijl details. “The definition of done then becomes: does it pass the model-based test? If so, you’ve implemented this piece correctly. You can do this when building from scratch, but usually, you’re extending existing software, in which case you just model the new addition.”
The pinnacle being the ability to produce verifiably correct software where you can prove system properties. “That’s the holy grail,” acknowledges Van der Bijl. “Each line of code you add is automatically checked against the requirements and when you’re done, fault-free software is generated with the push of a button. Unfortunately, our models are simply too big for the existing model checkers. It’s the infamous state space explosion – and not just by a factor ten. As a workaround, you can have two experts independently make a specification model and an implementation model, and check those against one another – our toolset is really good at that. If the implementation model implements the specification model, you’re good to go. Meanwhile, we’re exploring different avenues to get closer to that grail, including this new technology, the Z3 theorem prover, which looks very promising.”
No-code, Lean, Agile – Axini is certainly benefitting from the sign of the times. “The world has evolved. The Scaled Agile framework didn’t exist when we started almost fifteen years ago; now, almost all our customers are using it,” observes Van der Bijl. “We used to have a hard time explaining what we do. Test automation? People had heard about it, but they didn’t exactly know what it was and they certainly weren’t doing it. With the advent of Scaled Agile, we can talk about how we’re contributing to shift-left and quality built-in and that attracts immediate attention. This has definitely made it easier to sell our story.”
“We’ve also gained more insight into who we need to sell our story to,” adds Jonkers. “In the beginning, we targeted test engineers because we thought they would benefit the most from our tooling. We couldn’t have been more wrong. They’re not the ones with a problem – unit testing is doing the job for them just fine. You need to look for the real pain, for the project managers who are struggling to get to a working deliverable, for the product owners who are trying to fit a huge wish list into two or three releases a year, for the managers who have lost control of their software projects. You need to find the people who have a reason to change. They’ll become your champions and sell your story for you.”
One of the key lessons Van der Bijl and Jonkers have taken home from their many trips to the south of the Netherlands is that in the Dommel valley, before anything else, they needed a partner to get a foot in the door. “For a long time, we were convinced that we could do it on our own. We did some research projects with Brainport partners like ESI and Philips, we did some proof of concepts. But each time we wanted to do more with a big company, we got the door slammed in our face. ‘It’s super interesting,’ we were told, ‘but we’re not going to do this with you. You’re too small.’ Still, we thought we knew best,” Van der Bijl looks back. “Until 2018. With becoming a scale-up, it dawned on us that this idea of doing everything ourselves didn’t scale. Serving the likes of ASML, Philips and Thermo Fisher ourselves would require hiring a hundred consultants, if not more. We understood that we would have to team up with partners who know their way around the Dommel valley – partners like ICT and Sioux.”
Employing 21 in 2021, Axini hasn’t exactly skyrocketed since its incorporation at the end of 2007. “If you would have told me back then that that’s where we would be at in fifteen years’ time, I think I would have been disappointed,” concedes Jonkers. “Looking back now, however, I’m actually very pleased with the way we’ve pioneered and organically grown while remaining self-funded and super flexible. Since 2018, I feel we’ve really been going full steam ahead. We’re on our way to 30 people and our customer and user base are growing rapidly.” Next stop: Brainport.
This article was written in close collaboration with Axini.