Do Agile and architecture really clash?

Some years ago, I attended a graduation ceremony at a university and started a chat with a professor in software architecture. When he learned I lectured about Agile principles, his immediate reaction was that he and I were teaching conflicting concepts. Although I’m aware of the many misconceptions about Agile, I still got a bit irritated to hear this from a professor in software engineering.

There’s a widespread impression that Agile is a trial-and-error method – no plans, no structure, and we’ll see what and when the activity will deliver. It’s true that with an Agile way of working, the teams often deliver something quite different from what was expected at the start. But that’s on purpose; it’s because the expectations change! As a result of a strong collaboration with various stakeholders and a continuous discussion on what deliverables are really needed, the scope of the activities is continuously adapted rather than just blindly executing the plan drafted a long time ago. To reduce avoidable waste, decisions are postponed until they’re really needed.

Sometimes, however, we need to make early decisions, even if that would potentially lead to waste. In Agile, contrary to what people may think, we think carefully and use the available information to come to the best possible decision, even when it’s early in the process. Agile is just a way to deal with the intrinsic uncertainty in many things we do rather than deny and neglect it or recover in a late phase when waste can no longer be avoided. It’s about being ready for change and this requires a lot of upfront thinking.

A lot has been written on Agile architecting, an adaptive process of continuously updating the architecture and postponing architecture decisions to avoid waste. Moreover, in Agile, the responsibility for architecture isn’t just with a few architects throwing a design over the fence and leaving the implementation to a team; architecture and implementation are a joint responsibility. I won’t go into detail here, and instead focus on one aspect, which some people may simply consider good craftmanship, but what I like to refer to as the ability to deal with uncertainty.

More than a decade ago, I was involved in the development of a new consumer navigation device. The software on that device consisted of three parts: the navigation core with all the business logic, the navigation user interface and the user interface dealing with non-navigation topics like connectivity, subscription management and downloading new maps. These were all delivered by different teams.

I remember that one day, the hardware design department had decided – without looking into the possible software consequences, a common phenomenon in many companies – to implement a last-minute change and modify the pixel aspect ratio of the display. In those days, this had quite some impact on both user interfaces. The navigation UI team changed some settings and within hours, the user interface looked nice and neat again as expected. The other UI team struggled; their text was all over the display and not aligned at all. The navigation user interface clearly was more robust against these late hardware changes and designed with the possibility of changes in mind.

For me, this is also an important aspect of product architecture: it should be robust against changes. Textbooks talk about modifiability and often start with the question: what’s likely to be changed? That’s a valid question. However, architecture must also deal with unknown unknowns, and we may face some unexpected changes. But even when something isn’t likely to be changed, it’s good software engineering practice to be explicit on our assumptions and minimize the dependencies on these assumptions. And don’t assume something will never change.

When I look at the curriculum of university software engineering programs, they’re a lot about how to design complicated architectures for well-defined problems and even create robustness against unexpected inputs, sometimes using formal methods to prove correctness under all circumstances. However, I see few opportunities for the students to practice design for changing requirements. Why not let them take existing software and implement a new or changed requirement and identify what past architectural design decisions have now blocked an efficient implementation of the change? That’s probably closer to the real world of software engineering than designing from scratch. It’s closer to the Agile world, where we organize ourselves for uncertainty and are ready for change.

So, I agree with the professor that Agile principles conflict with the current curriculum in many universities. But isn’t he in the driver’s seat to make that curriculum a better fit with the real needs of the industry?