IEEE Software - The Pragmatic Designer: Why Is It Getting Harder to Apply Software Architecture?
This column was published in IEEE Software, The Pragmatic Designer column, July-August 2021, Vol 38, number 4.
ABSTRACT: By the late 1990s, we understood the basic abstractions of software architecture, and it was possible to choose an architecture that suited the problem at hand. Over two decades later, it’s difficult to apply these ideas to our projects, but why? Because we have changed our philosophy to iterative development using factory metaphors, and have reinforced that change with tooling. Developers now focus primarily on incremental changes, and secondarily on the overall design or architecture. These circumstances are inhospitable to the holistic system reasoning that architecture requires, and can explain why so many teams today regard technical debt as their biggest challenge.
Today, we find ourselves in a surprising situation: We understand software architecture fairly well, but we find it difficult to put it into practice. This is because most software developers work within iterative processes that focus attention on what’s new, and because factory-inspired metaphors encourage ever-quicker movement from feature requests to deployed code. How did that happen?
A big idea from Bertrand Russell’s A History of Western Philosophy is that “[T]he circumstances of men’s lives do much to determine their philosophy, but, conversely, their philosophy does much to determine their circumstances.”  Around the year 2000, we changed our philosophy of software development in two ways. Those two changes, in turn, led to the circumstances that we see today.
Two Philosophy Changes
The first change in philosophy was in the object-oriented community’s attention, from design to management. In the 1990s, the discussion was about how to design object-oriented systems. Thought leaders prescribed engineering activities, like choosing objects, modeling the problem domain, expressing it with a modeling notation, allocating responsibilities, and describing contractual behavior including preconditions and postconditions. Patterns were the means of communicating how to design.
Around 2000, thought leaders shifted their attention to management activities: organizing meetings, interacting with the business, seating arrangements, and, most importantly, the use of iterations. Despite broad agreement that iterative processes worked better than waterfall processes, most teams in the 1990s were still using waterfall. My inference is that agile advocates found it hard to apply their design ideas under these circumstances, so they set out to create circumstances suitable for object-oriented design and programming.
The second change in philosophy was the popularization of factory-style production metaphors, including: automation, minimization of work in progress, and reducing cycle times. These ideas found a natural home in the emerging practice of DevOps. The traditional split between software developers, testers, and system operators was joined into a single role. This created the desired circumstances: a developer with this combined role had the perspective and ability to identify inefficiencies, automate them, and seek continuous improvements.
These two changes in philosophy led to the intended changes in circumstances, but also other changes. Today, developers find it easier to focus on incremental changes instead of the system as a whole, and they seek improvement by reducing the time between a feature request and its moving to production. These circumstances make it easy to pile up technical debt, despite warnings and advice. The rest of this article shows how these circumstances arose: through changes in the dominant decomposition, reinforcement by tooling, overloading of developers, and the inevitable dominance of quantifiable metrics over intangible design concepts.
Please read the rest on the IEEE website.