ABSTRACT: Lehman’s Laws distinguish between stable S-type and volatile E-type code. Developers can decompose unique problems methodically by seeking standard sub-problems, which reduces complexity and boosts productivity. This illuminates the nature of software engineering, which is needed whether code is written by humans or machines.

Software is written when there’s a novel problem to solve. For example, students write linked list code as an exercise, but industry software developers do not; they reuse the library code. So, how do developers divide a novel problem into many smaller problems, some of which are solved by libraries?

Software developers divide big problems into small problems every day, often without much thought. Unlike algorithms and data structures, this is a skill that tends to be learned through experience, not coursework. Perhaps that’s because we don’t understand it very well. We do have some guidance though.

David Parnas advised us to think of our current software as the precursor to many future systems. [1] We don’t know what the software will be asked to do tomorrow, so we should divide our current system into parts that we might be able to rearrange and reuse in a future system.

Edsger Dijkstra warned us that there are many ways to build something that works, but most of those will be difficult to reason about. The rationale (or in Dijkstra’s case, proof) and the code should “grow hand in hand”. [2] If we want to reassemble several modules into a new system tomorrow, it’s easier if we can understand what each of them does individually, rather than reason about them together. [3]

Like most developers, I’ve read the advice about how to decompose a big problem into smaller ones, and yet in practice, I tend to decompose by gut feel. I’ve recently learned about a technique that makes the choice of decomposition more methodical.

To do this, we must use our experience and ingenuity to recognize stable sub-problems within every unique problem presented to us. That’s because the code for stable sub-problems behaves differently than most code we are familiar with.

This approach differs from typical practice. Few developers look for stable sub-problems, except by intuition. Inexperienced developers, in particular, often take the unique problem they are presented with and decompose it into smaller but still unique sub-problems. It’s custom code all the way down. Beware of this easy path! There are no off-the-shelf solutions to unique problems. When we seek and find standard sub-problems, then we can either reuse an existing library or we can write code to put into a library.

Lehman’s Laws

Perhaps this technique seems obvious but I could not see it until I studied Lehman’s Laws of Software Evolution. [4] Meir Lehman spent his life seeking to describe how large software evolves, much the way the Ideal Gas Law describes how gases behave. When folks like Dijkstra and Hoare were predicting that we’d soon be proving all of our software correct, Lehman was skeptical. [5]

To say that software is proven correct, we need two things: a description of its behavior (i.e., a specification or just “spec”) and a proof that the code conforms to that specification. Lehman’s insight was that not all specs are stable. [6] The code that Dijkstra and Hoare were proving correct, such as sorting algorithms, was anchored to a stable problem, so the spec was stable. It would be very surprising to learn tomorrow that sorting and searching have changed their nature but, in contrast, nobody should be surprised if tax rates change tomorrow. That’s because sorting and searching algorithms are anchored to a stable problem; tax calculations are anchored to an unstable problem.

Lehman called these two kinds of code S-type (“S” for specified) and E-type (“E” for evolutionary).¹ To be S-type, code needs a spec and it must be anchored to a stable problem. The critical part isn’t the existence of a spec because with enough effort you can specify anything, even ever-changing tax rates. It’s the anchoring of a spec on a stable or unstable problem that makes it S-type.

Lehman observed that essentially all of the programs we use are E-type because they solve a problem in the world, and since the world isn’t stable, those programs evolve alongside the world to stay relevant. Lehman’s Uncertainty Principle states that “No E-type program can ever be relied upon to be correct” because you could prove it correct today only for the world to change tomorrow, invalidating the spec. [7]

Lehman observed that our E-type programs are composed of code that can be either E-type or S-type. If we are insightful enough, potentially all of an E-type program can be composed of S-type code. Consider a simple shell script that consists exclusively of calls to existing programs. If each of those programs is S-type, then the only E-type code is the script itself.

This is when I had my aha moment. Lehman wrote about programs² and sub-programs. The S-type / E-type distinction works equally well for any methods, procedures, or functions within a program. I realized that sometimes I’m calling sort() from the standard library – that’s S-type code. Other times, I’m writing custom methods for unique problems – that’s E-type code. Before I call or write any code, I must decompose the bigger problem I’m facing into sub-problems. Often I do this instinctively, without much thought, and I find E-type sub-problems, but the opportunity is there for me to seek out more S-type code.

So the technique boils down to this: decompose problems into stable sub-problems, not into volatile sub-problems dependent on whims of the world, like today’s tax rates.

Finding S-type code

Once I learned this technique, I recognized cases of it from my past where I looked at a unique problem and recognized a stable problem embedded within it. Here’s an example.

I worked with someone who was instinctively better at this decomposition than I was. I’d written some code to extract the top 3 items from a collection, which I’d done with a for loop, based on some custom logic to define what “top” items meant. That is, I’d decomposed an E-type problem into another E-type problem. His review comment to me was: think of this as three problems.

1. Figure out a way to compare two items (a comparison function).
2. Put the items into an ordered collection, sorted by your comparison function.
3. Take the top 3 items from the collection.

At the time, neither of us knew about Lehman’s terminology, but now I’d see the first part (the comparison function) as unique code, so it’s E-type, and the second and third parts as S-type because they’re standard library code.

Programs that solve business or information technology problems won’t be as stable as math libraries, but I’ve found that I can make a problem more stable by defining abstractions. One example is activeCustomer. Even when the world changes and we must redefine what constitutes an active customer, the rest of the code is typically unaffected because it rests on the activeCustomer abstraction. From this, I conclude that Lehman’s sharp distinction between E-type and S-type code should more accurately be seen as a gradient, with some problems more stable than others. Math and informatics problems tend to be the most stable and business problems are less stable.

When I initially decompose a problem, I rarely find all of the opportunities for S-type code. I often decompose naively into E-type code – perhaps because I have so many years of bad habits – and then critically examine what I’ve just done, looking for S-type opportunities. It fits into a virtuous cycle of code improvement that I’ve written about before. [8]

My co-worker had trained himself to find S-type problems embedded in E-type ones. At the time I thought he was applying a functional programming technique but it’s more general than that.

It takes time to train yourself to recognize opportunities for S-type code. I want to emphasize, however, that once you’re able to seek out S-type code, it’s not slower than decomposing into E-type code. In fact, it’s typically faster because you’re reusing more and writing less. That’s how expertise works: experts are faster than novices.

Impact

Why bother with S-type code? I see at least three reasons. The first is that S-type code accumulates differently than E-type code. As we’ve seen, because it solves a stable problem, S-type code may already be in a library, and if not you could add it to an existing library.

The world presents us with an unending series of problems to solve. If we resort to writing custom solutions to each – that is, we write E-type code – then we’ll be accumulating a lot of custom code. Each time we instead recognize a stable problem hidden within what’s presented to us, that’s code we don’t have to write, test, or maintain. If your company finds S-type code more often than your competitors, you will build systems faster than they do.

Consider the three graphs below. These are just cartoons to help you see how S-type code accumulates differently than E-type. Graph A shows the unlikely but theoretically possible situation where you always recognize S-type code, which rarely needs to change. Graph C shows the opposite, the all-too-easy to imagine situation where code is always written from scratch, never reused. Graph B is a mixture.

Notice how the code in Graph A accumulates like pancakes or stratified layers, the code in Graph C like vertical bars, and the code in Graph B in-between as diagonal stripes. As the size of the system grows, the cost of writing and re-writing in (Graph C) grows rapidly, while the cost to recognize and reuse (Graph A) is minimal.

Figure 2. Patterns of code accumulation. These three notional graphs show how long code lives before being rewritten. (A) shows the theoretically possible 100% S-type code. (B) shows a mix of E-type and S-type. (C) shows the all-too-easy to blunder into 100% E-type code. Assuming it’s equally costly to write a line of S-type or E-type code, the total cost to write and maintain the system in (A) is much less than the system in (C) because there’s minimal rewrite or complexity cost in (A).

These three graphs are cartoons but Rich Hickey showed graphs that looked just like A and B. [9] His Graph A was for Clojure, where code that went into its libraries had a brief juvenile period where there was some churn, but after that, it leveled out into horizontal pancakes. Graph B was for another language. A strategy of seeking S-type code is possible and pays off with low code churn.

The second impact of seeking S-type code is reduced complexity. As Rob Pike wrote:

“Organic growth is not simple; it generates fantastic complexity. Each piece, each change may be simple, but put together the complexity becomes overwhelming. Complexity is multiplicative. In a system, like Google, that is assembled from components, every time you make one part more complex, some of the added complexity is reflected in the other components. It’s complexity runaway.” [10]

Each of those unique E-type parts you build will have wrinkles custom to the unique problem. As you compose them into a system, those wrinkles multiply and you slow down.

The final impact of seeking S-type code is that it can help steer a project to health. Project leads will have developed a gut feel for good and bad code. They can’t transfer that gut feel to their teammates, but they can teach others to seek out S-type code. They can track the ratio of E-type to S-type code on the project: the ES-Ratio. The ES-Ratio acts like a compass indicating health trends on the project.

The nature of software engineering

The nature of software engineering doesn’t change; it’s the same discipline whether a human or artificial intelligence (AI) types the code. Its nature has been obscured because, since the beginning, developers and academics have disagreed about the effectiveness of various software engineering advice and heuristics.

Empirical validation has been elusive because it’s difficult to create identical conditions between experimental groups and, if you could do so, it’s wildly expensive to build full-scale software multiple times. Now that AI is writing code, it’s easy to recreate identical conditions and cheap to build software. As a result, we may learn more about the nature of software engineering in the next five years than we have in the past fifty.

Some people’s first thought is that since AI will be writing code, it’s less important to understand software engineering. In fact, the opposite is true. Regardless of who writes the code, the world runs on software so it matters how it is written. A better understanding of software engineering leads to better code: higher quality, easier to write, and less expensive.

Understanding the role of E-type and S-type code in software engineering is a piece of the puzzle. Mary Shaw has observed that by understanding software design, we enable anyone to do what formerly required a virtuoso. Soon, we’ll expect that from AI.

References

1. D. L. Parnas, “On the Design and Development of Program Families,” IEEE Trans. Softw. Eng., vol. SE-2, no. 1, pp. 1-9, Mar. 1976.
2. E. W. Dijkstra, “The Humble Programmer,” Commun. ACM, vol. 15, no. 10, pp. 859-866, Oct. 1972.
3. E. W. Dijkstra, “On the Role of Scientific Thought,” in Selected Writings on Computing: A Personal Perspective. New York, NY, USA: Springer-Verlag, 1982, pp. 60-66.
4. M. M. Lehman, “Programs, Life Cycles, and Laws of Software Evolution,” Proc. IEEE, vol. 68, no. 9, pp. 1060-1076, Sept. 1980.
5. M. M. Lehman, interview by W. Aspray, IEEE History Center, Sept. 23, 1993. [Online]. Available: https://ethw.org/Oral-History:Meir_Lehman.
6. M. M. Lehman, “Laws of Software Evolution Revisited,” in Software Process Technology (EWSPT 1996) (Lecture Notes in Computer Science, vol. 1149), C. Montangero, Ed. Berlin, Heidelberg: Springer, 1996, pp. 108-124.
7. I. Herraiz, D. Rodriguez, G. Robles, and J. M. Gonzalez-Barahona, “The Evolution of the Laws of Software Evolution: A Discussion Based on a Systematic Literature Review,” ACM Comput. Surv., vol. 46, no. 2, pp. 1-28, Dec. 2013.
8. G. Fairbanks, “Fix Tech Debt With Virtuous Cycles,” IEEE Softw., vol. 40, no. 2, pp. 111-116, Mar./Apr. 2023, doi: 10.1109/MS.2022.3228623.
9. R. Hickey, “A History of Clojure,” Proc. ACM Program. Lang., vol. 4, no. HOPL, pp. 1-46, June 2020.
10. R. Pike, “Simplicity,” Command Center, Dec. 2023. [Online]. Available: https://commandcenter.blogspot.com/2023/12/simplicity.html (accessed Nov. 30, 2025).

ABSTRACT: As GenAI tools handle more routine programming tasks, the software developer’s responsibilities shift to supervising these tools. Accordingly, the software developer’s success will depend on a skillset that emphasizes reading, critiquing, and modifying code rather than writing programs.

Generative AI (GenAI) and other automated tools are increasingly handling the routine nuts and bolts of creating code. To use them effectively, however, you need to know precisely what you want the tools to generate. It requires close knowledgeable human supervision to specify what you really want (often it is fuzzy), to determine whether generated code does what you specified (often it does not), to judge the quality of the code (often it is poor), to repair AI-generated code if it’s defective (often it is), and to decide whether what you specified to the tool actually describes what the you intended (often it does not).

In other words, the responsibilities of software developers are rapidly becoming more about designing and less about programming. Reading, understanding, evaluating, and repairing someone else’s code is now more important than writing code from scratch. Judging whether you requested the right thing looms larger when the code is not written by you. Attempting to develop complex software without these skills can put a programmer in the position of the Sorcerer’s Apprentice – able to invoke the technology but lacking the skills to control it.

In this column, we examine this shift from coding to design skills through the lens of how to teach “programming” in the new AI age. This reveals the capabilities that current software developers should be cultivating – no longer programming in the traditional sense, but rather software development in cooperation with GenAI tools. We start with the introductory programming course as an example, because it is familiar, it establishes the students’ essential mindset about software development, and it shows the challenges raised by the shift to AI. From there we move on to how GenAI affects software development across the spectrum, not only for university students but also for established software designers.

Our principal point is that the relevant skills for software development are now understanding and modifying code you didn’t write yourself and that this will best be achieved by in-depth study of exemplars of excellent software.

A university curriculum contains both surface topics that are evident to everyone and also deeper objectives. Students in data structures are asked to write linked lists (the surface topic) so that they develop deeper skills: problem analysis, algorithmic thinking, debugging, and evaluation. No one needs another linked list implementation. It is assigned for two reasons: in struggling with it, the student makes progress on the deeper objectives, and the evaluation of the code serves as a proxy for those deeper objectives. Unfortunately, the assignments are often quite imperfect proxies.

Instructors have guided students this way for thousands of years but in just a few years GenAI has disrupted that guidance. GenAI is such a convincing mimic of a student making progress on the deep objectives that instructors are blinded to the actual progress. Reshaping courses for the GenAI world will require rethinking assignments to be more directly connected to the deep objectives.

The implications are profound and are forcing changes throughout the university curriculum. A curriculum is a complex system with many interacting parts. It has, or should have, a description of the capabilities of its graduates. The curriculum achieves this not only through a set of discrete courses, but also through a culture of intellectual engagement that extends beyond the courses and creates a coherent philosophy of learning.

What’s called for is re-examining the course objectives, resetting them if necessary to prioritize enduring principles, then ensuring that student activities serve those objectives. Anyone focusing on the superficial topics (the proxies for the deep objectives) must become more aware of how the curriculum works. Some instructors will need to be reminded of the deep objectives. Students, who can ask GenAI to do their homework in a click, must be acutely aware that the value of their education arises from their struggle. It’s easy for students to conflate industry assignments (where the code is put into production) and course assignments (where the code is discarded), and believe it’s OK to use GenAI on both.

Most disciplines study outstanding exemplars before expecting students to create their own, progressing to more sophisticated works as the students master earlier material. Students study not just the surface text of the exemplars but also their deep structure, their context, and criteria for critiquing them. We think students and experienced software developers should study great exemplars, too, with progressively more complexity as they gain experience. Computer science, alas, has a tradition of expecting students to write code as the primary way to learn software design.

What’s called for is a source of carefully curated exemplars of software systems with interpretive material that supports study of the important qualities of good software. Even more than in years past, developers will need to read, understand, and critique relevant aspects of software in order to modify it – often code written by GenAI.

The introductory computer science course

The principal objective of an introductory computer science course was once – and still should be – for students to learn problem solving using computational tools by applying enduring principles. That is, the course should be about principles such as critical thinking, problem analysis/solving, and careful critique – in other words, about design. Students learn these principles by applying them to specific problems, learning skills with their tools in the process. The initial tasks are small, but this is the time to establish a mindset for quality and good habits for developing software.

Unfortunately, the introductory course has in many places evolved to a programming course in which the objectives are largely the skill of writing small programs to solve given well-specified problems in traditional imperative languages such as C, Java, and Python. That is, the surface topic has become the primary objective. Evaluation is based on the code the students write, rather than the analysis and design steps the students go through in producing that product. The current (2023) ACM/IEEE/AAAI curriculum is still as focused on writing programs as ever. The section on introductory programming, “Fundamental Programming Concepts and Practices”, is all about writing code: the illustrative learning outcomes are mostly (9 of 14) “design, write, test, and debug” one thing or another, and only one is “read a given program and explain…”

Let’s change the introductory course to emphasize reading, understanding, and critiquing code – and applying these skills directly to supervising GenAI tools for code generation.

Today this means teaching students how to supervise genAI tools, which entails designing new activities that lead students through the steps of understanding both the strengths and the weaknesses of these tools and processes for using them well. This involves guiding students in effective use of the tools and revising assignments to ask for interpretation, evaluation, and revision of GenAI results rather than solely coding programs from scratch.

Students must of course continue learning to write code, not just to read it, but the current balance is wrong. Reading itself can be staged, as can writing. This might begin with hand simulation then move to asking whether a given piece of code does X, then revising the code manually or with GenAI so it actually does (or comes closer to doing) X. To develop critical reading skills as students progress, common types of defects and failures could be introduced so that students develop personal checklists of flaws to watch out for.

One special challenge of GenAI is that, unlike previous advances in programming technology and tools, which could be trusted to generate correct results, GenAI cannot be trusted to produce high quality or correct results, which changes the relation between the software developer and tool [1]. We now have tools that can relieve humans of writing large amounts of code, but this comes with uncertainty about the quality and correctness of that code, which together shifts the software developer’s responsibility from coding to design.

Students should be taught to read good exemplars at a scale appropriate to the course. They should learn to think not only about what the examples compute and whether it’s correct, but also about the structure of the software and whether and how the examples achieve quality attributes such as performance, evolvability, and maintainability. This will prepare them to study larger examples later.

Ongoing professional development

Software engineers need to be lifelong learners to keep abreast of emerging technology, and senior engineers need to mentor junior engineers. Success depends on mastering principles, theories, models, and concepts that endure through several changes of technology, in order to learn new tools and examples as they emerge.

In The Mythical Man Month, Brooks observed that moving from a program to a product increases the complexity and effort by a factor of 3; moving from a program to a system does likewise. Doing both, moving from a program to a system product, costs a factor of 9 – about an order of magnitude.

Introductory courses use simple problems, yet these early courses establish the learners’ essential mindset about software quality. Both senior students and junior practitioners must learn to appreciate and handle this growth in complexity. For software that’s part of a system, the internal design decisions matter as much as the surface functionality. As developers mature, they must create robust well-engineered software that satisfies increasingly higher demands for reliability, safety, security, correctness, performance, and other quality attributes. This involves exercising good judgment and tacit knowledge.

Developing that mastery comes not from coding small programs but instead from engaging deeply with good exemplars of large systems. This entails not just reading the code, but also studying interpretive material, observing them in execution, and experimenting with modifications in order to understand them deeply.

Other disciplines develop this judgment by studying shared, well-defined systems with a rich history of analysis and commentary from multiple points of view. Often known as model systems, exemplars, or type problems, they provide a way to compare methods and results, work out new techniques on standard examples, and set a minimum standard of capability for new participants. Biology, for example, has the fruit fly and the lab rat. Software engineering is short of these but there are efforts to develop such exemplars in specific areas [2].

The skill of supervising AI tools thus depends critically on knowing the technology of the implementation in order to supervise the tools – critically evaluating the AI outputs and, as is often necessary, improving them either manually or with tool-based refinement. Developing that mastery requires studying good exemplars, a task that is more difficult the more complex the systems are.

Let’s develop a curated collection of great examples of well-engineered software, with supplemental material to interpret why they’re great.

Empirical research shows that GenAI tools can be helpful to developers who have a broad and solid knowledge base, and who understand the strengths and weaknesses of the tools. However, in the hands of inexperienced developers. GenAI tools can be counterproductive, presumably because they lack understanding of the tools and how to use them [3]. The next generation of developers must master skills and judgement that go beyond classroom examples, such as the skills to design safety-critical, high-assurance, and secure code.

The transition to genAI must support developing software that is fit for its intended purpose. Doing this requires determining the level of quality required for a particular task. Some software is sufficiently critical to justify the effort of extensive validation. For other software, “good enough” is good enough. The question of whether the software is “good enough” depends on the consequences of failure, the odds that someone (or some thing) will actually intervene before failure, and problem-specific context. If the quality standards are not set appropriately, fitness for purpose may be lost [4].

Today’s students must grow into tomorrow’s developers who have both real appreciation of the technology and a mindset that seeks correctness over mere box-checking. They must develop their own judgement for what “good enough” means based on the consequences of failure.

The challenge

This column challenges the status quo in computing education. It argues that we should revisit educational objectives to be sure they lead to the durable, deep skills (algorithmic thinking, clear formulation of problems, solving those problems with computational tools, debugging, and evaluating candidate solutions, and code repair) in the context of current technology. It argues that we should revise – likely drastically – educational content to emphasize the analysis that leads to the solution to an assignment, not merely the finished product.

Technology is advancing and educators should embrace it. Let’s teach learners critical thinking with current technologies and let them use the tools that help with lower-level tasks; the need to check and revise GenAI results provides the setting for teaching them the skills that used to be front and center.

We have identified the challenge and some avenues to explore. We don’t pretend to solve everything and many questions remain. For example:

• How do we find excellent exemplars? How do we decide that they are excellent? Can we create them, or must we find them in the wild? Can we identify some that serve us as model problems? Several existing efforts are identified in [2]
• How do we encourage learners to shift from hacking out code to understanding that there are problems for which mere hacking is unsuitable?
• How can they develop the judgement about what quality standard to use on a system – when quick and dirty is ok, and when only the best will do?
• How do we strike the right balance between learners studying good examples and creating their own?
• How do we reach the vast numbers of programmers who don’t get formal education in programming – the ones who pick it up informally, on their own?

GenAI has disrupted the standard way that instructors evaluate students’ progress. Though many instructors are grumbling loudly, we should treat this as a gift because it guides us to reconsider our standard ways of teaching and consider alternatives.

Acknowledgements

Thanks to Dan Gillmor, Irving Wladawsky-Berger, André van der Hoek, Owen Cheng, Marian Petre, Ipek Ozkaya, David Kosbie, Dave Eckhardt, John Mackey, Dave Farber, Rich Kulawiec, Ed Frakenberry, Nick Kelly, Rohan Padhye, David Garlan, Titus Winters, Andrea Scaduto, Sushil Birla for constructive comments and suggestions.

References

1. Eunsuk Kang and Mary Shaw. 2024. “Tl;dr: Chill, y’all: AI Will Not Devour SE.” Proc. Onward! ‘24, pp 303–315. https://doi.org/10.1145/3689492.3689816.

2. Mary Shaw and Eunsuk Kang. “Model Problems in Software Engineering.” Accessed: Oct. 1, 2025. [Online]. Available: https://modelproblems.org.

3. Matthew Kam, et al.. 2025. “What do professional software developers need to know to succeed in an age of Artificial Intelligence?” Proc 33rd ACM int’l Conf on Foundations of Software Engineering (FSE Companion ‘25), pp 947–958. https://doi.org/10.1145/3696630.3727251.

4. Mary Shaw. 2022. “Myths and mythconceptions: what does it mean to be a programming language, anyhow?” Proc. ACM Program. Lang. 4, HOPL, Article 234 (June 2020), 44 pages. Refer to pp. 234:21-22. https://doi.org/10.1145/3480947.

Author bios

• Mary Shaw is the A. J. Perlis University Professor of Computer Science in the School of Computer Science at Carnegie Mellon University, specializing in software engineering.
• Dr. Michael Hilton is a Teaching Professor in the School of Computer Science at Carnegie Mellon University, specializing in software engineering education.

Here are the slides.

Below is a lightly-cleaned transcript of the talk (hesitations and false starts removed, meaning preserved). Each slide appears before the portion of the talk that discusses it. An appendix with the full audience Q&A follows the talk.

Contents:

• What is SWEdu?
• What’s in SWEdu?
• Does it work?
• Reflection
• Bibliography

Introduction — David Garlan (host)

David Garlan: So it’s a great pleasure to welcome George back. He’s no stranger to us, of course, having done a PhD here, but also having collaborated recently with Bradley and me in teaching and revamping the software architecture course, as well as several other activities. George is very interested in connecting back with CMU and welcomes interactions with people.

The meet-and-greet filled up almost immediately, so we’ll be scheduling another day here when people can come meet with George if they want to talk to him.

I think you know what George is going to talk about. I haven’t seen the slides, so I don’t know exactly, but it will help us understand a bit about what Google does in education and software engineering. For us, a particularly interesting question is: what are the gaps George is seeing from people coming in to work at Google that we should actually be addressing in our classes? Partly what to teach, but also how to teach. George’s form of delivery within Google has a lot of innovations compared to what we’d normally think of in the traditional classroom. And the analytics and data George has gathered is really impressive. We could be doing similar things. There’s a lot to be learned. With that, George, I’ll turn it back to you.

Opening & decoder ring

George Fairbanks: Thank you so much, David. It’s my distinct pleasure to be back here at CMU, seeing so many familiar faces. Hello, Bill — I didn’t get a chance to say hello. I hope not to give you a boring presentation today.

Feel free to interrupt me at any time for clarification questions, because the thing I’m most worried about is dropping jargon or slang that we use inside the company. Let me give you a quick decoder ring on the very first slide: “Four Years of the Google School of Software Engineering,” which we call SWEdu, which has got to sound strange.

At Google, they abbreviate the job “software engineer” into SWE. The department I’m in is EngEdu, which you could probably figure out, and we jammed those together and got SWEdu. Two other decoder-ring items. I’ll probably say “TL” instead of spelling out “tech lead” (a software engineer who is the lead engineer for a group of people). And I’ll say “CL,” which stands for “change list.” If you’re familiar with Git, it’s exactly the same as a PR, or pull request (a proposed change to the code base). So: SWEdu, CLs, and TLs.

At 12:30 today [about halfway through the presentation], the other person who started this with me, Titus Winters, is going to join, so he’ll be around for the Q&A part as well.

This is a picture of the guy I used to see in the mirror maybe 10 years ago.

I’ve been active compared to most practitioners in getting the word out about software design and software engineering: in IEEE Software magazine, a book, and presentations, including the SATURN Software Architecture Conference that Len Bass was also a big part of.

When I showed up at Google, it was a bit of a shock: in some ways they were incredibly ahead in software-engineering techniques, and yet they weren’t embracing some of the ones I thought were good ideas. That’s where all of this work began.

Today I’ll tell you what the project is and exactly what’s inside it (because you’re probably wondering, are they teaching agile? functional programming? software architecture?), present the evidence we have about whether it works, and then share some personal reflections.

A note on process: I had to get all these slides vetted, as industrial speakers do. The things on the slides are the parts approved for publication. Especially in that last section, on reflection, you’ll be hearing much more about me, my thoughts, and what I found hard. It’s not an official statement from Google.

What is SWEdu?

We started this back in 2021. Kevin O’Malley was the sponsor leading the department. Titus and I found that we had expertise in software design and in software testing, and we said, look, let’s just go out there.

Kevin encouraged us to shoot for the moon. The title “Google School of Software Engineering” sounds a little grand, and he said, “That’s what you eventually want, isn’t it? Let’s see how close we can get.”

What Titus and I experienced is that engineers at Google (probably like every other company) learn on the job. But what they learn is idiosyncratic: they may be exposed to some things and not others.

You may have a great lead who’s a strong mentor, or you might just have a knuckle-down-and-get-the-work kind of lead who isn’t really sharing knowledge. CMU graduates a bunch of very strong undergraduates, but it’s not clear which [gaps in their knowlege] will get filled in. It’s much by happenstance.

They do teach themselves, but there’s a critical problem.

When we talk to tech leads about these techniques and give them the names, they say: “I already know that, but I never knew somebody else had already given it a name, let alone 50 years ago.”

As a result, they’re unable to share that knowledge effectively. They can say “no, no, not like that, like this,” but they don’t have a name for a principle. They can’t point to a reference. They may have reinvented information hiding all over again, but they don’t have the term to let somebody else know.

Across the company you get a Tower of Babel. And I think this is pervasive in the industry. Self-taught lessons can’t be communicated very effectively.

Who’s been involved? The team is currently me as the only engineer, plus a part-time project manager, and someone helpful in running the logistics and editing videos. Our sponsor right now is Mohamed Dekhil. And there are several other people, including Titus Winters, who were key contributors.

When I started out, I had a gut feel there was a skills gap, based on having come to CMU myself and trying to absorb everything I possibly could. When I looked at what everybody else was doing, I thought, “I’m not sure they’ve heard of these ideas, or understand the benefit they’d get.”

So one of the first things we did once we had an education process was start surveying the heck out of the tech leads. We honestly buried them in surveys: before they show up, immediately after they finish, and for a year afterwards, to see what adoption looks like.

One of the first things that popped out: the new grads that show up don’t know all the ideas in our class. And in fact the majority of the class’s ideas are relevant even to entry-level engineers. That’s pretty good. It means we’re mostly shooting at the right target.

There is no way any school, even CMU, is going to [teach] everything an engineer needs in four years. You can’t prepare somebody for a 40-year career starting from an 18-year-old. The frontier keeps moving. The implication is we’re going to need to keep teaching things in industry.

How did the experienced engineers rate? They knew a whole lot more (maybe not the terminology, but they already understood information hiding, modularity, and so on). But we were delighted to see we were still on target: even the most experienced engineers could gain things from this training.

When we started, we had a substantial amount of discussion (“discussion” is the nice word) about what the pedagogy should be. Titus and Tom had had incredible successes with a certain kind of pedagogy: weekly tip-of-the-week articles and best practices. Some of you may have heard of Google’s “Testing on the Toilet” series, where every week a new piece of paper is hung in public places inside the company with a single tip. The rules are that the tip has to have consensus and be actionable. Not “let me talk to you about abstract data types,” but “this is a better library for parsing command-line flags.” Just because of that vehicle, we shy away from abstract topics there.

But I argued that when it comes to software design, there aren’t that many cut-and-dried best practices. Instead you have a bunch of ideas that, once they’re inside you, allow you to grapple with design decisions better. The variety of software Google makes is truly impressive: operating systems, device drivers, phones, telephone or essentially network switches. How would you boil down best practices for design across that range? That’s why we confronted what we’re doing here.

We were guided by a distinction between education and training. I got this terminology from Tim Halloran, another person here from CMU. In the military they make a strong distinction between the two. If you need to teach someone to disassemble and rebuild a helicopter engine, that is training. There’s a standard way, everyone conforms, and you know the outcome.

[The military] also has education, which would include the war colleges. (Apologies for the military references.) There are better and worse ways of winning a war, but that’s not the same character as disassembling and reassembling an engine. We ended up leaning into the education aspect rather than the training aspect in SWEdu.

Here’s my MBA metaphor. You might naively think that as long as your company takes in more money than it costs to do stuff, it will be successful. But when you get an MBA, you learn about cash-flow analysis: what matters is when you get the money, not just how much. Companies can fail because they don’t have the right money at the right time. It’s not a profound lesson, but once you internalize it, every problem you look at shifts perspective, and you’re more likely to avoid that mistake. That is the same character as the design education we’re doing.

We have both benefits and drawbacks of doing this education in industry. The big benefit is we don’t have to test anybody. ChatGPT is not throwing out our curriculum. But my point is that evaluation for us can be as simple as giving them a survey at the end, as opposed to an academic environment, where you’re much more structured around accurately gauging how well people learn.

Here’s the flip side. If you want to become a medical doctor, the curriculum can say you’ve got to take organic chemistry. That’s just the way it works. We basically don’t have that. Every one of our students can walk out anytime they want; we have to convince them to show up in the first place, and once they’re there, convince them to stay. That influences a lot of what we’re able to do.

Google’s culture includes a lot of what we call “go-links.” You can see one at the bottom of the page. It’s basically a small namespace: inside Google you type “go/” plus a link into a browser, kind of like Bitly or any link shortener, except the namespace is company-wide.

We designed SWEdu content to fit within that go-link structure. If you’re in the middle of mentoring somebody and don’t feel like writing a whole bunch about a topic, you can say, “I know George already wrote that one-page essay — type in the go-link.”

We encourage these students (meaning 10-year industry veterans) to mentor other engineers via go-links. We want them to understand the concept, remember the go-link, and the next time the topic comes up, point them to it. The idea is that this makes the company much more efficient and starts to coordinate the vocabulary inside the company.

As everybody knows, LLMs are a big deal these days. One of the most interesting things I’ve been seeing in the last couple of months is that our internal LLMs have now been trained on the [internal] content we’ve created. If you ask them anything about software design or software testing, they’re incredibly likely to cite SWEdu content (which is delightful to see). So we’re not only training engineers, we’re training the bots.

Here’s a short sequence of slides that puts in a nutshell how we hope to get ideas into the company.

Imagine you’re a tech lead reviewing some work (a design document or a proposed change, a CL). Somebody says, “I’d like to get your input on this.” We have a strong culture of doing that. If it’s a trivial change (you forgot a semicolon or you have a typo), no problem, you just point that out.

But as soon as you [suggest] the equivalent of “I think your characters in your story lack motivation; we need to talk about the fundamentals of storytelling,” then you’re in trouble as a mentor. You only have a few options.

Option one: teach on the fly. You try to type that essay into the comments section of a document. But it’s an incredible burden on the leader, and often it doesn’t happen because of the time burden.

Option two: walk over to somebody’s desk and say, “let’s chat through this. I was surprised to see this proposal.” Again, that’s incredibly expensive.

Option three: say “nope, not like that, like this” and hand them the solution. That’s not the best for teaching, and culturally you feel sheepish afterwards: “my TL knows how to do this, but I’m not very good at it.”

Option four: maybe you learned this idea from a textbook, and even remember which one. But if you’re in the middle of a review cycle (here are 20 lines of code, can you take a look?), the last thing you want to do is insert “please read textbook here before continuing.” It’s just impractical.

So what we ended up with is building a whole bunch of short pages, putting them behind go-links, and getting them out to the people doing the reviews (the tech leads). The idea is that this saves everybody time and starts to pull the company together.

How do we train these people? These are the tech leads.

[SWEdu’s flagship course is] a six-week program, two half-days per week. Typically I do Tuesdays on design and Titus does Thursdays on testing. It’s the format of a graduate seminar, so before each class there are a handful of readings they’ve done ahead of time. (At the end of the slide deck you’ll have available are the readings we’ve assigned.)

The whole model is to influence these technical leaders and send them back into the company (effectively a train-the-trainer, or train-the-mentor, situation). We want to encourage mentoring and make them strong, effective mentors.

Our multi-year game plan is to do a whole bunch of things that, summed together, will change Google’s culture and improve the state of the practice.

First, we have to create content. If you think about many core ideas in software engineering, there isn’t a single essay you can point to that reveals that idea. We have to write those essays (the one-pager on that thing). It might be in the middle of Len [Bass]’s book on software architecture, but they’re not going to read the first six chapters to get to chapter seven, get the idea, and get back to work.

Second, we try to nurture a community. If we’re not constantly drawing attention to these topics, people shift their attention elsewhere. There’s a renewing of interest.

Finally, we’re “priming the pump” by teaching all these folks, then patiently waiting for change in the company. If you’re familiar with this meme template, the “waiting patiently” part is the hard part. We’re starting to see real signs of change, but we wish it had happened already and that we were farther along.

What does one of these go-linked pages look like?

Here’s one on quality-attribute priorities (probably a bit of an eye chart, but you’ll have the slides later). In the blue box is a summary: “A quality attribute priority expresses the relative priority of several quality attributes.” It’s an idea you might need to reference, so you can go to go/quality-attribute-priority and drop that link when you’re mentoring anyone.

What’s in SWEdu?

So what’s the content, exactly?

Our project consists of four things: written materials, courses, recorded and live videos, and a community.

On written materials: we have a guide on software architecture (you saw one of the pages), plus a collection of miscellaneous design concepts that haven’t made their way into one specific guide. We have a Design by Contract guide, because we find DBC ideas are very compatible with testing. Code takes on a contractual nature. If you’re a functional-programming fan, you realize this is like the gateway drug for procedural programmers.

We also have a guide on design diagrams and diagramming tools, which as it turns out is the most popular thing we’ve done. Plus a guide on error handling, and a whole bunch of tech talks and video podcasts on the website.

Here’s what’s in the courses: a Design by Contract course, and the flagship course I keep referring to (the Tech Lead Seminar, the six-week one).

We sort of force-fit the content into small, medium, and large because we need a structure for the class.

In “small”: the very first topic is quality as a strategy. The idea is that Google started out making a high-quality distributed system from low-quality (i.e., unreliable) PCs. That was a wild innovation. But think of it this way: it’s a lot easier to build a distributed system out of high-quality parts than low-quality parts. If you can make stuff good, you have an easier time building a high wall. That’s the metaphor we use the whole time.

In “medium”: modules and coupling, error-handling topics, and various kinds of test doubles.

In “large”: a very brief overview of software development processes, where tech debt comes from, and continuous integration.

After teaching the course a couple of times, we realized that during class discussions (because it’s run as a seminar), several cross-cutting themes kept coming up that were never one specific topic.

One is this idea of intellectual and statistical control. As everyone knows, industry has gotten onto the testing bandwagon — that’s an example of statistical control: you look for specific cases and test them before you ship. You can imagine a factory spot-checking its products. We encourage people to do the thing every academic here is intimately familiar with: you should be able to think through your software, not just have empirical evidence that it works. We want you to have both. But it’s almost a radical idea at this point. People have so embraced empiricism that they’ve kind of abandoned the idea they might be able to think through whether their software works. Getting that on the table is a big part of these cross-cutting themes.

Does it work?

OK, here’s where the fun stuff begins: does it work?

The answer is yes. It’s working, but we’d like to do better. Let me dig into three topics: effectiveness, pedagogy, and adoption.

On effectiveness: before we did any of this training, we ran a survey asking about hypothetical training on these topics. How much time would it save your entry-level programmers? L3s and L4s are the entry- and mid-level ladders.

What we got back was it would save them five to six weeks per year (a pretty incredible number).

What was even more impressive: the more seniority, the more years of experience, the higher your level, or whether you’re in a leadership or management position, the higher the number. That’s great. It indicates people besides me were perceiving there was efficiency to be gained.

We did a similar survey after delivering the training, which has the benefit of being after the fact. They’ve actually seen the content. Now we can ask, how much will this content help your engineers? (With the caveat that they’re predicting productivity — I wish Ciera Jaspan were here to talk about how hard it is to measure and predict productivity.)

What we found was a pretty incredible number: every member of their team, as a result of the tech lead taking the training, would be about two months per year more efficient (with a standard deviation that’s off the charts).

That [wide standard deviation] made sense to me, because we have a buffet of ideas and say, “any of these any good?” Some people say, “yeah, our team needs this testing idea.” Some say “I wish I knew this design idea last year, because we just made a bunch of mistakes as a result.” So some people end up with very large numbers because they now recognize a mistake they’d had. But the number has been very consistent, despite the large distribution. And if you believe in the wisdom of crowds, we got data from 280 tech leads.

All in all, we got three different predictions: the one I showed before training; this one about time saved; and a third where we asked them to estimate the acceleration on their team. All are quite large.

One of our partners (we train a lot of Waymo engineers) did a back-of-the-envelope calculation: you’re investing on the order of 12 hours per week for your tech lead over six weeks, and you’re telling me you’ll get eight weeks per year out of your engineers? That’s like a 15x ROI in terms of time invested and time saved.

We were delighted to find that people loved this class. It actually made them happier and improved their opinion of their employer. So if you let employees take this training, they come out happier and more productive.

Shifting to pedagogy: for this to be effective, we need both the training materials (the class) and the written materials. Because the whole idea is to bring tech leads in, talk about topics, and make them effective mentors.

The effective-mentor part requires writing all those web pages. We hear this in the surveys: “It was great that you identified the concepts, and I really need those written references so I can cite them.”

They also overwhelmingly point out that the common vocabulary dramatically reduces the Tower of Babel effect. Engineers communicate with precise terminology.

One thing we were dismayed by was an experiment we set up the very first time we ran this.

We carefully got 90 candidates and divided them into three groups of 30: a control group; a group that got just the written materials but was not allowed to participate live; and a group that got the whole shebang (live class and all materials). We set it up to argue to management that it’s really effective to send people to live training.

What we found: all groups were very excited at the beginning — at least the live group and the written-materials group. But after two weeks, every single person in the written-materials group ghosted us. They weren’t operating at a lower level; they just stopped the training entirely. Two weeks ago, very excited. Two weeks later, we can’t get them to answer an email.

We were shocked — though it comes across as an obvious conclusion in hindsight.

You’ve got extremely busy people who always have something they need to be doing today. If you don’t have something on the calendar and part of a moving train (you’ve got to keep up, do the readings, show up Tuesdays and Thursdays), it’s like a gym buddy. There’s a reason these patterns exist for human beings. So our conclusion: do not consider making these videos and the training available on our website to be a solution. You need to continue to schedule classes and push this thing. Otherwise there’s always something else you should be doing.

Are teams adopting the ideas? Yes, they are.

But we’re still running into a marketing and publicity problem. Most engineers in the company aren’t aware we exist. In the last group, somebody said, “George, I know about your work because I read it when I was at ThoughtWorks, and I didn’t know you were at this company until I got the announcement for this course.” It’s a big company. Just getting attention is very hard.

They also tell us it’s not just having the materials. Having them on a website gives the TL credibility when they argue for a position. “Hey, I think we should do this — oh, by the way, the Google standard is over here.” That’s way more persuasive than “hey, that’s your idea.”

We saw a pretty good adoption rate across the different cohorts. Six cohorts have completed all the surveying; there are two more cohorts within the past year that haven’t finished all the surveying.

What I haven’t mentioned is that the first four cohorts, one through four, were taught live by myself and Titus. Then starting with five, six, and seven, they replayed the videos because Titus was at a different company by then. That actually works OK, though not as well. People aren’t as delighted, but they still come out pretty positive.

[Here’s] what we found. (This is an eye chart, look at the pretty colors).

[Recall that our materials are divided between design and testing]. These are the various topics for software design. Some topics were highly adopted. Some were barely adopted. That’s what I want you to take away.

And here are the testing topics: not quite as much of a drop-off, a bit more consistency. That’s probably because the idea of doing software testing at Google has a 15-year head start compared to the design ideas at Google.

Charlie Garrod (audience): Can you please clarify how you’re measuring whatever metric you’re actually talking about?

George: [The measurements come from surveys of the participants.] When you look at the slides afterwards, in the top-right corner, the decoder ring: the green says “my team and I think about testing this way”; the lighter green says “my team and I thought about testing this way before”; purple is “I now think about testing this way”; blue is “I thought about testing this way before, but my team did not.” There’s an equivalent version of this, and we track it immediately after class and then quarterly for a year, and we see the numbers creep up a bit.

If you think about it from those last two slides (are you successful in getting your curriculum into the company?), the answer is no. If you think about it a different way (we have a buffet of ideas; we can’t possibly come up with one curriculum that works for device drivers and backends and frontends and phones and you name it), then I think we’re doing OK, because everybody seems to find something to eat at the buffet. Essentially everybody comes away with multiple topics they’re excited about.

One thing we do is deliberately teach some topics adjacent to code (immediately actionable by the team) and we see the best adoption rates for those.

We see the worst adoption rates for the things I’m most passionate about, and that’s an embarrassment. (David and Mary are going to get after me later.)

But in some ways it’s understandable: those ideas have the least penetration into the industry, they’re the least mainstream at this point, and they’re the most abstract.

And I have just four hours to cover these topics, so maybe it’s not surprising they’re not fully adopted.

As for viral adoption, based on a very small survey and some individual interviews: it seems about 70% of the ideas make their way into tech leads, about a quarter of that jumps over to the team, and we’re not sure how many jump into the rest of Google. I’d love those numbers to be higher, but at least things are moving — which is good.

Leading up to this presentation, I realized that Google Analytics is hooked up to our internal websites too, so I pulled the numbers: how many people looked at go/software-design in the past year?

I was honestly shocked: 74,000 people read what I wrote last year. I’m still letting that wash over me. Here I am, some guy, saying you should know about quality attributes and tradeoffs, and now there are go-links for them.

I feel very encouraged this is actually working when I see a number like that. I assume it’s not product managers reading our website; I assume it’s engineers — but we don’t actually have that data.

So here’s a summary I’d like to leave as a capstone.

One alum says: these ideas are good. The importance of saturation cannot be underestimated. It is not one-and-done. It’s not that George or Titus said the right thing in class and now the company has changed. It never works like that.

And going back to ou [at CMU]: it’s impossible to imagine you could say the right thing to an undergraduate and solve industrial software engineering. It requires everyone to keep repeating the good ideas, saturating them. The more people you hear a good idea from — “hey, I really think we should test this,” “hey, I really think it’d be great if we had an architecture model” — the more practices change. If it’s said once, it drops on the floor and nobody’s behavior changes.

As a capstone for “does it work”: these are Likert-scale results: was it worth my time, I learned something valuable from another tech lead, it made me happier, attending improved my opinion of the company, I’d recommend it to others.

All I really want you to see is that it’s all over on one side. In general, people leave this course very happy. And that’s after bombarding [the participants] with surveys.

[Note that] every single one of them doesn’t carve out enough time. They always think, “I’m a very efficient person. I can sneak this course into my regular schedule.” And they complain nonstop, saying “You should warn us!” I say, “go back: We warn you every single day.” You need to carve time out. We make you get permission from your manager. I think it takes that much time.

When people come out of this, the median percentile they give for how it rates compared to all the other education they’ve done is the 90th percentile (which ain’t bad).

But what’s even more amazing is that a quarter of the people said literally the 100th percentile, which I interpret to mean a quarter of the people think this is the best [class] they’ve ever had in their life. I’m pretty happy about that.

Again, this is for reading later and it is an eye-chart. We’ve dug into many of these points already.

Reflection

So now I shift to my personal reflection, and I’ll try to connect it to some things relevant to the handoff and the relationship between academic and industrial education.

I’ve been to other presentations where someone at an engineering company says “I ran this project, we trained these people, we had some good results,” and I’m in awe that they got anything to work. Because having tried to do it myself, I realize so many things have to come together for this to work.

I’m incredibly grateful to the people who sponsored this project, because it’s hard. It’s much easier to spend your scarce resources on writing more code.

It also relies on a great number of good ideas (which I’ve liberally stolen from everyone here at Carnegie Mellon) and on soft skills: finding a way not to irritate people and to be persuasive to get where you want.

Another thing: many of the lessons we’re teaching are disruptive. There’s plenty of education that’s incremental, and I think everyone wants that model: “hey, here’s a great way to parse command-line arguments; I don’t have to throw away any ideas I hold dear; there’s a new library, it saved me 10 seconds, great.”

But many of the ideas we’re talking about say: hold on, you’re doing pretty well, but you need to back up, get rid of some bad habits in order to be even stronger. And what’s more, you now need to convince other people to do the same. That is hard. If you tell a team they’re not testing the right way, or they’ve been undervaluing software design, that’s a disruptive change.

Titus Winters used to say that we stumbled upon teaching design and testing in a way that was very convenient. They fit together like learning calculus and physics. One is the more abstract version. One is the more applied version. You can get an intuition for a phenomenon over here, and understand it in general over here. Being able to alternate those two was incredibly valuable in making these lessons stick. I’ve become a big fan of making sure you have the concrete along with the abstract.

We have a difficult problem: we aren’t doing skills transfer. We’re doing education. We need to send these tech leads back to their teams. Assume a team is at a local maximum of practices. They’re doing a good job. Now we’ve convinced the TL the team could be up here somewhere, but to get there they need to start doing things in a worse way until they can put everything back together in a better way.

So we are really training mentors and teachers to confront arguments and objections from their team that we can’t possibly predict. It’s a very difficult thing to prepare someone to do.

Finally, I think there may be an expectation in academia about what the baseline practices in industry are. I’m not just talking about Google. I’ve been a consultant at a bunch of different companies.

The median code basically looks like it’s from 1970. That’s not necessarily bad. You can make money writing 1970 code. But if you think you’ve taught them abstract data types so they’ll use them, they’ll use them in the standard library. They won’t necessarily write their own. They may never say, “wait a second, the big idea behind abstract data types is much bigger than lists, sets, and queues.”

So practice changes very slowly. There’s a lot of work to do to actually improve the state of the practice, which is wildly uneven — some teams are way ahead, and some are still writing mundane stuff you could write in BASIC.

So how can we improve? First, I think this kind of training needs to include both skills training and education. We do an OK job on the conceptual education part. We do not do a great job on “here’s how you disassemble and reassemble the helicopter.” When a tech lead goes back to their team, we’re essentially asking them to create educational materials or mentor without anything except the slides we gave them. And those slides were [written for] an experienced audience. When they’ve got early-career people, it’s inappropriate to ask them to do all that mentoring without skills training.

A lot of tech leads come back and say, “not only do I need to convince my team, I need to convince my management to see this the way I now see it.”

We need better channels. As I mentioned, not everybody inside the company knows we exist. Strangely enough, the best way we advertise what’s going on is an email newsletter that goes out once a month on education efforts, and we get thousands of people looking at our website as a result.

Google is way too big right now for the guerrilla tactics the Testing on the Toilet folks used. They had a wild idea and started putting posters up. You can’t do that within an established company like we are now.

The last point: we’re suffering from limited expertise. I’ve done everything I can to soak up good ideas like a sponge, but I think everyone here realizes there are so many other things you just don’t know.

I could never have created this course without Titus. He brought an incredible amount of knowledge, chose the topics, and explained them extremely well. We still rely on his lecturing for that.

Jonathan Schuster, who has a PhD from Northeastern, brought in a bunch of programming-language expertise. I wish we had more of that. I’m not trying to say this is the George project — this only works because we have these other folks who filled in the gaps in my skills.

Overall, the TLs say we’re on target.

The neat thing is now that we have these AI engines, I can take every bit of written content, put it into a text file, feed it to Gemini, and say “please don’t butter me up. Be as neutral as possible and tell me what they said about these topics.” So here are quotes from Gemini’s summary of the alumni written feedback.

Participants overwhelmingly appreciated the common language; they thought the ideas were universally relevant; they thought intellectual control was something they’d been missing; and they thought the ideas were applicable to their day-to-day work.

They love the content and think it’s useful in a world where we’re using LLMs more and more.

This is a subject of active debate. You’ve heard lots of opinions about what the future holds. Are we heading to a future where requirements people merely dictate their requirements, engage in some Q&A with an LLM, and the code pops out? Nobody knows.

But what the TLs tell us is that as a result of this training, they understand the world of software development better, and they feel that’s a necessary skill to guide the LLMs and use them successfully.

With that, I want to thank you all again. I couldn’t have gotten here without many of the people in this room, and I certainly couldn’t have gotten there without the people who helped create this course. Titus Winters, I think, may be listening in — thank you, Titus.

Discussion

Audience: Thank you for the talk. In one of your slides, you mention diagrams and diagramming. With my students, I’ve found they have a problem expressing themselves pictorially — whether it’s UML, arrows, whatever. What is the problem you refer to? What’s on those pages?

George: I can tell you what’s on those pages. First is a table listing a bunch of different diagramming tools with a summary of how they can be used. For example: can I edit the source code of the diagram, or is it always me visually moving things around? Can I drop it into Markdown — we have work documents based on Markdown? Can I drop it into Google Docs? That seems very mundane.

The other pages are my attempt to get people to recognize that a diagram is a model. The reason you’re having trouble drawing the diagram is that the model is unclear to you. It’s not because you don’t understand boxes and lines. Boxes and lines are simple. Models are surprisingly complicated. I don’t know how well that lesson lands, but I have a feeling engineers, because they’re guided through the design-doc template (“put diagram here”), go “OK, tell me about diagrams.” So it ends up being a very popular page.

Michael (audience): I’ve experienced it. We used the Google Design Doc template based on Titus’s book in our class, and we had exactly the same phenomenon. Although with LLMs, the diagrams have gotten a lot prettier but a lot more raw (that’s just been my experience). But I had an actual question. You mentioned at the beginning this “17% vs. 77%” skills-gap figure in intro software engineering, and you also talk about the value of providing a lingua franca — people recognize a concept but just didn’t have the terminology. You also had to give us a bunch of terminology that we as experts didn’t know.

So my question is: A) how much of this is teaching “the Google way,” and how much is arriving at an agreed-upon vocabulary? How much is vocabulary and how much is conceptual learning? Having a shared vocabulary is probably good — I’m not criticizing — but what’s your sense of the balance?

George: I think it’s a very fair question. Titus, are you on the call? Do you want to take this one? Well, actually, because you’re outside the company, you can say whatever you want, and you’re probably in a better position to chat about the vocabulary or the concepts that I’m bringing in for software design.

Titus Winters (remote): Yeah, I think it’s really important to remember: you can’t give someone a solution to a problem they don’t realize they have. Also, it’s a lot easier to put a name on an experience they’ve already had than it is to describe in the abstract a scenario they may experience in the future and give them a name for it — and then they take a test two weeks later, don’t think about it again until they encounter the problem. How would we expect them to name that correctly five or ten years later? One of the huge insights for me from this experience is that vocabulary is an incredibly powerful thing to focus on, specifically for early-to-mid-career engineers — like five-year-experience engineers — because they have so many working hours under their belt and a breadth of experience that’s just not available in the classroom. At that point, vocabulary becomes so much more relevant.

I’ve also spoken in the last year about a cultural phenomenon I see in tech (I can’t swear to it, but it all kind of lines up in my head): because of psychological-safety concerns, fitting-in concerns, and maybe just being polite and not interrupting the flow of conversation, our industry doesn’t stop to ask “hey, what does that word mean?” You just pick it up through context, and the person who just said it picked it up through context too. This is why words don’t actually mean anything in industrial spaces. The number of times I’ve had to ask “what does continuous integration mean?” terrifies me. Same applies in testing, design, and all these things, because we have an incredibly squishy understanding. No one has been given vocabulary at a time in their career when that vocabulary would stick. Does that answer your question, Michael, I assume?

Michael (audience): I mean, what I’m hearing is that the vocabulary is really important, but it’s not that they have a non-Google-specific word for this concept and you’re centralizing on the Google vocabulary. It’s that they’ve encountered the concept but don’t have a word to put to it — which is a different way of thinking about it.

Titus Winters (remote): I think it is incredibly helpful. Yeah, and as an example, we talk about test doubles, and broadly in the industry everyone just talks about “mocks” in general, but there are semantically different flavors of the tool. And I think Tom Manshreck (who’s also on the call here) tracked the terminology from Martin Fowler originally, like 20-odd years ago, and we sort of reconstituted a lot of those ideas. Once you start explaining the different use cases (you could just have a flow chart that tells you how to choose what the appropriate tool is for this task), so much muddle just crystallizes, and it’s kind of magic to watch.

George: Actually, I can speak to some of the testing stuff, because I can attest to Titus’s part. On the testing side, Titus hammers vocabulary about flakiness vs. brittleness (which for many folks is just a pejorative adjective to slap on a bad test). And he’s like, no, they’re two distinct things with two distinct remedies. So it’s important we give these names.

One thing Titus may not be aware of: I’ve been fighting the fight to use the term “quality attributes” instead of “non-functional requirements,” and I think we might actually be over the hump on that. Believe it or not, having the website up for five years, people are actually starting to say it in authoritative, very senior-level documents. So thank you, Leah Rivers, and a bunch of other folks — but I think we’re actually winning that one.

Another answer: [SWEdu training materials have] almost no Google technology in any of these topics. You’re not going to see Stubby, you’re not going to see our internal database names, any of that kind of stuff. It does lean toward using examples from languages we use at Google — not Haskell or ML. So mostly it’s conceptual.

David Garlan (audience): So what are the prospects that this might become available outside Google?

George: We’re working on it. It seems entirely possible. That’s probably all I can really commit to at this point. There are a lot of people who’d like to see it happen.

Because I think you and I have chatted about this idea I’ve got, which is: it’s preposterous that this is a single source, that I’m the only one contributing. I would love a Wikipedia kind of thing where everyone can start to improve this thing. I know Wikipedia sounds scary, but over time, with editors and attention, it has become truly impressive. I don’t see why we couldn’t do something similar. Or you could have it be a CMU-branded one, or you could have different forks and say, well, that’s the standard flavor of it, but here’s the CMU flavor — like the Linux kernel. Anybody can do anything you want, but we tend to follow Linus’s version.

Audience: Yes. One of the things you mentioned toward the end of the talk: you see a lot of code in industry that’s like 1970s code. The implication is that there are characteristics of this code that make it poor quality, and you mentioned a lack of use of abstract data types. Can you describe more of the characteristics or deficiencies of some of this code, to give a better idea of the challenges facing the industry in code quality?

George: So there’s a thing that’s going on in industry, and has been for a while, which is that the majority of the code is written by the least experienced programmers. The more senior you get, the more you get into an advisory, steering capacity, and you don’t get to write as much code. I think that’s less true at Google than at many other companies, but it’s still true to some extent. You have an army. It’s a business model: one expensive, experienced person plus five other people to magnify their influence. That’s what you want.

So you end up with mundane rookie mistakes being magnified. An example: engineers across the industry, I think, are not properly afraid of side effects. When you’ve got a hundred lines of code, it’s easy to keep track of the side effects in your head. But when you have tens of thousands or millions of lines, it becomes very difficult to reason about “I just made this change — is that going to be safe?” if you don’t have some discipline about where things are going to change and where they’re not going to change.

This is an example of me personally speaking for myself: it took me a long time to understand many of the things the programming-language community was saying. Now I can see they’re doing languages in a certain way, but I’m not allowed to use that language at work, or I’m not using it — the system’s already written in Java or whatever.

So I’ve been doing my best to mine all the good ideas I can find, but state [the ideas] independently of the language that carries them. So there’s a page on why you should be worried about side effects: they don’t compose very nicely. First of all, the idea of composition of code isn’t something entry-level programmers really think about.

And the idea of the contractual nature of code: “I give you an X and you give me a Y.” Even in procedural code, if something is non-contractual, it quickly becomes a kitchen sink: if there’s no contract, then when I have to put another feature in, I’ll just put it right here. So the better they can hit the target of “this method has a clear purpose,” the less likely everyone who edits it is to start adding weird stuff.

Does that give you a flavor?

Audience: Yeah, it does. Thank you.

Audience: Hi, George. You mentioned that there are both go-links that the SWEs can reference for the concepts, and there’s also Gemini that adjusts based on the links provided. Do SWEs usually consult Gemini more, or the go-links more? Under what circumstance do they lean more toward consulting Gemini as opposed to the go-links?

George: Well, unfortunately, the answer is it’s too early to tell. The use of LLMs in industry is just beginning. For a long time, the LLMs were only looking at external content. So me seeing them look at internal content – I was like, “yay, it’s finding internal stuff.” I expect this trend will continue, and I’ll be able to answer your question a bit better later on. But the one distinction is that in this kind of scenario, your tech lead is specifically reviewing something that you are doing.

Audience: And I would imagine LLMs might often be used prior to sending it for review. I think LLMs could also help with reviews.

George: But in this case, this is someone else trying to communicate with you via a compact essay, if you think of it that way. This is the essay they would have written if they had the time.

Audience: Thank you.

Bill Scherlis (audience): So thanks for your talk, George. I’m curious about follow-up. The TLs go through the class, go off, do things, and then they answer your questions, and they like it. The question, though, is: after maybe four months, six months, they may have some issues that show up in their experience that are late-breaking — not anticipated in your instruction or in their early adoption. Do you have a mechanism for refresh, recharge, revisit: do the TLs come back after some period of time and say, “great class, great ideas, except this one thing, we just couldn’t get any traction on it”?

George: So this is a part I want to point to [shows slide with the video podcast].

We’ve got the written materials and the courses. We have this podcast kind of thing where we invite the alumni to be the guests (I assume the same thing kind of works with Zoom). With Google Meet you can be invited to the meeting, or you can have a read-only link and watch the live stream. We invite the whole company to the live stream, and the alumni get to be in the meeting so they can raise their hands. So there are certain things we’re doing to try to provide a relief valve for that.

In the early days, we made a conscious effort to forge a community around each one of these cohorts, because in some areas that works well — like I understand MBA cohorts really stick. My brother got an MBA, and he stays in touch with those people as he moves forward. We found we could do that, but the effort on our organizing side was very high — we had to organize lunches, send emails, and do various things to keep it alive, because it’s just going to decay back to nothing. So we were unable to keep up that level of effort; we didn’t have the staffing to do that.

The idea of having them actually come back. I love that idea. Yeah, like a follow-up. They tell us what they don’t like.

Audience: So with organizational change being a peer-based hearts-and-minds game, I’m curious: were there any commonalities among the tech leads that determined successive mentoring, or successive adoption of the techniques?

George: Well, I’m trying to think back on what the feedback said. One of the things they said in the feedback was that when one person went back to their larger group, they felt like a voice alone in the forest. But as soon as they could advocate to their management to get two or more people, it started to get traction inside the group.

So they keep coming back to this idea of critical mass. If you have three tech leads saying “yeah, I think this is a good idea,” it stops feeling like George’s screwball idea and starts becoming “yeah, a lot of the people are talking about this idea.” But you’re exactly right as far as a social dimension, the organizational-change dimension (which I’m a novice at) I’m sort of wandering into it and doing my best.

David Garlan (host): So, George, I’m sure there are many more questions people could ask, but we should stop. Let’s thank George again.

Transcript prepared from two ASR passes of the talk audio, cross-checked against the recording. Timestamps are approximate (±30s).

Bibliography

The slide deck includes an appendix of readings assigned in the SWEdu Tech Lead Seminar, but each slide entry has just enough information to identify the work (a short title, author, and rough year), not a full citation. Below, each entry has been resolved to a full citation and, where one exists, linked to a legitimate open-access or archived copy rather than a paywalled one. Where no free copy could be found, the official (paywalled) link is given instead — no pirated or paywall-bypass links are used. A few entries reference internal Google documents (design guides, internal courses) that aren’t publicly available; these are listed for attribution only, with no link.

A machine-readable version of this list is available as a BibTeX file.

Design & general readings

• Fairbanks, George. “The Pragmatic Designer: Intellectual Control.” IEEE Software 36, no. 1 (January 2019). doi:10.1109/MS.2018.2874294
• Fairbanks, George. “The Pragmatic Designer: Fix Technical Debt with Virtuous Cycles.” IEEE Software 40, no. 2 (March–April 2023). doi:10.1109/MS.2022.3228623
• Beck, Kent. “Revealing Intent” (a named pattern in Smalltalk Best Practice Patterns). Prentice Hall, 1997. ISBN 013476904X.
• DeRemer, Frank, and Hans Kron. “Programming-in-the-Large Versus Programming-in-the-Small.” IEEE Transactions on Software Engineering SE-2, no. 2 (March 1976): 80–86. doi:10.1109/TSE.1976.233534 (paywalled; no free copy found)
• Pike, Rob. “Simplicity is Complicated.” Talk, dotGo, 2015. Video · Archived
• Wikipedia contributors. “OODA loop.” Wikipedia, The Free Encyclopedia. Accessed 6 August 2026. Stable revision
• Schuster, Jonathan, and George Fairbanks. “Design by Contract Class,” 2023. Internal Google document — not publicly available.
• Fairbanks, George. “E-type and S-type Code,” 2024. Internal Google document — not publicly available.
• Burrows, Michael. “Abstraction and Specification.” Internal Google document — not publicly available. (For a well-known public work on a similarly named topic, see Liskov, Barbara, and John Guttag, Abstraction and Specification in Program Development, MIT Press/McGraw-Hill, 1986 — connection unconfirmed.)
• Fairbanks, George. “Design Guide: Quality Attributes,” 2025. Internal Google document — not publicly available.
• Dijkstra, Edsger W. “On the Role of Scientific Thought.” EWD447, E.W. Dijkstra Archive, University of Texas at Austin, 1974. Origin of the term “separation of concerns.” Reprinted in Selected Writings on Computing: A Personal Perspective, Springer, 1982, pp. 60–66.
• Parnas, David L. “The Secret History of Information Hiding.” In Software Pioneers: Contributions to Software Engineering, edited by Manfred Broy and Ernst Denert, 398–409. Springer, 2002. doi:10.1007/978-3-642-59412-0_25 (paywalled; no free copy found)
• Parnas, David L. “On the Criteria To Be Used in Decomposing Systems into Modules.” Communications of the ACM 15, no. 12 (December 1972): 1053–1058. doi:10.1145/361598.361623
• Parnas, David L. “Designing Software for Ease of Extension and Contraction.” IEEE Transactions on Software Engineering SE-5, no. 2 (March 1979): 128–138. doi:10.1109/TSE.1979.234169 (paywalled; no free copy found)
• Abelson, Harold, and Gerald Jay Sussman. “Lisp: A Language for Stratified Design.” MIT AI Memo 986, August 1987. Also published in BYTE 13, no. 2 (February 1988): 207–218.
• Fairbanks, George. Just Enough Software Architecture: A Risk-Driven Approach, ch. 11 §3. Marshall & Brainerd, 2010. ISBN 9780984618101. Borrow the full book via the Internet Archive.
• Wikipedia contributors. “Conway’s law.” Wikipedia, The Free Encyclopedia. Accessed 6 August 2026. Stable revision
• Fairbanks, George. “Error Handling Guide,” 2023. Internal Google document — not publicly available.
• Dijkstra, Edsger W. “The Humble Programmer.” 1972 ACM Turing Award Lecture. Communications of the ACM 15, no. 10 (October 1972): 859–866. Also EWD340. doi:10.1145/355604.361591
• Miller, Rob, Max Goldman, and MIT 6.102 course staff. “Reading 7: Abstraction Functions and Rep Invariants.” MIT 6.102 (Software Construction), Spring 2023. (Closest verified match to the reading list’s “Mapping Internal Rep to External Abstraction”; no reading with that exact title was found.)
• Schuster, Jonathan. “Typeful Programming,” 2023. Internal Google document — not publicly available.
• Fairbanks, George. “Continuous Design of IT Systems.” Talk, SATURN Conference, 2019. (The reading list gives the year as 2024; no 2024-dated Fairbanks work by this title could be found — this 2019 talk is the closest verified match.)
• Fairbanks, George. “Code is King; Let’s Think in Code.” Talk, GSAS Conference, Barcelona, October 2019.
• Fowler, Martin. “Design Stamina Hypothesis.” martinfowler.com, 2007. Archived
• Fowler, Martin. “Tradable Quality Hypothesis.” martinfowler.com, 2011. Archived
• Fowler, Martin. “Is High Quality Software Worth the Cost?” martinfowler.com, 2019. Archived
• Naur, Peter. “Programming as Theory Building.” Microprocessing and Microprogramming 15, no. 5 (1985): 253–261. doi:10.1016/0165-6074(85)90032-8 · Archived

Software architecture readings

• Garlan, David, and Mary Shaw. “An Introduction to Software Architecture.” Technical Report CMU-CS-94-166, Carnegie Mellon University, January 1994. Also published as CMU/SEI-94-TR-21 and as a chapter in Advances in Software Engineering and Knowledge Engineering, Vol. I, World Scientific, 1993, pp. 1–39.
• Fairbanks, George. Just Enough Software Architecture: A Risk-Driven Approach, ch. 1. Marshall & Brainerd, 2010. ISBN 9780984618101. Free chapter excerpt (PDF) · Borrow the full book via the Internet Archive.
• Fairbanks, George. “Software Architecture Guide,” 2023. Internal Google document — not publicly available.
• Keeling, Michael. “The Psychology of Architecture Decision Records.” IEEE Software 39, no. 6 (November–December 2022): 114–117. doi:10.1109/MS.2022.3198195
• Keeling, Michael, and Joe Runde. “Share the Load: Distribute Design Authority with Architecture Decision Records.” Experience report, Agile2018 Conference, August 2018. Mirror (Agile Alliance)
• Shaw, Mary. “Procedure Calls Are the Assembly Language of Software Interconnection: Connectors Deserve First-Class Status.” Technical Report CMU/SEI-94-TR-002, Software Engineering Institute, Carnegie Mellon University, January 1994.
• Shaw, Mary, and Paul C. Clements. “A Field Guide to Boxology: Preliminary Classification of Architectural Styles for Software Systems.” 21st International Computer Software and Applications Conference (COMPSAC ‘97), 6–13. IEEE Computer Society, 1997. doi:10.1109/CMPSAC.1997.624691
• Fairbanks, George. “The Pragmatic Designer: Software Architecture Is a Set of Abstractions.” IEEE Software 40, no. 4 (July–August 2023). doi:10.1109/MS.2023.3269675

Testing readings

• “Google Testing Guide,” 2025. Internal Google document — not publicly available. All testing readings on the SWEdu reading list are drawn from this guide.

ABSTRACT: By adding a short, clear specification to an implementation and tests, developers build a stable unit: a stable triangle. Each leg can be analyzed, letting developers catch bugs and improve quality. They are a partial antidote to rising complexity and encourage edits in suitable places.

It’s time for developers to deliver units of code consisting of a specification, an implementation, and tests. This simple idea deserves a simple name. I call it a stable triangle because it reminds me of how mechanical engineering structures get their stability from triangles.

Some readers may already have a bias against specifications, so take a look at the stable triangle shown as pseudocode in Figure 1. Notice how normal it looks. Anyone who was worried that a specification (a spec) is a wall of text should be exhaling in relief. In fact, the spec is the shortest part.

Figure 1: A stable triangle for removeDups consisting of a specification, implementation, and several tests.

Specs are an old idea, just like tests. Today, developers typically deliver an implementation and tests, but it wasn’t always this way. For decades, developers delivered only the implementation code. Starting around the year 2000, practices shifted and developers began including tests alongside the implementation.

Consider what else was changing. In the 1990s, software was typically released just a few times per year, run in isolation on personal computers, distributed on compact discs, bought in brick-and-mortar stores, and sold in shrink-wrapped boxes. As the world grew more complex and less forgiving, developers improved their practices. Tests improved code reliability and developer productivity, so they were a good antidote for rising complexity.

Today, software is woven into our lives to the point where a bug can lock you out of your car, a missed email can trigger a tax audit, and a failed software update can lock millions of people out of their computers. Most people interact with software from the moment they wake up (perhaps from an alarm on their phone) until they go to bed (perhaps with software running their toothbrush). Software bugs pervade our lives, so our software engineering techniques aren’t working well enough.

A quarter century ago, developers faced with rising complexity dug into the software engineering toolbox and pulled out testing. The world has grown more complicated and it’s time to pull out another complexity-reducing technique: specifications. In the next few sections, I’ll explain what happens when you embrace specs, how they enable you to catch more problems, and how they discourage edits in inappropriate places.

Sidebar: Offered and imposed specifications

Developers are understandably allergic to specifications. They associate them with whole-program specifications, specifications “thrown over the transom”, waterfall processes, and non-experts oversimplifying the job of software development. Let’s examine specs from a different perspective.

Imagine that you own a factory that makes various bolts. Your customers are price-sensitive. To help them choose the right bolt, you offer specifications, telling them that your expensive stainless steel bolts are suitable for wet conditions and the cheaper ones are not.

Changing the perspective makes a world of difference. Developers aren’t allergic to all specifications; they are allergic to imposed specifications. The factory isn’t imposing specifications, it’s offering specifications. When you offer specifications, you are explaining your creation. Developers should offer specifications for their code so that clients know how to use it. When you design and implement code, you are the world expert on it: nobody knows that code better than you. When you offer your clients specs, they know what it does and you can warn them about any tricky bits.

Anatomy of a stable triangle

A stable triangle has three corners: specs, implementation, and tests. The spec says what the implementation promises to do, the implementation has the mechanism to do it, and the tests check that the implementation does what the specs promise. You may know that already.

What’s less commonly known is that having all three parts provides stability. Let’s approach that idea gradually and first consider what happens when we skip tests and specs, delivering only the implementation. What can and can’t we do with just an implementation? Without tests, we forfeit the assurance that edits don’t break what already works. Less obviously, we have no record of what the system is supposed to do – except in our memories – because tests provide a durable record of intended behavior. Hopefully you are already sold on tests, so let’s move on.

What if we had tests, but no specs? To some extent, tests act as specs, but unit tests check specific cases, not the general case. Unit tests can assert that add(1,1) is 2 and add(1,2) is 3, and so on, but those are specific cases. Tests are a safety net, but a net with many holes. Without a spec, there’s no durable record of what the system should do in general. There are two big implications: bugs and implementation details.

Without a spec for the general case, the idea of a “bug” is subjective. Imagine that your team’s code passes its tests but a user says there is a bug, pointing to an untested case. How do you decide if it’s a bug? Since there’s no test for that case, there’s no test to act as a spec either. One developer might agree with the user but another might disagree, insisting it’s not a bug. When we have a general spec, it’s easy to decide what’s a bug: A bug is a behavior of the implementation that breaks a promise in the spec.

Most developers believe they can identify the “implementation details” and can avoid testing them. But what is an implementation detail, exactly? It’s any behavior of the implementation that goes beyond the spec. For example, the code in Figure 1 sorts the list before returning it. Without a spec, developers may disagree about whether sorting should be tested. With a spec, it’s easy to decide: test everything that’s promised in the spec, ignoring anything else the implementation does (i.e., sorting and using a hashmap).

Stable triangles let us compare each part with the others, which gives us confidence that the entire thing does what we expect.

Figure 2. A stable triangle has a specification, implementation, and several tests (the corners). Each leg (A, B, and C) can be analyzed, which provides stability.

• (A) Spec-to-implementation. Does the implementation omit or break any specified behaviors? Those are bugs. Pay particular attention to failure cases. Does the implementation do more than is specified? Those are implementation details, as long as they don’t break any promises in the spec.
• (B) Implementation-to-tests. Does the implementation pass all the tests? Do the tests cover any implementation details?
• (C) Tests-to-spec. Is every case in the spec covered by one or more tests? Sometimes the spec will have multiple clauses, such as “if x is positive, then …, otherwise …” which alerts you to cases. Are the chosen tests representative coverage of the cases? Is the strongest form of testing being used, for example using a property or fuzz test when possible?

Why now?

Developers should embrace specs because the world has changed in gradual and abrupt ways. The gradual changes are the slow rise in the complexity of our systems and the quickening of the development cycle. The abrupt change is generative artificial intelligence (GenAI) that can write and edit code.

Developers embraced testing when compute power became cheap. They were confronting rising complexity and they used the compute power to run tests. There’s still more opportunity to use stronger testing, such as model-based testing, property-based testing, and fuzzing.

That cheap compute power also sped up development cycles and helped developers with complexity. Today, some developers are lucky and experience nearly instantaneous development cycles (i.e., code that’s continually incrementally compiled and deployment happens in seconds). Most are not so lucky and must wait hours or days.

That brings us to the elephant that has trumpeted into the room: GenAI. We all expect a big shakeup but are unsure about when and what. Here, I’ll make some observations about the nature of software design and engineering that I hope are fundamental and stay relevant despite our uncertain future.

People have cognitive limits. As they solve problems, they can keep only so many ideas in their head. They have an easier time solving problems that are well-defined and can be reasoned about locally. GenAI seems to follow this pattern too. It has limits on its attention and the amount of context it can use productively. It works better on well-specified problems because it can check that its solutions are correct. And it solves big problems by breaking them into smaller problems, so local reasoning is desirable.

Specifications seem to help people and GenAI in similar ways. Specified problems are well-defined problems. When a written spec has ambiguities or oversights, as they often do, it’s possible to identify them and improve the spec. As the saying goes: a problem well-stated is half-solved.

A spec makes it possible to create different solutions. Without a written spec, both humans and GenAI must guess (as discussed above) about implementation details and bugs. From a spec, you can write a suitable set of tests, and with both specs and tests in hand you can write one or more implementations.

Nesting specs makes it possible to limit what’s on your mind. Both people and GenAI solve problems through a divide-and-conquer strategy. If you have a spec for the overall problem and specs for each of the sub-problems, you can collapse entire sub-trees of the problem and think only about their specs. For example, your car audio system might need 12 volts and there is a lot of complex engineering to ensure a stable 12 volts, but you can collapse all of that complexity into its spec – there’s 12 volts on this wire – freeing those details from your mind and making space to work on the problem at hand. This is exactly Edsger Dijkstra’s argument for separation of concerns: when we focus on one aspect of a problem, it’s not that we are blind to complexity elsewhere, it’s that we allow our full brainpower to work on one aspect of the problem before turning to the next.

Evolution

As developers, we’d like to believe that more time leads to better code. Instead, what happens in practice is that systems grow buggier and more complicated with each edit. Why is this? Some edits degrade the code. They change code that was working, complicating it, and sometimes even introduce new bugs.

Consider two scenarios where you want to add a feature. First, imagine that you have an unspecified method with a vague purpose, something like handleEvents. That vague method looks like fair game to be edited. So tempting! Just add a few lines of code, write one more test, and you’re done. Second, imagine that you find a stable triangle with a well-defined spec, something like hideElements. As soon as you see that clear spec you pause. This is already a cohesive unit. Your edit doesn’t fit – and you might break callers depending on the existing contract.

Stable triangles encourage edits in suitable places. They invite you to compose triangles rather than edit an existing triangle. This is where evolution connects to process. A developer who scatters edits across the code isn’t building up coherent, stable, trustworthy units. A developer who focuses on one thing, gets it working, checks its quality, then moves onto the next is building up value.

Confidence from simplicity

When I write specs, I strive to keep them at 1-2 lines of text. For simple methods I just say what happens in the success and failure cases, often using this template: “Returns X if Y, otherwise Z.” When the method is more complicated, I ask myself if I can simplify it. Sometimes I can. Sometimes it’s hard to describe because my vocabulary (i.e., the types in my program) is too simple, and adding a type lets me state the spec simply. The more complex my spec is, the more testing I must do and the less I trust it will work as I expect, so it’s worth my time to simplify.

Stable triangles are an affordable luxury. Tests and implementations become stable and easy to analyze when you add specs. Working without specs forces me to guess how the code works, so it’s a self-imposed cognitive impairment. I’m long past any “real programmer” bravado where I pride myself on writing tricky code. It’s hard to write good code and I’m more likely to succeed when I can navigate the options to find simple code. By cycling through the spec, implementation, and tests, I engage in a virtuous cycle where I evaluate the problem from those distinct perspectives and look for ways to simplify. [1]

Most of all, I appreciate the incremental gratification. Some developers zoom across the code making edits in many places, like a modern Icarus. I feel a bit like Daedalus, focusing on one thing at a time, accumulating each stable triangle as a small win, knowing those wins add up, and I have the confidence that each stable triangle will support what comes next. The higher you fly, the more you must trust your contraptions.

References

1. G. Fairbanks, “Fix Tech Debt With Virtuous Cycles” in IEEE Software, vol. 40, no. 02, pp. 111-116, March-April 2023, doi: 10.1109/MS.2022.3228623.

ABSTRACT: Programming is an example of extended cognition: A developer’s ability to program is enhanced by source code that is a suitable external representation and by tools to manipulate it. When structures are aligned between mind and code, they can be swapped in and out like virtual memory pages; using external tools can be quicker and more accurate than thinking alone. A developer, source code, and tools fuse into a coupled system, which enables work on bigger problems, more quickly, and with fewer errors. However, the extended cognition phenomenon is fragile. Code is typically treated as a machine that can be left alone unless new features are desired. As a consequence, the coupling between developer and code slips away, taking productivity with it.

Software engineering is an intensely cognitive activity. Faced with millions of lines of code, however, no developer tries to memorize it all, nor do they try to use their minds alone to test, typecheck, or compile it. To enable their work on anything larger than tiny programs, developers amplify their minds by externalizing their memory and reasoning with external tools that are larger, faster, and more accurate than their own minds.

Software engineering is an intensely cognitive activity. Faced with millions of lines of code, however, no developer tries to memorize it all, nor do they try to use their minds alone to test, typecheck, or compile it. To enable their work on anything larger than tiny programs, developers amplify their minds by externalizing their memory and reasoning with external tools that are larger, faster, and more accurate than their own minds.

Few developers or managers notice the extent to which productivity depends on collaboration with the environment. Consequently, they may take developer productivity for granted and focus exclusively on financial concerns to steer projects. To steer well, it’s important to understand where developer productivity comes from and how it slips away.

Our own culture is invisible to us until we can step outside of it. To see programming as the remarkable feat that it is, we must shift our perspective. I’ll show a few simple examples – Tetris, Rubik’s cube, and addition – so we have a clear view of the cognitive phenomenon called extended cognition, then it will be easier to see programming as extended cognition.

What is extended cognition?

Extended cognition is the idea that “… parts of the world can be parts of cognitive processes and that cognitive processes are not all in the head.” [1] Philosophers have a variety of names for variations on this idea: extended cognition, embedded cognition, embodied cognition, and enactive cognition. [2]

Playing Tetris is a simple example of extended cognition. You could play the game fully in your head, deciding how to rotate a shape and where it will fit. That’s not what players do, however, because the game will instantly and flawlessly rotate the shape for you, much faster than you can do it in your head. The same thing happens with Rubik’s cubes: it’s faster and more accurate to rotate the cube face in the world than it is to imagine rotating it in your mind. Your mind speeds up by collaborating with an external tool.

Tetris and Rubik’s cube have another similarity: There’s a lot of detail to keep track of. A Rubik’s cube has nine squares per face and six faces – quite a lot to keep in your head – and each turn of the cube rearranges those squares. In Tetris, the accumulated shapes at the bottom change as the game progresses. Players remove a burden from their memory by collaborating with the world around them. They give their memory a rest and let the world remember it. Your mind speeds up because the external representation is quicker and more accurate than your own memory.

Extended cognition works because a mind can collaborate with the world in two ways: perception and action. People perform better when they can speed up their Observe - Orient - Decide - Act (OODA) loop. If it takes less time to push a key so the computer rotates a shape than it does to rotate a shape in our heads, then we’ve sped up our OODA loop and boosted our productivity.

A fragile superpower

Extended cognition gives us a superpower, but that superpower is fragile: The productivity boost can disappear when there are minor changes in the world that break the alignment between our thoughts and the world. The difference in productivity is called the representation effect. People can add small numbers in their heads, but after a few digits it’s hard to keep straight, so they use extended cognition to add big numbers. [3] Children are taught how to make marks on paper so that adding huge numbers becomes easy.

Consider the two sets of marks on paper in Figure 1. Both the left and right have exactly the same text, but only one of them is a suitable external representation.

Figure 1: Two external representations for adding numbers. The right side is a more effective external representation for extended cognition.

In hindsight, it’s obvious that, for superpowered addition, we need to have the marks line up in columns, so using a proportional font disrupts the columns and productivity collapses.

Consider how extended cognition might break for a Rubik’s cube or Tetris. If a cube had different shades of the same color, not distinct colors, we’d likely be much slower to solve it. The Tetris game is fully playable if a shape could be rotated at most three times, but a player would need to know in advance what rotation they want, which means they’d need to use their mind.

The basic finding [of psychological studies on the representation effect] is that different representations of a problem can have dramatic impact on problem difficulty even if the formal structures are the same. [3]

There are two distinct lessons here. The first lesson is that seemingly innocuous changes can break extended cognition and collapse productivity. The second, non-obvious lesson is that it’s hard to imagine extended cognition being different than it is today. We anchor our expectations on how we can think now. Accountants using Roman numerals didn’t complain that they were being held back and computers (the name for people who manually calculated without machines) thought that what they did was natural. Just a few decades ago, developers could not imagine the productivity boost we enjoy today from automated testing, integration, and deployment. History tells us that we should not anchor to the present and should expect that a changed environment will bring us new superpowers.

Developers can thrash too

Success at programming depends on extended cognition. Programs are too big to fit into our heads and programs require analysis – such as typechecking, testing, and compiling – that far exceeds our minds. The only reason we can program is because of the extended cognition enabled by tools and a suitable external representation in code; unsuitable tools or code destroys that superpower.

As a thought experiment, imagine a program that would take you a month to write, then consider what happens when you take away the environment you depend on: an external record of the source code, tools like searching and replacing, semantic cross-referencing, automated regression tests, compilers, linters, type checkers, production monitoring, and so on. How long would it take you to write that same program take with just pen and paper? Or entirely within your mind? Your ability to program is intricately linked to externalized source code and its tools.

A computer has a hierarchy of memory that varies in speed of access and size: registers are small and fast, spinning disks are big and slow. To operate on data, it first moves data into registers, and it may use special-purpose hardware, such as matrix multiplication. Developers using extended cognition work this way too. They turn their attention to a relevant subset of the external representation, much like loading data into registers, and they use external computation instead of their slow and error-prone minds.

Thrashing happens when a computer’s working memory is smaller than the working set of data it needs. Instead of running at the speed of electronic memory, it runs at the speed of the hard drive – the slowest link in the chain – shuffling data from there into electronic memory.

Developers can thrash too. When their thoughts are misaligned with the source code or tools, the system becomes hard to understand and evolve. Consider: when automated tests are working reliably, developers can quickly decide if recent code changes are working as intended, but a few flaky tests will reduce productivity to a crawl. Using just their minds, developers must decide if the test failures are genuine or false alarms. Instead of (automatically) evaluating thousands of tests per second, a developer (manually) takes minutes or hours to decide that a test failure is erroneous. When extended cognition fails, developers drop back to operating at the speed of their minds – the slowest link in the chain – not the superpowered speed they can reach with extended cognition.

Ur-technical debt

Today, the term technical debt is used to describe any code that’s undesirable. I believe that Ward Cunningham meant something different when he coined the term, so I use the term ur-technical debt (where the prefix ur- means original) to discuss the narrower meaning. [4] To me, the way he described tech debt sounds like extended cognition. [5]

[I]f you develop a program for a long period of time by only adding features and never reorganizing it to reflect your understanding of those features, then eventually that program simply does not contain any understanding and all efforts to work on it take longer and longer.

There are two ways to keep alignment between an external representation and a mind. The first is curation, where you adapt the external representation to match the mind. That’s what I think Cunningham was talking about. The second way is retraining, where you adapt your mind to the external representation. In practice, you must do both curation and retraining. Like gardeners, a team must continually curate their code so it is a suitable external representation and a new developer must retrain their mind to match a team’s existing code.

Developers can compensate for ur-technical debt, but it hurts their extended cognition. They can memorize quirks (“when the code says X, think Y”) but that mental translation is slow and error-prone. If a method is named save but sometimes doesn’t save then, sooner or later, a developer will mistakenly assume it does what its name says. So, developers can compensate but they have cognitive limits and each compensation hurts productivity.

Systems start out small and easily within our cognitive limits, so it’s easy for developers to cope. They can look at code that disagrees with their thoughts, yet stay productive. Adding features and fixing bugs makes the system more complex, which accumulates ur-tech debt and pushes developers closer to their cognitive limits. I’ve seen teams transition quickly: One month they’re mostly adding features and the next they’re mostly fighting fires. I’ve also seen them improve productivity by investing in the code and automation.

Code as machine and as thought

To see our own culture, we must shift our perspective. If the examples here have done their job, you may now have a different perspective on ur-tech debt: it hinders extended cognition and developer productivity. You can recognize that code has two natures. Code’s first and obvious nature is that it’s a machine that does valuable work. Code’s second and less obvious nature is that it’s the external representation of developers’ thoughts. [6] Ward Cunningham contrasts code’s machine nature with its thought nature this way [7]:

Although immature code may work fine and be completely acceptable to the customer, excess quantities will make a program unmasterable, leading to extreme specialization of programmers and finally an inflexible product. … Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation …

Code can “work fine” because of its machine nature and yet be “unmasterable” because of its thought nature. Code’s machine nature is easy to measure; code’s thought nature, like anything related to developer productivity, is stubbornly resistant to measurement.

Leaders can fall into the McNamara Fallacy: they measure what they can and discard what’s not measurable. They treat ur-tech debt, developer productivity, and extended cognition as unmeasurable and therefore irrelevant. This is a mistake because, sooner or later, it steers every project into low productivity. Focusing exclusively on what’s measurable isn’t practical or hard-nosed; it ensures a bad outcome.

So, how do you create and maintain good conditions for extended cognition? The answer is simple: train your developers, keep track of where the code diverges from their thoughts (i.e., ur-tech debt), allocate time to clean up that ur-tech debt, and invest in automation. There’s lots of nuance but that’s the gist. On a well-managed project, both natures are nurtured.

If your team has fallen into the McNamara Fallacy or you’ve inherited code with a lot of ur-technical debt, you can recover. First, gain consensus about the situation, specifically that ur-tech debt and extended cognition isn’t measurable but it’s still important. Second, gather evidence including anecdotes. The trauma of a recent system outage or loss of a developer isn’t quantitative, but it can inform a rational decision. Finally, strengthen your team’s ability to make apples vs. oranges decisions. By starting small, you can calibrate your instincts and build a track record that leads to good conditions for extended cognition.

References

1. Chalmers, David J., Extended Cognition and Extended Consciousness, in Matteo Colombo, Elizabeth Irvine, and Mog Stapleton (eds), Andy Clark and His Critics (2019; online edn, Oxford Academic, 23 May 2019), https://doi.org/10.1093/oso/9780190662813.003.0002.

2. Sprevak, Mark. Extended Cognition, 2019, doi:10.4324/9780415249126-V049-1. Routledge Encyclopedia of Philosophy, Taylor and Francis.

3. Jiaje Zhang, Donald A. Norman, Representations in distributed cognitive tasks, Cognitive Science, Volume 18, Issue 1, 1994, Pages 87-122, ISSN 0364-0213.

4. George Fairbanks, Ur-Technical Debt, IEEE Software, Vol 37 number 4, July/August 2020.

5. Ward Cunningham, Ward Explains the Tech Debt Metaphor, February 14, 2009, Video transcript.

6. George Fairbanks, Code Is Your Partner in Thought, IEEE Software, Vol 37 number 5. September/October 2020.

7. Ward Cunningham, The WyCash Portfolio Management System, OOPSLA 92, Vancouver, British Columbia, Canada, Addendum to the Proceedings, Experience Report, 5 - 10 October 1992.

ABSTRACT: Developers want to steer the quality attributes in their system. For example, they want to keep latency low, security high, and the code maintainable. To achieve this, they typically roll up their sleeves and get to work rather than deliberately employing design techniques that would steer the desired qualities. That’s because the connection between software design and quality attributes is not widely understood. This column explains the connection and shows how software design can steer software qualities.

Engineering is complicated, so it’s easy to overlook a basic truth: The qualities that a system exhibits derive primarily from its design. But there’s a catch: We influence these qualities indirectly. So, if a bridge withstands high winds safely, you can thank the engineers who used abstractions. Yes, the bridge is constructed from bolts and steel, but stresses and strains guided how those bolts and steel could make it resilient.

Formal study of software architecture abstractions started in the 1990s and revealed how design was connected to qualities. In hindsight, it’s now clear that the reason a software developer would pay attention to architecture is precisely because it can steer quality attributes. To steer something, you need to be able to grasp a control, like a rudder, commanding it to do your bidding. Software architecture reveals the essential abstractions that give you the ability to steer.

Most developers do not learn this in school and developer-oriented articles or blog posts often call abstraction the villain, not the hero. Because they don’t understand how to steer the qualities of their software, they try things that don’t work so well. Software stubbornly ignores job titles, developer enthusiasm, and story points. This column provides a capsule understanding of how you can steer the qualities in your system using software design.

Steering at Netflix

In the 2000s, Netflix had a problem: when one of their services slowed down, its callers repeated their requests, which slowed the service down further. A vicious cycle. They fixed this with the Chaos Monkey [1], which, counter-intuitively, improved reliability by stopping services that were running in production. Reliability emerges from that technique because rare overload events become commonplace, so developers handle them, so the system becomes more reliable. It’s ingenious.

What’s less well-known is that Netflix developers already had a library that could handle the service overload problem, but they ignored that library. Overload conditions were rare enough that developers didn’t seek out solutions. Netflix could have changed developer behavior in various ways, such as adding compliance steps to their process, sending developers to education, or harsh punishment. Chaos Monkey changed developer behavior by making failures more common, thereby making the reliability problem visible. It’s even more ingenious.

Features are local, qualities are emergent. You can point to the code in Netflix that handles the streaming or account activation features. There’s no single place that handles qualities like security, usability, latency, or resiliency. To build a feature, you write some code. To influence a quality, you typically need to write code, but only after using abstractions that help you reason about how that quality can emerge from the system as a whole.

Netflix improved reliability by structurally changing the overall system. They created Chaos Monkey as a lever of control and they wrote a software library to handle overload conditions. In those actions, you can glimpse a general framework to steer software qualities: choose a strategy, create levers of influence, and apply design techniques. The next few sections describe that framework. There are a lot of figures that summarize the ideas and introduce terminology and you may want to glance at the figures first, then return to the text.

Strategy

Figure 1 shows three broad strategies for steering software qualities. The first is named ad hoc because developers using this strategy work on qualities only when the need becomes acute. Some developers at Netflix appear to have been focused on feature development, oblivious to design techniques to achieve reliability.

Many teams create a stream of work that combines both feature and quality requests, prioritized by customer needs. Developers metaphorically roll up their sleeves and get to work on qualities, which works ok if they lucked into an architectural style that matched their needs, say a three-tier style for an IT system. Too often they are unlucky: quality needs become apparent long after the system design is set in stone.

A local strategy recognizes the need to steer qualities from the beginning. Developers may be advised to always make certain modules idempotent, or stateless, or free of side effects. The advice often takes the form: when X, don’t forget Y. If such advice were easy to follow, we would have many fewer buffer overruns and memory leaks.

A structural strategy is the one I wish all developers understood. Often, you can set up initial conditions on a project so that it’s easy to achieve the qualities you want with little effort or vigilance. Chaos Monkey set up conditions by which Netflix became reliable without constant vigilance. It creates the opportunity for a few experts to own something complicated, such as the queueing theory needed to optimize message flow between services, removing that burden from most developers.

Figure 1: Strategies for achieving desired qualities

Few projects start with a clear understanding of quality goals and tradeoffs, e.g., latency is prioritized over maintainability. Those that do tend to work locally and with vigilance to achieve the quality. Few use a structural strategy: promoting desired qualities via the system’s architecture. [3]

Levers of influence

Trying to change the qualities in a system or organization feels like pushing Jello. You must first create levers of influence, then you can operate them. Before Chaos Monkey, Netflix had a clear desire to improve reliability – they had even written a library – but lacked a suitable lever of influence. If you are lucky, your organization already has some levers, but typically they are missing. There might also be obstacles, for example your company has an education curriculum but you cannot change it.

Levers of influence exist in different scopes. Figure 2 shows three scopes: software architecture and design, the software lifecycle, and systems engineering. There is a power-generality trade-off between them [2]. Systems engineering is the most general, but the least powerful; software architecture is the most powerful but least general.

Figure 2: Three scopes: architecture & design, software lifecycle, and systems engineering

They exhibit a power-generality tradeoff [2]: Systems engineering has many levers of influence over quality attributes that apply to many problems. Software architecture has a few levers that act only on the software itself, but they affect quality attributes directly and forcefully.

Figure 3 shows several examples of levers of influence in systems engineering. Chaos Monkey, educating developers, and exerting management pressure to use the library are levers in the scope of systems engineering. Perhaps Chaos Monkey should be in the scope of software architecture or the lifecycle – it can be hard to categorize some levers.

Changing hiring practices and curriculum can have a big impact on the software, though indirectly and after a long time. And consider that what gets staffed gets done: are there quality-specific job roles such as user experience engineer and site reliability engineer?

Between architecture and systems engineering is the software development lifecycle. Figure 4 shows several levers of influence in the software lifecycle (except design, detailed next). Every stage in the software development lifecycle is an opportunity to steer qualities. Do not skip past the requirements / analysis stage, as systems become tangled when quality priorities and tradeoffs are unknown or chosen inconsistently on each sub-team. Notice that Chaos Monkey by itself reduces reliability, but works great in combination with monitoring and alerting.

Software architecture and design has the most direct influence on qualities. Figure 5 shows four primary levers of influence in software architecture. Two classic levers of influence are architectural styles and tactics [3]. Both are large-scale patterns, with tactics typically nested within a style. Each style is known to promote (or inhibit) certain qualities. Once the style is chosen, tactics can be applied according to Attribute-Driven Design to further refine qualities.

More recently recognized levers include architectural hoisting [4] and thinking [5]. When you move a responsibility out of developers’ hands and into the infrastructure, that’s hoisting. Without hoisting, developers require vigilance. With hoisting, the infrastructure ensures that it’s done. Architectural thinking is an umbrella term for the use of architecture abstractions to achieve intellectual control over a system [5]. An alternative to architectural thinking is testing and statistical control.

Figure 5: Software architecture’s levers of influence

In principle, these levers are always available to steer qualities, but on any given system you may need to develop them. For example, a system that has become a big ball of mud lacks an architectural style [6], so you cannot use the style lever to promote a quality. [3] [4] [5]

Architecture and design techniques

Design techniques are the most fine-grained influence. Figure 6 shows several design techniques organized by viewtype: compile-time, run-time, and deployment/allocation. All developers will have experience with these techniques as they are daily or at least weekly activities. For example, refactoring code into a reusable library (compile-time viewtype) or reusable service (run-time viewtype) is common.

Design techniques work with architectural thinking. Most developers employ heuristics for writing code. If a system has challenging latency needs, however, developers must trade off latency with modifiability. In a few computation-intensive modules, they would optimize the code for speed, not modifiability. They are thinking architecturally: keeping quality priorities and tradeoffs in mind and identifying critical paths of computation.

The value of software design

You can promote or inhibit quality attributes through your software design choices. This column sketches a framework for understanding how to do that. With the ideas and terms here, you are in good shape to dig into each idea further.

From a bird’s eye perspective, this approach looks reasonable and perhaps even obvious. In practice, however, software development is often filled with intense pressure to build features and fix bugs. A simple version of the system is often demanded within a few weeks – little more than a prototype – but this prototype defines the architecture of the system. With these pressures, it’s tempting to assemble something from what’s handy or use your last system as a template for the next one. The project’s conditions can make it hard to use design techniques to steer qualities.

It’s increasingly common to see software development as a factory, with developers expected to be increasingly efficient at their stations. From this perspective, value is created only when deployed features are making money for the company, and you seek improvement by running a tighter factory, efficiently moving features into deployed code. [7]

I have a different perspective. The developer role is a combination of engineer and factory worker. A developer’s attention must be balanced between engineering work (such as design) and implementation work (such as coding and testing). If we embrace the factory metaphor too strongly then we’ll seek coding efficiency at the expense of engineering and design quality. In a tight factory, when will developers have time to learn and apply software design techniques?

The levers of influence over quality attributes aren’t just there for the pulling and pushing: to be effective, they must be created and nurtured. If a developer joins a project that’s already a big ball of mud, it may require heroic efforts to create those levers and steer quality attributes.

Some experts think that software architecture will naturally make its way into standard industrial practice: it’s following historical technology adoption patterns. I think software design has a marketing problem. People conflate software design with waterfall processes, which they reject, and with know-nothing corner-office architects, who they resent.

I want to overcome this marketing problem. I wish developers could see software design the way I do, that developers can design every day, and that design can range from tiny tasks all the way to systems engineering. Asking for time for design and architecture might sound like I’m asking for the software process to slow down, but that’s the opposite of what I want. There’s no evidence that cutting corners makes software development – or any kind of engineering – go faster, except for a temporary boost at the start. Ask anyone working on a big ball of mud about their velocity. No, I don’t want slow, I want to see developers at the top of their game, zooming along with every design technique available to them.

References

1. S. Anand, Keeping Movies Running Amid Thunderstorms, Nov 20, 2011, QCon San Francisco.

2. A. C. Bock, “The Power/Generality Trade-Off in Decision and Problem Modeling: Theoretical Background and Multi-level Modeling as a Resolution, Lecture Notes in Business Information Processing, vol 318, Springer, Cham., 2018, doi: 10.1007/978-3-319-91704-7_14.

3. L. Bass, P. Clements and R. Kazman, Software Architecture in Practice, Addison Wesley Longman, 2021.

4. G. Fairbanks, Architectural Hoisting in IEEE Software, vol. 31, no. 4, pp. 12-15, July-Aug. 2014, doi: 10.1109/MS.2014.82.

5. G. Fairbanks, Intellectual Control, IEEE Softw., vol. 36, no. 1, pp. 91–94, Jan./Feb. 2019. doi: 10.1109/MS.2018.2874294.

6. B. Foote and J. Yoder, Big Ball of Mud Pattern Languages of Program Design, vol. 4, 1997.

7. G. Fairbanks, Why Is It Getting Harder To Apply Software Architecture?, in IEEE Software, vol. 38, no. 4, pp. 126-129, July-Aug. 2021, doi: 10.1109/MS.2021.3071520.

ABSTRACT: Software architecture is a set of abstractions that helps you reason about the software you plan to build, or have already built. Our field has had small abstractions for a long time now, but it has taken decades to accumulate larger abstractions, including quality attributes, information hiding, components and connectors, multiple views, and architectural styles. When we design systems, we weave these abstractions together, preserving a chain of intentionality, so that the systems we design do what we want. Twenty years ago, in this magazine, Martin Fowler published the influential essay “Who Needs an Architect?” It’s time for developers to take another look at software architecture and see it as a set of abstractions that helps them reason about software.

Twenty years ago, in this magazine, Martin Fowler published the influential essay “Who Needs an Architect?” [1] Today, when I ask developers where they learned about software architecture, they point me to that essay, directly or indirectly. The essay provides three definitions of software architecture, quoting Ralph Johnson for the definitions and commentary. After weighing the options, Johnson is critical of all three and metaphorically throws up his hands:

So, this makes it hard to tell people how to describe their architecture. “Tell us what is important.” Architecture is about the important stuff. Whatever that is.

Twenty years is a long time in computer science. When those words were written, some of the engineers I work with today were in diapers. With the benefit of hindsight, I will make a case for the third definition, that software architecture is a set of abstractions.

Johnson rejects the definition that architecture is “the highest level concept of a system in its environment,” retorting that “[t]here is no highest level concept of a system” because each stakeholder sees the system differently and developers are just one stakeholder. I agree. Describing architecture as “high level” is a convenient crutch when introducing the idea, but it does not stand up to careful scrutiny.

I prefer a variant of this definition that is in the same vein but avoids the “high level” trap: “The set of structures needed to reason about the system, which comprises software elements, relations among them, and properties of both.” [2] It focuses on reasoning, not levels. Architecture is what you need to reason about a system: Software elements, relations, and properties. Those are abstractions that let you reason about software, both before and after you build it.

Missing abstractions

But surely we already have plenty of abstractions! Computer science is swimming in abstractions. So many, in fact, that it’s easy to overlook what’s missing. Let’s review how we grew to understand architecture. Way back in 1975, Frank DeRemer and Hans Kron observed that developers have abstractions for writing data structures, methods, and modules, but not for assembling modules into systems [3].

[S]tructuring a large collection of modules to form a “system” is an essentially distinct and different intellectual activity from that of constructing the individual modules. That is, we distinguish programming-in-the-large from programming-in-the-small.

A few decades later, in 1993, David Garlan and Mary Shaw strengthened and generalized the argument, sketching out what we now call software architecture [4].

As the size and complexity of software systems increases, the design problem goes beyond the algorithms and data structures of the computation: designing and specifying the overall system structure emerges as a new kind of problem. Structural issues include gross organization and global control structure; protocols for communication, synchronization, and data access; assignment of functionality to design elements; physical distribution; composition of design elements; scaling and performance; and selection among design alternatives. This is the software architecture level of design.

Pause to consider their point. Could you reason about a system using only algorithms and data structures? Or do you find yourself going beyond these abstractions when you talk to other developers or think through the design of your system?

By 2010, after two more decades of innovation, the set of architecture abstractions had settled down. A group of authors collected the abstractions into a book titled “The Secret Abstractions of Software Architecture, Finally Revealed!” No, I’m pulling your leg. They actually titled it “Documenting Software Architectures” and released it when companies were abandoning heavyweight processes in favor of agile ones that discouraged documentation [2]. I think that’s why it’s not more famous.

A few years ago, when I read a book that told a story of innovations in mathematics, it stitched together ideas that had been independent islands in my mind. So, instead of reciting an inventory of the architecture abstractions, what I’ll do here is sketch a story of innovation. As summarized in Figure 1, the story has several related plotlines: specifications, structure, views, and patterns. I’m surveying decades of work, so this is an overview, not a complete inventory.

| Specification Abstractions             | Structure Abstractions             |
|----------------------------------------|------------------------------------|
| 1850s: Software and hardware           | 1950s: Modules                     |
| 1940s: Features and quality attributes | 1970s: Information hiding          |
| 1960s: Interfaces and implementations  | 1990s: Components and connectors   |

| View Abstractions                      | Pattern Abstractions               |
|----------------------------------------|------------------------------------|
| 1960s: Compile-time and run-time views | 1990s: Named architecture patterns |
| 1990s: Multiple views                  | 1990s: Styles promote qualities    |
| 2000s: Viewtypes                       | 2010s: Patterns in every viewtype  |

Specification abstractions

A specification is a precise description or clear identification of something. In the earliest days of computing, there were no specifications of software because the concepts of hardware and software were intertwined. People set out to build useful machines such as Babbage’s analytical engine that, in the mid-1800’s, was designed to compute tables of numbers. Ada Lovelace not only wrote the first algorithm for it, she recognized that with suitable instructions the hardware could operate on any kind of symbol – that software could be distinct from hardware.

By the late 1940’s, software was in fact distinct from hardware. We had multiple algorithms for the same function, differing in their use of storage space or run time. People talked about speed-space tradeoffs. This is the germ of a critical idea in software architecture: specifying features vs qualities. Qualities, or more fully, quality attributes, go beyond just speed and space; they include latency, usability, modifiability, portability, and many more “-ities”. As you reason about software, you want to know not only about its features (what it computes) but its qualities (how it computes them). (You may have heard the term “non-functional requirements,” but it carries baggage: “non-functional” means “broken” and “requirements” can imply a waterfall process. It is convenient to discuss qualities of designs – say that one has better latency – without stating a requirement or suggesting a development process.)

By the 1960’s, it was commonplace to compile and link programs in separate steps. That required a new abstraction: specifying a subroutine interface separately from its implementation. A program could depend on an interface – say, sorting – but wait until linking to choose a fast or space-efficient implementation. This abstraction helps you reason about a program from its interfaces, letting your mind skip past the implementations. As with all abstractions, you lose detail that way, but in return you gain the ability to reason about larger programs.

Structure abstractions

Software architecture depends on a second group of abstractions, ones related to the structure of the program. In the early 1950’s, David Wheeler identified the need for reusable chunks of code, introducing subroutines and libraries. Subroutines were grouped into modules and modules, like subroutines, were split between interface and implementation. By the 1970’s, David Parnas saw that some ways of modularizing are better than others. Sometimes a change to a module required changes to its neighbors, other times not. Module interfaces could be designed to hide the details that might change, called information hiding. A program designed with information hiding would work the same (have the same features), but be easier to modify (have different quality attributes).

By the mid 1970s, programs were large enough that programmers complained that while they could understand any given piece, they had difficulty understanding the whole program. “[C]urrent languages discourage the accurate recording of the overall solution structure; they force us to write programs in which we are so preoccupied with the trees that we lose sight of the forest, as do the readers of our programs!” [3]. This is where the terms programming-in-the-small and programming-in-the-large originated.

In object-oriented programming, there is a clear distinction between a class and an object: A class representing a person can have several instances, one each for Ann, Bob, and Carl. A similar type-versus-instance distinction for modules was not made clear until the early 1990s. At that point, the terms module and component were no longer used interchangeably: Modules exist at compile-time and components at run-time.

With the advent of computer networking in the late 1960’s, it was necessary to describe the interactions as protocols. It was not until the early 1990s, however, that interactions between components had a first-class abstraction: connectors. Connectors express protocols and much more. Examples of connectors include call-return, publish-subscribe, and pipes. The implementation of a connector often requires a lot of code, organized into many modules. A procedure call is a simple connector that developers use to implement more complex connectors, for example, remote procedure calls or event-based connectors.

View abstractions

When designing physical structures like houses or bridges, it’s common to create diagrams showing the structure from different perspectives, or views. Views play a critical role in software architecture. In the late 1960’s, Edsger Dijkstra observed that mentally animating code is difficult and error-prone, so programs should be written in a structured way, so that it’s easier to look at the code and envision how it behaves. Said another way, developers stare at one view (the code), imagine another view (its runtime behavior), and reason about how changes to one affect the other.

In the mid-1990’s, Philippe Kruchten identified views as a useful abstraction for software architecture. Not just compile-time and run-time views, but also concurrency and deployment to hardware. Each view enables different kinds of reasoning, perhaps needed by different people on the team.

The halting problem says that we cannot always look at code and say if it will run forever. It’s the extreme case of Dijkstra’s point: it’s hard to use one view (say, the source code) to reason about another (say, it’s runtime behavior). It’s similarly hard to look at code and answer: is this code running in production, and if so, where? Yet developers confronted by a bug report must reason not about the code in their repository, but the version of the code that users are interacting with. This leads to a final view abstraction: viewtypes. Viewtypes are a grouping of views, the most common of which are compile-time, run-time, and deployment, and they cannot be easily reconciled with each other.

Pattern abstractions

All kinds of engineers give names to recurring patterns, like truss bridges or hybrid cars, and software engineers have done the same. In the early 1990s, Mary Shaw created a catalog of architectural patterns that had been in use for decades, such as client-server, pipe-and-filter, and batch-sequential. As she and David Garlan formalized these patterns (also called styles), two ideas emerged.

First, architectural patterns are inherently linked to quality attribute tradeoffs. For example, if the system needs to be low latency, then client-server is more suitable than map-reduce, and if your system needs high throughput, then the choice is reversed. The linkage of architecture patterns to promoted/inhibited qualities lifts a fog covering architectural design. Qualities like latency or availability emerge from all the design choices in a system. It’s a relief to be able to influence them directly instead of just “rolling up your sleeves” and hoping that daily vigilance pays off.

Typical systems have an inconsistent mish-mash of patterns. Consider a house that’s been adapted over the years using the patterns and materials of the day. You’d like to reason about the house, for example “it has insulation, so I’ll be warm in it.” But can you? Perhaps one room has insulation, but another room does not. When patterns are applied inconsistently, you don’t get much reasoning power.

This leads to the second idea. Consistency leads to stronger reasoning power: The more consistently a system applies a pattern, the easier it is to reason about. People tend to use the term architectural style when a pattern is applied consistently as opposed to piecemeal. When a system conforms to an architectural style, it plays by the style’s rules. Consider the POSIX standard in which programs communicate with signals such as SIGTERM and SIGKILL. A well-behaved program listens and acts appropriately; a poorly-behaved program learns that kill -9 is the boss. Returning to the house analogy, if you consistently follow style rules about insulating a house, then you can reason about its warmth.

In 2010, a final architecture abstraction took me by surprise. I knew about the three main viewtypes (compile-time, run-time, and deployment), and I knew about architectural patterns. However, all of the patterns I knew about were run-time patterns, so I was shocked when I read about patterns for how source code is arranged, and patterns about how components are deployed to datacenters [2]. An example of a pattern in the deployment viewtype is redundant deployment to datacenters: by deploying the same components to many datacenters, the system can keep running if a datacenter goes offline. I’d like to say this was an easy extension of what I already knew but in reality I had to struggle before I internalized it.

Chain of intentionality

A few years ago, my software development team was at an offsite team building event where we learned how to cook. Because I like to cook, I already had some of the skills being taught, and I ended up coordinating several of the dishes that our team was preparing. We had some vegetarians so we cooked two pans of Brussels sprouts, one with and one without bacon. Just before serving, however, someone picked up the pans and combined them. Luckily, there were other vegetarian dishes that evening.

The well-meaning person saw there were two pans, of course, but incorrectly inferred that the intent was to feed more people, not to accommodate different diets. This mistake happened because design intent was lost, and I’m to blame for that. (It’s also an example of why it’s dangerous to hoard the architectural knowledge of a system). There was a design to this meal, so to speak, and satisfying our audience led to arranging the cooking in a certain way.

Using two pans is a small detail, but it had a big impact. Was it architectural? As I said earlier, Ralph Johnson was right, “[t]here is no highest level concept of a system” because each stakeholder sees the system differently. Architecture isn’t just the “high level” anything, even if that phrasing is a convenient shorthand. To our vegetarian stakeholders, however, the two-pan design detail was indeed architectural.

Does that mean all details are architectural? No, our systems are big and complex, so we must aggressively simplify if we hope to reason about them. Software architecture is a set of abstractions that lets you reason about your system, especially about quality attributes. Our field has had small abstractions like subroutines, algorithms, and data structures for a long time now. It has taken decades to accumulate larger abstractions like information hiding, components and connectors, multiple views, and architectural styles. When we design systems, we weave these abstractions together, preserving a chain of intentionality, so that the systems we design do what we want [5].

Twenty years ago, Martin Fowler asked “Who needs an architect?” and, with help from Ralph Johnson, helped shape the way a generation of software developers thought about architecture. It’s not just twenty years later; our systems are twenty years bigger. Today, what’s more important than asking about job roles is: Can you reason about the software you plan to build, or have already built? It’s time for developers to take another look at software architecture and see it as a set of abstractions that helps them reason about software.

References

1. Martin Fowler, Who Needs an Architect? alt link, IEEE Software, vol 20 issue 5, pp. 11-13, Sept-Oct 2003
2. Paul Clements et al., Documenting Software Architectures, 2nd Edition, SEI Series in Software Engineering Addison-Wesley, Upper Saddle River, NJ, 2010.
3. Frank DeRemer and Hans Kron, Programming-in-the Large Versus Programming-in-the-Small, Proceedings of the International Conference on Reliable Software, pp. 114–121, April 1975.
4. David Garlan and Mary Shaw, An Introduction to Software Architecture, In V. Ambriola and G. Tortora (ed.), Advances in Software Engineering and Knowledge Engineering, Series on Software Engineering and Knowledge Engineering, Vol 2, World Scientific Publishing Company, Singapore, pp. 1-39, 1993.
5. George Fairbanks, Just Enough Software Architecture, Marshall & Brainerd, 2010.

ABSTRACT: In recent years, teams have found it easy to quickly deliver a working system, but increasingly hard to deliver new features because of tech debt. Tech debt arises from what teams do – and more importantly, what they don’t do – each day.

Teams can use virtuous cycles to produce great code and keep improving it. Virtuous cycles can be found at the scale of methods, modules, and systems. With minimal effort, developers can slow the buildup of tech debt by shifting their perspective on development activities from a checklist to virtuous cycles.

In recent years, teams have found it easy to quickly deliver a working system, but increasingly hard to deliver new features because of tech debt. Tech debt arises from what teams do each day – and more importantly, what they don’t do [1]. If current team practices create tech debt, then different practices can avoid or repair it. My teams have done exactly that by using virtuous cycles.

You can avoid and repair tech debt by shifting your perspective on writing code from a checklist of obligations to a virtuous cycle. Virtuous cycles exist at different scales: when building a method, a module, and a system. They are the core of every continuous improvement process. Whenever you reflect on what you’ve done and decide to change your ways, you are in a virtuous cycle. I also see it in the writing about agile software development, especially when the author talks about the spirit of agility.

Virtuous cycles are a feedback loop that improves code over time. As a result, a lot of tech debt can be avoided with frequent minimal effort, in the vein of gardening, not rewrites. Here, I offer a sketch of some virtuous cycles. It is not a full development process, so I expect that it will work on many software projects exactly because it is incomplete.

The problem

Today, many teams endlessly repeat a checklist like this: pick up the highest priority feature, write a test for it, write the implementation, and maybe refactor – all while conforming to a coding style guide. Systems grow organically, feature by feature. You might think that what distinguishes this from older processes is the addition of testing, which became mainstream in the early 2000s, but it is instead the radical idea that such a short checklist might possibly work [1].

A few decades of experience shows that this short checklist works, but there’s a catch. It works in the same sense that doctors can perform operations without washing their hands. Such success is short-lived, as patients suffer from infection and software projects suffer from technical debt. So, the short checklist works, but the price of radical process simplicity is, ironically, the buildup of project-crushing complexity, which I call sedimentary development [2].

Organic growth rapidly creates complexity. Since there was no master plan, when you read the code you will not find consistency. There will be no cross-cutting principles or decisions to aid your reasoning. To understand it, you must memorize all its quirks.

As Tony Hoare said nearly half a century ago in his Turing award lecture:

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.

Organic feature growth quickly exceeds the intellectual control of its authors, so they settle for statistical control through testing. A simple checklist is a kind of statistical control, where the tests ensure that specific cases work. What is lacking is intellectual control – an understanding that developers hold in their minds that convinces them that the system works in the general case [3]. Intellectual control is possible only when the design is simple.

If refactoring alone were strong enough then teams would not still be suffering from tech debt. Refactoring can slow and even repair tech debt, but not when refactoring is a step in a checklist. People use checklists grudgingly. They admit that they make mistakes and that checklists can catch those mistakes, but emotionally they want to zip through the checklist as quickly as possible and return to fun activities like building more features.

Vicious and virtuous cycles

A vicious cycle is a chain of events that reinforces a bad outcome. Here’s how a checklist becomes a vicious cycle: You pick up a feature request, write some code, and write a test. Sometimes you refactor. You feel pressure to be productive, so each day you try to zip through this checklist a bit faster. Your manager cannot measure tech debt or code complexity, but can see all that code you are cranking out. As weeks pass, it becomes harder to add the next feature because it’s hard to understand the code it sits on. But you want to keep improving productivity, so you refactor less often. You test fewer cases. You use the first thing you think of. The more you seek productivity, the more you compromise quality, which leads you further away from productivity.

Increasing productivity means getting more done in the same amount of time, so you cannot improve productivity by slowing down. But speeding through a checklist ends up reducing productivity and slowing down also reduces productivity. Is this not a paradox?

You would be right in thinking that the answer is a virtuous cycle, which is a chain of events that reinforces a good outcome. As many have observed, writing a test or refactoring code creates an opportunity for reflection. The tests and the refactoring are themselves good, but the real driver of a virtuous cycle is the reflection and the opportunity to course-correct that follows. The virtuous cycle starts when you ask “can I do better?” and then improve.

Reflection leads you to fix your first attempt. It leads you to recognize trouble piled on top of trouble. At those moments, the checklist transforms into a virtuous cycle. You return to writing code with new insight, you revisit your bureaucratic tests with a way to lighten them, and you refactor beyond superficial code de-duplication. When you follow that virtuous cycle, productivity improves as you avoid and repair tech debt a little bit each day.

You may be skeptical. Consider any team that has good testing practices. Ask them if they would be more productive without their tests. They will tell you: of course not. But doesn’t writing tests take time away from coding? Yes, but because the team is in a virtuous cycle, the time investment pays off immediately. If you are still skeptical, please delete the tests on your current project and let me know if your productivity improves.

Here’s an overly tidy way of summarizing this idea. When you try to improve productivity by working harder, you encourage a vicious cycle; when you work smarter, you encourage a virtuous cycle. The virtuous cycle guides you to keep the code as simple as possible, which preserves your intellectual control, so feature requests are easy to reason about and incorporate.

In my experience, the virtuous cycles I’ve used aren’t strong enough to keep a system healthy forever. If you imagine two teams, one using a checklist and the other using virtuous cycles, you’ll see the virtuous cycle team consistently being more productive and its system has a few more years of useful life. Let’s see how that works at three scales, starting with methods.

Method virtuous cycle

Let’s work through an example of building a method using a virtuous cycle (See Figure 1). Revealing the virtuous cycle would be easier if we could work on a project together for a few hours, staring at the same code. Let’s acknowledge the awkwardness of text and do our best. Assume we are working on a first draft implementation of a method, where c is the ID of a customer and a is the ID of an account:

bool isOwner(int c, int a)

We have already written another method:

void setOwner(int c, int a)

The first step in the virtuous cycle is to write a test for the success case. The test calls setOwner(), then asserts that a call to our new method – isOwner() – succeeds. So far, there’s no opportunity for insight. Nothing we’ve done encourages us to revisit the body of the method. Some developers would stop now.

The second step is to write a contract for the method that explains what a caller cannot know from the method signature, such as: “Returns true iff customer owns the account.” (“iff” is shorthand for “if and only if”). We’d write this in a comment to document the method. Most callers would have guessed this is the contract. Again, there’s not much opportunity for insight so let’s move to the next step, which is interesting.

The third step is to consider error handling. Our test case works because we call setOwner() before isOwner(). What happens if we omit that? Let’s write a failure test and assert that isOwner() returns false, as the contract says it should.

We are surprised when this test fails: Instead of returning false, the isOwner() method crashes with an error about an index out of bounds. For brevity, the body of isOwner() isn’t shown here, but let’s say that we notice the customerId parameter is used as an index into a table of customers and accounts. Aha! In the success case, setOwner() configures the table, but our new test omits the call to setOwner(), which triggers the crash.

Instead of rushing to refactor the body and re-run the test, let’s reflect on what we’ve designed. The isOwner() method can tell callers about ownership (the boolean return value) but not about errors caused by unexpected parameters or conditions. We’ve designed a loaded question where the isOwner() method is forced to assume its parameters and conditions are valid, but in practice they might not be.

Let’s revise the contract: “Returns true iff customer owns the account and parameters are valid IDs.” This helps because it warns callers that IDs can be invalid. It doesn’t say what happens though. The method is now underspecified: it’s a partial function. Methods like this are a burden on callers, who must ensure that the IDs are valid before calling isOwner(). Remember this burden, as we will revisit it.

Writing a contract takes seconds, so that has no effect on your productivity. Pondering and deciding a contract does take time, but you must spend that time when writing the test anyway. Before you could write a test case, you had to decide the expected behavior and avoid testing the method’s implementation details. That’s the contract. When you spend time deciding a contract, you feed a virtuous cycle, leading to tests that cover all cases, methods that are simpler, and callers who find the method easy to use.

The fourth step is to consider typeful programming, which is the pervasive use of types that can be checked by the compiler. So far, the isOwner() method takes two integer parameters. Callers could accidentally transpose them yet the compiler could not catch that mistake. You can help the compiler by creating two new types, CustomerId and AccountId. The signature becomes:

bool isOwner(CustomerId c, AccountId a)

Not only does this make transposing the parameters impossible, it also feeds our virtuous cycle. Recall that the contract burdened callers with ensuring that the IDs are valid. Now that the IDs are their own types, you can enforce ID validity when the types are created, which removes the caller’s burden. That means we can simplify the contract, so it is again simply “Returns true iff customer owns the account.”

You could add more steps to this virtuous cycle, such as looking for chances to make data immutable and functions total. The goal of this detailed example was to reveal the difference between a checklist and a virtuous cycle. Next, let’s look at virtuous cycles for modules.

Module virtuous cycle

Modules consist of hundreds or thousands of lines of code, so they are too big to show here, let alone show them evolving as you work through a virtuous cycle (See Figure 2). To make that cycle apparent, I’ll resort to metaphors and general descriptions.

What is a module? A module groups together a bunch of methods and creates a distinction between inside and outside. (Here comes the metaphor). Consider your kitchen. Your refrigerator is a kind of module that you can use to keep food cold. It has an inside and an outside. Inside, there are a bunch of parts. Outside, it interacts with the world in limited ways: the electrical plug and its doors. You can understand how to use your refrigerator without understanding all the parts inside it.

Some modules are better than others. Your kitchen likely has a drawer where you keep miscellaneous tools. Both have an inside and an outside. Both contain parts on the inside. Your refrigerator, however, is a better module. The drawer cannot be understood except by understanding each thing it contains. When you build source code modules, you prefer ones that can be understood simply, like a refrigerator, not an arbitrary bag of parts that must be mastered individually, like the drawer.

In source code, organic growth leads to complexity, bit by bit, like tools accumulating in your miscellaneous drawer. Virtuous cycles tame complexity, guiding you to group methods to hide inner complexity while presenting a simple interface.

The virtuous cycle for modules has familiar activities including writing methods, tests, and contracts. At the scale of a module, however, you use them differently than when writing a single method. When writing tests for a module, you have an opportunity to evaluate the module’s interface. You can scrutinize what you’ve kept inside versus allowed out across its interface. With each test, you are looking for secrets leaking out of the module.

One example I’ve seen many times is that implementation details leak out via exceptions. When you are testing error cases and the test must catch a specific exception (such as PostgresException or OracleException), that means users of your module now depend on your technology choice. This is a failure of what David Parnas called information hiding. Better information hiding means changes to one module don’t ripple and force changes to its neighbors.

The virtuous cycle for modules also includes contracts, but, compared to methods, the stakes are higher. The contracts on module interfaces are often used by people not on your team, perhaps not even at your company. They might not be able to read your source code to figure things out, so the contract is their only insight into how the module is intended to work. Writing the contracts helps you build a better module interface. Modules with clear contracts can, if necessary, be rewritten. Rewriting a module with fuzzy contracts means chasing quirk-for-quirk compatibility, which is so hard that you rarely ever attempt it.

The virtuous cycle for modules also includes new activities, like evaluating dependencies. Some of your modules will be leaf modules that depend on no other modules. Fewer dependencies leads to easier testing and reuse, so you should be looking for opportunities to prune dependencies.

To build good modules and fight creeping complexity, you must keep tinkering over the life of your system. It’s easy to create bad modules, ones like that drawer in your kitchen instead of a refrigerator. Activities like testing, writing contracts, create opportunities to evaluate the module as a division between inside and outside, a division that keeps some secrets inside while presenting a useful interface to users.

System virtuous cycle

At the scale of an entire system, the virtuous cycle shifts your focus to quality attributes like latency, security, modifiability, or reliability. All systems have architectures, whether chosen consciously or not, that promote or even ensure certain qualities. The virtuous cycle guides you to an architecture that is best suited to the qualities you prioritize.

As before, this is just a sketch of the virtuous cycle (See Figure 3). And again you start the cycle with testing. Writing an end-to-end test that shows the whole system working for a simple task forces you to use all of the modules in the system. If you pause at this moment, you will notice that setting up this first system test is painful, difficult, and fragile. Take the opportunity to remove some friction, to make the next test less painful to write, to make the test setup less likely to break.

Next, look for inconsistency across the modules in your system. That could be in the vocabulary of types used in module APIs: perhaps you can remove unnecessarily different types used in those APIs, which makes testing easier and removes corner cases. Look at how errors are signaled and handled across the modules because unnecessary diversity leads to mistakes. Look for places where developers must be vigilant in their coding practices, as architecture hoisting can reduce that burden.

Then evaluate how you have partitioned your system into modules. Perhaps there is a better way to divide up the code. Perhaps a single responsibility has been smeared across many modules, leading to unwanted coupling. As you change code, does your edit touch a single module, or ripple out into its neighbors? Ripples are a sign that the partitioning into modules needs attention. Also look for standard problems, such as parsing, validation, and loading/storing. Align your modules with these problems instead of entangling them.

As your system grows, it’s increasingly important that it has a hierarchy of modules and a pattern of organizing them. You may start out with a neat tree but don’t expect that to last. There’s no single right way to organize modules, but neglect quickly leads to tangled dependencies.

The final step in the virtuous cycle is to re-evaluate your desired qualities, accepted trade-offs, and chosen architectural styles. If you haven’t prioritized which qualities you need most then you are giving up the opportunity to choose a style that matches your needs. It’s expensive to change architectural styles, so, in the early days of your system, this step is critical because if you need to change, you’d prefer to do that before the system is huge.

A systematic approach

The Boy Scout rule says to leave the campsite cleaner than you found it. How exactly do you do that? In software development, you can use virtuous cycles at three different scales: a method, a module, and a system. The steps here are ones that have worked for me in a career spent in application development. If you build device drivers or self-driving cars, you might have different steps, but you can find your own virtuous cycles.

Some teams perform better than others even when doing similar activities under similar conditions. I think that the higher-performing teams are linking those activities into virtuous cycles, not speeding through tasks on a checklist. Each step in a virtuous cycle gives you a different perspective on your work and an opportunity to notice a better solution. That opportunity is critical. Use it to recognize and repair tech debt, often before your code reaches production.

Some people think that seeking productivity this way is too expensive. Can teams instead seek productivity by being scrappy and simple? History says no. The most productive people aren’t amateurs with a checklist, they are reflective experts who are continually improving. It takes practice and reflection to make something look effortless.

References

1. G. Fairbanks, The Rituals of Iterations and Tests, IEEE Software, Vol 37 number 6. November-December 2020.
2. G. Fairbanks, Ur-Technical Debt, IEEE Software, Vol 37 number 4. July/August 2020.
3. G. Fairbanks, Intellectual Control, IEEE Software, Vol 36 number 1, January/February 2019.

ABSTRACT: There are two kinds of iteration, but they are commonly conflated. The first is the evolution of a design and its implementation to become more suitable over time: design-focused iteration (DFI). The second is the evolution of an artifact (like code) to become more suitable over time: code-focused iteration (CFI).

CFI improves only the code and ignores the refinement relationship between design and code. In contrast, a goal of DFI is nurturing and improving the refinement relationship so that the design becomes more stable and valuable over time.

Refactoring happens in both kinds of iteration. In CFI, the refactoring is shallow and textual. In DFI, the refactoring is conceptual and yields what Domain Driven Design calls “deep models” and “supple designs”.

In the allegory of the cave, Plato argued that invisible concepts, such as geometry, could be more true than any figures that we imperfectly scratch in the sand. The triangles and squares that we can observe with our eyes are just shadows cast on the wall of a cave by the pure ideas that we cannot observe directly. This presents us with a choice: should we fixate on the shadows we can see, or use them to discover hidden truths?

Today, two and a half millennia after Plato wrote his allegory, software developers make that same choice. Some developers see source code as the truth. Others see source code as the shadow on the wall that provides clues about the truth, which is the problem and solution that cannot be observed directly. I doubt Plato would be surprised that we are still debating.

Many developers consider themselves as pragmatic and therefore decide that seeking invisible truth is something left for philosophers and academics. I disagree. The Wright brothers were deeply pragmatic, yet in their quest to be the first to fly they both built airplanes and developed theories about aviation. They built an airplane before others precisely because they pursued both.

Importantly, the Wright brothers used an iterative approach. Iterations forced them to build something instead of spending all their time on philosophy. Iteration is what makes it possible, and indeed pragmatic, for engineers to both get things working and seek the invisible truths that explain how to make things work better.

Here’s the rub: Iteration means different things to different people. “Code is the truth”developers iterate on the code, adding features over time. I call this code-focused iteration (CFI). “Shadow on the wall”developers iterate on their understanding and on the code, making both better over time. I call this design-focused iteration (DFI).

Because the word “iteration”is ambiguous, developers can declare “we are iterating”and yet be doing quite different things. Small changes in day-to-day activities lead to different outcomes after just a few months. Developers doing code-focused iteration erode their designs, impair their readiness for the next requirement, and reduce their productivity. In contrast, developers doing design-focused iteration strengthen their design with each iteration, solve their problems better, and enjoy their work.

Kinds of iteration

What do CFI and DFI look like in practice? Let’s start with some familiar non-software examples of iteration. When you get a new pair of eyeglasses, your optometrist uses iteration to adjust them to fit your head. The two of you alternate between wearing and adjusting the glasses until both of you are satisfied with how well they fit. The optometrist is using a hill-climbing algorithm: examining the situation and making a change for the better. In this kind of iteration, no one is seeking invisible truths. You and the optometrist attend solely to what is visible, using a technique to improve a machine (your eyeglasses) for the better. This is code-focused iteration, but with eyeglasses instead of code.

Car engines are another example. When a new generation of engine comes out, it typically has unforeseen problems. The automotive engineers identify and fix these problems iteratively and, over several years, as design flaws are fixed, that generation of engine becomes more reliable. This is code-focused iteration, but with engines instead of code.

Those same automotive engineers, however, are also doing something else. Across generations of engines, they are building up their understanding of everything involved with building engines: the materials, the combustion, the wear on parts, the machines that create the engines, the environment the engines will be placed into, etc. They are using their experience with the tangible to learn about the invisible. By building up their understanding of the invisible truths, their next generation of engines will be better than the previous. This is design-focused iteration applied to engines.

Let’s return to software development. Imagine a system used to schedule university classes that already handles semesters, and let’s say the developers receive a new requirement: trimesters. The developers make minor changes to the code to support the requirement. (You can imagine many similar changes that would not force any significant reflection on the nature of university classes or on the design of the software.) Developers can make those changes by attending solely to the code itself, applying a hill-climbing algorithm. This is code-focused iteration.

Consider a different requirement for this university software: that teachers can attend classes. Let’s say the code has one data structure for teachers and another for students. If professor John Doe wants to take a class, the system would be tracking him twice, with his information duplicated in the two data structures. So, this requirement forces developers to reflect on what they understand about university classes. They iterate on their invisible understanding of how things work and revise their ideas. Perhaps they land on the idea of introducing two new concepts: people and roles. Where they previously thought of teachers and students, they now think of people who play the roles of teachers and students. They revise the code to match this new understanding. This is design-focused iteration.

Code refines a design

It’s tempting to ignore distinctions between CFI and DFI, instead thinking only of developers making a series of edits to the code to improve it. After all, developers may interleave thinking and coding, and in fact this can accelerate their design-focused iterations. But failure to distinguish CFI from DFI can doom a project. When Ward Cunningham coined the term technical debt, he described how iteration, done poorly, could bring “[e]ntire engineering organizations …to a stand-still” [1]

So, what is CFI missing? In a word, refinement. Design-focused iteration improves both the design and the code so that, over time, the design becomes an increasingly good fit to the problem at hand. In contrast, code-focused iteration, by accumulating features, improves only the code.

Refinement is the relationship between design and code. Your design guides your code and limits some of your implementation choices. Anything present in the design must also be in the code, but not vice-versa. Consider the university class scheduling system in the example above. You have a lot of implementation choices. You could implement it in any programming language, using any variety of algorithms, and on any hardware platform. However, there are limits. The ideas from the design -- people and roles, semesters and trimesters -- may not be contradicted in the code.

As a developer, why should you voluntarily constrain yourself? How can shackles help you solve problems? A good design makes it easier to write good code. In the example above, the design change from teacher-students to people-roles isn’t a shackle, it’s a gift. Clear thoughts in the design can avoid any number of corner cases in the code. A good design allows you to make broad conclusions without reading through every line of code. For example, a map-reduce design insists that each map job be idempotent, so you can conclude that it’s safe to reschedule jobs that are running slowly. The idea of idempotence is one of those invisible truths in a design that you cannot see directly in code.

Geometry is more true, and in ways more real, than any imperfect diagrams we might draw. In the same vein, a design can feel more true than source code. Consider the vending machine problem that’s often used in introductory programming courses along with a finite state machine design. Is that design not more true and real than any student’s code that implements it? And if you had a new requirement, perhaps to handle a new coin, wouldn’t you revise the state machine and then edit your code to match it?

Iteration with a goal

Years ago, when waterfall processes were common, refinement was a fact of life. Developers were forced to confront the refinement relationship between design and code because design happened early in the project and code not until later. All developers were aware of how their design related to their code.

When I mention waterfall processes, some people misinterpret this as me advocating for up-front design. The goal is to have a refinement relationship between design and code, but that goal can be accomplished through up-front design or iterative design. As Desmond D’Souza and Alan Cameron Wills said: “Refinement is a relationship, not a sequence.”[2] Plenty of articles have demonized up-front design, but the bigger problem is neglecting the goal of refinement.

Consider the two iterative processes shown in Figure 1. One will help you improve the code, while the other improves both the code and the design [3]. Teams using design-focused iteration are bringing the design with them on their journey. It is a constant companion. CFI and DFI are both iterative, but only DFI has the goal of nurturing the refinement between design and code.

Code-focused iteration is vulnerable to problems that grow worse over time [4]. The first problem is the sedimentary buildup of old ideas. In the university example above, you probably could have edited the code so that your teacher and student data structures survived. Obsolete ideas can accumulate in code like sediment, making it hard for other developers to understand the design and reason about it.

The second problem is loss of intellectual control. If you iterate only on the code, whatever design you have will deteriorate and provide less value. You lose your ability to reason through the system using abstraction and instead must trace the code line by line. When obsolete ideas accumulate and intellectual control is lost, projects become technical zombies without vitality.

Refactoring the design, not just the code

For decades, refactoring has been held up as the way to repair iteration’s flaws. That’s only partly right. Most projects that use refactoring merely to textually rearrange code. How do developers make that mistake? If you look at books and websites on refactoring techniques, you’ll see them describe mechanical activities, but those are shadows on the wall. Such refactoring is helpful, but it’s akin to fixing the grammar and spelling in an essay with half-baked or obsolete ideas.

The truly valuable part of refactoring is invisible. The best description of how to use refactoring to evolve your design is in the Domain Driven Design book section on “Refactoring Toward Deeper Insight”[5]. It suggests that the goal is to develop “deep models”and “supple designs” which happens during breakthroughs:

[C]ontinuous refactoring prepares the way for something less orderly. Each refinement of code and model gives developers a clearer view. This clarity creates the potential for a breakthrough of insights. A rush of change leads to a model that corresponds on a deeper level to the realities and priorities of the users. Versatility and explanatory power suddenly increase even as complexity evaporates.

That is exactly what you hope to achieve by iterating. However, 17 years after that was written, most developers are still refactoring superficially, doing code-focused iteration. If I had to guess why, I would say it’s because most developers haven’t heard of the idea, or think that the entire DDD package of ideas is a poor fit for their project and so neglect this critical technique.

Iterate toward a clean design

Plato wrote the Allegory of the Cave to teach us that invisible ideas can be more important than the visible shadows on the wall. We read his words thousands of years later not because they are easy but because they are uncomfortable. It’s far easier and comfortable to attend to what we can see directly than to heed someone ranting about hidden truths. In fact, the second half of the allegory discusses how people who have only ever seen shadows would react when told about the invisible figures casting those shadows. Plato’s verdict was grim: they would kill the messenger.

When I look around our industry at what teams are doing, I see many good practices such as iteration, refactoring, testing, and automated deployments. Despite those similarities, some teams are succeeding and others are suffering. What distinguishes them is how well they nurture their design (see Figure 2). As teams abandon up-front design, I fear that many of them are doing code-focused iteration, accumulating technical debt through sedimentary layers of obsolete ideas, and building technical zombies.

For a long time we believed that iteration and refactoring were sufficient to keep a design healthy, but we can no longer believe that after seeing so many tangled designs and zombie projects. Software development is an intensely cognitive activity that cannot be reduced to simple activities repeated mechanically. Good design, while invisible, is critical and must be a goal of refactoring. By recognizing the distinction between code-focused and design-focused iteration, developers can adjust their activities slightly to keep their design healthy.

References

1. W. Cunningham, The WyCash Portfolio Management System, in Proc. OOPSLA 92, Vancouver, Canada, Oct. 5–10, 1992.
2. D. D’Souza and A. C. Wills, Objects, Components, and Frameworks with UML: The Catalysis Approach. Boston: Addison-Wesley, 1998.
3. M. Keeling, T. Halloran, G. Fairbanks, Garbage Collect Your Technical Debt, IEEE Software, vol 38, no. 5. Sept.-Oct. 2021.
4. G. Fairbanks, The Rituals of Iterations and Tests, IEEE Software, vol. 37, no. 6, pp. 105–108, Nov.–Dec.2020.
5. E. Evans, Domain-Driven Design, Addison-Wesley, 2004.

ABSTRACT: The iterative process that a team follows is a bit like a garbage collection algorithm, and we can compare software development processes like we can any algorithm. A process can help developers do two things: clean up tech debt after it exists, or avoid creating it. When an iterative process does neither, tech debt buildup will lead to bankruptcy, so it is only suitable for projects with a short lifespan. A process that does both has the best chance at minimizing tech debt over a long lifespan. In particular, focusing on the system’s design will keep tech debt low.

There is a kind of design distortion that happens when a team chooses to build iteratively instead of looking at all the requirements at once. Ward Cunningham coined the term technical debt to describe those design distortions. (Today, people use the term tech debt to describe a wide range of problems, including bad code written by novices, and known-bad designs used in desperation to meet a deadline. Throughout this essay, we are talking about the original, narrower kind of tech debt, also called ur-technical debt [1].) By understanding the causes of tech debt and connecting them back to a team’s actions (or inactions), it’s possible to minimize the buildup of tech debt and keep a system healthy, indefinitely. The way to minimize tech debt is to view a software development process as an algorithm, consider several algorithms, and choose the right one for the circumstances.

However, most developers don’t think about their process as an algorithm, so let’s ease into the idea by looking at garbage collection algorithms. Watching tech debt build up on a project is a bit like watching a program allocate memory.

Running a program creates garbage, which is memory that’s been allocated but is unused. Garbage creation is unavoidable, so we must occasionally pause to collect garbage. It’s nicer when those pauses are predictable and short. There are various garbage collection algorithms that have different properties.

And here’s a description of tech debt, using the same phrasing:

Running a timeboxed iteration creates tech debt, which is working code with an obsolete design. Creating tech debt is unavoidable, so we must occasionally pause to refactor the code. It’s nicer when those pauses are predictable and short. There are various iterative software development processes that have different properties.

Consider this: a team’s software development process is an algorithm, run by the team itself, that generates and cleans up a kind of garbage that we call tech debt. We know how to analyze algorithms, so let’s analyze a team’s process just like any other algorithm.

Software development processes control tech debt using two techniques. The first technique is cleaning up existing tech debt. Most teams already do this by refactoring. The second is avoiding the creation of tech debt. This is less common but more interesting. Let’s look at each in turn.

Technique: Tech Debt Cleanup

You can control tech debt by cleaning it up after it exists. Often, a team “bolts on” a feature without regard to the existing design, identifies tech debt, and only then refactors to clean it up. Sometimes the cleanup happens immediately, but it could be much later.

Small problems can be refactored in minutes, but bigger problems can take days, weeks, or months to clean up. When developers take a break from writing features to fix tech debt, that’s like a garbage collector pausing to clean up garbage. Spending time on refactoring means less time for new features. The bigger the problem, the longer it takes to refactor.

Because it requires stealing time from feature building, teams can find themselves under pressure to do less refactoring, especially large refactorings. As a result, they clean up the small problems but delay cleaning up big problems, such as the system’s architecture.2 Postponing a small cleanup can transform it into a big cleanup because, over time, code builds up around the problem, and it too must be refactored.

Technique: Tech Debt Avoidance

You can control tech debt by creating less of it—that is, by avoiding it. Teams do that by considering design alternatives and choosing the one that creates the least tech debt. When asked to add a new feature, a team considers how well the current design can accommodate that feature. If the design is already suitable, they add the feature. But if the design is unsuitable, they update the design first, then add the feature.

Kent Beck summarized it this way: “[F]or each desired change, make the change easy (warning: this may be hard), then make the easy change.” (K. Beck, twitter, Sept 25, 2012.) The wordplay in Beck’s quote is delightful, but the idea here is not a linguistic trick. Figure 1 shows two possible software development processes to control tech debt. The first allows tech debt to happen, then cleans it up. The second looks for upcoming trouble and avoids it by redesigning before implementing the feature.

To many developers, avoiding problems sounds better than cleaning them up. Be aware, however, that some tech debt is unavoidable. Sooner or later, a new requirement will be an unpleasant surprise. You might wonder if peeking ahead at future requirements would work, but that’s not foolproof because the fog of design obscures our view of the future [3].

Some teams worry that tech debt avoidance is a waterfall process in disguise, or worse, Big Design Up-Front. That’s clearly not the case, as a waterfall process would have the team look at all the requirements and deliver one system to handle them all. Tech debt avoidance means that the team works on the requirements iteratively, delivering a working system with each iteration.

Some teams worry that tech debt avoidance will lead to analysis paralysis. Today, we see lots of teams struggling to control their tech debt, but we don’t know of any teams using an iterative process that are stuck in analysis paralysis. Perhaps that’s because there are strong forces pushing the team to deliver features with each iteration.

Choosing a Tech Debt Algorithm

We’ve explored two techniques to keep tech debt low: cleanup and avoidance. Expanding the combinations of those two techniques yields four kinds of tech debt algorithms to choose from: None, Reactive (cleanup only), Proactive (avoidance only), and Balanced (both cleanup and avoidance). These algorithms are summarized in Figure 2.

We’ve seen teams succeed with all of these algorithms. We’ve also seen teams choose an unsuitable algorithm and suffer, then conclude that tech debt is an untameable monster. Choosing the right algorithm for your team depends on circumstances including the team and project size, domain knowledge, design experience, technology experience, and schedule pressure.

None. Some teams don’t do anything to control tech debt, and the parallel with garbage collection holds up: There are no-op garbage collectors. If you write a quick script for yourself, and you don’t plan to reuse it, why worry about tech debt? The same thinking applies to bigger projects, such as commercial computer games where the team knows they will start a fresh codebase for the next game. The developers only suffer with tech debt until the game is released.

Reactive. The Reactive algorithm, using only tech debt cleanup, is what most teams do today. Teams can focus primarily on the stream of features to build, pausing occasionally to clean up tech debt “garbage”. Bigger cleanup efforts are hard, so early mistakes linger because they are too expensive to refactor later. It’s easier to recognize problems than it is to avoid them, so Reactive makes sense when the developers have limited design skills.

Proactive. The Proactive algorithm, using only tech debt avoidance, is uncommon today. If you can avoid tech debt with a bit of thinking, that’s more efficient than blundering into obvious problems. On the other hand, if you don’t have experience with the technology being used, you may waste time based on bad assumptions. Despite efforts to avoid tech debt, it will happen, so teams that start with the Proactive algorithm may switch to the Balanced algorithm to clean it up.

Balanced. Most teams wish their tech debt were lower, so most teams should use the Balanced algorithm because it includes both cleanup and avoidance. Depending on the circumstances, they can do more or less of each technique.

Here’s an example of a Balanced algorithm that we find pragmatic. At the start of each iteration, the team discusses how the feature requests will affect the current design. That keeps the iteration design-focused, and the design fresh in everyone’s minds.

They may peek ahead at future feature requests, even though they aren’t working on them now, because knowing what’s coming may help them answer today’s design questions.

Sometimes a feature is hard to add to the design. It could contradict an assumption about the domain, or it could be hard to build within the current architecture. If the team can rework the design and add the feature within the current iteration, that’s great. When they cannot, they chat with the product owner. They weigh political, economic, and social forces as well as schedule pressure and engineering risk before deciding. The answer might be to bolt the feature on and clean up the tech debt later, postpone the feature entirely, or something in between.

Finite and Infinite Games

Perhaps the most important factor in deciding which tech debt algorithm suits your team is whether your team is playing a finite or infinite game. Finite games can be lost or won. Infinite games can be lost, but winning just means you can keep playing. Tech debt feels a bit like an infinite game: If you can keep it under control, you can keep playing. Otherwise, you lose and declare tech debt bankruptcy.

Teams with a strict schedule are playing a finite game. One of the authors (Halloran) developed a military wargame simulation, StratWar, that had to be completed so students could use it in the next semester. He met the deadline, but built up vast amounts of tech debt [4].

Startup companies are playing a series of finite games. They operate in do-or-die mode to reach the next milestone, and failure means the company dies. Halloran also worked at a static code analysis startup company that scrambled to build a product to show at the JavaOne conference. As you would expect, the demo built up a lot of tech debt, but showing up at the trade show with working software let the company live another day and kept hope alive to switch to playing an infinite game.

Inexperience can force you to play a finite game. If developers don’t know the problem domain or the implementation technologies, they are in a finite game until they can build something that works. Prototyping can build experience faster than up-front design or refactoring, but tech debt will make that code unsuitable for the long term.

Switching from a finite to an infinite game runs the risk of tech debt bankruptcy. Sometimes you discard the code from the finite game, as we did in the StratWar example. Other times you nurse the code back to health, as we did in the analysis startup.

If you declare bankruptcy and decide to rewrite the system, it is critical to re-evaluate your tech debt algorithm. Don’t keep using an algorithm tuned to a finite game and hope it’s suitable for an infinite game. It’s a good time to try the Balanced algorithm, both avoiding tech debt through good design practices and cleaning up the inevitable debt through refactoring.

Minimize Your Tech Debt

Managing tech debt is a bit like managing memory allocation. By choosing your software development process, you can control how tech debt accumulates, just like a garbage collector reclaiming memory. It’s helpful to think of your software development process as an algorithm that controls your system’s tech debt.

Building software iteratively leads inevitably to tech debt because we choose to deliver systems before we have looked at all the requirements. Not knowing what’s next distorts our designs and that distortion is the tech debt. In theory, waterfall could avoid that distortion, but in practice it introduces other design distortions by peering far into a foggy future [5].

Software processes have a dominant decomposition: either a stream of features or the system’s design. Today, most teams focus on a stream of features, and it follows naturally that those teams rely primarily, or even exclusively, on tech debt cleanup [6].

We have spoken with teams that work differently. In addition to refactoring, they also proactively avoid tech debt. They have flipped the dominant decomposition, making the system’s design their primary concern. Their iterations are design-focused, not feature-focused.

Today, teams struggle with tech debt. Some managers believe it’s uncontrollable and expect tech debt bankruptcy after a few years. The idea that our own software development process is contributing to tech debt is liberating because our process is under our control. By looking at tech debt as analogous to garbage creation, you change your perspective. Tech debt might be inevitable, but you can minimize it by choosing a suitable algorithm.

References

1. G. Fairbanks, Ur-Technical Debt, IEEE Software, Vol 37 number 4. July-August 2020.
2. M. Keeling, Headwinds to Redesign. IEEE Software, Vol 38 number 2, March-April 2021.
3. T. Halloran, Fog of Software Design, IEEE Software, Vol 38 number 3,May-June 2021.
4. T. Halloran, Development of the StratWar Wargame Software, Practicum Report, DTIC ADA428794, February 2004.
5. G. Fairbanks, The Rituals of Iterations and Tests, IEEE Software, Vol 37 number 6. November-December 2020.
6. G. Fairbanks, Why Is It Getting Harder to Apply Software Architecture? IEEE Software, July-August 2021, Vol 38, number 4.