Kent Beck, original signer (and first in alphabetical order) of the Manifesto for Agile Software Development, delivered the closing keynote at QCon San Francisco this week. The title of his lecture was Tidy first? – which is also the name of his next book.
He began by presenting his personal mission to help geeks feel safe in the worldwhich he explains in this Facebook post.
He spoke about the recent review of Structured Design by Ed Yourdon and Larry Constantine. He describes it as containing Newton’s laws for software development, the concepts and ideas are as relevant to programming today as they were when the book was first published. The book introduced the concepts of coupling and cohesion which remain as much of a challenge for software engineering today as they were when the book was published in 1975.
He says he has spent the last 17 years learning how to explain cohesion in software design, which led him to write a series of three books that will explore software design. Books are accessible as content is published on Substack.
He says that
Software design is an exercise in human relations
And it is by exploring these relationships that software systems emerge in the world.
The first relationship is between the idea being explored and the behavior that will materialize that idea. The relationship is two-way and evolutionary – the idea defines the behavior and the existence of the behavior can generate more ideas (now that I see this, what about that…). Beneath the idea and the behavior is the structure of the system. It is architecture and it has a profound effect on visible behavior.
The structure of the system has a profound effect on behavior and on the ideas that are generated because behavior can change. Structure influences behavior, resulting in new ideas asking for behavioral changes that can impact structure, in a continuous loop. The Tidy first? the workflow makes us pause and ask if the structure needs to be changed or do we just need to implement the behavior change. If the change is going to impact the structure, clean up the structure first – refactor the underlying architecture if necessary. Do NOT try to make structural and behavioral changes at the same time, because that’s where systems become technical debt.
He says there are two viewpoints involved when systems are built, and particularly when they need to be changed. The two points of view are waiters and changers. The servers have the idea and want the behavior to do the new thing as quickly as possible, the changers have to maintain the code and want to tidy up the structure first so the behavior change can be done safely.
The complexity increases even more when multiple changers deal with different areas of the same product. It is highly likely that server incentives and competing incentives from different changers are in tension.
Software design skills are about maintaining good relationships between disparate manufacturers and between changers and servers.
The question a changer faces when receiving a product change request often results in this code is messy – should I tidy it up before making changes. The answer to this question is often dogmatic sure and Of course not – we want the change to be made as soon as possible AND we want the codebase to be free of technical debt. Unfortunately you can’t have both and a decision has to be made and the correct answer has to be it depends.
He drew this image to illustrate the different relationships in the server-changer ecosystem.
He then explained why large-scale design was a bad idea in the 1990s and is still a bad idea today. He said that the waterfall is back and some organizations are again trying to define success in software development in terms of time, cost and scope defined in advance and how iterative and incremental development has always been the most economically rational approach to building systems software, and even more so today.
He explained how Constantine’s equivalence shows that the effective cost of a software system is the cost of changing the system. Over the lifetime of the system, the cost of maintaining it will make the initial development cost an insignificant part of the overall investment. He then showed how the cost of maintaining a product is directly related to the cost of coupling in the system. When a change to one element of the system results in a cascading change to another element, which results in a change to another element and so on…
The overall cost of the system can be thought of as the cost of this coupling plus the cost of decoupling the elements of the system. (He used elements to indicate that the actual nature of the code is irrelevant in this context – it could be functions, remote procedures, microservices or any other component that has dependencies)
The underlying structure (architecture, design) of the system is the most important factor in the coupling/decoupling equation.
When looking at changes to a system, most will likely be made in areas that don’t have a lot of cascade couplings, but there will likely be a few changes over the life of the product that are jackpot changes – those where the coupling cascade is so important that the cost of carrying out the change is astronomical. These jackpot changes are likely to make up the vast majority of the cost of maintaining the system over its useful life.
He ended by encouraging the audience to make changes in small increments, to keep their systems structure clean and as decoupled as possible, and to take a Tidy First approach to software maintenance.