The Story Of The System, Discovering the “Why” Behind The System
Software development can be a fast-paced world, different fads come and go through the industry, many times what was old becomes new again and there are no less than three dozen vendors vying for you to build your system using their technology. This is just the nature of the industry. One of the many problems with this; however, is that often multiple fads or competing technologies can be found in the same system. Is it just a failed or abandoned migration? Are certain technologies to be used for certain use cases and others not? The way X was built seems wrong, why was it built that way? These very well might have been questions that you have asked yourself before. Having clear answers to these questions can have a profound impact on how effective you are within a system and thus are worth exploring.
No matter the age of the system, every system and application has a story. Some systems are new and thus their stories may be short and hopefully uneventful. Other systems may have existed for decades and have a long and winding history that feeds into why it is the way that it is. Understanding this story is often the first step in making a change in a system, particularly if it is a substantial one. The problem is, how do you discover this history? In our fast-paced world, we do not have time to reinvent the wheel and try the same things over and over if they won't work. Conversely, if we are unable make changes to systems as time goes on we have entered a state of stasis and as the Jeff Bezos quote goes, stasis is "followed by irrelevance. Followed by excruciating, painful decline. Followed by death." No one wants that.
Let's dig into some ideas of how to share the story of the why of a particular system's architecture.
Storytelling #
This first idea is admittedly very fragile and can often not be possible but when you can make it work it can be extremely effective in concert with some of the other ideas that will be shared below. This is the idea of storytelling. When working with another engineer on a system share the stories of how the piece that is being worked on fits in the whole. Share the stories and the reasons why it is the way that it is. Share the downfalls and the issues that have been experienced with the system. This process obviously requires an initial person to know the story but after that, the story can be passed down by others. There is obviously a risk that the story will mutate as it goes down the generations and that is where additional methods can act as guardrails but the narrative format seems to stick well with people.
I have worked with a fellow software architect before that had a process that after a developer had been working in the system for a certain amount of time he would set up a meeting with them and they would sit at the whiteboard and he would have them draw out the architecture of the system and explain to him how it worked. This was not a test. While obviously someone working in a system without any idea of how it works would be concerning, there were no repercussions from how people explained things. This process served a few purposes. One, it allowed the architect to see how the knowledge of the system was being shared. If people had misunderstandings of how things worked he could correct them, if they were correct in their explanations that was a sign that the knowledge of the system was being disseminated. Having to share the story of the architecture also served as a benefit to the person describing it as it cemented in their minds how things work.
Quality Attributes #
Sometimes also called the "ilities" for the fact that many of them end with the suffix "-ility", quality attributes serve as documented wildly important non-functional attributes of a system. These are the most important non-functional attributes the system must achieve in order to be successful. Architecture and system design is a game of trade-offs. While we would love all of our systems to possess all the good things in the world this is not possible. Having well-documented quality attributes allows those working in the system to refer to them to help understand why the system is built the way that it is. It also provides a framework to test new decisions against to make sure the system isn't being taken in the wrong direction.
I have worked in a system before that was extremely complex. The pieces of the application were extremely loosely-coupled and distributed. None of these items by themselves is a problem. The problem came when trying to answer the question of "Why?" There really wasn't much of an answer. You would occasionally get an answer such as, "We need to support X amount of requests per second bursting to 1000X" which could serve as a good basis for a performance or scalability quality attribute, the only problem was those numbers were made up. Digging into the actual business requirements we discovered that the true quality attributes were speed of development, maintainability, data integrity, modest performance needs, etc. The original architecture which was partially first of its kind, had a high learning curve and data integrity issues. This was directly at odds with the quality attributes. This original architecture was later abandoned and a much simpler architecture was adopted that allowed us to meet those original requirements.
Architecture Decision Records #
Another way to explain the story behind the "why?" of part of a system is via an architecture decision record. An Architecture Decision Record (ADR) is a documented decision about an architecturally significant part of an application. There are many formats of architecture decision records that can be used but the general pieces of information encompassed in them is a status (whether it is accepted, obsoleted, etc.), what the decision was, the environment where the decision was being made, accepted downsides to the decision, and other options considered before arriving at the current answer. Additional sections can be added to a record if it is useful but this is the common core of what it encompasses.
One of the beauties of architecture decision records is that they can get across a whole lot of information very quickly in a fairly concise manner while only requiring a text editor to get started. This is likely why many people have suggested their usage and it has shown up on ThoughtWorks' tech radar, industry leaders like Michael Nygard have shared their merits, and more.
One of the places that the above format has been useful is when the environment changes. I have worked with a group that had a strong anti-microservice rule. This was largely accepted but they hit a point where they wanted to start leveraging a more service-oriented architecture. The problem was they had this documented decision not to use microservices. It was only when they discovered the reason why that decision was made that they could move forward. Historically the software this company built was deployed and managed on-premise by non-technical users. In that environment, the operational complexities of microservices were not acceptable. However, this company had since pivoted to deploying and managing the software themselves and offering it as a SaaS solution. Thus with the environment change, a new decision could be made using the new current environment as its guide. If the original environment and reasonings had not been documented, coming to the decision to pivot could have faced more complications than required.
Architecture Fitness Functions #
An architecture fitness function is a concept borrowed from evolutionary computing where a system is developed to determine how well a given system achieves a particular aim. These fitness functions rely on already having quality attributes and scenarios laid out to guide the test. This concept was originally explained in Evolutionary Architecture by Neal Ford, Rebecca Parsons, and Patrick Kua. These fitness functions often exist as automated tests in your system. If they are not running consistently and visibly to the developers in an application, their usefulness is significantly decreased. Much like more traditional automated tests can serve as documentation of what a system does and how it should work, automated fitness functions can serve as documentation in their own right about how a system works, should work, and why.
One of my favorite tools that helps support this functionality is a tool called ArchUnit. ArchUnit is a great tool that slides into your existing test framework and allows you to quickly start testing and documenting your architecture decisions. While there is a limit to the scope of what it can cover it performs its job well in its scope of influence. It also allows extension into enforcing new capabilities that it doesn't initially ship with. I have successfully used this library in order to build pre-emptive tests that make sure future developers are following the architecture guidance we are after in various capacities. Examples of enforcements I have written is making sure all tests are tagged with one of the accepted tags in the organization (unit vs integration in this case). Enforcing that particular parts of the application that would be treated differently in a production environment would work in that environment. This sped up the feedback when things were not designed in a way that would work in other environments. As a final example, I have been able to use it to enforce that code is not calling into non-published APIs of different parts of the system. These are just a few ideas of what ArchUnit can do.
These are just some ideas of how to document the "Why?" behind the architecture of a system (or even just a part of it). None of this work is free, it definitely takes effort. That said, it will pay dividends if you invest in the work. Not only will people know the critical parts of what makes up your system and why, but they will also be better able to recognize those parts of the system that are not critical that are more open to change. This all leads to accelerated development and happier developers which, I believe, leads to a more quality application.