As an architect you are tasked with providing the initial structure and arrangement of software systems that will grow and change over time, will have be to reworked, will have to talk to other systems, and almost always in ways you and your stakeholders did not foresee. Even though we are called architects, and we borrow many metaphors from building and engineering, great software is not built, it is grown.
The single biggest predictor of software failure is size; on reflection there’s almost no benefit to be had from starting with a large system design. Yet at some point we will all be tempted to do exactly that. As well as being prone to incidental complexity and inertia, designing large systems upfront means larger projects, which are more likely to fail, more likely to be un-testable, more likely to be fragile, more likely to have unneeded and unused parts, more likely to be expensive, and more likely to have a negative political dimension.
Therefore resist trying to design a large complete system to “meet or exceed” the known requirements and desired properties, no matter how tempting that might be. Have a grand vision, but not a grand design. Let you and your system learn to adapt as the situation and requirements inevitably change.
How to do this? The best way to ensure a software system can evolve and adapt is to evolve and adapt it from the very outset. Inducing a system to evolve means starting with a small running system, a working subset of the intended architecture – the simplest thing that could possibly work. This nascent system will have many desirable properties and can teach us much about the architecture that a large system, or worse, a collection of architectural documents never can. You are more likely to have been involved in its implementation. Its lack of surface area will be easier to test and therefore less prone to coupling. It will require a smaller team, which will reduce the cost of coordinating the project. Its properties will be easier to observe. It will be easier to deploy. It will teach you and your team at the earliest possible moment what does and does not work. It will tell you where the system will not evolve easily, where it is likely to crystallize, where it is fragile. Where it might break. Perhaps most important, it will be comprehensible and tangible to its stakeholders from the beginning, allowing them to grow into the overall design as well.
Design the smallest system you can, help deliver it, and let it evolve towards the grand vision. Although this might feel like giving up control, or even shirking your responsibilities, ultimately your stakeholders will thank you for it. Do not confuse an evolutionary approach with throwing requirements out, the dreaded phasing, or building one to throw away.