Category: Software Architect

As software developers we initially understand software as a system of commands, functions and algorithms. This instruction-oriented view of software aids us in learning how to build software, but it is this very same perspective that starts to hamper us when we try to build bigger systems.

If you stand back a little, a computer is nothing more than a fancy tool to help you access and manipulate piles of data. It is the structure of this data that lies at the heart of understanding how to manage complexity in a huge system. Millions of instructions are intrinsically complicated, but underneath we can easily get our brains around a smaller set of basic data structures.

For instance, if you want to understand the UNIX operating system, digging through the source code line-by-line is unlikely to help. If however you read a book outlining the primary internal data-structures for handling things like processes and the filesystem, you‘ll have a better chance of understanding how UNIX works underneath. The data is conceptually smaller than the code and considerably less complicated.

As code is running in a computer, the underlying state of the data is continually changing. In an abstract sense, we can see any algorithm as just being just a simple transformation from one version of the data to another. We can see all functionality as just a larger set of well-defined transformations pushing the data through different revisions.

This data-oriented perspective — seeing the system, entirely by the structure of its underlying information — can reduce even the most complicated system down to a tangible collection of details. A reduction in complexity that is necessary for understanding how to build and run complex systems.

Data sits at the core of most problems. Business domain problems creep into the code via the data. Most key algorithms, for example, are often well understood, it is the structure and relationships of the data that frequently change. Operational issues like upgrades are also considerably more difficult if they effect data. This happens because changing code or behavior is not a big issue, it just needs to be released, but revising data structures can involve a huge effort in transforming the old version into a newer one.

And of course, many of the base problems in software architecture are really about data. Is the system collecting the right data at the right time, and who should be able to see or modify it? If the data exists, what is its quality and how fast is it growing? If not, what is its structure, and where does it reliably come from? In this light, once the data is in the system the only other question is whether or not there is already a way to view and/or edit the specific data, or does that need to be added?

From a design perspective, the critical issue for most systems is to get the right data into the system at the right time. From there, applying different transformations to the data is a matter of making it available, executing the functionality and then saving the results. Most systems don’t have to be particularly complex underneath in order for them to work, they just need to build up
bigger and bigger piles of data. Functionality is what we see first, but it’s data that forms the core of every system.

Things are usually easier said than done, and software architects are notoriously good at coming up with things to say. To keep your words from becoming a lot of hot air (generally the key ingredient in making vaporware), you need a good team of developers to make it happen. The role of an architect is usually to impose constraints, but you also have the opportunity to be an enabler. To the extent your responsibilities allow, you should do everything possible to empower your developers.

Make sure developers have the tools they need. Tools shouldn’t be imposed on developers, they should be carefully chosen to make sure they are the right tools for the job at hand. Repetitive and mindless work should be automated wherever possible. Also, it is well worth the investment to make sure developers have top-notch machines to work with, adequate network bandwith and access to software, data and information necessary to carry out their work.

Make sure they have the skills they need. If training is required, make sure they get it. Invest in books and promote active discussions about technology. The work life of a developer should be hands-on and practical, but also should be actively academic. If you have the budget for it, send your team to technical presentations and conferences. If not, get them involved in technical mailing lists and look for free events in your city. As much as possible, participate in the developer selection process as well. Look for developers that are hungry to learn, that have that little “spark” that says they really dig technology (also make sure they can play ball with the team…). It’s hard to get a big bang out of a team of duds.

Let developers make their own decisions wherever it won’t contradict the overall goal of the software design. But put constraints where they count, not only to guarantee quality, but also to further empower developers. Create standards for the sake of consistency, but also to reduce the number of troublesome, insignificant decisions that aren’t part of the essential problem developers are solving. Create a clear roadmap for where to put their source files, what to call them, when to create new ones, and so on. This will save them time.

Lastly, protect developers from nonessential parts of their job. Too much paperwork, and too many office chores adds overhead and reduces their effectiveness. You (usually) aren’t a manager, but you can have influence on the processes surrounding software development. Whatever processes are used, make sure it is designed to remove obstacles, not increase them.