Why complexity happens

Software gets complex as you add more features to it. We all know that. Designers, developers, engineers, product managers, and your drunk neighbor would all agree. But, if you’re not working directly on building software, it’s easy to under-appreciate how quickly complexity happens. I’ve coded for over 10 years now (my God), and even today, I have to stop my inner ignoramos from saying “Oh that should be easy” for a piece of new functionality applied to an application I haven’t written a single line of code for.

A simple way to understand complexity is to think of software ultimately as a bucket of states. When your actually using the application, at any given point in time, the application is in some state. So, how many potential states are there? As programmers, we don’t really think about how many potential states you can be in…but the code we write ultimately dictates how many there could be.

Complexity in simple raw numbers
Since we just released our web-based issue tracking tool DoneDone last month, this particular piece of software is still fresh in my head.

Let’s just talk about one screen in DoneDone, the issue detail page. How many states can an issue be in?

From a pure data standpoint, an issue has several parameters that make up its “state”. It can be in one of nine statuses (open, in progress, not an issue, not reproducible, missing information, ready for next release, ready for retest, pushed back, and closed). It can have one of four priority levels (low, medium, high, critical). Suppose we have 10 people on a particular project. There are 10 potential issue creators that can then assign the issue to nine potential issue resolvers. Issues can have attachments or not. For simplicity, we can say an issue has one of three attachment states (none, one, or more than one). An issue also may or may not have a description (we require only a title).

So where does this get us? 9 potential issue statuses, 4 potential priority levels, 10 potential creators, 9 potential resolvers, 3 potential attachment states, and optionally, a description.

9 * 4 * 10 * 9 * 3 * 2 = 19,440 potential states. Each potential state is an opportunity for a bug.


It’s fair to assume that ain’t no one gonna be testing 19,440 possibilities to make sure they all behave as we expect. Not even in today’s economy, nuh uh. And fortunately, it’s pretty safe to say we don’t have to.

For issues, who actually creates or resolves an issue doesn’t really matter (Craig Bryant, Lindsay Woods, or Mustafa Shabib should have the same results). So, we can get rid of the 10 * 9 = 90 potential creator/resolver combos. That gets us down to 216. Sweet!

But, actually, people do matter. Administrators have different access levels than regular users. A creator may or may not be an admin, and a resolver may or may not be an admin. 216 * 4 = 864. Crap.

You can continue playing this game of “state volley” in your head for hours. Do these parameters really effect the state of an app? No? OK, get rid of them from the equation. Ah, but this does. Multiply it back in.

You’ll find that the mere exercise of trying to talley the number of states of your application that need to be tested for bugs is, itself, difficult. Back to the DoneDone example, it may be that three of the issue statuses behave in roughly the same way, and the other six are in their own world of uniqueness. Or, some parameters only matter sometimes, and when they matter is a function of what the other parameters happen to be at that point in time.

Complexity as a game of pick-up sticks
All this points to the age old game of pick-up sticks…

Apparently, this game dates back to 500 B.C.E. I wonder if they came in that sweet canister back then.

Anyways, the objective of pick-up sticks is simple. Try removing sticks one-by-one without disturbing the rest.

Software sometimes feels alot like a game of pick-up sticks. Each stick metaphorically represents some new “feature” to your app. Sometimes (though rarely), a feature just lives outside the whole mess of sticks in the middle. Sometimes, it touches a stick. Sometimes it’s all entwined around a bunch of sticks.

Implementing a new feature is like adding a new stick into the mix and trying to remove it without disturbing other sticks. Complexity adds up fast. Everytime you add a new feature, it can disrupt a host of other features that might not at first seem directly connected. Not only is it hard to predict how many other features might be disrupted, but it’s also hard to predict which ones. As you add more features, those multipliers and disruption points grow pretty rapidly.

To that end, the next time you’re ready to throw in that new feature or that new parameter into your beautiful app, really consider what it does to the scale of your complexity. You may be surprised how quickly the numbers add up.

3 responses to “Why complexity happens

  1. Right on, right on. Interesting to consider how a micro-arch like Cairngorm helps us ensure we can, as often as possible, place that ‘new stick’ on top … and make it easy to get in and out. Certainly not always the case …

  2. True. I’d say that when you write good code, abstracting things as necessary, it goes a long way to preventing bugs. But, the number of manifested ‘states’ is still there…

    Those run-time combination of parameters and micro-states that make up the overall-state-of-the-app get really hard to predict.

    The benefit of good architecture, at the very least, is it makes it much easier to unit test / debug why software is breaking.

    I guess the other part I should address is, we find bugs at a very low order of magnitude. I.e., an app can work 99.9% well, but that still might mean….10 bugs. 10 bugs feels bad – and is – but relative to how complex software gets, it actually might mean 99.9% bug-free.

  3. The problem is that complexity has become embedded in the software. Partially due to OOP methods.

    Lets compare scripting and OOP for a moment. And assume that without user driven complexity they are basically equal in the wild for arguments sake.

    The OOP team will decide that the way to make the app flexible is to make sure every object is instantiated with a factory and use dependency injection for testing, use at least 3 “layers” of tightly coupled code and also to build a completely generic UI which requires templates and prebuilt components. Add massive amounts of XML configuration and other head scratching. (eg Rails code generation for *differently named* objects which are also all basically the same.

    Ok that sounds complex!

    A scripted application can make different choices. A scripted application which modestly respects normal boundaries of code but does not induce layering or factories or depends injection, can still use templates and prebuilt components.

    When something comes along the scripting team didnt expect, they can simply in 1 line of code call out to a copy and paste version that does the right/different thing.


    This is what OOP people dont get. They WANT to impact and refactor because they simply have no choice because of their earlier failures at making false, predictive choices.

    Scripting languages can fork entire apps with a few scripts copied and pasted and ahem “injected” into the main application.

    Done. So you spend the rest of your life working on well defined interfaces for a constantly changing requirement.

    Ill just use scripts and get the same job done in a fraction of the time, and go fishing on the weekends.

    But I do sadistically admire the irrational compulsion to keep fishing in a dry lake.

Comments are closed.