🧿 Complexity is not the enemy

We fight complexity every step of the way when we should be focusing on embracing it and instead focus on managing uncertainty.

Howdy Tribe -

It’s Aadil.

Lately, I am struggling with how to think and approach complexity within software product development. Most common reaction that I have seen to it is one of fear. A poltergeist here to ruin everything; a bug; something to be crushed or removed; a road block; failure to a certain degree.

I believe really great product teams (combination of engineering, design, product, project, marketing, testing) create fantastically complex solutions capable of achieving supremely complex functions. In fact, I encourage and advocate that complex solutions aren’t a bug but a feature. What needs to be feared, tackled, and managed is uncertainty.

Any fairly modern system - whether it be an app, e-commerce solution, banking platform, voice AI hardware - will be inherently complex. Then, why are we so hell bent on removing complexity from systems that are by their very nature complex?

Mantra’s like keep it simple stupid, keep it straightforward, you are making this way too complex, it’s too complex for our current system, these aren’t in my mind a fear of complexity but rather a misguided reaction to uncertainty.

It is human nature when we encounter something we don’t understand, we deem it complex. So, it is not unexpected that when developing solutions to difficult problems our focus is trying to find the simplest solution, or the easiest one to implement because there is an implicit measure of certainty that we have on how these solutions will react in various scenarios. Simple solutions can be complex. It’s not mutually exclusive.

Photo by Shahadat Rahman on Unsplash

Developing complex solutions results in a few existential moments for product development teams:

• We are forced to realize the current system isn’t capable of scaling to where we need it to go. (Read: This will be a lot of work. So much rework or retooling. 😭)

• Building the right solution is complex and unfortunately it does not fit our business timelines. (Read: Product teams are taking too long to build things. 😒)

• Should we just build it ourselves or perhaps buy a solution? Investment $$$. (Read: This will need a lot of money and headcount. Let’s just buy a solution instead and retrofit it to make this work. 😩)

Part of the role of an Engineering Project Manager is helping shape the technical conversation from the fantastically wild and imaginative towards achievable and focused. In other words, leadership will always have an expectation that product leaders are driving towards solutions that are:

• Simple and straightforward

• Don’t cost too much (human and financial resources)

• Can be delivered in the shortest time possible.

Should then complex solutions be avoided from the very start? No. A healthy amount of complexity is natural and should be expected. Product teams that cannot appreciate and accept this reality will always struggle with endless discovery, constant rework and roadmap slog. Starting from “keep the solution simple” creates an artificial restriction on the product teams creativity; something that is unnecessary.

What then is a healthy amount of complexity? Here are a set of questions, in no particular order, to help you know when your complexity has become unhealthy:

• Are we certain the solution will achieve the original goal?

• Do we understand what pieces constitute the critical path?

• Are there any edge cases we have not solutioned for?

• Do we understand what impacts this solution will have on downstream systems?

• If the entire product team was replaced today, will the new team understand the basic components and happy path functionality without needing to rewrite large parts if not all of it?

• Do we understand the weakest parts of the system?

• How brittle is the solution?

• Have we tried breaking this system/solution?

• Is the solution bespoke to the current problem?

• Do we understand the performance characteristics of the solution?

• Do we understand how this solution impacts the way business teams do work today?

• Are we taking on any technical debt with this solution?

• Are we not taking this solution because we will need to rewrite large parts of our existing system?

• Have or do we need to consider backwards compatibility issues?

• Do we have a fail-safe switch mechanism (read: can we revert or feature flag incase of catastrophic failure)?

If you notice, the questions above are not focused on how quickly can we deliver the solution/product or cost. Time and cost estimate accuracy is entirely dependent on the product scope and understanding of how the solution/product works. Thus, removing uncertainty of both technical and non-technical kind should be the primary goal.

You will be tempted, as the Engineering Project Manager, to be that person in the room to constantly state the two most annoying one-liners that trigger me:

• KISS

• We are agile, not waterfall.

DO NOT BE THAT PERSON.

When product teams are doing initial discovery and formulation of a product or solution, let the creativity flow, don’t stifle it. Let the discussion flow from divergence to convergence but when you converge, your objective is not to remove complexity but laser focus on uncertainty. Sure you can help by timeboxing the conversations but let product teams have their green field pastures to let them think before you suddenly start asking about MVP, v1.0, etc.

If you know how a system works and can fine tune it as your requirements change, well then, it’s not complex in my books. 🧐

Until next time 👋🏽 🙏🏽 !

-Aadil

---

Additional Reading

Great article by Boz on complex systems and failures.

Boz @boztank

Simple systems fail simply. Complex systems fail catastrophically. Adding complexity to reduce failure can be healthy or it can backfire in a major way. My latest blog post is up and this one feels very fundamental to how I approach systems:

boz.comFailure TheoryWhile the rate of failure can be reduced over time that often comes at the expense of introducing the possibility of more catastrophic failures. It can be helpful to model failure as a fixed sum which will be released either in a large number of small increments or a small number of large ones.

---

Was this edition useful? Relevant? Help me to improve!

• 🤩 Awesome

• 🙂 Good

• 😐 Okay