BA 46/52: Tech Debt is Good
Strategies and lessons from how we dealt with tech debt at Apple in a robust manner.
I have a straightforward point of view on Tech Debt. I believe it is good. Why? It means a few things:
You as a team or leader made hard decisions when push came to shove.
You changed your mind and deviated from the set plan.
You wanted to move fast and go for big gains.
You code is robust and evolving.
Yes, tech debt can also mean negative things but for this piece, I presume optimism. I will keep the cynicism for another post.
For me, unpaid tech debt is the worst, obviously. In my experience, unpaid debt manifests in a few ways:
Performance and quality issues.
Unexpected failures and issues.
Hampers feature development velocity.
Bad consumer outlook of your product and service.
There are a wide range of strategies to be employed to dealing with tech debt ranging from the most basic to the most comprehensive.
Here is how we dealt with tech debt at Apple:
Dedicated Development Block
Overview — When I first joined Apple, during iOS 6 development, we had a month or so blocked out during the development phase which we called performance month.
Additional Details — During this part of development, all teams focused on resolving performance, quality and stability issues with the feature work completed thus far. Additionally, teams could take this opportunity to address other/existing tech debt issues.
Result — this strategy worked for awhile until the features became more complex and Apple platforms become more intertwined (iOS, MacOS, WatchOS etc). This is great for paying down debt as you incur it but not a great tool for dealing with historical tech debt.
Top 100 List Approach
Overview — In between major iOS and MacOS releases, the Program Office would track a list of Top 100 issues which we would target to address in minor releases (iOS 16.X.X).
Additional Details — An established criteria would determine whether an issue goes on the Top 100 list. Primary sources for this list would be customer feedback, bugs pushed out from previous releases, engineering teams requests. The senior engineering leadership would collective agree what made the list or not including input from design, marketing and program leadership.
Result — This method works great for keeping on top of the tech debt that you are creating and address the most egregious customer issues however, often times only the loudest voices or teams would make the cut. Additionally, 100 is a tricky number but mentally, this value is ideal for keeping teams focused, scope small and nimble, yet the impact of these issues once fixed large.
Dedicated % Capacity to Quality
Overview — at the tail end of my Apple years, Craig Federighi and his staff constituted that every engineering team will dedicate 50% of their capacity to quality issues including tech debt.
Additional Details — this mandate was the direct result of the degrading iOS quality in press reviews as teams were primarily focused on shipping new and bigger features every year. This in my opinion is a blunt, and perilous approach which can yield great results. However, it can also produce lower than expected results.
Results — majority of the teams who are not delivering tentpole features or significant portions of new tech in the iOS release could meet this mandate. Many teams focused on tentpole features would often have no capacity left to deal with quality issues. However, those teams that stuck to this mandate achieved great results.
Nuclear Option: Rewrite
This one is straightforward — sometimes you have to declare bankruptcy and start all over. This requires data and information to prove to senior leadership that any further development must be paused and a complete rewrite is required.
Again — tech debt can be a positive signal. Where teams struggle with the most is how to address the debt.
If you ask me, it’s always best to keep some capacity to focused on paying down this debt through the development lifecycle. Treat debt like any feature development project. Then, when the time comes, acknowledge that the current implementation and architecture has reached its peak, now we begin again with something more modern and meets our current needs.
Until next time 👋🏽!
-Aadil
How is this week’s newsletter?