Lessons On Evaluating Engineering Team Performance
Issue 010 - Drawing upon my time working with industry’s best engineering leaders at Apple and Google, here is how to think about measuring engineering team performance and diagnosing output issues.
Howdy Tribe -
Aadil here.
24 people signed up for this newsletter since my last essay. This edition of the Building Romes Newsletter is going out to 145 people and counting.
If you have been enjoying reading Building Romes, please share it with your friends and get them to sign up!
I would love to hear from you about topics, ideas, questions you may have for me so, HIT REPLY, and I reply to every email. 🙏🏼😬❤️
Here is the list of essay’s I have written in the month of November that you may have missed:
Onward to today’s essay.
E.
What are we solving today?
Every leader wants their team to do more in less time. Performance not only reflects the caliber of the leader but the individual engineers as well. To track performance you must measure.
It is a myth that engineers don’t like to measure their performance. An engineer by nature is analytical. I have yet to meet an engineer or engineering manager who was repulsed by data. What is repulsive to them is how that data is most commonly used. The worst approach would be to take convergence charts and beat individual engineers over the head with:
“WHY AREN’T YOU FIXING FASTER”
“WHY ARE BUGS OPEN FOR SO LONG”
“WE ARE NOT CONVERGING, WORK LONGER”.
So, let’s say you exercise your powers responsibly, and the data shows an team efficiency issue. Bugs take a long time to solve, feature work keeps slipping, and despite your best efforts you have concluded the engineering talent is sub-optimal and the course of action is to replace them.
Hold your horses. I think you’re missing an important piece of the puzzle.
Lessons Learned and A Solution
The commonly accepted method of measuring performance and output for any engineering team is quantitative:
Pull Request KPIs
Bug burndown reports
Velocity Charts
Bug Life cycle KPIs
Sprint Over Sprint Tracking
Unfortunately, this is only half the story. You need to dig deeper and look into qualitative measures for the complete picture before deciding a drastic course of action.
Here is a checklist of qualitative measures, both at individual and team level, that I have learned from really smart engineering leaders from Apple and Google to diagnose performance issues..
Some engineers are fantastic at programming but not so great at communication.
Are there communication barriers or differences that are creating performance issues?
Is the team able to articulate or communicate blockers, issues, code concern during code reviews or engineering sessions?
Does the team take an unusual amount of time to parse and accept Product Requirements?
Team is taking longer to solve bugs, they must be bad engineers - hang on, dig deeper:
Do you have bugs open longer but code is being submitted. Perhaps bad bug hygiene? Easy one to diagnose.
Perhaps the team is holding on to bigger bugs when they should be breaking them down into smaller more manageable ones?
Is there churn in the team to understand what the bug is actually about?
Is the team raising blockers at the right time to prevent delays. If not, why? Do they feel a psychological safety within the org where they don’t expect to be blamed for highlighting mistakes?
Do you have engineering burnout?
Perhaps there are issues outside of the team’s control.
If engineering teams are expected to be high performance, are the supporting functions like Project Management, Product Manager, Design, QA, Marketing also performing at high levels?
No two features are the same especially when you are building something new. Have you taken that into account? Perhaps complexity wasn’t totally resolved between Feature A versus Feature B which creates the perception of the team being slow?
Are you, the leader, the problem?
Are you making decisions at high efficiency so as to ensure the team is not blocked?
Do you have a vision problem? Does the team feel motivated and connected with the broader vision of the organization?
Is the team leader leading with a growth mindset or are they empire building and just managing up or micromanaging thus alienating the individual engineers?
Do you have motivation issues within the team because of the leader?
Inter-team relationships can be a massive source of contention and impede performance.
Sometimes having a toxic personality on your team can create hurdles and issues especially when you are relying on said person to work on a dependency in a collaborative manner. Do you have a toxicity problem?
Are the “A” players within the team spending time teaching and helping the new or weaker engineers to grow or dictate solutions? Easiest way to diagnose this problem - take a look at the Code Reviews and how the feedback is being delivered to junior engineers.
By no means are these questions the end all be all. The goal of this checklist is to add another perspective and perhaps help you shift the view to find a problem you may not be able to see with the data and information you have right now.
Final Thoughts
Output and performance is analytical in nature thus the approach is to solve math with math. But, unlike a math problem where the core components are numbers, engineering output’s core component is human. Analytical issues are also easier to solve - slow output, try adding more people; quality issues, give more time to work on quality bugs versus features; bugs taking longer, get help from peers etc.
Qualitative issues require a human touch which often means solutions and answers not easy to implement with difficult decisions and conversations. They are harder to diagnose let alone identify and measure.
When solving problems we want quantitative data so we can measure, analyze, investigate, and make conclusions. Qualitative data requires a nuanced and thoughtful approach because this kind of measure is shaped by our own experiences and opinions thus a much harder task than tape measuring. However, often times the delta between perceived and actual problem scope lies not in numbers but in the qualitative aspects, the human bit.
I hope this helps.
Until next time 👋🏽 🙏🏽.
- Aadil
Was this essay useful? Relevant? Pointless Topics? Help me to improve!
