I believed we could start the shift towards becoming an “outcome-focused” team by improving and defining our sprint review process.
Writing Useful Performance Reviews: Assembling the data
Writing performance reviews for your engineering teams can be daunting. Here are some steps you can take to make the process less onerous, no matter your company's review format or timeline.
Kevin Goldsmith is the Chief Technology Officer at DistroKid. He has previously worked at Anaconda, Onfido, Avvo, Spotify, Adobe, and Microsoft. In addition to being a dad, musician, and photographer, Kevin has run a record label for 30 years.
This article was originally published on Kevin Goldsmith's blog.
It’s January, and that can mean only one thing. For many of us, it is now – or soon will be – time to write performance reviews for our team. Writing reviews can be daunting for many, especially those with large groups or little experience. I often hear managers (even senior leaders) bemoaning the effort it takes to write the reviews for their group members. However, there are some things you can do that will make the process less onerous, no matter what format or schedule your company has.
I’m going to break this subject into four parts:
- Assembling the data (this article)
- Evaluating the data and writing the review
- Making salary recommendations
- Delivering the review
Why do we do performance reviews?
The rationale we used to hear for performance reviews is that they are for the employee to know how they are doing, to give them helpful feedback on what they are doing well and where they need to improve. Today we try to provide this feedback often, throughout the year. I often tell the managers on my teams that there shouldn’t be any surprises in the performance review. It should instead be a summing up of the feedback that the person has been receiving all along.
If we give feedback throughout the review period, why do we need to do the performance review? It is for the company and us almost as much as it is for the person receiving it. Ideally, we maintain a narrative across the year with our feedback, reviewing months of our notes and prior communication before each one-on-one. All too often, the larger arc gets lost in the whirlwind of work. The feedback we give is usually very transactional about what has just occurred. If there are significant overarching discussions, we may be able to tie the feedback to that, but often the narrative gets lost.
The review is a chance to look across all that has transpired over a much lengthier period than the time between one-on-ones. It is a chance for us to take stock and find new patterns or trends that we may have missed. To look at the bigger picture and then build a shared understanding of that picture with the team member.
The review is also for the company because the company keeps a record of employee performance to justify bonuses, promotions, salary increases, and stock offerings. It is also vital to have a history of performance for a new manager if you move on from your role. Sometimes you will move to a new job, or the employee will move to a new team. When that happens, all the shared understanding you have built up is lost unless it’s written down. An employee who has been working years towards a new role may be set back significantly if their new manager doesn’t understand the efforts they have made over time and is looking only at what they see in the present moment.
A well-written review is a valuable document for the person receiving it. First, it is a checkpoint for them to refer to as they work towards their career goals. Second, it is a useful document for you to help them on their career path. Third, it is a favor to their future managers at your company. Finally, it is a critical document for the company and your manager to understand how to manage compensation for the person.
Preparing for the performance review. Start early!
Often writing reviews seems like a great deal of work because we wait until our company’s official “kick-off” of the review period. The people/HR team lets all the managers know the schedule, does a few meetings to discuss/update the process, and opens the forms for managers to enter data. If you wait for that moment to begin preparing your reviews, you may find yourself spending a lot of nights and weekends trying to get your evaluations prepared, since your regular work continues during this time. In the past, I’ve spent more than a few sad weekend days sitting in a ski lodge huddled over my laptop, writing reviews while my family was out on the slopes having fun.
Reviews happen at the same time every year.
Your company may adjust the dates slightly, but you can be confident that reviews will happen around the same time each year. When the dates are announced, you should be prepared. If you are incredibly diligent, you may be collecting and organizing data for your reviews year-round. If you haven’t done that, you can start reviewing, amassing, and organizing supporting data as review time approaches so that you don’t have to struggle and potentially miss things. While the format of the reviews in your company may change periodically, the general things that are measured likely won’t.
There are many sources of data you can assemble for the review.
As I start my preparation, I create a folder on my computer for the review period and a subfolder for each person. In each folder, I assemble all the documents and data for the performance assessment. I prefer to keep local copies because it is less likely that I will accidentally share the folder or file. To focus, I often go offsite to work on reviews, and sometimes these places have sketchy connectivity. Having the documents stored on my computer has functioned well for my process.
My primary data always comes from the notes I take during my one-on-ones and in meetings.
I used to store all my notes in Evernote organized by meeting (for recurring meetings) and tagged with the people in the discussion. This storage approach made it easy to find all my notes for each person to track what we spoke about across the review period. However, during the pandemic, I switched to paper notebooks. Now I keep an index of which pages people appear on. This index makes it easy to find all my notes referencing someone.
As I review my meeting notes, I assemble meaningful comments or things I notice into a new document in the person’s folder to organize my data for the review. I include where I got it from for each item in case I need to go back to the source.
The person’s prior reviews are essential.
I always download copies of any previous reviews for the person in the system and put them in the folder. It is vital to remember our prior review conversations and see any reviews before they reported to me. Reading previous reviews helps me understand the different challenges and strengths they have had and understand their career story at the company so far.
E-mail and slack exchanges may remind you of other events from the year.
Occasionally, things come up and are resolved between one-on-ones or meetings, so they don’t appear in your notes. I scan over the e-mail and Slack exchanges I have had with the person during the review period to see if I missed an event in reviewing my meeting notes.
I copy/paste these exchanges into the notes document in the person’s review folder, or summarize them there.
Make a list of peers of the person from whom you want to request feedback.
Your company may include a formal peer-review element in your performance reviews process. However, if it isn’t part of the company process, you will still find it valuable to ask for peer review feedback. The first step is to list the people you would like to ask for feedback, so you are ready. You may also write the template for the peer feedback request to prepare you to send them out.
I’ve noticed that in companies with a formal peer feedback process as part of their reviews, people quickly become inundated with feedback requests. Your chance of getting valuable (or any) feedback is greatly improved if you send the request early, before people have feedback fatigue.
If you know that peer feedback will not be part of your company’s process, you may still want to send out the feedback requests early to get the responses with enough time to follow up if there are questions. However, make sure that you specify a date by which you would like the feedback returned, and don’t make that date too far in the future, or people will put the request aside and forget about it.
The message template goes into the top-level performance reviews folder, and the list goes into the person’s folder. If you want to be tricky, you can put the list in a CSV file to make it easier for a mail merge. You may generate a lot of e-mail performance feedback requests as part of this process. I’ve automated this over the years.
When you receive the feedback, save a copy of it to the folder.
You may need to ask for a self-evaluation.
If your company does not include self-evaluation as part of the review process, you may ask the people you review to do that for you. If you are unsure what to ask, use your company’s career pathing rubric for their job/level. Ask them to compare themselves to the rubric and give examples of how they have met, exceeded, or missed the expectations. If you use individual goals or OKRs, they should talk about how they achieved or missed them. They should also talk about the areas they want to improve on for the coming review period.
You want them to complete their self-review early enough that you have time to follow up with them or others on anything that comes up in that document. Your company will set the dates for you if it includes self-review as part of the review process.
Save the self-evaluation to the person’s folder.
Review the work output.
A critical part of the performance review is reviewing the actual value the person created for the customers and company. A portion of your performance review as a lead or manager covers what your team achieved. Think through your teams’ accomplishments and think about how this person contributed to or detracted from those projects. Add concrete examples to the notes document.
Look over the person’s commits to the code of the project. Did they review others’ code? Did they contribute helpful comments? Did their code require many fixes? Did they contribute to the project documentation? Look over their comments in your project and bug tracking systems. Did they contribute helpful information? Did they help others?
It can be very tempting to try to be “objective” when looking at work output. Counting lines of code produced, number of commits, number of issues filed or closed, or story points completed might seem like unbiased data. Avoid this temptation at all costs. People have different approaches to knowledge work. Even if your team has strong guidelines on how work should be done, people will always have methods that your seemingly objective process might miss. Instead, focus on the value they contribute to the team and watch in the peer feedback for what they contribute that won’t show up in the source management or issue tracking systems.
Save your observations on their work output in your notes document.
What if the person didn’t report to me for the entire review period?
If the person is a new hire and is still eligible for a performance review, you will use this process, but just for their time in the company. You will have to make allowances for their onboarding and focus more on how they learn to contribute than on their actual contributions.
If you are a new manager to an existing team, spend as much time as you can with the prior manager to understand how they have approached each person’s development. Read the reviews for each person before talking to the manager. If the manager has left the company, you can still reach out to them. Hopefully, they will still want the best for their old team. Depending on how long you were in the team during the review period, you may need to emphasize the peer review component more than you would have otherwise. Be aware that changing a team’s manager is very disruptive to the team. You will only have seen the results of that disruption and how the team now works.
If the person joined the team from a different group in the company, consider doing a joint review with their prior manager to cover their work before joining your team. If that doesn’t seem necessary, you should still have an extended conversation with their former manager after going over the person’s previous reviews.
This process seems like a lot of work!
It is! It should be. It is important stuff. Think about the best reviews you have received from your current or former managers. Not just the performance reviews that were the most positive, but the ones that made you feel like your manager cared about your development. A good review inspires you with the knowledge that your manager and the company recognize the worthy work you’ve done. You know that your areas of improvement have been considered and are essential for your career development.
A good review requires good data. Therefore, it is incumbent on you to make sure you are going over as much as you can, not just what you can remember at the end of the review period (also known as recency bias).
The first time you go through this process, it will take a great deal of effort, but the payoff will be worth it. For the next period, you will learn to collect and organize this data as you go. If you assemble and categorize data all the time, it will be helpful in your one-to-ones as well and not just at performance review time.
Now you have the data. What next?
Now that you have assembled your data, you can evaluate the data against the expectations of the role and level. I will discuss that in the next part of this four-part series.
Thanks to Laura Blackwell for editing assistance