Posted by Emerson Murphy-Hill, Research Scientist, Central Product Inclusion, Equity, and Accessibility
At Google, we often study our own software development work as a means to better understand and make improvements to our engineering practices. In a study that we recently published in Communications of the ACM, we describe how code review pushback varies depending on an author’s demographics. Such pushback, defined as “the perception of unnecessary interpersonal conflict in code review while a reviewer is blocking a change request”, turns out to affect some developers more than others.
The study looked at pushback during the code review process and, in short, we found that:
We estimate that this excess pushback roughly costs Google more than 1,000 engineer hours per day – something we’re working to significantly reduce, along with unconscious bias during the review process, through solutions like anonymous code review.
Last year, we explored the effectiveness of anonymous code review by asking 300 developers to do their code reviews without the author’s name at the top. Through this research, we found that code review times and review quality appeared consistent with and without anonymous review. We also found that, for certain types of review, it was more difficult for reviewers to guess the code’s author. To give you an idea, here’s what anonymous code review looks like today at Google in the Critique code review tool:
In the screenshot above, changelist author names are replaced by anonymous animals, like in Google Docs, helping reviewers focus more on the code changes and less on the people making those changes.
At Google, we strive to ensure there is equity in all that we do, including in our engineering processes and tools. Through continued experimentation with anonymous code review, we’re hoping to reduce gaps in pushback faced by developers from different demographic groups. And through this work, we want to inspire other companies to take a hard look at their own code reviews and to consider adopting anonymous author code review as part of their process as well.
In the long run, we expect that increasing equity in developers’ experience will help Google – and our industry – make meaningful progress towards an inclusive development experience for all.
