With the ongoing reviewer crisis in our community, I asked myself a simple but uncomfortable question: Am I reviewing as much as I am asking others to review my work?
Fortunately, I have kept records of almost every review I have written over the past 11 years, which allows me to reflect on my own balance. The dataset is not perfect. It does not include re-reviews in revise and resubmit rounds (such as at CHI), and it combines different types of contributions, including full papers, posters, workshops, and student research competitions. Still, it provides a solid picture of my reviewing activity.
Between summer 2014 and summer 2025, I wrote 541 reviews across 59 different venues such as CHI and MobileHCI. That works out to around 4 reviews every month, consecutively for 11 years (or 0.94 reviews per week – if this is better to grasp). This is in addition to other community roles I have taken on, such as serving three times as CHI Subcommittee Chair and multiple operational roles (e.g., Publications Chair). Figure 1 provides an overview of my scientific service roles.
The next question is whether I review as much as I require others to review when I submit my own work. This is not easy to calculate, but some assumptions help. Most of my publications receive about four reviews (three reviews plus one meta review). Like everyone else, I also resubmit papers, which further increases the number of reviews requested from the community. As of this writing, I list 197 publications on my webpage across papers, journals, posters, and workshops. If each of these would generate four reviews, that would mean at least 788 reviews were requested by my co-authors and me.
The good news is that I have co-authors who also contribute reviews. When I account for my share, this works out to about 196 reviews over 11 years. In comparison, I actually contributed 541 reviews during that period. This means I have built up a substantial surplus. Figure 2 shows this breakdown, with blue bars for reviews I generated and red bars for my review share.
Of course, the picture is more complex. I do not have single-author publications, so I never personally generate the full four reviews per paper. On the other hand, I often publish with undergraduate and master’s students who are not (yet) contributing reviews, which is not reflected in this calculation.
Even with these caveats, the comparison is clear. Against a personal “review debt” of 196, I have contributed 541 reviews, which puts me well into surplus.
Next, I looked at the quality of the reviews I provided. Of course, review length is not a perfect measure of quality, but it is the most consistent metric I have. For 92.6% of my reviews, I was able to retrieve the full text. The missing 7.5% have two main explanations: either I forgot to save them, or the paper was desk rejected and the process never resulted in a written review, but in a reviewed paper. From this data, I extract that my reviews are on average 547 words long (median 479). For context, the CHI 2024 Blog reported mean review lengths of 699 words for regular reviews, 611 words for 2AC reviews, and 311 words for 1AC reviews. Since my dataset also includes 1AC and 2AC reviews, I would argue that my reviews are roughly in line with these typical lengths.
Lastly, I asked myself what is happening over time. Am I getting lazier as I get older? Before looking at the graph, I was convinced that my reviews had become shorter. These days, with more experience, I can assess the quality of a paper more easily, focus on the major concerns, and avoid spending time on minor issues when the paper already has major flaws.
When I exclude my first review year (2015) to avoid skewing the trendline, the data confirms a slight decline: my review length decreases by about 2.87 words per year. Still, with an overall average of 547 words, I would say my reviews remain fairly consistent (see Figure 4).
What has changed is the variance (and maybe in language, see Figure 5). I now write some very short reviews and some very long ones. In practice, this means I put more effort into a review when there is something substantial to say that can help the authors, while I keep it brief when the main issues are already clear. This also reflects my more frequent roles as Associate Editor or Associate Chair, where I often provide a high-level summary rather than a full, detailed review.
Looking back over 11 years of reviewing, I am happy to find that it seems I am not contributing to the reviewer crisis. This exercise was extremely valuable for me because it made the invisible work of reviewing visible. But it also reminded me how dependent our field is on the voluntary labor of many researchers. The reviewer crisis is real, and it will not be solved by any single person doing more. Still, reflecting on our own contributions might help us all recognize the importance of reviewing as part of our scholarly practice.
