Assuming you have given everything to write the best and most beautiful paper
you can ever create, it is obvious that the reviewers must see your points and
therefore write you a favorable review with a recommendation of strong accept.
Unfortunately, this is not always the case and reviewers may miss some points or
misunderstand some of your contributions.
Many conferences have therefore introduced a rebuttal phase that allows authors
to respond to the (initial) set of reviews. The rebuttal is an opportunity to
clarify misunderstandings, answer questions the reviewers may have, or to expand
on a given point the reviewers complained about. There are many different
forms of rebuttals with slight twists. Generally, a rebuttal allows you to
discuss and clarify certain aspects in a review but it is not intended to add
new material, so keep it short and focused.
Reviewing generally is not an adversarial setting and most reviewers are not
against you or against your research. Due to the increasing review burden, some
reviews may end up being on the short end or not as deep as you would have
wanted. The rebuttal is not the time to complain about such reviews. As
mentioned above, the rebuttal serves the purpose to clarify and to respond to
the reviews. If you must complain about the reviews themselves, consider taking
it up with the PC chairs.
Over time, I've settled on the following three step process to write rebuttals,
which helps me work through the reviews and to extract the points reviewers
raised. I encourage my students to always write rebuttals even if a conference
is not using a rebuttal process. Rebuttals allow you to digest reviews and to
reflect on your paper from the reviewer's point of view, hopefully identifying
the weaknesses and, if the paper is not accepted, improve the paper for the next
Read the reviews
Reading reviews is an art. It is incredibly difficult to read between the
lines. Try to identify what annoyed the reviewer: where did they stop paying
attention? What is, according to their view, the main issue with the paper? What
are the shortcomings? Additionally, try to figure out what they liked and what
they think the strength of the paper is. Great reviews also contain a section
that highlights the path to acceptance, i.e., what the reviewer thinks needs to
change to get the paper accepted. If no such section is present, try to identify
what would have helped swing the reviewer in your favor.
Reading reviews can be disturbing. You may ask yourself why reviewers did not
get a certain point as it was clearly discussed in the paper. After going
through the reviews, it is best to take some time off to digest the reviews,
allowing you to regain your objectivity.
Extract the main criticisms, group, and rank
Start marking the main criticisms in the paper. Pay attention to the topics
identified in the first phase and highlight them. Scribble over the
reviews to highlight individual comments. In this second phase your goal is to
identify the main topics that need to be addressed. Creating an outline of these
main points can be helpful. As you are working through the reviews again and
again, start grouping the comments of individual reviewers based on topics, and
then rank the topics according to importance. If multiple reviewers brought up
the same points it may be crucial to clarify that aspect.
An interesting question that often pops up is what aspects a rebuttal should
focus on. Should the ranking be purely technical, according to reviewer
expertise, or according to the review score? For example, is it better to
convince a non-expert weak accept to bump up their score or to clarify some
issues that an expert raised? I've heard many different approaches and each
approach has pros/cons. Also, having seen the process from the other side as a
reviewer, I cannot say if any given approach has advantages. In my rebuttals I
generally try to address the technical points, not focusing on individual
reviewers or experts too much. If an expert is strongly polarizing, it may be
worthwhile to highlight some misunderstanding or to keep the discussion of that
review short. But these issues quickly evolve into politics and may be for
people with more social skills.
The key issue you want to likely avoid is alienating reviewers. Keep sarcasm,
irony, and other subtle forms of communication out of your rebuttal and stick to
technical facts. Try to clarify technical items and write in a way that gives
reviewers a way out to adjust their scores for the better. I.e., instead of
writing "reviewer A is a moron who ignored our section 2.1 where we clearly
describe the design of our Flubb system" write something along the lines: "In
section 2.1 we describe how Flubb satisfies the Blubb assumption. We will
clarify these constraints based on reviewer A's feedback." If a reviewer takes
the time to note a certain point as part of their review then they felt that
this was an issue and it is the author's job to clarify that issue. The reviewer
is not wrong but may have been misguided by the paper. Improving your writing
will make it easier for the reviewer to digest your points.
Formulating a response
Now that you are clear about the major (perceived) weaknesses of your paper and
after you have identified the main topics that need clarification, it is time to
write the actual rebuttal. I like to write the rebuttal based on topics and then
highlight which reviewers have raised that topic. Note that at a top tier PC,
reviewers have 20-25 papers on their stack and reading 25 rebuttals can be
taxing, make it easy for them to identify which parts address their points. Also
important: stick to the word limit, many reviewers hate overlong rebuttals and
I've seen great rebuttals ignored if they were over (I've also seen rebuttals
that were 4x the allowed length). It is good tone to start the rebuttal by
thanking the reviewers for their reviews and to highlight any general issues
such as that you plan to open source your implementation or to give a quick one
sentence introduction into the main topic.
After the initial lead in you can dive into the individual points starting with
a quick introduction that summarizes the issue or question and an answer. Try to
keep the discussion short. You're not writing a new paper but are clarifying
some details. The rebuttal is not the place to introduce new topics but you may
mention that you have some additional results or to highlight certain
Generally, keep the tone polite. An aggressive rebuttal will rarely be read to
end and is not helpful in convincing the reviewers of your case. Snarky comments
or insults are not a good idea either.
When you submit the rebuttal, note that HotCRP sends out an email with the
rebuttal to all the reviewers. I've received several rebuttals that were heavily
modified and received a couple of updates. It is generally in your interest to
only submit the latest version, especially if earlier versions are not yet
Thanks to Nathan Burow for feedback on the article. I updated the discussion of
politics and rephrased the outline construction slightly.