The revision process nobody teaches you — how to write rebuttals, handle harsh feedback, and strategically improve your paper. Based on 7 weeks of research paper writing classes I taught at IIT Kharagpur.
You submit your paper. You wait weeks, sometimes months. Then the reviews arrive.
Your heart sinks.
Reviewer 2 calls your experimental design “fundamentally flawed.” Reviewer 3 says your contribution is “incremental.” The meta-reviewer wants a major revision.
What do you do?
I remember my first peer review experience vividly. It was during my MS at IIT Kharagpur. I was so personally wounded by the criticism that I wanted to give up on the paper entirely. My supervisor read the same reviews and said: “This is fixable. Most of this is fixable.” He was right — and the paper eventually got published at WebSci 2019 in Boston.
That experience shaped how I think about peer review. When I later taught a 7-week course on “Research Paper Writing” at IIT Kharagpur in mid-2024, I dedicated two full weeks to rebuttals and the revision process — because it’s a skill that’s rarely taught but absolutely essential. This article condenses those lessons along with what I’ve learned across my PhD, my time at L3S Research Center, and my current work at Stanford Medicine.
Table of Contents
- First, Take a Breath: The Emotional Reality of Peer Review
- Understanding the Reviewer’s Perspective (They’re Not Evil)
- Why First Impressions Matter Even After Submission
- How to Read Reviews Strategically
- The Rebuttal Process: A Step-by-Step Method
- A Template for Point-by-Point Responses
- When to Push Back (and How)
- When to Accept Criticism Gracefully
- Handling Contradictory Reviews
- Journal Revisions vs. Conference Rebuttals
- What to Do After a Rejection
- The Bigger Picture: How Rebuttals Connect to Writing Well
- Common Mistakes in Rebuttals
- Final Words
First, Take a Breath: The Emotional Reality of Peer Review
Let me be honest about something that research training rarely acknowledges: receiving critical reviews hurts.
You’ve spent months on this work. You’ve stayed up late debugging code, iterating on experiments, polishing every sentence. And now someone who spent maybe an hour reading it is telling you it’s not good enough.
Here’s what I’ve learned: don’t respond to reviews on the same day you receive them. Give yourself at least 24-48 hours. Read them once, then close the laptop. Come back with fresh eyes.
When you re-read them the next day, you’ll often find that the criticism is more specific and actionable than it first seemed. What felt like a personal attack often turns out to be a reasonable request for additional experiments or clearer writing.
Understanding the Reviewer’s Perspective (They’re Not Evil)
This is something I emphasized strongly in my research paper writing classes at IIT Kharagpur. The reviewer is:
- A human — with limited time and attention
- A busy professional — balancing hundreds of commitments
- Not evil — but may definitely be rushed (more on this below)
This matters because your paper needs to work within these constraints. Here’s how I explained it in my course:
Reviewers operate in two rounds. In the first round — which can look low-effort — they assess your abstract, the first paragraph of your introduction, your list of contributions, and whether you’ve linked your code. They glance at figures, tables, and their captions. This is enough to form a strong first impression.
Only when they’re satisfied with this first pass do they invest more effort and read in depth. If the first impression is weak, they start looking for reasons to recommend weak reject or reject.
This has a direct implication for rebuttals: if your paper’s first impression was poor, the reviewer may have been predisposed to be critical before they even reached your Methods section. Your rebuttal needs to overcome not just specific objections, but the overall impression.
Why First Impressions Matter Even After Submission
In my Week 1 lecture, I covered five common mistakes in research paper writing. Several of them directly impact how your paper is reviewed and, consequently, what your rebuttal needs to address:
Mistake: Lack of clarity and purpose. If the main contribution of the paper isn’t clear in your own mind, it won’t be clear to the reviewer. As I told my students: your key contribution should be clear enough to explain to a friend who isn’t in your field. If you can’t do that, rewrite your introduction before submitting — and before writing your rebuttal, re-examine whether the contribution is really as clear as you think.
Mistake: Ignoring first impressions. Your paper should have one or two key takeaways, illustrated with examples. Main results should be evident from just the figures, tables, and their captions. If reviewers struggled to find your key message, your rebuttal should open by stating it clearly.
Mistake: Lack of documentation during the research process. This is the invisible mistake that bites you during rebuttals. If you didn’t keep proper documentation or logs during your research project cycle, you won’t have the data you need to respond to reviewer requests for additional analysis. I always advise students to maintain short paper summaries of every paper they read, justifications for evaluation setup choices, result logs with general trends and conclusions, and TODO items. When a reviewer asks “why did you choose this baseline?” — your logs should have the answer.
How to Read Reviews Strategically
When you first read your reviews, create a structured document. For each review, extract:
The overall sentiment: Positive, mixed, or negative? Look at the recommendation and the confidence score. A “weak reject with low confidence” is very different from a “strong reject with high confidence.”
Explicit requests: What specific things does the reviewer ask for? More experiments, better baselines, clearer writing, additional analysis? Make a list.
Implicit concerns: Sometimes reviewers don’t directly state what bothers them. “The experimental section is thin” might mean they want more baselines, ablation studies, or significance tests. Read between the lines.
Factual errors by the reviewer: Note these carefully — you’ll need to address them diplomatically.
The fatal vs. fixable distinction: Some concerns are fixable in a revision (add an experiment, rewrite a section). Others may be fundamental. Identify which is which.
The Rebuttal Process: A Step-by-Step Method
In Week 6 of my research paper writing course, I presented a systematic rebuttal process inspired by Devi Parikh’s excellent guide. I’ve since refined it based on my own experience across multiple submissions. Here are the four steps:
Step 1: Itemize reviewer comments.
Use a spreadsheet to organize individual comments, questions, and concerns raised by each reviewer. I use columns for: Reviewer, Comment Number, Comment Text, Category (major/minor), Author 1 Notes, Author 2 Notes, Status.
Laying everything out in the same place helps identify common concerns across reviewers and keeps you from missing anything. Do this as soon as possible — it helps you identify any necessary new experiments or analyses early, while you still have time.
Step 2: Brain dump possible responses.
The spreadsheet has a column for each author to reply to each reviewer comment. Dump your thoughts here in rough text without worrying about style or length. Being convincing and concise is a subtractive process — start with everything, then trim.
Step 3: Write a draft rebuttal.
Turn your consensus from the spreadsheet into concrete responses in a rebuttal draft. Write concisely but don’t worry about space yet. Cover every point — you’ll prioritize and trim later.
Step 4: Review and revise.
Reread the initial reviews and your spreadsheet to make sure everything has been addressed. Prioritize major concerns and start working toward meeting space limitations.
Why the spreadsheet matters: The target audience for your rebuttal includes two different groups with different needs:
For the reviewers — who have read your paper to varying degrees — your job is to clarify doubts, answer questions, correct misunderstandings, push back on mischaracterizations, and make a good-faith effort to incorporate feedback.
For the Area Chair (AC) — who is likely even less familiar with your work — a good guiding principle is to assume that all they will read is the set of reviews and the rebuttal. Your job is to convince them you’ve made a good-faith effort, present a representative summary of the reviews, help them understand whether concerns were addressed, call out bad-faith reviewing, and help them make a decision.
A Template for Point-by-Point Responses
Here’s the template I use and teach:
Reviewer Comment [R1-C1]: “The authors should compare against [Specific Baseline Model].”
Response: We thank the reviewer for this suggestion. We have added a comparison with [Baseline Model] in Table 2 (Section 4.2). Our approach outperforms [Baseline Model] by X% on [metric], which we attribute to [brief explanation]. We have also added a discussion of this comparison in Section 5.
Reviewer Comment [R1-C2]: “The motivation for using [Method X] over [Method Y] is unclear.”
Response: We appreciate this feedback. We have expanded Section 3.1 to clarify our choice. Specifically, [Method Y] requires [specific limitation], which makes it unsuitable for our clinical dataset because [reason]. We have also added a footnote comparing computational costs.
Reviewer Comment [R2-C3]: “The paper claims state-of-the-art results, but the comparison is limited.”
Response: We acknowledge this concern and have significantly expanded our experimental comparison. We now include results from five additional baselines (Table 2), including the recent work by [Author et al., 2024]. We have revised our claims to be more precise — our method achieves state-of-the-art on [specific metric/dataset] rather than broadly.
Key principles:
- Start with acknowledgment, never defense
- Be specific about what changed and where
- If you added something, point to the exact location in the paper
When to Push Back (and How)
Sometimes reviewers are wrong. They misunderstand your method, cite irrelevant criticism, or make factually incorrect claims. You should push back — but carefully.
Do: Present evidence calmly. “We respectfully note that [specific point]. As stated in Section 3.2, [clarification]. We have added additional clarification in the revised manuscript.”
Don’t: Be combative. “The reviewer clearly didn’t read our paper carefully” will never help you, even if it’s true.
Do: Redirect constructively. If a reviewer asks for an experiment that doesn’t make sense, explain why and offer an alternative that addresses the underlying concern.
Don’t: Ignore concerns you disagree with. Even invalid concerns need to be addressed. Saying nothing reads as avoidance.
When to Accept Criticism Gracefully
Not all criticism is unfair. Some of it genuinely improves your paper.
Signs you should accept and improve:
- Multiple reviewers raise the same concern (almost always legitimate)
- The concern is about missing experiments that are feasible to run
- The concern is about unclear writing (always fixable)
- The suggested approach would genuinely strengthen the paper
My personal rule: If I can address a concern with less than a week of additional work, I always do it, even if I think my original approach was defensible. The revised paper will be better, and the reviewer will feel heard.
Handling Contradictory Reviews
Reviewer 1 says your paper has too much technical detail. Reviewer 3 says there’s not enough.
How to handle it:
- Acknowledge the contradiction in your response
- Lean toward the meta-reviewer’s guidance if provided
- Make targeted changes: Contradictory reviews can often be resolved by restructuring. Move detailed derivations to an appendix (satisfying R1) while adding a summary table in the main text (satisfying R3).
Journal Revisions vs. Conference Rebuttals
These are fundamentally different processes:
Journal revisions give you time to actually change the paper. You submit a revised manuscript plus a detailed response. You can run new experiments, add sections, and substantially improve the work.
Conference rebuttals are typically limited to 500-1,000 words. You cannot change the paper — you can only respond to concerns. Focus on the most critical concerns, especially factual misunderstandings. Lead with the strongest response.
For conference rebuttals, prioritize:
- Correcting factual misunderstandings by the reviewer
- Addressing concerns about correctness over concerns about presentation
- Offering to add requested experiments in the final version if accepted
- Conciseness — every word counts
What to Do After a Rejection
Rejection is part of research. Even the best researchers get rejected regularly.
If your paper is rejected:
- Re-read the reviews after a few days. Extract the actionable feedback.
- Decide whether to revise or pivot. If the core contribution is sound but execution was weak, revise and resubmit. If reviewers fundamentally disagree with your approach, consider whether they have a point.
- Don’t submit unchanged. Reviewers at different venues may overlap, and submitting an unchanged paper is bad practice.
- Consider the target venue. Maybe your paper was a poor fit for a top-tier conference but would be well-received at a strong workshop or a domain-specific journal.
- Talk to your advisor or a trusted colleague. An outside perspective helps.
For the right venues, check my conference deadlines article and my article on publishing without a conference.
The Bigger Picture: How Rebuttals Connect to Writing Well
In my research paper writing course at IIT Kharagpur, I covered seven topics across seven weeks:
- Common mistakes to avoid
- Practical case studies (comparing different versions of an ECAI 2024 submission)
- How to write a great abstract
- The importance of focus and “theatrical cuts”
- Making effective figures
- Writing rebuttals (Part 1)
- Writing rebuttals (Part 2) and obtaining feedback from co-authors
The reason I teach rebuttals alongside writing — not separately — is that they’re deeply connected. Many rebuttal problems are actually writing problems. If the reviewer doesn’t understand your contribution, the fix isn’t a better rebuttal — it’s a better introduction. If the reviewer can’t find your main result, the fix isn’t explaining it in the response — it’s making it evident from your figures, tables, and captions.
The “theatrical cut” concept from Week 4 is especially relevant: your paper is not a dump of all your results. It’s a curated story — a theatrical release. Everything that doesn’t support your main point should be removed or moved to supplementary material. Clutter draws attention away from your point and leaves the reader exhausted. If reviewers complain about unfocused writing, the theatrical cut is your fix.
Obtaining feedback from co-authors (Week 7) is also critical for rebuttals. Prepare the first draft of your rebuttal early — your co-authors are busy, and giving them time to respond improves the quality of the response. Use meetings to discuss strategy, not just edits. Ask specific questions: “Does our main result seem convincing enough? What’s the strongest objection a reviewer might raise?”
Common Mistakes in Rebuttals
Mistake 1: Being defensive. The rebuttal is not a debate. It’s a demonstration that you can improve the paper.
Mistake 2: Promising changes you can’t deliver. If you promise to add an experiment in the final version, you must actually add it.
Mistake 3: Ignoring “minor” comments. Reviewers sometimes embed important concerns in their “minor comments” section. Address everything.
Mistake 4: Writing too much. For conference rebuttals especially, conciseness is key. Reviewers won’t read a 5-page response.
Mistake 5: Not formatting the response clearly. Use clear numbering (R1-C1, R1-C2), bold the reviewer’s comments, and use a different style for your responses.
Mistake 6: Delaying the first draft. As I teach in my paper writing course, the first draft should be prepared quickly — dump all raw materials from your notes and logs (Stage 1, 1-2 days), then iteratively refine (Stage 2, ongoing until deadline). The same applies to rebuttals: dump your responses into the spreadsheet immediately, then refine.
Final Words
The peer review process is imperfect. We all know it. As Bill Freeman noted at the “Good Citizen of CVPR” panel, it’s more like a large crowded marketplace with limited time, attention spans, partial information, and people juggling multiple responsibilities.
But learning to navigate it skillfully is one of the most important research skills you can develop. The best researchers I’ve worked with — from my PhD advisor at IIT Kharagpur to my collaborators at L3S and Stanford — all share one trait: they treat reviews as data, not as verdicts.
If you’re at the beginning of your research career, I promise you: your response to reviews will improve dramatically with experience. Every revision makes you a better writer, a more rigorous scientist, and a more resilient researcher.
Related articles that may be of interest to you
- A Self-Help Guide to Starting Your Own ML Research Project
- Writing Research Papers in LaTeX — Complete Guide
- Can You Publish Without a Conference?
- ML & AI Conference Deadlines 2025–2026
- The Side-Hustle Scientist: Publish AI Papers While Working Full-Time
- How to Prepare a Research Presentation
Lecture Referenced
Discover more from Medical AI Insights
Subscribe to get the latest posts sent to your email.