Planning Next Steps When Things Don’t Work Out

---

Table of Contents

---

Planning Next Steps When Things Don’t Work Out

In the previous lesson of this series, we learned how to plan and write a research paper. Because of the very nature of research, it can become as frustrating as it is exciting when things don’t seem to work out. Despite its nature, there will always be frustrating periods when you feel like quitting and starting something new.

You can feel frustrated when progress is slow, experiments don’t go as planned, you start having doubts about the work, or the work has been rejected (perhaps multiple times).

Regardless of the reason, it is essential to understand that frustration is normal and very common in research. It is more important to deal with it properly, not let it affect your self-confidence and hours of research work.

This blog post will discuss coping with frustrations and planning your next steps when things don’t work out.

In this series, you will learn how to publish novel research.

This lesson is the 4th in a 5-part series on How to Publish Novel Research:

1. Choosing the Research Topic and Reading Its Literature
2. Ideating the Solution and Planning Experiments
3. Planning and Writing a Research Paper
4. Planning Next Steps When Things Don’t Work Out (this tutorial)
5. Ensuring Your Research Stays Visible and General Tips

To learn how to plan and write a research paper, just keep reading.

---

Planning Next Steps When Things Don’t Work Out

---

Research Frustrations

Research and frustrations go hand in hand. While pursuing research, we often don’t exactly know what we are looking for, which can become frustrating. As Albert Einstein (Figure 1) said, “If we knew what we were doing, it would not be called research, would it?” Hence, you should re-motivate yourself by saying why you got into the research at first and that these frustrations are common in the research community.

Frustrations can arise because of several reasons, and it is important to identify them before constructively dealing with them. You can get frustrated when you are stuck thinking of a solution or the experiments are not going as expected for a long period. This can put you in doubt about whether your research will be successful and if there are other better opportunities that you are missing.

At times, some outsiders may criticize your work, leading you to think about whether it will be meaningful or not. You may also start realizing that you are not spending enough time with your family and are constantly thinking about your research even before sleep and on weekends. Once you have calmed yourself and identified your reasons, plan your next steps.

---

Making a Research Diary

Making a research diary (Figure 2) provides an overall picture of what you have tried, what did or did not work, and what else you can take from literature to inspire and improve your work. Organize all important related results, the ideas, and the experiments you have tried so far. It can be virtual or on paper, depending on what suits your style.

Summarize the motivation, contributions, and main idea of important literature. Mention your thoughts on how your research can be improved and inspired by those works. Next, describe all the ideas you have tried so far. Write the novel aspects about it, and add a rough figure that illustrates the thought and the results of the experiments you have done.

Refer to this blog on how to keep a research diary.

---

Stick with Your Research

Research needs a different mindset that does not expect any profits. Hence don’t leave your research without giving it a fair chance. Stick with your experiments and believe that you will get your expected results sooner or later. If you feel as if you are being struck indefinitely, start reading something else. Read papers that are recent, exciting, and yet unrelated to your current research. Doing so will help you get your mind off the project for some time and will most likely spark new ideas in you.

---

Keep the Discussion Going

Keep in touch with your mentors and advisors and regularly discuss your progress (Figure 3). You can share your research diary with them so that it helps them track what you have accomplished. They can guide you properly by advising you on getting through your research study and will perhaps share some alternative ways of thinking and carrying out your research.

---

Retrospection

Take a break for a few days from thinking and experimenting and instead look back at your implementation for alternatives to see if you could have done anything differently. Have a closer look at your codebase and identify any silly mistakes ruining your efforts.

Visualize the results using graphs, attention maps, or loss landscapes and identify the root cause of things not working out. Consider how they can be mitigated via implementation, additional loss functions, or modules if you have identified any.

---

Paper Rejections

Whenever we submit a work in an academic venue, we hope for the best. However, because of the selective nature of academic venues, our papers will have more chances of getting rejected. As shown in Figure 4, Academic venues (conferences and journals) usually have a 20-25% low acceptance rate. Statistically, it will take around 10-11 re-submissions to have a 95% chance of acceptance. Thus, one should understand that rejection is part of the academic process and should not take it negatively if they have multiple rejections.

Rejections can be harsh, disappointing, and sobering, especially for young and new researchers. The important thing to keep in mind is not to make any impulsive decisions based on the reviews. For example, don’t complain to program chairs about the awful reviews. Since the decision has already been made, there is little chance that your complaint will have any impact unless there is a clerical error (where the paper is rejected even when reviews say accept).

Avoid publishing the reviews online. Doing so can tarnish the reputation of you and your co-authors. Additionally, publishing reviews online can violate the copyright of reviews as they didn’t consent for their words to be made public. Some conferences even have explicit rules about posting reviews online that can jeopardize your research career.

Don’t write an angry rebuttal to prove your point and criticize the reviewer. This will make things only worse even when you are right. It is entirely reasonable to get disappointed and stow yourself away for a couple of days. Instead, it is recommended that you take your time to process and heal. When you are over with that, revisit your work, read the reviews, and start to act as follows.

---

Understand the Reason for Rejection

Rejections can happen either because

• your approach is not argued and appropriately explained
• results are insignificant
• your experiments section is weak as in you did not follow standard practices or more experiments could have been done
• the idea did not sound original and novel to the reviewers
• your motivation is not clear
• reviewers had a misunderstanding
• the paper is not well written and presented

Figure 5 reviews the reasons for rejection for the benefit of aspiring authors and readers.

It is also important to understand the rejection from the viewpoint of the reviewer and area chairs. Among the pile of papers assigned to an area chair, one-third are apparent rejects. In the whole set, maybe one or two are well written (oral or spotlight) and have nice ideas and results. The rest are borderlines.

With a 75-80% rejection rate, the question is, “How can I reject the borderline paper?” There are two types of borderline papers for an area chair (Figure 6).

Cockroach: There is no flaw that you can identify to kill the paper. The paper is well written. However, the reviews are okayish, results have shown incremental improvement, etc. A typical average poster paper.

Puppy with six toes: A delightful paper has flaws that are easy to point out. Even though the flaws are not that important, it makes it easy for the area chair to reject.

---

Reading the Reviews

Read the reviews and meta-reviews thoroughly and identify the major and minor reasons for the rejection. Major concerns usually include insufficient experimentation, insignificant results, limited originality and novelty of approach, etc., which need to be addressed. Minor comments can include writing, clarity, and presentation style issues which can be handled while revising or re-iterating the paper.

Reviewers can sometimes misunderstand things or intentionally provide irrational comments to reject the work. Hence, don’t blindly believe all of the reviews. Rather look beyond the words to determine if each comment is reasonable or irrational.

---

Deciding Where to Submit Next

Now that you have an idea of how reviewers perceive your work, you should start looking for alternate venues where you think the revised work has a chance. You need to look for those alternatives which provide you with enough time to revise the paper and resubmit in time. There are four alternative venues where you can decide to resubmit your work.

Conference: Conferences offer a fast way of publishing your work. They provide you with immediate feedback within 1-2 months and can take a couple more months to decide on your paper. However, the reviewing can become sloppier because of the time constraint. You can look at aideadlines to see upcoming conferences and decide where to submit next. Consider conferences where the deadline is sufficiently close so that you won’t have to wait too long for the acceptance.

Journal: Journals involve a continuous dialog with reviewers and editors to improve your work. There are journals like Nature, Science, TPAMI, and IJCV, which are considered as competitive as top tier conferences like CVPR, NeurIPS, etc. However, they have a long turnaround time, with the complete process (from submission to acceptance) taking as long as two years. Check if your paper is apt for a journal.

Usually, papers which are complex, present theoretical models, literature surveys, or are too long to be conveniently discussed at conferences are more suitable for journals. Sometimes journals announce special editions which take submissions pertaining to certain research fields. These are usually faster compared to standard journal submissions.

Figure 7 briefly differentiates conferences and journals.

Workshops: Workshops have relatively higher acceptance rates and are suitable for works where novelty is limited or content is not verbose enough to utilize the 8- to 10-page limit for conferences and journals. You can browse conference websites for the list of workshops and can decide which one suits you best.

Note that many workshops are archival and don’t have their proceedings. You might want to look for this information if you are looking to publish your work in proceedings.

ArXiv: If you can’t find a suitable venue for your work, consider uploading it on public forums like ArXiv and look for better alternatives. It might also be suitable for outdated works and have a meager chance of getting through the peer-review process.

---

How to Improve

Here are a few suggestions for improving your work for your next submission (Figure 8). The gist is to carefully address reviewers’ comments so that they can be avoided in the next round of submission.

Obtaining additional results: Reviewers will likely suggest additional experiments and analyses that might be needed to access the work better. Make a list of all new experiments and ablations you need to run. Try to include those results in the main manuscript rather than supplementary (as reviewers often skip it). If it is not possible due to space constraints, add the results in the supplementary and refer to them explicitly in the main manuscript. This ensures the reviewers that you have done such an experiment.

Another important thing to look at is whether any new works have been published in the duration (time from the first submission to the second submission). Ensure that you compare your approach with these latest approaches or discuss them in related work (if not planning to compare).

Invite more collaborators: Having more collaborators can help you address the reviewers’ comments and shape the paper accordingly.

Presentation style issues and misunderstandings: Note if any misunderstandings could have impacted the reviewer’s decision. While writing a rebuttal, don’t completely blame the reviewer for misunderstanding the work. Instead, own that the misunderstanding could be because of your writing and presentation style. Add additional visuals and text to clarify it upfront. For presentation style issues, ensure that you run a grammar and spell check and correctly refer to equations, tables, and figures.

AJE Scholar provides a checklist for revising and resubmitting a research paper for publication.

---

Summary

Because of the very nature of research, there will be times when things won’t work out.

You might get frustrated if you are stuck in your research for a long period or get slobbered if the work is rejected multiple times. It is reasonable to stow yourself away and take your time to heal and process.

At times, you will feel like quitting, but you should not let these negative feelings affect your hours of research work. Hence, it is important to properly deal with frustrations and rejections and plan your next steps.

Before planning your next steps, it is important to identify the reason for your frustration. For example, frustration can arise when you are stuck for a long time and start having doubts about whether the research will be successful.

Keeping and maintaining a research diary can provide you with a picture of what went well, what you have tried, and what else can be done. You can share the same with your collaborators and advisors so that they can track your progress regularly and suggest alternative ways of approaching the problem whenever you are struck.

Sometimes reading papers that are recent, exciting, and yet unrelated to your current research can help you get your mind off the project for some time and will most likely spark new ideas in you.

Paper rejections can be harsh and sobering, especially for young researchers. However, one should understand that rejection is part of the academic process and should not take it negatively if they have multiple rejections. Don’t complain to area chairs or post the reviews online on impulse. Rather calm yourself and go through the reviews thoroughly to identify the major and minor reasons for rejection.

Decide on possible venues where you can submit work. You can look for upcoming conferences and workshops or can decide to submit to a journal. Ensure that the deadline is sufficiently close so that you won’t have to wait for too long for the acceptance. If you don’t find an adequate venue or your work is outdated and has little chance to go through the peer-review process, consider uploading it to arXiv.

Revise and improve your work by running additional experiments, comparing with the latest state-of-the-art methods, clarifying any misunderstandings, and working on any presentation and writing style issues. I hope this lesson will assist you in planning your next steps when things don’t work out. Stay tuned for the next lesson on ensuring your research stays visible and general tips.

---

Citation Information

Mangla, P. “Planning Next Steps When Things Don’t Work Out,” PyImageSearch, P. Chugh, R. Raha, K. Kudriavtseva, and S. Huot, eds., 2022, https://pyimg.co/hvxgp

@incollection{Mangla_2022_Planning_Next_Steps,
  author = {Puneet Mangla},
  title = {Planning Next Steps When Things Don’t Work Out},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Ritwik Raha and Kseniia Kudriavtseva and Susan Huot},
  year = {2022},
  note = {https://pyimg.co/hvxgp},
}

---

---