Have you seen this review?

We recently submitted a structured literature review to a well-ranked journal, and got a review back complaining about how badly our controlled experiment had been carried out. We inquired with the editor about this, but got no answer back. The review (by reviewer 2, no less) is so generic, I suspect it has been used many times before. I’d be curious to hear from you if you received this (same) review in the past.

Reviewer: 2

Recommendation:

Reject (do not encourage resubmit)

Comments:

The authors need to better explain the context of this research, including why the research problem is important.

Contributions should be highlighted more. It should be made clear what is novel and how it addresses the limitations of prior work.

==== RELATED WORK ====

The authors should explain clearly what the differences are between the prior work and the solution presented in this paper.

The authors should add a table that compares the key characteristics of prior work to highlight their differences and limitations. The authors may also consider adding a line in the table to describe the proposed solution.

==== PROBLEM DEFINITION ====

Authors should give a clear formal definition of the problem.

==== METHOD ====

A novel solution is presented but it is important to better explain the design decisions (e.g. why the solution is designed like that)

It is important to clearly explain what is new and what is not in the proposed solution. If some parts are identical, they should be appropriately cited and differences should be highlighted.

The description of the proposed solution should be more formal.

==== EXPERIMENTS ====

The experiments should be updated to include some comparison with newer studies.

A statistical analysis should be carried out to demonstrate that the experimental results are significant.

There is not enough discussion of the experimental results.

The experiments have been carried with a few datasets. It is necessary to add more datasets so as to make experiments more convincing.

Some experiment(s) should be added to show that the proposed solution can be used in real applications.

Some additional experiments are required:

• Scalability
• Runtime
• Memory
• Sensitivity analysis