The most important considerations for an author when submitting an article to a journal are the journal’s impact factor, the indexes in which the journal is listed, whether or not there is a publication charge, and of course, the speed of its evaluation process (1, 2, 3).
To understand the speed and the results of a journal’s evaluation process, it is necessary to know and understand the background work done by the journal’s editorial team (4). From an editorial standpoint, the objective, timely, and high-quality evaluation of a manuscript is the work of a large network that begins with the author and involves the editor, the associate editors, and the reviewers (5).
As the editorial board of the Turkish Archives of Otorhinolaryngology (TAO), we would like to share with authors, reviewers, and readers the editorial work we do in the background and give some details about the peer-review process.
Between 2015-2019, 646 manuscripts were submitted to TAO. When the manuscripts are analyzed according to their subspecialties, 176 (27.24%) were about rhinology-facial plastic surgery and sleep disorders, 169 (26.16%) about head and neck surgery, 156 (24.15%) about otology, 74 (11.46%) about pediatric otorhinolaryngology, 26 (4.02%) about laryngology, and 45 (6.97%) were classified as other.
In the initial editorial review of the 646 manuscripts, the editorial board had rejected 265, and 381 were assigned to reviewers for the peer review process. A total of 2,122 reviewers were assigned to 381 manuscripts (5.57 per manuscript). Table 1 shows the responses of the reviewers to invitations.
Reviewers evaluated 381 manuscripts, of which 154 were rejected after the initial review, 196 were returned for major revision, 20 were returned for minor revision, and 11 were accepted for publication as-is. After all revisions, 207 manuscripts were accepted for publication.
The first step of a paper is pre-editorial evaluation. If the manuscript is found to be written in line with the guidelines of TAO, then it is sent to the Editor. After editorial evaluation, the editor either rejects the manuscript or sends it to one of the associate editors. The associate editors also evaluating the manuscript, can either directly reject or send it to the reviewers according to the subject of the manuscript and the field of expertise of the reviewer.
In 2015 to 2019, the pre-editorial time after submission of a manuscript to the journal was 18.0±45.29 days, and the time to appointment of an associate editors was 1.5±3.01 days after editorial evaluation. Thus, it can be seen that the pre-editorial period brought about an average time loss of 18.5 days for the authors. Pre-editorial time was mostly associated with the manuscript’s inconsistency with the TAO’s writing and uploading guidelines or with carelessly written content. Accordingly, authors can shorten the evaluation process by 18 days by ensuring that the manuscript is prepared according to the guidelines of the journal and uploaded correctly.
The associate editor then selects two to four reviewers and assigns the manuscript for review. The average response time of the reviewers who had accepted the review was 2.98±2.58 days and their average review time was 5.81±3.93 days. Once invitations to review are sent to candidate reviewers, the associate editors allows a response time. When the evaluation period of the reviewers who did not respond is accepted as 21 days, the average evaluation time of the reviewers who completed the review was 13.46±4.38 days. The total mean evaluation time for a manuscript after submission was 55.88±65.38 days. The time from submission to the final decision was 37.38±48.19 days. The appointment of the reviewers is critical in the manuscript evaluation process. The editorial team expects each candidate reviewer to respond to their invitation, whether positive or negative. Reviewers who do not respond to the invitation, and particularly those who do not review the manuscript despite having agreed to do so, cause the journal's editorial service to be lengthened and delayed. The response and review completion data of the assigned reviewers are detailed in Table 1.
It can be seen that the most important issue in the manuscript evaluation time in terms of reviewers is the reviewers who do not respond. Those who accept or decline the review assignment have no impact on the duration of the review time.
To determine the features of completed reviews, we evaluated the character and word counts of each reviewer report. The average word count was 101.705±88.033, and the average character count was 610.997±515.796. The average word count for accepted papers, major revisions, minor revisions and rejected papers were different. The reviewers wrote shorter reports on the articles they accepted, and longer reports on the articles they rejected or for which they suggested major revision (Table 2).
For the purposes of this study, we also developed a scoring system for monitoring the progress of the manuscripts. In this respect, we calculated a decision score for each paper. According to this scoring system, each reviewer’s decision was scored as
• Accept = 3 points,
• Minor revision = 2 points,
• Major revision = 1 point,
• Reject = 0 point,
then the total score was divided by the number of reviewers. To put it briefly, decision score of manuscript = total of scores from all reviewers / total number of reviewers. This scoring system was used both for first evaluations and for revisions.
In the first decision, the totally mean decision score was 1.371±0.756. The decision score for accepted manuscripts was 2.538±0.598. It was 2.075±0.574, 1.641±0.569 and 0.844±0.616 for minor revisions, major revisions, and rejected manuscripts, respectively. The manuscripts requiring minor revision increased their decision score after revision from 2.075±0.574 to 2.342±0.543. Those requiring major revision increased their decision score after revision from 1.641±0.569 to 2.176±0.622. The overall decision score for manuscripts requiring both minor and major revisions increased from 1.685±0.576 to 2.172±0.614 after revision.
To sum up, the editorial process can be shortened with more attention paid to certain points. Particularly, it is critical that authors should prepare their manuscripts in accordance with the instructions to authors guidelines, the editors should act fast and unbiased, and finally, reviewers should provide timely, sufficient and reliable evaluations.
All contributors of the journal are responsible for shortening the evaluation process. While it is very important for the authors to prepare their manuscripts in accordance with the journal’s writing guidelines, it is crucial that reviewers promptly respond to invitations and complete evaluations within the given timeframe once they accept the invitation. Editors and associate editors, on the other hand, can shorten the evaluation processes by prioritizing those reviewers who provide positive and fast response to invitations.
We hope that all stakeholders of the journal, including members of editorial board, authors, reviewers, and readers will benefit from this paper aimed to present a detailed account of TAO’s review processes.