Peer-Review Mode survey: Results

The survey in numbers (not finished yet)

• N = 25 (26 % of the mantle community)

​

​

• Geographic distribution

​

Statement 1: We should adopt an interactive peer-review process (EGU Solid Earth style)

Pro comments:

• Added value of this model requires an engaged community to comment on the preprint alongside the assigned referees.
• Phase approach: interactive peer-review could be implemented once the journal establish itself.
• This feels most in line with the spirit of Diamond Open Access.
• Community reviews help to improve papers
• Might force more objectivity and avoid conflict of interest.

Con comments:

• Has benefits, but the peer-review process is quite long compared to the classic peer-review process. This might be an obstacle for many ECR.
• Although I appreciate the interactivity of Solid Earth, it is a feature that nobody really uses.
• Not necessarily a good thing for everyone to weigh in. If we trust our editors and reviewers, they shouldn't need to.
• Some may feel uncomfortable publicly criticizing the work of well-known scientists.

Suggestions:

• Encourage authors to pre-print their work (e.g., EarthArXiv) and encourage the community to engage and comment in parallel to the traditional peer-review process.
• Not necessarily made in the exact same way as the EGU-SE, but interactive and open.

Statement 2: We should adopt a classic peer-review process

Pro comments:

• Manuscripts are improved by reviewer commentary before being seen by the public.
• Would attract more authors as they are more familiar with this mode.
• No major issues with the classical process, if all the other aspects of publishing are solved.
• Simpler, more straight-forward, more efficient.

​

Con comments:

• Classic style peer-review process lacks review of the scientific community.
• The classic review process has worked well in general, but we should try something new.

​

Statement 3: Peer-review reports should be published once the paper is published

Pro comments:

• Keeps reviewers more honest (rigorous) and kind (professional).
• This may be useful in providing a bit further incentive for reviews to be constructive.
• Championing diamond open access implies making the whole peer review process transparent.

Con comments:

• As an author, I'm not sure I want my mistakes (or what could be perceived as my mistakes) on display. Choice should be given to authors.
• The peer-review reports are based on the reviewed paper and the authors ideas, they do not belong to the reviewers to be published separately.
• If it's a classic review system, there is no merit in publishing the previously confidential reports when the issues raised have been addressed.

Suggestions:

• Make it optional.
• Some moderation by the editor might be good.

Statement 4: The peer-review reports should be published even if the paper is rejected

Pro comments:

• Since most controversial (i.e., misguided) review comments might lead to “reject” decisions, it is worth thinking about making everything transparent.
• If a rejection is fair, nobody should fear this, right?

Con comments:

• Adds insult to injury by publishing all the things wrong with a rejected paper.
• May cause bias if the authors submit a revised version elsewhere. Why pour salt in the wound of rejection?
• Where would these go? Requires hosting rejected papers.

Statement 5:  Reviewers can choose to either be anonymous or disclose their identity

Comments:

• Anonymous if rejected, but not if paper is published.
• Requiring disclosure would help temper the worst of negative anonymous reviews.
• There should be an editorial suggestion to disclose their identity.
• Mutual accountability leads to more fair review process. But requiring people to disclose identity would definitely bias [narrow?] the reviewer pool.
• Reviewers should be identified as part of open science.
• Reviewers should only be anonymous if authors are.
• Either everyone involved (including authors) should be anonymous or no one gets to be anonymous.
• Reviewers should be willing to be identified and stand by their comments, or else they shouldn't make them.

Statement 6: Authors can request reviewers that are willing to share their identity

Pro comments:

• This may be a good option to not enforce either anonymous and non anonymous reviews; this will help bring together authors and reviewers that share the same values on the peer review process, and thus, will both be happy with the outcomes.
• This seems fairer than the option to not have a certain reviewer.
• Only if the reviewers are also intitled to deny this request.

Con comments:

• it is hard to get reviewers sometimes, can't be that picky and it might bias against ECR agreeing to review (or other people vulnerable to retaliation).
• it should be the journal who decides if the reviews can or cannot be anonymous. And it is the editor who should decide if a review has been fair or not.
• It could lead to some difficult situations with authors wanting reviewers to name themselves so they can pressure them (even if a decision has been made).
• Impossible for reviewers to know whether they would like to be anonymous or not before reading the manuscript.

Statement 7: Authors can choose double-blind review

Pro comments:

• Not convinced whether double blind review works, but if the authors have a reason to believe this, we could let them try it. Perhaps no one will request it, but it’s at least nice to offer the option.
• I'd love to see this, even as a trial period and then re-evaluated.
• Equalizes the author/reviewer relationship if identification of reviewers is not required.
• If the subject area is known to have conflicts this may help mitigate evil reviewing.
• Double blind review is best. No bias by author name/affiliation

Con comments:

• Does not make much sense given our smallish tectonics community.
• There are many cases where double-blind reviews did not work for improved equity and diversity.

Statement 8: “Structured review form” in addition to the free-form review report

Pro comments:

• Forces reviewers to discretize their comments.
• Structured review would greatly help new reviewers and those who want to review more but have limited time.
• It is a good way to make sure all aspects are covered but is also an extra hassle for reviewers.
• Way more objective to answer these surveys than to just waffle on about reviewer feelings on the paper.
• Provides a standardized review across the journal. No ratings out of 100 though!
• This will ensure clarity in the title/abstract and will hopefully mean key points will be picked up when searching online.

Con comments:

• Such forms can be frustrating when no option seems appropriate, and you still must choose one.
• It's impossible (and undesirable) to add enough varied options to anticipate every possibility.
• Adds to the reviewer’s workload. Seems a waste of time to me.
• The nuance of a review is in the written comments. Forms, regardless of how exhaustive, will fail to catch all important points.

Suggestions:

• Maybe a form is good, but with an "other (explain)" option for some questions?
• The form should not be as strict as e.g., in Frontiers. A checklist works fine.

Statement 9: As a DOA journal, Tektonika should make pre-printing mandatory

Pro comments:

• We are diamond OA, right?
• An open review has less value if the manuscript isn't also "open"

Con comments:

• This will discourage many to publish in Tektonika, particularly ECRs.
• As an author, I hate to put out unfinished (e.g., unreviewed) work out there, as I want people to read the finished, polished, and accepted version of some papers.
• The advantage to pre-printing is to make a version available if the final is not open-access but with a DOA journal, that benefit is not important.
• What happens the article is rejected?
• This will discourage many form submitting to Tektonika.

Suggestions:

• Not mandatory but encouraged.
• Not necessary before submission, however before e.g., going to review, or before the review process is completed.
• In partnership with EarthArXiv.
• There may be some situations when this is not appropriate. Authors should be given the option to specify this to the editor during the submission.
• It should be part of the submissions process and Tektonika should provide that preprint service, rather than obliging the authors to do it elsewhere.

​

Statement 10: Tektonika should allow post-publication comments and discussion

Pro comments:

• I think this is a great idea and would help set us apart.
• Absolutely. This would be the equivalent of the more formal "comments and reply" short papers that many journals have featured in the past. Comments and discussion in the community are good - but as I said earlier, not necessarily during the review process itself.
• As easy as having hypothes.is on them
• It can be great, and mimics the vibe of a conference presentation, so long as discussions are moderated.

Con comments:

• This is available through social media and other websites. Not sure replicating functionality on Tektonika itself is worthwhile. Move the conversation to existing platforms.

Suggestions:

• Is this something OJS can handle? How would we moderate these forums and keep out spam?
• This is a nice idea but would require additional work to moderate. If we have the community bandwidth, I say go for it. Should the people allowed to comment be required to have a Tektonika website account (keep out the anonymous internet trolls)? An open review has less value if the manuscript isn't also "open”.
• Must be moderated by editor and replies from authors should be encouraged.

​

​