A PDF version of the reviewer guidelines (download)
We are grateful for your willingness to serve as a reviewer for WASPAA 2023. Your role is pivotal in ensuring the quality and success of the workshop.
What Happens Next?
Update your information for the Toronto Paper Matching System (TPMS). Also, in your CMT profile, add conflict domains and set your maximum review quota between 4 and 6 (we strongly encourage you to keep the default value of 6). Shortly after the first submission deadline you will be asked to bid for papers and identify specific conflicts of interest. Based on this information you will be assigned paper submissions for review. As a reviewer for those papers you will have 3 main responsibilities: (1) to identify potential COIs or other desk issues upon assignment; (2) to complete a thorough review for each assigned paper; and (3) to actively participate in the discussion phase and change your reviews based on the authors’ rebuttal accordingly. Below we outline some guidelines to assist you in this process.
Upon Review Assignment:
- Do an initial scan of your assigned papers as soon as you receive them. Identify potential conflicts of interest, disclosure of authors’ identities, and other issues needing quick clarification from authors, meta-reviewers or the TPC chairs.
- Note that the review process is double-blind this year. We are asking authors not to identify themselves and their institutions in the paper, but acknowledge that they may decide to post their papers on arXiv, while we still discourage advertising the work on social media until accepted. Therefore, don’t go out of your way to identify the authors, and if by any chance you are able to do so, try your best not to let that knowledge affect the outcome of the review process. If inadvertently identifying the authors leads to a potential conflict of interest, let the meta-reviewer and TPC chairs know immediately.
For Your Review:
- Provide substantial, constructive and timely feedback. Write as if you were addressing a close colleague or friend. Write directly, thoughtfully and politely.
- For individual questions in the review form, provide short, written justification of your recommended scores. Individual reviewers interpret the rating scale differently, so a written justification helps the authors, meta-reviewers and TPC chairs to understand your rating better.
- The section with comments for the authors is the most important in the form. We expect a substantial review with multiple paragraphs and itemized lists and recommend the following structure: (1) a 1-paragraph summary of the paper identifying core contribution(s), main strengths and weaknesses; (2) a detailed account of both major and minor issues with the paper, with a clear description and rationale for each criticism. Use citations as needed. Each major issue needs to be described in sufficient detail for the authors to be able to respond and/or revise the paper accordingly.
- Provide definitive accept/reject recommendations in your review scores, and avoid marginal recommendations as much as possible. Your score should be consistent with your assessment of the strengths and weaknesses of the paper. Accepts imply that any issues can be addressed in the camera-ready version, while rejects imply that there are major issues that cannot reasonably be expected to be addressed by the end of the review process. Papers should not be rejected for minor issues.
- If no or minimal justification is provided, or if the review is deemed to be of insufficient quality, the meta-reviewer or TPC chairs might request that you revise your review. Help us ensure the quality of WASPAA and help researchers in the field understand how to strengthen their work.
- If you encounter an exceptional paper, we encourage you to nominate the paper for an award and provide a brief rationale. Your nomination won’t be visible to the authors.
- Complete the review process within the specified time, so that the rebuttal, discussion and decisions can be completed without delays.
Rebuttal and Discussion:
- Authors have the option of writing a rebuttal letter prior to the discussion period. The purpose of the rebuttal is to address issues raised in the reviews and propose revisions if the paper is accepted. Keep in mind that, if accepted, only minor revisions are typically allowed for the camera-ready version of the paper. Any suggestion of a major revision in the rebuttal should be backed by evidence and only considered in special circumstances.
- At the start of the discussion phase, we expect reviewers to read the rebuttal letter and the other reviews.
- The discussion will be led by the meta-reviewer for each paper. Please plan to actively participate in all discussions, engaging early and meaningfully. Expect to commit 1-2 hours during the discussion phase across all your papers.
- Be respectful of other reviewers’ views, but do not hesitate to try to persuade others if you have a strong opinion. Whenever possible, try to form a consensus towards a final recommendation for each paper.
- If the rebuttal/discussion changes your mind, update both your scores and commentary accordingly, and please acknowledge the rebuttal letter.
Thank you once again for your invaluable service as a reviewer. If you have any questions or concerns, please do not hesitate to contact us. We truly appreciate your contribution and look forward to working with you.
Timeline for the Review Process
| Timeline | Action Item |
| 4/21 | Abstract/title deadline (for authors) |
| 4/24-5/3 | Paper bidding (for reviewers / meta-reviewers) |
| 4/28 | Full paper deadline (for authors) |
| 5/5 | Meta-reviewer assignments complete (for TPC chairs) |
| 5/12 | Reviewer assignments complete (for meta-reviewers) |
| 6/2 | Review deadline (for reviewers) |
| 6/12 | Reviews released to the authors |
| 6/23 | Rebuttal deadline (for authors) |
| 7/4 | Discussion period ends (reviewers / meta-reviewers) |
| 7/7 | Meta-reviews deadline (meta-reviewers) |
| 7/12 | Final notifications (TPC chairs) |