The Association for the Advancement of Artificial Intelligence (AAAI) has not yet opened its reviewer data. However, there's a noticeable interest in the community to view how scores are distributed. In light of this, we are exploring the possibility of gathering ratings and confidence levels anonymously to create a visualization of scores that meets this interest.
We acknowledge that this method **
might lead to bias**. The number of filled data will be updated on the Paper Copilot site
without modification and
no personal data will be collected.Please fill in the ratings in the same order split by commas ',' or semicolon ';', or just space ' ' e.g.:
Ratings: 3,3,4
Confidence: 1,2,3
[Total Submitted Responses (update every 8 hours PST daytime)]
- All Valid Responses (Remove Redundancy and Invalid Input)
- Valid Responses with Rebuttal (press [s] and switch to rebuttal with overlaid bars or scattered chart)
- Phase 1 and Phase 2 (press [s] and switch to decision): Phase 1 (Reject), Phase 2 (Unknown)
Notice:
- You can update your submission anytime when you re-visit this link, we'll consider only the latest submitted data for every paper ID (paper ID is only used for the accuracy of the review curve, false paper ID will introduce noise to the collected data).
- 10: Top 5% of accepted papers, seminal paper; 9: Top 15% of accepted papers, strong accept; 8: Top 50% of accepted papers, clear accept; 7: Good paper, accept; 6: Marginally above acceptance threshold; 5: Marginally below acceptance threshold; 4: Ok but not good enough - rejection; 3: Clear rejection; 2: Strong rejection; 1: Trivial or wrong
- This Google form will be removed if there's no response or if the reviews are publicly available. We'll also update this on Twitter during the discussion, follow me if needed.