Jury-Based Self-Evaluation: Benchmarking LLM Problem-Solving in Specialized Domains
Varuni Sarwal, Seungmo Lee, Aingela Kattapuram, Rosemary He, Mandy Wang, Eleazar Eskin, Serghei Mangul, Wei Wang
Department of Computer Science, University of California Los Angeles
INTRODUCTION & CONTRIBUTION
We created BioinformaticsBench, which is to-date the largest benchmark set containing 602 human annotated college level questions across 9 different bioinformatics domains
CAN LLM’s REPLACE HUMANS FOR ERROR ANALYSIS?
LEADERBOARD
DATASET EXAMPLE
ERROR ANALYSIS OF LLM RESPONSE
Varuni is on the job market starting December’24!
Google Scholar
We are looking for collaborators to grow BioinformaticsBench, join us here!
BioinformaticsBench consists of two major datasets, the public set and the test set. The public dataset was created by searching for domain specific textbooks and curating a problem set. The set was preprocessed according to the rulebook and the questions are checked for correctness via skilled human annotators. The test set consisted of exam problems for college level bioinformatics classes, which were not publicly released.
Human domain experts randomly sampled 100 error annotations and distilled the errors into 10 essential skills the models might lack, namely: chain of thought, assumption identification, problem deduction skills, unit conversion, domain knowledge, hallucination, logical reasoning, arithmetic correct response but missing output, and incorrect response missing output.
Given the high cost of human annotations required to attribute the cause of incorrect solutions to specific skill deficiencies, we proposed a novel self-critique protocol as follows. We employed a set of 3 LLMs to serve as jury of classifiers and determine whether a specific error results from the lack of a particular problem-solving skill. Finally, we employed a consensus vote of the classification results to determine the class. This could potentially lead to the lack of human intervention while still maximizing the classification accuracy.
However, this approach emphasizes the requirement of human eval, while inter-human agreement is 95%, there is only a 37.5% agreement among GPT-4o and GPT-4-Turbo models, and a 36% agreement among GPT-4o and GPT-3.5 models. Furthermore, there is only a 50% agreement between a human and a language model. This proves that while certain LLMs can be used to solve bioinformatics problems, error analysis still remains a challenge, and the role of expert human evaluators is imperative.