Seminar Details
AI in Science: Foundations for an Explosion of Discovery
- Start Date: October 22, 2026
- End Date: October 22, 2026
- Event Start Time: 12:00 PM
- Event End Time: 1:00 PM
- Event Location: CoRE Building | Room 431
- Seminar Series: Rutgers EconCS Seminar
- Presenter(s): Justin Payan, Postdoctoral Research Associate, Carnegie Mellon University
- Presentation Type: Stand Alone Presentation
- Abstract:
AI is rapidly accelerating math and science: OpenAI recently released hundreds of proofs for unsolved problems in mathematics, and Anthropic’s Claude drove the discovery of a promising enzyme system in a wet lab. Yet AI also allows anyone with an internet connection to generate ostensibly correct pseudoscience at the click of a button. Our scientific infrastructure is buckling under the increased volume.
I propose leveraging AI at key points to focus human oversight, and I demonstrate two applications of AI in peer review. Many peer review venues have deployed prototype AI reviewers to serve alongside human experts, claiming that AI reviewers can help identify errors and weed out obviously poor quality submissions. But do these AI reviewers accurately identify problems in submissions? I present a benchmark called FLAWS (Fault Localization Across Writing in Science), which inserts major flaws into papers to evaluate AI reviewer capabilities. At the time of evaluation (November 2025), we found that the best-performing model (GPT-5) identified inserted errors in approximately 40% of instances. The journal Transactions on Machine Learning Research (TMLR) also used FLAWS to evaluate AI reviewers, finding the top-performing system’s performance satisfactory for experimental deployment alongside human reviewers.
Even with AI reviewers, peer review venues still struggle to keep up with increasing submissions of AI-generated articles – articles that the “authors” may not have even read! We develop a test called greCAPTCHA, which verifies author understanding by asking authors questions about their own papers. In a study with 31 researchers, we show that we are able to distinguish researchers answering questions about their own versus unfamiliar papers with an ROC AUC of 0.9. Semi-structured interviews show broad support for greCAPTCHA, and suggest changes to make before deployment. A public demo is available at grecaptcha.com. These projects examine how AI can support scientific integrity, while preserving human judgment and accountability.