« DIMACS Workshop on Foundation Models, Large Language Models, and Game Theory
October 19, 2023 - October 20, 2023
Location:
DIMACS Center
Rutgers University
CoRE Building
96 Frelinghuysen Road
Piscataway, NJ 08854
Click here for map.
Organizer(s):
Tamra Carpenter, DIMACS
Denizalp Goktas, Brown University
Amy Greenwald, Brown University
David Pennock, DIMACS
Segev Wasserkrug, IBM Research
Foundation models—models trained on large datasets which can easily adapt to many tasks using few-shot or zero-shot learning—are a major paradigm shift in AI. A primary example are Large Language Models (LLMs), such as ChatGPT, which, through natural language prompting and interaction, can already provide significant value in use cases such as document and code generation. The widespread deployment of such models, however, is creating new challenges, both technical and societal.
This workshop aims to initiate work at the intersection of foundation models and game theory. We are interested both in how to use game theory to address some of the issues arising from the use of foundation models, and in how we might advance game theory through the use of foundation models. Our goal, therefore, is to attract researchers who work on foundation models, natural language processing (NLP), and game theory, and to facilitate a structured, academic exchange among them.
The workshop will focus on:
Avenues to improve foundation models and LLMs through the use of game-theoretic models and tools: Leveraging multiagent learning in games to study interactions among foundation models and LLMs, and to improve their training algorithms.
Advancing game theory and algorithmic game theory using foundation models and LLMs: Studying the potential use of foundation models and LLMs to better model human preferences and to solve games, especially those that arise in economic, political, and social contexts.
Assessment of the societal impact of foundation models and LLMs using game theory: Investigating methodologies to quantify and predict the effects of LLMs on technology and society using game theory.
The exchange of ideas among researchers from the various communities has the potential to push the frontiers of foundation model development and AI more broadly. This workshop is intended as a platform in which to initiate this exploration.
Thursday, October 19, 2023
Breakfast & Check in
Welcome & Opening Remarks
What Would It Take to Train Foundation Models That Are Strategic?
Constantinos Daskalakis, Massachusetts Institute of Technology
Break (30 minutes)
On Proper Loss Functions for Evaluating Generative Models
Bo Waggoner, University of Colorado
Chat Games: Strings as Strategies
Ian Gemp, Google DeepMind
When the Majority is Wrong: Modeling Annotator Disagreement for Language Tasks
Eve Fleisig, University of California, Berkeley
Lunch
Recognizing Failures in the Successes of Large Language Modeling
Kathleen McKeown, Columbia University
Break (30 minutes)
Building Strategic AI Agents Using Language Models and Game Theory
Athul Paul Jacob, Massachusetts Institute of Technology
The Interaction of Game Theory and Natural Language Processing
Roma Patel, Google DeepMind
A Meta-Game Evaluation Framework for Multiagent Training Algorithms
Michael Wellman, University of Michigan
Break (30 minutes)
Rump Session
Dinner
Getting Computers to Do What We Want: Programming Meets Machine Learning
Michael Littman, Brown University & National Science Foundation
Friday, October 20, 2023
Breakfast
Welcome & Remarks
Large Language Models as Economic Agents: What Can We Learn from Home Silicus?
John Horton, Massachusetts Institute of Technology
Break (30 Minutes)
Scaling Human Feedback Using Foundation Models
Minae Kwon, Stanford University
Fine-tuning Games: Bargaining and Adaptation for General-Purpose Models
Hoda Heidari, Carnegie Mellon University
Recursive Self-improving Code Generation
Adam Kalai, Microsoft Research
Lunch
Panel Discussion
Fei Fang, Carnegie Mellon University
Gabriele Farina, Massachusetts Institute of Technology
Amy Greenwald, Brown University
Kevin Leyton-Brown, University of British Columbia
Matthew Stone, Rutgers University
Breakout Groups
Breakout Group Readouts
Presentations at this workshop are by invitation but others are welcome to attend. There is no fee to attend but registration is required. Please register using the button at the bottom of the page. Space is limited, so please register early if you plan to attend.
Update (9/26/2023): This event has reached capacity, so we have removed the registration link. If you would like to join a waiting list to attend please send email to Nicole Clark Johnson.
Poster session: The workshop will feature a poster session. If you would like to present a poster please apply using the form referenced below. 9/26/2023: Submissions are now closed.
Request support: There are limited funds available to support travel by those whose attendance is contingent on support. Please apply by September 23, 2023 and do not book your tickets until you hear from us!
To apply for travel support or to apply to submit a poster: Please complete this form. (It is a single form through which you can apply for support or to present a poster, or both.) We especially encourage diverse and inclusive participation. We will prioritize applications for support from students presenting posters and those from minority or underrepresented groups.9/26/2023: Submissions are now closed.
Presented in association with the SF on Mechanisms & Algorithms to Augment Human Decision Making.