March 17, 2021 - March 19, 2021
Raf Frongillo, University of Colorado
David Pennock, DIMACS
Bo Waggoner, University of Colorado
Following the successful EC 2017 Workshop on Forecasting, we will hold the DIMACS Workshop on Forecasting in 2021. We welcome submissions describing recent research on crowd-sourced, data-driven, or hybrid approaches to forecasting. We especially encourage contributions that leverage forecasts to improve decisions. Please see the Call for Participation below for details.
Recent advances in crowdsourced forecasting mechanisms, including Good Judgment’s superforecasting, prediction markets, wagering mechanisms, and peer-prediction systems, have risen in parallel to advances in machine learning and other data-driven forecasting approaches. Innovations have come from academic researchers, companies, data journalists, and government programs like IARPA’s Aggregative Contingent Estimation program and Hybrid Forecasting Competition.
The workshop will emphasize forecasts embedded inside decision-making systems, where the value of a forecast comes from increasing the expected utility of a key decision. Our ultimate goal is to modernize organizations, markets, and governments by improving how they collect and combine information and make decisions.
The workshop embraces the diversity of this exciting and expanding field and encourages submissions from a rich set of empirical, experimental, and theoretical perspectives. We invite theoretical computer scientists studying algorithmic game theory, incentivized exploration, and NP-hard counting problems; AI researchers studying machine learning, human computation, Bayesian inference, peer prediction, and satisfiability; statisticians studying scoring rules and belief aggregation; economists studying prediction markets, financial markets, and wagering mechanisms; data journalists and marketing scientists studying surveys and polls; blockchain pioneers implementing decentralized prediction markets and other experimental market constructs; social and behavioral scientists studying human behavior modeling; human-computer interaction researchers designing interfaces to facilitate elicitation or convey uncertainty; and practitioners working to improve forecasts as a business or service.
Uncertainty is hard to communicate. Forecasters argue that they are “right”, and critics that forecasters are “wrong” (for example about Brexit or the US Presidential election), despite the fact that probabilistic forecasts can only be evaluated in bulk relative to other forecasts. We invite contributions discussing ways to communicate uncertainty and educate the public about modeling, forecasting, and scoring, building on the excellent 2018 Nova episode “Prediction by the Numbers”.
Topics of interest for the workshop include but are not limited to:
Wednesday, March 17, 2021
Welcome & Opening Remarks
Invited Talk: How to Increase the Accuracy of Human Forecasts and Check the Reasons for Improvement
Barbara Mellers, University of Pennsylvania
Ville Satopää, INSEAD
Asymptotic Behaviour of Prediction Markets
Philip Dawid, University of Cambridge
Timely Information from Prediction Markets
Chenkai Yu, Tsinghua University
Invited Talk: A Heuristic for Combining Correlated Experts
Yael Grushka-Cockayne, University of Virginia
Thursday, March 18, 2021
Invited Talk: Predicting Replication Outcomes
Anna Dreber, Stockholm School of Economics
From Proper Scoring Rules to Max-Min Optimal Forecast Aggregation
Eric Neyman, Columbia University
Forecast Aggregation via Peer Prediction
Juntao Wang, Harvard University
Comparing Forecasting Skill vs Domain Expertise for Policy-Relevant Crowd-Forecasting
Emile Servan-Schreiber, Mohammed VI Polytechnic University
Forecasting Startup Founders Panel
Pavel Atanasov, pytho
Andreas Katsouris, PredictIt
Kelly Littlepage, OneChronos
Emile Servan-Schreiber, Hypermind
Friday, March 19, 2021
Invited Talk: Information, Incentives, and Goals in Election Forecasts
Andrew Gelman, Columbia University
Models, Markets, and the Forecasting of Elections
Rajiv Sethi, Columbia University
Boosting the Wisdom of Crowds Within a Single Judgment Problem: Weighted Averaging Based on Peer Predictions
Ville Satopää, INSEAD
Crowdsourced Forecast Elicitation: Methods vs. Individuals
Pavel Atanasov, pytho
Invited Talk: Models vs. Markets: Forecasting the 2020 U.S. election
Harry Crane, Rutgers University
Attend: This workshop is open to all to attend, but you must register using the link at the bottom of the page. We will send instructions on how to join the event on or before March 15, 2021. If you do not receive them, please check your spam folder or contact Nicole Clark. Please note that you may not be able to register once the event has begun.
Present: We invite both full contributions and poster contributions. A full contribution is an unpublished or recently published research manuscript. A poster contribution can be a preprint, a recently published paper, an abstract, or a presentation file. Preference may be given to more recent and unpublished work. We especially encourage poster contributions from students and postdocs.
Please submit your contributions using this Google Form by February 19, 2021. The workshop is non-archival, meaning contributors are free to publish their results later in archival journals or conferences. Panel discussion proposals and invited speaker suggestions are also welcome. Email questions or suggestions to the organizers.
The workshop will include invited and contributed talks, open discussion, and may include a poster session and a rump session. Workshop registration will be open. Once registered, you will join the workshop through Virtual Chair.
Presented in association with the SF on Mechanisms & Algorithms to Augment Human Decision Making.