This workshop will explore new ways to make better use of expert opinion for planning and decision making. The goal is to discover new hybrid approaches that engage the best and brightest experts, regardless of their location, in collaborations yielding valid and reliable plans or decisions in an expeditious manner. This topic is timely for three reasons. First, domain expertise is scarce and expensive, so it is a resource that needs to be managed well. Second, improvements in communications and computer networking technology have created a new scientific "frontier'' for exploring expert opinion and its applications, making this a ripe topic for Algorithmic Decision Theory. Third, recent social and political events have underscored the need for more disciplined approaches to planning and decision making--ones that address the validity, reliability and traceability of results, and can identify sources of expert bias.
The workshop will address questions like the following: (a) How should experts be selected and qualified? (b) What elicitation techniques and protocols yield the most useful data? (c) What is the best way to combine data elicited from experts? (d) What techniques are available for validating and assessing the reliability of analytical results? (e) What role might social networking, ubiquitous communications and distributed computing play in the timely acquisition, analysis and validation of decisions or plans derived from expert opinion? (f) What are the relative advantages and disadvantages of mathematical and social/behavioral models, and might an integration of the two be more efficacious than either applied alone? (g) How can expert opinion and quantitative data be used together most effectively? (h) What situations favor pooling of expert opinion and what ones favor so-called "crowd sourcing" approaches?
There are two fundamental approaches to risk-sensitive planning and decision making--those that have been categorized as "mathematical" and those categorized as "behavioral'' or "social''. The former category includes mathematical algorithms for eliciting and combining expert opinions, such as the non-Bayesian, Bayesian and psychological scaling methods, while the latter category relies more on communications and social processes to drive consensus estimation, for example the Delphi, Nominal Group and other methods. While the potential of approaches in both categories has been demonstrated under limited conditions, none are without critics. Nor has there been much effort to integrate these approaches to make best use of both quantitative data and more qualitative expert opinion. This WS will explore both approaches.
Agencies are sometimes reluctant to use expert opinion in planning and decision making because of a perception that expert opinion is unreliable and prohibitively expensive. This perception is unfortunate since for many planning and decision making activities, e.g., evacuation planning in response to terrorism, there are no alternative sources for data. Moreover, new advances in computing and communications technology (hand-held computers, social media, "tweeting'') could be applied to reduce the cost of recruiting and convening experts, effectively eliminating need for "face-to-face'' collaborations.
Recent developments in social, mathematical, statistical and computer sciences have created an opportunity to examine the potential of combining improved algorithms in hybrid approaches. Algorithmic developments in decision making and risk assessment that we will explore include those in: ubiquitous communication and computing, social networking, data elicitation and imputation, and analysis of small data sets, including computationally-intensive statistics. The workshop will evaluate these and determine where improvements might be made by combining them.
Next: Call for Participation
Workshop Index
DIMACS Homepage