Meena Jagadeesan, University of Pennsylvania

Abstract

When designing compound AI systems, a common approach is to query multiple copies of the same model and aggregate the responses to produce a synthesized output. Given the homogeneity of these models, this raises the question of whether aggregation unlocks access to a greater set of outputs than querying a single model. In this work, we investigate the power and limitations of aggregation within a principal-agent framework. This framework models how the system designer can partially steer each agent's output through reward specification, but still faces limitations due to prompt engineering ability and model capabilities. Our analysis uncovers three natural mechanisms—feasibility expansion, support expansion, and binding set contraction—through which aggregation expands the set of outputs that are elicitable by the system designer. We show that any aggregation operation must implement one of these mechanisms in order to be elicitability-expanding. Moreover, we show that strengthened versions of these mechanisms provide necessary and sufficient conditions which fully characterize elicitability-expansion. Altogether, our results take a step towards characterizing when compound AI systems can overcome limitations in model capabilities and in prompt engineering. Based on joint work with Nivasini Ananthakrishnan.