Title: Large Margin Generative Models
Speaker: Tony Jebara, Columbia University
Date: May 7, 2004 10:00 am
Location: DIMACS Center, CoRE Bldg, Room 433, Rutgers University, Busch Campus, Piscataway, NJ
Abstract:
Generative models such as Bayesian networks, exponential family distributions and mixtures are elegant formalisms to setup and specify prior knowledge about a learning problem. However, the standard estimation methods they rely on, including maximum likelihood and Bayesian integration do not focus the modeling resources on a particular input-output task. In applied settings when models are imperfectly matched to the real data, discriminative learning is crucial for improving performance with such models. We consider classifiers built from the log-likelihood ratio of generative models and find parameters for these models such that the resulting discrimination boundary has a large margin. Through maximum entropy discrimination, we show how all exponential family models can be estimated with large margin using convex programming. Furthermore, we consider interesting latent models such as mixture models and hidden Markov models where the additional presence of latent variables makes large margin estimation difficult. We propose a variant of the maximum entropy discrimination method that uses variational bounding on classification constraints to make computations tractable in the latent case. The method finds large margin settings reliably by iteratively interleaving standard expectation steps with large margin maximization information projection steps. Interestingly, the method gives rise to Lagrange multipliers that behave like posteriors over hidden variables. Preliminary experiments are shown.
See Webpage: http://www.stat.rutgers.edu/~madigan/mms/spring04.html