2003-08: On Bayesian Learning of Sparse Classifiers

DIMACS TR: 2003-08

On Bayesian Learning of Sparse Classifiers

Authors: Wen-Hua Ju, David Madigan, and Steven L. Scott

ABSTRACT
Figueiredo (2001) and Figueiredo and Jain (2001) described a particular sparseness-inducing Bayesian model for probit regression. For several standard datasets, they reported predictive performance for their model that was as a good as, or better than, previously reported results. This paper explores several aspects of the Figueiredo and Jain model in an attempt to better understand its performance. We modify the Figueiredo and Jain approach in three ways. First, we introduce an alternative prior distribution. Second, we propose a fully Bayesian MCMC learning algorithm. Third, we replace their kernel based classifier with a linear classfier. We measure the impact of these modifications on three publicly available test data sets. Preliminary results indicate that while each change can produce a noticeable impact on test error rates, no one approach dominates the others in all cases.

Paper Available at: ftp://dimacs.rutgers.edu/pub/dimacs/TechnicalReports/TechReports/2003/2003-08.ps.gz

DIMACS Home Page