2007-16: Saddle Point Feature Selection in SVM Classification

DIMACS TR: 2007-16

Saddle Point Feature Selection in SVM Classification

Authors: Yuri Goncharov, Ilya Muchnik and Leonid Shvartser

ABSTRACT
SVM wrapper feature selection method for the classification problem, introduced in our previous work, is analyzed. The method based on modification of the standard SVM criterion by adding to the basic objective function a third term, which directly penalizes a chosen set of variables. The criterion divides the set of all variables into three subsets: deleted, selected and weighted features. We give more formal derivation of the saddle point problem to which SVM wrapper method reduces. Saddle point algorithm described, proof of its convergence and estimation for the step size of the algorithm done. Effective calculations of projections used in the saddle point algorithm are described. The algorithm is examined on a classification Benchmark and its ability to improve the SVM recognition results is shown.

Paper Available at: ftp://dimacs.rutgers.edu/pub/dimacs/TechnicalReports/TechReports/2007/2007-16.pdf

DIMACS Home Page