## DIMACS TR: 2007-16

## Saddle Point Feature Selection in SVM Classification

### Authors: Yuri Goncharov, Ilya Muchnik and Leonid Shvartser

**
ABSTRACT
**

SVM wrapper feature selection method for the classification
problem, introduced in our previous work, is
analyzed. The method based on modification of the standard SVM
criterion by adding to the basic objective function a third term,
which directly penalizes a chosen set of variables. The criterion
divides the set of all variables into three subsets: deleted,
selected and weighted features. We give more formal derivation of
the saddle point problem to which SVM wrapper method reduces.
Saddle point algorithm described, proof of its convergence and
estimation for the step size of the algorithm done. Effective
calculations of projections used in the saddle point algorithm are
described. The algorithm is examined on a classification Benchmark
and its ability to improve the SVM recognition results is shown.

Paper Available at:
ftp://dimacs.rutgers.edu/pub/dimacs/TechnicalReports/TechReports/2007/2007-16.pdf

DIMACS Home Page