[R] svmpath: fitting the entire SVM regularization path

From: Trevor Hastie <hastie_at_stanford.edu>
Date: Wed 07 Jul 2004 - 04:24:28 EST

svmpath is a contributed package that fits the entire regularization path for a two-class SVM model.

The SVM (with any kernel), has a regularization or cost parameter C, which controls the amount of overlap
at the soft margin. When the SVM criterion is expressed in terms of a hinge loss plus lambda x quadratic penalty, then lambda=1/C. In many situations, the choice of C can be critical, and different regimes for C are called for as the other kernel tuning parameters are changed.

Most software packages come with a default value for C (typically very large), and the user is left to explore different values of C. It turns out that the lagrange multipliers which define the SVM solution for any C are piecewise linear in C (and more usefully piecewise linear and mostly piecewise constant in lambda) This means that we can compute the entire sequence of solutions for all values of C exactly. svmpath does this with essentially the same cost as fitting a single SVM model with a specified value of C.

See the paper (joint work with Saharon Rosset, Ji Zhu and Rob Tibshirani) http://www-stat.stanford.edu/~hastie/Papers/svmpath.pdf for details.

This code has been tested on moderate sized problems, with up to 1000 observations. The current version is not industry ready; occasionally it will run into situations where the steps are too small, leading to machine zero situations. Usually increasing the parameter eps from its default 1e-10 will avoid this.

Trevor Hastie

  Trevor Hastie                                  hastie@stanford.edu
  Professor, Department of Statistics, Stanford University
  Phone: (650) 725-2231 (Statistics)          Fax: (650) 725-8977
  (650) 498-5233 (Biostatistics) Fax: (650) 725-6951   URL: http://www-stat.stanford.edu/~hastie   address: room 104, Department of Statistics, Sequoia Hall

           390 Serra Mall, Stanford University, CA 94305-4065

R-help@stat.math.ethz.ch mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Wed Jul 07 07:38:27 2004

This archive was generated by hypermail 2.1.8 : Wed 03 Nov 2004 - 22:54:44 EST