Understanding SVM (and associated kernel machines)
through the development of a Matlab toolbox

Stéphane Canu

Lectures given at the Institute of Mathematics and Statistics, University of Sao Paulo, feb 2014
Moodle site of the lecture at IME-USP thanks to Pr. R. Hirata Jr Sylabus (pdf)
The Objective of this course is to provide an overview of all these issues related with kernels machines. To do so, we will introduce kernel machines and associated mathematical foundations through practical implementation. All lectures will be devoted to the writing of some Matlab functions that, putting all together, will provide a toolbox for learning with kernels.

Lecture 1: Two classes classification problem: Introduction to Support Vector Machines (SVM), Linear SVM in the primal as a quadratic program and LPSVM for variable selection.
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file   Data (.mat)

Lecture 2: Karush-Kuhn-Tucker (KKT) optimality conditions, lagrangian duality, Linear SVM in the dual, support vectors
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file

Lecture 3: Linear SVM in the non separable case, reformulations: convex hulls and regularized hinge loss minimization. Libsvm and other QP solver
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file   Data (.mat)

Lecture 4: Positive definite kernels, associated RKHS, differentiability, subdifferentiablility and other kernels
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file

Lecture 5: SVM in a reproducing kernel Hilbert spaces (RKHS), representer theorem and other kernel machines (splines and kernel logistic regression)
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file   Checker data (.mat)

Lecture 6: Novelty detection, Support Vector Data Description (SVDD), One Class SVM, minimum enclosing balls and level sets
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file, An other solution for 2 class SVDD as a matlab file   Data (.mat)

Lecture 7: Tuning the hyperparamters, C, the bandwidth and the KKT gap
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file   Splice data (.mat)

Lecture 8: MultiClass SVM, estimating the posterior probabilities and structured output
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file   Data (.mat)

Lecture 9: How to choose the kernel, multiple kernel learning
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file   Data (.mat)

Lectrure 10: L0 SVDD,
Slides (pdf)   Practical Session (pdf)   My solution as a matlab file

Matlab toolbox and other usefull software resources:
A lecture on SVM in french Dernière mise à jour : 21 Mai 2014,
Stéphane Canu ( scanu@insa-rouen.fr)