Introduction to Support Vector Machines

given by Li He

Time
7:30 pm Sep 22, 2008
About 1h

Outline

• Several Results from Learning Theory
• Design a Simple SVM and its Kernelized Version
• Design an SVM in Nonseparable Case
• A General Formulation of SVM

Aims
This short introduction acts as a reminder of the audiences for later topics on S3VM, because there has been some time since Prof. Junping Zhang gave us the talk on SVM.

Related materials
Two Books, Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond; slides (only part of it is covered in the seminar).

Introduction to Semi-Supervised Learning (SSL)

given by Li He

Time
6:30 pm Sep 22, 2008
About 1h

Outline

• What is semi-supervised learning?
  • Formulations of different learning problems;
  • Do we do SSL?
  • Transduction v.s. SSL
    
• How does semi-supervised learning work?
  • Assumptions;
  • Classes of Algorithms
  • For Future Research
    
• About our seminar on semi-supervised learning.

Related Materials
Jerry Zhu's tutorial on SSL in ICML 2007; chap 1 and 25; slides.

Schedule

Please make sure you get everything ready before your own show.

2008.09.22 Introduction

• Introduction to Semi-supervised Learning (chap1, 25)
  
  given by Li He
  
  Related materials
  Jerry Zhu's tutorial on SSL in ICML 2007; chap 1, 2 and 25; slides.
  
• Introduction to Support Vector Machines
  
  given by Li He
  
  Related materials
  Two Books, Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond; slides (only part of it is covered in the seminar).
  

2008.10.13 S3VM Algorithms Part I: Lower Density Separation

• Transductive SVM and ▽S3VM
  given by ??
  Related materials
  chap 6, Joachims' paper, workshop on AI and stat 2005.
  
• SDP Convex Relaxation and its Application in S3VM
  given by ??
  Related materials
  chap 7, NIPS 2007,
  

2008.10.20 S3VM Algorithms Part II: Global Optimization Algorithms

• Continuation and its Application in S3VM
  Related Materials
  ICML 2006, about continuation.
  
• Deterministic Annealing and its application on S3VM
  Related Material
  ICML 2006,
  

2008.10.27 S3VM Algorithms Part III: Other Optimization Algorithms

• Branch and Bound and its Application in S3VM
  Related material
  NIPS 2006,
• CCCP and its Application in S3VM
  Related Material
  ICML 2006, NIPS 2001,
  

2008.10.27 Graph Based Method
Laplacian Smoother and its Application in SSL (including S3VM)

Seminar in Fall of 2008

Reading Material
Semi-supervised Learning, edited by Olivier Chapelle, Bernhard Scholkopf and Alexander Zien
Recent published papers in ICML and NIPS.

Time and Place
6:30 pm, Monday
Conference Room 203, Yifu Building

Attendees
Changyou Chen, Xin Cheng, Li He, Siqiang Luo, Jian Pu, Ben Tan, Xiaodan Wang, Xiaofan Xu, Junping Zhang, Ji Zhou.

Topics

• S3VM(Semi-Supervised Support Vector Machines)
  
  • Transductive SVM (chap 6)
  • ▽S3VM
    
  • continuation
  • deterministic annealing
  • CCCP
  • Branch and Bound
  • SDP convex relaxation (chap 7)
    
  • Laplacian smoother (chap 12)
    
• Generative Models
• Graph-based Methods
  • Graph cuts
  • Harmonic
    
• Low Density Separation
• Multiview
  • self-training
  • co-training
    
• Change of Representation
• Misc
  

Accepted Suggestions
if any, please comment.

Tips for Reading this Blog
Please try a news reader (RSS aggregator) that is able to manage the RSS of this blog. E.g. Google reader (for google fans, unnecessary to install), akregator(for KDE users), or whatever you like (if you don't know, please find one for yourself here.).

Please comment on each entry here for further discussion. E.g. please tell me what kind of topics you like to present to us and in what kind of style.

Paper Scanning
Since the field of semi-supervised learning is still evolving, we hope whenever you encounter an interesting paper on semi-supervised learning, please share it here with a simple introduction of the idea proposed by the paper with links to the paper and its most important references.

About Labelling Entries in this Blog
I haven't decide the most proper style. But I think for paper scanning, the keywords should be the best. For seminar entries it's better to label it with ``seminar.''

Administration
Currently this blog has one administer, Li He. If you have any further request, please send him an email.