Date 
Topic 
Readings/Further References 
Deadlines 
Slides/Notes 
Aug 23 
Introduction and Class Logistics 
Background material cribsheet,
MATLAB tutorial, [M06] 
HW0 out 
slides (printversion) 
Supervised Learning 
Aug 25 
KNearestNeighbors and Decision Trees 
knn.pdf, dtrees.pdf, [Q86], [I04], [WBS06] 

slides (printversion), infotheory notes 
Aug 30 
Decision Trees (Contd.) and Data Representation 

HW0 due 
slides (printversion) 
Sep 6 
Learning Models by Fitting Parameters: Linear and Ridge Regression (+Maths Refresher) 
linreg.pdf, some notes, [KD08], [MD10], [PP08] 
HW1 out 
slides (printversion) 
Sep 8 
Learning Hyperplane Separators: Perceptron and (Intro to) Support Vector Machines 
perceptron.pdf, svm.pdf, [B98] 

slides (printversion) 
Sep 13 
Support Vector Machines (Contd.), Loss Fuctions and Regularization 
svm.pdf, [B98], [BL07], [C07], [YHL11], [SFR09] 

slides (printversion) 
Sep 15 
Kernels Methods and Nonlinear Classification 
svm.pdf (section 7),
Learning with Kernels, [HSS08],
Learning Kernels 

slides (printversion) 
Sep 20 
Learning Probabilistic Models: Linear Regression (revisited) and Logistic Regression 
parameterestimation.pdf, [M03] 
HW1 due 
slides (printversion) 
Sep 22 
Model Selection and Feature Selection 
evaluation.pdf, An Introduction to Variable and Feature Selection, [K95] 
HW2 out 
slides (printversion) 
Sep 27 
Learning Theory 
computationallearningtheory.pdf 

slides (printversion) 
Sep 29 
Supervised Learning: Odds and Ends 


no slides 
Unsupervised Learning 
Oct 4 
Clustering: Kmeans and hierarchical clustering 
clustering.pdf, [JMF99], [J08] 

slides (printversion) 
Oct 6 
Probabilistic approaches to clustering (Gaussian Mixture Models via Expectation Maximization) 
mixturemodelsem.pdf (Sections 9.29.3.2), EM for Mixture of Gaussians 
HW2 due 
Gaussian Mixture Models notes 
Oct 11 
Fall Break 



Oct 13 
Fall Break 



Oct 18 
GMM Recap and the general Expectation Maximization algorithm 
EM for Mixture of Gaussians, Expectation Maximization tutorial, 
Project Proposal 
The Expectation Maximization algorithm 
Oct 20 
Linear Dimensionality Reduction 
pca.pdf, PCA Tutorial (good intuitive explanations and different perspectives), [BCR04] (advanced reading)

HW3 out 
slides (printversion) 
Oct 25 
Nonlinear Dimensionality Reduction: Kernel PCA, Manifold Learning (LLE, ISOMAP) 
LLE, Isomap, Spectral Methods for Dimensionality Reduction 

slides (printversion) 
Assorted Topics 
Oct 27 
Ensemble Methods: Bagging and Boosting 
Bagging, Boosting, and C4.5, Ensemble Methods (SDM'10 Tutorial) 

AdaBoost Introduction, slides 
Nov 1 
Imbalanced Data and Multiclass Classification 
The Class Imbalance Problem,
Survey on Multiclass Classification Methods,
Label Embedding Trees for Large MultiClass Tasks (optional,
but read section 4 on related work) 

Imbalanced and Multiclass Classification (up to section 5.2),
Perceptrons for Imbalanced and Multiclass Classification 
Nov 3 
Ranking and Collective Classification 
Ranking Tutorial (optional reading) 
HW3 due (now due on 5/11) 
Ranking and Collective Classification (required reading) 
Nov 8 
Semisupervised Learning 
SemiSupervised Learning Literature Survey (required reading: sections 14, 5, 5.1, 67; other sections optional) 

slides (printversion) 
Nov 10 
Active Learning 
Active Learning Literature Survey (sections 13; other sections optional) 

slides (printversion) 
Nov 15 
Naïve Bayes Classification; Generative vs Discriminative Models 
Principled Hybrids of Generative and Discriminative Models (optional reading) 

Book Chapter (required reading: section 1,2,4,5) 
Nov 17 
Structured Prediction (1): Hidden Markov Models 

HW4 Due 
HMM notes 
Nov 22 
Structured Prediction (2) 
(Optional readings) MEMM, CRF, Other references can be found from this seminar webpage 

Structured Prediction 
Nov 24 
Thanksgiving Break 



Nov 29 
Reinforcement Learning (1): Discrete MDPs, Value Iteration, Policy Iteration 
(Optional reading) Reinforcement Learning: An Introduction (chapters 1,3,4) 

slides (printversion) 
Dec 1 
Reinforcement Learning (2): Continuous MDPs 


Andrew Ng's notes (section 4 onwards) 
Dec 6 
Intro to Bayesian Learning, and class wrapup 
(Optional readings) Bayesian Modelling in Machine Learning: A Tutorial Review, MCMC tutorial,
Conjugate Prior Relationships 

Bayesian Learning intro (up to section 3) 
Dec 8 
Final Exam 


