MLRG/fall10

'''Semisupervised and Active Learning'''

'''Fri 2:00-3:20pm'''

'''MEB 3105'''

==Synopsis==

Supervised learning algorithms usually require a good amount of labeled data in order to learn a reliable model. Since getting large quantities of labeled data can be expensive and/or difficult, much effort in machine learning has been devoted on coming up with ways to learn with a limited amount of labeled data. There are many ways of doing this. Two very important paradigms we will be looking at in this seminar are (1) ''semi-supervised learning'' which involves augmenting a small amount of available labeled data with a large amount of additional unlabeled data (which is usually very easy to obtain), and (2) ''active learning'' which involves judiciously selecting the most informative/useful labeled examples to be given to a supervised learning algorithm. In this seminar, we will be looking at some representative papers from both these paradigms. As it will not be possible to cover all important papers in a single seminar, for those interested, a bunch of papers will be added under the suggested readings.

== Participants ==
* [http://www.cs.utah.edu/~piyush Piyush Rai], PhD Student, School of Computing
* [http://www.cs.utah.edu/~suresh/web/ Suresh Venkat], Asst. Prof, School of Computing
* [http://www.cs.utah.edu/~huangrh Ruihong Huang], PhD Student, School of Computing
* [http://www.cs.utah.edu/~alnds Lalindra De Silva], PhD Student, School of Computing
* [http://www.cs.utah.edu/~sandeepp Sandeep P], MS Student, School of Computing
* [http://www.nealrichter.com Neal Richter], PhD (Montana State U), Working at the Rubicon Project
* [http://www.cs.utah.edu/~ngilbert Nathan], PhD Student, School of Computing

==Schedule== 
(subject to change; * means will probably need a rescheduling)

{|  border="1" style="width: 100%; text-align:left" class="content"
|-
! Date || Topic || Outline and Paper(s) || Presenter 
|-
| Sep 3 || Outline, Motivation || Seminar logistics. Introduction to semisupervised learning and active learning || [http://www.cs.utah.edu/~piyush Piyush]
|-
| colspan="4" bgcolor="#dddddd" style = "text-align:center" | '''Semisupervised Learning''' 
|-
| Sep 10 || Bootstrapping/weak-supervision || [http://www.cs.cmu.edu/~tom/10-702/cotraining.ps Combining Labeled and Unlabeled Data with Co-Training] (for some theoretical results, also see [http://books.nips.cc/papers/files/nips14/LT14.pdf PAC Generalization Bounds for Co-training]) || [http://www.cs.utah.edu/~piyush Piyush]
|-
| Sep 17 || Low density regions and the cluster assumption for SSL || [http://www.kyb.mpg.de/publications/pdfs/pdf2899.pdf Semi-Supervised Classification by Low Density Separation], (also see section 5 of the [http://pages.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf SSL survey] for other methods and further references) || [http://www.cs.utah.edu/~huangrh Ruihong]
|-
| Sep 24 || Imposing function smoothness: Graph based SSL || [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.61.8355&rep=rep1&type=pdf Manifold Regularization for Semi-supervised Learning], (also see section 6 of the [http://pages.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf SSL survey] for other methods and further references)  || [http://www.cs.utah.edu/~sandeepp/ Sandeep]
|-
| Oct 1 || Probabilistic approaches: Expectation Maximization for SSL || [http://www.cs.umass.edu/~mccallum/papers/semisup-em.pdf Semi-Supervised Text Classification Using EM] ||
[http://www.cs.utah.edu/~alnds Lalindra]
|-
| Oct 8 || Using unlabeled data to learn predictive functional structures || [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.77.9121&rep=rep1&type=pdf A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data] || [http://www.cs.utah.edu/~sandeepp/ Sandeep]
|-
| Oct 22 || Semi-supervised Learning for Ranking || [https://ssli.ee.washington.edu/people/duh/papers/sigir.pdf Learning to Rank with Partially-Labeled Data] || [http://www.nealrichter.com/ Neal]
|-
| Oct 29 || <s>Semi-supervised Learning Theory</s> || [http://www.cc.gatech.edu/~ninamf/papers/labeled_unlabeled.pdf A PAC-style Model for Learning from Labeled and Unlabeled Data] CANCELED|| <s>[http://www.nealrichter.com/ Neal]</s>
|-
| Nov 5|| Semi-'''unsupervised''' Learning (for Clustering/Dimensionality Reduction) || [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1.5745&rep=rep1&type=pdf Integrating constraints and metric learning in semi-supervised clustering], [http://www.siam.org/proceedings/datamining/2007/dm07_073Zhang.pdf Semi-Supervised Dimensionality Reduction] || [http://www.cs.utah.edu/~huangrh/ Ruihong]
|-
| colspan="4" bgcolor="#dddddd" style = "text-align:center" | '''Active Learning''' 
|-
| Nov 12 || Pool-based active learning, Query by committee, Query by uncertainty || [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.89.4888&rep=rep1&type=pdf Support Vector Machine Active Learning with Applications to Text Classification], [http://pages.cs.wisc.edu/~bsettles/pub/settles.activelearning.pdf Active Learning survey (sections 3.1 and 3.2)] || <s>[http://www.cs.utah.edu/~ngilbert Nathan]</s>
|-
| Nov 19 || Pool-based active learning, Query by committee, Query by uncertainty, Stream-based active learning || [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.89.4888&rep=rep1&type=pdf Support Vector Machine Active Learning with Applications to Text Classification], [http://pages.cs.wisc.edu/~bsettles/pub/settles.activelearning.pdf Active Learning survey (sections 3.1 and 3.2)], [http://www.jmlr.org/papers/volume7/cesa-bianchi06b/cesa-bianchi06b.pdf Worst-Case Analysis of Selective Sampling for Linear Classification] || [http://www.cs.utah.edu/~ngilbert Nathan]
|-
| *Nov 26 || Dealing with sampling bias and using cluster-structure for active learning || [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.149.5701&rep=rep1&type=pdf Hierarchical Sampling for Active Learning] ||
|-
| Dec 3 || Semi-supervised learning and active learning || [http://pages.cs.wisc.edu/~jerryzhu/pub/zglactive.pdf Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions] || Piyush
|-
| Dec 10 || Multiview active learning || [http://www.isi.edu/~muslea/PS/muslea06a.pdf Active Learning with Multiple Views] || (To be rescheduled)
|}

==Suggested Readings==

Some survey papers:
* [http://pages.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf Semisupervised Learning Literature Survey]
* [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.87.10&rep=rep1&type=pdf Learning with Labeled and Unlabeled Data]
* [http://pages.cs.wisc.edu/~bsettles/pub/settles.activelearning.pdf Active Learning Literature Survey]

Other papers:
* On Co-Training: [http://books.nips.cc/papers/files/nips17/NIPS2004_0399.pdf Co-Training and Expansion: Towards Bridging Theory and Practice], [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.144.3460&rep=rep1&type=pdf Bayesian Co-training], [http://www.icml2010.org/papers/275.pdf A New Analysis of Co-Training]
* [http://cseweb.ucsd.edu/~dasgupta/papers/twoface.pdf Two faces of active learning]

Will be updated with more papers.