The following is a tentative list (a superset) of the topics to be covered in the class. Note: This list will change as we get closer to the start of the semester and also as the semester progresses. For a listing of lectures and schedules, visit the lectures page.
- Basic concepts and Bayesian statistics
- Probability space, random variables, CDF, PDF, expectation, variance, independence, etc.
- Probability distributions, e.g., (Multivariate) Gaussian distribution, student t-distribution, Beta distribution, Gamma and inverse Gamma distributions, Dirichlet distribution, etc.
- Maximum likelihood estimation (MLE), maximum a posterior estimation (MAP), predictive distribution, type II MLE, empirical Bayes
- Bayesian decision theory, Bayesian model selection
- Noninformative priors, exchangeability, de Finetti's theorem, Bayesian philosophy
- Exponential family and conjugate priors
- Generalized linear models
- Bayesian linear regression, logistic regression and probit regression
- Multi-class logistic regression and ordinal regression
- Generalized linear models in exponential family
- Probabilistic graphical models
- Bayes networks and Markov random fields
- Conditional independence, Markov blanket, Bayesian ball algorithm
- Factor-graphs
- Approximate inference
- Laplace approximation
- Variational inference, EM algorithm, variational model evidence lower bound, mean-field, convex conjugate, variational message passing
- Markov Chain Monte-Carlo algorithms: Metropolis Hasting, Gibbs sampling, Hamiltonian Monte Carlo
- Bayesian neural networks
- Bayes back-propagation
- Variational auto-encoder
- Reparameterization tricks
- Diffusion Modeling
- Gaussian process
- Gaussian process regression
- prediction and training
- Connection to Bayesian NNs