Principal Surfaces and Manifold Learning
Manifold learning is a specific approach to nonlinear dimensionality
reduction based on the assumption that data points are sampled from a low
dimensional manifold embedded in a high dimensional ambient space. The
aim is to uncover the low dimensional manifold structure from the
samples in the high ambient space. Many methods for manifold learning
have been proposed in the machine learning literature. Much of the
recent work focused around what can be called global or spectral methods.
These methods have a closed form solution based on the spectral
decomposition of a matrix that is compiled based on local properties of
the input data.
In many use cases of manifold learning it is necessary to map from
ambient space to manifold coordinates or construct data points given
manifold coordinates, e.g. build a generative model. Manifold learning
methods thus far are mostly concerned with finding a low dimensional
parametrization, the manifold coordinates, of the data, but often do not
provide the tools to project or construct new data points, as for example
PCA does in the linear case.
We propose an approach, kernel map manifolds, that provides the
tools to project data points onto the manifold and reconstruct data
points on the manifold. The approach is firmly rooted in the concept of
principal surfaces, a conceptual extension of principal component
analysis to nonlinear data. Informally principal surfaces pass through
the middle of a distribution. A variational formulation of principal
surface leads to a manifold model that converges to a principal manifold
as the number of samples increases.
Regularization of principal curves / manifolds is a challangeing
problem. We develop a novel method, based on the ideas of the kernel map
manifold approach, to estimate principal curves that solves the
regularization problem in principal curve estimation.
Related publications
Samuel Gerber, Ross Whitaker
"Regularization Free Principal Curve Estimation",
Accepted JMLR 2012. [pdf]
Samuel Gerber, Tolga Tasdizen, Ross Whitaker
"Dimensionality Reduction and Principal Surfaces via Kernel Map Manifolds",
In Proceedings of the 2009 International Conference on Computer Vison
(ICCV 2009). [pdf]