Emanuel Parzen [125] invented this approach in the early 1960s, providing a rigorous mathematical analysis. Since then, it has found utility in a wide spectrum of areas and applications such as pattern recognition [48], classification [48], image registration [170], tracking, image segmentation [32], and image restoration [9].

Parzen-window density estimation is essentially a data-interpolation
technique [48,171,156]. Given an instance of the random sample, ,
Parzen-windowing estimates the PDF from which the sample was derived. It essentially
superposes *kernel* functions placed at each observation or datum. In this way, each
observation contributes to the PDF estimate. There is another way to look at the estimation
process, and this is where it derives its name from. Suppose that we want to estimate the value of
the PDF at point . Then, we can place a *window function* at and determine how
many observations fall within our window or, rather, what is the contribution of each
observation to this window. The PDF value is then the sum total of the contributions
from the observations to this window. The Parzen-window estimate is defined as

where is the window function or kernel in the -dimensional space such that

(54) |

and is the *window width* or *bandwidth* parameter that corresponds to the
width of the kernel. The bandwidth is typically chosen based on the number of available
observations . Typically, the kernel function is unimodal. It is also itself a PDF,
making it simple to guarantee that the estimated function satisfies the properties of a
PDF. The Gaussian PDF is a popular kernel for Parzen-window density estimation, being infinitely
differentiable and thereby lending the same property to the Parzen-window PDF estimate .
Using (2.53), the Parzen-window estimate with the Gaussian kernel becomes

(55) |

where is the standard deviation of the Gaussian PDF along each dimension. Figure 2.5 shows the Parzen-window PDF estimate, for a zero-mean unit-variance Gaussian PDF, with a Gaussian kernel of and increasing sample sizes. Observe that with a large sample size, the Parzen-window estimate comes quite close to the Gaussian PDF.