Results

This section gives experimental results on numerous real and synthetic images along with the analysis of UINTA's behavior and qualitative and quantitative comparisons with the state-of-the-art wavelet methods. UINTA exposes only three parameters to the user: (i) the size $\vert\mathcal{N}_t\vert$ of the neighborhoods, (ii) the standard deviation $\sigma_{\mathrm{spatial}}$ of the Gaussian PDF that defines the extent from which local samples are taken for density estimation (for stationary images such as textures, a global-sampling scheme will work best), and (iii) the number of iterations, or other parameters, related to the stopping criterion. Empirical results show that UINTA's performance degrades gracefully--no drastic effects as in typical PDE-based filtering schemes--for suboptimal values of these parameters. We use masked, rotationally-symmetric, 9 $\times$ 9 pixels neighborhoods, as described in Section 3.5.3. Parzen windowing in all of the examples uses a local Gaussian random sampling ( $\sigma_{\mathrm{spatial}} = 40$ pixels) in the image domain with $1000$ samples (i.e., $\vert\mathcal{A}_t\vert = 1000$), as explained in Section 3.5.2. For certain experiments, we simulate i.i.d. additive Gaussian noise. We recompute the size of the Parzen window $\sigma$ after each iteration, as explained in Section 3.4. The computation for each iteration of UINTA is $O (\vert\mathcal{A}_t\vert \vert\mathcal{T}\vert \vert\mathcal{N}_t\vert)$. Typically, UINTA takes about $5$ iterations for the restoration. The implementation runs about twice as fast on a dual-processor shared-memory Pentium machine. For $\vert\mathcal{A}_t\vert = 1000$, it takes about $25$ seconds to process a 256 $\times$ 256 pixels image on a Pentium-IV $2.8$GHz dual-processor workstation. The implementation in this chapter relies on the Insight Toolkit [2].

All original (uncorrupted) images have intensities ranging from $0$ to $100$. As a visualization aid for comparing different images/results, the intensities of all images within a set have been consistently rescaled to span the available range of intensities.

Figure 4.5 shows the result of UINTA filtering on the Lena image. UINTA preserves and enhances fine structures, such as strands of hair or feathers in the hat, while removing random noise without imposing a piecewise-constant intensity profile. The results are noticeably better than any of those obtained using other methods shown in Figure 4.1. Figure 4.5 also shows the results of processing an MR image of a human head.

Figure 4.5: UINTA results. (a1),(a2) Noisy images: Lena and MR image of the human head. (b1),(b2) UINTA-restored images after about 5 iterations. (c1),(c2),(e1),(e2) and (d1),(d2),(f1),(f2) show magnified portions of the degraded and filtered images, respectively.

Figure 4.6 shows the result of UINTA processing on electron-microscopy data--Figure 4.7 shows the zoomed insets. These examples show UINTA's ability to adapt to a variety of grayscale features in real images approximated by piecewise-stationary models.

Figure 4.6: UINTA results. (a) Corrupted electron-microscopy image of rabbit retinal cells. (b) UINTA-restored image after 5 iterations.

Figure 4.7: UINTA results. Zoomed insets of the (a) corrupted electron-microscopy image of rabbit retinal cells, and (b) UINTA-restored image after 5 iterations.

The fingerprint image in Figure 4.8 is an example where the degradation involves smudges (blurring), which is clearly not additive noise. UINTA enhances the light and dark lines without significant shrinkage. UINTA performs a kind of multidimensional classification of neighborhoods--therefore some features in the top-left are lost because they resemble the background more than the ridges. For the stopping criteria, we use the relative change in entropy as described in Section 4.5. Figure 4.8 also presents the results with other restoration strategies for visual comparison with UINTA. The piecewise-smooth image models associated with anisotropic smoothing, bilateral filtering, and curvature flow (Figures 4.8(g)-(i)) are clearly inappropriate for this image.

Figure 4.8: UINTA results compared with the state of the art. (a) Degraded fingerprint image with (b),(c) zoomed insets. (d) UINTA restored image with (e),(f) zoomed insets. Zoomed insets of the fingerprint image processed with (g) anisotropic diffusion: $K$=0.45 grayscale values, $99$ iterations, (h) bilateral filtering: $\sigma_{\mathrm{domain}}$=3 pixels, $\sigma_{\mathrm{range}}$=$15$ grayscale values, (i) curvature flow: time step=$0.2$, $5$ iterations, (j) coherence-enhancing diffusion: $\sigma$=$0.1$ pixels, $\rho$=$2$ pixels, $\alpha$=$0.0001$, $C$=$0.0001$, $15$ iterations, (k) unrestricted mean shift [10]: $\sigma_{\mathrm{domain}}$=$2$ pixels, $\sigma_{\mathrm{range}}$=$5$ grayscale values, $5$ iterations, and (l) wavelet denoising [133]: $\sigma_{\mathrm{noise}}$=$14$ grayscale values.

The coherence-enhancing filter (Figure 4.8(j)) does not succeed in retaining or enhancing the light-dark contrast boundaries. It also forces some elongated structures to grow or connect. An unrestricted mean-shift filtering (Figure 4.8(k)) on image intensities (with the PDF not changing with iterations) yields a thresholded image, while retaining most of the noise. Wavelet denoising (Figure 4.8(l)) is unable to get rid of the smudges and excessively smoothes other regions of the image.

Figure 4.9 gives an example of restoring the standard House image [133] corrupted with i.i.d. additive Gaussian noise having variance $10^2$. The wavelet denoising technique yields a lower RMS error for this image, but introduces ringing-like artifacts in smooth regions. Table 4.1 shows the RMS errors with the standard test images of the House, Lena, Barbara, and Peppers [133].

Figure 4.9: UINTA results. (a) House image and its (b) zoomed inset. Zoomed insets of the (c) Noisy image. (d) UINTA filtered image. (e) Wavelet denoised [133] image.

Table 4.1: RMS errors comparing UINTA with the current state-of-the-art wavelet denoisers. Note: The standard test images of Barbara  [133] and Peppers  [133] do not appear in this dissertation. All uncorrupted images have an intensity range between $0$ and $100$.

Example	Initial RMS error	UINTA	[133]	[152]	[128]
Standard image: House	10.0	3.5	2.9	3.1	3.5
Standard image: Lena	10.0	4.6	3.6	3.8	4.1
Standard image: Barbara	10.0	4.8	3.8	4.2	4.5
Standard image: Peppers	10.0	4.5	3.5	3.7	3.9
Hand-drawn curves	25.0	15.4	16.0	18.5	18.0
Simulated fingerprint	10.0	3.4	4.1	4.7	4.7
Simulated range data (head)	1.0	0.35	0.34	0.36	0.5
Reptile	10.0	3.5	2.9	3.0	3.4
Building Facade	10.0	4.5	4.4	5.1	5.4
MRI (with learning)	10.0	3.1	3.4	3.7	3.9
MRI (multimodal)	10.0	3.3	3.4	3.7	3.9

Figure 4.10 shows the application of UINTA to an image of hand-drawn curves (noise $N(0,25^2)$). The noise level is high enough so that thresholding can not yield the original image. UINTA learns the pattern of black-on-white curves and forces the image to adhere to this pattern. However, UINTA does make mistakes when curves become too close, exhibit very sharp bends, or when the noise introduces ambiguous gaps. The wavelet denoised image depicts significant artifacts around the edges, giving a higher RMS error (Table 4.1).

Figure 4.10: UINTA results. (a) Hand-drawn curves with a (b) zoomed inset. Zoomed insets of the (c) noisy image, (d) UINTA-filtered image, and (e) wavelet-denoised [133] image.

Figure 4.11: UINTA results. (a) Hand-drawn curves. (b) and (c) show UINTA filtered images after $100$ and $200$ iterations, respectively.

The entropy reduction associated with UINTA does impose a kind of statistical simplification on the image, and that statistical simplicity corresponds, in many cases, to geometric simplicity. Figure 4.11 shows the results of many UINTA iterations on the hand-drawn image of Figure 4.10(a). UINTA has no explicit geometrical model and yet it gradually smooths out the bends in these curves producing progressively simpler geometric structures. The entropy of straighter curves is lower, because of reduced variability in the associated neighborhoods. The result is qualitatively similar to that of curvature-reducing geometric flows [118,153,117], suggesting a strong link between variational and statistical characterizations of images [188].

Figure 4.12: UINTA results. (a1) Simulated fingerprint image. (b1) Noisy image. Difference between the filtered and the noiseless images for (c1) UINTA and (d1) the wavelet denoiser [133]. (a2) Head range data. (a3) Reptile image [55]. Zoomed insets of the (b2)-(b3) noisy images, (c2)-(c3) UINTA filtered images, and (d2)-(d3) wavelet denoised images [133].

In order to better analyze the behavior of UINTA and compare its performance with state-of-the-art wavelet denoisers, we present results with a diverse collection of synthetic images. We provide examples on the simulated fingerprint image (Figure 4.12(a1)), the simulated range data of the human head (Figure 4.12(a2)), and the synthetic Reptile  image [55] (Figure 4.12(a3)). Table 4.1 shows the RMS errors. UINTA performs better on the fingerprint, almost equally well on the range data and poorer on the Reptile image. Thus, UINTA performs better as a denoiser when it can find sufficiently many patterns in the degraded image to be able to distinguish the degradation from the underlying signal. Indeed, this stems from the stationarity assumption on the MRF model underpinning UINTA. Moreover, the statistical models underlying the wavelet denoisers are empirically derived from photographs, similar to the Reptile image. Figure 4.13 shows a photograph of a building facade that exhibits a certain degree of redundancy. UINTA is able to exploit that to perform almost as well as the best wavelet denoiser in terms of RMS error (see Table 4.1) and with fewer visual artifacts.

Figure 4.13: UINTA results. (a) Building facade image. Zoomed insets of the (b) noisy image, (c) UINTA-filtered image, and (d) wavelet-denoised image [133].

Figure 4.14: UINTA results in a supervised scenario. (a) Image used for learning neighborhood statistics. Zoomed insets of the (b) noisy image, (c) original image, (d) UINTA-filtered image, and (e) wavelet-denoised image [133].

When operating within a specific application domain, UINTA can perform much better by learning from ideal or noiseless-image examples. Figure 4.14 shows a demonstration of this concept on simulated MRI data from the BrainWeb [31] project. We corrupt a head MRI T1 image with i.i.d. additive Gaussian noise and use two other similar, but not identical, images for learning the neighborhood statistics of typical brain MR images. Figure 4.14(a) shows one of the two images representing the nonparametric prior model. This example shows the power of such learning--the UINTA restored image exhibits structures that are barely visible in the degraded version and fares considerably better than the wavelet denoiser, both qualitatively and quantitatively.

Figure 4.15: UINTA results. (a)-(c) Multimodal MR images comprising T1, T2, and PD scans. (d) Zoomed inset of the UINTA-restored image.

The UINTA formulation also generalizes easily to simultaneous restoration of a sequence of images, e.g., multimodal MRI, exploiting the relationships between images to further enhance performance. Figure 4.15 shows an example with multimodal restoration. This entails a simultaneous restoration of T1, T2, and PD images in a coupled manner, treating the combination of three images as an image of vectors, and analyzing PDFs in the combined probability space. Although in this chapter we show results with multimodal images that are well aligned, our experiments suggest that the restoration is fairly robust to minor registration errors. Here again, UINTA fares better than the wavelet denoiser.

We now provide qualitative comparison between UINTA and the NL-means algorithm [23]. The updates in both methods have similar mathematical form. However, there are several important differences. While UINTA is iterative and formulated in an information-theoretic context, NL-means is not iterative and relies on optimal nonparametric regression estimation [156]. The derivation of the NL-means update is closely related to the ICE update. UINTA relies on a stopping criterion based on an information-theoretic or statistical optimality metric. Concerning the engineering aspects, while UINTA chooses the Parzen sample $\mathcal{A}_t$ stochastically from a Gaussian PDF over the image coordinates, NL-means chooses $\mathcal{A}_t$ from a small square neighborhood. More importantly, while UINTA dynamically tunes the Parzen-kernel parameter $\sigma$ via a data-driven manner optimality metric, NL-means exposes this $\sigma$ as a free parameter. NL-means relies on a heuristic to tune $\sigma$ to approximately $10$-$15$ times the estimated standard-deviation of the (assumed) i.i.d. Gaussian noise in the image. UINTA, on the other hand, automatically chooses $\sigma$ to be close to the noise level.

Figure 4.16(a1)-(a4) gives some images denoised by the NL-means algorithm. Each of the original images was corrupted with $10$% i.i.d. additive Gaussian noise. For an accurate comparison with UINTA, we used the same 9 $\times$ 9 pixels neighborhood mask in NL-means as we do for UINTA (see Figure 3.3). We found that choosing $\sigma$ as $10$ times the noise level leads to extreme smoothing/averaging that destroys all significant image details. We choose $\sigma$ to be $6$ times the noise level. The RMS errors are: $3.2$ for the House , $4.75$ for the building facade, $3.8$ for the simulated fingerprint, and $15.9$ for the hand-drawn curves. Comparing these values with those in Table 4.1, we observe that UINTA produces better results on the three images other than the House  image. Moreover, the edges in the NL-means-restored images appear noisy. Figure 4.16(b1)-(b4) and Figure 4.16(c1)-(c4) show the difference between degraded images and restored images (termed method-noise [23]) for NL-means and UINTA, respectively. While the method noise in UINTA has a wider intensity range showing poor performance for unique image structures, e.g., corners, that in NL-means appears more correlated along long edges. Figure 4.16(d1)-(d4) and Figure 4.16(e1)-(e4) show the difference between restored images and original images for NL-means and UINTA, respectively--for a perfect restoration, these images would comprise all zero values. We can observe the higher correlation along long structures in the NL-means-restored images a bit more clearly as compared to the method-noise images. Note: the method-noise images in Figure 4.16(b1)-(b4) and Figure 4.16(c1)-(c4) can be obtained by negating the images in Figure 4.16(d1)-(d4) and Figure 4.16(e1)-(e4) followed by addition of the noise.

Figure 4.16: Comparison of UINTA with NL-Means. (a1)-(a4) Images denoised via NL-means [23]: cropped and zoomed for comparison with UINTA-restored images shown previously. (b1)-(b4) Difference between the degraded images and the restored images (method-noise [23]) for NL-means [23]. (c1)-(c4) Difference between the degraded images and the restored images (method-noise [23]) for UINTA. (d1)-(d4) Difference between the restored images and the original images for NL-means. (e1)-(e4) Difference between the restored images and the original images for UINTA.