Restoration via Entropy Reduction on Markov Statistics

UINTA models images as derived from stationary MRFs. Thus,

$$\forall t \in \mathcal{T}, P (\tilde X_t, {\bf\tilde Y}_t) = P (\tilde X, {\bf\tilde Y}) = P ({\bf\tilde Z}).$$ (106)

Degraded images, by definition, have less regularity in the Markov statistics as compared to their original nondegraded versions. This increases the randomness associated with the Markov PDF $P ({\bf\tilde Z})$ or the conditional Markov PDFs $P (\tilde X \vert {\bf\tilde y}_t)$ at each pixel $t$. In simpler words, degradations reduce the predictability of pixel values from the values in their neighborhoods. UINTA attempts to counter the degradations by increasing this regularity. One measure of randomness associated with a PDF is the entropy [34] and, hence, UINTA attempts to restore images by reducing the entropy of the stationary Markov PDF $P ({\bf\tilde Z})$.

The choice of entropy as the optimization measure is also consistent with several other observations. If we assume i.i.d. additive zero-mean noise, the addition of two independent random variables, i.e., the signal and additive noise, increases the entropy [154,34]. Entropy reduction reduces the randomness in corrupted PDFs and tries to counteract noise. Of course, continued entropy reduction might also eliminate some of the normal variability in the signal (original image). However, we have found that nondegraded images tend to have very low entropy relative to their degraded counterparts. Therefore, entropy reduction first affects random degradations substantially more than the signal. Furthermore, the entropy reduction is limited by the entropy-based stopping criterion, as described in Section 4.5.

The UINTA strategy is to reduce the entropy $h ({\bf\tilde Z})$ of the Markov PDF by manipulating the pixel values $\{ \tilde x_t \}_{t \in \mathcal{T}}$. This requires the entropy of the Markov PDFs $P ({\bf\tilde Z})$ to be expressed as a function of each pixel value $\tilde x_t$. This follows naturally from the Parzen-window density-estimation technique, based on the proposed adaptive-MRF image model. Thus, all pixel-neighborhood values ${\bf\tilde z}_t = (\tilde x_t, {\bf \tilde y}_t)$ in the image are observations that participate in defining the PDFs.

To update every pixel value in order to reduce the entropy, UINTA employs a gradient-descent strategy. Note that a gradient descent on $h ({\bf\tilde Z}) = h (\tilde X, {\bf\tilde Y})$ has components corresponding to both the center-pixel value $\tilde x_t$, and the neighborhood values ${\bf\tilde y}_t$. Thus, at each pixel $t$, a gradient-descent scheme can potentially update the entire region $( \tilde x_t, {\bf\tilde y}_t )$. In practice, however, we update only the center-pixel value $\tilde x_t$, i.e., we project the gradient onto the direction associated with the center pixel.