CS 7960 - Project 5
Deformable Snakes
| Name: | Jonathan Bronson |
| Date: | April 15, 2010 |
Table of Contents |
|
1. Introduction
Overview
The goal of this project is to implement and understand a form of
deformable contours, as described in Trucco and Verri. They show
that an otherwise continuous formulation can be solved discretely
using a simple greedy algorithm. While not optimal, it displays the
type of behavior we expect from deformable contours, and is excellent
as an introduction to such techniques. After reading this document it
should become clear how to formulate the energy minimization problem
that leads to the snake-like behavior. Further, the limitations to this
simple greedy approach should become clear, motivating a more robust
formulation for future projects.
Design Choices
|
For this project and future projects for this course, I will continue
to build a comprehensive framework application. This application allows
me to easily open and save images using a GUI, and quickly access
all previous projects through their respective toolbox's. Using an MDI
display model allows myself or another user to have any number of
images open at one time, and compare results of methods more easily.
My implementation is coded in C++, using Qt for the GUI, and VISPack for image handling. I use CMake to automate the build process and except for some limitations of VISPack, the application is completely cross-platform. The software has been successfully built on OpenSUSE, Ubuntu, and Windows 7. |
 |
2. Deformable Contours
At a high level, the idea behind deformable contours, or Snakes, is both ingenious and relatively simple. We would like to have a contour, a closed loop or line, that deforms locally to move towards some goal. If the goal is to segment objects with strong edges, we want the contour to travel until it finds edges and stop. Obtaining this type of behavior from deformable contours usually involves formulating an energy minimization problem. In this way, the snake will continually 'move' toward a lower energy level. A commonly found energy function is:
In this formulation, the total energy of the contour is the integral of three energy terms evaluated every location along the parameterized contour. These three terms, Econt, Ecurv, Eimage account for continuity, smoothness, and image forces, respectively. The continuity and curvature terms enforce internal properties of the contour while the image term helps to guide it to the desired location in the image. The following subections describe each term in more detail.
2.1 Continuity Term
An intuitive way to understand the continuity term, is that it strives for uniform sampling of the contour. Thus, if we must sample our continuous contour with a fixed set of discrete points, these points should be spaced as equally as possible along the contour. Mathematically, this means our parameterization of the contour should be as uniform as possible. We can write this as:
This is a continuous definition of the first derivative with respect to the parameterization, however, and in our implementation we must use a discrete formulation. For simplicity, we can use backwards differences between points as an approximation. We write this as:
As Trucco and Verri mention in their work, this method could lead to the formation
of clusters of snake points. This is because each point is independently seeking to
minimize these terms. By incorporating a global term, 
, the
average distance between point pairs, we can force a smoother behavior. We write
this new formulation as:
This new energy term penalizes changes that too quickly deviate from the average, ensuring a smoother global change.
2.2 Smoothness Term
The smoothness term is another internal contour force designed to ensure more consistent behavior. In particular, it helps to avoid high oscillations that are most likely due to noise in the image. It does this by penalizing extremely high curvatures. Since curvature is defined from second derivatives, we can use them to form the smoothness energy term as:
This is simply a discrete approximation to the magnitude of the second derivative.
2.3 Image Term
The previous two terms helped restrict the shape of the deformable contour to a reasonable subset of all possibilities. What is still lacking is a force to hone in on features of the image. This term provides that force. For our purposes, we are interested in detecting edges, and hence we want to minimize the cost of being near edges. We accomplish this by using the gradient magnitude such that:
In this way, as points of the snake reach edges in the image, the negative energy will reduce the total energy and cause the snake to slow down or stop alltogether.
3. Greedy Algorithm
The algorithm we implement is based on the greedy algorithm proposed by Trucco and Verri. The idea behind it is that to minimize the global energy of the contour, it is sufficient to minimize the energy of each discrete snake sample. In this way, local optimizations should lead to a better global solution. We can phrase this discretely by minimizing the energy functional:
In order to minimize this functional, our algorithm performs as follows:
1: Compute average distance between points,
2: For i = 1,...,N, move point pi to local minima.
3: For i = 1,...,N, estimate curvature k as:
If k is local maxima, and k > threshold, set 
4: If more than some fraction of points move, repeat.
This algorithm is appealing mostly due to its simplicity. We can't expect it to perform optimally, but well enough to demonstrate the purpose and usefullness of deformable contours.
When we say local minima, a small window of potential locations must be evaluted. For this project, we use 3x3 and 5x5 neighborhood windows. This helps keeps the contour movement somewhat stable, while minimizing computational complexity.
4. Experiments
This section contains a subset of the experiments I ran to evaluate the performance of our discrete and greedy implementation of deformable contours. These experiments were chosen because they highlight some of the important issues worth considering when dealing with deformable contours.4.1 Control Point Count
Probably the most obvious drawback to this approach, is the discrete sampling. For even basic shapes, such as this circle and square, it can difficult or impossible to get a good representation. For the circle, the problem is not so bad, but for sharp objects like the rectangle, corners can be cut off quite deeply. Further, there's no way to know apriori how many samples are good enough. If I needed to use this system on a regular basis, I would use as many control points as computationally possible, at all times.
4.2 Concavities
 |
 |
 |
|
| initial a=1.0, b=1.0, g=1.5 |
final a=1.0, b=1.0, g=1.5 |
initial a=0.2, b=0.2, g=1.5 |
final a=0.2, b=0.2, g=1.5 |
As shown in the subset of images above, without extensive blurring to the underlying image, capturing concavities is extremely difficult, if not impossible. The problem is there is no term in our energy formulation that moves the contour inward, such as a normal force. Evening setting alpha and beta to zero would not fix the problem, since then no force is felt on the points.
4.3 Open Contours
 |
 |
 |
 |
| initial a=1.0, b=1.0, g=1.5 |
final a=1.0, b=1.0, g=1.5 |
initial a=1.0, b=1.0, g=1.5 |
final a=1.0, b=1.0, g=1.5 |
The behavior of this implementation on open contours exposes a problem with the formulation. In particular, the term we thought was supposed to enforce smoothness only works on straight gaps, such as the false rectangle above. For the broken circle on the right, smooth should look like a circle, not a gear. Unfortunately, no values of alpha or beta can fix this. The problem boils down to a breakdown in our assumption that locally optimal choices will lead to globally optimal solutions. In this case, neighboring points need to share curvature information to remedy this problem.
 |
|
| initial a=1.0, b=1.0, g=1.5 |
final a=1.0, b=1.0, g=1.5 |
The images above of the two circles side by side shows that this tight fitting is not necessarily a bad thing. In cases like this, a tight convex hull may actually be what we want. However, we should ideally be able to specify when this is the case.
4.4 Real Image
 |
|
| initial a=0.3, b=0.6, g=0.9 |
final a=0.3, b=0.6, g=0.9 |
 |
 |
| initial a=0.1, b=0.8, g=0.1 |
final a=0.1, b=0.8, g=0.1 |
As one might expect, for real-world images, tweaking the parameters to obtain an ideal shape match is extremely difficult. Unlike the previous examples, the strength we give the image term really effects the contour evolution. In a grayscale image, there is a balance that must be obtained between internal and external forces. Finding that balance is essentially trial and error by the user. For a repeated task, one might hope to aid the user by learning a relationship between the parameters they ultimately settled on, and the gray level pixels along the parameterized contour.
5. Conclusion
After experimentation, I've come to the conclusion that the discrete implementation of deformable contours, as described by Trucco and Verri is insufficient. The force terms, while well intentioned, do not actually induce the behavior they were designed to. The ideas and reasoning behind them make perfect sense. It is there implementation as described that is flawed.
First, the continuity term does help distribute points more evenly around the contour. However, it also induces a shrinking behavior of the contour. These two effects really should not be intertwined together.
The bigger problem, however, is the smoothness term. It's designed to smooth out the contour by keeping the curvature, and hence the second derivative small, an excellent idea. The problem is, the snake point that suffers from a discontinuity is not necessarily the one that can fix the problem. For instance, if one point is stuck to an edge, and it's neighbor continues on into a void, it is the neighbor that must remedy this by backing out. Instead, under the Trucco and Verri scheme, the point in the void evaluates itself to have a good smoothness measure, and stays put. The point on the edge has a terrible smoothness measure, but correctly stays on the edge it was designed to find.
I propose we remedy this by replacing the smoothness term by the following:
This is an average of the two neighbors second derivatives with the local one. This new form makes more sense, since moving one point actually effects three derivatives at once. This will allow points to move to correct problems with points which cannot move, for other reasons.
Finally, there really needs to be a term whose soul purpose is to drive the movement of the contour, in the absence of information. This is perhaps a more controversial point, but one could argue that if there are no forces on the contour than there is nothing of interest in the area of the image, and the contour needs to move on.
All of these criticisms aside, the problem is clearly just this implementation. Despite its shortcomings, deformable contours in general can be very powerful. It is surprising that such a simple implementation can give as good results as it does, compared to the complexity of better methods. Finally, it's clear that any one application may want to specify custom energy functionals, as there are an uncountable number of ways one might want these snakes to behave.