Project 5: Snakes: Deformable
Contour Segmentation
Manasi Datar
1. Overview
In this project we implement the deformable contour segmentation approach due to Trucco & Verry. This approach, also known as deformable snake segmentation optimizes a user-specified contour to segment an image. Such user interaction results in a robust algorithm for segmentation, but does not preclude complete automation of the process. The following sections discuss the theory of deformable contours and one possible greedy implementation. We further apply this algorithm to various contour images and examine the effects of various factors governing the optimization process. Finally, we conclude with a few remarks about the details of the implementation.2. Deformable Contours:
Snakes
The snake model is desribed as a controlled continuity model under the
influence of image forces. Internal forces control the bending
characteristics of the line, while image forces, such as the gradient, serve to push the snake toward
image features. When the snake coordinate vector is defined
parametrically as v(s) = (x(s), y(s)) , the total energy of the snake
is defined asE*snake = ∫ Esnake(v(s))ds = ∫ Eint(v(s))ds + ∫ Eext(v(s))ds
where Eint and Eext are the internal and
external energy terms, respectively. Minimization of this combined
energy functional implies that the internal energy works to ensure that
a consistent shape representation for the snakse, while the external
energy term attempts to change the shape properties of the snake (as it
evolves) to capture relevant features.The internal energy of a snake seeks to preserve the continuity and curvature through the evolution process. Thus, the internal energy can be defined as:
Eint
= Econt + Ecurv
Each of the above energies can be computed in terms of the contour
points defined by v(s). The continuity can be quantified as the rate of
change of the contour points, whereas smoothness is the change in
curvature, and can be measured as the rate of change of the tangent
vectors along the contour. These energies are therefore given by the
first and second derivatives of the contour v(s).The external forces mostly depend on the application. For image segmentation, we want the deformable contour to find a particular feature within an image. Typically, features have well defined edges, and we wish to evolve the snake to match these edges. Thus, the external energy influencing the evolution of the snake must be related to this feature ! The most relevant and obvious measure of edges in an image is the gradient magnitude, and this is precisely what is used to denote the external energy. Mathematically,
Eext
= -|| 
I ||
Since we wish to minimize both internal and external energies, we
construct the external energy to equal the negative of the gradient
magnitude.I ||Variational calculus can be used to formulate a solution to minimize E*snake and evolve the snake to match the desired contour in the image. However, this method is complex and it is tough to obtain a stable implementation.
3. Snakes: Discrete
Implementation
Alternately, we can devise a discrete solution to the snakes evolution
problem as follows.We model the contour as c = {p1,p2,...,pN}, where pi represents the x- and y-coordinates of the ith point on the contour. Now, we can minimize the following equation in a
Each of the energies related to the snake can be defined using discrete differences (as shown in class). The continuity energy can be defined in terms of the distance between points along the snake, as compared to the average distance between neighboring points. Thus, this term will seek to keep points along the snake as close together as possible.
The curvature energy is nothing but the second derivative at each point along the snake. Finally, each of these energies are normalized by division with the largest value in any given neighborhood.
As discussed earlier, the external energy is nothing but the negative gradient magnitude. However, a pre-processing step is required prior to computing the image gradient. Gaussian blurring is applied as a pre-processing step. This ensures a greater range for the deformable snake to latch onto image contours. Normalizing the gradient magnitude can lead to detection of spurious or weak edges and should be avoided. The influence of the external energy term can instead be controlled by the constant γ related to it.
4. Results: Synthetic Data
*hyperlink to image
in actual size
This section describes the various experiments and provides a
discussion on the corresponding observations. Most of the
experiments are performed on a U-shaped image. This image is
challenging and provides avenues to stretch the implementation and
examine the contour evolution over a region involving concavities.4.1 Contour Initialization and
Number of Points
As a first step, we experiment with different initial contours.
All other parameters remaining the same, the initial contour plays a
very important part in the convergence of the snake evolution process.
The images below describe the effect of generic contours
versus user-defined contours. Figures in the left column indicate the
initial contour in red, while the animation in the middle column shows
the evolution of the deformable snake at regular intervals in the
optimization process. The final contour is displayed in green in
figures of the right column.We try to automate the initialization process by using geometric
primitives such as a circle and an ellipse as initial contours. As we
can see in the first and second rows of the figure above, this approach
does not converge very well ! In fact, the final contour due to the
second initialization actually loops on itself !! The constants also
play a major part in the eovlution, but this effect is not biased due
to those values since they are the same for all three results displayed
below.
One the other hand, user-defined contours
result in much better segmentation. Thr third row displays the result
of a meticulously initialized snake with a dense collection of points
whereas the fourth row has a sparsely populated initialization and yet
generates a decent segmentation. This does not imply that the results
are invariant to the number of points on the initial contour. It is
important to note that the granularity of the feature governs the
required number of points on the initial snake. In the last row of the
figure below, the user-initialized snake has lesser number of points on
the outer curve of the U, but has a dense population of points near the
concave section. This sensitivity is important to generate good results
using deformable snakes.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
From the above figure, we can conclude that the deformable snake algorithm is highly sensitive to initialization of the snake. One also needs to understand the granularity of the feature being segmented while deciding on the number of points and their distribution in the initialization process.
4.2 Effect of Gaussian Smoothing
The
second experiment involves an examination of the external image forces
governing the optimization process. We keep all parameters constant,
and only vary the amount of Gaussian blurring applied to the image
prior to computing the gradient magnitude. The figures below display
the quiver plots of the gradient image after different amout of
blurring. As we can see from these images, apart from the scaling
factor γ the Gaussian blurring plays an important role
in guiding the
contour evolution. |
 |
 |
Synthetic
data: Gradient image after Gaussian blurring (
Note the increasing range for the snake to move)
(l) σ = 0.25 (c) σ = 0.5, (r) σ = 1.0
(l) σ = 0.25 (c) σ = 0.5, (r) σ = 1.0
The amount of blurring determines the size of the region in which the snake can move under the influence of the external forces. As we can see from the images below, blurring with a wider Gaussian kernel enlarges the range of possible locations any point on the snake can move to. While this may be an advantage in most cases, it may turn out to be a flaw in cases where we have overlapping objects, or the discrimination at an edge is poor (e.g. MRI images of brain tissue).
4.2 Effect of Continuity constant α
This experiment is designed to look at the effect of the constants α, β, and γ on the segmentation results. The parameter α governs the continuity. For contour like the 'infinity' symbol in the figure below, it is important that the snake deform as much as possible to achieve the desired shape. α however acts to keep the points along the snake close together and reduces its 'elasticity'. Thus, a low value of α is desirable to segment shapes like the 'infinity' symbol. In the figure below, we see the user-initialized snake to the left, along the with evolution animation in the center and the final contour on the right. A high value of α will result in a much more 'circular' segmentation ! |
 |
 |
Synthetic
data:
(l) Initial Contour*, (c) Snake Evolution*, (r) Final Contour*
From the above figure, we can also conclude that the deformable snakes algorithm is fairly robust to noise added into this image using the GIMP software. Even though the noise results in spurious edges and holes in the contour, energy minimization only happens in the center and the snake evolves to fit this contour correctly.
4.4 Open (Illusory) Curves: the effect of Curvature constant β and Image constant γ
In the process of looking at shapes with missing information, we also discover the actions of the curvature constant β, and the constant γwhich governs the image force. In the figure below, we examine the results of snake evolution on a shape with missing edges which simply imply a boundary to the human visual system. In the figure below, the gray level contour has gaps in it, and we wish to segment out the entire contour as a single feature. As we can see from the second row, the deformable snake does a decent job with the right choice of parameters.In the first row, the initial snake is roughly a user-initialized ellipse which is slightly biased toward the protrusion on the left as compared to the one on the right. The image in the center shows the animation of snake evolution with a relatively high value of β. The final contour on the right shows that the snake has not been able to recede into the right hand corner as well as it did on the left side. This behavior can be attributed to the value of β which governs the smoothness of the snake, and does not allow bending around corners unless the image forces are strong enough to govern the evolution. It must be noted that this experiment was run with a relatively low value of γ because of which the image force was not able to pull the snake into the right hand corner !
 |
 |
 |
 |
 |
 |
The images in the second row are results from a more balanced configuration of β and γ. This balance allows the snake to follow the contour into both corners and achieve a much better segmentation. Of course, the user-initialization of the snake near the desired contour also helps ! In general, it seems preferable to initialize the snake just outside the feature to be segmented and let it work its way on to the contour. Perhaps, this is due to the nature of our discrete implementation as there is really nothing in the theory which implies such a preference.
This experiment also demonstrates the difficulty in segmenting corners. This problem can again be attributed to the inclusion of the curvature constraint, and its accompanying constant β. The inherent trade-off between trying to maintain smoothness and squeezing into a corner results in the snake rounding off at corners as seen in the results above. The remedy would be to possibly make β a function of each point-location along the snake such that as this point approaches a corner the corresponding β becomes lower in value to allow this point to propagate into the corner.
5. Results: Real Data
*hyperlink to image
in actual size
We briefly look at an example of a 2D CT image of the abdomen and
try to use the deformable snake model to segment the lung. In the
figure below, the left image shows the initial contour, the animation
of the snake evolution process can be seen in the middle image and the
final contour of the lung can be seen in the right image. For real
data, it is tricky to get the balance of parameters right. Also,
initializing the contour very close to the feature we wish to segment
is the key.
 |
 |
 |
Real
data:
(l) Initial Contour*, (c) Snake Evolution*, (r) Final Contour*
Unless there is an automated way to propagate the segmented contour onto the next slice (in a 3D scenario) it would be a tedious process to manually initialize the snake in each frame. One method might be to register each frame to the first one (to get rid of motion artifacts, etc.) and then propagate the segmented contour to the corresponding voxels in subsequent slices and optimize the snake further to capture the contour in that slice.