Project 4: Active Shape Models (ASM)

Manasi Datar

1. Overview

In this project we explore statistical shape modeling using the Active Shape Model (ASM) proposed by Cootes & Taylor. ASM is a satistical model that allows the user to comment about the variation present in an ensemble of shapes using corresponding landmarks on each of the shapes.

The following sections discuss the construction of a statistical shape model using ASM and introduce the subsequent analysis. We also look at results of applying the above analysis to a set of synthetic shapes and on real data involving a study of the corpus collosum. We conclude with a few remarks about the applications of ASM and details of the implementation.

2. ASM: a tool for statistical shape analysis

Statistical shape analysis uses geometrical information from a set of shapes to compute statistics describing the properties of the shape ensemble. Such statistics can further be used to study the variation in shapes in a given population, test differences between population groups (e.g. normal and pathological anatomy) and also to test other similar for other hypothesis tests.
One of the important aspects of analyses such as those described above is the notion of "distance" between shapes. Given a measure of distance between shapes, we can proceed to compute a statistical model (e.g. "average" shape) from the given ensemble and further analyze the complete ensemble as described above, using our model as a template (or reference shape).

The Point Distribution Model (PDM) is a prominent model used for statistical shape analysis, and is the static form of the ASM. Cootes and Taylor describe the PDM of a shape as a collection of ordered landmarks. These landmarks are typically chosen to be points of anatomical/geometrical interest. Such selection and ordering of landmark points implies correspondence across shapes and simplifies further analysis. The following discussion details the construction of the PDM as described by Cootes & Taylor:
In a given ensemble of N 2D shapes, imagine the i-th shape to be represented by n landmark points, each of the form (x,y). Thus, the i-th shape can be written as an order vector of length 2n as shown below:

Further, assuming that the shapes are in reasonable alignment, we can compute a vector representation of the mean shape as follows:

PDM uses this mean shape as a template/reference or "model" to compute and study the variation in the geometric characteristics of the given ensemble of shapes.

3. Population Analysis using ASM

As explained in the material due to Stegman & Gomez, further analysis (or shape decomposition) can be carried out using Principal Component Analysis (PCA). Consider the covariance matrix w.r.t to the mean shape representation constructed in Sec. 2 , given as follows:

This matrix represents the variation-from-the-mean for each landmark point. PCA then seeks a linear transformation such that the resulting landmark representation is devoid of correlation between landmark points. Further, we can pick a sub-space defined by eigen-vectors with the most prominent eigen-values to effectively reduce the dimensionality of the shape representation by an order-of-magnitude, and still study effectively, the geometric variation in the underlying ensemble.
The problem of choosing the appropriate sub-space can be resolved heuristically by choosing the eigen-vectors which cumulatively describe a certain percent of the overall variation in the shapes. For this report, this cut-off is set to 90% as described in the project instructions.

An appropriate sub-space can be denoted as a collection of the corresponding eigen-vectors as follows:

where P is a matrix of the first k eigen-vectors chosen as described above.
Now, any shape from the given ensemble can be approximated as a weighted sum of deviations of these k modes from the mean shape as shown below:

where b = (b₁, b₂, ... , b_k)^Tis a vector of weights, one for each eigen-vector. The sub-space P is orthogonal and so b is easy to compute and gives us a representation of each shape in a reduced-dimension space. Furthermore, we can use the above analysis to explore the shape variation in each of the modes using a user-defined parameter s, applied to the standard deviation and ranging from -1 to +1, as shown below:

The above expression allows us to create all the variations of the shapes described by the i-th mode, corresponding to eigen-value λ_i.

4. Results: Synthetic Data

^*hyperlink to image in actual size

This section discusses the creation (from Sec. 2) and analysis (from Sec. 3) of ASM on a set of synthetic shapes. The shape chosen was a square of side-length 2, centered about the origin, and each of the corners was chosen to be a landmark - giving a total of 4 x 2 = 8 values to represent each shape. An ensemble of 25 shapes was used to test the algorithm under various transformations, with and without noise. The following discussion presents details of each of the experiments and shows the corresponding results.

4.1 Anisotropic Translation

The first experiment involves translation of each landmark, independently in each dimension. Thus, each of the 8 components of the shape vector are perturbed independently. Since the relative distance between landmarks describing a particular shape does not change by much, the mean shape is still a square of side-length approximately equal to 2. The table below shows contours of the various squares in the ensemble in shape space, along with the mean shape (leftmost figure)^*. The image in the center shows the distribution of the eigen values^* and the figure to the right is the correlation image.
The correlation image is not especially interesting, however it does reiterate the experimental setup since the corresponding dimensions are highly correlated (i.e. all x-values are correlated and so are all y-values.

Synthetic data: (l) contours in shape space, (c) distribution of eigen-values, (r) correlation matrix image

The eigen value distribution from the table above indicates that there are 2 major modes of variation. The table below depicts the variation in the shape in these modes. We can see that the major modes of variation capture the translation introduced in the experiment to a great extent, with the first mode primarily capturing horizontal translation while the second mode mostly involves variation due to vertical translation.

Modes of variation for the synthetic dataset under anisotropic translation
	-1.0 σ	-0.5 σ	ľ	0.5 σ	1.0 σ
m1
m2

4.2 Isotropic Scaling

The second experiment involves isotropic scaling of each shape, thus shifting all of the landmarks outwards. The table below shows contours of the various squares in the ensemble in shape space, along with the mean shape (leftmost figure)^*. The image in the center shows the distribution of the eigen values^* and the figure to the right is the correlation image.
The correlation image makes sense since isotropic scaling would mean that neighboring landmarks would undergo similar changes and would thus be correlated. Thus we see large sub-regions with high correlation along the diagonal.

Synthetic data: (l) contours in shape space, (c) distribution of eigen-values, (r) correlation matrix image

The eigen value distribution from the table above indicates that there is only 1 major mode of variation. This is imperative since we apply isotropic scaling ! The table below indicates the changes in scale as captured by the first mode of variation.

Modes of variation for the synthetic dataset under isotropic scaling
	-1.0 σ	-0.5 σ	ľ	0.5 σ	1.0 σ
m1

4.3 Translation + Scaling

The third experiment involves a combination of translation and scaling from the above experiments. The table below shows contours of the various squares in the ensemble in shape space, along with the mean shape (leftmost figure)^*. The image in the center shows the distribution of the eigen values^* and the figure to the right is the correlation image.
The correlation image is a combination of the checkerboard image seen in the translation experiment, with the blocky image from the scaling experiment. This is probably due to the fact that translation and scaling are independent operations and hence their combination has an additive or subtractive effect (depending on the direction of translation and the nature of scaling) on the correlation between landmarks. We can observe this fact in the correlation image shown to the right in the table below.

Synthetic data: (l) contours in shape space, (c) distribution of eigen-values, (r) correlation matrix image

The eigen value distribution from the table above indicates that there are 3 major modes of variation. This is again a combination of the 2 independent translations and a single isotropic scale applied ! The table below indicates the changes in the shapes along the 3 major modes. We can see that these form a combination of horizontal and vertical translation, along with the effect of scale.

Modes of variation for the synthetic dataset under translation + scaling
	-1.0 σ	-0.5 σ	ľ	0.5 σ	1.0 σ
m1
m2
m3

4.4 Noise

The fourth experiment is designed to inspect the effect of noise on the ASM. This experiment involves adding a zero-mean Gaussian noise with a very small variance to each of the landmark points after they are translated and scaled. The table below shows contours of the various squares in the ensemble in shape space, along with the mean shape (leftmost figure)^*. The image in the center shows the distribution of the eigen values^* and the figure to the right is the correlation image.
It is heartening to see that the correlation image is still a combination of the checkerboard image seen in the translation experiment, with the blocky image from the scaling experiment. This implies that the ASM is robust to noise, and can still produce a model that distinguishes shape variations due to the underlying translation and scaling even in the presence of noise.

Synthetic data: (l) contours in shape space, (c) distribution of eigen-values, (r) correlation matrix image

The eigen value distribution from the table above indicates that there are 3 major modes of variation. We can see from the table below that these modes correspond to shape variation introduced due to translation and scaling. This provides further evidence that the method is robust to noise.

Modes of variation for the synthetic dataset under translation + scaling + noise
	-1.0 σ	-0.5 σ	ľ	0.5 σ	1.0 σ
m1
m2
m3

4.5 Rotation

The fifth and final experiment looks at the effect of rotation on the ASM. In this experiment, each of the squares is randomly rotated. The table below shows contours of the various squares in the ensemble in shape space, along with the mean shape (leftmost figure)^*. The image in the center shows the distribution of the eigen values^* and the figure to the right is the correlation image.
The correlation matrix is somewhat symmetric in nature. This is expected since rotating a square changes the relative positions of the landmark points, but does not affect the symmetry of the shape itself. As such, the checkerboard pattern from the translation experiment is repeated, albeit in blocks representing a group of landmarks rather than each individual x- or y- value.

Synthetic data: (l) contours in shape space, (c) distribution of eigen-values, (r) correlation matrix image

The eigen value distribution from the table above indicates that there are 2 major modes of variation. This does not quite fit, since planar rotation should be sufficiently explained by just one mode of variation. However, it must be noted that the rotation was not applied w.r.t the center of the square, and hence leads to a shift and slight scaling of the shapes, as shown in the leftmost image in the table above. The mean shape is thus smaller than the original square with side length 2. Inspecting the two major modes of variation in the table below, we can see that they represent nearly identical angles of rotation but at different scales.

Modes of variation for the synthetic dataset under anisotropic translation
	-1.0 σ	-0.5 σ	ľ	0.5 σ	1.0 σ
m1
m2

This observation reiterates the need for a pre-processing step involving image alignment. The method may be robust to noise, but is definitely sensitive to initial alignment of the shapes in the ensemble.

5. Results: Corpus Callosum

^*hyperlink to image in actual size

The table below shows the ensemble contours in shape space for the corpus callosum data^*, along with the mean shape overlaid in green. The graph^* in the middle depicts the distribution of the eigen-values and the image to the right shows the correlation image for the various landmarks.

Corpus Callosum data: (l) contours in shape space, (c) distribution of eigen-values, (r) correlation matrix image

It turns out that the first 5 modes are sufficient to account for 90% of the total variance in the dataset. Hence, we analyze the variation in the shapes for each of these modes in the table below. Even though the shapes are aligned in the atlas space (using AC - PC points), rotation seems to feature in all the major modes of variation. If we analyze the variation sans rotation (only by visual inspection at the moment...), it appears that the following characteristics vary significantly:
1. The first mode seems to vary the eccentricity of the shapes. As we move from -1.0σ to 1.0σ along this mode, we see that the corpus callosum becomes "flatter"
2. The second mode seems to correspond to the size of the cross-section of the shapes. This should not be mistaken for pure scale, though that might be a component too.
3. The third mode varies the curve at the extremes of the shapes, or changes the uniformity of the cross-section. As we go from left to right, the cross-section changes from being of uniform size (like a bent pipe) to being different at various locations along the shape.

Modes of variation for the Corpus Callosum dataset
	-1.0 σ	-0.5 σ	ľ	0.5 σ	1.0 σ
m1
m1	rotation + eccentricity
m2
m2	rotation + size of cross-section (thickness)
m3
m3	rotation + uniformity of cross-section
m4 m5

	complex variation

The fourth and fifth modes imply complex variation and are not as simple to explain. Perhaps with better alignment (e.g. Procrustes analysis), we could remove the effects of external transformations like rotation and scale to understand the underlying internal shape variation better.

6. Conclusion

ASM is a simple and robust tool for statistical shape analysis. Once correspondences are established across shapes, this method is readily extended to 3D and performs well. However, the problem of establishing correspondences on 3D surfaces is hard and demands a whole new method of placing lamndmarks. Another problem is related to the number of modes to select following PCA. Heuristic measures such as the one suggested in the project instructions are o often used, along with domain knowledge. However, parallel analysis seems to be a promising statistical tool to determine the number of modes to choose for analysis. Finally, shape alignment is cruicial for the method to perform well in real applications.

7. Implementation Notes

For this project, I decided to switch to Matlab for the following reasons:

My research at SCI is based on the 3D version of the ASM and hence I already had access to C++ code (Project: ShapeWorks) for most of the analysis presented here. We have been through quite a few iterations of this code and however much I tried, I could not write a version much different from ShapeWorks.
I was really interested in the Corpus Callosum data which was given as a Matlab workspace !

The in-built functions in Matlab provide a rich source of code for rapid testing of prorotype algorithms. Also, synthetic data generation is simplified. My code is a collection of .m files and uses in-built Matlab functions for each of the steps in the analysis.