Lit-Tensors

Streamlines used in vector visualization are sometimes hard to interpret because they lack the shading cues that we are accustomed to seeing on surfaces. However, illuminated streamlines (``lit-lines'') have shading and highlights that give information about their direction and curvature, creating the appearance of shiny filaments [40]. In the case of diffusion tensor visualization, we have made tensors opaque and colored based on their anisotropy, but we lacked a means of shading tensors in the presence of a virtual light source. We designed a shading technique termed lit-tensors which can indicate the type and orientation of anisotropy, by following certain constraints:

1. In regions of complete linear anisotropy, the lighting model should be identical to that of illuminated streamlines. Complete linear anisotropy means movement by diffusion is constrained to one dimension, so it is sensible for the lighting model to degenerate to one already developed for vector fields.

2. In regions of complete planar anisotropy, the lighting model should be the same as with traditional surface rendering. The obvious choice for the ``surface normal'' for a planar anisotropic tensor is the third eigenvector, perpendicular to the plane formed by the span of the first two eigenvectors (associated with the largest two eigenvalues).

3. There has to be a smooth interpolation between these two extremes. Since tensor data can exhibit a wide variety of anisotropies, allowing small variations in anisotropy to lead to large changes in shading will probably create a more confusing image.

This can be seen as a problem of how to interpolate illumination between different codimensions. The codimension of the diffusion tensor's representative ellipsoid is two in the linear anisotropy case, and one with planar anisotropy. Previous work [1] has rigorously developed illumination methods for general manifold dimension and codimension, but did not cover cases part-way between different codimensions. Unlike that work, no claim to physical accuracy or plausibility is made for the model presented here; it is just one simple way of satisfying the constraints above.

Lit-Tensor Formulation

We take as our starting point the Blinn-Phong lighting model [4]:

$$I = I_{\mathrm{ambient}} + I_{\mathrm{diffuse}} + I_{\mathrm{specular}}$$

$$= k_\mathrm{a} A_\lambda O_\lambda + I_\lambda (k_\mathrm{d} O_\lambda \mathbf{L}\cdot\mathbf{N} + k_\mathrm{s} (\mathbf{H}\cdot\mathbf{N})^n)$$ (5)

$k_\mathrm{a}$, $k_\mathrm{d}$, and $k_\mathrm{s}$ control the contributions of ambient, diffuse, and specular reflection to the final image. Following Foley et al. [13], we add the subscript $\lambda$ to those variables which vary according to color. For example, there are separate values $I_\mathrm{r}$, $I_\mathrm{g}$, $I_\mathrm{b}$, for the red, green, and blue components of the directional light source. The ambient light color is $A_\lambda$. Instead of representing the intrinsic object color with different $k_\mathrm{a}$ and $k_\mathrm{d}$ for red, green, and blue, we use $O_\lambda$ for object color and keep $k_\mathrm{a}$ and $k_\mathrm{d}$ as separate controls. In our case, the intrinsic object color is determined by any of the methods described in this paper (barycentric maps, hue-balls, or reaction-diffusion textures). $\mathbf{L}$ is the vector pointing towards the directional light source, $\mathbf{V}$ points towards the eye, and $\mathbf{N}$ is the surface normal. Note that instead of using $(\mathbf{V}\cdot\mathbf{R})^n$ for the specular component, where $\mathbf{R}$ is the reflection of $\mathbf{L}$ across $\mathbf{N}$, we are using the ``half-way'' vector $\mathbf{H}$ in $(\mathbf{H}\cdot\mathbf{N})^n$. $\mathbf{H}$ is the normalized average of $\mathbf{L}$ and $\mathbf{V}$, and $n$ is the shininess exponent.

Because a streamline is one-dimensional, at any given point along it there is an infinite set of normals, all perpendicular to the tangent direction $\mathbf{T}$, radiating outwards in a circle. If naively using Equation 5 to illuminate a streamline, one must find the normal which is in the plane spanned by $\mathbf{L}$ and $\mathbf{T}$ to evaluate $\mathbf{L}\cdot\mathbf{N}$. Similarly, another specific normal must be found to evaluate $\mathbf{H}\cdot\mathbf{N}$. The insight which makes lit-lines simple is that one does not need to actually find a specific normal in order to evaluate a dot product with it. With the Pythagorean theorem, the dot product with $\mathbf{N}$ can be expressed in terms of the tangent $\mathbf{T}$:

$$\mathbf{U}\cdot\mathbf{N} = \sqrt{1 - (\mathbf{U}\cdot\mathbf{T})^2}$$ (6)

where $\mathbf{U}$ is either $\mathbf{L}$ or $\mathbf{H}$, for the diffuse and specular terms, respectively.

The relevant property of Equation 6 is that the lighting calculation depends on a tangent vector $\mathbf{T}$ that gives the object's direction, instead of its surface normal $\mathbf{N}$. The direction and orientation of a diffusion tensor is determined by not one, but two vectors: the first and second eigenvectors². Both of these could be interpreted as tangents, but their relative importance is determined by the magnitudes of the corresponding eigenvalues. To control the relative importance of the first two eigenvectors in determining the tensor's orientation, we introduce a parameter $c_\theta$ that characterizes anisotropy type. Assuming that the eigenvalues are ordered $\lambda_1 \geq \lambda_2 \geq \lambda_3$, we define

$$c_\theta = \frac{\pi}{2}\frac{c_p}{c_a} = \frac{\pi(\lambda_2 - \lambda_3)} {\lambda_1 + \lambda_2 - 2\lambda_3}$$ (7)

As anisotropy varies from completely linear ($c_l = 1$; $c_p = 0$) to completely planar ($c_l = 0$; $c_p = 1$), $c_\theta$ varies from 0 to $\frac{\pi}{2}$. The role of $c_\theta$ is to control how much the second eigenvector contributes to the lighting of the diffusion tensor. In the linear case, only the first eigenvector determines the tensor orientation, and in the planar case, both the first and second eigenvectors matter equally.

The expression to be used in lieu of dot products with $\mathbf{N}$ is:

$$\mathbf{U}\cdot\mathbf{N} = \sqrt{1 - (\mathbf{U}\cdot\mathbf{e}_1)^2 - (\mathbf{U}\cdot\mathbf{e}_2\,\sin(c_\theta))^2}$$ (8)

In the case of linear anisotropy, $\sin(c_\theta) = \sin(0) = 0$, so the contribution from $\mathbf{e}_2$ vanishes, and the expression reduces to the formula for lit-lines (Equation 6), with the principal eigenvector $\mathbf{e}_1$ taking the role of the tangent $\mathbf{T}$. This is appropriate, since in linear anisotropy, the principal eigenvector points in the direction of movement, as does a streamline's tangent vector.

In planar anisotropy, $\sin(c_\theta) = \sin(\frac{\pi}{2}) = 1$, and the contributions of the two dot products are equal. This means that for any other vector $\mathbf{W}$ such that

$$(\mathbf{W}\cdot\mathbf{e}_1)^2 + (\mathbf{W}\cdot\mathbf{e}_2)^2 = (\mathbf{U}\cdot\mathbf{e}_1)^2 + (\mathbf{U}\cdot\mathbf{e}_2)^2$$ (9)

Equation 8 will have the same value. Therefore, in planar anisotropy the lighting model is rotationally symmetric around $\mathbf{e}_3$. Rotational symmetry in this case is actually an important feature of the lighting model. In planar anisotropy, the diffusion tensor ellipsoid degenerates to a disc, and any vector in the plane spanned by the disc is an eigenvector. Because of this numerical instability, the calculated directions of the first and second eigenvectors will be essentially random. The illumination should not be sensitive to this arbitrary orientation, and should only be a function of the third eigenvector. In fact, one can use the Pythagorean theorem to show that if $c_\theta = \frac{\pi}{2}$, Equation 8 gives an exact formula for $\mathbf{U}\cdot\mathbf{e}_3$. Interpreting both $\mathbf{e}_1$ and $\mathbf{e}_2$ as surface tangents, then the surface normal $\mathbf{N}$ is aligned along $\mathbf{e}_3$. Therefore the model contains standard surface shading as a special case.

Figure 6: Sequence of volumes of differing anisotropy, rendered with lit-tensors. Anisotropy varies gradually between the nine volumes, going in scanline order.

To demonstrate lit-tensors, Figure 6 shows nine different synthetic diffusion tensor datasets that were direct volume rendered with a fixed viewpoint and light. The anisotropy index $c_a$ of the sphere is also constant in every case, but $c_\theta$ is changing. The dataset in the upper left has complete linear anisotropy in a concentric circular pattern (along lines of latitude). The dataset in the middle has complete planar anisotropy (and hence looks just like a standard surface rendering). The dataset in the lower left has complete linear anisotropy along lines of longitude, going from pole to pole. The images provide a convincing sense of surface anisotropy, which is not a typical trait in direct volume renderings.

Lit-Tensors mixed with Opacity Gradient Shading

Figure 7: Lit-tensor model shading whole brain, with same opacity and light direction as the first image in Figure 5.

The spheres shown in Figure 6 have well-behaved anisotropy in the following sense: the changes in the orientation of anisotropy are directly correlated to the changes in the orientation of the sphere's surface normal. Experience has shown that measured diffusion tensor data is generally not so well-behaved, so that still images created using lit-tensors tend to be confusing³. The underlying problem is that lit-tensors were designed to indicate anisotropy type and direction, not the shape of the structure made opaque by the opacity function. On a complex structure with significant self occlusion, the lack of surface shape cues can lead to a rather ambiguous, water-color effect, as seen in Figure 7.

Our current solution to this problem is to perform a separate (and significantly simpler) shading calculation, using standard Phong shading with the normalized gradient of opacity serving as the surface normal. This is accomplished by a two-step pre-process: the opacity at each data point is determined, and then the gradient of the opacity field is calculated. The normalized negative gradient of opacity is stored at each sample point. During volume rendering, these directions are interpolated to determine a surface normal at every point along the ray. The interpolated surface normal is used in the Phong shading equation. This results in an image in which shading is entirely determined by the opacity assignment and the shape of the opaque structures selected by it.

Figure 8: Mixing between lit-tensor and opacity gradient shading, for a synthetic sphere (left), and a portion of brain data (right). Going from top to bottom, the contribution of lit-tensors is $1.00$, $0.66$, $0.33$, and $0.00$.

However, it is also possible to arbitrarily mix the results of lit-tensor shading and opacity gradient shading, as shown in Figure 8. Both shading calculations are performed, and then the results are mixed on a per-voxel basis by a user-defined parameter. This sort of mixing is quite different than varying the anisotropy type as was done in Figure 6. Instead of one specular highlight changing shape gradually, there are two different specular highlights which cross-blend.

The range of possibilities illustrated by Figure 8 demonstrates an important difference between scalar and tensor volume rendering. In scalar volume rendering, opacity is nearly always determined as a function of the (scalar) data value, hence the opacity gradient is always aligned with the gradient of original data value. This means that the data value gradient can be computed only once per dataset and used to shade the output of any opacity function. Unfortunately, such a pre-process is not possible with tensor data under barycentric opacity maps, as the domain of the opacity function is a multi-dimensional space which varies non-linearly with the tensor matrix component values. On the other hand, given the overall computational expense of tensor volume rendering, we have found the cost of having to compute the opacity gradient once per opacity function to be acceptable.

Footnotes

... eigenvectors²

Because the eigenvectors always form an orthogonal basis, and because we are adopting two-sided lighting, the third eigenvector does not contribute any additional information.

... confusing³

Animations that vary viewpoint location can disambiguate surface shape while enhancing the effect of lit-tensors by showing motion of the specular highlights.